ssd(SingleShotMultiBoxDetector)代码解读之(三)multiboxloss损失函数

得到正样本和负样本后&＃xff0c;我们要怎么运用起来呢&＃xff1f;

首先我们要知道SSD网络输出的分类预测是矩阵conf_data如下&＃xff1a;

conf_data记录的是每个default box对应的 num_class个分类的预测值。num_class个值中最大的那个就作为此框的类别。

由于 pos矩阵和neg矩阵中的值都是1或者0。因此pos矩阵相当于记录了default box中正样本框的序号&＃xff0c;同理neg矩阵记录的就是负样本框的序号。

我们要明白一件事就是 pos矩阵中为1的肯定是正样本&＃xff0c;但并非说为0的就一定是负样本。neg矩阵中为1的肯定是负样本&＃xff0c;也并非说为0的就一定是正样本&＃xff0c;因为由于Hard Negative Mining的关系&＃xff0c;负样本的数目被设置成了是正样本的三倍&＃xff0c;所以这就导致了并非所有pos中为0的框都是负样本。

然后使用一个技巧&＃xff0c;所有default box中 pos 和 neg中的值加起来大于0的框&＃xff0c;才作为真正的训练样本 conf_p。意思就是conf_p中的框不是正样本就是负样本。意在去除pos&＃61;0且neg&＃61;0的框&＃xff0c;这类框的特点是&＃xff0c;与GroundTrue不怎么沾边&＃xff0c;被分类成背景框&＃xff0c;且作为背景框的损失值还很小&＃xff0c;就是说这类框必是背景框了。要作为训练数据的框&＃xff0c;要么是与GroundTrue交叠比较大的正样本&＃xff0c;要么是损失值大的背景框&＃xff0c;就是说它看上去是背景&＃xff0c;但好像又不是背景的框。

最后把 conf_t和targets_weighted输入到交叉熵损失函数就可以得到最终的分类损失值。

得到回归损失值和分类损失值后&＃xff0c;就加起来就可以计算出最终的 multibox 损失函数值了。

代码&＃xff1a;

pos_idx &＃61; pos.unsqueeze(2).expand_as(conf_data) #[batchsize,num_priors,num_class]为1的是大于阈值的框neg_idx &＃61; neg.unsqueeze(2).expand_as(conf_data) #[batchsize,num_priors,num_class]为1的是负样本conf_p &＃61; conf_data[(pos_idx&＃43;neg_idx).gt(0)].view(-1, self.num_classes)#pos&＃61;0是小于阈值的框&＃xff0c;neg&＃61;0是损失很小的框#pos_idx&＃43;neg_idx大于0的数据保留&＃xff0c;其余舍去targets_weighted &＃61; conf_t[(pos&＃43;neg).gt(0)]loss_c &＃61; F.cross_entropy(conf_p, targets_weighted, size_average&＃61;False)

multibox loss完整代码&＃xff1a;

# -*- coding: utf-8 -*-
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
from data import coco as cfg
from ..box_utils import match, log_sum_expclass MultiBoxLoss(nn.Module):"""SSD Weighted Loss FunctionCompute Targets:1) Produce Confidence Target Indices by matching ground truth boxeswith (default) &＃39;priorboxes&＃39; that have jaccard index > threshold parameter(default threshold: 0.5).2) Produce localization target by &＃39;encoding&＃39; variance into offsets of groundtruth boxes and their matched &＃39;priorboxes&＃39;.3) Hard negative mining to filter the excessive number of negative examplesthat comes with using a large number of default bounding boxes.(default negative:positive ratio 3:1)Objective Loss:L(x,c,l,g) &＃61; (Lconf(x, c) &＃43; αLloc(x,l,g)) / NWhere, Lconf is the CrossEntropy Loss and Lloc is the SmoothL1 Lossweighted by α which is set to 1 by cross val.Args:c: class confidences,l: predicted boxes,g: ground truth boxesN: number of matched default boxesSee: https://arxiv.org/pdf/1512.02325.pdf for more details."""def __init__(self, num_classes, overlap_thresh, prior_for_matching,bkg_label, neg_mining, neg_pos, neg_overlap, encode_target,use_gpu&＃61;True):super(MultiBoxLoss, self).__init__()self.use_gpu &＃61; use_gpuself.num_classes &＃61; num_classesself.threshold &＃61; overlap_threshself.background_label &＃61; bkg_labelself.encode_target &＃61; encode_targetself.use_prior_for_matching &＃61; prior_for_matchingself.do_neg_mining &＃61; neg_miningself.negpos_ratio &＃61; neg_posself.neg_overlap &＃61; neg_overlapself.variance &＃61; cfg[&＃39;variance&＃39;]def forward(self, predictions, targets):&＃39;&＃39;&＃39; output &＃61; ( --->predictionsloc.view(loc.size(0), -1, 4),conf.view(conf.size(0), -1, self.num_classes),self.priors&＃39;&＃39;&＃39;"""Multibox LossArgs:predictions (tuple): A tuple containing loc preds, conf preds,and prior boxes from SSD net.conf shape: torch.size(batch_size,num_priors,num_classes)loc shape: torch.size(batch_size,num_priors,4)priors shape: torch.size(num_priors,4)targets (tensor): Ground truth boxes and labels for a batch,shape: [batch_size,num_objs,5] (last idx is the label)."""loc_data, conf_data, priors &＃61; predictions #loc是ssd生成的偏置值&＃xff0c;priors是PriorBox方法画出来的boxnum &＃61; loc_data.size(0) #num &＃61; batch_size loc_data&＃61;[batchsize,8732,4] priors&＃61;[8732xbatchsize,4]priors &＃61; priors[:loc_data.size(1), :] #[num_priors,4] 逐个batchsize处理 [8732,4]num_priors &＃61; (priors.size(0)) #num_proirs&＃xff0c;一个batchsize所有框num_classes &＃61; self.num_classes# match priors (default boxes) and ground truth boxesloc_t &＃61; torch.Tensor(num, num_priors, 4) #num&＃61;batchsizeconf_t &＃61; torch.LongTensor(num, num_priors)for idx in range(num): #num &＃61; batch_size 一张张图片拿出来truths &＃61; targets[idx][:, :-1].data #target 有5个整数&＃xff0c;前四个是坐标&＃xff0c;框长宽&＃xff0c;最后一个是类别labels &＃61; targets[idx][:, -1].data #[num_objs,1]defaults &＃61; priors.data #[num_priors,4]match(self.threshold, truths, defaults, self.variance, labels, #truths:[num_objs,4]loc_t, conf_t, idx)if self.use_gpu:loc_t &＃61; loc_t.cuda()conf_t &＃61; conf_t.cuda() #conf_t维度原为[num_priors]&＃xff0c;通过阈值处理后&＃xff0c;变成[ 0 #pos为 conf_t大于0的部分&＃xff0c;pos维度[batchsize,num_priors] [0,1,1,1,0,1.....]num_pos &＃61; pos.sum(dim&＃61;1, keepdim&＃61;True) #不因jard被舍去的框的个数# Localization Loss (Smooth L1)# Shape: [batch,num_priors,4]pos_idx &＃61; pos.unsqueeze(pos.dim()).expand_as(loc_data) #在最后加一维度batchsize&＃xff0c;num_priors,4]#下面两步&＃xff0c;1是从预测中舍去不通过jard的框&＃xff0c;2是从 match中舍去不通过jard的框loc_p &＃61; loc_data[pos_idx].view(-1, 4) #loc_p 是网络预测出来的loc_t &＃61; loc_t[pos_idx].view(-1, 4) #loc_t 是overlap>阈值的 prioris和对应GT框偏置值loss_l &＃61; F.smooth_l1_loss(loc_p, loc_t, size_average&＃61;False) #预测偏置值#loc_t 相当于 真正训练的label&＃xff0c;loc_p则相当于真正训练的预测输入# Compute max conf across batch for hard negative miningbatch_conf &＃61; conf_data.view(-1, self.num_classes) #[batch_size x num_priors,num_classes]batch_conf包含全部框#conf_t 是每个num_priors 框最符合的类别label 维度[num_priors]#con_t是不少于阈值的每个prior对应的类别(最大overlay重叠的GT框的类别)#loss_c是每个batchsize中每个priors框的类别 和 预测出的类别的损失loss_c &＃61; log_sum_exp(batch_conf) - batch_conf.gather(1, conf_t.view(-1, 1)) #conf_t&＃61;(batchsize, num_priors)#上式中conf_t --->[batchsize X num_priors&＃xff0c;1]#后一项是选出大于阈值的框 的分类#loss_c 是各个框分类的损失值# Hard Negative Miningloss_c &＃61; loss_c.view(num, -1) #一个batchsize每张图的 loss_closs_c[pos] &＃61; 0 # filter out pos boxes for now 剩下分类为背景的框的损失值_, loss_idx &＃61; loss_c.sort(1, descending&＃61;True) #整个batchsize的loss_c排序_, idx_rank &＃61; loss_idx.sort(1)#各个框loss_c(分类损失)的排名&＃xff0c;从大到小 [batch,num_priors]num_pos &＃61; pos.long().sum(1, keepdim&＃61;True) # #不因jard被舍去的框的个数 #[batch,1]num_neg &＃61; torch.clamp(self.negpos_ratio*num_pos, max&＃61;pos.size(1)-1) ##[batch,1]neg &＃61; idx_rank