前言
考虑到道路这一特殊地物的拓扑结构信息,本博文引入两个指标 Road Completeness 和 Road Connectivity 分别用于统计道路结果的完整性与连接性。
Road Completeness
Lpred为预测出来的路的长度,Ltruth是真实标签中路的长度,其中,路的预测结果与真实标签都是单像素的栅格数据。Lpred是通过统计Ltruth在预测出来的道路路面范围内的长度。
下图表示了计算 Road Completeness 的方式。其中黄色为预测出来的道路,蓝色为真实道路,经过细化之后为蓝色的细线,即Ltruth,在第三幅中,经过叠加,落在预测结果上的绿色的代表预测成功的长度,即Lpred,红色的为缺失的长度。
import numpy as np
import skimage
from skimage import morphology
import cv2def thin_image(mask_dir, mask_file):im = cv2.imread(mask_dir + mask_file, 0)im = im > 128selem = skimage.morphology.disk(2)im = skimage.morphology.binary_dilation(im, selem)im = skimage.morphology.thin(im)return im.astype(np.uint8) * 255mask_dir = 'E:/shao_xing/out/result_boost12345/'
gt_dir = 'E:/shao_xing/test/lab/'
region = 'd'mylogs = open('E:/shao_xing/out/tiny_evalog/new_metric/' + region + '_boost12345.log','w')
ratio_list=[]
for i in range(-4, 4):for j in range(-4, 4):gt_file = region + '_' + str(i) + '_' + str(j) + '_osm.png'mask_file = gt_file[:-7] + 'merge.png'mask = cv2.imread(mask_dir + mask_file, 0)thin_gt = thin_image(gt_dir, gt_file)num_mask = np.sum(thin_gt[mask > 128]) / 255num_gt = np.sum(thin_gt) / 255ratio = num_mask / (num_gt+0.00001)if num_gt != 0:ratio_list.append(ratio)print('test image ', str(i), '_', str(j), 'ratio:', round(ratio, 2), file=mylogs)print('test image ', str(i), '_', str(j), 'ratio:', round(ratio, 2))print('********************************', file=mylogs)
print('Average Ratio:', round((sum(ratio_list) / len(ratio_list)), 2), file=mylogs)
print('********************************')
print('Average Ratio:', round((sum(ratio_list) / len(ratio_list)), 2))
mylogs.close()
Road Connectivity
下式表示 Road Connectivity 的计算方法。Nconnected代表预测结果中连续的道路片段的数量,Ntotal代表真实值中道路的片段的数量。统计的方法是将真实值的矢量数据分割成等长的片段,然后统计这些片段是否被完整预测出来,如果完整预测出来,则记为connected,没有完全预测出来记为unconnected。
下图表示 Road Connectivity 的统计过程。黄色为预测出来的道路,蓝色和红色的细线为矢量格式的道路真实标签,由很多片段组成。红色的表示断裂的部分,即 unconnected,蓝色的为 connected。
import sys
sys.path.append("./discoverlib")
from discoverlib import geom, graph
import numpy as np
import cv2
import skimage
from skimage import morphology"""
evaluate connectivity based on ground truth graph
We split the total graph into segments with length of around 20 pixels
Then, statistic the number of fully connected segments in segmentation masks.
The connectivity ratio is the percentage of fully connected segments.
"""log_name = 'mask'
mask_dir = '~/data/out/mask/' total_gt_number = 0
total_connected_number = 0
total_not_connected_number = 0
total_pred_number = 0total_connected_length = 0
total_gt_length = 0
total_pred_length = 0
mylog = open('~/data/out/eval_log/' + log_name + '_connect.log', 'w')region_name_list = [["amsterdam",-4,-4,4,4], ["chicago",-4,-4,4,4], ["denver",-4,-4,4,4]]
for region_info in region_name_list:print("test region: "&#43;region_info[0])graph_name &#61; &#39;~/data/graph_gt/&#39;&#43; region_info[0] &#43; ".graph" gt_graph &#61; graph.read_graph(graph_name)edge_nodes&#61;[]for i,edge in enumerate(gt_graph.edges):if i % 2 &#61;&#61;0:edge_nodes.append([edge.src.point.x,edge.src.point.y,edge.dst.point.x,edge.dst.point.y])base_gt_mask&#61;np.zeros((1024, 1024))edge_nodes&#61;np.array(edge_nodes)for i in range(region_info[1], region_info[3]):for j in range(region_info[2], region_info[4]):mask_file &#61; region_info[0] &#43; &#39;_&#39; &#43; str(i) &#43; &#39;_&#39; &#43; str(j) &#43; &#39;_fusion.png&#39;mask &#61; cv2.imread(mask_dir &#43; mask_file, 0)/255patch_gt_number&#61;0patch_connected_number&#61;0patch_not_connected_number&#61;0patch_connected_length &#61; 0patch_gt_length &#61; 0offset&#61;[-i*1024, -j*1024, -i*1024, -j*1024]patch_nodes&#61;edge_nodes&#43;offsetfor seg_edge in patch_nodes:if (seg_edge>&#61;[0,0,0,0]).all() and (seg_edge<[1024,1024,1024,1024]).all():base_gt_mask &#61; np.zeros((1024, 1024))patch_gt_number&#43;&#61;1 base_gt_mask&#61;cv2.line(base_gt_mask,(seg_edge[0],seg_edge[1]),(seg_edge[2],seg_edge[3]),color&#61;1, thickness&#61;1)pred_seg_length&#61;np.sum(mask[base_gt_mask>0])gt_length&#61;np.sum(base_gt_mask>0)patch_gt_length &#43;&#61; gt_lengthif pred_seg_length < gt_length:patch_not_connected_number&#43;&#61;1else:patch_connected_number&#43;&#61;1patch_connected_length &#43;&#61; gt_lengthelse:passim &#61; (mask*255) > 128selem &#61; skimage.morphology.disk(2)im &#61; skimage.morphology.binary_dilation(im, selem)im &#61; skimage.morphology.thin(im)thin_mask &#61; im.astype(np.uint8) * 255patch_pred_length &#61; np.sum(thin_mask > 0)patch_pred_number &#61; patch_pred_length / 20.0 ratio &#61; 2*patch_connected_length/(patch_gt_length&#43;patch_pred_length&#43;0.00001)print(&#39;test image {}_{} connected:not:total {}/{}/{}, ratio: {}&#39;.format(i,j,patch_connected_number,patch_not_connected_number,patch_gt_number,round(ratio, 4)))print(&#39;test image {}_{} connected:not:total {}/{}/{}, ratio: {}&#39;.format(i, j, patch_connected_number,patch_not_connected_number,patch_gt_number,round(ratio, 4)), file&#61;mylog)total_gt_number &#43;&#61; patch_gt_numbertotal_connected_number &#43;&#61; patch_connected_numbertotal_not_connected_number &#43;&#61; patch_not_connected_numbertotal_pred_number &#43;&#61; patch_pred_numbertotal_connected_length &#43;&#61; patch_connected_lengthtotal_gt_length &#43;&#61; patch_gt_lengthtotal_pred_length &#43;&#61; patch_pred_length
total_ratio &#61; 2*total_connected_length/(total_gt_length&#43;total_pred_length)print(&#39;********************************&#39;)
print("total connected:not:total {}/{}/{}, ratio: {}".format(total_connected_number,total_not_connected_number,total_gt_number,round(total_ratio, 4)))
print("total_gt_length:{}".format(total_gt_length))
print("average gt length:{}".format(total_gt_length/total_gt_number))
print(&#39;********************************&#39;, file&#61;mylog)
print("total connected:not:total {}/{}/{}, ratio: {}".format(total_connected_number,total_not_connected_number,total_gt_number,round(total_ratio, 4)),file&#61;mylog)
print("total_gt_length:{}".format(total_gt_length),file&#61;mylog)
print("average gt length:{}".format(total_gt_length/total_gt_number),file&#61;mylog)mylog.close()
参考
https://github.com/astro-ck/Road-Extraction