openpose脚部标注问题梳理

openpose脚部标注问题梳理

版本

person_keypoints_train2017_foot_v1.zip

person_keypoints_val2017_foot_v1.zip

下载地址

Human Foot Keypoint Datasethttps://cmu-perceptual-computing-lab.github.io/foot_keypoint_dataset/

总结

问题挺多的，如果想要使用此数据集最好先清洗一遍

openpose标注的训练集总数：11065

问题一

训练标注与验证标注格式不同，训练标注将脚部标注与COCO标注相融合，验证标注没有融合

person_keypoints_train2017_foot_v1:

{"segmentation":......,"num_keypoints":13,"area":26215.66095,"iscrowd":0,
"keypoints":[0,0,0,0,0,0,252,156,2,0,0,0,248,153,2,198,193,2,243,196,2,182,245,2,244,263,2,0,0,0,276,285,2,197,298,2,228,297,2,208,398,2,266,399,2,205,475,2,215,453,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],
"image_id":209468,"bbox":[178.38,120.54,114.59,354.6],"category_id":1,"id":183062}

person_keypoints_val2017_foot_v1

{"segmentation":......,"num_keypoints":14,"area":17702.97795,"iscrowd":0,
"keypoints":[215,485,2,206,479,2,223,451,2,335,492,2,346,489,2,328,455,2],
"image_id":579902,"bbox":[200.03,232.57,182.53,270.8],"category_id":1,"id":191310}

很明显train2017中将新标注的脚部数据和原始的COCO标注融合到一起，

问题二

"segmentation"格式不对

标准COCO格式

{"segmentation": [[267.03,243.78,314.59,154.05,357.84,136.76,374.05,104.32,410.81,110.81,429.19,131.35,420.54,165.95,451.89,209.19,464.86,240.54,480,253.51,484.32,263.24,496.22,271.89,484.32,278.38,438.92,257.84,401.08,216.76,370.81,247.03,414.05,277.3,433.51,304.32,443.24,323.78,400,362.7,376.22,375.68,400,418.92,394.59,424.32,337.3,382.16,337.3,371.35,388.11,327.03,341.62,301.08,311.35,276.22,304.86,263.24,294.05,249.19]],......}

person_keypoints_train2017_foot_v1格式

{"segmentation":[187.03,472.97,192.43,350.81,181.62,335.68,182.7,317.3,180.54,301.08,193.51,293.51,195.68,268.65,178.38,248.11,182.7,216.76,194.59,194.05,210.81,171.35,200,141.08,230.27,120.54,260.54,134.59,257.3,177.84,246.49,186.49,257.3,199.46,254.05,253.51,276.76,281.62,285.41,275.14,292.97,275.14,275.68,291.35,252.97,288.11,286.49,380,288.65,417.84,243.24,475.14,195.68,475.14],......}

查看了很多标准COCO中的segmentation格式，大多数都是一个二维的列表，但是OPENPOSE给出的格式不同，按正常COCO格式进行调用会报错

问题三

关键点数量错误

例：

"image_id": 478636

标准COCO格式

{"segmentation": ......, "num_keypoints":15,"area":11888.2415,"iscrowd":0,"keypoints":[452,139,2,0,0,0,448,135,2,0,0,0,434,136,2,460,160,2,413,167,2,477,184,2,410,186,2,454,197,2,410,208,2,457,239,2,423,240,2,455,295,2,420,295,2,455,348,2,426,352,2,451,351,2,456,350,2,450,347,2,395,351,2,399,350,2,421,350,2],
"image_id":478636,"bbox":[391.89,113.53,87.95,246.47],"category_id":1,"id":185068}

OPENPOSE增加脚部关键点

{"segmentation": ......,"num_keypoints": 15,"area": 11888.2415,"iscrowd": 0,"keypoints": [452,139,2,0,0,0,448,135,2,0,0,0,434,136,2,460,160,2,413,167,2,477,184,2,410,186,2,454,197,2,410,208,2,457,239,2,423,240,2,455,295,2,420,295,2,455,348,2,426,352,2],
"image_id": 478636,"bbox": [391.89,113.53,87.95,246.47],"category_id": 1,"id": 185068}

针对这帧图像的标注，OPENPOSE增加了脚部关键点的内容，却没有增加脚部关键点的数量，这种情况在运行时容易报错，数量对不上，或者无法被学习

问题四

明显的关键点顺序错误

OPONPOSE标准的循序应该如下图，

18: 左脚大拇脚指、19:左脚小拇脚指 、20:左脚脚跟 、21:右脚大拇脚指 、22:右脚小拇脚指 、23：右脚脚跟

但是在查看过程中发现较多左右脚标注错误的情况

问题五

明显漏标

"image_id": 478636

验证集中下面这张图明显应该标注，但是OPENPOSE没有标，类似的情况在训练集中应该也有