PaddlePaddle跳舞机(IntelNuc+OpenVino)

项目介绍

基于之前本人的项目如果感到快乐你就跳跳舞 进行升级迭代开发。项目通过比较在音乐鼓点时姿态的相似度进行计分统计&＃xff0c;有较强的互动娱乐性&＃xff0c;特别是对于作者这种老肥宅的程序员有很不错的减脂效果。欢迎大家尝试体验&＃xff01;

本次项目主要升级点&＃xff1a;

• 采用了最新的更轻量级的PP-TinyPose升级版&＃xff0c; 姿态点相对以前稳定性提升不少
• 使用了更轻量级的Picodet行人检测模型&＃xff0c;推理性能超实时
• 在Intel Nuc硬件上基于OpenVino加速推理框架&＃xff0c;性能非常惊艳
• 移植了sort跟踪算法&＃xff0c;在这种简单场景下已足够支持几个人的跟踪
• 每次游戏结束还会通过PPLCNet提取行人特征&＃xff0c;可简单大致查找选手之前的分数&＃xff0c;要做到更精确识别需要进行人脸识别
• 选手在镜头前举手动作持续5秒可自动开始或结束当前游戏
• 同时为了提升体验&＃xff0c;增加了音效和背景

todo 视频

注意: 所有素材均来自互联网&＃xff0c;若有侵权&＃xff0c;请联系删除

界面设计

界面布局主要分成3部分&＃xff0c;上方中间显示姿态匹配正确的次数:

• 左侧上方是鼓点时刻参考舞蹈的姿态
• 左侧下方是参考舞蹈视频源
• 右侧上方是鼓点时实时视频的姿态
• 右侧下方是摄像头实时视频源头

菜单管理

这次升级中暂无明显改动&＃xff0c;在参考舞蹈视频源区域右键单击显示菜单&＃xff0c; 主要包括

• 摄像头设置
• 歌曲管理
• 开始游戏(游戏未开始时)
• 停止游始(游戏运行时)
• 全屏显示
• 退出

点击歌曲管理后可以添加删除舞蹈视频, 也可以在歌曲列表切换当前选中歌曲。

行人检测模型转换

# 下载openvino
!pip install openvino-dev --user


# 下载框架代码
!git clone https://gitee.com/paddlepaddle/PaddleDetection.git


Cloning into &＃39;PaddleDetection&＃39;...
remote: Enumerating objects: 254557, done.[K
remote: Counting objects: 100% (235027/235027), done.[K
remote: Compressing objects: 100% (42643/42643), done.[K
remote: Total 254557 (delta 193625), reused 232717 (delta 191595), pack-reused 19530[K
Receiving objects: 100% (254557/254557), 411.82 MiB | 24.67 MiB/s, done.
Resolving deltas: 100% (208174/208174), done.
Checking connectivity... done.


# 导出行人检测静态图模型
%cd /home/aistudio/PaddleDetection
!python tools/export_model.py -c configs/picodet/application/pedestrian_detection/picodet_s_320_lcnet_pedestrian.yml \-o weights&＃61;https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/picodet_s_320_lcnet_pedestrian.pdparams \export.benchmark&＃61;False export.nms&＃61;False \--output_dir&＃61;output_pedestrain_onnx


/home/aistudio/PaddleDetection
Warning: import ppdet from source directory without installing, run &＃39;python setup.py install&＃39; to install ppdet firstly
[11/22 00:28:41] ppdet.utils.checkpoint INFO: Finish loading model weights: /home/aistudio/.cache/paddle/weights/picodet_s_320_lcnet_pedestrian.pdparams
[11/22 00:28:41] ppdet.data.source.category WARNING: anno_file &＃39;dataset/coco/annotations/instances_val2017.json&＃39; is None or not set or not exist, please recheck TrainDataset/EvalDataset/TestDataset.anno_path, otherwise the default categories will be used by metric_type.
[11/22 00:28:41] ppdet.data.source.category WARNING: metric_type: COCO, load default categories of COCO.
[11/22 00:28:41] ppdet.engine INFO: Export inference config file to output_pedestrain_onnx/picodet_s_320_lcnet_pedestrian/infer_cfg.yml
[11/22 00:28:46] ppdet.engine INFO: Export model and saved in output_pedestrain_onnx/picodet_s_320_lcnet_pedestrian


# 导出onnx
# !pip install paddle2onnx
%cd /home/aistudio/PaddleDetection
!paddle2onnx --model_dir output_pedestrain_onnx/picodet_s_320_lcnet_pedestrian/ \--model_filename model.pdmodel \--params_filename model.pdiparams \--opset_version 11 \--input_shape_dict "{&＃39;image&＃39;:[1, 3, 320, 320]}" \--save_file pedestrain.onnx


/home/aistudio/PaddleDetection
[1;31;40m2022-11-22 00:29:45 [WARNING] [Deprecated] The flag &＃96;--input_shape_dict&＃96; is deprecated, if you need to modify the input shape of PaddlePaddle model, please refer to this tool https://github.com/jiangjiajun/PaddleUtils/tree/main/paddle [0m
[Paddle2ONNX] Start to parse PaddlePaddle model...
[Paddle2ONNX] Model file path: output_pedestrain_onnx/picodet_s_320_lcnet_pedestrian/model.pdmodel
[Paddle2ONNX] Paramters file path: output_pedestrain_onnx/picodet_s_320_lcnet_pedestrian/model.pdiparams
[Paddle2ONNX] Start to parsing Paddle model...
[Paddle2ONNX] Use opset_version &＃61; 11 for ONNX export.
[Paddle2ONNX] PaddlePaddle model is exported as ONNX format now.
2022-11-22 00:29:46 [INFO] &＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;Make PaddlePaddle Better!&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;
2022-11-22 00:29:46 [INFO] A little survey: https://iwenjuan.baidu.com/?code&＃61;r8hu2s


# 安装onnx和onnxsim
!pip install -U onnx onnxsim onnxruntime


# 优化onnx模型
%cd /home/aistudio/PaddleDetection
!onnxsim pedestrain.onnx pedestrain_sim.onnx


/home/aistudio/PaddleDetection
Simplifying[33m...[0m
Finish! Here is the difference:
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃[1m [0m[1m [0m[1m [0m┃[1m [0m[1mOriginal Model[0m[1m [0m┃[1m [0m[1mSimplified Model[0m[1m [0m┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Add │ 81 │ 81 │
│ BatchNormalization │ 82 │ [1;32m0 [0m │
│ Clip │ 78 │ 78 │
│ Concat │ 7 │ 7 │
│ Constant │ 774 │ [1;32m0 [0m │
│ Conv │ 98 │ 98 │
│ Div │ 78 │ 78 │
│ GlobalAveragePool │ 6 │ 6 │
│ HardSigmoid │ 2 │ 2 │
│ MatMul │ 4 │ 4 │
│ Mul │ 90 │ 90 │
│ Relu │ 2 │ 2 │
│ Reshape │ 12 │ 12 │
│ Resize │ 2 │ 2 │
│ Sigmoid │ 12 │ 12 │
│ Softmax │ 4 │ 4 │
│ Split │ 1 │ 1 │
│ Sqrt │ 4 │ 4 │
│ Transpose │ 4 │ 4 │
│ Model Size │ 4.6MiB │ [1;32m4.5MiB [0m │
└────────────────────┴────────────────┴──────────────────┘


姿态模型转换

# 导出姿态模型静态图
%cd /home/aistudio/PaddleDetection
!python tools/export_model.py -c configs/keypoint/tiny_pose/tinypose_256x192.yml \-o weights&＃61;https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/tinypose_256x192.pdparams \--output_dir&＃61;output_pose


/home/aistudio/PaddleDetection
Warning: import ppdet from source directory without installing, run &＃39;python setup.py install&＃39; to install ppdet firstly
[11/21 23:24:52] ppdet.utils.download INFO: Downloading tinypose_256x192.pdparams from https://bj.bcebos.com/v1/paddledet/models/keypoint/tinypose_enhance/tinypose_256x192.pdparams
100%|█████████████████████████████████████| 5439/5439 [00:01<00:00, 3179.96KB/s]
[11/21 23:24:55] ppdet.utils.checkpoint INFO: Finish loading model weights: /home/aistudio/.cache/paddle/weights/tinypose_256x192.pdparams
[11/21 23:24:55] ppdet.engine INFO: Export inference config file to output_pose/tinypose_256x192/infer_cfg.yml
[11/21 23:25:10] ppdet.engine INFO: Export model and saved in output_pose/tinypose_256x192


# 导出onnx模型
%cd /home/aistudio/PaddleDetection
!paddle2onnx --model_dir output_pose/tinypose_256x192/ \--model_filename model.pdmodel \--params_filename model.pdiparams \--opset_version 11 \--save_file tinypose.onnx


/home/aistudio/PaddleDetection
[Paddle2ONNX] Start to parse PaddlePaddle model...
[Paddle2ONNX] Model file path: output_pose/tinypose_256x192/model.pdmodel
[Paddle2ONNX] Paramters file path: output_pose/tinypose_256x192/model.pdiparams
[Paddle2ONNX] Start to parsing Paddle model...
[Paddle2ONNX] Use opset_version &＃61; 11 for ONNX export.
[Paddle2ONNX] PaddlePaddle model is exported as ONNX format now.
2022-11-21 23:25:45 [INFO] &＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;Make PaddlePaddle Better!&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;&＃61;
2022-11-21 23:25:45 [INFO] A little survey: https://iwenjuan.baidu.com/?code&＃61;r8hu2s


# 将模型转为openvino的&＃xff29;&＃xff32;格式
%cd /home/aistudio/PaddleDetection
!mo --framework onnx --input_model tinypose.onnx --output_dir ./ --input_shape [1,3,256,192]


行人检测和姿态识别联合推理

结合使用sort跟踪算法&＃xff0c;推理第100帧图片, 目前在aistudio中无法直接运行openvino, 会报错　ModuleNotFoundError: No module named ‘openvino.runtime’

%cd /home/aistudio/track/
!python detector.py


运行推理结果示例如下&＃xff1a;

REID模型转换

本项目使用REID模型仅仅用于特征保存&＃xff0c;方便快速查找某个舞者的历史分数&＃xff0c;若要进一步更准确的查找&＃xff0c;也可以用人脸识别模型替换。

参考转换姿态模型的步骤&＃xff0c;简单几步即可完成:

CUDA_VISIBLE_DEVICES&＃61;0 python tools/export_model.py -c configs/mot/deepsort/reid/deepsort_pplcnet.yml -o reid_weights&＃61;https://paddledet.bj.bcebos.com/models/mot/deepsort/deepsort_pplcnet.pdparamspaddle2onnx --model_dir output_inference/deepsort_pplcnet \--model_filename model.pdmodel \--params_filename model.pdiparams \--opset_version 11 \--save_file reid.onnxmo reid.onnx --output_dir . --input_shape [1,3,64,192]


写在最后

本项目需要本地运行&＃xff0c;基本可以运行&＃xff0c;程序运行稳定性和匹配效果还需要进一步优化。

后期将做更多的功能升级&＃xff0c;例如尝试3D姿态&＃xff0c;多人模式&＃xff0c;Unity等等。

关于作者

• 成都飞桨领航团团长
• PPDE
• PFCC
• AICA三期学员

我在AI Studio上获得青铜等级&＃xff0c;点亮0个徽章&＃xff0c;来互关呀~ https://aistudio.baidu.com/aistudio/personalcenter/thirdview/89442

请点击此处查看本环境基本用法.

Please click here for more detailed instructions.

此文章为搬运
原项目链接