CornerNet-Lite是刚开源不久的实时目标检测方法&＃xff0c;据说比YOLO3 牛逼&＃xff0c;于是打算尝鲜一波。
首先说运行的环境要求&＃xff1a;
1&＃xff09;Python 3.7
2&＃xff09;PyTorch 1.0.0
3&＃xff09;CUDA 10 &＃xff08;意思是你得有一张NVIDIA的显卡&＃xff0c;比如我的是GTX1060&＃xff09;
4&＃xff09;GCC 4.9.2 or above
1. 去github下载源码
https://github.com/princeton-vl/CornerNet-Lite
git clone https://github.com/princeton-vl/CornerNet-Lite.git
2. 跟着官方readme&＃xff0c;设置环境&＃xff0c;这会下载一大堆东西&＃xff0c;比如opencv, pytorch
cd
conda create --name CornerNet_Lite --file conda_packagelist.txt --channel pytorch
这可能是一个漫长的过程&＃xff08;实验室这破网速&＃xff0c;几kb的下载速度玩个XX&＃xff09;&＃xff0c;且在下载的pytorch的时候&＃xff0c;可能会失败&＃xff0c;失败&＃xff0c;再失败&＃xff0c;请不要气馁&＃xff0c;奇迹总会出现&＃xff0c;比如大清早&＃xff0c;实验室的网速最佳&＃xff0c;成功的几率更高哦。
3. active 你的环境
conda activate CornerNet_Lite
注意&＃xff0c;这里是 conda &＃xff0c;不是source。执行完后&＃xff0c;你会发现终端命令行前面多了(CornerNet_Lite)
这个时候你还会发现&＃xff0c;当你在该终端下运行python时&＃xff0c;默认是3.7版本了
4. 编译Corner Pooling Layers
cd
python setup.py install --user
5. 编译NMS
cd
make
6. 下载模型文件
原下载地址&＃xff1a;
CornerNet-Saccade
CornerNet-Squeeze
CornerNet
百度云走起&＃xff1a;
链接: https://pan.baidu.com/s/1gmMpx6EUOVjfIVys_POGqQ 提取码: xtzx 复制这段内容后打开百度网盘手机App&＃xff0c;操作更方便哦
把下载的文件放在正确的目录下&＃xff1a;
Put the CornerNet-Saccade model under
7. 测试一下demo&＃xff0c;看看效果如何
python demo.py
糟糕&＃xff0c;有问题&＃xff0c;由于本人安装了ROS&＃xff0c;cv2.so冲突了吗&＃xff1f;错误如下&＃xff1a;
Traceback (most recent call last):File "demo.py", line 7, in
ImportError: /opt/ros/kinetic/lib/python2.7/dist-packages/cv2.so: undefined symbol: PyCObject_Type
检查一下python路径
Python 3.7.1 (default, Oct 23 2018, 19:19:42)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
[&＃39;&＃39;, &＃39;/home/communicationgroup/rgbdslam2_catkin_ws/devel/lib/python2.7/dist-packages&＃39;, &＃39;/opt/ros/kinetic/lib/python2.7/dist-packages&＃39;, &＃39;/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python37.zip&＃39;, &＃39;/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7&＃39;, &＃39;/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7/lib-dynload&＃39;, &＃39;/home/communicationgroup/.local/lib/python3.7/site-packages&＃39;, &＃39;/home/communicationgroup/.local/lib/python3.7/site-packages/cpools-0.0.0-py3.7-linux-x86_64.egg&＃39;, &＃39;/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7/site-packages&＃39;, &＃39;/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7/site-packages/torchvision-0.2.1-py3.7.egg&＃39;]
罪魁祸首正是&＃39;/opt/ros/kinetic/lib/python2.7/dist-packages&＃39;
那就删除它吧
对demo.py加入如下两行代码
#解决ros中cv2的冲突
import sys
sys.path.remove(&＃39;/opt/ros/kinetic/lib/python2.7/dist-packages&＃39;)
2019/4/27更新&＃xff1a;
这样的后果是不能在.py中 import rospy&＃xff0c;若想嵌入到ROS中&＃xff0c;则不能采用该方法。若不用嵌入ROS&＃xff0c;使用该方法没得问题&＃xff0c;若要嵌入ROS中&＃xff0c;解决办法参考后面第8节的更新。
重新运行demo.py
python demo.py
终端输出大概是这样的&＃xff1a;
total parameters: 116969339
loading from /home/communicationgroup/CornerNet-Lite/core/../cache/nnet/CornerNet_Saccade/CornerNet_Saccade_500000.pkl
/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode&＃61;bilinear is changed to align_corners&＃61;False since 0.4.0. Please specify align_corners&＃61;True if the old behavior is desired. See the documentation of nn.Upsample for details."See the documentation of nn.Upsample for details.".format(mode))
结果还会在CornerNet-Lite目录下保存一张名为 demo_out.jpg 的图片
可以看出&＃xff0c;已经把小狗仔们框出来了。
其实也并不快嘛&＃xff0c;我的电脑&＃xff08;GTX1060&＃xff09;跑一张图像需要0.5s左右&＃xff0c;难道是没用上GPU&＃xff1f;&＃xff1f;&＃xff1f;&＃xff08;2019/4/27注&＃xff1a;平均测试速度单张图片244ms&＃xff0c; Titan X (PASCAL) GPU&＃xff09;
8. 尝试一下能不能用在ROS工程中
思路&＃xff1a;相机节点通过相机获取视频&＃xff0c;并通过ROS消息发布&＃xff0c;图像处理节点订阅图像消息&＃xff0c;并调用CornerNet进行目标检测。
1)C&＃43;&＃43;和python参数传递
将.cpp获取的图像作为参数传给.py&＃xff0c;实现目标识别&＃xff0c;再将.py得到的bboxes传回给.cpp
(以后慢慢做&＃xff0c;等更新)
2) 写一个.py节点&＃xff0c;来订阅图像消息&＃xff0c;并发布bboxes
由于这个CornerNet既要python3.7&＃xff0c;又要pytorch1.0.0&＃xff0c;还要opencv3&＃xff0c;而ros还没有完全支持python3&＃xff0c;所以暂时放弃该尝试。
-------------------------------------------------------------我是分割线--------------------------------------------------------------------
2019/4/27更新&＃xff1a;
经过一番波折&＃xff0c;实现了 .py节点的方法。测试方式&＃xff1a;Kinect2 获取图像&＃xff0c;test_ros.py 订阅图像消息&＃xff0c;并调用CornerNet实现目标检测。
该方法主要是解决Python3 环境下运行ROS的问题&＃xff0c;其中主要涉及cv_bridge&＃xff0c;cv2的问题。网上方法形形色色&＃xff0c;但是到我这里基本是都不得行&＃xff0c;还害得我重装了一遍ROS。为了让大家少走弯路&＃xff0c;这里总结一下我对Python3环境下运行ROS&＃xff0c;并调用CornerNet的探索。
8.1.1 首先&＃xff0c;解决import cv2的问题
前面第7节提到的方案在这里就不适用了&＃xff0c;sys.path.remove(&＃39;/opt/ros/kinetic/lib/python2.7/dist-packages&＃39;) 会导致 import rospy失败。
cd /opt/ros/kinetic/lib/python2.7/dist-packages/
sudo mv cv2.so cv2_ros.so
意思就是重命名一下&＃xff0c;真是简单粗暴还管用啊!
8.1.2 解决Python3 和 cv_bridge的问题
ROS默认使用python2.7&＃xff0c;这时我们需要从源码编译cv_bridge&＃xff0c;并在编译前设置好Python版本为Python3
1)新建ros工作空间&＃xff0c;用于测试
mkdir py3_ws
cd py3_ws
mkdir src
2)将前面测试通过的CornerNet-Lite目录全部拷贝到 src/目录下
现在&＃xff0c;目录结构大概是这样的 py3_ws/src/CornerNet-Lite
3&＃xff09;使用conda激活CornerNet-Lite环境
cd src/CornerNet-Lite/
conda activate CornerNet_Lite
跟前面一样&＃xff0c;成功的话&＃xff0c;终端命令行前面或多出 (CornerNet_Lite)
之所以激活该环境&＃xff0c;是因为后续需要在该环境下调用ROS&＃xff0c;所以得先切换到该环境下&＃xff0c;安装rospkg等包。
4) 使用pip在CornerNet_Lite环境下安装rospkg、catkin_pkg等&＃xff0c;这样才能在该环境下调用ROS
pip install catkin_pkg pyyaml empy rospkg numpy
更改CornerNet_Lite环境下opencv版本为3.4.1&＃xff08;CornerNet_Lite环境下的opencv版本为3.4.2&＃xff0c;该版本在调用cv2.imshow()函数时会报错&＃xff0c;也不知道大家的存在这个问题不&＃xff09;
conda install opencv&＃61;&＃61;3.4.1
终端输出如下
Collecting package metadata: done
Solving environment: done## Package Plan ##environment location: /home/communicationgroup/anaconda3/envs/CornerNet_Liteadded / updated specs:- opencv&＃61;&＃61;3.4.1The following packages will be downloaded:package | build---------------------------|-----------------ffmpeg-3.4 | h7985aa0_0 8.0 MBlibopencv-3.4.1 | h8fa1ad8_3 41.5 MBlibprotobuf-3.5.2 | h6f1eeef_0 4.2 MBopencv-3.4.1 | py37h6fd60c2_3 8 KBpy-opencv-3.4.1 | py37h8fa1ad8_3 1.2 MB------------------------------------------------------------Total: 54.9 MBThe following NEW packages will be INSTALLED:libprotobuf pkgs/main/linux-64::libprotobuf-3.5.2-h6f1eeef_0The following packages will be DOWNGRADED:ffmpeg 4.0-hcdf2ecd_0 --> 3.4-h7985aa0_0libopencv 3.4.2-hb342d67_1 --> 3.4.1-h8fa1ad8_3opencv 3.4.2-py37h6fd60c2_1 --> 3.4.1-py37h6fd60c2_3py-opencv 3.4.2-py37hb342d67_1 --> 3.4.1-py37h8fa1ad8_3Proceed ([y]/n)? yDownloading and Extracting Packages
py-opencv-3.4.1 | 1.2 MB | ##################################################################################################################################### | 100%
opencv-3.4.1 | 8 KB | ##################################################################################################################################### | 100%
libopencv-3.4.1 | 41.5 MB | ##################################################################################################################################### | 100%
libprotobuf-3.5.2 | 4.2 MB | ##################################################################################################################################### | 100%
ffmpeg-3.4 | 8.0 MB | ##################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
安装完成后&＃xff0c;你可以使用conda list查看该环境下都有哪些包&＃xff0c;主要关注以下opencv、 catkin_pkg、 pyyaml、 rospkg
conda list
输出大概是这样的 &＃xff08;我这里多了个opencv-python 3.4.5.20&＃xff0c;是我自己手贱安装的&＃xff09;
# packages in environment at /home/communicationgroup/anaconda3/envs/CornerNet_Lite:
#
# Name Version Build Channel
blas 1.0 mkl
bzip2 1.0.6 h14c3975_5
ca-certificates 2019.1.23 0
cairo 1.14.12 h8948797_3
catkin-pkg 0.4.12 pypi_0 pypi
certifi 2019.3.9 py37_0
cffi 1.11.5 py37he75722e_1
cuda100 1.0 0 pytorch
cycler 0.10.0 py37_0
cython 0.28.5 py37hf484d3e_0
dbus 1.13.2 h714fa37_1
defusedxml 0.6.0 pypi_0 pypi
docutils 0.14 pypi_0 pypi
empy 3.3.4 pypi_0 pypi
expat 2.2.6 he6710b0_0
ffmpeg 3.4 h7985aa0_0
fontconfig 2.13.0 h9420a91_0
freeglut 3.0.0 hf484d3e_5
freetype 2.9.1 h8a8886c_1
glib 2.56.2 hd408876_0
graphite2 1.3.12 h23475e2_2
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb453b48_1
harfbuzz 1.8.8 hffaf4a1_0
hdf5 1.10.2 hba1933b_1
icu 58.2 h9c2bf20_1
intel-openmp 2019.0 118
jasper 2.0.14 h07fcdf6_1
jpeg 9b h024ee3a_2
kiwisolver 1.0.1 py37hf484d3e_0
libedit 3.1.20170329 h6b74fdf_2
libffi 3.2.1 hd88cf55_4
libgcc-ng 8.2.0 hdf63c60_1
libgfortran-ng 7.3.0 hdf63c60_0
libglu 9.0.0 hf484d3e_1
libopencv 3.4.1 h8fa1ad8_3
libopus 1.2.1 hb9ed12e_0
libpng 1.6.35 hbc83047_0
libprotobuf 3.5.2 h6f1eeef_0
libstdcxx-ng 8.2.0 hdf63c60_1
libtiff 4.0.9 he85c1e1_2
libuuid 1.0.3 h1bed415_2
libvpx 1.7.0 h439df22_0
libxcb 1.13 h1bed415_1
libxml2 2.9.8 h26e45fe_1
matplotlib 3.0.2 py37h5429711_0
mkl 2018.0.3 1
mkl_fft 1.0.6 py37h7dd41cf_0
mkl_random 1.0.1 py37h4414c95_1
ncurses 6.1 hf484d3e_0
netifaces 0.10.9 pypi_0 pypi
ninja 1.8.2 py37h6bb024c_1
numpy 1.15.4 py37h1d66e8a_0
numpy-base 1.15.4 py37h81de0dd_0
olefile 0.46 py37_0
opencv 3.4.1 py37h6fd60c2_3
opencv-python 3.4.5.20 pypi_0 pypi
openssl 1.1.1b h7b6447c_1
pcre 8.42 h439df22_0
pillow 5.2.0 py37heded4f4_0
pip 19.1 pypi_0 pypi
pixman 0.34.0 hceecf20_3
py-opencv 3.4.1 py37h8fa1ad8_3
pycparser 2.18 py37_1
pyparsing 2.2.0 py37_1
pyqt 5.9.2 py37h05f1152_2
python 3.7.1 h0371630_3
python-dateutil 2.7.3 py37_0
pytorch 1.0.0 py3.7_cuda10.0.130_cudnn7.4.1_1 [cuda100] pytorch
pytz 2018.5 py37_0
pyyaml 5.1 pypi_0 pypi
qt 5.9.7 h5867ecd_1
readline 7.0 h7b6447c_5
rospkg 1.1.9 pypi_0 pypi
scikit-learn 0.19.1 py37hedc7406_0
scipy 1.1.0 py37hfa4b5c9_1
setuptools 40.2.0 py37_0
sip 4.19.8 py37hf484d3e_0
six 1.11.0 py37_1
sqlite 3.25.3 h7b6447c_0
tk 8.6.8 hbc83047_0
torchvision 0.2.1 py37_1 pytorch
tornado 5.1 py37h14c3975_0
tqdm 4.25.0 py37h28b3542_0
wheel 0.31.1 py37_0
xz 5.2.4 h14c3975_4
zlib 1.2.11 ha838bed_2
5&＃xff09;从源码编译cv_bridge
先装点依赖项吧&＃xff0c;其实很多都有了
sudo apt-get update
sudo apt-get install python-catkin-tools python3-dev python3-catkin-pkg-modules python3-numpy python3-yaml ros-kinetic-cv-bridge
初始化工作空间&＃xff0c;并设置Python版本为Python3&＃xff08;注意&＃xff0c;这里使用的是系统默认安装的Python3.5m&＃xff0c;测试通过&＃xff0c;管他呢&＃xff0c;或许Python3都可以&＃xff09;
cd ../../ #回到py3_ws/目录
catkin init
catkin config -DPYTHON_EXECUTABLE&＃61;/usr/bin/python3 -DPYTHON_INCLUDE_DIR&＃61;/usr/include/python3.5m -DPYTHON_LIBRARY&＃61;/usr/lib/x86_64-linux-gnu/libpython3.5m.so
catkin config --install
下载cv_bridge源码并编译
git clone https://github.com/ros-perception/vision_opencv.git src/vision_opencv
注意&＃xff0c;这里直接下载到了src/目录下&＃xff0c;所以不用先cd src/
修改src/vision_opencv/cv_bridge/CMakeLists.txt 第11行内容&＃xff08;解决找不到boost_python3的问题&＃xff09;
gedit src/vision_opencv/cv_bridge/CMakeLists.txt
将第11行的
find_package(Boost REQUIRED python3)
修改为
find_package(Boost REQUIRED python-py35)
修改src/vision_opencv/cv_bridge/setup.py第1行内容&＃xff08;话说这样才会在执行的时候使用Python3&＃xff1f;&＃xff1f;&＃xff1f;&＃xff09;
gedit src/vision_opencv/cv_bridge/setup.py
将第1行的
#!/usr/bin/env python
修改为
#!/usr/bin/env python3
开始编译
catkin build cv_bridge
source install/setup.bash --extend
此处使用catkin build编译cv_bridge包&＃xff0c;并source了一下setup.bash&＃xff0c;让系统知道cv_bridge要用咱们这里这个。
这时可以简单测试以下能不能用
pythonPython 3.7.1 (default, Oct 23 2018, 19:19:42)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from cv_bridge.boost.cv_bridge_boost import getCvType
>>>
不报错或许就是莫得问题啦&＃xff0c;那就开始写代码吧
8.1.3 来个test_ros.py脚本测试测试
cd src/CornerNet-Lite/
gedit test_ros.py
test_ros.py内容如下
#!/usr/bin/env python3
#!coding&＃61;utf-8import sys
import rospyfrom std_msgs.msg import String
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeErrorimport cv2
print("cv2.__version__: ", cv2.__version__) from core.detectors import CornerNet_Saccade
from core.vis_utils import draw_bboxesdef callback(data):global count, bridge, detectorcount &＃43;&＃61; 1if count &＃61;&＃61;1:count &＃61; 0cv_img &＃61; bridge.imgmsg_to_cv2(data, "bgr8")bboxes &＃61; detector(cv_img)cv_img &＃61; draw_bboxes(cv_img, bboxes)cv2.imshow("frame", cv_img)cv2.waitKey(3)else:pass def image_pro():# In ROS, nodes are uniquely named. If two nodes with the same# name are launched, the previous one is kicked off. The# anonymous&＃61;True flag means that rospy will choose a unique# name for our &＃39;listener&＃39; node so that multiple listeners can# run simultaneously.rospy.init_node(&＃39;image_pro&＃39;,anonymous&＃61;True)global count,bridge,detectordetector &＃61; CornerNet_Saccade()count &＃61; 0bridge &＃61; CvBridge()rospy.Subscriber("/kinect2/qhd/image_color_rect", Image, callback)# spin() simply keeps python from exiting until this node is stoppedrospy.spin()if __name__ &＃61;&＃61; &＃39;__main__&＃39;:image_pro()
注意&＃xff1a;
第一行 #!/usr/bin/env python3 很重要
请修改订阅图像信息的话题名&＃xff0c;我这儿使用的Kinect2
rospy.Subscriber("/kinect2/sd/image_color_rect", Image, callback)
8.1.4 run起来&＃xff0c;go&＃xff01;go&＃xff01;go&＃xff01;
新开一个终端&＃xff0c;运行roscore
roscore
再新开一个终端&＃xff0c;运行相机节点&＃xff08;这里使用Kinect2&＃xff09;
rosrun kinect2_bridge kinect2_bridge
在之前那个激活了CornerNet_Lite环境的终端下运行test_ros.py
./test_ros.py 或者python test_ros.py
终端输出大概是这样的
cv2.__version__: 3.4.5
total parameters: 116969339
loading from /home/communicationgroup/py3_ws/src/CornerNet-Lite/core/../cache/nnet/CornerNet_Saccade/CornerNet_Saccade_500000.pkl
/home/communicationgroup/anaconda3/envs/CornerNet_Lite/lib/python3.7/site-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode&＃61;bilinear is changed to align_corners&＃61;False since 0.4.0. Please specify align_corners&＃61;True if the old behavior is desired. See the documentation of nn.Upsample for details."See the documentation of nn.Upsample for details.".format(mode))
嘻嘻&＃xff0c;稍等片刻&＃xff0c;画面出现了&＃xff0c;感谢刚哥友情出镜
效果不错啊&＃xff0c;框出来这么多&＃xff0c;美中不足就是有点卡。卡。卡。
8.1.5 出错总结&＃xff1a;
1&＃xff09;如果出现如下类似错误&＃xff0c;就是找不到&＃39;torch&＃39;&＃xff0c;应该是你没有激活环境
Traceback (most recent call last):File "./test_ros1.py", line 14, in
ImportError: No module named &＃39;torch&＃39;
解决办法&＃xff1a;在终端执行
conda activate CornerNet_Lite
2&＃xff09;如果出现如下错误&＃xff0c;就是cv_bridge的错误&＃xff0c;应该是你没有source 一下
[ERROR] [1556354686.771577]: bad callback:
Traceback (most recent call last):File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/topics.py", line 750, in _invoke_callbackcb(msg)File "./test_ros1.py", line 22, in callbackcv_img &＃61; bridge.imgmsg_to_cv2(data, "bgr8")File "/opt/ros/kinetic/lib/python2.7/dist-packages/cv_bridge/core.py", line 163, in imgmsg_to_cv2dtype, n_channels &＃61; self.encoding_to_dtype_with_channels(img_msg.encoding)File "/opt/ros/kinetic/lib/python2.7/dist-packages/cv_bridge/core.py", line 99, in encoding_to_dtype_with_channelsreturn self.cvtype2_to_dtype_with_channels(self.encoding_to_cvtype2(encoding))File "/opt/ros/kinetic/lib/python2.7/dist-packages/cv_bridge/core.py", line 91, in encoding_to_cvtype2from cv_bridge.boost.cv_bridge_boost import getCvType
ImportError: dynamic module does not define module export function (PyInit_cv_bridge_boost)
解决办法&＃xff1a;如果在py3_ws/src/CornerNet-Lite目录下的话&＃xff0c;在终端执行
source ../../devel/setup.bash
-------------------------------------------------------------我是分割线--------------------------------------------------------------------
2019/4/28更新&＃xff1a;
8.2 既然得到了目标的框框&＃xff0c;那就发布出来吧&＃xff0c;以待后续工程中能够应用
8.2.1 自定义BoundingBox消息
1&＃xff09;首先&＃xff0c;在py3_ws/src目录下创建cornernet_ros_msgs包&＃xff0c;里面主要是定义了我们自己的BoundingBox消息
cd py3_ws/src/
catkin_create_pkg cornernet_ros_msgs message_generation actionlib_msgs sensor_msgs std_msgs message_runtime
创建后会发现 py3_ws/src目录下生成了一个名为cornernet_ros_msgs的文件夹&＃xff0c;打开该文件夹&＃xff0c;我们可以看到里面生成了CMakeLists.txt 和 package.xml 。
2&＃xff09;进入cornernet_ros_msgs/目录&＃xff0c;并创建msg文件夹&＃xff0c;并在msg/目录下新建两个文件&＃xff0c;分别为BoundingBox.msg 和BoundingBoxes.msg
cd cornernet_ros_msgs/
mkdir msg
cd msg/
gedit BoundingBox.msg
BoundingBox.msg 中写入如下内容
string class
float64 prob
int64 x1
int64 y1
int64 x2
int64 y2
意思就是自定义了名为 BoundingBox 的消息类型&＃xff0c;其中class为类名&＃xff0c;prob也就是得分嘛&＃xff0c;可以根据这个得分滤掉低于设定阈值的框框&＃xff0c;后面的x1,y1,x2,y2就是boundingbox的坐标了。
保存一下&＃xff0c;再创建 BoundingBoxes.msg (很多时候框框不止一个)
gedit BoundingBoxes.msg
BoundingBoxes.msg 内容如下
Header header
BoundingBox[] bounding_boxes
意思就是里面有很多BoundingBox&＃xff0c;存在数组&＃xff08;或者叫列表&＃xff09;中。
3&＃xff09;接下来就是修改修改CMakeLists.txt 和 package.xml 中的内容了。
CMakeLists.txt
cmake_minimum_required(VERSION 2.8.3)
project(cornernet_ros_msgs)## Compile as C&＃43;&＃43;11, supported in ROS Kinetic and newer
add_compile_options(-std&＃61;c&＃43;&＃43;11)find_package(catkin REQUIRED COMPONENTSroscpp rospy message_generationactionlib_msgsmessage_runtimesensor_msgsstd_msgs
)# find_package(Boost REQUIRED COMPONENTS system)add_message_files(FILESBoundingBox.msgBoundingBoxes.msg)generate_messages(DEPENDENCIESactionlib_msgssensor_msgsstd_msgs)catkin_package(CATKIN_DEPENDS actionlib_msgs message_runtimesensor_msgs std_msgs
)
package.xml
4&＃xff09;那就开始编译吧&＃xff08;就像前面编译cv_bridge一样&＃xff09;
cd ../../../ #回到py3_ws/目录下
catkin init
终端输出大概是这样的
Catkin workspace &＃96;/home/communicationgroup/py3_ws&＃96; is already initialized. No action taken.
--------------------------------------------------------------------------------
Profile: default
Extending: [cached] /home/communicationgroup/catkin_ws/devel:/home/communicationgroup/rgbdslam2_catkin_ws/devel:/opt/ros/kinetic
Workspace: /home/communicationgroup/py3_ws
--------------------------------------------------------------------------------
Build Space: [exists] /home/communicationgroup/py3_ws/build
Devel Space: [exists] /home/communicationgroup/py3_ws/devel
Install Space: [exists] /home/communicationgroup/py3_ws/install
Log Space: [exists] /home/communicationgroup/py3_ws/logs
Source Space: [exists] /home/communicationgroup/py3_ws/src
DESTDIR: [unused] None
--------------------------------------------------------------------------------
Devel Space Layout: linked
Install Space Layout: merged
--------------------------------------------------------------------------------
Additional CMake Args: -DPYTHON_EXECUTABLE&＃61;/usr/bin/python3 -DPYTHON_INCLUDE_DIR&＃61;/usr/include/python3.5m -DPYTHON_LIBRARY&＃61;/usr/lib/x86_64-linux-gnu/libpython3.5m.so
Additional Make Args: None
Additional catkin Make Args: None
Internal Make Job Server: True
Cache Job Environments: False
--------------------------------------------------------------------------------
Whitelisted Packages: None
Blacklisted Packages: None
--------------------------------------------------------------------------------
Workspace configuration appears valid.
--------------------------------------------------------------------------------
他说我已经initialized。。。&＃xff08;或许不必执行catkin init???&＃xff09;
既然这样&＃xff0c;那就直接编译吧
catkin build cornernet_ros_msgs
终端输出大概是这样的
--------------------------------------------------------------------------------
Profile: default
Extending: [cached] /home/communicationgroup/catkin_ws/devel:/home/communicationgroup/rgbdslam2_catkin_ws/devel:/opt/ros/kinetic
Workspace: /home/communicationgroup/py3_ws
--------------------------------------------------------------------------------
Build Space: [exists] /home/communicationgroup/py3_ws/build
Devel Space: [exists] /home/communicationgroup/py3_ws/devel
Install Space: [exists] /home/communicationgroup/py3_ws/install
Log Space: [exists] /home/communicationgroup/py3_ws/logs
Source Space: [exists] /home/communicationgroup/py3_ws/src
DESTDIR: [unused] None
--------------------------------------------------------------------------------
Devel Space Layout: linked
Install Space Layout: merged
--------------------------------------------------------------------------------
Additional CMake Args: -DPYTHON_EXECUTABLE&＃61;/usr/bin/python3 -DPYTHON_INCLUDE_DIR&＃61;/usr/include/python3.5m -DPYTHON_LIBRARY&＃61;/usr/lib/x86_64-linux-gnu/libpython3.5m.so
Additional Make Args: None
Additional catkin Make Args: None
Internal Make Job Server: True
Cache Job Environments: False
--------------------------------------------------------------------------------
Whitelisted Packages: None
Blacklisted Packages: None
--------------------------------------------------------------------------------
Workspace configuration appears valid.
--------------------------------------------------------------------------------
[build] Found &＃39;5&＃39; packages in 0.0 seconds.
Starting >>> cornernet_ros_msgs
Finished <<
[build] Ignored: 4 packages were skipped or are blacklisted.
[build] Warnings: None.
[build] Abandoned: None.
[build] Failed: None.
[build] Runtime: 0.9 seconds total.
我们最喜欢看到的就是succeed&＃xff01;
6) 敲代码&＃xff0c;敲代码&＃xff0c;把框框发布出去&＃xff08;只需要对前面的test_ros.py稍作修改&＃xff09;
暂且命名为test_ros1.py吧&＃xff0c;内容如下&＃xff0c;且放在py3_ws/src/CornerNet-Lite/目录下
#!/usr/bin/env python3
#!coding&＃61;utf-8import sys
import numpy as npimport rospyfrom std_msgs.msg import String
from sensor_msgs.msg import Image
from cv_bridge import CvBridge, CvBridgeError#自己定义的BoundingBoxes消息
from cornernet_ros_msgs.msg import BoundingBoxes
from cornernet_ros_msgs.msg import BoundingBoximport cv2
print("cv2.__version__: ", cv2.__version__) from core.detectors import CornerNet_Saccade
from core.vis_utils import draw_bboxesdef pub_bboxes(bboxes, thresh&＃61;0.5):"""Public bounding boxes msgs.Args:bboxes: A dictionary representing bounding boxes of different objectcategories, where the keys are the names of the categories and thevalues are the bounding boxes. The bounding boxes of category should bestored in a 2D NumPy array, where each row is a bounding box (x1, y1,x2, y2, score).thresh: (Optional) Only bounding boxes with scores above the thresholdwill be drawn.Returns:None""" global pubbboxes_msg &＃61; BoundingBoxes()for cat_name in bboxes:keep_inds &＃61; bboxes[cat_name][:, -1] > threshfor bbox in bboxes[cat_name][keep_inds]:bbox_msg &＃61; BoundingBox( str(cat_name), #class namebbox[-1].astype(np.float), #prabbbox[0].astype(np.int32), #x1bbox[1].astype(np.int32), #y1bbox[2].astype(np.int32), #x2bbox[3].astype(np.int32)) #y2bboxes_msg.bounding_boxes.append(bbox_msg) # print(bboxes_msg)pub.publish(bboxes_msg) def callback(data):global count, bridge, detectorcount &＃43;&＃61; 1if count &＃61;&＃61;1:count &＃61; 0cv_img &＃61; bridge.imgmsg_to_cv2(data, "bgr8")bboxes &＃61; detector(cv_img)cv_img &＃61; draw_bboxes(cv_img, bboxes)cv2.imshow("frame", cv_img)pub_bboxes(bboxes) #发布消息cv2.waitKey(3)else:pass def image_pro():# In ROS, nodes are uniquely named. If two nodes with the same# name are launched, the previous one is kicked off. The# anonymous&＃61;True flag means that rospy will choose a unique# name for our &＃39;listener&＃39; node so that multiple listeners can# run simultaneously.rospy.init_node(&＃39;image_pro&＃39;,anonymous&＃61;True)global count,bridge,detector,pubdetector &＃61; CornerNet_Saccade()count &＃61; 0bridge &＃61; CvBridge()rospy.Subscriber("/kinect2/qhd/image_color_rect", Image, callback)pub &＃61; rospy.Publisher(&＃39;boundingboxes&＃39;,BoundingBoxes,queue_size&＃61;10)# spin() simply keeps python from exiting until this node is stoppedrospy.spin()if __name__ &＃61;&＃61; &＃39;__main__&＃39;:image_pro()
7&＃xff09;试一试&＃xff0c;看看消息能发布出来不&＃xff08;看看程序能跑不&＃xff09;
先激活CornerNet_Lite环境&＃xff0c;并source一下工作空间
cd py3_ws/src/CornerNet-Lite
conda activate CornerNet_Lite
source ../../devel/setup.bash
运行test_ros1.py&＃xff08;假设你已经启动了roscore 和相机节点&＃xff0c;参考8.1.4节&＃xff09;
./test_ros1.py
东西框出来了&＃xff0c;那消息发布出去了么&＃xff1f;那就rostopic list看一看
新开一个终端
rostopic list
这是终端会输出ROS发布的所有消息&＃xff0c;我的大概是这样的&＃xff0c;很棒&＃xff0c;第一个就是我们需要的
communicationgroup&＃64;Z370:~/py3_ws/src/CornerNet-Lite$ rostopic list
/boundingboxes
/kinect2/cap/camera_info
/kinect2/cap/cap_bgr
/kinect2/cap/cap_bgr/compressed
/kinect2/cap/cap_mono
/kinect2/cap/cap_mono/compressed
/kinect2/hd/camera_info
/kinect2/hd/image_color
/kinect2/hd/image_color/compressed
/kinect2/hd/image_color_rect
/kinect2/hd/image_color_rect/compressed
/kinect2/hd/image_depth_rect
/kinect2/hd/image_depth_rect/compressed
/kinect2/hd/image_mono
/kinect2/hd/image_mono/compressed
/kinect2/hd/image_mono_rect
/kinect2/hd/image_mono_rect/compressed
/kinect2/qhd/camera_info
/kinect2/qhd/image_color
/kinect2/qhd/image_color/compressed
/kinect2/qhd/image_color_rect
/kinect2/qhd/image_color_rect/compressed
/kinect2/qhd/image_depth_rect
/kinect2/qhd/image_depth_rect/compressed
/kinect2/qhd/image_mono
/kinect2/qhd/image_mono/compressed
/kinect2/qhd/image_mono_rect
/kinect2/qhd/image_mono_rect/compressed
/kinect2/sd/camera_info
/kinect2/sd/image_color_rect
/kinect2/sd/image_color_rect/compressed
/kinect2/sd/image_depth
/kinect2/sd/image_depth/compressed
/kinect2/sd/image_depth_rect
/kinect2/sd/image_depth_rect/compressed
/kinect2/sd/image_ir
/kinect2/sd/image_ir/compressed
/kinect2/sd/image_ir_rect
/kinect2/sd/image_ir_rect/compressed
/rosout
/rosout_agg
再用rostopic echo /boundingboxes 看看里面具体是些啥
communicationgroup&＃64;Z370:~/py3_ws/src/CornerNet-Lite$ rostopic echo /boundingboxes
ERROR: Cannot load message class for [cornernet_ros_msgs/BoundingBoxes]. Are your messages built?
哎呀&＃xff0c;看到ERROR就尴尬了&＃xff01;其实这都是小问题啦&＃xff0c;因为新开这个终端没有source工作空间&＃xff0c;他怎么理解我们自己定义的消息呢&＃xff0c;那就source一下
source ~/py3_ws/devel/setup.bash
然后再echo
rostopic echo /boundingboxes
终端输出大概是这样的
communicationgroup&＃64;Z370:~/py3_ws/src/CornerNet-Lite$ rostopic echo /boundingboxes
header: seq: 1stamp: secs: 0nsecs: 0frame_id: &＃39;&＃39;
bounding_boxes: - class_: "bottle"prob: 0.650504469872x1: 1y1: 300x2: 43y2: 444- class_: "bottle"prob: 0.597384572029x1: 123y1: 310x2: 154y2: 381- class_: "cup"prob: 0.739920139313x1: 94y1: 379x2: 182y2: 472- class_: "chair"prob: 0.736968696117x1: 719y1: 368x2: 917y2: 536- class_: "tv"prob: 0.689738988876x1: 283y1: 246x2: 428y2: 350- class_: "mouse"prob: 0.543673157692x1: 381y1: 388x2: 423y2: 408- class_: "keyboard"prob: 0.587634801865x1: 228y1: 386x2: 369y2: 427- class_: "book"prob: 0.500504910946x1: 501y1: 291x2: 590y2: 306
---
没错&＃xff0c;这就是我们想要的。以后具体应用的时候&＃xff0c;还是该把Header加上&＃xff0c;这样就可以根据时间戳等信息知道这些框框对应哪一帧图像的。
最后备注&＃xff1a;
代码中detector&＃61;CornerNet_Saccade()&＃xff0c;是否可以改一改呀&＃xff1f;不是有三个models嘛&＃xff0c;这个就留给大家去探索吧&＃xff08;悄悄告诉大家&＃xff0c;CornerNet_Squeeze很快哦&＃xff09;
那就这样吧&＃xff0c;本篇博客正式完结。
参考链接&＃xff1a;
https://github.com/princeton-vl/CornerNet-Lite
https://stackoverflow.com/questions/49221565/unable-to-use-cv-bridge-with-ros-kinetic-and-python3?rq&＃61;1
https://community.bwbot.org/topic/499/%E5%9C%A8ros%E4%B8%AD%E4%BD%BF%E7%94%A8python3
京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有 