python+opencv+caffe+摄像头做目标检测的实例代码

yipeiwu_com5年前Python基础

首先之前已经成功的使用Python做图像的目标检测,这回因为项目最终是需要用摄像头的,

所以实现摄像头获取图像,并且用Python调用CAFFE接口来实现目标识别

首先是摄像头请选择支持Linux万能驱动兼容V4L2的摄像头,

因为之前用学ARM的时候使用的Smart210,我已经确认我的摄像头是支持的,

我把摄像头插上之後自然就在 /dev 目录下看到多了一个video0的文件,

这个就是摄像头的设备文件了,所以我就没有额外处理驱动的部分

一、检测环境

再来在开始前因为之前按着国嵌的指导手册安装的opencv3.2当时没有开启V4L2及GTK_2.x的支持,

所以後面遇到了一连串的问题,请大家如下面方法检测

$ python

1.检测Python的V4L2支持及摄像头驱动是否正常

进入Python之後如下命令

Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> cap = cv2.VideoCapture(0)
>>> print cap.isOpened()
True
>>>

如果 返回True就代表摄像头及你的opencv的V4L2支持就已经完全正常了

如果返回False就代表opencv或是摄像头有问题叁考後面的修改方式

2.再来因为我们要把摄像头的影像生成窗口,所以我们需要检测Python的gtk支持如下

>>> import cv2
>>> cv2.namedWindow('test',cv2.WINDOW_AUTOSIZE)
>>> 

如果没有报任何错误就代表gtk也是正常的

如如果出现下面提示

OpenCV Error: Unspecified error (The function is not implemented. Rebuild the library with Windows,
 GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config, 
then re-run cmake or configure script) in cvNamedWindow,

那就代表opencv不支持gtk

如果上面两个测试都是好使的你可以跳过本步骤

首先我发现我不论如何重新编译opencv发现python一直都是有问题的,

最後发现是因为不知道什麽时候已经装过 python-opencv的包了

可以用命令

$ pip list |grep opencv

如果发现有任何跟opencv有关的包都可以利用 pip uninstall xxx 来移除

假设看到 pythom-opencv的包那就 pip uninstall opencv-python 来移除

还有检查dpkg -i |grep opencv 如果跟python的opencv有关的包也得移除

$ sudo apt-get remove python-opencv

然後到你之前安装opencv3.2的那个源码目录,

$ cd build
$ make uninstall

这样就会卸载之前安装的opencv

再来V4L的头文件已经改名了,但是opencv会默认使用linux/videodev.h所以要做个软鍊接

$ ln -s /usr/include/libv4l1-videodev.h /usr/include/linux/videodev.h

opencv安装过程中会自动的检测相关的包,以及一些依赖,

先列出我安装的包,但是因为环境多少有点不同,下面会教大家如何看缺少的包

$ sudo apt-get install libgphoto2-dev v4l2ucp libv4l-dev dv4l libwebcam0-dev libgtkglext1-dev libunicap2-dev 

再来执行cmake

$ cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local \
-D WITH_TBB=ON -D WITH_V4L=ON -D WITH_GTK=ON -D WITH_GTK_2_X -D WITH_OPENGL=ON \
-D WITH_CUDA=ON -D WITH_CUBLAS=ON -D BUILD_PYTHON_SUPPORT=ON \
-D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib-3.2.0/modules/ ../opencv-3.2.0

其中如果V4L2使用share库也可以把-D WITH_V4L: = ON 换成

-D WITH_LIBV4L=ON

在cmake的过程中会有类似如下的提示

Detected version of GNU GCC: 54 (504)
FP16: Feature disabled
Found OpenEXR: /usr/lib/x86_64-linux-gnu/libIlmImf.so
Checking for module 'libucil'
 No package 'libucil' found
Looking for linux/videodev.h
Looking for linux/videodev.h - found
Looking for linux/videodev2.h
Looking for linux/videodev2.h - found
Looking for sys/videoio.h
Looking for sys/videoio.h - not found
Checking for module 'libavresample'
 No package 'libavresample' found
Found TBB: build
found IPP (ICV version): 9.0.1 [9.0.1]
at: /mnt/sdb/ubuntu/install/opencv/build/3rdparty/ippicv/ippicv_lnx
CUDA detected: 8.0
CUDA NVCC target flags: -gencode;arch=compute_20,code=sm_20;-gencode;arch=compute_30,code=sm_30;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-D_FORCE_INLINES
LAPACK_IMPL: Atlas, LAPACK_LIBRARIES: /usr/lib/liblapack.so;/usr/lib/libcblas.so;/usr/lib/libatlas.so
Could NOT find JNI (missing: JAVA_AWT_LIBRARY JAVA_JVM_LIBRARY JAVA_INCLUDE_PATH JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH) 
Could NOT find Matlab (missing: MATLAB_MEX_SCRIPT MATLAB_INCLUDE_DIRS MATLAB_ROOT_DIR MATLAB_LIBRARIES MATLAB_LIBRARY_DIRS MATLAB_MEXEXT MATLAB_ARCH MATLAB_BIN) 
VTK is not found. Please set -DVTK_DIR in CMake to VTK build directory, or to VTK install subdirectory with VTKConfig.cmake file
 
General configuration for OpenCV 3.2.0 =====================================
 Version control:    unknown
 
 Platform:
 Timestamp:     2017-08-14T00:31:55Z
 Host:      Linux 4.10.0-30-generic x86_64
 CMake:      3.5.1
 CMake generator:    Unix Makefiles
 CMake build tool:   /usr/bin/make
 Configuration:    Release
 
 C/C++:
 Built as dynamic libs?:  YES
 C++ Compiler:    /usr/bin/c++ (ver 5.4.0)
 C++ flags (Release):   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DNDEBUG
 C++ flags (Debug):   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -g -O0 -DDEBUG -D_DEBUG
 C Compiler:     /usr/bin/cc
 C flags (Release):   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wno-narrowing -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -O3 -DNDEBUG -DNDEBUG
 C flags (Debug):    -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wno-narrowing -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -g -O0 -DDEBUG -D_DEBUG
 Linker flags (Release):
 Linker flags (Debug):
 ccache:      NO
 Precompiled headers:   YES
 Extra dependencies:   /usr/lib/x86_64-linux-gnu/libpng.so /usr/lib/x86_64-linux-gnu/libtiff.so /usr/lib/x86_64-linux-gnu/libjasper.so /usr/lib/x86_64-linux-gnu/libImath.so /usr/lib/x86_64-linux-gnu/libIlmImf.so /usr/lib/x86_64-linux-gnu/libIex.so /usr/lib/x86_64-linux-gnu/libHalf.so /usr/lib/x86_64-linux-gnu/libIlmThread.so gtk-x11-2.0 gdk-x11-2.0 pangocairo-1.0 atk-1.0 cairo gdk_pixbuf-2.0 gio-2.0 pangoft2-1.0 pango-1.0 gobject-2.0 glib-2.0 fontconfig freetype gthread-2.0 gtkglext-x11-1.0 gdkglext-x11-1.0 GLU GL Xmu Xt SM ICE pangox-1.0 X11 gmodule-2.0 gstbase-1.0 gstreamer-1.0 gstvideo-1.0 gstapp-1.0 gstriff-1.0 gstpbutils-1.0 unicap dc1394 avcodec-ffmpeg avformat-ffmpeg avutil-ffmpeg swscale-ffmpeg gphoto2 gphoto2_port exif dl m pthread rt /usr/lib/x86_64-linux-gnu/libGLU.so /usr/lib/x86_64-linux-gnu/libGL.so /usr/lib/liblapack.so /usr/lib/libcblas.so /usr/lib/libatlas.so cudart nppc nppi npps cublas cufft -L/usr/local/cuda-8.0/lib64
 3rdparty dependencies:  libwebp zlib libjpeg tbb
 
 OpenCV modules:
 To be built:     cudev core cudaarithm flann imgproc ml video cudabgsegm cudafilters cudaimgproc cudawarping imgcodecs photo shape videoio cudacodec highgui objdetect ts features2d calib3d cudafeatures2d cudalegacy cudaobjdetect cudaoptflow cudastereo stitching superres videostab python2
 Disabled:     world
 Disabled by dependency:  -
 Unavailable:     java python3 viz
 
 GUI: 
 QT:       NO
 GTK+ 2.x:     YES (ver 2.24.30)
 GThread :     YES (ver 2.48.2)
 GtkGlExt:     YES (ver 1.2.0)
 OpenGL support:    YES (/usr/lib/x86_64-linux-gnu/libGLU.so /usr/lib/x86_64-linux-gnu/libGL.so)
 VTK support:     NO
 
 Media I/O: 
 ZLib:      build (ver 1.2.8)
 JPEG:      libjpeg (ver 90)
 WEBP:      build (ver 0.3.1)
 PNG:       /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.2.54)
 TIFF:      /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 - 4.0.6)
 JPEG 2000:     /usr/lib/x86_64-linux-gnu/libjasper.so (ver 1.900.1)
 OpenEXR:      /usr/lib/x86_64-linux-gnu/libImath.so /usr/lib/x86_64-linux-gnu/libIlmImf.so /usr/lib/x86_64-linux-gnu/libIex.so /usr/lib/x86_64-linux-gnu/libHalf.so /usr/lib/x86_64-linux-gnu/libIlmThread.so (ver 2.2.0)
 GDAL:      NO
 GDCM:      NO
 
 Video I/O:
 DC1394 1.x:     NO
 DC1394 2.x:     YES (ver 2.2.4)
 FFMPEG:      YES
  avcodec:     YES (ver 56.60.100)
  avformat:     YES (ver 56.40.101)
  avutil:     YES (ver 54.31.100)
  swscale:     YES (ver 3.1.101)
  avresample:    NO
 GStreamer:     
  base:      YES (ver 1.8.3)
  video:      YES (ver 1.8.3)
  app:      YES (ver 1.8.3)
  riff:      YES (ver 1.8.3)
  pbutils:     YES (ver 1.8.3)
 OpenNI:      NO
 OpenNI PrimeSensor Modules: NO
 OpenNI2:      NO
 PvAPI:      NO
 GigEVisionSDK:    NO
 Aravis SDK:     NO
 UniCap:      NO
 UniCap ucil:     NO
 V4L/V4L2:     YES/YES
 XIMEA:      NO
 Xine:      NO
 gPhoto2:      YES
 
 Parallel framework:   TBB (ver 4.4 interface 9002)
 
 Other third-party libraries:
 Use IPP:      9.0.1 [9.0.1]
   at:      /mnt/sdb/ubuntu/install/opencv/build/3rdparty/ippicv/ippicv_lnx
 Use IPP Async:    NO
 Use VA:      NO
 Use Intel VA-API/OpenCL:  NO
 Use Lapack:     YES (/usr/lib/liblapack.so /usr/lib/libcblas.so /usr/lib/libatlas.so)
 Use Eigen:     NO
 Use Cuda:     YES (ver 8.0)
 Use OpenCL:     YES
 Use OpenVX:     NO
 Use custom HAL:    NO
 
 NVIDIA CUDA
 Use CUFFT:     YES
 Use CUBLAS:     YES
 USE NVCUVID:     NO
 NVIDIA GPU arch:    20 30 35 37 50 52 60 61
 NVIDIA PTX archs:
 Use fast math:    NO
 
 OpenCL:      <Dynamic loading of OpenCL library>
 Include path:    /mnt/sdb/ubuntu/install/opencv/opencv-3.2.0/3rdparty/include/opencl/1.2
 Use AMDFFT:     NO
 Use AMDBLAS:     NO
 
 Python 2:
 Interpreter:     /usr/bin/python2.7 (ver 2.7.12)
 Libraries:     /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.12)
 numpy:      /usr/local/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.1)
 packages path:    lib/python2.7/dist-packages
 
 Python 3:
 Interpreter:     /usr/bin/python3 (ver 3.5.2)
 
 Python (for build):   /usr/bin/python2.7
 
 Java:
 ant:       NO
 JNI:       NO
 Java wrappers:    NO
 Java tests:     NO
 
 Matlab:      Matlab not found or implicitly disabled
 
 Documentation:
 Doxygen:      /usr/bin/doxygen (ver 1.8.11)
 
 Tests and samples:
 Tests:      YES
 Performance tests:   YES
 C/C++ Examples:    NO
 
 Install path:     /usr/local
 
 cvconfig.h is in:    /mnt/sdb/ubuntu/install/opencv/build
-----------------------------------------------------------------
 
Configuring done

开始的部份会检查一堆依赖,请仔细看如可有缺包,可以百度一下怎么安装,因为我已经装完了就不提示了,

不过有些包我也找不到我就没解决了如上面的

Checking for module 'libucil'
 No package 'libucil' found

这个我还真没找到怎么解决,不过反正问题不大

再来下面的部份一定要注意几个部分一定要有

 GUI: 
 GTK+ 2.x:     YES (ver 2.24.30)
Video I/O:
 V4L/V4L2:     YES/YES
 FFMPEG:      YES
 Python 2:
 Interpreter:     /usr/bin/python2.7 (ver 2.7.12)
 Libraries:     /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.12)
 numpy:      /usr/local/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.1)
 packages path:    lib/python2.7/dist-packages

主要就是上面几个个非常重要,必須要装上

再来就正常 make

$ make -j8
$ make install

设置nccl的ld环境

$ vi /etc/ld.so.conf.d/nccl.conf

加上下面目录

/usr/local/nccl/lib/

然后执行

$ sudo ldconfig

安装完后回python按步骤一再次检查是否环境都好使了

二、撰写Python测试程序

# -*- coding:utf-8 -*-
# 用于模型的单张图像分类操作
import os
os.environ['GLOG_minloglevel'] = '2' # 将caffe的输出log信息不显示,必须放到import caffe前
import caffe # caffe 模块
from caffe.proto import caffe_pb2
from google.protobuf import text_format
import numpy as np
import cv2
import matplotlib.pyplot as plt
import time
import skimage.io
 
global num
num = 0
 
 
 
def detect(image1,net):
 # 传进来的image1的dtype为uint8
 # print image1.shape
 # print image1.dtype
 # print image1.size
 
 # image = np.array(image1, dtype=np.float32)
 # image = caffe.io.resize_image(image1, (480, 640))
 image = skimage.img_as_float(image1).astype(np.float32)
 # image = caffe.io.resize_image(image2, (300, 300))
 
 # skimage.io.imsave("photo.png", image)
 # cv2.imwrite("photo.png", image)
 # image = caffe.io.load_image(caffe_root + 'examples/images/bird.jpg')
 # 以下方式读取的imaged的dtype为float32
 # image = caffe.io.load_image(caffe_root + 'photo.png')
 # image = caffe.io.load_image(image1)
 
 # 改变dtype
 # image.dtype = 'float32'
 # print 'mode:'+image.mode
 # print image.shape
 # print image.dtype
 # print image.size
 
 # plt.imshow(image)
 
 # * Run the net and examine the top_k results
 # In[5]:
 global num
 num += 1
 print 'image num:' + str(num)
 
 transformed_image = transformer.preprocess('data', image)
 net.blobs['data'].data[...] = transformed_image
 
 time_start=time.time()
 # Forward pass.
 net.forward()
	
 time_end=time.time() 
 print 'time:' + str(time_end-time_start) + ' s'
 
 
 
 loc = net.blobs['bbox-list'].data[0]
 print(loc)
 #查看了结构文件发现在CAFFE一开始图像输入的时候就已经将图片缩小了,宽度1248高度384
 #然后我们在net.blobs['bbox-list'].data得到的是侦测到的目标座标,但是是相对于1248*384的
 #所以我们要把座标转换回相对原大小的位置,下面im.shape是保存在原尺寸的宽高,
 for l in range(len(loc)):
		xmin = int(loc[l][0] * image.shape[1] / 1248)
		ymin = int(loc[l][1] * image.shape[0] / 384)
		xmax = int(loc[l][2] * image.shape[1] /1248)
		ymax = int(loc[l][3] * image.shape[0] / 384)
		#在该座标位置画一个方框
		cv2.rectangle(image, (xmin, ymin), (xmax, ymax), (55 / 255.0, 255 / 255.0, 155 / 255.0), 2)
 # 显示结果
 
 #plt.imshow(image, 'brg')
 #plt.show()
 cv2.imshow('img', image)
 
 
 
def show_info(cam):
 print 'POS_FRAMES:'+str(cam.get(1))
 print 'FRAME_COUNT:'+str(cam.get(7))
 print 'FORMAT:'+str(cam.get(8))
 print 'MODE:'+str(cam.get(9))
 print 'SATURATION:'+str(cam.get(12))
 print 'FPS:'+str(cam.get(5))
 
#CPU或GPU模型转换
caffe.set_mode_gpu()
#caffe.set_mode_cpu()
#caffe.set_device(0)
 
caffe_root = '/var/smb/work/mycode/'
# 网络参数(权重)文件
caffemodel = caffe_root + 'module/detectnet/snapshot_iter_2391.caffemodel'
# 网络实施结构配置文件
deploy = caffe_root + 'module/detectnet/deploy.prototxt'
 
 
img_root = caffe_root + 'data/'
 
# 网络实施分类
net = caffe.Net(deploy, # 定义模型结构
    caffemodel, # 包含了模型的训练权值
    caffe.TEST) # 使用测试模式(不执行dropout)
 
# 加载ImageNet图像均值 (随着Caffe一起发布的)
print(os.environ['PYTHONPATH'])
#mu = np.load(os.environ['PYTHONPATH'] + '/caffe/imagenet/ilsvrc_2012_mean.npy')
#mu = mu.mean(1).mean(1) # 对所有像素值取平均以此获取BGR的均值像素值
 
# 图像预处理
transformer = caffe.io.Transformer({'data': net.blobs['data'].data.shape})
transformer.set_transpose('data', (2,0,1))
#transformer.set_mean('data', mu)
transformer.set_raw_scale('data', 255)
transformer.set_channel_swap('data', (2,1,0))
 
# 处理图像
cam = cv2.VideoCapture(0) 
if cam.isOpened():
 cam.set(3, 400)
 cam.set(4, 300)
 cam.set(5, 3)
 time.sleep(6)
 cam.set(15, -8.0)
 size = (int(cam.get(3)), int(cam.get(4)))
 print 'size:'
 print size
 
cv2.namedWindow('img', cv2.WINDOW_NORMAL)
 
# cnt=2
# while cnt:
#  cnt -= 1
while cam.isOpened():
 ret, img = cam.read()
 if ret:
  #show_info(cam)
  detect(img,net)
 
 if 0xFF == ord('q') & cv2.waitKey(5) == 27:
  break
 # time.sleep(0.033)
cam.release()
cv2.destroyAllWindows()

介面上会打印bbox也就是侦测到的目标在图像的座标,另外请自行修改python代码里的相关目录,

我用自己训练的KITTI数据集,用于侦测车辆,因为拍不到车子拿手机欺骗一下,好使

以上这篇python+opencv+caffe+摄像头做目标检测的实例代码就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持【听图阁-专注于Python设计】。

相关文章

Python assert语句的简单使用示例

写在前面 最近每日一更,我这菜鸡都有点儿不好意思了 简单介绍 简单用法是: assert expression 让我们用程序来测试这个expression,如果expressio...

Python实现的选择排序算法原理与用法实例分析

Python实现的选择排序算法原理与用法实例分析

本文实例讲述了Python实现的选择排序算法。分享给大家供大家参考,具体如下: 选择排序(Selection sort)是一种简单直观的排序算法。它的工作原理是每一次从待排序的数据元素中...

python模块smtplib实现纯文本邮件发送功能

python模块smtplib实现纯文本邮件发送功能

今天学到了如何使用Python的smtplib库发送邮件,中间也是遇到了各种各样的错误和困难,还好都一一的解决了。下面来谈一谈我的这段经历。 配置你的邮箱 为什么要配置邮箱呢?具体要配置...

Python语言描述连续子数组的最大和

题目描述 HZ偶尔会拿些专业问题来忽悠那些非计算机专业的同学。今天测试组开完会后,他又发话了:在古老的一维模式识别中,常常需要计算连续子向量的最大和,当向量全为正数的时候,问题很好解决。...

Python面向对象之类和实例用法分析

本文实例讲述了Python面向对象之类和实例用法。分享给大家供大家参考,具体如下: 类 虽然 Python 是解释性语言,但是它是面向对象的,能够进行对象编程。至于何为面向对象,在此就不...