CSI-camera-2.0.8

From Tinker Board Wiki
Revision as of 09:02, 18 January 2019 by WikiSysop (talk | contribs)
Jump to navigation Jump to search

Camera Config

SETTING UP THE CAMERA HARDWARE

Csi.png

Warning: Cameras are sensitive to static. Earth yourself prior to handling the PCB. A sink tap or similar should suffice if you don’t have an earthing strap.
The camera board attaches to the Tinker Board via a 15-way ribbon cable. There are only two connections to make: the ribbon cable needs to be attached to the camera PCB, and to the Tinker Board itself. You need to get the cable the right way round, or the camera will not work. On the camera PCB, the blue backing on the cable should face away from the PCB, and on the Tinker Board it should face towards the Ethernet connection.

Although the connectors on the PCB and the Tinker Board are different, they work in a similar way. On the Tinker Board itself, pull up the tabs on each end of the connector. It should slide up easily, and be able to pivot around slightly. Fully insert the ribbon cable into the slot, ensuring it is set straight, then gently press down the tabs to clip it into place. The camera PCB connector also requires you to pull the tabs away from the board, gently insert the cable, then push the tabs back. The PCB connector can be a little more awkward than the one on the Tinker Board itself.

SETTING UP THE CAMERA SOFTWARE

Camera software is default built inside the release image. You don’t need to do extra setup.
To test whether the Camera is working, try the following Gstreamer command:
 

$ gst-launch-1.0 rkcamsrc io-mode=4 ! video/x-raw,format=NV12,width=640,height=480 ! rkximagesink
 

The display should show a preview from the Camera, whilst displaying various informational messages.
 

TROUBLESHOOTING

If the Camera Module isn't working correctly, please use below command to check whether the Camera Module be detected by Tinker Board.

$ sudo apt-get install v4l-utils
$ media-ctl -p

You will see imx219 or ov5647 in entity 7. If not, there are number of things to try:

  • Is the ribbon cable attached to the Camera Serial Interface (CSI), not the Display Serial Interface (DSI)? The ribbon connector will fit into either port. The Camera port is located near the HDMI connector.
  • Are the ribbon connectors all firmly seated, and are they the right way round? They must be straight in their sockets.
  • Is the Camera Module connector, between the smaller black Camera Module itself and the PCB, firmly attached? Sometimes this connection can come loose during transit or when putting the Camera Module in a case. Using a fingernail, flip up the connector on the PCB, then reconnect it with gentle pressure. It engages with a very slight click. Don't force it; if it doesn't engage, it's probably slightly misaligned.
  • Is your power supply sufficient? The adapter should be 5V/3A that is more stable for the use of the camera.

USING THE CAMERA

The CSI Camera of Tinker Board support IMX219 and OV5647. Here describes the basic usage of Camera.

Switch Camera (default is IMX219)

#switch to ov5647(Raspberry Pi Camera Module V1)
$ sudo tinker-config (Interfacing Options --> Camera --> OV5647)
$ sudo reboot

#switch to imx219(Raspberry Pi Camera Module V2)
$ sudo tinker-config (Interfacing Options --> Camera --> IMX219)
$ sudo reboot

Gstreamer

#preview
$ gst-launch-1.0 rkcamsrc device=/dev/video0 io-mode=4 ! video/x-raw,format=NV12,width=640,height=480 ! rkximgesink
$ gst-launch-1.0 rkcamsrc device=/dev/video0 io-mode=4 isp-mode=2A tuning-xml-path=/etc/cam_iq/IMX219.xml ! video/x-raw,format=NV12,width=640,height=480 ! rkximgesink

#capture
$ gst-launch-1.0 rkcamsrc num-buffers=10 io-mode=4 ! video/x-raw,format=NV12,width=640,height=480 ! jpegenc ! multifilesink location=image_%02d.jpg
$ gst-launch-1.0 rkcamsrc num-buffers=10 device=/dev/video1 io-mode=4 ! video/x-raw,format=NV12,width=1920,height=1080 ! jpegenc ! multifilesink location=image_%02d.jpg
$ gst-launch-1.0 rkcamsrc num-buffers=10 io-mode=4 ! video/x-raw,format=NV12,width=640,height=480 ! jpegenc ! multifilesink location=image.jpg
$ gst-launch-1.0 rkcamsrc num-buffers=10 io-mode=4 isp-mode=2A tuning-xml-path=/etc/cam_iq/IMX219.xml ! video/x-raw,format=NV12,width=640,height=480 ! jpegenc ! multifilesink location=image.jpg

#recording
$ gst-launch-1.0 rkcamsrc num-buffers=512 io-mode=4 isp-mode=2A tuning-xml-path=/etc/cam_iq/IMX219 ! video/x-raw,format=NV12,width=640,height=480,framerate=30/1 ! queue ! mpph264enc ! queue ! h264parse ! mpegtsmux ! filesink location=/home/linaro/vga.ts
#show picture
$ gst-launch-1.0 playbin uri=file:///home//linaro//image.jpg
$ gst-launch-1.0 filesrc location=image.jpg ! decodebin ! imagefreeze ! autovideosink
 
#play video
$ gst-launch-1.0 playbin video-sink=rkximagesink uri=file:///home/linaro/vga.ts
$ gst-launch-1.0 uridecodebin uri=file:///home/linaro/vga.ts ! rkximagesink

#rtsp
$ gst-launch-1.0 rtspsrc location=rtsp://140.114.188.219/axis-media/media.amp latency=1200 ! rtph264depay ! h264parse ! mppvideodec ! rkximagesink sync=false

#usb camera
$ gst-launch-1.0 v4l2src device=/dev/video4 ! mppvideodec ! rkximagesink

MJPG-streamer

# Switch camera form IMX219 to OV5647 (Optional)
$ sudo tinker-config (Interfacing Options -> Camera -> OV5647)
$ sudo reboot

# Set isp format for OV5647 (Optional)
$ sudo apt-get install -y v4l-utils
$ media-ctl -d /dev/media0 --set-v4l2 '"rkisp1-isp-subdev":0[fmt:SBGGR8/1296x972]'
$ media-ctl -d /dev/media0 --set-v4l2 '"rkisp1-isp-subdev":0[fmt:SBGGR8/1296x972]' --set-v4l2 '"rkisp1-isp-subdev":0[crop:(0,0)/1296x972]'
$ media-ctl -d /dev/media0 --set-v4l2 '"rkisp1-isp-subdev":2[fmt:YUYV2X8/1296x972]'
$ media-ctl -d /dev/media0 --set-v4l2 '"rkisps-isp-subdev":2[fmt:YUYV2X8/1296x972]' --set-v4l2 '"rkisp1-isp-subdev":2[crop:(0,0)/1296x972]'
#Install
# Install for mjpg-streamer
$ sudo apt-get install libjpeg62-turbo-dev
$ git clone https://github.com/TinkerBoard/mjpg-streamer.git
$ cd mjpg-streamer/mjpg-streamer-experimental
$ make
$ sudo make install
#Start MJPG-streamer
$ cd ~/code/mjpg-streamer
$ ./mjpg_streamer -i "./input_uvc.so -y -d /dev/video0" -o "./output_http.so -w ./www"

You will see camera preview on your browser http://IP_ADDRESS:8080/?action=stream.
Note: IP_ADDRESS is 127.0.0.1 on Tinker Board browser or use wlan0 address (sudo ifconfig to check) on remote PC browser

OpenCV in python (Face Detection)

#Install
$ sudo apt-get update
$ sudo apt-get upgrade
#Install a few developer tools
$ sudo apt-get install -y build-essential git cmake pkg-config
#Install image I/O packages which allow us to load image file formats such as JPEG, PNG, TIFF, etc.
$ sudo apt-get install -y libjpeg-dev libtiff5-dev libpng-dev
#Install video I/O packages
$ sudo apt-get install -y libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libxvidcore-dev libx264-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
#Install the GTK development library
$ sudo apt-get install -y libgtk2.0-dev
#Various operations inside of OpenCV (such as matrix operations) can be optimized using added dependencies
$ sudo apt-get install -y libatlas-base-dev gfortran
#Install the Python 2.7 and Python 3 header files 
$ sudo apt-get install -y python2.7-dev python3-dev python-opencv


$ wget https://github.com/opencv/opencv/archive/3.3.0.zip
$ unzip 3.3.0.zip
$ cd opencv-3.3.0
$ mkdir build
$ cd build
$ cmake -D CMAKE_BUILD_TYPE=Release -D WITH_LIBV4L=ON -D CMAKE_INSTALL_PREFIX=/usr/local ..
$ sudo make install

[Important Note] Please use the adapter 5V 3A.

sample code in Python

import sys, os, cv2

print(cv2.__version__)
cv2.setNumThreads(4)

#please copy ~/opencv-3.3.0/data/harrcascades_cuda/haarcascade_frontalface_alt.xml to /home/linaro/
faceCascade = cv2.CascadeClassifier("/usr/local/share/OpenCV/haarcascades//haarcascade_frontalface_alt.xml")
cap = cv2.VideoCapture("rkcamsrc io-mode=4 ! video/x-raw,format=NV12,width=640,height=480 ! videoconvert ! appsink")

while cap.isOpened():
    ret, frame = cap.read()
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = faceCascade.detectMultiScale(
        gray,
        scaleFactor=1.1,
        minNeighbors=5,
        minSize=(30, 30),
        flags=cv2.CASCADE_SCALE_IMAGE
    )
    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
    cv2.imshow("Frame", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

NOTE: If it has a error as like Error in `python’: free(): invalid pointer, please try below command

$ sudo apt-get install libtcmalloc-minimal4
$ ldconfig or export LD_PRELOAD=”/usr/lib/libtcmalloc_minimal.so.4”

Sample code in c++
Sample code - test.c

#include <opencv2/highgui.hpp>
#include <iostream>

using namespace cv;
using namespace std;


const String keys
    = "{ help           | false | print usage         }"
      "{ camera_width   | 640   | camera device width  }"
      "{ camera_height  | 480   | camera device height }"
;

int main(int argc, char** argv)
{
    CommandLineParser parser(argc, argv, keys);


    if (parser.get<bool>("help"))
    {
        parser.printMessage();
        return 0;
    }

    VideoCapture cap;

    cap = VideoCapture("v4l2src ! video/x-raw,format=NV12,width=640,height=480 ! videoconvert ! appsink");
    if(!cap.isOpened())
    {
        cout << "Couldn't find camera: " << cameraDevice << endl;
        return -1;
    }

    cap.set(CAP_PROP_FRAME_WIDTH, parser.get<int>("camera_width"));
    cap.set(CAP_PROP_FRAME_HEIGHT, parser.get<int>("camera_height"));

    for(;;)
    {
        Mat frame;
        cap >> frame; // get a new frame from camera/video or read image

        if (frame.empty())
        {
            waitKey();
            break;
        }

        imshow("image", frame);
        if (waitKey(1) >= 0) break;
    }

    return 0;
}

Build test.c

$ gcc test.c `pkg-config --cflags --libs opencv`

Gstreamer in C code
Sample code - test.c (camera preview)

#include <gst/gst.h>

int main(int argc, char *argv[]) {
    GstElement *pipeline;
    GstBus *bus;
    GstMessage *msg;
    
    /* Initialize GStreamer */
    gst_init (&argc, &argv);
    
    /* Build the pipeline */
    pipeline = gst_parse_launch ("rkcamsrc num-buffers=512 io-mode=4 ! video/x-raw,format=NV12,width=640,height=480 ! rkximagesink", NULL);
    
    /* Start playing */
    gst_element_set_state (pipeline, GST_STATE_PLAYING);
    
    /* Wait until error or EOS */
    bus = gst_element_get_bus (pipeline);
    msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);

    /* Free resources */
    if (msg != NULL)
        gst_message_unref (msg);
    gst_object_unref (bus);
    gst_element_set_state (pipeline, GST_STATE_NULL);
    gst_object_unref (pipeline);
    return 0;
}

Build test.c

$ sudo apt-get install libgstreamer1.0-dev
$ gcc test.c -o test1 `pkg-config --cflags --libs gstreamer-1.0`