Difference between revisions of "Linbian Usage Guide"

From Lindenis Wiki
Jump to: navigation, search
(Camera Module)
(Dual Camera view)
(39 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
[[zh:Linbian使用指南]]
 
[[zh:Linbian使用指南]]
 
== Terminal ==
 
== Terminal ==
 +
 +
On Linbian OS, the default terminal application is LXTerminal.
 +
 +
[[File:Lxterminal.png|frameless]]
 +
 +
You can open it from '''Application Menu''' -> '''System Tools'''.
 +
 +
== Camera Module ==
 +
===download source code from github===
 +
*ldcam developed base on gstreamer
 +
:git clone https://github.com/lindenis-org/ldcam-gst.git
 +
===compile===
 +
*compile on v5 board===
 +
::you can execute "make" directly
 +
*compile on your own env
 +
::you should create the Makefile by yourself
 +
===user commd===
 +
:you can use "./ldcam -?" or "./ldcam --help" to get the cmd instructions
 +
 +
== Webcams ==
 +
 +
== Gstreamer ==
 +
=== Camera source ===
 +
*'''lindeniv4l2src: Video (video4linux2) Source'''<br>
 +
Use for CSI Camera<br>
 +
Example:<br>
 +
sudo gst-launch-1.0 lindeniv4l2src device=/dev/video1 ! video/x-raw,width=1920,height=1080,format=I420 ! videoconvert ! sunxifbsink video-memory=24
 +
 +
=== Omx hardware H.264 decoder and encoder ===
 +
*'''omxh264dec: OpenMAX H.264 Video Decoder'''<br>
 +
Example:<br>
 +
sudo gst-launch-1.0 filesrc location=1c.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! videoconvert ! sunxifbsink
 +
 +
*'''omxh264videoenc: OpenMAX H.264 Video Encoder'''<br>
 +
Example:<br>
 +
sudo gst-launch-1.0 lindeniv4l2src device=/dev/video1 ! video/x-raw,width=1920,height=1080,format=I420 ! videoconvert ! omxh264videoenc target-bitrate=5000000 ! mpegtsmux ! filesink location=1.ts
 +
 +
=== Display sink ===
 +
*'''sunxifbsink: Accelerated console framebuffer video sink for sunxi-based devices'''<br>
 +
Example:<br>
 +
sudo gst-launch-1.0 -v videotestsrc pattern=ball name=left ! sunxifbsink
 +
 +
== Play Video ==
 +
 +
== Play Audio ==
 +
 +
== Python ==
 +
 +
== GPIO ==
  
 
== MPP ==
 
== MPP ==
  
 +
MPP is the media process platform of Allwinner V5, including following components.
 +
* system control, API family: AW_MPI_SYS
 +
: This component deal with the media hardware initialization, other components initialization and component state management and etc.
 +
* video input, API family: AW_MPI_VI
 +
* video output, API family: AW_MPI_VO
 +
* image stitch engine, API family: AW_MPI_ISE
 +
* video encode, API family: AW_MPI_VENC
 +
* video decode, API family: AW_MPI_VDEC
 +
* mux, API family: AW_MPI_MUX
 +
* demux, API family: AW_MPI_DEMUX
 +
* audio input, API family: AW_MPI_AI
 +
* audio output, API family: AW_MPI_AO
 +
* audio endode, API family: AW_MPI_AENC
 +
* audio output, API family: AW_MPI_AOUT
 +
* Region, API family: AW_MPI_RGN
 +
 
=== Allwinner official documents ===
 
=== Allwinner official documents ===
 
* MPP Application Development Reference Manual, [http://files.lindeni.org/lindenis-v5/documents/soft_design/MPP/AW%20IPC%e5%aa%92%e4%bd%93%e5%a4%84%e7%90%86%e8%bd%af%e4%bb%b6%e5%bc%80%e5%8f%91%e5%8f%82%e8%80%83.pdf Chinese version].
 
* MPP Application Development Reference Manual, [http://files.lindeni.org/lindenis-v5/documents/soft_design/MPP/AW%20IPC%e5%aa%92%e4%bd%93%e5%a4%84%e7%90%86%e8%bd%af%e4%bb%b6%e5%bc%80%e5%8f%91%e5%8f%82%e8%80%83.pdf Chinese version].
Line 16: Line 81:
 
Please refer to [[#Demo_programs | Demo programs]].
 
Please refer to [[#Demo_programs | Demo programs]].
  
== Camera Module ==
+
== Demo programs ==
 +
 
 +
Allwinner release a series of MPP samples. You can download from [https://github.com/lindenis-org/mpp_sample Github].
 +
<pre>
 +
$ git clone https://github.com/lindenis-org/mpp_sample ~/mpp_sample
 +
</pre>
 +
 
 +
After downloading, You will get the following directory.
 +
<pre>
 +
app/
 +
Makefile.param
 +
README.md
 +
sample_adec/
 +
sample_aenc/
 +
sample_ai/
 +
......
 +
</pre>
 +
'''There is a instruction name readme.txt in every sample, you should read the document first when you running the sample'''
 +
 
 +
'''And then you can know how to config the sample, some sample needed be configured,maybe config file is sample_xxx.conf.'''
 +
 
 +
'''app/''' includes a set of MPP samples with a command line based menu. '''sample_xxx/''' is a separate sample of the scene. Root privileges is required when run these sample. e.g.
 +
<pre>
 +
$ sudo ./app/release/app_mpp_sample
 +
 
 +
***************************************************************
 +
  1 : vi+(venc+rtsp)+vo
 +
  2 : vi+(venc+rtsp)
 +
  3 : vi+vo
 +
  4 : vi+ise+(venc+rtsp)+vo
 +
  5 : [vi+venc+vo]+[ai+aenc+ao]+rtsp
 +
  6 : vi->yuv+venc+rtsp
 +
  7 : vi+venc+mux
 +
  8 : vi+(venc+mux)+vo
 +
  9 : demux+vdec+vo
 +
10 : ai+aenc+mux+ao
 +
11 : demux+adec+ao
 +
12 : Quit
 +
***************************************************************
 +
Please choice 1~12 num:
 +
</pre>
 +
 
 +
=== RTSP ===
 +
 
 +
==== Location ====
 +
 
 +
app_mpp_sample
 +
 
 +
==== Building ====
 +
 
 +
<pre>
 +
$ cd ~/mpp_sample/app
 +
$ make
 +
</pre>
 +
 
 +
==== Running ====
 +
 
 +
<pre>
 +
$ sudo ./release/app_mpp_sample
 +
</pre>
 +
There will shows the following message when you execultive the cmd, and type the num according the menu.
 +
<pre>
 +
***************************************************************
 +
  1 : vi+(venc+rtsp)+vo
 +
  2 : vi+(venc+rtsp)
 +
  3 : vi+vo
 +
  4 : vi+ise+(venc+rtsp)+vo
 +
  5 : [vi+venc+vo]+[ai+aenc+ao]+rtsp
 +
  6 : vi->yuv+venc+rtsp
 +
  7 : vi+venc+mux
 +
  8 : vi+(venc+mux)+vo
 +
  9 : demux+vdec+vo
 +
10 : ai+aenc+mux+ao
 +
11 : demux+adec+ao
 +
12 : Quit
 +
***************************************************************
 +
Please choice 1~12 num: 1
 +
 
 +
***************************************************************
 +
  1 : Set VI+VENC scene    (Resolution,    default:4K@25fps+720P@25fps)
 +
  2 : Set VENC Payload Type (H264/H265,    default:H264)
 +
  3 : Set VENC RC Mode      (CBR/VBR/FIXQP, default:CBR)
 +
  4 : Set VENC Profile      (BL/MP/HP,      default:Main Profile)
 +
  5 : Set VENC rotate      (0/90/180/270,  default:rotate_0)
 +
  6 : Save confige and run this sample
 +
***************************************************************
 +
Please choice 1~6 num: 1
 +
 
 +
***************** Choice VI+VENC scene **************************
 +
[0]:VI_4K@30fps + VENC(4K@30fps+VGA@30fps)
 +
[1]:VI_4K@25fps + VENC(4K@25fps+720P@25fps) [default]
 +
[2]:VI_2K@30fps + VENC(2K@30fps+720P@30fps)
 +
[3]:VI_1080P@30fps + VENC(1080P@30fps+720P@30fps)
 +
[4]:VI_2880x2160@30fps + VENC(2880x2160@30fps+1080P@30fps)
 +
[5]:VI_2592x1944@30fps + VENC(2592x1944@30fps+1080P@30fps)
 +
Please choose VI+VENC scene ID 0~3 or (q-Quit): 1
 +
 
 +
***************************************************************
 +
  1 : Set VI+VENC scene    (Resolution,    default:4K@25fps+720P@25fps)
 +
  2 : Set VENC Payload Type (H264/H265,    default:H264)
 +
  3 : Set VENC RC Mode      (CBR/VBR/FIXQP, default:CBR)
 +
  4 : Set VENC Profile      (BL/MP/HP,      default:Main Profile)
 +
  5 : Set VENC rotate      (0/90/180/270,  default:rotate_0)
 +
  6 : Save confige and run this sample
 +
***************************************************************
 +
Please choice 1~6 num: 6
 +
</pre>
 +
 
 +
==== Test result ====
 +
[[File:Rtsp.png|frameless]]
 +
 
 +
*You can see that the two remote device can get the data at the same time
 +
 
 +
*There are two output data channel, you can get 720p from channel0 and 4K data from channel1
 +
 
 +
*Tt also can be viewed by mobile phone, a simple IPC mode.
  
'''Sys_config.fex config'''
+
=== Dual Camera ===
  
See the 13 chapter page 37 to page 52 [http://files.lindeni.org/lindenis-v5/documents/guide/sys_config.fex%E4%BD%BF%E7%94%A8%E9%85%8D%E7%BD%AE%E8%AF%B4%E6%98%8E.pdf sys_config guide]
+
==== Location ====
  
'''Kernel menuconfig'''
+
sample_virvi2ise2vo
  
Device Drivers  ---> 
+
==== Building ====
<*> Multimedia support  --->
 
            [*]  V4L platform devices  --->
 
                  [*]      imx317 platform select  --->
 
                        --- imx317 platform select
 
                        [*]  select IPC platform config otherwise SDV platform
 
                        <M>  imx317_mipi sensor driver 
 
  
the imx317 sensor driver code
+
<pre>
 +
$ cd ~/mpp_sample/sample_virvi2ise2vo
 +
$ make
 +
</pre>
  
kernel/drivers/media/platform/sunxi-vin/modules/sensor/imx317_mipi.c
+
==== Running ====
  
if you want to support a new sensor, you can add the driver code in this path, and modify the Makefile and Kconfig like imx317. Of course you should config the sys_config.fex first.
+
<pre>
 +
$ sudo ./sample_virvi2ise2vo -path sample_virvi2ise2vo.conf
 +
</pre>
  
'''Insmod the sensor module'''
+
==== Test result ====
 +
[[File:Dual camera.jpg|frameless]]
  
1.cp the driver ko to debian/target/lib/modules/4.4.55+/
+
*Two ISP can work at the same time
  
2.replace the imx317_mipi with the new sensor ko's name in this file
+
*And each one can send 4K 30fps data out when you preview the picture
  
device/eagle/rootfs/etc/modules-load.d/mpp.conf
+
*Device can support 4K 15fps encode when two ISP work at the same time
  
3.build the sdk
+
=== Face detection ===
  
'''User Interfaces'''
+
Face detection based on EVE.
  
1.Use the MPP interfaces. see the [http://files.lindeni.org/lindenis-v5/documents/soft_design/MPP/AW%20IPC%e5%aa%92%e4%bd%93%e5%a4%84%e7%90%86%e8%bd%af%e4%bb%b6%e5%bc%80%e5%8f%91%e5%8f%82%e8%80%83.pdf MPP Interface Usage] page 20-53,
+
==== Location ====
  
The demo code is here [https://github.com/lindenis-org/mpp_sample/tree/master/sample_virvi2vo sample_virvi2vo]
+
sample_face_detect
  
2.Use the /dev/videoX directly, you can see the demo in  [http://files.lindeni.org/lindenis-v5/documents/Driver/SUNXI-VIN%e6%a8%a1%e5%9d%97%e4%bd%bf%e7%94%a8%e6%96%87%e6%a1%a3.pdf Camera Driver Guide]
+
==== Building ====
  
== Webcams ==
+
<pre>
 +
$ cd ~/mpp_sample/sample_face_detect
 +
$ make
 +
</pre>
  
== Gstreamer ==
+
==== Running ====
  
== Play Video ==
+
<pre>
 +
$ sudo ./sample_face_detect
 +
</pre>
  
== Play Audio ==
+
The picture of face crop and store at '''./pic''' directory when detected.
  
== Python ==
+
==== Test result ====
 +
[[File:Face detect.jpg|frameless]]
  
== GPIO ==
+
*Support face detect which the data source minmum pixels is 20
  
== Demo programs ==
+
*Support face detection and tracing at the same time, the maximum number is 16
  
=== Face recognition ===
+
*The maximum detect face number is 128
  
 
=== Motion detection ===
 
=== Motion detection ===
 +
 +
Motion detection based on CVE.
 +
 +
==== Location ====
 +
 +
sample_AILib/sample_vi2MOD
 +
 +
==== Building ====
 +
 +
<pre>
 +
$ cd ~/mpp_sample/sample_AILib/sample_vi2MOD
 +
$ make
 +
</pre>
 +
 +
==== Running ====
 +
 +
<pre>
 +
$ sudo ./sample_vi2MOD -path sample_vi2MOD.conf
 +
</pre>
 +
 +
The information of detected target was stored at ./DTCAResult.bin .
  
 
=== Human counting ===
 
=== Human counting ===
 +
 +
Hunman counting based on CVE.
 +
 +
==== Location ====
 +
 +
sample_AILib/sample_vi2HCNT
 +
 +
==== Building ====
 +
 +
<pre>
 +
$ cd ~/mpp_sample/sample_AILib/sample_vi2HCNT
 +
$ make
 +
</pre>
 +
 +
==== Running ====
 +
 +
<pre>
 +
$ sudo ./sample_vi2HCNT -path sample_vi2HCNT.conf
 +
</pre>
 +
 +
The information of detected target was stored at ./HCNTResult.bin .
  
 
=== VLPR ===
 
=== VLPR ===
 +
 +
VLPR based on CVE.
 +
 +
==== Location ====
 +
 +
sample_AILib/sample_vi2VLPR
 +
 +
==== Building ====
 +
 +
<pre>
 +
$ cd ~/mpp_sample/sample_AILib/sample_vi2VLPR
 +
$ make
 +
</pre>
 +
 +
==== Running ====
 +
 +
<pre>
 +
$ sudo ./sample_vi2VLPR -path sample_vi2VLPR.conf
 +
</pre>
 +
 +
The information of detected vehicle license plate was stored at ./VLPRResult.bin .
 +
 +
=== Binocular depth information image ===
 +
 +
BDII based on CVE.
 +
 +
==== Location ====
 +
 +
sample_AILib/sample_BDII
 +
 +
==== Building ====
 +
 +
<pre>
 +
$ cd ~/mpp_sample/sample_AILib/sample_vi2BDII
 +
$ make
 +
</pre>
 +
 +
==== Running ====
 +
 +
<pre>
 +
$ sudo ./sample_vi2BDII -path sample_vi2BDII.conf
 +
</pre>
 +
 +
The output is a 8bit-gray-scale picture, stored at ./BDIIResult.bin .
  
 
=== Image stitch ===
 
=== Image stitch ===
  
=== Binocular DOF image ===
+
Camera module is in development.
 
 
=== Speech recognition ===
 

Revision as of 07:46, 23 January 2019

Terminal

On Linbian OS, the default terminal application is LXTerminal.

Lxterminal.png

You can open it from Application Menu -> System Tools.

Camera Module

download source code from github

  • ldcam developed base on gstreamer
git clone https://github.com/lindenis-org/ldcam-gst.git

compile

  • compile on v5 board===
you can execute "make" directly
  • compile on your own env
you should create the Makefile by yourself

user commd

you can use "./ldcam -?" or "./ldcam --help" to get the cmd instructions

Webcams

Gstreamer

Camera source

  • lindeniv4l2src: Video (video4linux2) Source

Use for CSI Camera
Example:
sudo gst-launch-1.0 lindeniv4l2src device=/dev/video1 ! video/x-raw,width=1920,height=1080,format=I420 ! videoconvert ! sunxifbsink video-memory=24

Omx hardware H.264 decoder and encoder

  • omxh264dec: OpenMAX H.264 Video Decoder

Example:
sudo gst-launch-1.0 filesrc location=1c.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! omxh264dec ! videoconvert ! sunxifbsink

  • omxh264videoenc: OpenMAX H.264 Video Encoder

Example:
sudo gst-launch-1.0 lindeniv4l2src device=/dev/video1 ! video/x-raw,width=1920,height=1080,format=I420 ! videoconvert ! omxh264videoenc target-bitrate=5000000 ! mpegtsmux ! filesink location=1.ts

Display sink

  • sunxifbsink: Accelerated console framebuffer video sink for sunxi-based devices

Example:
sudo gst-launch-1.0 -v videotestsrc pattern=ball name=left ! sunxifbsink

Play Video

Play Audio

Python

GPIO

MPP

MPP is the media process platform of Allwinner V5, including following components.

  • system control, API family: AW_MPI_SYS
This component deal with the media hardware initialization, other components initialization and component state management and etc.
  • video input, API family: AW_MPI_VI
  • video output, API family: AW_MPI_VO
  • image stitch engine, API family: AW_MPI_ISE
  • video encode, API family: AW_MPI_VENC
  • video decode, API family: AW_MPI_VDEC
  • mux, API family: AW_MPI_MUX
  • demux, API family: AW_MPI_DEMUX
  • audio input, API family: AW_MPI_AI
  • audio output, API family: AW_MPI_AO
  • audio endode, API family: AW_MPI_AENC
  • audio output, API family: AW_MPI_AOUT
  • Region, API family: AW_MPI_RGN

Allwinner official documents

Samples

Please refer to Demo programs.

Demo programs

Allwinner release a series of MPP samples. You can download from Github.

$ git clone https://github.com/lindenis-org/mpp_sample ~/mpp_sample

After downloading, You will get the following directory.

app/
Makefile.param
README.md
sample_adec/
sample_aenc/
sample_ai/
......

There is a instruction name readme.txt in every sample, you should read the document first when you running the sample

And then you can know how to config the sample, some sample needed be configured,maybe config file is sample_xxx.conf.

app/ includes a set of MPP samples with a command line based menu. sample_xxx/ is a separate sample of the scene. Root privileges is required when run these sample. e.g.

$ sudo ./app/release/app_mpp_sample

***************************************************************
  1 : vi+(venc+rtsp)+vo
  2 : vi+(venc+rtsp)
  3 : vi+vo
  4 : vi+ise+(venc+rtsp)+vo
  5 : [vi+venc+vo]+[ai+aenc+ao]+rtsp
  6 : vi->yuv+venc+rtsp
  7 : vi+venc+mux
  8 : vi+(venc+mux)+vo
  9 : demux+vdec+vo
 10 : ai+aenc+mux+ao
 11 : demux+adec+ao
 12 : Quit
***************************************************************
 Please choice 1~12 num:

RTSP

Location

app_mpp_sample

Building

$ cd ~/mpp_sample/app
$ make

Running

$ sudo ./release/app_mpp_sample

There will shows the following message when you execultive the cmd, and type the num according the menu.

***************************************************************
  1 : vi+(venc+rtsp)+vo
  2 : vi+(venc+rtsp)
  3 : vi+vo
  4 : vi+ise+(venc+rtsp)+vo
  5 : [vi+venc+vo]+[ai+aenc+ao]+rtsp
  6 : vi->yuv+venc+rtsp
  7 : vi+venc+mux
  8 : vi+(venc+mux)+vo
  9 : demux+vdec+vo
 10 : ai+aenc+mux+ao
 11 : demux+adec+ao
 12 : Quit
***************************************************************
 Please choice 1~12 num: 1

***************************************************************
  1 : Set VI+VENC scene     (Resolution,    default:4K@25fps+720P@25fps)
  2 : Set VENC Payload Type (H264/H265,     default:H264)
  3 : Set VENC RC Mode      (CBR/VBR/FIXQP, default:CBR)
  4 : Set VENC Profile      (BL/MP/HP,      default:Main Profile)
  5 : Set VENC rotate       (0/90/180/270,  default:rotate_0)
  6 : Save confige and run this sample
***************************************************************
 Please choice 1~6 num: 1

 ***************** Choice VI+VENC scene **************************
 [0]:VI_4K@30fps + VENC(4K@30fps+VGA@30fps)
 [1]:VI_4K@25fps + VENC(4K@25fps+720P@25fps) [default]
 [2]:VI_2K@30fps + VENC(2K@30fps+720P@30fps)
 [3]:VI_1080P@30fps + VENC(1080P@30fps+720P@30fps)
 [4]:VI_2880x2160@30fps + VENC(2880x2160@30fps+1080P@30fps)
 [5]:VI_2592x1944@30fps + VENC(2592x1944@30fps+1080P@30fps)
 Please choose VI+VENC scene ID 0~3 or (q-Quit): 1

 ***************************************************************
  1 : Set VI+VENC scene     (Resolution,    default:4K@25fps+720P@25fps)
  2 : Set VENC Payload Type (H264/H265,     default:H264)
  3 : Set VENC RC Mode      (CBR/VBR/FIXQP, default:CBR)
  4 : Set VENC Profile      (BL/MP/HP,      default:Main Profile)
  5 : Set VENC rotate       (0/90/180/270,  default:rotate_0)
  6 : Save confige and run this sample
***************************************************************
 Please choice 1~6 num: 6

Test result

Rtsp.png

  • You can see that the two remote device can get the data at the same time
  • There are two output data channel, you can get 720p from channel0 and 4K data from channel1
  • Tt also can be viewed by mobile phone, a simple IPC mode.

Dual Camera

Location

sample_virvi2ise2vo

Building

$ cd ~/mpp_sample/sample_virvi2ise2vo
$ make

Running

$ sudo ./sample_virvi2ise2vo -path sample_virvi2ise2vo.conf

Test result

Dual camera.jpg

  • Two ISP can work at the same time
  • And each one can send 4K 30fps data out when you preview the picture
  • Device can support 4K 15fps encode when two ISP work at the same time

Face detection

Face detection based on EVE.

Location

sample_face_detect

Building

$ cd ~/mpp_sample/sample_face_detect
$ make

Running

$ sudo ./sample_face_detect

The picture of face crop and store at ./pic directory when detected.

Test result

Face detect.jpg

  • Support face detect which the data source minmum pixels is 20
  • Support face detection and tracing at the same time, the maximum number is 16
  • The maximum detect face number is 128

Motion detection

Motion detection based on CVE.

Location

sample_AILib/sample_vi2MOD

Building

$ cd ~/mpp_sample/sample_AILib/sample_vi2MOD
$ make 

Running

$ sudo ./sample_vi2MOD -path sample_vi2MOD.conf

The information of detected target was stored at ./DTCAResult.bin .

Human counting

Hunman counting based on CVE.

Location

sample_AILib/sample_vi2HCNT

Building

$ cd ~/mpp_sample/sample_AILib/sample_vi2HCNT
$ make

Running

$ sudo ./sample_vi2HCNT -path sample_vi2HCNT.conf

The information of detected target was stored at ./HCNTResult.bin .

VLPR

VLPR based on CVE.

Location

sample_AILib/sample_vi2VLPR

Building

$ cd ~/mpp_sample/sample_AILib/sample_vi2VLPR
$ make

Running

$ sudo ./sample_vi2VLPR -path sample_vi2VLPR.conf

The information of detected vehicle license plate was stored at ./VLPRResult.bin .

Binocular depth information image

BDII based on CVE.

Location

sample_AILib/sample_BDII

Building

$ cd ~/mpp_sample/sample_AILib/sample_vi2BDII
$ make

Running

$ sudo ./sample_vi2BDII -path sample_vi2BDII.conf

The output is a 8bit-gray-scale picture, stored at ./BDIIResult.bin .

Image stitch

Camera module is in development.