当前位置：网站首页>RIoTBoard开发板系列笔记（四）—— 使用VPU硬件解码
RIoTBoard开发板系列笔记（四）—— 使用VPU硬件解码
2022-07-20 05:33:00 【倾我一生来读】
视频解码分为软解和硬解两种，简单的来说纯粹依靠CPU来解码的称为软解，而使用GPU/VPU解码的称为硬解。常规地直接用CPU解码会极大地消耗CPU的运算能力，硬件解码是通过显卡的视频加速功能对高清视频进行解码,能够将CPU从繁重的视频解码运算中释放出来。
RIoTBoard使用的芯片是i.mx6 solo，芯片中包含一个VPU （video processing unit），可以进行硬件编解码加速，心血来潮，折腾一下视频硬件解码加速。
NXP官方提供了一个libimxvpuapi lib封装了vpu的使用方法，libimxvpuapi 又依赖libimxdmabuffer，所以只要编译出libimxvpuapi和libimxdmabuffer，运行libimxvpuapi 的testcase 就可以实现硬件编解码。
1 交叉编译libimxdmabuffer
libimxdmabuffer代码下载地址：
git clone https://github.com/Freescale/libimxdmabuffer.git
git 仓库里的readme有简单的说明，支持那些平台、如何编译等等，可以进行参考。
编译前需要配置toolchain以及imx-linux-headers-path，imx-linux-headers-path指向BSP中的linux-imx/include/即可。
./waf configure CC=arm-none-linux-gnueabi-gcc \
--imx-linux-headers-path=/home/zhy/code/mypc/linux/linux-imx/include/ \
--prefix=/home/zhy/code/rIoTboard/libimxdmabuffer
配置的过程中出现了linux/mxcfb.h not found的error:
Setting top to : /home/zhy/code/mypc/gstreamer/third_lib/libimxdmabuffer
Setting out to : /home/zhy/code/mypc/gstreamer/third_lib/libimxdmabuffer/build
Checking for 'gcc' (C compiler) : arm-none-linux-gnueabi-gcc
Checking for compiler switch -O2 : yes
Checking for compiler switch -DPIC : yes
Checking for compiler switch -fPIC : yes
Checking for compiler switch -pedantic : yes
Checking for compiler switch -std=gnu99 : yes
Checking for compiler switch -Wall : yes
Checking for compiler switch -Wextra : yes
Checking for header linux/mxcfb.h : not found
Could not find linux/mxcfb.h in path "/home/zhy/code/mypc/linux/linux-imx/include/" specified by --imx-linux-headers-path
查看log发现是这个头文件中uint未定义。
../../../../../../linux/linux-imx/include/linux/mxcfb.h:113:2: error: unknown type name 'uint'
解决方法把uint更改unsigned int 即可。
unsigned int flags;
重新配置，这次就可以配置成功，会输出如下log：
checking for linux/fb.h and the IPU header linux/ipu.h : yes
Checking for library g2d : not found
Checking for header g2d.h : not found
G2D not found (library found: 0 header found: 0); disabling G2D allocator
checking for linux/pxp_device.h : no
linux/pxp_device.h was not found in i.MX linux headers path; disabling PxP allocator
libimxdmabuffer version 1.0.1
'configure' finished successfully (0.662s)
配置好后执行编译命令进行编译：
./waf
编译成功后显示如下log：
[5/8] Compiling libimxdmabuffer.pc.in
[6/8] Linking build/libimxdmabuffer.so
[7/8] Linking build/test-alloc
[8/8] Symlinking build/libimxdmabuffer.so
Waf: Leaving directory `/home/zhy/code/mypc/gstreamer/third_lib/libimxdmabuffer/build' 'build' finished successfully (0.422s)
执行install命令，install目录为prefix指定的路径：
./waf install
测试一下能不能链接到limxdmabuffer这个lib:
arm-none-linux-gnueabi-gcc -L/home/zhy/code/rIoTboard/libimxdmabuffer/lib -limxdmabuffer
输出一下log就代表可以链接到，编译OK。
/usr/local/complie_tool/fsl-linaro-toolchain/bin/../arm-fsl-linux-gnueabi/multi-libs/default/usr/lib/crt1.o: In function `_start': init.c:(.text+0x34): undefined reference to `main'
collect2: ld returned 1 exit status
最后将编译好的lib 打包一下，方便以后使用，这一步不是必须的。
tar czvf libimxdmabuffer.tar.gz libimxdmabuffer/
2 编译libimxvpuapi
编译好libimxdmabuffer就可以编译libimxvpuapi，首先下载代码下载：
git clone https://github.com/Freescale/libimxvpuapi.git
libimxvpuapi编译依赖libimxdmabuffer和libvpu这两个lib ,libimxdmabuffer已经编译好了，在libimxvpuapi目录新建一个third_lib文件夹，把libimxdmabuffer编译好的so和头文件放进取。至于libvpu的头文件和so,可以到开发板的里面去copy，需要copy ipu.h、vpu_io.h、vpu_lib.h这三个头文件以及libvpu.so、libvpu.so.4这两个so，同样把这写放到third_lib 下面。最后在third_lib/lib/pkgconfig目录下放上libimxdmabuffer.pc和 vpu_lib.pc这两个pkg-config文件，最终third_lib下面的目录结构：
├── include
│ ├── imxdmabuffer
│ │ ├── imxdmabuffer_config.h
│ │ ├── imxdmabuffer.h
│ │ ├── imxdmabuffer_ipu_allocator.h
│ │ └── imxdmabuffer_physaddr.h
│ ├── ipu.h
│ ├── vpu_io.h
│ └── vpu_lib.h
└── lib
├── libimxdmabuffer.so -> libimxdmabuffer.so.1.0.1
├── libimxdmabuffer.so.1 -> libimxdmabuffer.so.1.0.1
├── libimxdmabuffer.so.1.0.1
├── libvpu.a
├── libvpu.so
├── libvpu.so.4
└── pkgconfig
├── libimxdmabuffer.pc
└── vpu_lib.pc
其中libimxdmabuffer.pc中内容，这里面的路径根据自己的目录可以进行修改：
prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
exec_prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
libdir=${prefix}/lib
includedir=${prefix}/include
libimxdmabuffer_ion=0
Name: libimxdmabuffer
Description: library for allocating and managing physically contiguous memory ("DMA memory" or "DMA buffers") on i.MX devices
Version: 1.0.1
Libs: -L${libdir} -limxdmabuffer
Cflags: -I${includedir}
vpu_lib.pc中内容：
prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
exec_prefix=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib
libdir=${prefix}/lib
includedir=${prefix}/include
Name: vpu
Description: libvpu
Version: 1.0.1
Libs: -L${libdir} -lvpu
Cflags: -I${includedir}
准备工作完成后进行编译配置：
./waf configure CC=arm-none-linux-gnueabi-gcc \
--sysroot-path=/home/zhy/code/mypc/linux/linux-imx/include/ \
--prefix=/home/zhy/code/rIoTboard/libimxvpuapi \
--imx-platform=imx6
--imx-headers=/home/zhy/code/mypc/linux/linux-imx/include/
配置的时候显示找不到libimxdmabuffer 这个lib:
Checking if this combination works : yes
Checking for program 'pkg-config' : /usr/bin/pkg-config
Checking for 'libimxdmabuffer >= 0.9.0' : not found
The configuration failed
(complete log in /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build/config.log)
查看log 发现是没有找到libimxdmabuffer的pkg-config文件，
----------------------------------------
Checking for program 'pkg-config'
/usr/bin/pkg-config
find program=['pkg-config'] paths=['/usr/local/sbin', '/usr/local/bin', '/usr/sbin', '/usr/bin', '/sbin', '/bin', '/usr/games', '/usr/local/games', '/snap/bin', '/usr/local/complie_tool/fsl-linaro-toolchain/bin'] var='PKGCONFIG' -> ['/usr/bin/pkg-config']
----------------------------------------
Checking for 'libimxdmabuffer >= 0.9.0'
['/usr/bin/pkg-config', '--cflags', '--libs', 'libimxdmabuffer', '>=', '0.9.0']
err: Package libimxdmabuffer was not found in the pkg-config search path.
Perhaps you should add the directory containing `libimxdmabuffer.pc' to the PKG_CONFIG_PATH environment variable No package 'libimxdmabuffer' found
通过PKG_CONFIG_PATH关键字增加pkg-config文件的路径即可。
./waf configure CC=arm-none-linux-gnueabi-gcc \
PKG_CONFIG_PATH=/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/third_lib/lib/pkgconfig \
--imx-platform=imx6 \
--sysroot-path=/home/zhy/code/mypc/linux/linux-imx/include/ \
--prefix=/home/zhy/code/rIoTboard/libimxvpuapi \
--imx-headers=/home/zhy/code/mypc/linux/linux-imx/include/
再次配置，有以下log印出则代表配置成功：
Setting top to : /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi
Setting out to : /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build
Checking for 'gcc' (C compiler) : arm-none-linux-gnueabi-gcc
Checking for compiler switch -Wextra : yes
Checking for compiler switch -Wall : yes
Checking for compiler switch -pedantic : yes
Checking for compiler switch -fPIC : yes
Checking for compiler switch -DPIC : yes
Checking for compiler switch -O2 : yes
Checking for compiler switch -std=gnu99 : yes
Now checking the combination of CFLAGS -Wextra -Wall -pedantic -fPIC -DPIC -O2 -std=gnu99
Checking if this combination works : yes
Checking for compiler switch -std=c99 : yes
Now checking the combination of CFLAGS -Wextra -Wall -pedantic -fPIC -DPIC -O2 -std=c99
Checking if this combination works : yes
Checking for program 'pkg-config' : /usr/bin/pkg-config
Checking for 'libimxdmabuffer >= 0.9.0' : yes
Using "/home/zhy/code/mypc/linux/linux-imx/include" as sysroot path
Checking for library vpu : yes
checking if ENC_ENABLE_SOF_STUFF exists : no
i.MX linux headers path: /home/zhy/code/mypc/linux/linux-imx/include/
checking for the IPU header linux/ipu.h : yes
libimxvpuapi version 2.2.0
'configure' finished successfully (0.692s)
执行以下命令进行编译
./waf
编译时出现了’IPU_PIX_FMT_NV16’ undeclared的error。
../imxvpuapi2/imxvpuapi2_imx6_coda_ipu.c: In function 'get_ipu_pixel_format':
../imxvpuapi2/imxvpuapi2_imx6_coda_ipu.c:33:76: error: 'IPU_PIX_FMT_NV16' undeclared (first use in this function)
ipu.h里面没有定义这个format，目前不需要NV12这个color format，在get_ipu_pixel_format函数中注释掉IPU_PIX_FMT_NV16 这个color_format即可。
static uint32_t get_ipu_pixel_format(ImxVpuApiColorFormat color_format)
{
switch (color_format)
{
case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV420_8BIT: return IPU_PIX_FMT_YUV420P;
case IMX_VPU_API_COLOR_FORMAT_SEMI_PLANAR_YUV420_8BIT: return IPU_PIX_FMT_NV12;
case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV411_8BIT: return IPU_PIX_FMT_YUV410P;
case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV422_HORIZONTAL_8BIT: return IPU_PIX_FMT_YUV422P;
//case IMX_VPU_API_COLOR_FORMAT_SEMI_PLANAR_YUV422_HORIZONTAL_8BIT: return IPU_PIX_FMT_NV16;
case IMX_VPU_API_COLOR_FORMAT_FULLY_PLANAR_YUV444_8BIT: return IPU_PIX_FMT_YUV444P;
case IMX_VPU_API_COLOR_FORMAT_YUV400_8BIT: return IPU_PIX_FMT_GREY;
default: return 0;
}
}
修改后编译成功，会有以下log印出，并在build 目录会生成libimxvpuapi2.so。
[14/19] Linking build/libimxvpuapi2.so
[15/19] Linking build/example/decode-example
[16/19] Linking build/example/encode-example
[17/19] Linking build/example/jpeg-dec-example
[18/19] Linking build/example/jpeg-enc-example
[19/19] Symlinking build/libimxvpuapi2.so
Waf: Leaving directory `/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build' 'build' finished successfully (1.970s)
测试一下能否动态链接到libimxvpuapi：
arm-none-linux-gnueabi-gcc -L/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build -limxvpuapi2
有以下log印出则代表链接OK:
[email protected]:~/code/mypc/gstreamer/third_lib/libimxvpuapi/build$ arm-none-linux-gnueabi-gcc -L/home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build -limxvpuapi2
/usr/local/complie_tool/fsl-linaro-toolchain/bin/../lib/gcc/arm-fsl-linux-gnueabi/4.6.2/../../../../arm-fsl-linux-gnueabi/bin/ld: warning: libimxdmabuffer.so.1, needed by /home/zhy/code/mypc/gstreamer/third_lib/libimxvpuapi/build/libimxvpuapi2.so, not found (try using -rpath or -rpath-link)
/usr/local/complie_tool/fsl-linaro-toolchain/bin/../arm-fsl-linux-gnueabi/multi-libs/default/usr/lib/crt1.o: In function `_start': init.c:(.text+0x34): undefined reference to `main'
将编译好的lib进行打包，以便后续使用，这一部分为非必要步骤。
mkdir libimxvpuapi
mkdir -p libimxvpuapi/lib libimxvpuapi/include/imxvpuapi2 libimxvpuapi/pkgconfig
cp -rf build/*.so.* libimxvpuapi/lib/
cp -rf imxvpuapi2/*.h libimxvpuapi/include/imxvpuapi2
cp -rf build/libimxvpuapi2.pc libimxvpuapi/lib/pkgconfig
然后在更改一下libimxvpuapi2.pc ，将libdir和includedir改成可根据prefix动态配置。
prefix=/home/zhy/code/rIoTboard/libimxvpuapi
exec_prefix=/home/zhy/code/rIoTboard/libimxvpuapi
libdir=${prefix}/lib
includedir=${prefix}/include
最后打包lib:
tar czvf libimxvpuapi.tar.gz libimxvpuapi/
3 开发板上测试使用vpu解码
libimxvpuapi编译好后在build/example 下面后生成可执行文件，将example整个文件夹复制到开放板上，同时还要复制libimxvpuapi和libimxdmabuffer编译好的so到开放板的/lib/目录下，下载一个h264码流，用mediainfo看一下信息：
把这个码流也复制到开发板上libimxvpuapi/example目录，在example执行以测试使用vpu解码h264。
./decode-example -i ./ds.h264 -o out.yuv
用手机拍了一下执行结果：
把解码后的out.yuv复制到PC 上，再用mediainfo查看，已经解码成yuv数据了，解码前190kb，解码后18.7M。
用VLC也可以播放out.yuv：
边栏推荐
- 微信H5记录
- 基于弹性云服务器的昇腾AI应用开发随笔【与云原生的故事】
- SourceTree对代码版本管理的操作流程及故障处理(不定时更新)
- STM32CubeMonitor的使用第二部分-历史数据存储以及网络访问
- 交通安全管理毕业论文范文
- About encoding (ASCII / Unicode / utf8)
- [model course of the first cann training camp advanced class in 2022] the first big assignment and additional content
- 2D目标检测综述之提议生成和特征表示篇（三）
- 城市规划设计毕业论文范文
- GD32利用CubeMX构建代码的测试
猜你喜欢
Thesis study ---- urllc benefit from noma (1)
bgy开发小示例
【CANN训练营】CANN训练营_昇腾AI趣味应用实现AI趣味应用（下）随笔
![[cann training camp] cann training camp_ Shengteng AI interesting application realizes AI interesting application (Part 1) essay](/img/72/14514a0ba23c2b0a8fea1befbce5b1.png)
[cann training camp] cann training camp_ Shengteng AI interesting application realizes AI interesting application (Part 1) essay
![[deep learning] instantiate a simple convolutional neural network using MNIST data set](/img/49/587353ccdfe943638abe0804b5871d.png)
[deep learning] instantiate a simple convolutional neural network using MNIST data set
STM32CubeMX的正交编码器encoder
基于弹性云服务器的昇腾AI应用开发随笔【与云原生的故事】
（3） Pytorch deep learning: back propagation gradient decline
理解WGAN 和 Spectral Normalization(归一化)
STM32CubeMonitor的使用第二部分-历史数据存储以及网络访问
随机推荐
Neural networks: a review of 2D target detection
（3） Pytorch deep learning: back propagation gradient decline
Pytorch实现手写数字识别 | MNIST数据集（CNN卷积神经网络）
STM32CubeMX的正交编码器encoder
Tensorflow 1.x 和 Pytorch 中 Conv2d Padding的区别
uni-app配置
vsCode配置Eslint+Prettier结合使用详细配置步骤，规范化开发
STM32CubeMonitor的使用第一部分-数据绘图以及仪表显示
The first experience of a one-stop solution for general target detection and recognition based on shengteng AI heterogeneous computing architecture cann
直观理解Transpose Convolution
[camp d'entraînement can] essai de mise en œuvre du Gan basé sur Shengsi
The difference between struct and class in swift
石油工程毕业论文范文
（6） Pytorch deep learning: loading datasets
HPDC Huawei partners and Developers Conference 2022 essay
KL散度与交叉熵概念辨析
建筑工程测量与测绘毕业论文范文
基于昇腾AI异构计算架构CANN的通用目标检测与识别一站式方案初体验
Hyperledger fabric super ledger CA construction and use
Popular understanding of the principle of deep learning gradient accumulation