Add Raspberry Pi 4 and 5 V4L2 Stateless HEVC Hardware Acceleration with FFmpeg #27453
This PR enables V4L2 stateless HEVC hardware acceleration for Raspberry Pi 5 within OpenCV's videoio module. It leverages FFmpeg's drm acceleration ([FFmpeg API changes](ee1f79b0fa/doc/APIchanges (L1529))), significantly improving HEVC decoding performance on RPi5 for robotics and embedded vision applications.
I have a working proof-of-concept with local benchmarks showing clear gains.
Checklist Status:
Ready: License, branch (4.x), FFmpeg reference, and (linked) related issue (#27452).
Seeking Guidance: Need help with formal C++ performance/accuracy tests, opencv_extra integration, and full documentation/examples.
As a Python developer, I welcome C++ best practice feedback and assistance with testing setup.
### Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
Patch to opencv_extra has the same branch name.
- [ ] The feature is well documented and sample code can be built with the project CMake
videoio: HW decode/encode in FFMPEG backend; new properties with support in FFMPEG/GST/MSMF
* HW acceleration in FFMPEG backend
* fixes on Windows, remove D3D9
* HW acceleration in FFMPEG backend
* fixes on Windows, remove D3D9
* improve va test
* Copyright
* check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+
* CAP_MSMF test on .mp4
* .mp4 in test
* improve va test
* Copyright
* check LIBAVUTIL_BUILD >= AV_VERSION_INT(55, 78, 100) // FFMPEG 3.4+
* CAP_MSMF test on .mp4
* .mp4 in test
* .avi for GStreamer test
* revert changes around seek()
* cv_writer_open_with_params
* params.warnUnusedParameters
* VideoCaptureParameters in GStreamer
* open_with_params
* params->getUnused
* Reduce PSNR threshold 33->32 (other tests use 30)
* require FFMPEG 4.0+; PSNR 30 as in other tests
* GStreamer AVI-demux plugin not installed in Ubuntu test environment?
* fix build on very old ffmpeg
* fix build on very old ffmpeg
* fix build issues
* fix build issues (static_cast)
* FFMPEG built on Windows without H264 encoder?
* fix for write_nothing test on VAAPI
* fix warnings
* fix cv_writer_get_prop in plugins
* use avcodec_get_hw_frames_parameters; more robust fallback to SW codecs
* internal function hw_check_device() for device check/logging
* two separate tests for HW read and write
* image size 640x480 in encode test
* WITH_VA=ON (only .h headers used in OpenCV, no linkage dependency)
* exception on VP9 SW encoder?
* rebase master; refine info message
* videoio: fix FFmpeg standalone plugin build
* videoio(ffmpeg): eliminate MSVC build warnings
* address review comments
* videoio(hw): update videocapture_acceleration.read test
- remove parallel decoding by SW code path
- check PSNR against the original generated image
* videoio: minor fixes
* videoio(test): disable unsupported MSMF cases (SW and HW)
* videoio(test): update PSNR thresholds for HW acceleration read
* videoio(test): update debug messages
* "hw_acceleration" whitelisting parameter
* little optimization in test
* D3D11VA supports decoders, doesn't support encoders
* videoio(test): adjust PSNR threshold in write_read_position tests
* videoio(ffmpeg): fix rejecting on acceleration device name mismatch
* videoio(ffmpeg): fix compilation USE_AV_HW_CODECS=0, add more debug logging
* videoio: rework VideoAccelerationType behavior
- enum is not a bitset
- default value is backend specific
- only '_NONE' and '_ANY' may fallback on software processing
- specific H/W acceleration doesn't fallback on software processing. It fails if there is no support for specified H/W acceleration.
* videoio(test): fix for current FFmpeg wrapper
Co-authored-by: Alexander Alekhin <alexander.a.alekhin@gmail.com>
* moved DIS optical flow from opencv_contrib to opencv, moved TVL1 from opencv to opencv_contrib
* fixed compile warning
* TVL1 optical flow example moved to opencv_contrib
- allow installing samples sources on all platforms
even if BUILD_EXAMPLES is disabled, fixed minor
issues in sources installation process
- use 'example_<group>_<name>' scheme for target and binary file naming
- use single function for sample executable creation
The class `App` appears to have two unused methods: `message` and `checkRectSimilarity`. The is no definition or use of either of these methods. This appears to be dead code.
All of these: (performance) Prefer prefix ++/-- operators for non-primitive types.
[modules/calib3d/src/fundam.cpp:1049] -> [modules/calib3d/src/fundam.cpp:1049]: (style) Same expression on both sides of '&&'.
For current OpenCV-CL architecture, if the data buffer
allocated in UMat are cpu buffer(not ocl buffer) under
cpu mode, and then pass this UMat to an OpenCL kernel
as an argument, the OpenCL path will fail and fallback
to cpu mode. Take HOGDescriptor::oclSvmDetector as an example:
ocl::setUseOpenCL(false);
//data allocated in hog.oclSvmDetector will be cpu buffer
hog.setSVMDetector(HOGDescriptor::getDaimlerPeopleDetector());
ocl::setUseOpenCL(true);
//We enabled OpenCL, but hog.oclSvmDetector are cpu buffer,
//so it will fail in the function ocl_classify_hists
//when reach to this line
//idx = k.set(idx, ocl::KernelArg::PtrReadOnly(detector));
hog.detectMultiScale(img, found, hit_threshold, win_stride,
Size(0, 0), scale, gr_threshold);
Similar problems heppen on img_aux and img. So we should re-define
or re-set these UMat when do mode switch (CPU -> OpenCL) in order
to make their data be allocated by ocl and then OpenCL path will
succeed.
Signed-off-by: Chuanbo Weng <chuanbo.weng@intel.com>
