From 39a7b3d1869d31df92029503b9a8670071eac596 Mon Sep 17 00:00:00 2001 From: WU Jia <35290826+kaingwade@users.noreply.github.com> Date: Mon, 1 Jul 2024 15:55:35 +0800 Subject: [PATCH] Merge pull request #25813 from kaingwade:orbbec-astra Update the tutorial of using Orbbec Astra cameras #25813 This PR is the backport of Orbbec OpenNI-based Astra camera related changes from #25410 to the 4.x branch, which includes updating the tutorial of Orbbec Astra cameras, renaming `orbbec_astra.cpp`. ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [ ] There is a reference to the original bug report and related work - [ ] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake --- doc/tutorials/app/intelperc.markdown | 2 +- doc/tutorials/app/kinect_openni.markdown | 2 +- ....markdown => orbbec_astra_openni.markdown} | 24 ++++++++++++------- .../app/table_of_content_app.markdown | 2 +- .../openni_orbbec_astra.cpp} | 0 5 files changed, 18 insertions(+), 12 deletions(-) rename doc/tutorials/app/{orbbec_astra.markdown => orbbec_astra_openni.markdown} (86%) rename samples/cpp/tutorial_code/videoio/{orbbec_astra/orbbec_astra.cpp => openni_orbbec_astra/openni_orbbec_astra.cpp} (100%) diff --git a/doc/tutorials/app/intelperc.markdown b/doc/tutorials/app/intelperc.markdown index 574bfc9e6a..315bcf3e32 100644 --- a/doc/tutorials/app/intelperc.markdown +++ b/doc/tutorials/app/intelperc.markdown @@ -3,7 +3,7 @@ Using Creative Senz3D and other Intel RealSense SDK compatible depth sensors {#t @tableofcontents -@prev_tutorial{tutorial_orbbec_astra} +@prev_tutorial{tutorial_orbbec_astra_openni} @next_tutorial{tutorial_wayland_ubuntu} ![hardwares](images/realsense.jpg) diff --git a/doc/tutorials/app/kinect_openni.markdown b/doc/tutorials/app/kinect_openni.markdown index 737b935f5b..4bae1b0182 100644 --- a/doc/tutorials/app/kinect_openni.markdown +++ b/doc/tutorials/app/kinect_openni.markdown @@ -4,7 +4,7 @@ Using Kinect and other OpenNI compatible depth sensors {#tutorial_kinect_openni} @tableofcontents @prev_tutorial{tutorial_video_write} -@next_tutorial{tutorial_orbbec_astra} +@next_tutorial{tutorial_orbbec_astra_openni} Depth sensors compatible with OpenNI (Kinect, XtionPRO, ...) are supported through VideoCapture diff --git a/doc/tutorials/app/orbbec_astra.markdown b/doc/tutorials/app/orbbec_astra_openni.markdown similarity index 86% rename from doc/tutorials/app/orbbec_astra.markdown rename to doc/tutorials/app/orbbec_astra_openni.markdown index 01a685db57..a5d11108dd 100644 --- a/doc/tutorials/app/orbbec_astra.markdown +++ b/doc/tutorials/app/orbbec_astra_openni.markdown @@ -1,4 +1,4 @@ -Using Orbbec Astra 3D cameras {#tutorial_orbbec_astra} +Using Orbbec Astra 3D cameras {#tutorial_orbbec_astra_openni} ====================================================== @tableofcontents @@ -9,7 +9,7 @@ Using Orbbec Astra 3D cameras {#tutorial_orbbec_astra} ### Introduction -This tutorial is devoted to the Astra Series of Orbbec 3D cameras (https://orbbec3d.com/index/Product/info.html?cate=38&id=36). +This tutorial is devoted to the Astra Series of Orbbec 3D cameras (https://www.orbbec.com/products/structured-light-camera/astra-series/). That cameras have a depth sensor in addition to a common color sensor. The depth sensors can be read using the open source OpenNI API with @ref cv::VideoCapture class. The video stream is provided through the regular camera interface. @@ -18,7 +18,7 @@ camera interface. In order to use the Astra camera's depth sensor with OpenCV you should do the following steps: --# Download the latest version of Orbbec OpenNI SDK (from here ). +-# Download the latest version of Orbbec OpenNI SDK (from here ). Unzip the archive, choose the build according to your operating system and follow installation steps provided in the Readme file. @@ -70,6 +70,12 @@ In order to use the Astra camera's depth sensor with OpenCV you should do the fo echo "exit" @endcode + @note The last tried version `2.3.0.86_202210111154_4c8f5aa4_beta6` does not work correctly with + modern Linux, even after libusb rebuild as recommended by the instruction. The last know good + configuration is version 2.3.0.63 (tested with Ubuntu 18.04 amd64). It's not provided officialy + with the downloading page, but published by Orbbec technical suport on Orbbec community forum + [here](https://3dclub.orbbec3d.com/t/universal-download-thread-for-astra-series-cameras/622). + -# Now you can configure OpenCV with OpenNI support enabled by setting the `WITH_OPENNI2` flag in CMake. You may also like to enable the `BUILD_EXAMPLES` flag to get a code sample working with your Astra camera. Run the following commands in the directory containing OpenCV source code to enable OpenNI support: @@ -106,7 +112,7 @@ can be read using the OpenNI interface with @ref cv::VideoCapture class. The vid not available through OpenNI API and is only provided via the regular camera interface. So, to get both depth and color frames, two @ref cv::VideoCapture objects should be created: -@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Open streams +@snippetlineno samples/cpp/tutorial_code/videoio/openni_orbbec_astra/openni_orbbec_astra.cpp Open streams The first object will use the OpenNI2 API to retrieve depth data. The second one uses the Video4Linux2 interface to access the color sensor. Note that the example above assumes that @@ -119,12 +125,12 @@ For this example, we’ll configure width and height of both streams to VGA reso the maximum resolution available for both sensors, and we’d like both stream parameters to be the same for easier color-to-depth data registration: -@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Setup streams +@snippetlineno samples/cpp/tutorial_code/videoio/openni_orbbec_astra/openni_orbbec_astra.cpp Setup streams For setting and retrieving some property of sensor data generators use @ref cv::VideoCapture::set and @ref cv::VideoCapture::get methods respectively, e.g. : -@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Get properties +@snippetlineno samples/cpp/tutorial_code/videoio/openni_orbbec_astra/openni_orbbec_astra.cpp Get properties The following properties of cameras available through OpenNI interface are supported for the depth generator: @@ -156,7 +162,7 @@ As there are two video sources that should be read simultaneously, it’s necess threads to avoid blocking. Example implementation that gets frames from each sensor in a new thread and stores them in a list along with their timestamps: -@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Read streams +@snippetlineno samples/cpp/tutorial_code/videoio/openni_orbbec_astra/openni_orbbec_astra.cpp Read streams VideoCapture can retrieve the following data: @@ -177,7 +183,7 @@ two video streams may become out of sync even when both streams are set up for t A post-synchronization procedure can be applied to the streams to combine depth and color frames into pairs. The sample code below demonstrates this procedure: -@snippetlineno samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp Pair frames +@snippetlineno samples/cpp/tutorial_code/videoio/openni_orbbec_astra/openni_orbbec_astra.cpp Pair frames In the code snippet above the execution is blocked until there are some frames in both frame lists. When there are new frames, their timestamps are being checked -- if they differ more than a half of @@ -194,5 +200,5 @@ but the depth data makes it easy. ![Depth frame](images/astra_depth.png) The complete implementation can be found in -[orbbec_astra.cpp](https://github.com/opencv/opencv/tree/4.x/samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp) +[openni_orbbec_astra.cpp](https://github.com/opencv/opencv/tree/4.x/samples/cpp/tutorial_code/videoio/openni_orbbec_astra/openni_orbbec_astra.cpp) in `samples/cpp/tutorial_code/videoio` directory. diff --git a/doc/tutorials/app/table_of_content_app.markdown b/doc/tutorials/app/table_of_content_app.markdown index 6671f6b541..c4dc34b23d 100644 --- a/doc/tutorials/app/table_of_content_app.markdown +++ b/doc/tutorials/app/table_of_content_app.markdown @@ -6,6 +6,6 @@ Application utils (highgui, imgcodecs, videoio modules) {#tutorial_table_of_cont - @subpage tutorial_video_input_psnr_ssim - @subpage tutorial_video_write - @subpage tutorial_kinect_openni -- @subpage tutorial_orbbec_astra +- @subpage tutorial_orbbec_astra_openni - @subpage tutorial_intelperc - @subpage tutorial_wayland_ubuntu diff --git a/samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp b/samples/cpp/tutorial_code/videoio/openni_orbbec_astra/openni_orbbec_astra.cpp similarity index 100% rename from samples/cpp/tutorial_code/videoio/orbbec_astra/orbbec_astra.cpp rename to samples/cpp/tutorial_code/videoio/openni_orbbec_astra/openni_orbbec_astra.cpp