ab8210686e
Add additional image processing operators for Ascend NPU by utilizing DVPP #3608 The user base for [Ascend NPU](https://www.hiascend.com/en/) and programming with CANN is increasing rapidly, with a growing number of users joining each day. To facilitate the use of these users, this PR provides more support for Ascend backend operators. All operators this PR offers are using use DVPP as the computational unit. Digital Vision Pre-Processing (DVPP) is an image processing unit built into the Ascend AI processor. Its main functions include image and video encoding/decoding, as well as image cropping and scaling. The high-frequency operators with NPU as the backend and basic data structure AscendMat has been provided in #3552, while it still lacks many image processing operators. Moreover, only two interpolation algorithms for the resize operator are supported in #3552. In this PR, the bilinear interpolation algorithm and nearest neighbour interpolation algorithm are implemented for the resize operator, as well as the Ascend implementation of the copyMakeBorder operator. In addition, the serialization of image processing operations is widely used in the preprocessing and post-processing stages of computer vision deep learning methods. Therefore, providing integrated operators is very meaningful for improving the convenience of use for OpenCV and deep learning crossover users. For example, torchvision also provides similar operators: [RESIZED_CROP](https://pytorch.org/vision/stable/generated/torchvision.transforms.functional.resized_crop.html?highlight=resizedcrop). Thus, this PR also provides two serialization processing operators: cropResize and cropResizeMakeBorder. ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [N/A] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [x] The feature is well documented and sample code can be built with the project CMake
3.7 KiB
Ascend NPU Image Processing
Goal
In this guide, you will gain insights into the thread safety of Ascend operators already in use, as well as discover how to effectively employ Ascend operators for image preprocessing and understand their usage limitations.
Preface
We provide a suite of common matrix operation operators that support the Ascend NPU within OpenCV. For user convenience, the new 'AscendMat' structure and its associated operators maintain compatibility with the 'Mat' interface in OpenCV. These operators encompass a wide range of frequently used functions, including arithmetic operations, image processing operations, and image color space conversion. All of these operators are implemented utilizing CANN(Compute Architecture of Neural Networks). The Ascend operator facilitates accelerated operations on the NPU by making use of CANN. This acceleration effect is particularly noticeable when working with larger images, such as those with dimensions like 2048x2048, 3840x2160, 7680x4320, etc.
Instructions on Thread Safety
Our stream function is implemented by invoking the CANN operators. In the same stream, tasks are executed sequentially, while across different streams, tasks are executed in parallel. The use of event mechanisms ensures synchronization of tasks between streams, please refer to the Stream Management documentation for details.
Example for Image Preprocessing
In this section, you will discover how to use Ascend operators for image preprocessing, including functions below:
- Add
- Rotate
- Flip
code
@add_toggle_cpp @include opencv_contrib/modules/cannops/samples/image_processing.cpp @end_toggle
@add_toggle_python @include opencv_contrib/modules/cannops/samples/image_processing.py @end_toggle
Explanation
Input Image
@add_toggle_cpp @snippet opencv_contrib/modules/cannops/samples/image_processing.cpp input_noise @end_toggle
@add_toggle_python
# Read the input image
img = cv2.imread("/path/to/img")
# Generate gauss noise that will be added into the input image
gaussNoise = np.random.normal(mean=0,sigma=25,(img.shape[0],img.shape[1],img.shape[2])).astype(img.dtype)
@end_toggle
Setup CANN
@add_toggle_cpp
@snippet opencv_contrib/modules/cannops/samples/image_processing.cpp setup
@end_toggle
@add_toggle_python
@snippet opencv_contrib/modules/cannops/samples/image_processing.py setup
@end_toggle Image Preprocessing Example
@add_toggle_cpp
@snippet opencv_contrib/modules/cannops/samples/image_processing.cpp image-process
@end_toggle
@add_toggle_python
@snippet opencv_contrib/modules/cannops/samples/image_processing.py image-process
@end_toggle
Tear down CANN
@add_toggle_cpp @snippet opencv_contrib/modules/cannops/samples/image_processing.cpp tear-down-cann
@end_toggle
@add_toggle_python
@snippet opencv_contrib/modules/cannops/samples/image_processing.py tear-down-cann
@end_toggle Results
-
The original RGB input image with dimensions of (480, 640, 3):
-
After introducing Gaussian noise, we obtain the following result:
-
When applying the rotate operation with a rotation code of 0 (90 degrees clockwise), we obtain this result:
-
Upon applying the flip operation with a flip code of 0 (flipping around the x-axis), we achieve the final result:
