opencv/opencv_contrib
1
0
Fork 0
mirror of https://github.com/opencv/opencv_contrib.git synced 2025-10-18 17:24:28 +08:00
Code Issues Releases Wiki Activity

Merge pull request #2532 from AjitPant:mcc

Browse Source 
* First commit

* Updated mcc

* Fixed build warnings.

* Fixed warnings v2

* Added support for providing regions to look for the chart

* Added the neural network based detector

* Added Documentation and tests

* Fixed Warnings

* Fixed test failure

* Fixed warnings v2

* Fix Warning attemp 3

* Replaced size_t by int to fix warning

* fixed one more

* Fixed a size_t

* Fixed some bugs

* Modified the private interface a bit

* Just adding doc of what -t and -ci numbers are

* significantly improved performance of the macbeth chart detector (by two orders of magnitude, perhaps)

* Fixed the suggested changes.

* Removed imcodes from precomp

* Fixed warnings and namespace issue

* Replaced occurance of Matx33f by InputArray

Co-authored-by: Gary Bradski <garybradskigit@gmail.com>
Co-authored-by: Vadim Pisarevsky <vadim.pisarevsky@gmail.com>
This commit is contained in:
Ajit Pant
2020-08-03 14:02:46 +05:30
committed by GitHub
parent c63778cc52
commit 2554f41c47
33 changed files with 5828 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

97
modules/mcc/tutorials/basic_chart_detection/basic_chart_detection.markdown Normal file
View File

@@ -0,0 +1,97 @@
Detecting colorcheckers using basic algorithms{#tutorial_mcc_basic_chart_detection}
===========================
In this tutorial you will learn how to use the 'mcc' module to detect colorcharts in a image.
Here we will only use the basic detection algorithm. In the next tutorial you will see how you
can improve detection accuracy using a neural network.
Building
----
When building OpenCV, run the following command to build all the contrib module:
```make
cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/
```
Or only build the mcc module:
```make
cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/mcc
```
Or make sure you check the mcc module in the GUI version of CMake: cmake-gui.
Source Code of the sample
-----------
```
run
<path_of_your_opencv_build_directory>/bin/example_mcc_chart_detection -t=<type_of_chart> -v=<optional_path_to_video_if_not_provided_webcam_will_be_used.mp4> --ci=<optional_camera_id_needed_only_if_video_not_provided> --nc=<optional_maximum_number_of_charts_to_look_for>
```
* -t=# is the chart type where 0 (Standard), 1 (DigitalSG), 2 (Vinyl)
* --ci=# is the camera ID where 0 (default is the main camera), 1 (secondary camera) etc
* --nc=# By default its values is 1 which means only the best chart will be detected
Examples:
```
Run a movie on a standard macbeth chart:
/home/opencv/build/bin/example_mcc_chart_detection -t=0 -v=mcc24.mp4
Or run on a vinyl macbeth chart from camera 0:
/home/opencv/build/bin/example_mcc_chart_detection -t=2 --ci=0
Or run on a vinyl macbeth chart, detecting the best 5 charts(Detections can be less than 5 but never more):
/home/opencv/build/bin/example_mcc_chart_detection -t=2 --ci=0 --nc=5
```
@includelineno mcc/samples/chart_detection.cpp
Explanation
-----------
-# **Set header and namespaces**
@code{.cpp}
#include <opencv2/mcc.hpp>
using namespace std;
using namespace cv;
using namespace mcc;
@endcode
If you want you can set the namespace like the code above.
-# **Create the detector object**
@code{.cpp}
Ptr<CCheckerDetector> detector = CCheckerDetector::create();
@endcode
This is just to create the object.
-# **Run the detector**
@code{.cpp}
detector->process(image, chartType);
@endcode
If the detector successfully detects atleast one chart, it return true otherwise it returns false. In the above given code we print a failure message if no chart were detected. Otherwise if it were successful, the list of colorcharts is stored inside the detector itself, we will see in the next step on how to extract it. By default it will detect atmost one chart, but you can tune the third parameter, nc(maximum number of charts), for detecting more charts.
-# **Get List of ColorCheckers**
@code{.cpp}
std::vector<cv::Ptr<mcc::CChecker>> checkers;
detector->getListColorChecker(checkers);
@endcode
All the colorcheckers that were detected are now stored in the 'checkers' vector.
-# **Draw the colorcheckers back to the image**
@code{.cpp}
for(Ptr<mcc::CChecker> checker : checkers)
{
// current checker
Ptr<CCheckerDraw> cdraw = CCheckerDraw::create(checker);
cdraw->draw(image);
}
@endcode
Loop through all the checkers one by one and then draw them.

114
modules/mcc/tutorials/basic_chart_detection/chart_detection_enhanced_by_neural_network.markdown Normal file
View File

@@ -0,0 +1,114 @@
Detecting colorcheckers using neural network{#tutorial_mcc_chart_detection_enhanced_by_neural_network}
===========================
In this tutorial you will learn how to use the neural network to boost up the accuracy of the chart detection algorithm.
Building
----
When building OpenCV, run the following command to build all the contrib module:
```make
cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/
```
Or only build the mcc module:
```make
cmake -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib>/modules/mcc
```
Or make sure you check the mcc module in the GUI version of CMake: cmake-gui.
Source Code of the sample
-----------
You can run the sample code by doing
```run
<path_of_your_opencv_build_directory>/bin/example_mcc_chart_detection_with_network -t=<type_of_chart> -m=<path_to_neural_network> -pb=<path_to_models_pbtxt> -v=<optional_path_to_video_if_not_provided_webcam_will_be_used.mp4> --ci=<optional_camera_id_needed_only_if_video_not_provided> --nc=<optional_maximum_number_of_charts_in_image> --use_gpu <optional_should_gpu_be_used>
``'
* -t=# is the chart type where 0 (Standard), 1 (DigitalSG), 2 (Vinyl)
* --ci=# is the camera ID where 0 (default is the main camera), 1 (secondary camera) etc
* --nc=# By default its values is 1 which means only the best chart will be detected
Example:
```
Simple run on CPU (GPU wont be used)
/home/opencv/build/bin/example_mcc_chart_detection_with_network -t=0 -m=/home/model.pb --pb=/home/model.pbtxt -v=mcc24.mp4
```
```
To run on GPU
/home/opencv/build/bin/example_mcc_chart_detection_with_network -t=0 -m=/home/model.pb --pb=/home/model.pbtxt -v=mcc24.mp4 --use_gpu
To run on GPU and detect the best 5 charts (Detections can be less than 5 but not more than 5)
/home/opencv/build/bin/example_mcc_chart_detection_with_network -t=0 -m=/home/model.pb --pb=/home/model.pbtxt -v=mcc24.mp4 --use_gpu --nc=5
```
@includelineno mcc/samples/chart_detection_with_network.cpp
Explanation
-----------
-# **Set header and namespaces**
@code{.cpp}
#include <opencv2/mcc.hpp>
using namespace std;
using namespace cv;
using namespace mcc;
@endcode
If you want you can set the namespace like the code above.
-# **Create the detector object**
@code{.cpp}
Ptr<CCheckerDetector> detector = CCheckerDetector::create();
@endcode
This is just to create the object.
-# **Load the model**
@code{.cpp}
cv::dnn::Net net = cv::dnn::readNetFromTensorflow(model_path, pbtxt_path);
@endcode
Load the model, here the model supplied with model was trained in tensorflow so we are loading it in tensorflow, but if you have some other model trained in some other framework you can use that also.
-# **(Optional) Set the dnn backend to CUDA**
@code{.cpp}
net.setPreferableBackend(dnn::DNN_BACKEND_CUDA);
net.setPreferableTarget(dnn::DNN_TARGET_CUDA);
@endcode
Models run much faster on CUDA, so use CUDA if possible.
-# **Run the detector**
@code{.cpp}
detector->process(image, chartType, max_number_charts_in_image, true);
@endcode
If the detector successfully detects atleast one chart, it return true otherwise it returns false. In the above given code we print a failure message if no chart were detected. Otherwise if it were successful, the list of colorcharts is stored inside the detector itself, we will see in the next step on how to extract it. The fourth parameter is for deciding whether to use the net or not.
-# **Get List of ColorCheckers**
@code{.cpp}
std::vector<cv::Ptr<mcc::CChecker>> checkers;
detector->getListColorChecker(checkers);
@endcode
All the colorcheckers that were detected are now stored in the 'checkers' vector.
-# **Draw the colorcheckers back to the image**
@code{.cpp}
for(Ptr<mcc::CChecker> checker : checkers)
{
// current checker
Ptr<CCheckerDraw> cdraw = CCheckerDraw::create(checker);
cdraw->draw(image);
}
@endcode
Loop through all the checkers one by one and then draw them.

39
modules/mcc/tutorials/basic_chart_detection/debugging_the_system.markdown Normal file
View File

@@ -0,0 +1,39 @@
Customising and Debugging the detection system{#tutorial_mcc_debugging_the_system}
===========================
There are many hyperparameters that are involved in the detection of a chart.The default values are chosen to maximize the detections in the average case. But these might not be best for your use case.These values can be configured to improve the accuracy for a particular use case. To do this, you would need to create an
instance of `DetectorParameters`.
```
mcc::Ptr<DetectorParameters> params = mcc::DetectorParameters::create();
```
* `mcc::` is important.
It contains a lot of values, the complete list can be found in the documentation for `DetectorParameters`. For this tutorial we will be playing with the value of `maxError`. The other values can be configured similarly.
`maxError` controls how much error is allowed in detection. Like if some chart cell is occluded. It will increase the error. The default value allows some level of tolerance to occlusions, increasing(or decreasing) `maxError`, will increase(or decrease) this tolerance.
You can change its value simply like this.
```
params->maxError = 0.5;
```
To use this in the detection system, you would need to pass it to the process function.
```
Ptr<CCheckerDetector> detector = CCheckerDetector::create();
detector->process(image, chartType, params = params);
```
Thats how easy is it to play with the values. But there is a catch, there are a lot of parts in the detection pipeline. If you simply run it like this you would not be able to see the effect of this change in isolation. It is possible that the preceding parts detected no possible colorchecker candidates, and so changing the value of `maxError` will have no effect. Luckily OpenCV provides a solution for this. You can make the code output a multiple images, each one showing the effect of one part of the pipeling. This is disabled by default.
* This can only be used if you are compiling from sources. If you can't build from souces, and still need this feature,try raising as issue in the OpenCV repo.
To do this : Open the file `opencv_contrib/modules/mcc/include/opencv2/mcc/checker_detector.hpp`, near the top there is this line
```
// #define MCC_DEBUG
```
Uncomment this line and rebuild opencv. After this whenever you run the detector, It will show you multiple images, each corresponding to a part of the pipeline. Also you might see some repetetions like first you will see `Thresholding Output`, then some more images, and again `Thresholding Output` corresponding to same image, but slightly different from previous one, it is because internally the image is thesholded multiple times, with different parameters to adjust for different possible sizes of the colorchecker.