OpenCV Grip Pipeline

By Mahesh

Task: Develop An OpenCV GRIP Pipeline To Detect Skystones

With this year's game awarding 20 points to teams than can successfully locate Skystones during autonomous, a fast and reliable OpenCV Pipeline is necessary to succeed in robot game. Our other two choices, using Vuforia and Tensorflow, were ruled out due to high lighting requirements and slow performance, respectively.

With many different morphological operations existing in OpenCV and no clear way to visualize them using a control hub and driver station, we used FTC Dashboard to view camera output and change variables realtime. This allowed us to more rapidly debug issues and see operations on an image, like in a driver controller and expansion hub setup.

To rapidly develop different pipelines, we used GRIP, a program designed specifically for OpenCV testing. After experimenting with different threshold values and operations, we found that a 4 step pipeline like the following would work best.

The first step is a gaussian blur, used to remove any noise from the raw camera output and smoothen the darkness of the black skystone. Next, a mask is applied to essentially crop the blurred image, allowing the pipeline to focus on only the three stones. An HSV threshold is then applied to retain colors which have low values; essentially black. Afterwards, a blob of white pixels appears near the black skystone, who's center can be determined by using a blob detector, or even by finding contours, filtering them appropriately, and placing a bounding rectangle around them, then taking the center of that rectangle to be close to the centroid of the black skystone blob. Here is a visual representation of each stage of the OpenCV pipeline:

Next Steps

The next and only step is to integrate the GRIP pipeline with our existing FTC Webcam capture system, which uses Vuforia to take frames, and decide which x-coordinates of the skystone coorespond to which positions of the skystone. Specifically, we have to take the width of the final images and divide it into three equal sections, then take the boundaries of those three sections to decide the location of each skystone.