RoboCupVisionPipeline

From GT RoboJackets

The vision pipeline dictates how a camera frame is turned into pose information and the steps that occur. The elements are discussed in order of their position in the pipeline. A camera frame goes in, and after processing pose information is available.

Contents 1 Camera Frame 2 Image Correction 3 Color Segmentation 4 Grouping 5 Identification 6 Positioning

Camera Frame

The camera frame is defined to be an RGB image of fixed size. The size of the frame and color mode should be fixed once the system is started.

Input: A raw camera frame.

Output: An i x j pixel image with 3 color components per pixel (RGB).

Image Correction

Handles all image correction (software and hardware) that may need to occur. It expects an RGB image and will also output an RGB image of fixed but not necessarily the same size. Distortion correction and color correction happen here.

The image correction is done on the entire image with the use of OpenGL texture mapping. A test setup was created to perform performance and remapping accuracy tests. Two major variables were changed during the tests: the number of subdivisions/points used to remap the image, and the type of rendering techniques used. The tests varied the number of subdivisions and then tested the various techniques on each.

Input: An i x j pixel image with RGB color components per pixel.

Output: An n x m pixel image with RGB color components per pixel.

Color Segmentation

Pixel by pixel color segmentation. Input expects an RGB image of fixed size. After segmentation an N bpp image of the same size as the input image will be output. The output bitmap will be a color segmentation of the input image.

Input: An n x m pixel image with a byte per component per pixel.

Output: An n x m pixel image with 16 bytes per pixel. This is the color segmented bitmap.

Grouping

Takes in the color segmentation output bitmap and performs color grouping and pixel binning based on certain criteria and thresholds. No pruning or other modification of the groups occurs, only the creation of said groups.

Input: An n x m pixel image with 16 bytes per pixel from the Color Seg.

Output: List of groups.

Each group contains:

• color
• number of pixels
• bounding box
• centroid

Identification

Uses the created color groups or blobs/bins to identify objects. Will eliminate unneeded groups and will output only the identified objects.

Input: List of Groups.

Output: List of objects. Each object has a classification and then more information depending on the classification.

Classifications:

• Self Robot
• Opponent Robot
• Ball

Information based on Classification:

• Robot
  • Groups belonging to robot
  • ID of robot
• Ball
  • Group belonging to the ball

Positioning

Takes in the identified objects and color groups that belong to that object and uses the image space pixel information to position the object into world space on the field. Further tweaks on position on an object basis occur here (i.e. height adjustments). The output is pose information in world space for the identified objects.

Input: List of objects.

Output: World space pose information for each object. The number of input objects should match the number of output objects with pose information.

Objects and Information:

• Ball
  • X,Y Position
• Self Robot
  • X,Y Position
  • Robot ID
  • Direction
• Opponent Robot
  • X,Y Position
  • Robot ID