Machine Vision Localization System

From Mech
Jump to navigationJump to search

Overview

This is a machine vision system that tracks multiple targets and can send out the positions via the serial port. It was developed specifically for the Swarm Robotics project, but can be adapted for other uses. It is based upon the Indoor_Localization_System, but has several enhancements and bug fixes. Refer to Indoor_Localization_System for a basic overview of the setup of the system and the workings of the patter identification algorithm.

Major Enhancements/Changes

  • The system will now mark the targets with an overlay and display coordinate data onscreen.
  • The serial output is now formatted for the XBee radio using the XBee's API mode with escape characters.
  • The calibration routine has been improved, and only needs to be performed once.
  • A command interface for sending out commands via the serial port has been added.
  • The system will discard targets too close to the edge of the camera frame to prevent misidentification due to clipping.
  • The origin of the world coordinate is now in the middle, not the lower left corner.
  • The GUI displaying the camera frames is now full sized instead of a thumbnail. However, if your monitor isn't big enough, you can resize them.

Major Bug Fixes

  • Two major memory leaks fixed.
  • Calibration matricies are now calculated correctly.
  • File handling bug that causes an off-by-one error in LoadTargetData() fixed.


Target Patterns

The target patterns and preprocessor can be downloaded at Indoor_Localization_System#Pre-processing_program_source_with_final_patterns:.

Operation

Setting up the Cameras

Camera setup.png


The cameras should be set up according to Indoor_Localization_System with one caveat: targets at the edge of the camera frame will now be discarded. This prevents misidentification of patterns if one or more dots in the pattern fall off the screen, but it also means that there must be enough overlap that when the target is in the dead-zone of one camera, it is picked up by another camera.

Targets found in the margin are discarded.


The frames must overlap; overlap by the width of one target to ensure that it will never be discarded by both cameras.


Starting the Program

  1. When you first run the program, it will open up a console window and ask you to enter a COM port number. This is the number of the serial port that will be used to send out data. Enter the number, and press Enter. (e.g. Enter COM number: 4<enter>)
  2. The program should now connect to the cameras, and will open two new windows: one with numbered quadrants (the GUI window), and one displaying the view of one of the cameras. Use you mouse to click on the quadrant that corresponds to the camera view. Repeat until all four cameras have been matched to the quadrants.
  3. Make sure the GUI window is selected, and press Enter.
  4. Go back to the console window, and you should now see a message asking you if you want to recalibrate your cameras. Press y for yes and n for no, then press Enter. To recalibrate your cameras, see the calibration section.
  5. The GUI window should now show a thresholded image, and the console will display how many dots the camera sees. Adjust the thresholding parameters (+ and - for black/white, z and x for area) until you are satisfied with the thresholded image.
  6. Make sure the GUI window is selected, then hit Enter.
  7. Remove any calibration patterns, and hit Enter. The program should now be running.


Camera Calibration

The camera calibration routine used is explained in the document Image_Formation_and Camera_Calibration.pdf by Professor Ying Wu.

The calibration routine uses 9 equally-spaced points per camera to find a mapping from the image coordinates to the real-world coordinates. The field of view of the four cameras must overlap the center horizontal and vertical dots that form a '+' shape.

Twenty-five points are used to calibrate the cameras. The field of view must overlap the dots that form a '+'.


To calibrate:

  1. Start the program, and type in 'y' at the prompt that asks whether or not you wish to calibrate.
  2. Enter the horizontal spacing (parameter A in the diagram) and vertical spacing (parameter B in the diagram) when prompted. These numbers are recorded in the file calibration_dot_spacing.txt.
  3. The GUI window will then display a thresholded image and the console will display the number of dots seen by each camera. Adjust the thresholding values until each camera sees only the nine dots being used to calibrate the camera.
  4. Select the GUI window and hit Enter.
  5. Remove the dots.
  6. Select the GUI window and hit Enter.

The program should now be running. Your calibration information will be recorded in Quadrant0.txt, Quadrant1.txt, Quadrant2.txt, Quadrant3.txt, and calibration_dot_spacing.txt. If you choose not to recalibrate your cameras next time, the data in these files will be used to generate the calibration matrices.

When calibrating, the dots should be raised to the same height as the patterns on the robots. It is possible to calibrate one camera at a time using only 9 dots by placing the dots under one camera at a time and running the calibration routine four times, making a copy of the Quadrant_.txt file generated by the calibration routine for that camera (the other three files will be garbage) each time so that it will not be overwritten. You can then copy the four good files back into the directory.

Pausing

Hit 'p' to pause the program, and hit 'p' again to resume.

Using the Command Console

To enter the command mode, hit 'c'. When in the command mode, the main loop is not running; the command mode must be exited to resume. Type 'exit' to exit the command mode and resume the main loop.

Enter commands in the following syntax:

<command name> <target ID> <parameter 1> <parameter 2> ... <parameter N>

To broadcast the message, use ID 0.

You can also enter multiple commands at a time:

sleep 1 sleep 2 wake 4 goto 0 100 -100 deadb 0 150

Commands

Commands Table
Command Description Parameters Example
sleep The robot will stop moving and stop sending data. sleep <ID> sleep 0
wake The robot will wake from sleep and resume moving and sending data wake <ID> wake 0
goal Change the swarm's goal state. goal <ID> <Ix> <Iy> <Ixx> <Ixy> <Iyy> goal 0 100 300 160000 40000 40000
goto Similar to goal, but only changes <Ix> and <Iy>. goto <ID> <Ix> <Iy> goto 0 100 300
deadb Change the robot motor deadband. The motors will only move for
speeds faster than this. This removes some jittering.
deadb <ID> <velocity> deadb 0 150
egain Change the estimator gains. egain <ID> <KP> <KI> <Gamma> egain 0 0.7 0.1 0.05
cgain Change the motion controller gains. cgain <ID> <KIx> <KIy> <KIxx> <KIxy> <KIyy> cgain 0 2 2 0.001 0.001 0.001
commr Change the communication radius distance.
The agent will discard any packets from other agents further away than this distance.
commr <ID> <distance> commr 0 1500
speed Change the maximum allowed speed for the robot. speed <ID> <speed> speed 0 300
safed Change the obstacle avoidance threshold. speed <ID> <distance> safed 0 200

Exiting the Program

Press ESC to quit the program.