Mech 296: Vision for Robotic Applications. Logistics

Transcription

1 Mech 296: Vision for Robotic Applications Lecture 6: Embedded Vision and Control 6.1 Logistics Homework #3 / Lab #1 return Homework #4 questions Lab #2 discussion Final Project discussion 6.2 1

2 Embedded Vision and Control Embedded System: Hardware and software that forms a component of some larger system and is expected to function without human intervention. 1 Embedded Vision: An embedded systems that perform image processing operations on digital images or video inputs Embedded Visual Servoing: An embedded vision application that uses video input as a sensor for feedback control 1 techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi 6.3 Today s Summary 1. Embedded Vision Systems Elements of Embedded Visual Servoing Hardware Paradigms, Software Environments 2. Lab System Hardware Overview Microprocessor Software Communications Modeling 6.4 2

3 System Components Elements of an Embedded Visual Servoing System Lens Camera Frame Grabber Vision Processor μprocessor Power Electronics & Motor Transfer Protocol: NTSC USB Firewire Camera Link Communications Link Sensor Data OR Control Commands 6.5 Hardware Paradigms Commercially available hardware for embedded vision is shipped in a variety of forms Unpackaged Components Acquisition/Processing Boxes Camera/Processing Boxes 6.6 3

4 Unpackaged Components Many vision systems are quickly and inexpensively assembled using conventional PCs Typical software development with C/C++ and framegrabber drivers Accelerated development with code libraries (Intel s OpenCV) or Integrated acquistion and processing products (LabView, Matlab) Compact systems, using single-board computers or digital signal processors (DSPs), may require longer development time Specialized development software may be required DSP optimization requires knowledge of processor, memory, and bus Lens Camera Frame Grabber Vision Processor μcontroller Power Electronics & Motor Photo Sources: Acquisition/Processing Boxes Many companies package frame grabbers and general purpose processors together in a compact package In many case, these companies also offer a development environment or toolkit to assist in rapid product design Lens Camera Frame Grabber Vision Processor μcontroller Power Electronics & Motor Photo Sources:

5 Camera/Processing Boxes A few companies package cameras, frame grabbers and processors together in a compact package Lens Camera Frame Grabber Vision Processor μcontroller Power Electronics & Motor Photo Sources: Lab Project Control Boebot using offboard (overhead) vision and offboard processing Lens Camera Frame Grabber Vision Processor μcontroller Power Electronics & Motor Program Vision Sensor and Control Law in Matlab Wireless Communications Program BS2 Processor of a Parallax Boebot

6 Programming BS2 Microcontroller We ll cover the fundamentals of Basic Stamp 2 (BS2) programming in class For a complete guide to programming the BS2 microcontroller, see the Basic Stamp Manual Required equipment PC with Basic Stamp Editor installed Serial cable Basic Stamp Board 6V Power: Power supply / 9 V battery / 4 AA battery* * Rechargeable batteries rated at 1.2 V, Alkaline at 1.5 V 6.11 Basic Stamp Editor Application: Stampw.exe or BASIC Stamp Editor v2.1 Download from Parallax Website Downloads Basic Stamp Software PBASIC Syntax: No character for line break Capitalization ignored Comments ('): Use single quote to comment remaining characters in line Single quote mark also used to identify processor type at beginning of program. Examples: '{$Stamp bs2} 'Standard BS2 Chip or '{$Stamp bs2sx} 'BS2sx Chip

7 RAM and Static Memory BS2, like most microcontrollers, is severely resource constrained Dynamic Memory (RAM): 26 Bytes available, plus an added 6 Bytes dedicated to I/O Static Memory (EEPROM): ~2kB available 6.13 Declaration & Assignment Integer Data Variable Size bit: 0-1 nib: 0-3 (4 bit) byte: (8 bit) word: (16 bit) Numbering Systems decimal: 99 binary: %1010 ascii: "a" Data Declaration Form: < name> VAR <data type> loop_count VAR WORD Constant Definitions Form: <name> CON <value> max_control CON 5 default_mode CON "b" Assignment Form: <name> = <constant> <name1> = <name2> loop_count = %101 'same as loop_count = 5 Note: Remember to declare and intialize variables before you use them in your code

8 Input/Output Pins Vdd: Regulated Voltage (5V) Servo Pins: P12-P15 Vss: Ground Dedicated Serial I/O Digital I/O Pins: P0-P Pin Variables: Assignment Pin variables are automatically declared for the 16 Input/Output (I/O) pins. Each pin variable has three binary member variables. Input state (0 or 1) (0 V or 5 V) Output state (0 or 1) (0 V or 5 V) Direction (0 or 1) (Input or Output) To assign direction use INPUT or OUTPUT (default direction is input) Form: OUTPUT <pin number / pin variable> OUTPUT 0 'sets I/O port 0 to output To assign voltage value to an output pin, use HIGH or LOW (these commands automatically switch pin direction to output) Form: HIGH <pin number / pin variable> HIGH 0 'sets direction to output and state to

9 Mathematical Operations Integer Arithmetic (+, -, *, /) All variables are unsigned integers Overflow: Excess bits lost (ex. % %0001 = %0000) Subtraction: 2 s complement rule (ex. % %0001 = %1111) Division round down (ex. %0011 / %0010 = %0001) 6.17 Flow Control Labels: PBASIC instructions execute consecutively. Labels, indicated by a trailing colon (:) can redirect flow. 1. Subroutines: GOSUB <label> (Note: Subroutines use shared namespace all variables are global) led_toggle: high 0 'toggle LED on pause 50 'delay execution for 50 ms low 0 'toggle LED off pause 50 'delay execution for 50 ms return 'return to line following GOSUB 2. While loops: GOTO <label> main: gosub led_toggle 'call subroutine LED_TOGGLE goto main Incremented Loops: FOR < name> = <constant/variable> TO <constant/variable> NEXT for loop_count = 1 to 10 gosub led_toggle 'call subroutine LED_TOGGLE next

10 Comparison Conditional Statements Form: IF <condition> THEN <statement> if control < maxcontrol then control = maxcontrol Form: IF <condition> THEN <address> if loop_count = 5 then led_toggle led_toggle: 'THEN address acts as a goto Comparison Operators Comparisons: =, <>, >, <, <=, >= Logic: AND, OR, NOT, XOR Note on '=': The same operator used for assignment and comparison Unsigned integers: All comparisons are made assuming unsigned integers. (Be careful of negatives!) 6.19 Blink light at 10 Hz Sample Program ' Blinking Light Sample Program ' {$Stamp bs2} 'Initialization high 0 'Main Loop main: low 0 pause 50 high 0 pause 50 goto main 'LED on by default, pin zero 'toggle LED off 'delay execution for N = 50 ms 'toggle LED on 'delay execution

11 Running Sample Code on BS2 Hardware With power off, connect LED and resistor in series to ground and pin 0 Use at least a 300 Ohm resistor Each individual pin can source up to 20 ma All pins together can source no more than 50 ma 5 V / 300 Ω = 17 ma Attach serial cable to computer and BS2 Apply power to BS2 P0 LED R = 300 Ω + Vss Download code Use Run Menu 6.21 The Boebot The Boebot package puts a BS2 microcontroller on wheels and adds two servo motors For our lab project, we will also add a wireless coms board

12 Servo Motor Control Continuous Rotation Servos Servo motor combines DC motor, gearbox, and H-bridge, and additional circuitry Motor Control Lines: Black (ground), Red (+5 V), White (command signal) make sure pins hooked up correctly The servo controls DC motor speed (open-loop) Commanding the Servo Microcontroller issues pulse trains on an output pin Servo measures width of each pulse (between 1 and 2 ms) to determine control command Servo accepts commands once each 20 ms period (50 Hz) 1-2 ms 20 ms 6.23 Motor Control in PBASIC Generate a pulse in PBASIC Form: PULSOUT <pin>, <width in 2 μs increments> PULSOUT 12, 500 'Generate a 1 ms pulse 'Toggle output + pause + toggle Generate a pulse train in PBASIC loop_count VAR BYTE 'declare counter LOW 12 'initialize pin 12 (servo) main: pause 1000 'wait for 1 s (1000 ms) GOSUB spin_motor 'call subroutine goto main '/********** SUBROUTINES ********************/ spin_motor: 'motor subroutine FOR loop_count = 1 TO 50 PULSOUT 12, 500 'generate a 1 ms pulse PAUSE 20 'pause for 20 ms NEXT RETURN

13 Motor Calibration Nominal zero Nominally, motor stops rotating at pulse width of 1.5 ms Pulse widths of ms result in forward rotation Pulse widths of ms result in reverse rotation Linear range The motor rotational speed will vary approximately linearly between ms Rotational speed will saturate quickly beyond this range Calibration Calibrate zero: For actual motor, zero rotational speed will occur near, but generally not at, 1.5 ms Calibrate velocity: The slope of the linear range (speed vs. pulse width) is a function of the assembled vehicle 6.25 Serial Communications Lab system uses two stage serial communication to transmit data from Matlab to Boebot 1. Wired connection from computer to basic stamp transmitter RS-232 Voltages (± 12 V) 2. Wireless connection from basic stamp transmitter to Boebot TTL Voltages (0 or 5 V)

14 Wireless Coms Ensure Connections Check 4 pins on each wireless carad Pin 1: GND to Vss Pin 2: +5VDC to Vdd Pin 3: MODE to Vdd Pin 4: TXD to pin Pin 6: RXD to pin Boebot BS2 Transmitter 6.27 Serial I/O One set of PBASIC commands may be used to send and transmit data over both connections (wired and wireless) For wired link to computer, Pin 16 is dedicated serial port For wireless card TXD and RXD, any pin P0-P15 can be used Read serial string Form: SERIN <Pin>, <BaudMode>, <Data> xpos VAR BYTE SERIN 10, 16780, xpos '2400 BAUD, No parity bit on Pin 10 Write serial string Form: SEROUT <Pin>, <BaudMode>, <Data> xpos VAR BYTE xpos = 127 SEROUT 11, 16780, xpos '2400 BAUD, No parity bit on Pin 11 Note: The BaudMode code depends on the version of the BS2 microcontroller. Consult Basic Stamp Manual for details. Note: If the SEROUT command is not functioning, try pausing at least 15 ms between characters

15 Intermediate Serial I/O Serial communication parameters Baud rate (bits per second): try 2400 to start Parity: parity allows error checking disable parity to start Asynchronous/synchronous: we ll operate in asynchronous mode (without an extra line dedicated to clock) Serial string headers If one side of the serial link drops a byte, the reading and writing devices will fall out of synch. To ensure proper synchronization of bytes in a serial string, consider adding a header byte. The SERIN <Data> block can be modified with a WAIT statement to hold processing until the header byte is received SEROUT 11, 16780, [WAIT("s"), xpos] 'wait for s to 'be transmitted 6.29 Basic Stamp Transmitter Sample Code for Basic Stamp Transmitter 'Declarations xpos VAR BYTE ypos VAR BYTE main: 'Read 2 byte serial string with 1 byte header SERIN 16, 16780, [Wait(0), xpos] 'Input from DB9 port SERIN 16, 16780, ypos 'Write 2 byte serial string with 1 byte header SEROUT 11, 16780, 0 'Output to wireless device SEROUT 11, 16780, xpos SEROUT 11, 16780, ypos GOTO main When the serial string is short, communication will not be continuous. There is a distinction between bit rate (baud) and the number of bits transmitted per second. This code can be made more compact by using a PBASIC byte array (see Basic Stamp Manual)

16 Matlab Serial Out Matlab can talk to the BS2 transmitter via a serial line plugged into the DB9 port (Pin 16) Sample Matlab code %Initialize Serial object s = serial('com1'); set(s,'baudrate',2400); fopen(s); %Create serial port object on COM1 %Default is 8 bits, no parity %Open serial port Looping = true; while looping fprintf(s,char(0)) %Print out 3 bytes starting with 0 header fprintf(s,char(100)) %Data bytes are in the range fprintf(s,char(121)) pause(0.2); %force 5 Hz output %Cleanup fclose(s); %Port must be closed prior to reuse delete(s); 6.31 Designing Serial Message Considerations Longer message contains more data Maximum number of bytes per second at 2400 baud is 300 For 10 Hz sample rate, max of 30 bytes per message However, longer message means greater control delay Two possibilities using two bytes of data Transmit sensor data, implement control law on Boebot X-Position resampled to range of Y-Position resampled to range of Transmit servo commands, with control law in Matlab Right servo motor commands between ms Left servo motor commands between ms Because one byte can t span this range, lab uses with steps at intervals of 3 milliseconds

17 Control Limiting Real servos are not ideal actuators Be nice to your servos Use nonlinear limiters in your control code Control saturation: The control law may request a larger motor speed than the no-load (maximum possible) speed Add a block to your control code that limits the maximum amplitude of the control command Slew-Rate Limiting Rapid changes in control cause wear to servos Plastic gears not built for dramatic acceleration Add a block to your control code that limits the maximum change in control between subsequent time steps 6.33 Control Design Design control using the Race-Car model from Lecture 5 Do not neglect rotational damping I θ = τ b( θ) and Damping dominates over inertia e = vθ θ b 1 ( τ ) and e = vθ Actuator commands control wheel velocities, reducing the 3 rd order problem to 2 nd order Open Task: determine actuator map

18 Boebot Kinematics Assuming no-slip between wheel and ground, kinematics are: vr () = vr ( ) + θ ( rr) 0 0 θ v L v R v 1 2 ( R L) ( )/ v= v + v θ = v v Δl Rough calibration: Assume motor performance in forward and backward directions is the same; calibrate vehicle velocity vs. servo command using vision sensor So R L 6.35 Review Commercial embedded vision products are still heavily reliant on general purpose processors Lab Project: Control Boebot using offboard (overhead) vision and offboard processing Lens Camera Frame Grabber Vision Processor μcontroller Power Electronics & Motor Program Vision Sensor and Control Law in Matlab Wireless Communications Program BS2 Processor of a Parallax Boebot

19 Final Projects Final Project will extend Lab Project using the Boebots and overhead vision 1. Speed Racer: Refine software to minimize time to complete two laps around oval 2. Dodge Ball: Complete two laps around oval, avoiding obstacles placed on the track 3. Follow the Leader: Track and pursue a Boebot pace car for two laps around the oval