Admin Stuff Course Website: http://robotics.mem.drexel.edu/mhsieh/courses/mem456/ MEM455/8 Robotics II/Advance Robotics Winter 9 Professor: Ani Hsieh Time: :-:pm Tues, Thurs Location: UG Lab, Classroom Side, Curtis Hall Instructor: M. Ani Hsieh 59 Curtis Hall Location: NEW UG Lab in Curtis Hall Pre-requisites: MEM 455 OR Instructors Approval Robots! Admin Stuff Cont. Main Text: Introduction to Mobile Autonomous Robots Supplemental Texts: Computer Vision: A Modern Approach Computer Vision Probabilistic Robotics Principles of Robot Motion: Theory, Algorithms, and Implementations Computer Grading: Assignments 3% Project 5% Class Participation 5% Teamwork 5% Grad level students will be graded at a different scale Misc: Assignments are due @ :59 pm on due date (unless otherwise specified). Assignments, w/ exception of code, must be submitted in electronically via Bb Vista in PDF format. The file must a single PDF file and must NOT exceed 5MBs. Late submissions WILL NOT be accepted 3 4
Assignments Assigned Reading Coding Exercises In-Class Presentations Project Write-ups Project Work Lower UG Lab Self-Scheduled SAS Lab (Hess Basement) Assignments Equipment SRV- Lynxmotion 5 DOF Robotic Arm IR Beacons Matlab, C/C++, Java, Python, Ruby Robot Play Pens (Setup in Lower UG Lab) Additional equipment in SAS Lab 5 6 SRV- Robot Hockey SRV- Localization SRV- Hockey Stick Puck Detection Task Allocation Projects In-Class Presentations/Discussions Based on projects Research papers, books, etc. Theory, Simulation and/or Experimental Results Lynxmotion Robotic Arm Assembly Interfacing with a PC Open-loop control of the arm Vision Based Closed-loop control SRV- Discussions Algorithm/Code design Theory and Implementation IR Beacons Assembly Interfacing w/ Arduino + PC Interfacing into SRV- Robot Cat & Mouse Game Player/Stage & USARSim HIGH level of comfort w/ C/C++ & Unix 7 Lynxmotion 5 DOF Robotic Arm IR Beacons Final Words Research focus class YOU lead discussions and topic areas Grad level students - LaTeX 8
Introduction Mobile Robotics Historical perspective 9 first use of the word robot by Czech playwright Karel Capek R.U.R. (Rossum's Universal Robots) Science fiction Asimov Metropolis (97) Formal definition (Robot Institute of America): "A reprogrammable, multifunctional manipulator designed to move material, parts, tools, or specialized devices through various programmed motions for the performance of a variety of tasks". OBSERVE ORIENT DECIDE & ACT Perception Localization Control 9 Mobile Robots Three key questions in Mobile Robots Where am I? Where am I going? How do I get there? To answer these questions the robot has to Have a model of the environment (given or obtained) Perceive and analyze the environment Find its position within the environment Plan and execute the movement How do we see the world? Sensing Light sun Course Outline Perception /3 Localization /3 Planning & Control /3 Place a piece of film in front of an object, can we get a reasonable image? 3
Pinhole Camera Pinhole Camera Let s add a barrier, w/ a *VERY* small opening. Purpose: To block off most of the light rays Reduces blurring The opening is known as aperture What happens to the image? Pinhole Model: Captures pencil of rays all rays through a single point This point is called Center of Projection (COP) The image is formed on the Image Plane Effective focal length, f, is the distance from COP to the Image Plane 3 4 Home-made Pinhole Camera Camera w/ Lense www.debevec.org/pinhole/ Why so blurry? Purpose for the lens: Ideal Pinhole Model is unattainable; light gathering mechanism Keep image in sharp focus 5 6 4
Anatomy of a modern camera Image Sensors Sensor Arrays CMOS sensor Images are formed by the interaction of the incident image irradiance with light sensitive elements on the image plane Light sensitive elements Film Charge Coupled Device (CCD) CMOS imaging element Forsythe & Ponce 7 8 Digital Imaging Systems CCD or CMOS imaging array When light falls on the cells in these arrays a charge accumulates which is proportional to the incident light energy A/D conversion unit Host Computer Digital Snapshots A digital image is an array of numbers indicating the image irradiance at various points on the image Image intensities are spatially sampled Intensity values are quantized (8-bits, -bits, -bits, etc) Video Imagery For a video camera, Images are taken sequentially by opening and close the shutter 3x/sec (i.e. 3 frames/sec) 9 5
Sensing Color Practical Color Sensing: Bayer Grid In a 3-CCD video camera the light path is split 3 ways which are passed through colored light filters and then imaged As a result a color image contains three channels of information: red, green, and blue image intensities In a -CCD color camera color information is obtained by converting the individual elements with a spatially varying pattern of filters, RGB Bayer pattern used to capture color images on a single imaging surface http://en.wikipedia.org/wiki/bayer_filter Images in a Computer Example An image is a -D table of numbers or D Matrix 3 4 6
D Image to D Pixel Optical Center (O c, O r ) 5 6 Images in a Computer An image is a -D table of numbers or D Matrix Binary Image Analysis Binary Image Image w/ only and as entries Notation: B denotes the binary image B[r,c] denotes a pixel in the image B[,] upper leftmost pixel Neighborhoods 4 Neighborhood 8 Neighborhood N NW W * E W S SW N * S NE E SE How would you convert any image into a Binary Image? 7 8 7
Applying Masks to Images Convolution of the Image w/ another Signal Masks have origins Symmetric masks origins are the center pixels 4 9 3 Segmentation Connected Components Labeling Recursive Labeling Algorithm Let I denote the image. Step : Convert I to B, Step : Let B = - B, label = for i = :rows for j = :columns if B (i,j) == - label = label + ; Add (i,j) onto Stack and set B (i,j) = label; 8 7 6 5 4 3? 9? while (Stack is not empty) Remove pixel P from Stack Let M = neighbors of P whose pixel value = -; Add M onto Stack; Set B (M) = label; 3 3 8
Algorithm Pictorially Recursive Labeling Algorithm - - - - - - - - - - - - - - - - - - - Let I denote the image. Step : Convert I to B, Step : Let B = - B, label = for i = :rows for j = :columns if B (i,j) == - label = label + ; Add (i,j) onto Stack and set B (i,j) = label; - - - - - - - - - - - - - - - - - - (3,) (3,) (3,3) (,3) while (Stack is not empty) Remove pixel P from Stack Let M = neighbors of P whose pixel value = -; Add M onto Stack; Set B (M) = label; 33 34 Result Morphological Operators Structuring Elements (S) BOX(3,5) DISK(5) RING(5) Basic Operators Translation X t of a set of pixels by a position vector t is given by X t = {x+t x X} 35 36 9
Morphological Operators Morphological Operators Dilation: Minkowski addition Erosion: 37 38 Morphological Operators Morphological Operators Opening: Closing: 39 4
Region Properties Area Total # of pixels in the region of interest Centroid Perimeter Circularity Mean Radial Distance Standard Deviation/Variance of Radial Distance Bounding Box & Extremal Points Spatial Moments Ellipse Properties Assignment Form project groups and subgroups Set up a time w/ Instructor to select project and project timeline Color Blob Extraction Due Monday /3 @ :59 PM EST ONLY submit your CODE on Blackboard!! Pair Programming, i.e. you WILL need a partner!! NO EXCEPTIONS Be ready to discuss your code in class on Tues /4 MEM455 Students Start from scratch Think of ways to optimize your code Hint: Matlab commands for your consumption reshape() 4 4