Visual Servoing. Charlie Kemp. 4632B/8803 Mobile Manipulation Lecture 8

Size: px

Start display at page:

Download "Visual Servoing. Charlie Kemp. 4632B/8803 Mobile Manipulation Lecture 8"

Tamsyn Rich
6 years ago
Views:

1 Visual Servoing Charlie Kemp 4632B/8803 Mobile Manipulation Lecture 8

2 From:

3 4 th floor 4100Q M Building 167 First office on HSI side From:

4 Find the KUKA cup.

5 Manipulate the KUKA cup.

6 Overview Manipulation Global approach Local approach Visual Servoing

7 Manipulating a rigid body Grasping it Transporting it Placing it

8 Rigid body transformations Initial pose Final pose

9 We have a generative model!

10 Motion planning (search for an entire trajectory) From: James Kuffner's home page (CMU)

Motion planning Global solutions Drawbacks Difficult to move from simulation to the real world Sensing rarely included Usually assume known state Usually

11 Motion planning Global solutions Drawbacks Difficult to move from simulation to the real world Sensing rarely included Usually assume known state Usually assume well-modeled transitions between states May not meet real-time constraints From: Steve LaValle's home page at UIUC (

12 Can we find an efficient and robust local solution?

13 Rigid body transformations Initial pose Final pose Define an error function Locally minimize this error function Simple feedback control (PID) Not sensor specific e.g., Rod Grupen's group at UMass Amherst

14 proportional-integral-derivative controller (PID controller) From: wikipedia article on PID control

15 What poses? Relative to what? Grasping it Transporting it Releasing it

16 Visual Servoing From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

17 Visual servoing S. Hutchinson, G. Hager, and P. Corke, A tutorial on visual servo control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

18 Examples

19 Examples

20 Major options Camera placement End-effector mounted Eye-in-hand EOL (endpoint open-loop) Fixed in the workspace Might be PTZ ECL (endpoint closed-loop) Error function 3D pose Image features Control hierarchy

21 Error function in 3D From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

22 Generative Model (Basic Image Formation) Pinhole camera model Perspective projection From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

Camera Calibration http://www.vision.caltech.

23 Camera Calibration Estimates parameters focal length, principal point, skew coefficient, distortions (radial and tangential) Rectify the images

24 Example fixed camera, point to point From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

25 Example eye-in-hand, EOL From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

26 servo without reconstruction (error as a function of image features) Multi-camera Single camera Error measurements imply 3D

27 feature based servoing From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

28 Image Jacobian From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

29 Example Image Jacobian From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

30 Inverting the Image Jacobian Overconstrained Underconstrained From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

31 Resolved rate motion control From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

32 servo without calibration Estimate the jacobian online From: Jägersand M. Nelson R. Visual Space Task Specification, Planning and Control In Proc. of IEEE Int. Symp. on Computer Vision 95, p , 1995.

33 Martin Jägersand University of Rochester -> Yale -> University of Alberta Jägersand M. and Nelson R., Visual Space Task Specification, Planning and Control. In Proc. of IEEE Int. Symp. on Computer Vision 95, p , A short description of higher level aspects of uncalibrated visual control. Many experiments solving complex manipulation tasks in unstructured environments. - Martin Jägersand From:

34 From: Jägersand M. Nelson R. Visual Space Task Specification, Planning and Control In Proc. of IEEE Int. Symp. on Computer Vision 95, p , 1995.

35 Non-rigid Manipulation From: Jägersand M. Nelson R. Visual Space Task Specification, Planning and Control In Proc. of IEEE Int. Symp. on Computer Vision 95, p , 1995.

36 From: Jägersand M. Nelson R. Visual Space Task Specification, Planning and Control In Proc. of IEEE Int. Symp. on Computer Vision 95, p , 1995.

37 Discussion questions? When is visual servoing a good idea? When is visual servoing a bad idea?

38 What will you do?

39 Next week You're the presenters!

40 Extra

41 Velocity of a Rigid Object Velocity screw From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

42 Velocity of a Rigid Object (continued) From: Hutchinson et. al., A Tutorial on Visual Servo Control, IEEE Transactions on Robotics and Automation, vol. 12, pp , October 1996.

Summary of robot visual servo system

Abstract Summary of robot visual servo system Xu Liu, Lingwen Tang School of Mechanical engineering, Southwest Petroleum University, Chengdu 610000, China In this paper, the survey of robot visual servoing