Marine Robot and Dexterous Manipulatin for Enabling Multipurpose Intevention Missions WP7 Multisensory Based Manipulation Architecture GIRONA 2012 Y2 Review Meeting Pedro J Sanz IRS Lab http://www.irs.uji.es/
IRS Lab http://www.irs.uji.es/
IRS Lab http://www.irs.uji.es/ Main expertise HRI Mob. Manipulation
WP s Relationships WP9: Project Coordination and Management UJI WP8: Dissemination, Education and Training UdG WP5: Floating Manipulation UNIGE-ISME WP1: Navigation and Mapping UdG WP7: Multisensory Based Manipulation Architecture UJI WP3: Vehicles Intelligent Control Architecture HWU WP2: Single and Multiple Vehicles Control IST WP6: Hand+Arm Mechatronics System and Control UNIBO WP4: Visual/Acoustic Image Processing UIB May 5th, 2011 UdG, SPAIN 4
WP1 Navigation & Mapping Management Summary WP7: Multisensory Based Manipulation Architecture State of the work UJI UdG UIB UNIBO UNIGE- ISME IST HWU GT WP T7.1: Sensor integration 8 8 T7.2: ] Specification of 11 1 1 13 interfaces T7.3: Software development 18 1 19 T7.4: Experimental validation 8 2 1 2 1 1 15 and benchmarking 45 2 1 4 2 1 55 Total DELIVERABLE 7.1: [month 18, Responsible UJI] TR on the methodology aspects and requirements on the Multisensory and knowledge-based approach architecture for grasping and dexterous manipulation MILESTONE M2: [month 18, Responsible UJI] Object recovery from a fixed base manipulator
WP 7 Long Term Objective Related with Increasing the performance, focused on the physical interaction problem Grasping / manipulation by using different unknown objects in poor conditions (i.e. overlapping, bad visibility, etc.) First field experiments (Roses Harbour), on recovery a specific object has been demonstrated
WP7 - Multisensory Based Manipulation Architecture WP 7 Task [7.1] Sensor integration [UJI 8]; Months 1 to 12 (12 months) Task [7.2] Specification of interfaces [UJI 11] [UNIBO 1] [UNIGE-ISME 1]; Months 7 to 18 (11 months) Task [7.3] Software development [UJI 18] [UNIBO 1]; Months 11 to 34 (24 months) Task [7.4] Experimental validation and benchmarking [UJI 8] [UdG 2] [UIB 1] [UNIBO 2] [UNIGE-ISME 1] [GT 1]; Months 16 to 18 and 29 to 34 (9 months)
WP7 - Multisensory Based Manipulation Architecture T7.1 Sensor Integration [UJI 8]. Months 1 to 12 (12 months) The Physical Interaction Framework Methodology Multisensory-based framework for the specification and robust control of physical interaction tasks, where the grasp and the task are jointly considered on the basis of the Task Frame Formalism (TFF) and the Knowledge-based approach to grasping Our previous work Moving to underwater The physical interaction frames
WP7 - Multisensory Based Manipulation Architecture T7.2 Specification of interfaces [UJI 11] [UNIBO 1] [UNIGE-ISME 1]. Months 7 to 18 (11 months) UJI: Manipulation planning & supervision. High-level sensor-based closed loop control of the end-effector UNIGE: Arm control & Floating manipulation Manipulation service Input: Task request Visual info Vehicle pose and velocity Internal: Closed loop control of the end-effector for the given task Output: Resultant force/torque to apply on the vehicle. Arm/Hand sensors info: FK, joint values, etc. Current state of the action. GT: Arm development UNIBO: Hand development & control 9
UNIBO UNIGE WP7 - Multisensory Based Manipulation Architecture T7.2 Specification of interfaces [UJI 11] [UNIBO 1] [UNIGE-ISME 1]. Months 7 to 18 (11 months) The Robot Operating System (ROS) was adopted for UNIBO-UNIGE-UJI communications Service /unige/switchcontrol Service /unige/switchpriorities Topic /unige/vtg (input) Topic /unige/joint_state (output) Topic /unige/arm_state (output) Topic /unige/vte (output) Switches between different control states: initialize, park, unpark, control, etc. Sets the priorities Transform-to-goal. Used when in control mode. Report joint values Reports the current control mode of the arm and other info that might be of interest Report FK with respect to the vehicle frame Service /unibo/initialize Service /unibo/setpreshape Action /unibo/grasp Topic /unibo/joint_state (output) Topic /unibo/force_feedback (output) Initialize hand (if needed). Sets a hand preshape: cylindrical, one-finger, etc. Applies the grasp on the object. Reports joint values Report the force feedback Regarding the interface with the high-level architecture, the messages defined in D3.1 have been implemented, thus connecting the manipulation sub-architecture with the Intelligent Control Architecture of WP3. Initial experiments of the above communication mechanisms have been already carried out in simulation.
WP 7 M2 (D7.1) Object recovery from an underwater fixed base manipulator Autonomous hooking sequence of a flight data recorder prototype in water tank conditions, with the arm mounted on an aluminium structure and under manual disturbances March 2011
WP 7 M2 (D7.1) May 2011 Recovery experiments with a mobile base (Girona 500 I-AUV): Water tank, UdG
WP7 - Multisensory Based Manipulation Architecture T7.3 & T7.4 Task 7.3: Software development Object tracking and pose estimation Visual control of the manipulator 3D Reconstruction and grasp planning Task 7.4: Experimental validation and benchmarking Simulation experiments (Validating grasping controllers in UWSim) Real life experiments (Intervention experiments at Roses Harbour)
WP7 - Multisensory Based Manipulation Architecture T7.3 & T7.4 Ongoing Research 2 1 UWSim A new software tool for visualization and simulation of underwater robotic missions http://www.irs.uji.es/uwsim Multisensory based Autonomous Mobile Manipulation
1 A diagram of the main parts that compose UWSim
1 A virtual visualization of the CIRS water tank (UdG). A printed posted is placed on the bottom and loaded in multiresolution mode (seafloor texture provided by Pam Reid, Univ of Miami and Nuno Gracias, UdG)
1 Testing a tracking algorithm on virtual images
1 Arm control and manipulation actions can also be simulated
1 A survey being reproduced in UWSim, from the dataset captured during the real survey in ROS bag format. The complete sequence can be found at http://goo.gl/hekrg
1 The manipulation architecture integrated with the free-floating controller of WP5 has been validated
2 The testbed used for the experiments at IRS-Lab, UJI The arm is attached to a floating platform that is placed in the water
2 A module that combines a laser emitter with a vision system in order to recover the 3D structure of unknown objects was developed, enabling the grasp planning for autonomous execution
2 Scanning of the surface with the laser stripe while doing tracking and estimating the platform motion
2 A laser scan of an amphora and a sea urchin (left), and the camera view and user grasp specification (right). The laser stripe is detected on the image and used for the reconstruction of the 3D points.
2 Autonomous execution of the grasps by the robot
2 Motion disturbances (vehicle trajectory) generated on the floating platform during the amphora recovery experiment (top), and grasping of the sea urchin (bottom).
WP 7 Dissemination EURON http://www.springer.com/engineering/robotics/book/978-3-642-33240-1 10 th G. Giralt PhD Award : PhD Thesis of M. Prats (7 th April 2011)
WP 7 Dissemination Handling ROS Tutorial Grasp and Motion Planning with Underwater Intervention Vehicles running ROS: the experience of TRIDENT EU project 2 Regular Papers + 2 papers Workshop on Mobile Manipulation + Poster of TRIDENT 1. Fernández et al., Manipulation in the Seabed: A New Underwater Robot Arm for Shallow Water Intervention, IEEE Robotics & Automation Magazine, Submit. 2 nd review. 2012 2. Prats et al., Towards Autonomous Intervention in Underwater Environments: Experiments on Object Recovery, Journal of Ocean Engineering, Submit. 2011 3. Prats et al., Reconfigurable AUV for intervention missions: a case study on underwater object recovery, Intelligent Service Robotics, vol 5 (1), pp. 19-31, 2012. 4. García et al., USER INTERFACE ORIENTED TO THE SPECIFICATION OF UNDERWATER ROBOTIC INTERVENTIONS. Journal of Maritime Research, vol VIII (2), pp. 49-62, 2011 5. Prats et al., Reliable non-prehensile door opening through the combination of vision, tactile and force feedback. Journal of Autonomous Robots, 29(2), pp. 201-218, 2010.
WP 7 Conclusions With the combination of laser peak detection, target tracking, 3D reconstruction and grasp execution, this is a unique system in the underwater robotics literature. M2 has been successfully reached, and further experiments demonstrate the suitable progress of WP7 towards the TRIDENT objectives.
WP 7 Last Remarks