Prospective Teleautonomy For EOD Operations

Transcription

1 Perception and task guidance Perceived world model & intent Prospective Teleautonomy For EOD Operations Prof. Seth Teller Electrical Engineering and Computer Science Department Computer Science and Artificial Intelligence Laboratory MIT

2 Goal: robots that join human teams Our lab develops machines that: Have a rich, live awareness of surroundings Understand natural speech and gesture Act autonomously, under human supervision Self-driving car (DARPA urban challenge) MIT vehicle one of only 6 to complete final course, of 89 original entrants Unmanned, voice-operated operated forklift Developed for DDR&E and U.S. Army Now part of fielding effort at ARDEC Self-driving di i wheelchair h & robot arm Learns from human-narrated motions, actions Prospective teleautonomous manipulator Effort funded by ONR starting 1/1/12 Subject of today s presentation

3 What problem are we tackling? ONR BAA asked the research community to develop and demonstrate emerging technologies for dismounted missions to detect/locate, access, diagnose/identify, and render safe/neutralize explosive hazards, including IEDs and UXO. From EOD personnel at SOCPAC, SPAWAR: Degree of autonomy provided by existing platforms is very low Existing teleoperation interfaces are clumsy, even for experts Even simple missions can take tens of minutes via teleoperation Significant fraction of missions can t be completed via teleoperation; in these cases, personnel must dismount and approach manually We re pursuing a new approach that, t if successful: Will significantly reduce the need for manual approach Will significantly reduce average mission duration, error rate Could mitigate issues of cost, weight, comms req ts, s/w complexity

4 Traditional Task Decomposition H U M A N All levels of planning All scene interpretation Uses OCU to control robot DOFs Tracking of workspace dynamics Reacting to unexpected events R O B O T Sends video feed to operator Follows commands of OCU Limited Cartesian motions Motor velocities/forces/torques Raw sensor data Key limitation: operator must control high-dof arm with low-dof OCU

5 Novel Proposed Decomposition (SRI/Sarnoff Video-Trek?) Perception and task guidance Perceived world model & intent H U M A N New component on human side: Multi-touch human-robot interface R O B O T Changes human-robot division of labor: New components on robot side: Lidar (captures RGB point cloud) Prospective autonomy algorithms High-level mission, task planning Sends video, lidar feed to operator Scene interpretation, motion guidance Scene perception (human-assisted) Cues about objects, desired actions Low-level motion planning via multi-touch gestures on lidar data Scene-adaptive motion control Rather than make robot smart, make it able to follow detailed instructions

6 Video (but please keep in mind ) Proof-of-concept demonstration Research robot (WG PR2), touchscreen Actions shown: pulling a lever, opening a box flap Imagine, next, what we re working toward: Typical EOD robot (Talon, Packbot), rugged tablet Actual EOD tasks (dig, break window, open car door, detach circuit board, clip wire, use disruptor)

7 Prospective Teleautonomy

8 Novel system aspects (1/7) Mouse/touchscreen interaction No teleop knobs/joysticks/gloves/controllers/armatures* Operator no longer forced to do inverse kinematics, i.e., move joysticks to make arm & gripper move as desired Comment: elimination of complex OCU mechanism could reduce both system weight and cost *But system could retain standard OCU as backup manual control method

9 Novel system aspects (2/7) Operator s free view of workspace View rendered from bot-pov lidar point cloud Operator can change view without moving robot or arm Comment: running change detection locally, and sending new HD/scans to operator only when needed, could significantly reduce bandwidth needs

10 Novel system aspects (3/7) Operator selection of object of interest Can add/subtract lidar points while changing view System responds by highlighting its idea of object

11 Novel system aspects (4/7) Operator has free 3D view of highlighted object Comment: could integrate give me a better look at that or give this fancy sensor a better look at that

12 Novel system aspects (5/7) Operator indicates object s degrees of freedom then specifies object motion based on indicated DOFs Comment: in future, system could infer DOFs automatically by matching selected point cloud to object library

13 Novel system aspects (6/7) System animates its idea of what operator wants Waits for operator confirmation before executing plan

14 Novel system aspects (7/7) Robot executes the operator-specified motion Uses local sensing, closed-loop control to track object Comment 1: no longer need low-latency comms; could even tolerate intermittent link Comment 2: in future, operator could provide guidance continuously, during execution Comment 3: motion planner is platform-agnostic, reducing software complexity

15 Prospective Teleautonomy

16 Technical Challenges / Risks Which sensors should the system use? Size/weight/power/cost vs. utility outdoors Point clouds (esp. dynamic) don t compress well How should the human-robot interface work? Rich enough to convey operator mental model Intuitive enough for training, predictability, trust Robust enough to generalize across manipulands How to achieve robust motion planning? Must incorporate operator-indicated constraints (+/-) Must hide details of robot morphology from operator How to achieve robust motion control? Bot must track non-rigid changes in workspace, and adapt to moving manipuland geometry in real time

17 Milestones (5 parallel thrusts over 48 mos.) Bidirectional human-robot interface Touchscreen to hand gestures; don t touch constraints; articulation/manipulation gestures; complex task sequences Workspace perception Terrain classification; object segmentation; manipuland categories; fine-resolution perception; clutter; night operations Multi-modal guidance and control Simple route planning; pre-grasp stance planning; execution of don t touch plans; macro capability for task sequences End-to-end capability Simple mobility; 1-DOF actions; single platform; path adjustment; multi-dof actions; multiple platforms; autonomous approach and pre-grasp p stance; bimanual manipulation with stance adjustment Frequent engagement with EOD personnel Observation of EOD training; feedback about prototypes

18 Conclusions New approach to conduct of EOD missions Treats robot as junior partner to human operator Operator conveys interpretation, action plan to robot Robot reflects its understanding, then executes plan Requires advances along multiple fronts Human-robot interaction: assisted robot perception Sensing and perception: objects from point clouds Platform-independent motion planning & control Proposed several concrete milestones From actions to complex action sequences From single platform to multiple platforms

19 Questions / Comments? Reactions from EOD folks? Highest-priority capability gaps? Stats on mission durations, success rates? Access to training materials? Opportunities to observe training? Opportunities for EOD test/evaluation?

20