USARsim for Robocup. Jijun Wang & Michael Lewis

Transcription

1 USARsim for Robocup Jijun Wang & Michael Lewis

2 Background.. USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in Urban Search & Rescue Katia Sycara CMU- Multi agent systems Illah Nourbakhsh CMU/NASA- Robots Mike Lewis Pitt- People Jijun Wang- primary developer

3 We wanted it to: Study Teams of Robots controlled by People

4 The Idea for USARsim in Robocup High fidelity simulation environment that provides: Detailed models of USAR environments Detailed models of robots & sensors The user brings Individual robot & team control code User interface

5 Strategy vs. Dropped Leads: Why we need something between Robocup Rescue & the NIST arenas The Devil is in the details.. USAR missions fail because the operator can t figure out: Why the robot is stuck (all of us) That the robot has flipped (Robin Murphy) That the camera never pointed at the victim That the camera is pointing at a victim but we didn t see.etc! We can be strategically perfect & still fail There needs to be a level of planning & execution that mixes strategy & sensing & control issues

6

7 NIST s Existing Transportable Arenas FORMING A CONTINUUM OF CHALLENGES (from Jacoff et al. 2003) Yellow Region Simple to traverse, no agility requirements Planar (2-D) maze 60' Isolates sensors with obstacles/targets Reconfigurable in real time to test mapping 4' RED COURSE 4' 20' 28' SHALLOW RAMP (5'/15') ORANGE COURSE Orange Region More difficult to traverse, variable floorings STAIRS (5'/8') 24' Spatial (3-D) maze, stairs, ramp, holes 60' Similarly reconfigurable 4' [START/FINISH] Red Region Difficult to traverse, unstructured environment 24' Simulated rubble piles, shifting floors Problematic junk (rebar, plastic bags, pipes ) 4' YELLOW COURSE 52'

8 Human Factors Challenges World through a straw (restricted FOV) Camera control for search/navigation Survey knowledge (mapping environment) from restricted FOV & impeded movement Visual smearing from close surfaces unfamiliar ground level perspective Difficult distance judgments from degraded 2D image Difficult orientation judgments from visual cues in disorderly environment Difficult locomotion due to out-of-view & negative obstacles

9 Multi-Robot Problems Common mapping & awareness Perceptual cooperation (you see what I m stuck on) Exploiting heterogeneous capabilities Daisy chaining & communication modeling Team planning in failure-prone environments Human interaction/control of teams Etc.

10 Simulation Desiderata Expense and availability of simulation hardware and software to USAR robotics community Ease of programming to reflect targeted aspects of design Fidelity of simulation w.r.t. aspects of design to be tested

11 Simulation Requirements Video feed for teleoperation and visual search and identification Sensor simulation- for autonomous control and fused displays Simulated robot dynamics- for teleoperation and autonomous control Multiple entity simulation- to allow interaction and cooperation among teams of robots

12 Hardware Why a Game Engine? (Lewis & Jacobson, CACM 2002) Best available graphics now on Nvidia/ATI gpu s for commodity priced pc s Software Cheap Takes advantage of current gpu features Sophisticated IDE s for both behaviors (mods) and environments (levels) Client-server architecture to support multiple robots Ranging commands used by bots provide hooks needed for sensor simulation Sophisticated physics engines such as the Karma engine allow high fidelity kinematical modeling

13 Why the Unreal Engine? Lewis, M. & Jacobson, J. (2002) Game Engines in Research Communications of the Association for Computing Machinery, NY: ACM 45(1) Good graphics, good physics, object orientation, good tools, & good documentation Unreal is the defacto game engine used in research: Pitt & PARC- multiscreen (CAVE-like) displays John Laird/ U Mich- AI test environment J. Anderson & C. Leibiere- Act-R & characters K. Sycara, J. Hodgins, & G. Sukanthar- synthetic characters Architectural modeling (Notre Dame), etc. Mike Zyda & NPS- America s Army, etc. ITC center at USC, Alterne project University of Teeside & others (using our cave set-up) (crystal space, garage games, etc.)

14 Alternatives Excellent 3D graphics & drivers from VR community like UCF tools Good physics engines like ODE (being used for 3D soccer simulation) But hard to put them together.. With ODE you have to work with OpenGL in Xwindows & lose the standard formats that let you use modeling tools like 3D studio max or Maya With VR tools you have to program physical behaviors yourself

15 USARsim Architecture Team Cooperation High Level Control Controller High Level Control Controller Middle Level Control Middle Level Control Control Interface Control Interface Video Feedback Unreal Client (Attached spectator) Video Feedback Unreal Client (Attached spectator) Network Unreal Data Gamebots Control Data Unreal Engine Map Models (Robots model, Sensor model, victim model etc.)

16 USARsim Architecture Team Cooperation High Level Control Controller High Level Control Controller Middle Level Control Middle Level Control Control Interface Control Interface Video Feedback Unreal Client (Attached spectator) Video Feedback Unreal Client (Attached spectator) Network Unreal Data Gamebots Control Data Unreal Engine Map Models (Robots model, Sensor model, victim model etc.)

17 Gamebots Shot

18 Robot control architecture (simulation) Attached Unreal Spectator Control software G A M E B O T S Unreal Server

19 Karma Physics engine rigid multi- body simulation Vehicle class lets us characterize robot kinematics precisely..

20 Sensor hierarchy The trace command gives ground truth for range information Sound is attenuated according to distance Human motion (pyroelectric) uses line of sight & magnitude of movement Sensor Range Sensor Sound Sensor HumanMotion Sensor RangeScanner Sensor

21 Sensor Class HiddenSensor This Boolean value is used to indicate whether the sensor will be visually showed in the simulator. MaxRange It is the maximum distance that can be detected. Noise It is the relative random noise amplitude. With the noise, the data will be data = data + random(noise)*data OutputCurve It s the distortion curve. It is constructed by a series of points that describe the curve.

22 Sensor visualizations (courtesy of Player)

23 Can be controlled through: Native Gamebots interface Pyro middleware (including a hack for Windows) Player.. (USARsim plays the role of Stage)

24 The Arenas

25 Arena Simulations ProEngineer solid model converted to Unreal format Digital photographs used to create textures to be applied to the model Glass, mirrors, orange safety fencing, and other special effects added Rubble, debris, and victim models added to simulation Robot characteristics adapted from Karma vehicle class

26 Data Collection at NIST Arenas Gaithersburg, MD

27 Illumination levels in Lux

28 Yellow Arena

29 Yellow Arena Simulation

30 Fisheye view of Orange Arena (from Jacoff et al. 2003)

31 Simulation of Orange Arena

32

33 Red Arena Simulation

34 Orange Arena Platform: photo & simulation

35 Parts & materials to build your own

36 The Robots

37 Activmedia Pioneer P2AT P2AT has: Four wheels Skid-steer Size: 50cm x 49cm x 26cm Wheel diam: 21.5cm equipped with: PTZ camera Front sonar ring Rear sonar ring

38 Pioneer P2DX

39 irobot ATVJr

40 CMU experimental: Personal Explorer Rover (PER)

41 CMU experimental: Corky

42 First generation interface (runs with both Corky & simulation)

43 Observations Both Corky and the simulation are very difficult to control with same problems: Nonvisible obstacles Surface blinding Disorientation Camera control Difficulty with stairs or rubble

44 Validation We are currently planning validation studies comparing real & simulated PER & Pioneer robots in Orange Arena Hardware: Sensors, Kinematics Behavior: Sensors x Kinematics Automation: scripted behaviors HRI: human x automation Stefano Carpin has a student doing a validation for laser rangefinder model

45 SimpleUI sample interface

46 Pyro Controls

47 Jijun s USARsim Practicals Today (Saturday) after lunch Basic- architecture & installation Late afternoon- Controllers Pyro Player Video feedback Sunday afternoon- Programming in Unreal build a sensor, robot

48 Hardware Issues Requires current (2000 +) pentium 4 pc For Linux requires Nvidia video card Server uses few resources Clients (attached spectators) are resource hogs Without modification server handles 32 spectator clients & unlimited robots without spectators To add robot Create robot in simulation Connect socket to control robot Add spectator to provide camera view

49 Near term Mods Rewrite vehicle classes to use downloadable Unreal Runtime (warfare) Engine Extend manual instructions for Assembling new arenas from parts More pointers to available Unreal tutorials & FAQs Extend video control demo d in SimpleUI to other releases of DirectX Add Aibo (courtesy of Sheila Tejada) Send us questions so we can write a better manual & FAQ

50 USARsim & Robocup Who is the community? Our (Mike, Katia, Illah, & Jijun) interests are in: Cooperating teams Robots with adjustable autonomy Human-Robot Interaction So we would like to see USARsim in Robocup evolve from a simulation analgue of the USAR league into something in between the USAR physical robot league & the Robocup Rescue simulation league

51 USARsim & Robocup So we would favor rules that: Encourage multiple robots Encourage heterogeneous robots Encourage Human-Robot Interaction BUT For now the IMPORTANT thing is to AGREE ON RULES HERE AT THIS CAMP

52 Modest Proposal Accept USAR rules (but maybe with just a nod to the previous slide?)

53 Issues Fixing or scoring platforms & sensors a) Everybody runs the same robot & sensor suite b) Use weighting factor to reward performance using low capability platform or sensors c) As robots are added allow higher capability platforms

54 Issues Arenas & Nike Silo or Novel Environments Who is going to make them What should be modeled

55 Issues USAR/USARsim Differences Size of venue Could add dynamic events such as collapses Hazards such as fire, water, etc. Radio drop out Smoke, etc. Not this year but 2006?

56 USARsim contol & maintenance Maintain informally within USAR community Transfer to NIST to administer like physical arenas Move to Sourceforge or similar opensource repository Ideas?

57 Download simulator & docs at