Task and Motion Policy Synthesis as Liveness Games

Transcription

1 Task and Motion Policy Synthesis as Liveness Games Yue Wang Department of Computer Science Rice University May 9, 2016 Joint work with Neil T. Dantam, Swarat Chaudhuri, and Lydia E. Kavraki 1

2 Motivation Industrial Robots Picture from robots.co 2

3 Motivation Industrial Robots Picture from robots.co Highly structured environment Pre-computed Plan 2

4 Motivation Industrial Robots Personal Robots Picture from robots.co Picture from robohow.eu Highly structured environment Pre-computed Plan 2

5 Motivation Industrial Robots Personal Robots Picture from robots.co Picture from robohow.eu Highly structured environment Unstructured environment Pre-computed Plan? 2

6 Example Kitchen Scenario Task avoid collisions eventually pick up an object Assumptions perfect sensing of current state deterministic actions 3

7 Example Kitchen Scenario Task avoid collisions eventually pick up an object Assumptions perfect sensing of current state deterministic actions 3

8 Example Kitchen Scenario Task avoid collisions eventually pick up an object Assumptions perfect sensing of current state deterministic actions 3

9 Example Kitchen Scenario Task avoid collisions eventually pick up an object Assumptions perfect sensing of current state deterministic actions Pre-computed plan not working 3

10 Example Kitchen Scenario Task avoid collisions eventually pick up an object Assumptions perfect sensing of current state deterministic actions Pre-computed plan not working Need a policy 3

11 Example Kitchen Scenario Task avoid collisions eventually pick up an object Assumptions perfect sensing of current state deterministic actions Problem: Given (1) Task Specification, (2) Geometric description of Robot and Env, and (3) Discrete abstraction of Robot and Env actions, automatically synthesize a policy that planaccomplishes not workingthe task. Pre-computed Need a policy 3

12 Challenges Uncontrollable agents What is the proper model? Policy over large state space How to efficiently synthesize the policy? Integration of task and motion planning [e.g., Bhatia et al. 11; Kaelbling and Lozano-Perez 11; Srivastava et al. 14; He et al. 15] - Information from continuous geometry 4

13 Challenges Uncontrollable agents Games between Robot and Env What is the proper model? Policy over large state space How to efficiently synthesize the policy? Integration of task and motion planning [e.g., Bhatia et al. 11; Kaelbling and Lozano-Perez 11; Srivastava et al. 14; He et al. 15] - Information from continuous geometry 4

14 Challenges Uncontrollable agents Games between Robot and Env What is the proper model? Policy over large state space How to efficiently synthesize the policy? Integration of task and motion planning [e.g., Bhatia et al. 11; Kaelbling and Lozano-Perez 11; Srivastava et al. 14; He et al. 15] Policy Synthesis Algorithm - Information from continuous geometry 4

15 Related Work Static, Deterministic domain Uncertain domain Stochastic Adversarial (Worst case) MDP [e.g., Lahijanian et al. Reactive Synthesis 10 12; Ding et al. 11; Wolff [e.g., Kress-Gazit et Task and et al. 12; Luna et al. 14] al. 09, Decastro and Motion Kress-Gazit 15; Our Problem Planning (TMP) POMDP [e.g., Grady et al. Wongpiromsarn et [e.g., Bhatia et 13 15; Kurniawati et al. al. 10; Ulusoy et al. al. 11; Kaelbling 08; Somani et al. 13] 13; Alur, Moarref, and and Topcu. 15] Lozano-Perez Planning in belief space 11; Srivastava [e.g., Kaelbling and LozanoMobile et al. 14; He et Differential Perez. 13; Levin et al. 13; Manipulation al. 15] Dynamics Wong et al. 13; Hadfield(High DOF) Menell et al. 15] 5

16 Ideas we extend 6

17 Ideas we extend Program Synthesis Syntax guided synthesis (SyGuS) [Alur et al. 13]; Counterexample guided inductive synthesis (CEGIS) [Solar-Lezama et al. 06] Satisfiability Modulo Theories (SMT) [De Moura and Bjørner. 08] - efficiently handle quantitative constraints 6

18 Ideas we extend Program Synthesis Syntax guided synthesis (SyGuS) [Alur et al. 13]; Counterexample guided inductive synthesis (CEGIS) [Solar-Lezama et al. 06] Satisfiability Modulo Theories (SMT) [De Moura and Bjørner. 08] - efficiently handle quantitative constraints Games de Alfaro and Henzinger 00; Alur, Henzinger, and Kupferman 02 Solving infinite games [Beyene et al ] Liveness Games: eventually reach a certain state 6

19 Liveness Game structure Game state space Robot states Env states Game transitions valid moves for Robot and Env Winning condition Defined using a set dst of goal states - Winning play should eventually visit a state s dst. 7

20 Liveness Game structure Game state space Robot states Env states Policy: select a proper action for the robot for every state Game transitions valid moves for Robot and Env Winning condition Defined using a set dst of goal states - Winning play should eventually visit a state s dst. 7

21 Policy Synthesis as Games Input Game Structure 8

22 Policy Synthesis as Games Input Game Structure Geometric description of Robot and Env 8

23 Policy Synthesis as Games Input Geometric description of Robot and Env Placement Graph [Nedunuri et al. 2014] Game Structure Game state space 8

24 Policy Synthesis as Games Input Geometric description of Robot and Env Placement Graph [Nedunuri et al. 2014] Game Structure Game state space Discrete abstraction of Robot and Env actions 8

25 Policy Synthesis as Games Input Geometric description of Robot and Env Discrete abstraction of Robot and Env actions Placement Graph [Nedunuri et al. 2014] Game Structure Game state space Constraints on system transitions Game transitions 8

26 Policy Synthesis as Games Input Geometric description of Robot and Env Discrete abstraction of Robot and Env actions Placement Graph [Nedunuri et al. 2014] Game Structure Game state space Constraints on system transitions Game transitions Liveness Task Specification 8

27 Policy Synthesis as Games Input Geometric description of Robot and Env Discrete abstraction of Robot and Env actions Placement Graph [Nedunuri et al. 2014] Game Structure Game state space Constraints on system transitions Liveness Task Specification Game transitions Liveness winning condition 8

28 Policy Synthesis as Games Input Geometric description of Robot and Env Discrete abstraction of Robot and Env actions Placement Graph [Nedunuri et al. 2014] Game Structure Game state space Constraints on system transitions Liveness Task Specification Game transitions Liveness winning condition Construct a policy 8

29 Policy Synthesis as Games Input Geometric description of Robot and Env Discrete abstraction of Robot and Env actions Placement Graph [Nedunuri et al. 2014] Game Structure Game state space Constraints on system transitions Game transitions Liveness Task Specification Liveness winning condition Construct a policy Find a winning strategy 8

30 Policy Synthesis Algorithm Iteratively generate a candidate and verifies its correctness Counterexample guided [Solar-Lezama et al. 06] 9

31 Policy Synthesis Algorithm Iteratively generate a candidate and verifies its correctness Counterexample guided [Solar-Lezama et al. 06] Apply heuristic to generalize failures 9

32 Geometric-Based Generalization Generalize the counterexample to a set of similar examples: Explore geometric structure reduce necessary iteration numbers - improve efficiency 10

33 Geometric-Based Generalization Counterexample Generalize the counterexample to a set of similar examples: Explore geometric structure reduce necessary iteration numbers - improve efficiency 10

34 Geometric-Based Generalization Counterexample set Counterexample Generalization Generalize the counterexample to a set of similar examples: Explore geometric structure reduce necessary iteration numbers - improve efficiency 10

35 Experiments Kitchen environment 2 chefs moving within the blue region Kitchen Scenario increasing the size of the blue region (FoodPrep Region) Task requirements: avoid collisions eventually pick up an object Comparison with the GR(1) synthesizer [Piterman, Pnueli, and Saar 2006 ] back-end solver of LTLMoP [Finucane, Jing, and KressGazit. 10] 11

36 Results In the tested benchmark, our method scales better for large problems Generalization gives order-of-magnitude speedup 12

37 Performance with quantitative constraints energy limits Still scales well About one-time slower 13

38 Conclusion Game model for policy synthesis in adversarial domains Algorithm for solving liveness games utilize geometric information (generalization) efficiently handle quantitative constraints, e.g., energy limits Future extensions: other uncertainty sources, such as sensor noises investigate additional generalization heuristics for broader domains 14

39 Thank you! Questions? 15