Dynamic Optimization Challenges in Autonomous Vehicle Systems Fernando Lobo Pereira, João Borges de Sousa Faculdade de Engenharia da Universidade do Porto (FEUP) Presented by Jorge Estrela da Silva (Phd student at FEUP) OMPC 2013 - Summer School and Workshop on Optimal and Model Predictive Control September 9-13, 2013 Bayreuth, Germany 1
Acknowledgment The contents of this presentation builds on the effort of the LSTS researchers 2
Outline Overview of LSTS - Underwater Systems and Technologies Laboratory Overview of Optimization Issues for Autonomous Systems Some dynamic optimization developments by LSTS members 3
Networked vehicle Underwater systems and technologies lab Mission statement Design, construction and deployment of innovative vehicle/sensor systems for oceanographic, environmental, military and security applications History Laboratory established in 1997 Involves students and faculty from ECE, ME and CS Primary sponsors: DoD, FCT Additional sponsors: FP7, NATO, ADI, FLAD, Gulbenkian, PSP- UCB 4
System of systems Vehicles come and go UAV Data provisioning Intervention UAV Mixed-initiative interactions Communication links UAV Data mules DTN Control station Surface buoy Navigation beacon Autonomous surface vehicle Control station Operators come and go Drifters Oceanographic sensors Sensing links Moored sensors Localization links Persistent dirty, dull and dangerous operations over wide areas Sensor network Control station Moored sensors 5
Emergent engineering systems? Water cycle Geographically co-located (maximize synergies) Environment and oceans Defense Harbour security and surveillance 6
Unmanned Vehicles at LSTS (1) New Autonomous Underwater Vehicle (N) Autonomous Submarine for long range missions Acoustic Modem, ADCP, Sidescan Sonar, CTD, IMU, GPS Acoustic modem, Wi-Fi and GSM/GPRS communications Light Autonomous Underwater Vehicle (L) Low cost and small (lightweight) Modular sensors (altimeter, GPS, CTD, IMU, ) Acoustic modem, Wi-Fi and GSM/GPRS communications 7 vehicles built since 2008 7
Unmanned Vehicles at LSTS (2) ROV-KOS Built completely in FEUP IMU, LBL Navigation Onboard camera and robotic arm Remotely operated using a laptop/joystick On board real-time control Swordfish ASV Katamaran frame with two electric thrusters Wireless video camera, sonar Wi-Fi and GSM/GPRS communications 8
Unmanned Vehicles at LSTS (3) Lusitania UAV Picollo autopilot Radio, Wi-Fi and GSM/GPRS communications Wireless video camera Gas-powered thruster Antex X02 UAVs Frame built by the portuguese air force academy CPU stack and software developed in FEUP Wi-Fi + GSM/GPRS communications Wireless video camera 9
Overview of Optimization Issues for Autonomous Systems 10
The Role of Optimization Tool in the quest for autonomy UAV Why Optimization? Control synthesis targeting: Performance min time, min fuel Robustness worst case scenario Contraint enforcement region of operation, actuator saturation, QoS Control station Contexts : Tactical specific activity Surface buoy Autonomous surface vehicle Navigation beacon Strategical Drifters - purpose of the system Oceanographic sensors Control station Moored sensors UAV UAV Control station Moored sensors 11
Challenging Requirements System complexity UAV UAV Modelling (e.g., hydrodynamic effects) Environment rich in interacting processes high variability UAV Uncertainty Control station Surface buoy Perturbations randomness, unmodelled phenomena,... Navigation beacon Autonomous surface vehicle Limited Drifters resources (space, Oceanographic power sensors and time) Control station limited communications, sensing, computation Moored sensors partial information available Control station Moored sensors 12
Optimization Contexts Control Architecture UAV UAV UAV Supervision layer re-planning ( on-line Surface buoy optimization) Control station Control station Control station Partitions the overall problem into amenable subproblems (time horizon, level of abstraction): Organization layer planning (off-line optimization) Coordination layer pick feasible task with higher added value Drifters Oceanographic sensors Navigation beacon Maneuver layer control synthesis (feedback optimization) Structural Arrangement Moored sensors Activities logically organized to ensure task/mission completion Systems Engineering Process Autonomous surface vehicle Transformation of objectives, requirements & constraints into a System-Solution Moored sensors 13
An Application Scenario Two teams Positioning service (L team) Finding the minimum of a scalar field (S team) Teams have to coordinate activities Intra-team control: provide a service satisfying technological constraints & requirements. Inter-team control: implement a model of coordination (L team must follow S team) 14
Search team Coordinated gradient following Invariance problem 15
Some dynamic optimization developments by LSTS members 16
Dynamic optimization developments Control station Control station UAV Drifters Oceanographic sensors UAV Value Function based coordination J. Borges de Sousa, F. Lobo Pereira, A set-valued framework for coordinated motion control of networked vehicles, Journal of Computer and Systems Sciences International, 2006 F. Lobo Pereira, J. Borges de Sousa, Coordinated Control of Networked Vehicles: An Autonomous Underwater Systems, Automation and Remote Control, 2004. Model Predictive Control Surface buoy F. Lobo Pereira, J. Borges de Sousa, R. Gomes, P. Calado, MPC based coordinated control of Autonomous Underwater Vehicles, ICIAM, Vancouver, July 18-22, 2011 F. Lobo Pereira, Reach set formulation of a model predictive control scheme, MTNS 2012, 20 th Melbourne, Australia, July 9-13, 2012. Navigation beacon Moored sensors Dynamic Programming based controllers UAV Autonomous surface vehicle J. Estrela da Silva, J. Borges de Sousa, A dynamic programming based path-following controller for autonomous vehicles, Control and Intelligent Systems, Vol. 39, No. 4, 2011 J. Estrela da Silva, J. Borges de Sousa, Dynamic Programming Techniques for Feedback Control, IFAC18th World Congress, Milano, Italy, August 28 - September 2, 2011. J. Estrela da Silva, J. Borges de Sousa, F. Lobo Pereira, Experimental results with value function based control of an, NGCUV 2012 Workshop, Porto, Portugal, April 10-12, 2012. Control station Moored sensors 17
Dynamic programming based controllers Jorge Estrela da Silva, João Sousa, Fernando Lobo Pereira 18
Problem formulation Differential game (upper value solution) Subject to: 0 0 Optimal cost to reach a target: non-anticipative Adversarial (maximizing) input models disturbances and model uncertainty Input sequence a(.) is piecewise constant 19
The approach: dynamic programming for sampled data systems Value function: the main object of the dynamic programming (DP) approach: Optimal cost to reach - value function is time independent. Infinite horizon (more delicate) assume that solution converges to V(x)+ct. Approach: value function based feedback synthesis 20
Value function computation In general, it is not possible to find an analytical expression for the value function. Numerical methods are required. Numerical computation of the value function is expensive, but not impossible for systems of low dimension. And, for the considered problems, this can be done at the design stage. Our solver is based on the semi-lagrangian (SL) numerical scheme by Falcone and co-authors, see, e.g., 21
DP for sampled data systems SL scheme: iteratively apply the DPP on each grid node (value iteration) x Key to our approach: emulation of the behavior of the computer system. - Time step = control period. - Piecewise constant input sequences (sample and hold). 22
Implementation of the control law What to store on the target computer? Numerical approximation of the value function Constant control on each grid cell - Requires less computations (local optimization is avoided). - May require more storage space, depending on the dimension of the control input. - In the former approach, the computed control is, in general, closer to the optimal. 23
Example 1 - Path following model for an State variables: Cross-track error Angle relative to path Angular velocity Fin (angular) position Input r v defines the path curvature. 24
Value functions Specifications Maximum cross-track error: 6 m Maximum fin angle: 0.26 rad Inside the MCIS: Infinite horizon problem Running cost: xmin = -6.000000, -1.570796, -0.400000, -0.260000 xmax = 6.000000, 1.570796, 0.400000, 0.260000 nx = 121, 151, 7, 31 (3964807 nodes) Outside the MCIS: Minimum Time to Reach problem xmin = -9.000000, -3.141593, -0.400000 xmax = 9.000000, 3.141593, 0.400000 nx = 181, 301, 5 (272405 nodes) 25
Sea trials at Leixões Harbour (2011/07) 26
27
Final remarks (I) Further work (not discussed here) Refinements/extensions of the numerical solver - Dealing with the continuous-time nature of the disturbances in the presence of large sampling steps. - Input switching costs. Aperiodic control - Next sampling instant decided by the control law. 28
Final remarks (II) Numerical approximations lead to sub-optimality. And also to lack of the robustness: is the adversarial really doing its worst? Is this much different from the approximate dynamic programming approaches? What stability and invariance properties is it possible to assure? - Verification algorithm based on constrained convex optimization. - Partition of the state space (e.g., as given by the grid cells). - Check Lyapunov like decrease condition on each subset, using quadratic local approximation of the value function. - Very computationally expensive. 29
Thank you for your attention. 30