A Multidisciplinary Approach to Cooperative Pedro U. Lima Intelligent Systems Lab Instituto Superior Técnico Lisbon, Portugal
WHERE ARE WE?
ISR ASSOCIATE LAB PARTNERS Multidisciplinary R&D in and Information Processing Mobile robotics, intelligent systems, computer and robot vision, biomedical and evolutionary systems, signal processing, ocean robotics and dynamic systems R&D in Marine Sciences Description, experiment and modeling of oceanic ecosystems, within the areas of Ecology, Marine Biology, Physical and Chemical Oceanography, and Fisheries. R&D in Earth and Space Sciences Genesis, evolution and use of mineral resources, from land and the ocean floors, with fluid-rock interaction processes and with Mineralogy and Crystallography and their applications, including environmental management. R&D in Sustainable Technologies and Environmental Systems Sustainability, namely in terms of the needs to secure the quality of the environment, together with the management of energy resources and the economic development.
ISR ASSOCIATE LAB PARTNERS Multidisciplinary R&D in and Information Processing Mobile robotics, intelligent systems, computer and robot vision, biomedical and evolutionary systems, signal processing, ocean robotics and dynamic systems R&D in Marine Sciences Description, experiment and modeling of oceanic ecosystems, within the areas of Ecology, Marine Biology, Physical and Chemical Oceanography, and Fisheries. R&D in Earth and Space Sciences Genesis, evolution and use of mineral resources, from land and the ocean floors, with fluid-rock interaction processes and with Mineralogy and Crystallography and their applications, including environmental management. R&D in Sustainable Technologies and Environmental Systems Sustainability, namely in terms of the needs to secure the quality of the environment, together with the management of energy resources and the economic development.
ISR ACTIVITIES HIGHLIGHTS applications: land, sea and aerial robots enabling technologies: task planning, guidance, navigation, control, computer vision, image and signal processing, software architectures, multi-robot systems tech transfer: collaboration with Portuguese and foreign companies funding: from participation in several EU, ESA and nationally funded R&D projects
INTELLIGENT SYSTEMS LAB Driving theme: R&D on decentralized decision-making and control for multi-robot (networked, cooperative) systems (main focus) cognitive robots human-robot interaction large-scale systems Application-driven: e.g., monitoring and decision-making in hazardous/remote environments (e.g., space, contaminated areas, post-disaster scenarios) services (e.g., ambient assisted living, helping people in public spaces, energy consumption in buildings, health care management, water irrigation channels decentralized control) Distinctive feature: we bring together people form Artificial Intelligence Control Theory Pedro Lima, ISR/IST
ADJUSTABLE AUTONOMY RAPOSA (FOX): semi-autonomous search robot Consortium project with SME IdMind
ADJUSTABLE AUTONOMY Originally tele-operated Some recent novel autonomy features: Tether docking Stairs climbing
VISUAL TRACKING Aerial Blimp Tracking a Land Robot LAB
VISUAL TRACKING Aerial Blimp Tracking a Land Robot GYM
VISUAL TRACKING Aerial Blimp Tracking a Land Robot 2x real speed Step response (controller tuning) U trajectory tracking
VISUAL TRACKING Tracking a Ball in 3D by Moving Omni-Robot (Tajana et al, IROS 2007) 3D shape 2D contour
VISUAL TRACKING Tracking a Ball in 3D by Moving Omni-Robot (Tajana et al, SIMPAR 2008) Removing the 1 st similarity term, one can track balls of any color
SENSOR FUSION Tracking a Ball in 3D by Multiple Moving Omni-Robots communicating all particle states throughout the network is not practical, due to limited bandwidth a reduced dimension representation of the posterior distribution is required, but at the same time we want to handle the system nonlinearity Solution: model particle distribution by a Gaussian Mixture Model (GMM) and communicate GMM parameters
SENSOR FUSION Particle distribution for the two robots
SENSOR FUSION Include robot self-localization uncertainty
SENSOR FUSION Estimate GMM parameters by EM-algorithm
SENSOR FUSION Communicate GMM parameters to teammate
SENSOR FUSION Use GMM parameters to fuse info locally
ACTIVE COOPERATIVE PERCEPTION EC FP7 URUS Project Tracking People with Multiple Cameras + Robots (robots actively decide to move to improve accuracy)
PLANNING UNDER UNCERTAINTY EC FP7 URUS Project POMDPs map beliefs on events occurrence onto actions (move towards a person, move towards a fire)
STATIC + MOBILE SENSOR NETWORKS UPCOMING PROJECTS Sustainable Urban Energy Systems (MIT-Portugal) Large Scale Dynamic Sensing and Actuation Ecosystem (CMU-Portugal)
FORMATION CONTROL Formation Feasibility (Paulo Tabuada s PhD Thesis, Jan 2002) systematic framework for studying formation motion feasibility of multiagent systems algebraic conditions that guarantee formation feasibility (i.e., that the specified geometric constraints can be satisfied) given the individual robot kinematics were determined framework also enables to obtain lower dimensional control systems describing the group kinematics while maintaining all formation constraints.
FORMATION CONTROL Obstacle Compliant Formation Control (Pedro Fazenda s MSc Thesis, Jan 2008) extended Fiorelli's and Leonard's method of controlling a formation of holonomic vehicles to handle non-holonomic vehicle formations with a given geometry, compliant with nearby obstacles (including those represented by teammates) vehicles are virtually linked to each other by the influence of artificial potentials that asymptotically stabilize the formation and keep all the robots separated by specified distances each vehicle has access to the positions of all its teammates, and senses the obstacles within a limited range of its neighborhood. to avoid falling in local minima of the potential fields, the vehicles will recall the n latest positions of the leader and use this information to move around the obstacle and keep the formation
FORMATION CONTROL Formation Decentralized Low-Communication State Estimation (Sónia Marques PhD Thesis, waiting for defense applied to ESA project) each vehicle measures locally 3 distances (3 receiving antennas) to another vehicle and communicates its updated state estimates to yet another vehicle state estimates are considered as observations in the receiving vehicle navigation algorithm is based, at each vehicle, on an EKF for local measurements, and on a Covariance Intersection (CI) algorithm (plus the EKF prediction part) for the observations communicated by its neighbor CI algorithm avoids the possible divergence of the EKF at the receiving vehicle, due to correlations between measurements of the team vehicles Estimation error of each s/c Kalman filter
DISCRETE EVENT MODELS OF ROBOTIC TASKS most of the existing robotic task models are not based on formal approaches concern a small number of behaviors are tailored to the task at hand systems-theory-based task design methods for general robotic tasks can enable systematic approach to modeling, analysis and design scaling up to realistic applications analysis of formal properties design from specifications Pedro Lima, ISR/IST
DES SUPERVISION USING LOGIC SPECIFICATIONS (Lacerda, Lima, 2008) model for 1 robot + environment several models can be composed controllable events are start_receiving move_to_ball pass start_passing move_to_goal kick_ball unsupervised behavior enables several robots going to the ball or a robot start receiving a pass without a pass being made temporal logic specifications disable those undesired behaviors
PETRI NET MODELS FOR COOPERATIVE ROBOTS Pedro Lima, ISR/IST
TASK + ENVIRONMENT MODEL (Costelha, Lima, IROS 2007, AAMAS 2008) task model (2 alternatives) action model environment model Pedro Lima, ISR/IST
EXAMPLE IN SOCCER ROBOTS Programmed using Petri nets SYNCHRONIZATION Free Kick (simulated vs real)
(Palamara et al, 2008) EXAMPLE IN SOCCER ROBOTS Programmed using Petri nets RELATIONAL BEHAVIOR - PASS
DES MODEL VIEWS Levels of Abstraction Untimed Timed Stochastic Timed time associated to events duration stochastic time associated to events FSA Timed FSA STA time associated to transitions/ events duration stochastic time associated to transitions/events duration PN Timed PN SPN Pedro Lima, ISR/IST
BIO-INSPIRED SWARM CONTROL (Milutinovič, Lima, IEEE TRO 2006) Stochastic Hybrid Automaton model of large robot populations Each robot has a Hybrid Automaton model, whose input events are stochastic, leading to a Stochastic Hybrid Automaton a) Source 2 Source 1 Source 3 q=2 Population x 1 λ 21 λ 23 b) θ=π/4 x 2 θ=0 θ=-π/4 q=1 λ 12 λ 32 q=3 x 2 x 1 x 1 x 1 Pedro Lima, ISR/IST
BIO-INSPIRED SWARM CONTROL Hybrid State (x,q) pdf N discrete states Discrete State q dynamics and associated continuous part Pedro Lima, ISR/IST
BIO-INSPIRED SWARM CONTROL Pedro Lima, ISR/IST
BIO-INSPIRED SWARM CONTROL L S R 0 km 5 km Pedro Lima, ISR/IST
BIO-INSPIRED SWARM CONTROL T=3h, u 1,u 2,,u 3 [0,2] Pedro Lima, ISR/IST
ECONOMY INSPIRED INSTITUTIONAL ROBOTICS excludes ad hoc interventions (e.g., breaking legs to implement prohibition ) as institutional devices; broken legs can persist for a while, but cannot accumulate - except if transferred to genetic information ; excludes occasional, fortuitous feasible by the agents or their ancestors; excludes natural ; «Institutions are cumulative sets of persistent artificial modifications made to the environment or to the internal mechanisms of a subset of agents, thought to be functional to the collective order.» (Silva, Ventura, Lima, AAMAS 2008) it is not about specific collective goals or particular circumstances; it is about constitutional aspects of how many agents interact; e.g., at nation level, we can change policies and even the government without changing the constitution;
(Silva, Lima, ECAL 2007) INSTITUTIONAL ROBOTICS physical properties drivers must slow down and go left or right BUT... How to choose the appropriate direction not to crash one with the other?
INSTITUTIONAL ROBOTICS Portugal, Spain, Germany,... UK, South Africa, New Zealand,... Convention (road code): go right Convention (road code): go left
The roundabout case study INSTITUTIONAL ROBOTICS robots as car drivers in an urban traffic scenario shown basic behaviours: avoiding obstacles, following the wall next: recognize traffic signs, road code rules, social roles (police robots), meaningful material organization of the world (roundabouts) conformists: follow the rules; non-conformists: use utility-based considerations to defy orthodoxy
A Multidisciplinary Approach to Cooperative Pedro U. Lima Intelligent Systems Lab Instituto Superior Técnico Lisbon, Portugal