Autonomous Agents and MultiAgent Systems* Lecture 2

* These slides are based on the book byinspitinpired Prof. M. Woodridge An Introduction to Multiagent Systems and the online slides compiled by Professor Jeffrey S. Rosenschein. Modifications introduced by Prof. Ana Paiva or Prof. César Pimentel, are their sole responsibility. Autonomous Agents and MultiAgent Systems* Lecture 2 Ana Paiva, Pedro Sequeira

What is an Agent? (Wooldridge, 2009): An agent is a computer system that is situated in some environment, and that is capable of autonomous action in order to meet its delegated objectives.

What is an Agent? AGENT Perceptions Actions ENVIRONMENT Perception (via sensors) Decision Action (via effectors)

What is an Agent? Typically, control over the environment is only partial. Agents must be prepared for failure!!

Autonomy is a spectrum On one end: human beings On the other: software services What we want: to delegate high-level goals without having to specify how to achieve them A difficult balance!

What is an Intelligent Agent? (Wooldridge and Jennings, 1995): Reactivity Proactiveness Social ability

Balancing Proactiveness and Reactivity Each is easy, but the balance is not! We don t want agents that blindly execute procedures We don t want agents that never focus on goals long enough to achieve them

What properties can we use to characterize agents? Properties of Agents

Autonomy Autonomy is the agent s ability to independently determine how to achieve its delegated goals/ tasks.

Reactivity Most interesting environments are dynamic: Systems must adapt to changes and account for failure (consider abandoning current procedure) Reactivity is the agent s ability to maintain an ongoing interaction with its environment, and respond to changes that occur in it (in time for the response to be useful).

Proactiveness Reacting to an environment is easy (e.g., stimulus response rules) But we want agents to have initiative Proactiveness is the agent s ability to exhibit goaldirected behavior, by taking initiative to act in order to achieve its goals/tasks.

Social Ability An agent should not ignore the existence of other agents Other agents may not share the same goals: negotiation Other agents may have common goals: cooperation Social ability is the agent s ability to interact with other agents (and possibly humans), and eventually engage in negotiation or cooperation with them.

Adaptation Some agents may become more and more efficient over time (with experience). Adaptation is the agent s ability to learn, from experience, how to better interact with the particular environment.

Collaboration Agents may have different characteristics and competences. Agents may need each other to achieve their goals or to achieve them more efficiently. Collaboration is the agent s ability to interact with others for the purposes of achieving common goals.

Believability E.g., in animation films we, spectators, suspend our disbelief and accept the characters as if they were real. Believability is the agent s ability to create a suspension of disbelief, leading the user to temporarily accept that the agent is alive, or that it is a real character.

Mobility Among their available actions, agents can usually move within their environment. Mobility is the agent s ability to change its location in the environment, being it the physical world (robots), a virtual world (virtual agents), or an electronic network (software).

Personality. Personality is the agent s ability to behave in an individual way, that makes the agent distinguishable from its peers.

Rationality Rationality is the agent s ability to act in a way that maximizes some utility function.

Veracity Veracity is the agent s property of not knowingly communicating false information.

Agents and Objects Are agents just objects by another name? Object: encapsulates some state methods - operations that may be performed on that state communicates via message passing

Objects do it for free Agents do it because they want to

Agents and Objects Differences: agents are autonomous: agents embody stronger notion of autonomy than objects; they decide whether or not to perform an action on request from another agent agents are intelligent: capable of flexible (reactive, proactive, social) behavior agents are active: each agent continuously runs in its own thread(s) of control

Agents and Expert Systems Aren t agents just expert systems by another name? Expert systems are capable of solving problems or giving advice in some knowledge-rich domain.

Differences: Agents and Expert Expert systems are disembodied they do not interact with an environment but rather with a user Expert systems are not reactive or proactive Expert systems are not social cooperation, coordination and negotiation Systems

Intelligent Agents and AI Aren t agents just the AI project? Isn t building an agent what AI is all about? So, don t we need to solve all of AI to build an agent? AI aims to build systems that can (ultimately) communicate in natural language, recognize and understand scenes, learn, reason, use common sense, think creatively, plan, act, etc.

Intelligent Agents and AI When building an agent, we simply want a system that can choose the right action to perform, typically in a limited domain We do not have to solve all the problems of AI to build a useful agent: a little intelligence goes a long way! We made our agents dumber and dumber and dumber until finally they made money. Oren Etzioni, speaking about the commercial experience of NETBOT, Inc

Agents... Live in their environments A2 Control Environment

Agents and the Environment Agents: Typically have only partial control over the environment, in the sense that they may influence it by their actions. Have to decide what action to execute in order to attain their goals. Can perform only some of their actions, depending on the situation of the environment (preconditions).

Environment Properties Accessible vs. Inaccessible An accessible environment is one in which the agent can obtain complete, accurate, upto-date information about the environment s state Most moderately complex environments are inaccessible

Environment Properties Deterministic vs. Non-deterministic A deterministic environment is one in which any action has a single guaranteed effect The physical world is, for us humans, nondeterministic

Environment Properties Static vs. Dynamic In an static environment, the world does not change while the agent is deliberating A non-static environment is said dynamic

Environment Properties Discrete vs. Continuous In an discrete environment, there is a fixed, finite number of possible actions and percepts A non-discrete environment is said continuous

Environment Properties Episodic vs. Nonepisodic In an episodic environment, the agent s execution time can be divided in a series of intervals (episodes) that are independent from each other, in the sense that what happens in one episode has no influence on the other episodes

Agent s Interface with the environment Sensors: define the list of perceptions that the agent can use to percieve the world. Effectors/Actuators: define the list of possible actions that the agent can perform in the world.

Interacting with the Environment SENSORS ACTUATORS Cameras Speed sensor GPS Sonar Microphone Start car; accelerate; break Turn right; Turn left Change gears Start lights; start window cleaner

Applications

Application Areas Agents are usefully applied in domains where autonomous action is required. Intelligent agents are usefully applied in domains where flexible autonomous action is required. This is not an unusual requirement! Agent technology gives us a way to build systems that mainstream software engineering regards as hard!

Autonomous Agents for specialized tasks Agents (and their physical instantiation in robots) have a role to play in high-risk situations, unsuitable or impossible for humans The degree of autonomy will differ depending on the situation (remote human control may be an alternative, but not always)

Application Areas Space Industrial applications: real-time monitoring and management of manufacturing and production process, Telecommunication networks, Transportation systems, electricity distribution systems, etc. Business process management, decision support ecommerce, emarkets Information retrieving and filtering Human-computer interaction & Personal Assistants Education and Learning. Web-based learning Social Robotics and Multi-robotics teams CSCW Entertainment Social Simulation 39

During a highly successful primary mission, it tested 12 advanced, high-risk technologies in space. In an extremely successful extended mission, it encountered comet Borrelly and returned the best images and other science data ever from a comet. During its fully successful hyperextended mission, it conducted further technology tests. The spacecraft was retired on December 18, 2001. NASA Web site Agents in Space http://nmp.jpl.nasa.gov/ds1/ Deep Space 1 launched from Cape Canaveral on October 24,1998.

Agents in space NASA s Earth Observing-1 satellite, which began operation in 2000, was recently turned into an autonomous agent testbed. Image Credit: NASA NASA uses autonomous agents to handle tasks that appear simple but are actually quite complex. For example, one mission goal handled by autonomous agents is simply to not waste fuel. But accomplishing that means balancing multiple demands, such as staying on course and keeping experiments running, as well as dealing with the unexpected. "What happens if you run out of power and you're on the dark side of the planet and the communications systems is having a problem? It's all those combinations that make life exciting," says Steve Chien, principal scientist for automated planning and scheduling at the NASA Jet Propulsion Laboratory in Pasadena, Calif. 41

Agents for Air Traffic Control A key air-traffic control system suddenly fails, leaving flights in the vicinity of the airport with no air-traffic control support. Fortunately, autonomous air-traffic control systems in nearby airports recognize the failure of their peer, and cooperate to track and deal with all affected flights. Systems taking the initiative when necessary... Agents cooperating to solve problems beyond the capabilities of any individual agent

TAC SCM Negotiation was one of the key agent capabilities tested at the conference's Trading Agent Competition. In one contest, computers ran simulations of agents assembling PCs. The agents were operating factories, managing inventories, negotiating with suppliers and buyers, and making decisions based on a range of variables, such as the risk of taking on a big order even if all the parts weren't available. If an agent made an error in judgment, the company could face financial penalties and order cancellations. 43

Transport Systems Agents are used: In traffic simulations (for each driver/car, each pedestrian) As the entity controlling autonomous cars - the DARPA Grand Challenge

Autonomy for UAVs Autonomous Unmanned Aerial Vehicle (UAV), or drones. The flight tests were conducted in restricted airspace at the Australian Army s Graytown Range about 60 miles north of Melbourne. The UAV (Avatar) was guided by an on-board JACK intelligent software agent that directed the aircraft s autopilot during the course of the mission. 45

Mobile agents Agents in the Networked There is currently a lot of interest in mobile agents, that can move themselves around a network (e.g., the Internet) operating on a user s behalf Applications include: hand-held tablets/smartphones with limited bandwidth information gathering world

Internet agents Internet agents can do more than just search They can plan, arrange, buy, negotiate carry out arrangements of all sorts that would normally be done by their human user As more can be done electronically, software agents theoretically have more access to systems that affect the real-world

Intelligent IT Solutions Goal-Directed Agent technology. AdaptivEnterprise Solution Suite allow businesses to migrate from traditionally static, hierarchical organizations to dynamic, intelligent distributed organizations capable of addressing constantly changing business demands. Supports a large number of variables, high variety and frequent occurrence of unpredictable external events. 48

Agents in E-Commerce Another important rationale for internet agents is the potential for electronic commerce Most commerce is currently done manually. But there is no reason to suppose that certain forms of commerce could not be safely delegated to agents

Agents for e-commerce E-commerce Transactions - business-to-busines (B2B) Difficulties of ecommerce Trust Privacy and security Billing Reliability - business-to-consumer (B2C) - consumer-to-consumer (C2C) 50

Agents for e-commerce Help Desks Trust Privacy and security Billing Reliability 51

Agents as Interfaces Agents in interfaces The idea is to move away from the direct manipulation paradigm Agents sit over applications, watching, learning, and eventually doing things without being told taking the initiative.

Agents as Interfaces The butler metaphor: The agent can work in mobile phones or as a personal robot and is able to determine users' preferences and use the web to plan business and social events And like a real-life butler the relationship between the agent and user improves as they get to know each other better (memory) The learning algorithms will allow the butler to arrange meetings without the need to consult constantly with the user to establish their requirements. 53

The goal of the annual RoboCup competitions, which have been in existence since 1997, is to produce a team of soccer-playing robots that can beat the human world champion soccer team by the year 2050. http://www.robocup.org/ Agents & Robotics 54

Agents & Robotics Robots are agents They have sensors and effectors and live in very dynamic worlds!

A leap forward in robotics research by combining experts in microrobotics, in distributed and adaptive systems as well as in self-organising biological swarm systems. Facilitate the mass-production of microrobots, which can then be employed as a "real" swarm consisting of up to 1,000 robot clients. These clients will all be equipped with limited, pre-rational on-board intelligence. The swarm will consist of a huge number of heterogeneous robots, differing in the type of sensors, manipulators and computational power. Such a robot swarm is expected to perform a variety of applications, including micro assembly, biological, medical or cleaning tasks. Robotic Swarms 56

Social Simulation In social simulations: Agents are used as the base for the simulation of artificial societies, in order to study real societies Agents may be used to test adoption of new norms; cultural differences; etc.

Agents in elearning Agents role in e-learning Enhance e-learning content and experience give help, advice, feedback act as a peer learning participate in assessments participate in simulation personalize the learning experience Enhance LMSs facilitate participation facilitate interaction facilitate instructor s activities 58

An example 59

Agents for Entertainment Agents as Non Player Characters (NPCs) Agents to use in simulated worlds for the cinema industry Agents in stories

Discussion