Lecture 1: Introduction

Transcription

1 IIIA-CSIC Lecture 1: Introduction

2 IIIA-CSIC Basic definitions. Agent Applications. Agent Communication. Agent Architectures. Building Agents. Agents and ecommerce. Game theory. A simple example. Lecture plan

3 IIIA-CSIC Introduction to agents Basic definitions. Agent Applications. Agent Communication. Agent Architectures. Building Agents. Agents and ecommerce. Game theory. Robotic and virtual agents. A simple example.

4 IIIA-CSIC Basic Definitions

5 Vision Five ongoing trends in computing: ubiquity; interconnection; intelligence; delegation; and human-orientation. Programming has progressed through: sub-routines; procedures & functions; abstract data types; objects; to AGENTS

6 Spacecraft control When a space probe makes its long flight from Earth to the outer planets, a ground crew is usually required to continually track its progress, and decide how to deal with unexpected eventualities. This is costly and, if decisions are required quickly, it is simply impracticable. For these reasons, organisations like NASA are seriously investigating the possibility of making probes more autonomous -giving them richer decision making capabilities and responsibilities. This is not science fiction: NASA s DS1 was doing this!

7 Internet agents Searching the internet for the answer to a specific query can be a long and tedious process. So, why not allow a computer program -an agent- do searches for us? the agent would be typically be given a query that would require synthesizing pieces of information from various different internet information sources. Failure would occur when a particular resource was unavailable, (perhaps due to network failure), or where results could not be obtained.

8 ecommerce Finding the best product and the best conditions for a deal is difficult when the offer is distributed among many manufacturers or retailers. Why not endowing an agent with a description of our needs and preferences and make it negotiate on our behalf the terms and conditions of the transaction? so we get liberated from the tedious comparison among sellers and can get the best possible deal?

9 Trends in Computer Science + Ubiquity Interconnection Intelligence Systems capable of independent action representing its user s interests + Delegation Humanorientation _

10 Trends in Computer Science + Systems capable of interacting with other systems to achieve their tasks + Ubiquity Interconnection Intelligence Delegation Humanorientation _

11 What is it? An Agent is a computer system capable of independent action on behalf of its user. A multiagent system is one that consists of a number of agents that interact with one another. In order to successfully interact, agents need ability to cooperate, coordinate, and negotiate. TWO KEY PROBLEMS How do we build agents capable of independent, autonomous action in order to successfully carry out the tasks that we delegate to them? How do we build agents that are capable of interacting (cooperating, coordinating, negotiating) with other agents in order to successfully carry out the tasks the we delegate to them, particularly when the other agents cannot be assumed to share the same interests and goals?

12 Agents as a paradigm for software engineering What is the field? Software engineers have derived a progressively better understanding of the characteristics of complexity in software. It is now widely recognised that interaction is probably the most important single characteristic of complex sotware. Agents as a tool for understanding human societies Multiagent systems provide a novel new tool for simulating societies, which may help shed some light on various kinds of social processes. Agents as a new model for complexity Agents permit the exploration of many different complex systems where interaction is the key: economics, ecology, the grid.

13 What is an agent? A computer system capable of autonomos action in some environement. INPUT AGENT OUTPUT ENVIRONMENT Trivial agents: thermostat, Unix daemon (e.g. biff) An agent must be capable of flexible autonomous action: Reactive: continuous interaction with the environment reacting to changes occurring in it. pro-active: to take the initiative by generating and pursuing goals Social: to interact with other agents via some agent communication language

14 Reactivity If the environment of a program is guaranteed to be fixed the program doesn t need to worry about its success or failure -the program executes blindly (v.g. a compiler). The real world is not like this: things change, information is incomplete. Most interesting environments are dynamic. Software is hard to build for dynamic domains: programs must take into account the possibility of failure -ask itself whether it is worth executing! A reactive system is one that maintains an ongoing interaction with its environment, and responds to changes that occur in it (in time for the response to be useful).

15 Proactiveness Reacting to an environment is easy (v.g. stimulus -> response rules) We generally want agents to do things for us -> goal-directed behaviour Pro-activeness = generating and attempting to achieve goals; not driven solely by events; taking the initiative. Recognising opportunities

16 Social ability The real world is a multi-agent environment: we cannot go around attempting to achieve goals without taking others into account. Some goals can only be achieved with the cooperation of others Similarly for many computer environments: witness the internet. Social ability in agents is the ability to interact with other agents (and possibly humans) via some kind of agent-communication language, and perhaps cooperate with others.

17 Sometimes other properties are required: Other properties Mobility: the ability of an agent to move around an electronic network; Veracity: an agent will not knowingly communicate false information; Benevolence: agents do not have conflicting goals, and that every agent will therefore always try to do what is asked for. Rationality: agent will act in order to achieve its goals, and will not act in such a way as to prevent its goals being achieved -at least insofar as its beliefs permit. Learning/adaptation: agents improve performance over time.

18 Physical agents that act in a physical world. Robots They operate in a highly complex environment (the real world!): partially observable, dynamic, stochastic, continuous with limited sensors and effectors. Perception, action, and navigation are very complex tasks.

19 Robots Manipulator Mobile Humanoid

20 Softbots Agents interacting in actual-world software environments by means of simple sensors and effectors. Advantages Less development and empirical cost More focused on goals

21 MAS, AI & Distributed/Concurrent Systems (Stone, 1996)

22 MAS, AI & Distributed/Concurrent Systems (Stone, 1996) Multiagent systems (SMA) are distributed systems with special features, namely: In a MAS the coordination/synchronisation structures are not fixed at design time: they come about at run-time! Agents in a MAS have their own goals unlike cooperative entities in a distributed/concurrent system. Typically the design of a distributed system is centralised, whereas the design of a MAS is distributed.

23 Client-Server Please cook a chocolate Cake. Computer request answer Computer

24 Remote evaluation Please, cook a chocolate cake. Here is the recipe: take two eggs Computer request answer Computer

25 Code on demand Please, give me the recipe Computer request answer Compùter

26 Mobile agent Here I am. May I use your oven?. migration Computer Computer

27 Agents and objects Object: State encapsulation,message passing, Methods activated by messages Agents: Autonomy. Stronger notion of autonomy, they decide for themselves whether or not to perform an action on request of another agent; Are smart. Capable of flexible (reactive, pro-active, social) behaviour, and the standard model of object has nothing to say about such types of behaviour; Are active. A multiagent system is inherently multi-threaded, in that each agent is assumed to have at least one thread of active control. OBJECTS DO IT FOR FREE Agents do it because they want to; Agents do it for money.

28 Agents and objects GASSER: Yes. Here I think the critical insight about agents---the next step in the progression of adding structure to programming technology---is the idea of direct support for 'structured persistent action.' People talk about this as autonomy, they talk about this as goal-directed behaviour, or whatever. But for me it's the concept of structured persistent action. In effect, ideally what the idea of agents gives, is a new programming tool, along the lines of objects, and along the lines of concurrent objects, but now a bit farther along in terms of the control capabilities built into the tool or framework itself. With agents, persistent action becomes part of the toolkit--part of the framework that the programmer doesn't have to build explicitly from scratch.

29 Agents and Expert Systems Expert systems disembody expertise about some domain of discourse. For instance, MYCIN about diseases in humans. A wealth of knowledge in the form of rules Gives advice from facts and posing questions to users Main differences Agents are situated in an environment Agents act Some real-time expert systems are agents.

30 Agents and AI Aren t agents just the AI project? isn t building an agent what AI is all about? AI aims at building systems that understand natural language, thinks creatively, understands scenes, uses common sense, So, don t we need to solve all AI to build an agent? When building an agent, we want a system that can choose the right action to perform, typically in a limited domain. We do not have to solve all the problems of AI to build a useful agent: a little intelligence goes a long way Oren Etzioni speaking about the commercial experience of NETBOR, Inc. We made our agents dumber and dumber and dumber until finally they made money.

31 Environments Accessible vs inaccessible An accessible environment is one where agents can obtain complete, accurate, up-to-date information about the environment s state. Most moderately complex environments (including the physical world) are inaccessible. The more accessible an environment is, the simpler is to build an agent to operate in it. Deterministic vs non-deterministic A deterministic environment is one in which an action has a single guaranteed effect -there is no uncertainty about the state that will result from performing an action. The physical world can be regarded as non-deterministic. Non-deterministic environments present greater problems to agent designers.

32 Environments Static vs dynamic A static environment can be assumed to remain unchanged except by the performance of actions by the agent. A dynamic environment is one that has other processes operating on it, and which hence changes in ways beyond the agent s control. The physical world is highly dynamic. Discrete vs continuous An environment is discrete if there are a fixed, finite number of actions and percepts in it. Russell and Norvig give a chess game as an example of a discrete environment and taxi driving as an example of a continuous one.

33 Agents as intentional systems When explaining human activity it is often useful to describe it as for instance: Carme took her umbrella because she believed it was going to rain Carles worked hard because he wanted to finish the paper on time Human behaviour is predicted and explained through the attribution of attitudes, such as believing, wanting, hoping, fearing, These attitudes are called intentional notions. Daniel Dennett coined the term intentional system to describe entities whose behaviour can be predicted by the method of attributing belief, desires anb rational acumen. Dennet identifies different grades of intentional system: A first-order intentional system has beliefs and desires (etc.) but no beliefs and desires about beliefs and desires A second-order intentional system is more sophisticated; it has beliefs and desires (and no doubt other intentional states) about beliefs and desires (and other intentional states) - both those of others and its own. Is it legitimate or useful to attribute beliefs, desires and so on, to a computer system?

34 Intentional stance Almost every object can be described by the intentional stance: It is perfectly coherent to treat a light switch as a (very cooperative) agent with the capability of transmitting current at will, who invariably transmits current when it believes that we want it transmitted and not otherwise; flicking the switch is simply our way of communicating our desires (Yoav Shoham) Most adults would consider this description as absurd. Why? The answer seems to be that while the description is consistent it does not buy us anything, since we essentially understand the mechanism sufficiently to have a simpler mechanism description of its behaviour. (Yoav Shoham) Put crudely, the more we know about a system, the less we need to rely on animistic, intentional explanations of its behaviour. But with very complex systems, a mechanistic, explanation of its behaviour may not be practicable. As computer systems become ever more complex, we need more powerful abstractions and metaphors to explain their operation -low level explanations become impractical. The intentional stance is such abstraction.

35 Intentional systems Intentional notions are (further) abstraction tools as have been in the past: procedural abstraction, abstract data types, object. So why not use the intentional stance as an abstraction tool in computing -to explain, understand, and, crucially, program computer systems. Helps in characterising agents Provides us with a familiar non-technical way of understanding and explaining agents. Helps in nested representations Provides us with a means to include representations of other systems. Which is considered to be essential for agents to cooperate. Leads to a new paradigm: post-declarative systems Procedural programming: How to do something. Declarative programming: What to do. But how as well. Agent programming: What to do. No buts. Theory of agency determines the control.

36 IIIA-CSIC Agent applications

37 Some general application areas Industrial applications manufacturing process control telecommunications transportation systems Business Process Management Electronic Commerce electronic markets/auctions trading agents Business intelligence What-if analysis Entertainment and virtual environments Military

38 Industrial applications Manufacturing at Daimler Benz

39 production lines Daimler Benz Manufacturing and supply nets Daimler Benz Objective improve manufacturing and supply processes bodyn-white painting assembly Rastatt Untertürkheim Marienfelde plant network Sindelfingen inter-organizational supply net vehicle assembly plant

40 Growing surplus of industrial capacity and globalization lead to Daimler Benz shorter product life-cycles reduced time-to-market increased product variety volatile demand reduced investment costs New manufacturing requirements product flexibility volume scalability robustness

41 Production Flexible and robust production system Disadvantages of transfer lines dedicated machines rigid material flow introduce more hardware flexibility flexible transportation system flexible machine

42 Production An agent-based control system work pieces find their way work piece agent switch agent switches direct the traffic machine agent Control forces: work pieces push themselves machines limit work-in-process machines control their work load

43 IIIA-CSIC Production Field test productivity increase at least 10% by-pass to existing transfer line

44 Business process management Container transport in Rotterdam By Tryllian

45 Harbour logistics 10% of the 110 EUR on costs caused by logistic inaccuracies handling sea containers Not all parties involved in the scanning process of a sea container are efficiently (or even) connected High dependency on human availability and skill

46 Virtual Harbour. Container scanning Port of Rotterdam Demo Transporters Customs Container Selection 5% 95% Scan Appointment Scan Final destination Transportation Appointment Forwarder

47 Virtual Harbour. System overview Customs Agent system 1 4 Transport list Agent Forwarder Agents 5 External transporter Agents Scan Agent 9 Internal transporter Agent 6 7 Transporter Output Agent 8 Scan servlet XML-file Excel-file SMSmessage Transporter servlet

48 Virtual harbour benefits Improved communication Less human activity involved Automated interactions Tasks completed faster Not reliant on human availability Granular in implementation Easily extended with new features, services and functionality Robust but easy to build Implement at low cost and easy to maintain No central server needed, preventing a single point of failure Inherently flexible in design of business rules and relationships

49 Electronic commerce Multiagent system for fish trading by IIIA-CSIC & Autec

50 Electronic commerce Commercial version of an electronic institution It supports the federation of actual-world fish markets It allows for human and software buying agents Human buyers can simultaneously participate in several auctions thanks to agents with customisable buying strategies

51 MASFIT

52 Buyer Agents Virtual world Electronic Institution FM+ Real Auction house Auction Software Real Auction house MASFIT Real world Human buyers Auction Software Real world

53 Business intelligence Supply network management by P&G and BiosGroup Inc.

54 Agent-based model of supply chain operations Software agents represent the components of the supply system (e.g. trucks, drivers, stores, etc.) Behaviour of each agent programmed via rules that mimic actual behaviour. "Dispatch this truck only when it is full or "Make more shampoo when inventory falls to x days' demand." What-if analysis via simulations to test impact of new logistics rules on: inventory levels transportation costs in-store stock-outs The models considered alternate rules on ordering and shipping frequencies, distribution centre product allocation policies, demand forecasting and so on. From SAP to ASAP: Agents of Change Gary Anthes, Computerworld, January

55 P&G s conclusions Relaxation of rigid rules to improve the overall performance of the supply network. That required some cultural changes, such as convincing freight managers that it's sometimes OK to let a truck go half full. More flexibility in manufacturing. P&G s manufacturing processes no longer produce long runs of a single product: every product produced daily. Benefits: fewer stock-outs and happier customers. More flexibility in distribution (e.g. possible to restock a retailer in 24 hours rather than the customary 48 to 72 hours).

56 Entertainment Crowd generation for movies by MASSIVE

57 Entertainment 3D animation system for generating crowd related visual effects for film and television. Used in the epic battle sequences for The Lord Of The Rings film trilogy AGENTS OF DESTRUCTION: Thousands of digitally created fighters clash with humanity in the Battle of Helm s Deep.

58 Massive. Build the agent Massive agents begin life rendered as three-dimensional characters, not the stick figures of older software. Each body part has a size and a weight that obeys the laws of physics.

59 Massive. Inset Motion Movement data gleaned from human actors performing in a motion-capture studio is loaded into the Massive program and associated with a skeleton. The programmer can fine-tune the agents' movements using the controls on the bottom of the screen.

60 Massive. Create the brain The core of every agent is its brain, a network of thousands of interconnected logic nodes. One group of nodes might represent balance, another the ability to tell friend from foe. The agent's brain relies on fuzzy logic.

61 Massive. Replicate When a variety of agents have been developed, copies are created from each blueprint, then the copies are tweaked to give each fighter a unique mix of various characteristics height, aggressiveness, even dirtiness. The 50,000 characters in the scene will act as individuals. They are placed into a battlefield grid for testing.

62 Massive attack The simulations begin. Since agents are programmed with tendencies, not specific tasks to accomplish, there is no guarantee they will behave as the director needs them to. Only through a trial-and-error process, in which the characters' programs are constantly adjusted, is the battle scene finalized.

63 Military Grid infrastructure By DARPA

64 DARPA COABS The military environment is dynamic with quickly changing operations, hardware and software moving, connecting and disconnecting, and burst bandwidth availability. Inflexible stove-piped legacy systems never meant to be integrated are of vital importance to military planning and operations. Multiple hardware and software platforms as well as data interfaces and standards further complicate the picture. Military personnel suffer from information overload (from the modern battlefield) with no adequate tools to filter and correlate data.

65 COABS Grid Control of Agent-Based Systems (CoABS) (DARPA) to develop and demonstrate techniques to safely control, coordinate, and manage large systems of autonomous software agents. CoABS is investigating the use of agent technology to improve military command, control, communication, and intelligence gathering. A goal of CoABS is to enhance the dynamic connection and operation of military planning, command, execution, and combat support systems to quickly respond to a changing operational picture. CoABS Grid middleware enables the dynamic interoperability of distributed, heterogeneous, objects, services, and multi-agent systems. flexible run-time communications dynamic registration and discovery of relevant participants adaptive and robust, evolving to meet changing requirements without reconfiguring the network.

66

67 Conclusions Agents have a high potential Useful in complex domains: Highly dynamic environments Multiple parties involved High degree of coordination and collaboration required

68 IIIA-CSIC Agent communication

69 Speech acts Most treatments of communication in MAS take inspiration on speech act theory Speech act theories are pragmatic theories of language, i.e. Theories of language use: they attempt to account for how language is used by people every day to achieve their goals and intentions Origin: Austin s 1962 book, how to do things with words.

70 Speech acts Austin noticed that some utterances are rather like physical actions that appear to change the state of the world. Paradigm examples are: Declaring war Christening I now pronounce you man and wife But more generally, everything we utter is uttered with the intention of satisfying some goal or intention. A speech act theory is a theory of how utterances are used to achieve intentions.

71 Identified different types of speech act: Representatives: such as informing, e.g., it is raining Searle Directives: attempts to get the hearer to do something, e.g., please make the tea Commisives: which commit the speaker to doing something, e.g., I promise to Expressives: a speaker expresses a mental state, e.g., thank you! Declarations: such as declaring war or christening.

72 Components In general, a speech act can be seen as having two components: A performative verb: (e.g. request, inform, ) Propositional contents: (e.g. the door is closed ) Examples Performative = request Content = the door is closed Act = please close the door Performative = inform Content = the door is closed Act = the door is closed! Performative = inquire Content = the door is closed Act = is the door closed?

73 Plan-based semantics Cohen and Perrault (1979) used the precondition-deleteadd formalism from planning to give semantics Request(s,h,f) pre (s believe (h can do f)) AND (s believe (h believe (h can do f))) (s believe (s want f)) post (h believe (s believe (s wants f)))

74 KQML = Outer language defining performatives. Ask-if ( is it true that ) Perform ( please perform the following action ) Tell ( it is true that ) Reply ( the answer is ) KIF = inner language for content. Examples: (ask-if (> (size chip1) (size chip2))) (reply true) (inform (= (size chip1) 20)) (inform (= (size chip2) 18)) KQML+KIF

75 FIPA ACL 20 performatives Similar structure to KQML (inform :sender agent1 :receiver agent5 :content (price good ) :language s1 :ontology trade)

76 Inform and request inform and request are the two basic performatives, the rest are macros Meaning: Precondition: what must be true for the speech act to succeed Rational effect: what the sender hopes to bring about Inform: content is a statement, precondition is that sender holds that the content is true, intends the recipient to believe it and does not believe that the recipient is aware of whether the content is true or not. Request: content is an action, precondition is that sender intends the action to be performed, believes recipient is capable of doing it and does not believe that receiver already intends to perform it.

77 Social commitments semantics Criticism to the mentalist approach. Semantics of illocutions are given by the social commitments/ expectations they create. Illocutions make the state of commitments evolve.

78 IIIA-CSIC Agent Architectures

79 Three types: symbolic agents 1985-present reactive agents 1990-present hybrid agents Agent architectures are: Agent Architectures A particular methodology for building agents. It specifies how the agent can be decomposed into the construction of a set of component modules and how these modules should be made to interact. The total set of modules and their interactions has to provide an answer to the question of how the sensor data and the current internal state of the agent determine the actions and future internal state of the agent. An architecture encompasses techniques and algorithms that supports this methodology (Pattie Maes) A particular collection of software (or hardware) modules, typically designated by boxes with arrows indicating the data and control flow among the modules. A more abstract view of an architecture is as a general methodology to designing particular modular decompositions for particular tasks. (Kaelbling).

80 Symbolic agents They contain an explicitly represented symbolic model of the world They make decisions (for example about what actions to perform) via symbolic reasoning. They face two key problems: The transduction problem. Translate the real world into an accurate representation in time for it to be useful The representation/reasoning problem. How to represent information about real world entities and processes, and how to get agents to reason with this information in time for the results to be useful. These problems are far from being solved Underlying problem: symbol manipulation algorithms are usually highly intractable.

81 Deductive reasoning agents Idea: Use logic to encode a theory stating the best action to perform in any situation: ρ is the theory (usually a set of rules) Δ a logical database that describes the current state of the world Ac the set of actions the agent can perform Δ ρ φ actions means that φ can be proved Δ using ρ. %try to find an action explicitly prescribed for each a in Ac do if Δ ρ Do(a) then return a endif endfor %try to find an action not excluded for each a in Ac do if Δ / ρ Do(a) then return a endif endfor

82 Example The vacuum world. Goal is the robot top clear up all dirt.

83 3 predicates 3 actions Agent architecture In(x,y) Agent is at (x,y) Dirt(x,y) there is dirt at (x,y) Facing(d) the agent is facing direction d Ac = {turn, forward, suck} %turn means turn right Rules ρ for determining what to do: In(0,0) and Facing(north) and ~Dirt(0,0) then Do(forward) In(0,1) and Facing(north) and ~Dirt(0,1) then Do(forward) In(0,2) and Facing(north) and ~Dirt(0,2) then Do(turn) In(0,2) and Facing(east) then Do(forward) And so on! With these rules and other obvious ones the robot will clean it up.

84 Problems How to convert a video stream into Dirt(0,1)? Decision making assumes a static environment Decision making using first order logic is undecidable Even when using propositional logic decision making in the worst case means NP-completeness. Typical solutions Weaken the logic Use symbolic, non-logical representations

85 Agent0 and PLACA Shoham s notion of agent oriented programming (AOP) AOP is a new programming paradigm, based on a societal view of computation. The key idea of AOP is that of directly programming agents in terms of intentional notions like belief, commitment and intention Motivation: in the same way that we use the intentional stance to describe humans, it might be useful to use the intentional stance to program machines.

86 AOP 3 components for an AOP A logic for specifying agents and describing their mental states An interpreted programming language for programming agents An agentification process for converting neutral applications (e.g. Databases) into agents Results were reported just on the first two components. Relation between the first two = semantics.

87 Agent0 Extension to LISP Each agent has A set of capabilities A set of initial beliefs A set of initial commitments, and A set of commitment rules The key element that determines how the agent acts is the commitment rule set.

88 Commitment rules Each commitment rule has A message condition A mental condition, and An action On each agent cycle The message condition is matched against the received messages The mental condition is matched against the beliefs If the rule fires the agent becomes committed to the action (it is added to the agent s commitment set)

89 Commitment rules Actions may be Private: an internally executed computation, or Communicative: sending messages Messages are constrained to be one of requests to commit to action unrequests to refrain from actions informs which pass on information

90 Agent0 Control Initialize Update beliefs Update commitments Messages in Beliefs Commitments Execute Abilities Internal actions Messages out

91 A commitment rule Commit( ( agent, REQUEST, DO(time, action) ), ;;; msg condition ( B [now, friend agent] AND CAN(self, action) AND NOT [time, CMT(self, anyaction)] ), ;;;mental condition self, DO(time, action) )

92 Agent0 in context Agent0 provides support for multiple agents to cooperate and communicate, and provides basic provision for debugging It is a prototype, rather than a production language Thomas implemented an improvement in 1993: PLACA Permitted agents to plan and communicate requests for action via high level goals Agents in PLACA are programmed in much the same way, in terms of mental change rules.

93 (((self?agent REQUEST (?t (xeroxed?x))) AND (CAN-ACHIEVE (?t xeroxed?x)) NOT (BEL (*now* shelving))) NOT (BEL (*now* (vip?agent)))) ((ADOPT (INTEND (5pm (xeroxed?x))))) ((?agent self INFORM (*now* (INTEND (5pm (xeroxed?x))))))) PLACA If someone asks you to xerox something, and you can, and you don t believe that they are a vip or that you re supposed to be shelving books, then adopt the intention to do it by 5pm, and inform about the newly adopted intention

94 Concurrent METATEM Is a multiagent language in which each agent is programmed by giving it a temporal logic specification of the behaviour it should exhibit These specifications are executed directly in order to generate the behaviour of the agent Temporal logic is classical logic augmented by modal operators for describing the truth of propositions along time.

95 Reactive architectures Symbolic AI has many unsolved problems basically on complexity Many researchers have argued about the viability of the whole approach (Chapman) Although they share a common belief that mainstream AI is in some sense wrong they have very different approaches

96 Brooks-behaviour languages Three theses: Intelligent behaviour can be generated without explicit representations of the kind that symbolic AI proposes Intelligent behaviour can be generated without explicit abstract reasoning of the kind that symbolic AI proposes. Intelligence is an emergent property of certain complex systems. Two ideas inspiring him: Situatedness and embodiment: real intelligence is situated in the world, not in disembodied systems like theorem provers of KBS. Intelligence and emergence: intelligent behaviour arises as a result of the interaction with its environment.

97 Subsumption architecture Hierarchy of task-accomplishing behaviours Each behaviour is a simple rule-like structure Behaviours compete with others to control the agent Lower layers represent more primitive kinds of behaviour (such as avoiding obstacles) and have preference over higher levels The resulting system in extremely simple in terms of the amount of computation to perform Some of the robots do impressive tasks from the perspective of symbolic AI

98 Mars explorer system Steel s system uses the subsumption architecture and obtains nearly optimal behaviour The objective is to explore a distant planet, and in particular, to collect sample of a precious rock. The location of the samples is not known in advance, but it is known that they tend to cluster If detect an obstacle then change direction If carrying samples and at the base then drop samples If carrying samples and not at the base then travel up gradient If detect a sample then pick sample up If true then move randomly

99 Hybrid architectures The compromised position, two subsystems: A deliberative one, containing a symbolic world model A reactive one which is capable of reacting without complex reasoning Often the reactive has priority Architectures tend to be layered with higher layers dealing with information at increasing levels of abstraction

100 Layers Perceptual input Layer n... Layer i... Layer 1 Action output Action output Layer n... Layer i... Layer 1 Layer n... Layer i... Layer 1 Perceptual input Perceptual input Action output Horizontal layering Vertical layering One pass control Horizontal layering Two pass control

101 TouringMachines Horizontal layering. Layers: reactive, planning, modelling Reactive: situation-action rules Rule-1: kerb-avoidance if is-in-front(kerb, Observer) and speed(observer) > 0 and separation(kerb, Observer) < KerbThreshold then change-orientation(kerbavoidanceangle) Planning: constructs plans and selects actions Modelling: symbolic representations of the other entities The three layers are embedded in a control framework: censor-rule-1: if entity(obstacle-6) in perception-buffer then remove-sensory-record(layer-r, entity(obstacle-6))

102 IIIA-CSIC Building agents

103 Building Agents 1. Based on agent infrastructure and components (e.g. Jade, COABS Grid) Infrastructure supports agent-aware services, but build intelligence yourself 2. Agent Oriented Programming Close to agent theory Academic (e.g. AOP, 3APL) Commercial (e.g. Agentis, JACK)

104 access mode pattern matching timeout-based blocking-based polling-based behaviour 1 private inbox of ACL messages behaviour 2 behaviour n scheduler of behaviours JADE agents life-cycle manager active agent behaviours (i.e. agent intentions) beliefs capabilities application dependent agent resources The JADE framework includes a library of interaction protocols and generic agent behaviours, that must be customized for the specific application needs in order to create the agent capabilities JADE library of interaction protocols and of generic agent behaviours

105 3APL agents

106 PROGRAM "patrol_agent.3apl" CAPABILITIES: { at_east(self) } WalkWest() { NOT at_east(self), at_west(self) }, { at_west(self) } WalkEast() { at_east(self), NOT at_west(self) } BELIEFBASE: at_west(self) GOALBASE: patrol() RULEBASE: patrol() at_east(self) WalkWest() ; patrol(), patrol() at_west(self) WalkEast() ; patrol(). 3APL agents

107 IIIA-CSIC Agents founded on the BDI (belied-desire-intention) model Agents employ BDI to reason effectively: Beliefs to capture knowledge about the environment in which they operate Desires to specify what it is that an agent is trying to achieve Intentions to capture decisions about how to behave now and in the future

108 IIIA-CSIC

109 IIIA-CSIC

110 Example: Chip manufacturing process A goal-directed agent dynamically Assembles the application at run-time

111 IIIA-CSIC G o a l - d i r e c t e d a g e n t s automatically handle failure recovery. When an unplanned error occurs, the agent selects another plan until it achieves its goal.

112 IIIA-CSIC A JACK agent is a software component that can exhibit reasoning behaviour under both pro-active (goal directed) and reactive (event driven) stimuli. Each agent has: a set of beliefs about the world (its data set); a set of events that it will respond to; a set of goals that it may desire to achieve (either at the request of an external agent, as a consequence of an event, or when one or more of its beliefs change); and a set of plans that describe how it can handle the goals or events that may arise.

113 IIIA-CSIC The JACK Agent Language allows programmers to develop the components necessary to define BDI agents and their behaviour: Agents - which have methods and data members just like objects, but also contain capabilities, beliefset relations they can use to store beliefs, descriptions of events that they can handle and plans that they can use to handle them. Capabilities - encapsulate and aggregate functional components of the JACK Agent Language (e.g. events, plans, beliefsets or other capabilities). Beliefsets - to store beliefs and data that the agent has acquired. Views - way of modelling any data in a way easily manipulated by JACK. Events - circumstances and messages that it can respond to. Plans - executed in response to events.

114 IIIA-CSIC JACK Agent Language extends the Java execution engine: Multi-threading is built into the kernel and removed from programmer control. Execution follows the model of agents processing a set of plans and having access to a number of beliefset relations. The agent executes these plans in tasks to handle events when they arise, consulting its beliefset relations where necessary. A new data structure called a logical member is introduced. The agent's beliefset can be queried using logical members by attempting to unify the logical member with a desired result.

115 IIIA-CSIC AGENT TEMPLATE agent AgentType extends Agent [implements InterfaceName] { // Knowledge bases used by the agent are declared here. #private data BeliefType belief_name(arg_list); // Events handled, posted and sent by the agent are declared here. #handles event EventType; #posts event EventType reference; #sends event EventType reference; // Plans used by the agent are declared here. // Order is important. #uses plan PlanType; // Capabilities that the agent has are declared here. #has capability CapabilityType reference; // other Data Member and Method definitions }

116 IIIA-CSIC In order for an event to be processed by an agent, the agent must determine which plan (if any) should handle the event. To decide which of the plans are applicable, Jack Identifies the plans which handle the event type. Eliminates plans on the basis of the data associated with the particular event instance. Generates plan instances which are consistent with the agent's current beliefs. Result: applicable plan set. JACK chooses either the first entry in the set, or one at random.

117 IIIA-CSIC Agents and ecommerce

118 Autonomy. Why Agents may help EC Agents work proactively, reactively and independently of human intervention. They can wait for good deals without diverting our attention. Personalisation. Agents can be equipped with a personal profile to reflect preferences. Social ability. The communication ability of agents can be used to negotiate over prices, services and transactions. Intelligence. Agents can learn and hence perform better over time. In EC scenarios this may equate to making more money. Many AI techniques can be applied.

119 Agents in ecommerce Product brokers: Jango PersonaLogic Firefly Merchant brokering Bargainfinder, Jango, Kasbah Buyer broker Eyes. Negotiation through trusted third parties Kasbah, Auctionbot, Fishmarket Basic limitation: Only price brokering. No product differentiation.

120 Two examples Jango & PersonaLogic

121 Personalized seller A personal seller is (usually) an animated character that uses conversational interactions to help a customer to use an electronic commerce site. It suggests products according to a user profile and to his/her preferences. It helps as an answer to a user demand or proactively. It can be adapted through learning.

122 Ad-hoc applications Contract allocation. MAGNET. MAS for ecommerce Power load management as a computational market. Control of shipment processes (EDI). Maquiladora. Generic applications Auctions Auctionbot Fishmarket Virtual markets Kasbah Bazar Metamall

123 IIIA-CSIC Game theory

124 What is game theory about A game is played whenever people interact. Game theory is about rational interaction. Example: strategic voting:

125 Where does it come from The theory of Games and Economic Behaviour, 1944, Von Neumann and Morgenstern: First approach: strategic or noncooperative, searching for optimal. Solved for two-player cases with opposite interests, strictly competitive or zero-sum games. (Chess, backgammon, ) Second approach: Coalitional or cooperative, they classified patterns of coalition formation. They thought them to be indeterminate. No optimal solution given. Nash proposed the notion of equilibrium for noncooperative games and offered arguments to determine solutions for the cooperative case. Game theory has concentrated mainly on the non-cooperative case.

126 The rules of the game tell who can do what and when. Also, they tell how much each participant gets at the end. The structure of a game is usually represented as a tree: the initial state is the root, a play is a path in the tree. Terminal nodes correspond to the outcomes of the game. Each arc is the what and each node has a label with a who. Some games have chance moves represented as a probability. Each leave contains how much players get. Extensive form:

127 IIIA-CSIC

128 -

129 Pure Strategy Pure strategies determine what actions player i should take at each node where i must move. For instance, the strategic form (rows are pure strategies for I and columns pure strategies for II).

130 Zermelo s algorithm Zermelo used this method to analyze chess back in The analysis started from the end toward the beginning of the game (method called backwards induction, and that is programmed via dynamic programming). The method consists on considering all subgames and labelling with W if there is a winning strategy for the subgame.

131 Some results Definition A game is said to be of perfect information if the history of the game is unambiguously known. Theorem (Zermelo) Let T be any set of outcomes in a finite two-player game of perfect information without chance moves. Then, either player I can force an outcome in T, or player II can force an outcome outside T. Definition An strictly competitive game is such that for each outcome a and b a < I b b > II a Corollary Any finite, strictly competitive game of perfect information without chance moves has a value. Corollary Chess has a value.

132 IIIA-CSIC Definition A strategy pair (s,t) is a saddle point of the strategic form of a strictly competitive game if, from the point of view of player I the outcome of (s,t) is no worse that any outcome in the column corresponding to t and no better than any outcome in row s. Corollary The strategic form of a finite, strictly competitive game of perfect information without chance moves has a saddle point (s,t) Theorem If the strategic form of game G has a saddle point (s,t) with outcome v, then G has value v.

133 Rational play What should a player do: chose strategies that guarantee an outcome not worse than v, the value of the game. That is, chose a pair (s,t) that is in Nash equilibrium, s is optimal for a player I who knows that player II chooses t and t is optimal for II knowing that I chooses s. Corollary In a strictly competitive game any saddle point is in Nash Equilibrium.

134 Subgame-perfect equilibrium Nash equilibrium pairs only guarantee rational behaviour at nodes on the equilibrium path, and may contain irrational moves elsewhere. Zermelos algorithm selects pairs (s,t) that are not only in Nash equilibrium in the whole game but that are in Nash equilibrium in every subgame. A pair of strategies with this property are said to be a subgame-perfect equilibrium. A player could however be unwise to play the strategy suggested by a game theory book labelled as optimal because it is usually optimal is everyone plans to play optimally.

135 Conflict and co-operation The theory of co-operation is far less developed. These games tend to be represented as outcomes labelled ( x, y) meaning that player I gets x and player II gets y. A team game:

136 Co-operation Game theory is also about co-operation. For instance a seller and a buyer trading a house. There are many prices at which the house can be sold. There is conflict and common interest. Social institutions have arisen to govern such transactions. Houses are sold by a two-step process: bargaining + binding contract.

137 Von Neumann and Morgenstern argued that the result should lie in the bargaining set: the set of all individually rational pareto-efficient pairs in the co-operative pay-off region X. A point x of X is pareto-efficient if for all y y>x y X The bargaining set An agreement is individually rational it it assigns each player a utility bigger than the disagreement point pay-off.

138 IIIA-CSIC

139 Nash Bargaining problem A Nash bargaining problem is a pair (X,d) B where X is a set of feasible payoff pairs and d X represents the disagreement point and such that: The set X is convex The set X is closed and bounded above Free disposal is allowed (burning money allowed)

140 A Nash Bargaining solution is F: B R 2 such that F(X,d) X. (F(X,d) is the payoff pair agreed upon by rational players.) Nash bargaining solutions The function G: B R 2 defined to be s in the figure (s = αr + βt) is called the generalised Nash Bargaining solution and α and β are called the bargaining power of the players. The regular Nash solution considers α = β = 1/2. G(X,d) = max x X (x 1 -d 1 ) α (x 2 -d 2 ) β

141 IIIA-CSIC A simple case: Complexity from simplicity

142 Insects, Social Insects, and Ants living insects (rough estimate) ~2% of all insects are social Social insects are: All ants All termites Some bees Some wasps 50% of all social insects are ants Avg weight of one ant between 1 and 5 mg Tot weight ants ~ Tot weight humans Ants have colonized Earth for 100 million years, Homo sapiens sapiens for 50,000 years

143 Ant Colony Societies Ant colony size: from as few as 30 to millions of workers Work division: Reproduction queen Defense soldiers Food collection specialized workers Brood care specialized workers Nest brooming specialized workers Nest building & maintenance specialized workers

144 How Do Social Insects Coordinate? Communication is necessary Two types of communication: Direct: antennation, trophallaxis (food or liquid exchange), mandibular contact, visual contact, chemical contact, etc. Indirect: two individuals interact indirectly when one of them modifies the environment and the other responds to the new environment at a later time This is called stigmergy

145 Stigmergy La coordination des taches, la regulation des constructions ne dependent pas directement des oeuvriers, mais des constructions elles-memes. L ouvrier ne dirige pas son travail, il est guidé par lui. C est à cette stimulation d un type particulier que nous donnons le nom du STIGMERGIE (stigma, piqure; ergon, travail, oeuvre = oeuvre stimulante). Grassé P. P., 1959 [ The coordination of tasks and the regulation of constructions does not depend directly on the workers, but on the constructions themselves. The worker does not direct his work, but is guided by it. It is to this special form of stimulation that we give the name STIGMERGY (stigma, sting; ergon, work, product of labour = stimulating product of labour). ]

146 Stimulation of workers by the performance they have achieved Grassé P. P., 1959 Stigmergy

147 Example of Stigmergy: Trail Following and Ants Foraging Behaviour While walking, ants and termites: May deposit a pheromone on the ground Follow with high probability pheromone trails they sense on the ground

148 Pheromone Trail Following Ants and termites follow pheromone trails Movie by Jean-Louis Deneubourg

149 Ants Foraging Behaviour Example: The Double Bridge Experiment Nest 15 cm A Foo d % of passages % Passages A % Passages B XY (Scatter) 3 XY (Scatter) 4 B Time (minutes) Simple bridge % of ant passages on the two branches Goss et al., 1989, Deneubourg et al., 1990

150 Double Bridge Experiment Goss et al., 1989, Deneubourg et al., 1990 Movie by Jean-Louis Deneubourg

151 Artificial Stigmergy Indirect communication mediated by modifications of environmental states which are only locally accessible by the communicating agents Dorigo & Di Caro, 1999 Characteristics of artificial stigmergy: Indirect communication Local accessibility

152 What Are Ant Algorithms? Ant algorithms are multi-agent systems that exploit artificial stigmergy as a means for coordinating artificial ants for the solution of computational problems Most research is about the study of how far we can push stigmergic communication as a means to design distributed algorithms that solve difficult computational problems

153 Real Ants Inspire Ant Algorithms: The Asymmetric Bridge Experiment Goss et al., 1989 Dorigo & Bertolissi, 1998

154 Some Results % of experiments r = % of traffic on the short branch r = 2 % of experiments r = % of traffic on the short branch % of experiments 50 r is the length ratio among the two bridges % of traffic on the short branch Short branch added later

155 Artificial Ants and the Shortest Path Problem Source Pheromone trail depositing Memory? Probabilistic rule to choose the path Destination

156 Using Pheromone and Memory to Choose the Next Node j τ ijd i Memory of visited nodes τ ikd k τ ird r

157 Ants Probabilistic Transition Rule τ ijd is the amount of pheromone trail on edge (i,j,d) J i k is the set of feasible nodes ant k positioned on node i can move to

158 Ants Pheromone Trail Depositing where the (i,j) s are the links visited by ant k, and where quality k is set proportional to the inverse of the time it took ant k to build the path from i to d via j j τ ijd Source i d Destination

159 Using Pheromones and Heuristic to Choose the Next Node τ ijd is a value stored in a pheromone table η ijd is an heuristic evaluation of link (i,j,d) which introduces problem specific information j k τ ijd ;η ijd τ ikd ;η ikd ant s destination = d r i τ ird ;η ird

160 The Simple Ant Colony Optimization Algorithm Ants are launched at regular instants from each node to randomly chosen destinations. Ants are routed probabilistically with a probability function of: (i) some artificial pheromone values, and (ii) some heuristic values, maintained on the nodes Ants memorize visited nodes and costs incurred Once reached their destination nodes, ants retrace their paths backwards, and update the routing tables The pheromone trail is the stigmergic variable

161 Three important components: TIME: a s h o r t e r p a t h r e c e i v e s pheromone quicker (this is often called: differential length effect ) QUALITY: a shorter path receives more pheromone. COMBINATORICS: a shorter path receives pheromone more frequently because it is likely to have a lower number of decision points. Why Does it Work? j d Destination τ ijd Source i

162 How well does it work? It works very well on shortest path problems with dynamic costs (e.g., routing in telecommunications networks) constrained shortest path problems (e.g., NP-hard problems)

163 The Routing Problem The practical goal of routing algorithms is to build routing tables Destination node Next node Routing is difficult because costs are dynamic Routing table of node k 1 i j i j k-1 i k-1 k+1 i k N i N Adaptive routing is difficult because changes in the control policy determine changes in the costs and vice versa

164 AntNet: Experimental setup Realistic simulator (though not industrial) Many topologies Many traffic patterns Comparison with many state-of-the-art algorithms (OSPF, SPF, Adaptive Bellman-Ford, Q-routing, PQ-routing) Performance measures: throughput (bit/sec) measures the quantity of service, and average packet delay (sec) measures the quality of service Japanese NTT net American NSF net

165 Throughput (b/s) Avg packet delay (s) AntNet: Some Results (1) NSF net NTT net From Di Caro and Dorigo, 1998, Journal of Artificial Intelligence Research Increasing UP traffic UP traffic increased by reducing the mean session inter arrival time

166 Ant System Applied to the TSP Pheromone trail depositing Memory Dorigo, Maniezzo, Colorni, 1991 Gambardella & Dorigo, 1995? Probabilistic rule to choose the path