WEB-BASED, DYNAMIC AND INTELLIGENT SIMULATION SYSTEMS

Transcription

1 Proceedings of the IASTED International Conference Intelligent Systems and Control 2000 August 14-16, 2000 Honolulu, Hawaii, USA WEB-BASED, DYNAMIC AND INTELLIGENT SIMULATION SYSTEMS T. PANAYIOTOPOULOS, S.VOSINAKIS, J. KALLIGATSIS, K. KABASSI University of Piraeus, Department of Informatics Knowledge Engineering Laboratory 80 Karaoli & Dimitriou str Piraeus, Greece (+301) ABSTRACT In this paper we present a method and an architecture for constructing intelligent simulation systems over the World Wide Web. In order to achieve this, different technologies such as Logic Programming, Object-Oriented Programming and Virtual Reality have been integrated. The proposed approach is illustrated by the presentation of a multi-agent VRML game where agents appear to possess dynamic as well as intelligent behavior. We are currently working on developing a generic tool for providing intelligence to Virtual Reality worlds. KEYWORDS Virtual Reality, Intelligent Systems, Modeling and Simulation, Web-Based Applications. 1. INTRODUCTION During the last few years VRML has accomplished extraordinary advances towards the field of three-dimensional visual representation [1]. It has motivated people to construct complex real or imaginary worlds where the user can virtually navigate and explore them. Empowered with Java, with the use of EAI (External Authoring Interface) [2], VRML has abandoned the static modeling style of its first version and is capable of complex object manipulation. Quite recently, attempts have been made to integrate Desktop Virtual Reality (Desktop VR) with educational software to produce a higher level of interaction and visualization. Researchers have tried to model concepts such as Virtual University [3], three dimensional representations of chemical structures [4], Virtual Laboratory [5,6] etc. In spite of all these, VRML can still not handle dynamic, non-deterministic behavior of virtual objects. On the other hand, Logic Programming (LP) has shown rapid progress in areas like Problem solving [7], proof systems [8,9], temporal reasoning and planning [10,11], etc. Recently many attempts have been made to putting toge ther technologies like Logic Programming, Object - Oriented Programming, as well as Virtual Reality [12, 13, 14]. Therefore, a need for a step further was emerging gradually and the last few years it became rather imperative: The need to integrate all these technologies. In this paper, we present an architecture, which can be used as a highly expressive tool for combining the forces of virtual reality and intelligent behavior. The proposed methodology can be applied to various kinds of applications. As a case study, we present a program that simulates the game of musical chairs. The paper is structured as follows: Section 2 discusses the issue of incorporating intelligence into VRML objects. Section 3 introduces the specifics of intelligent and dynamic behavior, and section 4 proposes an architecture for constructing such applications. Consequently, in section 5, we present implementation issues. In Section 6, a sample application developed with the proposed architecture is described in detail

2 Finally, we state conclusions, discuss problems and possible extensions of such kind of applications. 2. INTELLIGENT VRML OBJECTS Recently, there have been many attempts towards the visualization of Logic Programs, i.e. the perception of the output of a logic program in a visual way. Standard Logic Programming languages such as Prolog produce only text output. However, this was not enough, as Logic Programming applications were in need of a better output representation. Wouldn t one prefer to perceive an agent walking into a maze rather than reading the printout of each movement action? E.Denti, A. Natali and A.Omicini [15], propose a way to merge Logic Programming into Web based technology. In other words, they use Java as an effective GUI between the user and a logic program. Although this is worth mentioned effort, they mostly use Java for its network facilities, as a plain mean for producing an output, rather than for representing visually a process or a sequence of certain actions. The availability of logic programs on the Internet is the subject of a research done by S.Beltagy, M.Rafea and A.Rafea [16]. It appears that they follow an architecture that uses a Java front end for visualizing logic programs. Nevertheless, the visualization is still static in the sense that the output of a LP is produced in text. An interesting application of the current research towards visualizing logic programs would be the introduction of intelligent behavior of virtual objects, i.e. the control of the virtual objects actions using techniques from the field of Artificial Intelligence. Jeffrey Coble and Karan Harbinson, who present a Multi Agent architecture for Virtual Environments (MAVE) [17], propose such an approach. They describe an effective platform for groupware and coordination of information systems. Although they have managed to visualize complex processes, such as planning, they insist very strongly upon the implementation part, skipping the clarification of the methodology that led them to the specific result. Other researchers are dealing with the visualization of complex behavior of synthetic characters using methodologies based on Artificial Intelligence and Intelligent Agents theories. They focus on systems that implement real-time virtual humans [18] and their applications, as well as the representation of autonomous intelligent agents as avatars in virtual worlds [19]. We have also done some preliminary work on incorporating artificial intelligence techniques into VRML worlds. In [12], we presented an intelligent agent framework for visualizing simple but intelligent agents. However that approach was rather problem-oriented. More recently, [13], we have tried to develop a Virtual Reality Intelligent Agent Language. The proposed language however, has not been completed, as it must take care of many complex issues concerning VRML objects behavior. An attempt to describe Distributed Intelligent Virtual Agents architecture has been made in [14]. We are particularly interested in the field of VRML, where we propose an architecture of implementing systems that integrate intelligent behavior defined in terms of a declarative Logic based approach (e.g. using Prolog), dynamic behavior by means of an object oriented approach (e.g. Java) and visualization of the above behavior by the means of powerful browsers on expressive 3D environments (e.g. VRML worlds). The application fields of such a system are innumerable. Every VRML world that requires complex behavior from its objects can be enriched with the use of the proposed architecture. 3. MOTIVATION AND SPECIFICS OF INTELLIGENT AND DYNAMIC BEHAVIOR The majority of the VRML worlds published on the Internet is either static or includes predefined (or at least predictable ) objects behavior. Therefore they lack realism and fail to capture user s attention. An interesting VRML world would be one whose objects would be able to adjust their behavior to the alterations of the environment and decide for their actions according to a set of rules. Such a behavior is both dynamic in the sense that the sequence of an - 2 -

3 object s actions is not directly predictable, but depends on the environment, and intelligent because objects follow a certain strategy for choosing their actions. In order to find a satisfactory solution for this problem, a number of protocols that enable the manipulation of VRML worlds by external programming languages have been introduced. One of the most promising ones is the External Authoring Interface that allows a Java Applet to control VRML Nodes and alter their attributes. As a result, one can create composite animations, which are sequences or partial orders of primitive VRML animations, and generate dynamic objects behavior. We call VRML handled animations, primitive animations, not because they are primitive but because they are considered as basic elements of more complex animations of our approach. Primitive animations normally concern the individual transformation of parts of complex objects. Examples of primitive animations are move hand, blink, move foot, etc. Composite animations, on the other hand, concern the synchronized animations of all the parts of complex objects resulting in a realistic motion. For example push, sit down, stand up, etc. However, composite animations happen to be primitive actions of artificial intelligence planning systems. Such systems tend to define primitive and composite actions (which are defined in terms of primitive actions) and, using a planner, they generate a sequence (or partial order) of actions for changing the world in order to achieve some predetermined goals. Intelligent and dynamic behavior must first be defined at an abstract level, than it must become more specific at a lower level and finally it must be visualized. A planner could handle all decisions by using its internal world representation and a set of rules that constitute the core of the reasoning mechanism. Such a system defines composite and primitive actions and generates a sequence of such actions to be performed. Each primitive action is received by an intermediate unit, which produces one (but sometimes a few) composite animation. These animations are finally visualized as a synchronized set of primitive animations. Say for example that the planner has decided that a character must move to the next room. This means that it must stand up, walk through the room without bouncing on objects, open a door, and walk towards a chair in the middle of the next room. The intermediate unit receives the decided actions and produces the appropriate composite animations. The visualization unit generates these animations at the lowest level as primitive animations. 4. ABSTRACTION - SPECIFICATION - VISUALIZATION The proposed architecture consists of three different parts: the Logic Unit, the Interface Unit and the Visual Representation Unit (fig. 1). The logic unit handles all possible decisions and actions in an abstract way. There is a solver, i.e. an engine that produces the result of a problem and a knowledge base (KB) that has all the rules and facts needed to describe the specific problem. The KB contains two kinds of information: the domain knowledge, that describes the world and its objects, and the strategic knowledge, that describes the set of rules the solver has to follow in order to decide for an action. Respectively, the Visual Representation Unit contains a Virtual Reality Management Unit, which is a 3D Engine that displays and manipulates the virtual objects that take part in the problem. It can handle a set of instructions that determine the geometric transformations on the virtual objects and its output is based on a 3D-Models database, where the geometric features of the objects are stored. Some of these models are not affected by any possible action (e.g. a wall), while others can change their geometry and/or material attributes in real time (e.g. a human). Therefore 3D Models are categorized into static and dynamic ones respectively. The communication between the Logic Unit and the Visual Representation Unit is accomplished through the Interface Unit. It instantiates the objects that were defined in the Logic Program using its own objects base, receives an abstract notion of the action sequence and transforms it into specific instructions. The Interface Unit consists of three parts: the Communication Unit, which is responsible for the interaction with the Logic Unit, the 3D Interaction Unit that handles - 3 -

4 the geometric transformations and the Object Instantiation and Action Management Unit (OIAMU), which coordinates the above units as well as transforms the abstract orders produced by the Logic Unit into specific 3D objects animation. Logic Unit Solver Strategic Knowledge Knowledge Base Domain Knowledge response The Server Side Communication Unit interacts with the Solver, transmitting requests and receiving notification. The latter is transmitted back to the Client Side Communication Unit. The 3D Interaction Unit is also divided into two parts. The first part is the 3D interface that enables the communication of the Visual Representation Unit with the Interface Unit. Above the 3D interface, stands a more abstract encapsulation of it, called the Node Interface. The node interface enables the OIAMU to focus on movement and animation problems, rather than the Visual Representation Unit s implementation issues. Interface Unit Object Instantiation & Action Management Unit Communication Unit 3D Intercation Unit Objects Base Finally, the OIAMU carries out the basic translation of the Logic Unit s orders to a visually perceived presentation. To achieve this, it contains theme specific object paradigms (Objects Base). These paradigms analyze an abstract movement order (produced by the Logic Unit and received via the communication unit) to elementary animations. Definitions of these animations are kept in a queue structure inside the paradigm, and are activated one at a time. Each of these animations is correlated with a Stop Event structure, discussed later on. Visual Representation Unit Virtual Reality Management Unit Static Models 3D Models Base Dynamic Models The main OIAMU engine consists of a timer and a chronically sorted queue of Stop Event structures. This is a global list (i.e. contains Stop Event structures produced by all active objects). A Stop Event structure is registered upon activation of animation by an object. Among other information, it contains the timestamp of the relevant animations termination time. When the clock reaches the time contained within the first Stop Event in the list, the corresponding object is notified. Figure 1: The proposed architecture The communication unit serves two purposes: it interacts with the logic unit and implements the optional but suitable client-server architecture discussed later on. This architecture places the logic unit on a server, while the visual representation unit resides on one or more clients. The communication unit is divided into two parts, which communicate via a protocol built especially for this cause. The Client Side Communication Unit receives requests from the OIAMU and forwards them to the Server Side Communication Unit. Upon reception of the The object s response depends on whether it has some more animation registered in its queue, or not. In the first case, it activates the next animation. Otherwise, it informs the OIAMU engine that it needs further instructions (from the Logic Unit) before proceeding. The OIAMU engine has the ability to generate global events that are not produced by objects, but by general rules of the application

5 5. IMPLEMENTATION One possible way of implementing our proposed architecture is based on a 2 tier client/server motif. Prolog/Java are on the server side while Java/VRML on the client. The communication between the client and the server takes place by using stream sockets. Needless to say, the Java client part should be an applet in order to be able to connect to the VRML world. On the contrary, the server part should be an application because the Java - VRML interface (Jasper) only functions with Java applications. Thus, parts of the reasons for choosing a client/server approach were technical. The logic unit corresponds to a compiled prolog program that any player can call in order to decide what to do next. In this implementation, the logic is common for all the players, but their physical abilities (e.g. speed) vary. The interface unit uses two special classes: Chair and Player. These define the behavior of the corresponding objects by providing a set of methods that accomplish specific complex animations. Being quite obvious that the logic part is implemented in Prolog, the interface in Java, and the visual representation unit in VRML, one has to model the conceptual level using logical terms, the object instantiation level using an object-oriented approach and the visualization level using VRML language or other modeling packages able to export VRML files. The system s units cooperate through a sequence of messages traveling through different units and different implementation programming languages. This communication scheme is required for the autonomy of each unit and enhances the system s flexibility, in a way that it is easy to regenerate the application to perform a different scenario. 6. A SIMPLE EXAMPLE: MUSICAL CHAIRS The example presented here is the known game of musical chairs. This game is played in turns, during which several players can take part. N players stand in a line. When the music begins they run around a rectangle formed by n-1 chairs. When the music stops, each player races to his/her nearest chair. At least two players will be heading to the same chair. One of them arrives there first, so the other looks for another vacant chair. The player who remains standing is disqualified. Before the next turn begins, the player removes a chair. The last player to stay in game is the winner. In our example we created a 3D simulation of musical chairs using a set of human models to represent the players. Picture 1: A sample screenshot The main engine is based on three loops that terminate on occurrence of special events. These loops appear in three similar methods. The first is terminated when there are no more actions, the second is terminated when a predefined generic Stop Event has occurred (e.g. the music has stopped), and the third one is terminated when all but one player have been seated down. A very large VRML file contains the static world s data, the players and chairs data and the required TimeSensors, Interpolators and routes, for the animations. For each complex animation a set of interpolators is connected to one TimeSensor. 7. CONCLUSIONS AND FUTURE WORK In this paper we discussed about the need to combine artificial intelligence with 3D visual representation, by defining the behavior of virtual objects with the use of logic programming, and the possible benefits of such an approach. We proposed an architecture in order to create dynamic and intelligent behavior - 5 -

6 in a virtual world and described a possible implementation. The proposed methodology and implementation provides a simple yet efficient way for embedding intelligent behavior in VRML worlds. Unfortunately, desktop VRML s performance does not yet allow the creation of very complex 3D worlds. Since our architecture and methodology is proposed for visualizing actions, we could apply those to intelligent Agent architectures [20, 21]. A similar work has already been done [12], but the methodology had not been stressed well enough. Visualizing an agent s actions is a very important issue as it raises matters concerning a more natural human computer interaction. We could also expand that thought and include as well, the communication of intelligent agents and the visual representation of it. We are also researching the possibility of extending our architecture to introduce prolog-driven Intelligent Virtual Agents in large virtual environments [13]. We plan to develop a generic tool for the description and creation of such systems in all levels of abstraction. Interactive stories are also subjects of great interest [22]. Our methodology could apply to the formation of such stories and possibly will have to take in mind the user s participation. Our current work does not include continuous feedback from the user, but it could be easily modified to do so. ACKNOWLEDGEMENT This work has been supported by the Greek Secretariat of Research and Technology under the PENED '99 project entitled: Executable Intensional Languages and Intelligent Multimedia, Hypermedia and Virtual Reality Applications", Contract No. 99ED265. REFERENCES [1] ISO/IEC :1997, VRML97 International Standard. The VRML Consortium, [2] Marrin, C., External Authoring Interface Reference. Silicon Graphics Inc., [3] Panayiotopoulos, T., Zacharis, N., Vosinakis, S., Intelligent Guidance in a Virtual University. IMACS International Symposium on Soft Computing in Engineering Applications, SOFTCOM 98, Athens, June 1998, appears also in Advances in Intelligent Systems : Concepts, Tools and Applications, (S. Tzafestas ed.), Chapter 10, pp , Kluwer Academic Publishers, Netherlands, [4] Krieger, J., Doing Chemistry in a Virtual World. Chemical & Engineering News, December 9, 1996, [5] Panayiotopoulos, T., Vosinakis, S., Katsirelos, G., Avradinis, N., Interactive experimentation in Virtual Laboratories. CAL99 International Conference, Virtuality in Educat ion, London, UK, March [6] Dede C., Salzman, M., Loftin, B., Ash, K., Using virtual reality technology to convey abstract scientific concepts. In M. Jacobson & R. Kozma (Eds.), Learning the sciences of the 21st century: Research, design and implementation of advanced technological learning environments. Lawrence Erlbaum, Hillsdale, NJ, [7] Breuker, J., Problems in Indexing Problem Solving Methods. Workshop on Problem Solving Methods for Knowledge Based systems in Connection with the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI-97) Nagoya, Japan, August 23-25, [8] Sofronie-Stokkermans, V., Representation Theorems and Automated Theorem Proving in Non-Classical Logics. Proceedings of ISMVL-99, [9] Felty, A., Thory, L., Interactive Theorem Proving with Temporal Logic. Journal of Symbolic Computation, April [10] Panayiotopoulos, T., Gergatsoulis, M., Intelligent Information Processing using TRLi. 6th International Conference andworkshop on Database and Expert Systems Applications, DEXA'95, London, U.K., September, 4-8, 1995, appears in Proceedings, N.Revell, A. M. Tjoa (Eds.), 1995, pp [11] Marinagi, C.C., Panayiotopoulos, T., Vouros, G.A. Spyropoulos, C..D., Advisor : A knowledge-based planning system. International Journal of Expert Systems, Research and Applications, 9(3), 1996, pp [12] Panayiotopoulos, T., Katsirelos, G., Vosinakis, S., Kousidou, S., An Intelligent - 6 -

7 Agent Framework in VRML worlds. Third European Robotics, Intelligent Systems & Control Conference, EURISCON 98, Athens, June 1998, also in Advances in Intelligent Systems : Concepts, Tools and Applications, (S. Tzafestas ed.), Chapter 3, pp.33-43, Kluwer Academic Publishers, Netherlands, [13] Panayiotopoulos, T., Anastassakis, G., Towards a Virtual Reality Intelligent Agent Language. 7th Hellenic Conference on Informatics, 1999, Ioannina, also in Advances in Informatics, D. Fotiadis and S.D. Nikolopoulos (Eds), World Scientific, [14] Vosinakis, S., Anas tassakis, G., Panayiotopoulos, T., DIVA: Distributed Intelligent Virtual Agents, Workshop on Intelligent Virtual Agents, Virtual Agents 99, The Centre for Virtual Environments, University of Salford, pp , Issn , Salford, U.K., September 13, [15] Denti, E., Natali, A., Omicini., A., Merging Logic Programming into Web-based technology : a Coordination-based approach. Proceedings of 2nd International Workshop on Logic Programming Tools for Internet Applications, Leuven, Belgium, July [16] El-Beltagy, S.R., Rafea, M., and Rafea, A., Practical Development of Internet Prolog Applications using a Java Front End. Proceedings of 2nd International Workshop on Logic Programming Tools for Internet Applications, Leuven, Belgium, July [17] Coble, J., Harbison, K., MAVE : A Multi agent Architecture for Virtual Environments. 11th International Conference on Industrial & Engineering Applications of Artificial Intelligence and Expert Systems. [18] Badler, N., Real Time Virtual Humans. Proceedings of the Pacific Graphics 1997: The Fifth Pacific Conference on Computer Graphics and Applications, Los Alamitos, CA, IEEE Computer Society Press,1997. [19] Elliot, C., Brzezinski, J., Autonomous Agents as Synthetic Characters, AI Magazine, Summer 1998, AAAI Press, pp [20] Wooldridge, M., Muller, J., Tambe, M., editors. Intelligent Agents II, Agent Theories, Architectures and Languages. Volume 1037 of Lecture Notes in Artificial Intelligence, Springer-Verlag, [21] Muller, J., The Design of Intelligent Agents, A Layered Approach. Volume 1177 of Lecture Notes in Artificial Intelligence, Springer-Verlag, [22] Sgouros N. M., Tsanakas P., Papakonstantinou G., A Framework for Plot Control in Interactive Story Systems. Proceedings of the 13th National Conference on Artificial Intelligence (AAAI-96), Portland OR, USA, AAAI/MIT Press, 1996, pp