UTSAF: A Simulation Bridge between OneSAF and the Unreal Game Engine

Transcription

1 UTSAF: A Simulation Bridge between OneSAF and the Unreal Game Engine Phongsak Prasithsangaree, Joseph M. Manojlovich, Jinlin Chen, Michael Lewis School of Information Sciences University of Pittsburgh, Pittsburgh, PA, USA. phongsak,josephm,jlchen,ml@sis.pitt.edu Abstract Rapid advances in consumer electronics have led to the anomaly that consumer off the shelf (COTS) gaming hardware and software now provide better interactive graphics than military and other specialized systems costing orders of magnitude more. UTSAF is bridging software written to take advantage of the power of gaming systems by allowing them to participate in distributed simulations with military simulators such as OneSAF which use the DIS protocol. UTSAF parses DIS PDUs and uses the information gained to control entities within the Unreal game engine using the Game- Bots modification. This paper describes the advantages of game engine based simulation and the UTSAF bridging and control architecture. Our main contribution is to build a simulation bridge that enables affordable highquality 3-D viewers for military simulations. Keywords: Military Simulation, Game Simulation, Simulation Bridging, Software Agent, ModSAF, One- SAF, OTB, Unreal Tournament, Gamebot. 1 Introduction Military simulations have relied on expensive hardware and specialized software for many years. Conducting research related to military-simulated applications such as team-based operations has historically been difficult and expensive. Supercomputer level graphics have, however, become commonplace on commodity priced Personal Computer (PC) graphics processing units (GPUs) in the past two years. The current record holder Nvidia s GeForceFX TM GPU delivers more than 200 million triangles per second at a fill rate of around four billion texture mapped pixels per second making it the world s fastest GPU at least for the next few months. This speed comes despite a raft of hardware operations including multiple shadings per pixel, multi-texturing, texture and lighting, bump mapping and other compute intensive operations not found until recently in animated PC graphics. Accessing these hardware capabilities, however, requires complex 3-D graphics code /03/$17.00 c 2003 IEEE. that must be constantly revised to take advantage of new options for acceleration. By piggy-backing on game engines whose market demands up to the minute support for new options, military simulations can incorporate technological advances in lockstep with the commercial market without bearing development expenses their low volume could not support. Another advantage of game engines comes from the wealth of already developed models that can be incorporated into military simulations. Game engine models of human skeletal dynamics, Newtonian physics, trauma, and explosion are particularly well developed due to the character of first-person shooter games. Their availability and efficiency within a single simulation environment eliminates the need to tradeoff figure animation for terrain rendering or vehicle dynamics in developing complex interactive simulations. A third advantage arises from the architecture of multiplayer games that allows networked PC s to generate independent views of the same scene. Using commodity priced PC s and software to dictate players viewpoints [6] these views can be stitched together driving inexpensive data projectors to produce special purpose displays which simulate high resolution powerwalls for situation rooms, cockpit canopies for aircraft, or Cavelike panoramic displays for infantry. This confluence of technologies make game engines an ideal way to produce inexpensive, high quality simulations for research and training. The missing key, however, is interoperation with planned and existing military simulations. In our work, we are not trying to advance the stateof-the-art military simulation but rather to bridge existing technology to produce a research platform providing high quality military simulations at an affordable level. In this paper, we describe military simulations and some other efforts to build distributed simulations in Section 2. In Section 3, we discuss our effort to bridge distributed military simulation to a multiplayer game simulation. In Section 4, we discuss an implemented architecture that bridges the military simulation to the game simulation called UTSAF. Section 5 explains and

2 shows snapshots of our experiments in the UTSAF testbed. Finally, we conclude our work and discuss our future work in Section 6. 2 BackGround Modern networked military simulation began with the SIMulator NETworking (SIMNET) project in 1984 [8]. SIMNET tank simulators portrayed a virtual battlefield on computer screens to their users so that drivers could maneuver their vehicles through a computer-generated terrain to engage other forces within the simulation. SIMNET introduced the notion of protocol data units or PDU s to describe typed data structures to be exchanged only as needed. Building on SIMNET protocols and practice a family of Distributed Interactive Simulation (DIS) protocols were published by the IEEE in 1995 [4] and became the basis for a current generation of networked simulations, most notably the widely used Modular Semi-Automated Forces (ModSAF) [1] simulation and its variants. While ModSAF and its descendents such as OneSAF Test Bed (OTB) now run on inexpensive PC platforms, the vehicle and dismounted infantry simulators they serve remain very expensive and usually dedicated to training. Making networked military simulations more generally accessible requires substituting general purpose computing equipment for these specialized simulators. Mike Zyda and his students at the Naval Postgraduate School [7] have worked on this problem in various forms since their first NPSstealth application allowed Silicon Graphics workstations to participate in SIMNET tank battles. Our effort differs in that we are not trying to advance the state of the art of distributed interactive simulation but rather to connect existing technologies in order to produce a research tool providing high quality shared military environments using ModSAF as a server. 3 Bridging Distributed Simulation Distributed simulation has been widely used in military applications such as battlefield training. It has the advantage that it is able to aggregate several training sites into a single training network. Groups of trainees can then perform complementary tasks in a training scenario which resembles a battlefield engagement in the diversity of personnel and complexity of their interactions. SIMNET, ModSAF, and OTB are examples of such distributed simulations. Two kinds of nodes are defined in a distributed simulation, server nodes and viewer nodes. The server node resides on the simulation engine where simulated information is generated and processed. The viewer node resides on a graphic user interface (GUI) to convey information to users. Currently, the GUI of the OTB simulation only shows a two-dimensional view. There is also a three-dimensional (3D) viewer for OTB but it requires a high performance workstation and expensive software. The main contribution of our work is to build a simulation bridge that enables affordable high-quality threedimensional viewer nodes for military simulations. We use the 3D computer game, Unreal Tournament (UT) [2] as a visualization tool to view OTB simulation environments. To bridge the OTB simulation to UT game environments, the basic problems are (i) matching terrains, (ii) matching entities, and (iii) synchronizing events between the visible simulation and the OTB server node as well as (iv) a problem of creating an interface to the UT game engine. 3.1 Matching Terrain The OTB, the most current version of the Mod- SAF platform, continues to use the CTDB (compact terrain database) format. We subject this data to a chain of translations through different formats (CTDB OpenFlight 3DS UNR) to produce a corresponding terrain in UNR (UT terrain format). First the CTDB is converted into the OpenFlight TM format using the Terra Vista TM DART program. OpenFlight is a format associated with a proprietary model development environment. Then, the OpenFlight format is converted to 3DS (3D Studio) format using NuGraf TM software. 3DS is a popular proprietary 3D file format used by 3D Studio Max software. Then, the 3DS format is converted into UNR format using the open source 3ds2unr utility. Because of restrictions on UT map level size, CTDB maps typically must be broken up into grids for this translation. While we experienced difficulties with BSP (binary space partitioning) holes in translations for the UT-2 engine, translations for the new UT-2003 engine have gone smoothly. 3.2 Matching Entities An entity in UT is composed of two main parts: a 3D model (or mesh, in UT terms), and a texture that covers the model as skin. There may also be sounds associated with the model, such as jet engines or rocket sounds. Most of the entity components in UTSAF were taken from existing public UT packages. Some were created manually using such 3D editing software as 3D Studio Max or MilkShape3D. These entity components are placed into a package source directory, compiled by UT, and then can be used by the UTSAF system. UTSAF has three main classes of entities that must be modeled: fixed-wing aircraft, missiles, and ground vehicles. Each has its associated creation difficulties: fixed-wing aircraft and missiles have no moving parts to model, yet have complex 3D designs, while ground vehicles typically have moving parts such as wheels, tracks, turrets, etc.

3 Meshes for displayable entities and code to associate them with types read from Entity Creation PDUs must be prepared ahead of time. UTSAF must be started before units are added to OTB. As entities are added to OTB they are created at corresponding locations within the UT game. 3.3 Event Synchronization Events in distributed simulation are synchronized by the use of DIS protocol. The DIS protocol runs on TCP/IP networks. It conveys information on events in the form of PDUs which are broadcasted over the network. The DIS protocol works in stateless manner, which means information about all entities and events is flooded onto the network increasing the network traffic. Therefore, dead reckoning algorithms are introduced in the DIS protocol. A dead reckoning algorithm is specified for each entity. An entity then generates an entity state PDU only when its state differs from that predicted by dead reckoning by some delta. To convey action smoothly these dead reckoning algorithms were accurately duplicated within the UT game engine. Using the DIS protocol to communicate between the OTB and the UT game engine, a protocol conversion is necessary. Due to the DIS stateless nature, the PDUs use a timestamp to implicitly show the order of information. However, in a congested network, PDUs received at a simulation node can be out of order; therefore, we need to add packet management to correctly transfer information from OTB to the UT game engine. We use middleware agents composed using the RETSINA [9] multiagent architecture to bridge communication from OTB to UT game engine. Using the agents to bridge simulations, we are able to control the DIS traffic and to convert information within PDUs into appropriate format for the UT game engine. 4 UTSAF Architecture Unreal Tournament Semi-Automated Force (UTSAF) is a bridging software written to take advantage of the power of gaming systems by allowing them to participate in distributed simulations with military simulators such as OneSAF/OTB which use the DIS protocol. UT- SAF is written to connect the UT 2003 game engine to the OTB simulation server. UTSAF listens to broadcasts from OTB servers and parses the PDUs it detects. It then uses the information from the PDUs it has read to update the state of the game. Communication is currently one-way from the server to the game engine. The operator observing the game must interact (in our case indirectly) through OTB to perform actions such as to control remote vehicles. In order to bridge a world of OTB simulation to a world of game simulation, we create the UTSAF software which is built on a multiple agent system (MAS) architecture [10]. In the MAS infrastructure, different types of agents perform different tasks. The agents do not have overall knowledge and do need to collaborate with other agents to obtain specific knowledge. This gives advantages of scalability, diversity and extensibility. Figure 1 diagrams the UTSAF architecture. It consists of SAF broker agents, PDU parser, a SAF manager agent, and Gamebot client agents. 3.4 Interface to the UT Game Engine The UT simulation supports networked game environments that involve multiple players in playing a game. However, the communication protocol between UT game engines is proprietary. Because we have based our development on the inexpensive (<$50) consumer game rather than licensing the game engine itself (>$250,000), we do not have access to the networking protocols used by the game. Therefore, we must control entities within the simulation in a different way. We have chosen to use GameBots [5], a modification to the game that allows entities to be created and controlled through a standard TCP/IP socket. By using Gamebots, we are able to create multiple agents inside the UT game and communicate and control the agents to reflect behaviors of entities in OTB simulation. Figure 1: UTSAF System Architecture 4.1 SAF Broker Agents and PDU Parser The SAF broker is an agent responsible for filtering and obtaining necessary information from OTB simulations. The SAF broker listens to a broadcast channel on an OTB simulation network to capture PDUs. Each SAF broker is associated with one OTB server. This enables our framework to support simulation in multiple domains. By using the PDU parser, the SAF broker gets partial information about the PDUs and processes only PDUs that are relevant to the game simulation. The PDU parser is responsible for getting data from different DIS PDUs. The SAF broker is also used for

4 filtering redundant PDUs to reduce traffic load to the game engine. After getting necessary information, the SAF broker passes the information to the SAF manager for further processing. 4.2 SAF Manager Agents The SAF manager is responsible for coordinating all information from all SAF brokers. It is responsible for creating and updating a database for entity and event information. When new updates arrive, the SAF manager conveys the updates to Gamebot agents relevant to those updates. The communication between agents is the message-based KQML agent language [3]. Figure 2 shows the UTSAF control GUI which is used to control traffic, to monitor particular entities, and to show current entity databases. Figure 2: SAF Manager GUI 4.3 Gamebot Agents To overcome the high cost of licensing the UT proprietary communication protocol, we use Gamebots as a communication interface to the UT game engine. Gamebots are controlled via a TCP/IP socket which is connected to a Gamebot agent. The Gamebot agent receives information relevant to its entity and creates a message-based command to control the Gamebot. Each Gamebot agent is used to control one UT server. Figure 3 shows the GUI for showing and controlling traffic from Gamebot agents to Gamebots. Figure 3: Gamebot agents GUI 5 Experimental Testbed For experiments, we set up our test bed in a local area network. We run a OTB simulation engine on a Pentium III Linux-based machine. The DIS protocol traffic from this machine is broadcast to a multicast group which includes a machine where RETSINA agents reside. Figure 4 shows the user interface of an OTB simulation using the Fort Knox terrain. In this scenario, two groups of A-10 Thunderbolts are flying over a biochemical depot which is surrounded by several T-80 tanks. Figure 5 shows the biochemical depot and T-80 tanks in OTB simulation. The same simulation scenario in 3D visualization using UT simulation engine is shown in Figure 6 where the biochemical depot and T-80 tanks are depicted. In the 3D visualization, we are able to change a view of a simulated battlefield using our agent. Figure 7 shows another view of the simulation, which several A-10 Thunderbolts are flying over the biochemical depot and T-80 tanks. By using the SAF broker, we are able to reduce network traffic from the OTB simulation network. The agent examines the entity information received by DIS PDUs, and simply ignores the PDU if the entity information is not changed. When running, the UTSAF system will display in 3D a running OTB simulation. The movement and appearance of the OneSAF entities are reproduced faithfully in real time. The user is able not only to spectate the simulation from any position in the game space, but also to connect their view to any of the entities in the simulation. 6 Conclusion and Future Work UTSAF was created to provide an inexpensive easily controllable stealth viewer for use in human experiments

5 Figure 4: ModSAF Figure 6: A Biochemical depot in Unreal Tournament Figure 5: A Biochemical depot in OneSAF Figure 7: A-10 Thunderbolts flying over a T-80 tank platoon

6 involving information fusion and control of remote vehicles. By providing the ability to inexpensively combine ground views on a panoramic display with simulated weapon video feeds and ground tracks on a targeting laptop it will allow us to investigate realistically complex tasks such as forward air control with teams of unmanned aerial vehicles (UAVs). This paper outlines our motivations (expense and capability) and approach for using game engines to solve more general simulation problems. There remains much work to be done before UTSAF is a fully viable system. Most of the problems that the system currently faces stem from the Unreal Tournament video game and not OneSAF or the agent system. Unreal Tournament currently has limits on both the possible map size and the number of entities allowed in a given simulation. To simulate very large OTB areas in Unreal Tournament, beyond the allowed map size in the game, we are currently investigating ways of dividing up the map into subunits. Each sub-map can be run on an individual game server, and then linked together to create a seamless representation of the overall map. Unreal Tournament itself does not support this feature, however, and a solution will have to be found outside of the video game itself. Gamebots currently has a preset limit of 16 entities that it will allow. This was easily extended to 64 by simply resetting this limit. Higher limits may be possible, and experimentation to determine the upper limit actually allowed by the game is being conducted. A major problem that exists in UTSAF now is latency. The time from when a PDU is emitted by Mod- SAF or OneSAF and when the contained information enters Unreal Tournament is both significant (on the order of 1 or 2 seconds) and variable. It is felt that this is probably related to both the agent system, and overall network latency. One final issue with UTSAF is user disorientation. A ModSAF or OneSAF map has both grid lines and a compass to orient users. UTSAF currently lacks both. Active research is being conducted into using the heads-up display provided by the game to provide these features. [1] A. Ceranowicz, Modular semi-automated forces, in Proceedings of the 26th conference on Winter simulation, 1994, Orlando, Florida, United States, pp [2] Epic Games Inc., Unreal Tournament 2003 Game, [3] T. Finin, Y. Labrou, and J. Mayfield, KQML as an agent communication language, invited chapter in Jeff Bradshaw (Ed.), Software Agents, MIT Press, Cambridge, [4] Institute of Electrical and Electronics Engineers, International Standard, ANSI/IEEE Std , Standard for Information Technology, Protocols for Distributed Interactive Simulation, March [5] G. Kaminka, M. Veloso, S. Schaffer, C. Sollitto, R. Adobbati, A. Marshall, A. Scholer, and S. Tejada, GameBots: A flexible test bed for multiagent team research, Communications of the Association for Computing Machinery (CACM), Vol. 45, No. 1, pp , January [6] M. Lewis, & J. Jacobson, Game Engines in Research, Communications of the Association for Computing Machinery (CACM), Vol. 45, No.1, pp , January [7] M. Macedonia, J. Zyda, D. Pratt, P. Barham, S. Zeswitz, NPSNET : A network software architecture for large scale virtual environments, Presence, Vol. 3, No. 4, pp , Fall [8] A. Pope, The SIMNET Network and Protocols, BBN Report No. 7102, BBN Systems and Technologies, Cambridge, MA, July [9] M. Rectenwald, RETSINA AFC Developers Guide, ( softagents/afc/) [10] K.P. Sycara, Multiagent Systems, AI Magazine, Vol. 10, No. 2, 1998, pp Acknowledgments This project is supported by AFOSR contract F The authors would like to thank Martin van Velsen, and the entire CMU RETSINA Software Agent team. The authors also want to thank the UTSAF team, Steve Hughes, Jijun Wang, Jeff Jacobson, Jeff Gennari and Yang Xu. References