Inventory of a Potential Mobile Augmented Reality Genre for Learning

Transcription

1 Inventory of a Potential Mobile Augmented Reality Genre for Learning Gunnar Liestøl Dept. of Media & Communication, University of Oslo Key words: Mobile Augmented Reality, Situated simulations, sitsim, situated learning Abstract: This paper describes experiments with a potential mobile augmented reality genre, named situated simulations. Four prototypes and their key features and functionalities (interface design; linking, viewing and display conventions; user generated content; network access; and dynamic sequences of actions and events) are presented and discussed as they have evolved over several years in the course of the design and development process. In summary, these constitute a preliminary inventory of a possible/potential digital genre for learning. The functions are also related to user feedback from real environment testing and evaluation from BA students. Finally, design features for further research and development are suggested. 1 Introduction Augmented reality and situated simulations A Situated Simulation (sitsim) is a subcategory, or subgenre, of nonhybrid mobile augmented reality (MAR) systems based on smartphones. In a sitsim the screen does not display a combination of live video information (of the real) and a 3D graphics layer representing the virtual environment, as in most mixed reality systems (MR) [1, 2, 3]. In a situated simulation there is approximate identity between a) the user s visual perspective and perception of the real physical environment and b) the user s visual perspective of the 3D graphics environment as it is represented on the 'clean' screen. This relative congruity between the real and the virtual is obtained by allowing the camera position and movement in the 3D environment to be determined by the GPS-coordinates (positioning), the accelerometer (vertical movement) and magnetometer/electronic compass (horisontal movement). As the user changes position and moves the phone in real space, the perspective inside the virtual space (on the screen) changes accordingly and with a dynamic refresh and frame rate high enough to achieve continuous movement [4, 5]. This paper reports on a series of sitsims that have been developed and tested in real user environments over the last couple of years (from October 2008 to May 2010). The purpose here is to present an inventory of the features that so far constitute the status of situated simulations as a prototyped mobile augmented reality genre. All the sitsims discussed here have historical topics. ICL 2010 Proceedings Page 1008

2 2 Early installments some basic features (Oseberg and Dolores) The first two sitsim prototypes (2008-9), The Oseberg Viking Ship and Mission Dolores, [4, 6] established and employed some basic features: Matching real and virtual. Because of the sitsim's nonhybrid, clean screen solution there is no simultanous graphic match between the real and the virtual on the screen. The real stays real outside the screen and the virtual environment displayed there. Thus a temporal link of continuity needs to be established, relating the real and the virtual. This is is done by introducing a transition mode, a simplified, 'naive' version of the real on the screen which can be easily identified by the user as a representation/simulation. When the relationship is established the user is ready for the full resolution a graphic reconstruction of an historic past object, in this case The Oseberg Viking ship or the Mission Dolores (see Figure 1) Figure 1. Real > naive > virtual. The low resolution, naive versjon of the present environment serves as an intermediary stage an exordium mediating between the immediate outside of the screen and the next mode of the 3D environment, the display of the full resolution past object and its environment. Static 3D objects and environments. A virtual environment that cannot be manipulated by the user, that is it cannot be modified. The 3D objects in this environment do not change over time, they are static. Links. Although the virtual environment in this version of a situated simulation cannot be modiefied or changed, the user may navigate and access information via links. These hypertextual link anchors are spatially placed and the link icon is itself connected to either an object or a position in the environment by means of a visual string. Because of these characteristics it has been given the name 'balloon link' (see Figure 2). Link layer. In order to be able to switch between an experiential mode (without active visible links) and an informational mode (with visible active links) the link layer can be turned on and off. This makes it possible to choose between the two modes of (re)mediation: immediacy and hypermediacy. [7] (See Figure 3.) Detail view. Due to the hardware limitations of the current platform (iphone 3GS), the poly count of displayed objects is limited to less than 20k to obtain an acceptable framerate. Thus in order to view details of optimal resolution the general environment is substituted with an abstract environment only showing the detailed object. The transition between the two 3D environments is achieved by means of a 'repositioning' of the object from its original place (low resolution) to a position close up in front of the ICL 2010 Proceedings Page 1009

3 user's position/camera (high resolution). This movement creates the impression that the object 'comes up to' the user and displays itself in front of his or her's physical position. Figure 2. The Oseberg Viking ship from 834 in Tønsberg south of Oslo, Norway. In the prow of the ship we see a balloon link with its string attached to the stem of the ship where the carving ("Utskjæring") is located. Figure 3. Mission Dolores in San Francisco anno 1791 with Twin Peaks in the background. With the link layer function all links can be made invisible and inactive, so that the user can switch between a experiential mode and an informational mode of exploration. Figure 4. Detail view in the Oseberg simulation. When the user is positioned on top of the mound, the balloon link 'Vikingportrett1 ('Viking portrait 1') is activated. The scanned 3D model of the carved head then 'comes up to' the user and can be rotated and enlarged by the user by means of touch conventions. ICL 2010 Proceedings Page 1010

4 3 Transparency and overlay (The Parthenon) In a sitsim displaying a reconstructed version of the original Parthenon temple on the Acropolis in Athens, the prototyped application was extended with two new features: transparency and overlay. Transparency. Since GPS signals are not accesible indoors, and in the case of the Parthenon, the visitor is restricted to only move around the building, not inside its remains, there is a need to look inside, through visible obstacles like walls, columns etc. In the print tradition, a solution to this problem is called cut-away illustration and is frequently used in books on architecture and in travel guides. In our case we used partial and full transparency. It was partial in order to still be able to see the outer limits of the construction (the architectural context), and full when the display of objects inside is the primary focus (see figure 5.). Figure 5. Reconstruction of The Parthenon Temple on the Acropolis in Athens anno 431 BC. When the temple is observed from the east the 'Transparency 1' balloon link is activated. This gives us a better view of the cella and the position of the frieze. The columns and upper parts of the Parthenon is still visible in semi-transparent mode. Figure 6. Parts of the Parthenon frieze is then selected for detail view. The detailed object can be enlarged and moved using touch conventions. After the balloon link 'Transparency 2' has been activated the cella becomes semi-transparent and the central object of the temple, the statue of the Godess Athen in the inner chamber can be observed. ICL 2010 Proceedings Page 1011

5 Overlay. Another solution borrowed from the print tradition of architecture is the overlay marking. To explain the elements of architecture and their corresponding names, balloon links were stringed to the objects. When these are activated, partly transparent color overlays mark the extension and limitation of each element; this is also described by a verbal audio commentary. Figure 7. The Parthenon observed from the west. Under the 'Building elements' link the user has chosen the 'Architrave' link, thus the lover compartment of the entablature resting on the column capitals is marked with a transparent red banner while a voice over provides additional information. 3 User generated links and sequences of events (Temple of Divius Iulius) In the most recent of the prototyped situated simulations - one reconstructing the Temple of Divus Iulius (The Temple of Deified Julius Caesar) anno 29 BC in the Roman Forum - we also made it possible for users to create links, and we included a sequence of actions and events. User generated links. In the sitsims mentioned above, all links are provided by the producers of the application. Users are limited to navigate and access the information that is provided for them to acquire. In the most recent version of the sitsim application users can create, name and position their own links. These user generated links are of two kinds: either a) they link to and display a node of written text when activated, or b) they link to a web page. In both cases, the user generated information is stored on a server independent of the local application (on the actual iphone in use). In the case of the web page, the application applies a built in simple web browser which is displayed inside the virtual environment on parts of the screen. This also makes it possible to update material for use in the simulation independent of the application itself. Users can also comment on other users links, thus turning the situated simulation into a spatial constructive hypertext [8]. ICL 2010 Proceedings Page 1012