Liquid Galaxy: a multi-display platform for panoramic geographic-based presentations

Transcription

1 Liquid Galaxy: a multi-display platform for panoramic geographic-based presentations JULIA GIANNELLA, IMPA, <julia@impa.br> LUIZ VELHO, IMPA, <lvelho@impa.br> Fig 1: Liquid Galaxy is a multi-display platform data visualization viewer that enables immersive panoramic experience through interactive tours with KML data, videos, photos, and 2D/3D graphics. This technical report presents Liquid Galaxy (LG) platform with particular interest in its authoring features for panoramic geographic-based presentations within the scope of the research agreement between the Institute for Pure and Applied Mathematics (IMPA) and the Moreira Salles Institute (IMS). 1 INTRODUCTION LG is a multi-display platform data visualization viewer that enables immersive panoramic experience through interactive tours with KML data, videos, photos, and 2D/3D graphics. LG was originally developed by Google as an open source project to showcase Google's geospatial technologies and to promote its image at trade shows and exhibits. Created in 2008 by Jason Holt, LG started out as a panoramic multi-display Google Earth viewer [1], but has been extended by End Point Corporation, a software consultancy based at New York, to become a more general data visualization tool for operations, real estate, marketing, tourism, museums and research [2]. End Point offers development, deployment, and support services involving LG systems. With over 100 LG installations and events worldwide (Fig. 2), End Point has extended

2 the platform capabilities [3]. In its corporate version, LG includes a content management system (CMS) as well as support for other media features (eg. images, videos, 3D models and panoramic photos and videos). Fig. 3 LG system at IMS's facilities. 2 LIQUID GALAXY SYSTEM Liquid Galaxy is composed of hardware and software systems. Fig. 2 LG installations and events worldwide. In 2017, End Point donated an LG installation to the Vision and Computer Graphics (Visgraf) laboratory based at the Institute for Pure and Applied Mathematics (IMPA), offering an unprecedented opportunity for researches in the field of geospatial data visualization and new media. In 2018, IMPA and the Moreira Salles Institute (IMS), holder of one of the most relevant cultural collections of Brazil, signed a research agreement aiming IMS s photographic heritage [4]. One of the objectives of this institutional collaboration is to research and develop immersive panoramic and geographic-based demo solutions to present IMS s photographic collections. At this time, the LG platform is properly installed at one of the IMS's facilities (Fig. 3) and teams of both institutes are developing a first demo. 2.1 HARDWARE SYSTEM In its most common configuration, the Liquid Galaxy hardware provided by End Point is comprised of 7 large-paneled HDTVs, a touchscreen podium tablet, a 3D joystick controller, and 3 rack-mount computers. Computers are configured to work as follows: the head node manages content and allows administration, and two display nodes run the displays screen. One display node runs the center three screens and the podium tablet, and the other runs the two outer screens on either side [5] (Fig. 4). Fig. 4 LG hardware system. 2.2 SOFTWARE SYSTEM End Point provides a rich content management application (CMS) for the Liquid Galaxy platform accessed through a web browser (Fig. 5). Based on Robot Operating System (ROS), the CMS provides a structured environment for creating content for LG presentations using different

3 assets such as geographic visualizations, graphic overlays, photos, video, sound, and web pages. End Point also provides a user's guide documentation for its CMS [5]. associated with a scene, it s through the viewport mapping feature that lists seven items that correspond to the physical displays: left tree, left two, left one, center, right one, right two and right three (Fig. 7). Fig. 5 LG content management system. 3 PRESENTATION COMPONENTS In this section, it's given further information about LG components involved during presentation tours. 3.1 DISPLAYS SCREENS LG standard setup is 7 HDTVs screens resulting in a wide FoV display with a high definition resolution (7056x1080). The screens are positioned side by side with an angle of 15 degrees (Fig. 6). Fig. 7 CMS viewport mappings. Conceptually, viewport mappings comprise three location areas: background, sides, and center (Fig. 8). Each viewport area runs independently. Fig. 8 Viewport areas of the displays. Fig. 6 LG display screens. The display screens are used for showing content. In the CMS, screens are called viewports and each time an asset is Background: content placed on this viewport area span all seven screens. It's normally used to show Google Earth media, panoramic media, and 3D models; Center: comprises the three central viewport mappings (left one, center

4 and right one). Content placed on this viewport area runs on the foreground and can span the three central screens. It's normally used to show highlighted content through static media such as titles, image, video, etc. Sides: comprises the lateral viewport mappings (left three and left two and right three and right two). Content placed on this viewport area runs on the foreground and can span the left lateral screens or the right lateral screens. It's normally used to show complementary content through static media such as titles, image, video, etc. 3.2 TOUCHSCREEN The standard control method for presentation and scene selection is a 22'' touchscreen mounted on a podium (Fig. 9). The left side of the touchscreen interface shows a menu with available presentations while the main area shows a grid of thumbnail images. Each thumbnail corresponds to a scene. joystick (Fig. 10) mounted on the same podium as the touchscreen. This controller enables 6-axis navigation. Fig. 10 LG standard navigation controller. 4 CONTENT STRUCTURE The CMS organizes content into distinct presentations consisting of a number of scenes. Each scene contains discrete elements such as KML data, videos, images, web pages, etc. Each of these elements is known as an asset, and each is assigned to a specific location on an LG display. The system combines presentations into presentation groups. This leads to the following hierarchy: Presentation Group Presentation Scene Asset The easiest way to create content is to determine groups, then presentations, then scenes, then assets to add. The three first operations are done through the sidebar of the CMS interface (Fig. 11) while adding an asset has a further step (Fig. 121). Fig. 9 LG standard touchscreen. 3.3 NAVIGATION CONTROLLER The standard control method for scene navigation is a SpaceNav 3-dimensional

5 presentation through the touchscreen, the system sequentially plays one scene at a time. Fig. 11 The sidebar of the CMS interface. 4.3 SCENES A scene is a temporal composition (even though it can have time configured as infinite) containing one or more features that are initiated at the same time. The duration is an important property of the scene as it affects the presentation narrative structure. The value declared in the duration field is the time in seconds that a scene will stay active until the next scene starts. Scenes automatically flow one to the next, in sequence, unless the user intervenes selecting scenes through the touchscreen in a non-ordered way. The best practice to have an infinite scene is to set its value very high. Fig. 12 Adding an asset. 4.1 PRESENTATION GROUP A presentation group is like a folder of presentations. Each presentation group can have a name and a description. An "active checkbox" feature determines whether the LG touchscreen will display the presentation group or not. 4.2 PRESENTATION A presentation is like a collection of scenes in a specific order. Each presentation can have a name and a description. An "active checkbox" feature determines whether the LG touchscreen will display the presentation or not. When the user starts a An "active checkbox" feature determines whether the LG touchscreen will display the scene or not. A "visible on touchscreen checkbox" determines if the scene will be shown through a thumbnail on the touchscreen. If this checkbox is left unchecked, no thumbnail will appear on the touchscreen, but the scene will be shown in the pre-established sequence. 4.4 ASSETS Content is assigned to the presentation through assets addition. An asset is a discrete element such as Google Earth data, videos, images, web pages, etc. A full list of LG assets is described in End Point user's guide documentation [5]. There are two families of assets: the one that runs on the background and spans all seven display screens and the one that runs on the foreground and can be mapped on

6 one or multiple viewport mappings, respecting the display nodes BACKGROUND ASSETS Assets that run on the background cannot be resized and fill the 7 HDTVs screens. There are 3 types of background assets: Google Earth: It's a KML or KMZ file that will be loaded by default on all screens of the LG and displayed by Google Earth viewer. It can be either a "Geographic Data" asset (eg. visual elements such as location pins and images tied to a geographic location) or a "KML Tour" asset (Google Earth instructions to fly from one location to another). It can also be a "Cesium" asset. Street View: as the name says, a Google Street View scene, identified by pano ID, heading, and tilt. Panoramic media: a panoramic file media (images or video) in a resolution and aspect ratio that resemble the native resolution of LG display screens (7056x1080). The assets are named "Panoramic Image" and "Panoramic Video". 3D model: either a "SketchFab" object model identified by a model ID or a "Unity" scene model identified by a scene ID FOREGROUND ASSETS Assets that run on the foreground can be mapped on one or multiple viewport mappings, respecting the display nodes. Foreground assets include: "Graphics" (a digital graphic image); "Video" and "Browser Video Player" (a digital video file) and "Webpage" (a webpage identified by a URL). 5 COMPLEMENTARY AUTHORITY ENVIRONMENTS The CMS is the main authority environment for creating content for LG. It's responsible for creating the narrative structure of the presentation and organizing different assets within a scene. Still, some assets need previous development before they can be upload on the CMS. This section presents some of them but is beyond our scope to cover these complementary authority environments in depth. 5.1 KML DEVELOPMENT Keyhole Markup Language (KML) is a file format used to display geographic data in an Earth browser such as Google Earth. KML uses a tag-based structure with nested elements and attributes and is based on the XML standard. KML development includes Google Earth [6] software user interface (Fig. 13) and KML coding through a text editor. By combining both software and basic KML coding is possible to create features such as: Google Earth annotation (eg. placemarks, paths, polygons, etc.); Google Earth image overlay (eg. overlay photographs and scanned maps); Google Earth screen overlay (eg. legends, logos and other images); Google Earth tour (eg. camera movement instructions that enable flying from one location to another) When a text file is saved with a.kml or.kmz extension, Earth browsers, including the one available on LG, know how to display it. Google has created detailed documentation and tutorials on the KML subject [7, 8].

7 5.4 BLENDER/SKETCHFAB Blender [11] is a professional, free and open-source 3D computer graphics software used for creating animated films, visual effects, 3D object models and video games [Fig. 16] Fig. 13 Google Earth software user interface. 5.2 PANOTOUR Panotour [9] is a virtual tour software that enables the development of interactive, and panoramic (360 o ) tours (Fig 14). Fig. 16 Blender. Although Blender is used for 3D media development, LG doesn't import Blender files. Instead, LG supports model IDs (a string of letters and numbers) from SketchFab [12], which is an online platform for 3D media distribution (Fig. 17). Fig. 14 Panotour. 5.3 UNITY Unity [10] is a cross-platform game engine developed by Unity Technologies that can be used to create both three-dimensional and two-dimensional scene modeling with navigation and interaction features (Fig. 15). Fig. 17 SketchFab. Fig. 15 Unity. 6 NARRATIVE STRUCTURES LG functionalities enable different narratives structures with different levels of interactivity for presentations. Within our scope of researching and developing demo LG solutions to cultural collections, we came up with a classification.

8 The first level of classification is between non-interactive and interactive narrative structures and it's based on the duration of the scene. Then we identified two sublevels of non-interactive structures and three sublevels of interactive structures: Non-Interactive : A linear narrative structure. The scene duration is different from zero: Linear: It's the basic structure in which a scene flows one to the next, automatically. The user passively watches the presentation on the display screen and doesn't interact with the touchscreen and the navigation controller; Linear Looping: It's a variant from the latter one. The difference is that the presentation is set to repeat, so the last scene flows to the first scene, making a loop. Interactive : A narrative structure that enables different levels of user interactivity. It can be both linear or non-linear: Linear: The scene duration is different from zero so one scene flows one to the next, automatically. The user can interact within a scene with navigation controller (3D joystick) performing tasks such as pan, zoom, and rotation. We call this interactive visualization ; Non-linear: The duration of the scene has a very high value, what conceptually corresponds to a scene with infinite duration. The user takes control of the narrative structure of the 7 REFERENCES presentation. Non-linear interactive narrative structures are divided into two types: Dynamic interaction: The 3D joystick is used for interactive visualization within a scene while the touchscreen is used for scene selection in a non-ordered way; Exploratory interaction: It's a variant from the latter one. The difference is that on this structure we hypothetically consider the use of a more advanced presentation controller that allows the user to perform actions with content within a scene. [1] < [2] < [3]< 4/expanding-capabilities-liquid-galaxy.html> [4]< mam-parceria-com-foco-em-acervo-fotografico/> [5] End Point Liquid Galaxy Roscoe CMS User Guide v [6] Google Earth. < [7] KML Documentation. < [8] Google Earth Documentation. < [9] Panotour. < [10] Unity. < [11] Blender. < [12] SketchFab. <