Hands-Off: Using Kinect to query the ancient Maya city of Copan, Honduras

Size: px

Start display at page:

Download "Hands-Off: Using Kinect to query the ancient Maya city of Copan, Honduras"

Ralf Kelley
6 years ago
Views:

1 Hands-Off: Using Kinect to query the ancient Maya city of Copan, Honduras Computer Applications and Quantitative Methods in Archaeology March 28 th, 2012 Southampton, UK Heather Richards-Rissetto 1,3, Jim Robertsson 2, Giorgio Agugiaro 1, Fabio Remondino 1 Jennifer von Schwerin 3, Gabrio Girardi 4, Maurizio Forte 5 (1) 3D Optical Metrology Unit, FBK Trento, Italy- hmrichards@fbk.eu, agugiaro@fbk.eu, remondino@fbk.edu (2) HUMlab, Umeå University, Sweden jim.robertsson@humlab.umu.se (3) Dept. of Anthropology, University of New Mexico, USA jvonschw@unm.edu (4) Fondazione Graphitech, Trento, Italy gabrio.girardi@grahpitech.it (5) University of California Merced, USA mforte@ucmerced.edu

2 OUTLINE 1. Research Design & Research Questions 2. Web VR in Archaeology 3. MayaArch3D Project a. QueryArch3D Tool b. Beta Testing Tool (Device-based) 4. QueryArch3D-Kinect Project 5. Conclusions & Future Direction 2

3 RESEARCH DESIGN I. Develop a prototype for an online tool called QueryArch3D to enable users to move through and query 3D models (of multiple resolutions) in a virtual reality (VR) environment II. III. IV. Beta-test QueryArch3D (navigation via mouse/keyboard) to acquire user feedback Develop interface to link QueryArch3D to Kinect Beta-test QueryArch3D-Kinect Interface to acquire user feedback V. Compare user experience using mouse/keyboard vs. Kinect (body gestures) VI. Apply findings to improve QueryArch3D for public education 3

4 RESEARCH QUESTIONS (1)What opportunities do 3d models, specifically when placed in the context of virtual reality, afford for educating the public about cultural heritage? (2) And how do non-immersive device-based vs. natural based interaction technologies impact learning of cultural heritage? (Virtual Museum Transnational Network (v-must) Carrozzio and Bergamasco 2010) 4

5 OUTLINE 1. Research Design & Research Questions 2. Web VR in Archaeology 3. MayaArch3D Project a. QueryArch3D Tool b. Beta Testing Tool (Device-based) 4. QueryArch3D-Kinect Project 5. Conclusions & Future Direction 5

edu/ The Etruscan tombs: Panoramic Virtual

6 DEVICE-BASED WEB VR IN ARCHAEOLOGY Rome Reborn: June 21, 320 AD Virtual World Heritage Laboratory (VWHL) The Etruscan tombs: Panoramic Virtual tours 3D Optical Metrology Unit (3DOM), Bruno Kessler Foundation (FBK) Google Earth 6

7 GESTURE-BASED WEB VR IN ARCHAEOLOGY Aquae Patavinae 7

8 WEB VR IN MESOMERICAN ARCHAEOLOGY The World at Night Tikal (flash video users rotate) Maya 3D: (3D models/apps) CyArk (Google Earth, 3D models/point clouds) Maya Skies (Unity 3D)

9 OUTLINE 1. Research Design & Research Questions 2. Web VR in Archaeology 3. MayaArch3D Project a. QueryArch3D Tool b. Beta Testing Tool (Device-based) 4. QueryArch3D-Kinect Project 5. Conclusions & Future Direction 9

MAYAARCH3D PROJECT http://mayaarch3d.unm.

and 3D digital tools for research, cultural heritage management, and

interdisciplinary, international project that brings together art

10 MAYAARCH3D PROJECT Research innovative approaches in digital technologies to integrate GIS and 3D digital tools for research, cultural heritage management, and education on architecture and landscapes in the ancient Americas An interdisciplinary, international project that brings together art historians, archaeologists, and cultural resource managers with experts in remote sensing, photogrammetry, 3D modeling, and virtual reality 10

11 MAYAARCH3D PROJECT Project Partners and Collaborators: University of New Mexico (USA) HUMlab, Umeå University (Sweden) Honduran Institute of Anthropology and History Bruno Kessler Foundation (Trento, Italy) Swiss Federal Institute of Technology (Zurich) University of California, Merced (USA) German Archaeological Institute (DAI, Bonn) Funding Provided by: National Endowment for the Humanities (USA) National Science Foundation (USA) Alexander von Humboldt Foundation (Germany) UNESCO 11

12 ARCHAEOLOGICAL SITE OF COPAN, HONDURAS 12

13 ARCHAEOLOGICAL SITE OF COPAN, HONDURAS 13

14 QUERYARCH3D TOOL In an online, navigable VR users can: 1. Integrate 2D and 3D data of multiple resolutions 2. Perform attribute and spatial queries of archaeological data 3. Compare and analyze 3D buildings, freestanding monuments, and sculpture 14

QUERYARCH3D TOOL (TECHNOLOGY & SOFTWARE)

Queries both geometries and attributes in the

Supports 3D visualization and navigation of

15 QUERYARCH3D TOOL (TECHNOLOGY & SOFTWARE) Functionality addresses needs in cultural heritage for a single tool that: 1. Handles multi-resolution models 2. Queries both geometries and attributes in the same virtual environment 3. Supports 3D visualization and navigation of the models 4. Permits both local and online access to the content Agugiaro et al. 2010,

16 QUERYARCH3D TOOL (LODs) LOD 1 (ca archaeological structures) LOD 2 (working CAD reconstruction of Temple) 16 LOD 3 (some sculpture added to Temple; 3D models of some stelae) LOD 4 (reality-based segmented 3D models)

17 BETA TESTING QUERYARCH3D TOOL Short survey (14 questions) 82 survey responses Faculty: Maya archaeology, architecture, university libraries, high performance computing Students: archaeology, architecture, art history, cultural heritage management 17

18 SURVEY: USER DEMOGRAPHICS 18

19 SURVEY: POPULAR FEATURES 19

20 SURVEY: DEVELOPMENT NEEDS 20

21 OUTLINE 1. Research Design & Research Questions 2. Web VR in Archaeology 3. MayaArch3D Project a. QueryArch3D Tool b. Beta Testing Tool (Device-based) 4. QueryArch3D-Kinect Project 5. Conclusions & Future Direction 21

22 QUERYARCH3D-KINECT: TECHNOLOGY & SOFTWARE 22

23 QUERYARCH3D-KINECT: MAPPING GESTURES Keyboard Onscreen Command Movement Kinect (gestures) W Move forward Right arm forward S Move backward Left arm forward Q left turn Rotate body left E right turn Rotate body right Y Look up Lean backwards H Look down Lean forwards T Start walkthrough Right arm down LOD 1 Keyboard Onscreen Command Movement Kinect (gestures) W Move forward Right arm forward S Move backward Left arm forward Q left turn Rotate body left E right turn Rotate body right Y Look up Lean backwards H Look down Lean forwards T Start walkthrough Right arm down 23

24 QUERYARCH3D-KINECT: TECHNICAL ISSUES Technical/Logistical Issues: 1. Light sensitivity (windows, glass objects) 2. Distance (Full body vs. Upper body) 3. Angle of Kinect (relationship to body movement) 4. Direction of Kinect 5. User Motion Speed 6. User Height (FAAST programmed using movement distance) 24

25 OUTLINE 1. Research Design & Research Questions 2. Web VR in Archaeology 3. MayaArch3D Project a. QueryArch3D Tool b. Beta Testing Tool (Device-based) 4. QueryArch3D-Kinect Project 5. Conclusions & Future Direction 25

26 QUERYARCH3D: CONCLUSIONS General advantages of 3D GIS in a Virtual Reality: 1. Contextualizes 3D models in larger spatial and environmental context (beyond individual objects or buildings) 2. Explore 3D models from multiple perspectives (e.g., bird s eye, façade view, interior view) 3. Offers a sense of embodiment in architecture or landscape (sense of place) (Steuer 1992) 4. Perform dynamic, interactive queries on 3D models 5. Increases awareness of spatial relationships between objects and associated data (Forte and Kurillo 2010) 26

27 QUERYARCH3D: ADVANTAGES FOR PUBLIC EDUCATION The online virtual environment is suited to today s youth visual learners who are intrigued by, and proficient in using interactive and dynamic technologies. Helps non-specialists to understand larger spatio-temporal context of objects multiple levels of accessibility and with multiple types of media (panoramas, photos, videos, database, etc.). guided virtual tours, with stops to access multimedia content (videos, maps, etc.) access to unrestricted parts of the database to query, compare, and analyze art and architecture Virtual and site-based multi-media exhibitions 27

28 QUERYARCH3D: LIMITATIONS FOR PUBLIC EDUCATION Blocky models are boring. We can add more high-resolution models, but further research is needed to define a data model and to implement database storage procedures for the textures contained in the *.obj files. Download time (up to 12 minutes) and crash rate (10 percent) may cause viewers to give up. The architectural reconstructions are not transparent -Must include on/off switch for show reality-based models. Much of the model is not within the archaeological park and online access to this information might lead to increased looting. Public users must be restricted from access to sensitive parts of the database thus limiting the true context. 28 What about sound?

29 FUTURE DIRECTION Benefits can access via browser, low cost, portable Sense of landscape, spatial relationships, different building sizes and heights Sense that this was a city not simply a ritual center Unity 3D game engine, smooth navigation Disadvantages- Kinect is light sensitive (reflections) Limited sense of reality in terms of zoom, rotate objects (not haptic) Unity 3D- 65k mesh limit (not ideal for reality-based models) Future Direction User tests comparing device-based vs. gesture-based - Add sound - Add constructed narratives - Add vegetation - Add additional models - Add transparency function - Add additional user - Freedom to move throughout entire site - Measure and search tools using kinect - Add avatars? People, artifacts - Offer more information on social, political, historical, ideological contexts of tool (vr and models) 29

30 FUTURE DIRECTION Device-base? Natural-based? or Both? Interactive Exhibit at Copan Archaeological Park or Children s Museum 30

31 ACKNOWLEDGMENT This project was supported by the National Endowment for the Humanities (insert #s), National Science Foundation (# ), Alexander von Humboldt Foundation (Germany), HUMlab, Umea University (Sweden) and the Foundation for the Advancement of Mesoamerican Studies (USA). The authors thank Fabrizio Galeazzi (UC Merced,USA), Raul Maqueda and Laura Ackley, who cooperated to build the 3D Studio Max model of Temple 22, and Alessandro Rizzi (FBKTrento), who helped in the processing of the reality-based 3D models. We also thank the Honduran Institute of Anthropology and History (IHAH) for the collaboration and permission to work at Copan. 31 Contact Info: Heather Richards-Rissetto Jim Robertsson Giorgio Agugiaro Fabio Remondino Jennifer von Schwerin Gabrio Girardi Maurizio Forte

An Agent-Based Architecture for Large Virtual Landscapes. Bruno Fanini

An Agent-Based Architecture for Large Virtual Landscapes Bruno Fanini Introduction Context: Large reconstructed landscapes, huge DataSets (eg. Large ancient cities, territories, etc..) Virtual World Realism