MoCap Tango: Traces of Complexity

Transcription

1 MoCap Tango: Traces of Complexity Jeroen Peeters Interactive Institute Swedish ICT and Department of Informatics, Umeå University Ambra Trotto Interactive Institute Swedish ICT and Umeå School of Architecture, Umeå University Stoffel Kuenen Umeå Institute of Design, Umeå University Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. TEI '16, February 14-17, 2016, Eindhoven, Netherlands Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM /16/02 $15.00 DOI: Abstract In this paper, we report on an ongoing design research project MoCap Tango. Tango is a form of partner dancing in which two bodies sense each other in a dynamic, physical dialogue that is known for its subtle complexities, beauty and intimate experience. In MoCap Tango, we explore how we can use our skills as designers to highlight and unravel these embedded qualities and use them as inspiration in designing interactions. Using an optical Motion Capture System and custom-made passive markers, the movements of two world-class tango dancers are visualized in realtime. We present our motivation for this project, describe the first prototype and conclude with reflections on what this prototype revealed in terms of design opportunities and its relevance for the TEI community. Author Keywords embodiment; design research; experience; dance; tango; performance; visualisation. ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous; 545

2 Introduction Similar to much work in the TEI community, our interaction design research is founded on ecological perception [4], phenomenology [7] and embodied cognition [1]. Coming from different perspectives, these theories share the primacy of action as a central notion in the sense-making process through the body. It follows from this notion that meaning is released in dialogue as we experience the world by acting in it using our bodies. Such meaning is ungraspable, ephemeral and dynamic, emanating from our interactions with the world around us and the people within it. One way to engage with such an experiential notion is practice-based design research [5], in which design action and reflection become generators of knowledge. Using such an approach, we have in previous work explored how to design the qualities that may elicit a meaningful experience for people interacting with a design, or with each other, through a design. In the MoCap Tango project, we use the same designerly approach from a different perspective. Rather than investigating and designing qualities that may elicit a specific experience in interaction, we start from an embodied experience to see how amplifying it may reveal something of the qualities it possesses. Related Work and Motivation Following an embodied perspective, the value of designing for movement in interaction is clear. Different approaches, tools and methods have been developed to design for movement-based interaction. Designing for the moving body also calls for techniques and methodologies to design from the moving body, i.e., ways of sensitizing designers to qualities of movement and providing ways to communicate and use these qualities in the design process. Dance, as a particular type of movement that has aesthetic and expressive qualities, is foundational to a number of different approaches and methods for designing from the moving body (e.g. [2,6] or [9] for an extensive overview) An example of notable work directly related to the MoCap Tango project investigates ways in which visualisation [8] or analysation [3] of qualities of movement in dance can be facilitated by interactive technologies. Similarly to this work, our intention in this project is to use technology and our skills to design a prototype that may give us insights into qualities of movement. Our aim is not at gaining an analytical understanding of the movements in dance. We rather aim at using our skills as designers to emphasize and amplify the personal perspectives of expert dancers. By working with the dancers to shape the technological mediation performed by the installation, we aim to find expressions of the qualities embedded in the skills of the dancers. In using the forming language of design, we aim at exposing their intentionality in the process of embodied sensemaking, highlighting qualities that otherwise would remain hidden. 546

3 Work-in-Progress Context The project focuses on tango as it is a partner dance, meaning that it revolves around the embodied dialogue between two dancers. Two of the authors dance tango and were involved in the organization of the yearly Midnight Light Tango Festival organized by Tango Norteño. This year, the festival was hosted at Sliperiet of Umeå University - the same venue as our research lab - presenting itself as the perfect opportunity to materialize our interest. Two world-class tango dancers, Murat Erdemsel and Sigrid van Tilbeurgh attended the Festival for workshops and demonstrations, and were eager to become part of the design project. Motion Capture System Motion Capture Systems are used to record the movement of objects or people in a 3D space. Typically, they are used to digitise physical movement. For example, they are used in the gaming industry, where movements of actors are recorded and then used as the basis for life-like digital animations. We used an Optitrack Prime 13W system (82 x 70 FOV, 240 FPS cameras), consisting of 12 cameras, positioned evenly around a 5x5m square dance floor (see Figure 3). The cameras tracked special custom-made markers - small infrared reflecting spheres - as they move in 3D space. The markers were positioned in different points on the dancers bodies, in clusters of three. Design Process Figure 1: Close-up images of wearables to track foot movement: reflective spheres as markers, mounted on laser-cut MDF and fastened using elastic straps In line with our theoretical foundations and usual approach, we used a making-centered design approach. The hardware, software and visualizations were rapidly developed in parallel. In shared reflections, the design team shared their points-of-view from their respective expertise, allowing the design of different elements to mutually inform one another. For example, the design and fabrication of the wearables (see Figure 1) was informed by the dancers as well as the programming of the software, in order to balance data quality with freedom of movement. Similarly, different styles and ways of visualizing the data informed experimentation in style of dance, and vice versa (see Figure 4). Technical Description. The first MoCap Tango prototype consists of three main components: a Motion Capture System, custom-built wearables and customized software. Below, we briefly explain these components: Wearables Actors in Motion Capture Systems typically wear special suits featuring a number of markers on key positions on the body. Since our prototype was to be part of a performance, we needed to design ways in which the markers could be worn in addition to typical tango attires. Different wearables were designed for different body parts and were attached using pins or elastic bands. For the performance, the wearables were attached to the feet, hips, shoulders and hands of both dancers (see Figure 2). Software The Motion Capture software records the coordinates for each tracker s position. This data was directly streamed to customised software that translated the raw data into visuals (see Figure 4). Using a HD projector, the visualisations were projected onto a vertical surface above and behind the dancers. 547

4 Work-in-Progress Performance The prototype was used for a public performance by Murat Erdemsel and Sigrid van Tilbeurgh at the Festival (see Figures 2 and 3). Additional private performances were held after the Festival, in order to explore different visualisations and provide additional documentation of the process. We recommend readers to watch the video teaser of the project [10], showing footage of several performances and interviews with part of the design team. Figure 4: (Above) Images showing examples of two distinctly different styles of visualisation from two different camera angles. Because movement data is stored, it can be played back, allowing for experimentation with different visualisation designs after the performance. Figure 2: Image showing Sigrid wearing the wearables during one of the performances. The markers have been digitally enhanced in post-production to be more visible. Figure 3: Image showing the public performance. Sigrid and Murat dance within the Motion Capture System for an audience. Their movement data is visualised behind them on a projection surface. 548

5 Reflections By reflecting upon the first prototype and the performances, we can identify a number of opportunities and directions to develop this project further. These range from technical and design opportunities to methodological ones. Below, we firstly discuss a number of opportunities related to the design and use of the prototype. Secondly, we briefly propose our reflections on the approach as a useful tool for the design of embodied interactions. Design Opportunities Building on the current components of the prototype, there is a great deal of parameters that can be changed to further explore the visualisations of tango movements. In terms of hardware, the design of the wearables could be improved to make them more durable and less obtrusive for the dancers. By increasing or reducing the number of wearables, or by changing their positions on the bodies of dancers, different elements of the dance could be highlighted or removed from the digital tracking. On a software level, we aim to experiment with the many different variations of visualisations that are possible: e.g. changes in colour according to speed of movement, timing of the decay of visual traces, the point-of-view onto the digital 3D space, etc. Lastly, the prototype features a HD projection onto a flat surface above the dancers. One particular interest of ours is to move this projection off the screen and into the space surrounding the dancers. Our goal is for the visualisation to become a materialisation of the dance, so that it gains a spatial presence. Research Opportunities In terms of research, we are intrigued by the perspective that our designerly approach has given us and expert dancers into the complexities involved in the tango dance. The tango expert and member of the design team Murat Erdemsel is currently using the material created by the prototype in his teaching activities. It allows him to point out subtleties and complexities of movement s sequences that are difficult to communicate verbally. Similar advantages can be seen from a design research perspective. Tango is a dance full of subtle complexities, difficult to analyse because of the dynamics and the close proximity of the dancers. The visualisation allows a new perspective on this complexity. Many details are unveiled through the visualisation. We are currently working to articulate what dynamic, relational and aesthetic qualities can be exposed, by means of the visualisations, Once we will have a better grip on the nature of the observable qualities, we will be able to play with the design and playback of the visualisation to adapt it to highlight or emphasize particular parts of the movement. We believe that further exploring this approach of amplifying expert s movement in dance to reveal embedded qualities can contribute to the body of tools that informs our design process in designing engaging interactions. 549

6 Future Work We are currently developing a new version of the prototype to be used in a performance, with responses to the insights we gained into the design opportunities it offers. We aim at using the design of a new iteration as a research vehicle to better understand movement qualities in tango and their relevance for embodied interaction. Acknowledgements MoCap Tango is an ongoing design research project at Interactive Institute Swedish ICT, in collaboration with Murat Erdemsel and Sliperiet, Umeå University. We would like to thank Tango Norteño in Umeå for their enthusiastic hosting of the performance and especially the dancers that helped us with the prototype: Sigrid van Tilbeurgh, Carlos Arena and Duyen Ngoc La. The project was made possible with contributions from Interactive Institute Swedish ICT, Sliperiet Umeå University and Tango Norteño. References 1. Andy Clark Being There: Putting Brain, Body and World Together Again. Cambridge, MA: MIT Press 2. Caroline Hummels, Kees C. Overbeeke, and Sietske Klooster Move to get moved: a search for methods, tools and knowledge to design for expressive and rich movement-based interaction.personal Ubiquitous Comput. 11, 8 (December 2007), DOI= y 3. Diego Silang Maranan, Sarah Fdili Alaoui, Thecla Schiphorst, Philippe Pasquier, Pattarawut Subyen, and Lyn Bartram Designing for movement: evaluating computational models using LMA effort qualities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, DOI= 4. James J. Gibson The Ecological Approach to Visual Perception. London: Lawrence Eerlbaum (reprinted in 1986) 5. Ilpo Koskinen, John Zimmerman, Thomas Binder, Johan Redstrom, and Stephan Wensveen Design Research Through Practice: From the Lab, Field, and Showroom (1st ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. 6. Lian Loke and Toni Robertson Moving and making strange: An embodied approach to movement-based interaction design. ACM Trans. Comput.-Hum. Interact. 20, 1, Article 7 (April 2013), 25 pages. DOI= 7. Maurice Merleau-Ponty Phenomenology of Perception, trans. Colin Smith. New York: Humanities Press 8. Sarah Fdili Alaoui, Frederic Bevilacqua, and Christian Jacquemin Interactive Visuals as Metaphors for Dance Movement Qualities. ACM Trans. Interact. Intell. Syst. 5, 3, Article 13 (September 2015), 24 pages. DOI= 9. Sarah Fdili Alaoui, Thecla Schiphorst, Shannon Cuykendall, Kristin Carlson, Karen Studd, and Karen Bradley Strategies for Embodied Design: The Value and Challenges of Observing Movement. In Proceedings of the 2015 ACM SIGCHI Conference on Creativity and Cognition (C&C '15). ACM, New York, NY, USA, DOI= 10. A video teaser of the project can be found at: 550