Ethnographic Design Research With Wearable Cameras

Transcription

1 Ethnographic Design Research With Wearable Cameras Katja Thoring Delft University of Technology Landbergstraat CE Delft The Netherlands Anhalt University of Applied Sciences Schwabestr Dessau Germany k.c.thoring@tudelft.nl Roland M. Mueller Berlin School of Economics and Law Badensche Str Berlin Germany roland.mueller@hwr-berlin.de Petra Badke-Schaub Delft University of Technology Landbergstraat CE Delft The Netherlands p.g.badke-schaub@tudelft.nl Abstract This paper presents a novel ethnographic research approach based on a wearable camera that automatically takes pictures according to sensor changes within the wearer s environment. The opportunities and challenges that are raised by such technologies are investigated and discussed. We describe an exemplary application of the approach and point out the arising possibilities and limitations. As the main contribution we present a method for evaluating the ethnographic research data that is being produced through this kind of research approach. Furthermore, we propose a set of frameworks for analyzing the resulting images and related metadata. This method is considered a first attempt to facilitate the analysis process of the produced qualitative and quantitative data and is subject to further investigation and development. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s). CHI'15 Extended Abstracts, Apr 18-23, 2015, Seoul, Republic of Korea. ACM /15/04. Author Keywords Digital Ethnography; Wearable Cameras; Autographer; Data Analysis; Data Mining ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous.

2 Figure 1: Autographer camera takes pictures automatically based on internal sensor data (acceleration, passive infrared, color and light, magnetometer, temperature, GPS coordinates). Figure 2: Autographer s metadata for one picture (grey = GPS, red = light and color, pink = motion, light blue = acceleration, yellow = magnetometer, dark blue = temperature). Introduction Autographer Digital Ethnography is mostly known for utilizing the Internet as a source for ethnographic research (also known as Netnography ) [8,9]. Ethnography has traditionally made use of digital recording devices, such as audio or video recording. Nowadays, innovative hardware devices offer new opportunities to gather ethnographic data. Wearable or head-mounted cameras have experienced great popularity during the past years especially among life-loggers and sportspeople who want to document parts of their lives or specific lifestyle activities, such as extreme sports. Examples are GoPro, Autographer, or Narrative Clip. These technologies offer also promising opportunities for ethnographic research purposes. Participants of a study would be equipped with such a wearable camera in order to document their activities, their environment, and their interactions with artifacts or other people. The researcher would not be required to actively accompany and observe the participants, but would analyze the returned data afterwards and conduct follow-up interviews with the participants to clarify any upcoming questions. However, other than a more focused ethnographic research approach, such technologies produce large amounts of qualitative and quantitative data with peculiar requirements for analysis. We suggest the utilization of wearable cameras for ethnographic research purposes and present insights from an exemplary application, using an Autographer camera. Based on the resulting data we propose an innovative preliminary method for analyzing the content produced through the use of wearable cameras. We conclude by providing a critical discussion and an outlook to future work. Autographer (Figure 1) is a wearable camera with 6 internal sensors that trigger the camera to automatically take a picture once a change is detected. Additional metadata is captured along with each picture: 1) The accelerometer measures the change of speed with which the camera is moving. 2) The passive infrared (PIR) sensor detects moving objects. 3) The color sensor measures light and brightness and adjusts the image accordingly. 4) The magnetometer is an internal compass and captures the direction in which the camera is facing. 5) The temperature sensor measures environmental temperatures. 6) Additionally, the integrated GPS locates the camera s position. See Figure 2 for a display of the metadata associated with each taken picture. People can wear the device around the neck or clip it onto a belt or any other object such as a bag or bicycle handle. They would only have to switch it on at the beginning of the research task. After that, the device will automatically take pictures based on the sensor changes or after certain elapsed time periods. It is possible to adjust the degree of sensitivity of the sensors (in three steps), in order to produce more or fewer images, as well as to start and stop the shooting of a series of images manually. The main advantage of this handling in terms of ethnographic documentation is that the wearer would not have to worry about how to handle the device and might even forget about the presence of the device. The pictures can be displayed in the provided software application, along with a GPS coordinate for each picture, and the additional captured metadata (Figure 3). Pictures can be viewed separately or as a slideshow that resembles a time-lapse video.

3 behavior and what way he would incorporate also offsite locations into his design process. After the end of the 2-weeks project we received more than 4000 images, each with related metadata. Figure 3 illustrates the quality of the returned data: each picture is tagged with a GPS location on the integrated map, so that it was possible to identify the respective location through a) the pictured environment on the photo itself, and b) through the marker on the map linked to the picture. Additional metadata is connected with each image (Figure 2). We were able to derive the behavior of the student and his interactions with people and objects from the pictured environment and context of the image. Arising questions were clarified in a follow-up interview. Figure 3: Autographer s application interface. Each image is tagged automatically with time and date, GPS location, and other metadata. Exemplary Application of the Approach To exemplarily test the usefulness of such a device as an ethnographic research tool, we handed an Autographer camera to one design student during a two-weeks project that had the assignment to design a new typeface. However, the design project itself was not the focus of our research. Instead, we were particularly interested in the actual behavior and workflow of the student, not only within the classroom, but in what way he would incorporate also other locations on and off campus into his work process, as well as his interactions with other people, artifacts, and environments. Our research question was in what way the physical environment of the campus could support or hinder the creative workflow. The chosen approach in which the student self-documented his activities using a wearable camera seems appropriate for this purpose, because we were interested in his actual Suggested Data Analysis Method The amount of produced data (thousands of pictures and related metadata) requires a new approach for data analysis. There are several software packages for qualitative data analysis (QDA) available that provide assistance with e.g. tagging and coding, but coding thousands of images would be very much of an effort. Also, these applications offer no features for easy adding and analyzing the mostly quantitative metadata, such as GPS coordinates, temperature, etc. Triangulating these different data sources (e.g. correlations of activities in relation to location, or a change of behavior in correlation to a change of temperature or sound level, etc.) might reveal valuable insights. Hence, we need a method that allows easy (e.g. semi-automated) triangulation and analysis of the returned data. Additionally, a visualization of the data would be helpful for interpretation, but most QDA software do not provide sophisticated visualization features that are targeted to this kind of data. In the

4 Chunk Generation Chunk Enrichment Visualization Interpretation iterate iterate Figure 4: Suggested 4-step data analysis process. following we present our preliminary data analysis approach, which is a 4-step analysis process (Figure 4). It can be executed either manually or by implementing semi-automated (data mining) methodologies. Step 1: Chunk Generation As the first step the images have to be reduced to a convenient number for further analysis. Consecutive sequences of similar images have to be identified and clustered into chunks. It is possible to review the images manually, and group those together to one chunk that show no significant change of information (either in the image itself or in the related metadata). However, a semi-automated method based on data mining technologies might reduce the effort for this task, significantly. Both ways, the goal is to identify sequences of similar images. As long as no significant change in the image or the metadata compared to the previous image is detected, that picture is added to the sequence. It should be taken into account that a change might come as a sudden (e.g. loud noise), or gradually (e.g. temperature rise). Outlier detection [1] or other change monitoring techniques [10] can be used for semi-automated identification of pictures that belong to one chunk. Step 2: Chunk Enrichment From each chunk (consecutive sequence of similar images), one image has to be identified that represents this chunk, in order to use only these key images for further analysis. That way it is possible to reduce the number of images significantly. Additionally, tagging of these key images according to identified activities (e.g. sketching, talking, computer work, walking, etc.) or according to indicated sensor changes helps for further analysis. We suggest summarizing these key images in a table overview and to add information like lengths of the identified activity and the respective tags (Fig. 5). Step 3: Visualization Synthesizing and visualizing the collected information helps with interpreting the data [7]. We suggest two types of frameworks for the evaluation of data produced by the Autographer camera: 1) A Space- Time-Activity Matrix that displays durations of identified activities in the form of bar charts aligned to the respective locations (Figure 6). Figure 5: Chunk Enrichment: Table overview (excerpt) of key images, tagging, and metadata. Figure 6: Suggested visualization: Space-Time-Activity Matrix. 2) An Activity Map that shows a map of the visited locations along with the respective activities performed in these (Figure 7: size of the circles indicate duration,

5 color codes indicate the type of the activity, lines and numbering indicate the sequence of activities). Also other kinds of mapping are possible, according to the research focus, such as an Artifact Interaction Map, or a People Interaction Map. The creation of the suggested frameworks can be accomplished manually or with the support of semi-automated visualization tools (future work). Existing tools for visualizing and mapping timecoded data, such as ChronoViz [4], do not synthesize the data in a framework as in our suggested data analysis process. 5, Figure 7: Suggested visualization: Activity Map ,9 10 Sketching Talking Modelmaking Computer Work Reading Watching Thinking Writing Step 4: Analysis and Interpretation The process concludes with a classical analysis and interpretation of the data: Researchers review the data, cluster according to tags and codes, try to identify patterns and correlations based on triangulation of data sources, and discuss open question with the participants in follow-up interviews. If necessary, they will have to iterate the process by returning to one of the previous steps, similar to a grounded theory approach [2,5]. Critical Discussion Contribution of the Research Approach The described research approach with participants self documentation through wearable cameras is particularly useful in situations, in which mobile activities are supposed to be researched in a real-life environment, instead of a laboratory setting [2,3]. Also, in situations where researchers cannot follow the participants all the time (like remote or long-term observations), the method seems to be helpful. Other than e.g. video documentation, which is usually limited to staged or rather immobile settings, observation with wearable cameras can take place outside the classroom or the research lab, without the presence of the researcher. However, other than video observation, Autographer cameras do not record sound, which leads to an increased effort for interpretation of the images or follow-up interviews, to clarify arising questions. The additional metadata provided by the various sensors of the Autographer camera become part of triangulation of different perspectives [6]. Sensors add information in cases when something is not clear. The data itself is not really interesting or relevant only in combination with other information from the metadata or the picture itself it might turn into insights (e.g. the temperature on the day the one particular picture was taken is not relevant in itself; but in combination with e.g. location data, it might provide insights about the climate threshold at which students move out of the classroom into the park for studying). Arising Problems during Data Collection 1) The GPS coordinates were imprecise. The location details varied about several meters (which became apparent as two pictures, which were taken clearly in the same room and within only a few seconds time,

6 were marked as being located on two different sides of one street). 2) Some of the captured metadata were difficult to interpret (see Fig. 2). For example, the magnetometer (the compass ) returns a triple of numbers (e.g. 0.0, 0.1, -0.2) that is somewhat difficult to transfer into a geographic direction. 3) The briefing of the participant is crucial. In our example, the student was independently shutting off the camera when he thought nothing interesting was happening, which is obviously not in the interest of the researcher. 4) Problems with privacy became evident. Of course, our participant was asked for his consent to be tracked by the device during the project, but the camera is constantly logging also the environment along with any other people in it. The student was instructed to inform the people he interacts with about the presence of the camera and to shut it off, if they would not consent to being photographed. The whole situation leads to an increased responsibility for the researcher to guarantee the privacy of the people involved. Next Steps and Current Limitations The development of the proposed evaluation method for this kind of ethnographic research data is still in its beginnings. We are planning to develop a prototype that uses semi-automated methods for image recognition and change detection to help the researcher with Chunk Generation, Chunk Enrichment and Visualization. The initial exemplary application of the presented method provides valuable insights about possible improvements and limitations of the research approach. The following actions appear to be necessary for the further development of the method: 1) finding a solution for inaccurate geo-tagging (e.g. implement indoor-positioning technologies); 2) finding a solution for privacy problems (e.g. by adding a filter that automatically blurs faces in the images); 3) applying the method to other research contexts and different types of cameras. We hope that the proposed research approach and data analysis method offers valuable new possibilities for HCI research and might inspire other researchers to investigate the potential of wearable cameras for their own research projects. References [1] Aggarwal, C.C. Outlier analysis. Springer, New York, [2] Crabtree, A., Rouncefield, M., and Tolmie, P. Doing design ethnography. Springer, New York, [3] Creswell, J.W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. SAGE, Thousand Oaks, [4] Fouse, A., Weibel, N., Hutchins, E., and Hollan, J.D. ChronoViz: a system for supporting navigation of time-coded data. CHI EA '11 (2011), [5] Glaser, B.G. and Strauss, A.L. The discovery of grounded theory. Transaction Publishers, Piscataway, [6] Jick, T.D. Mixing qualitative and quantitative methods: Triangulation in action. Administrative science quarterly 24, 4 (1979), [7] Kolko, J. Exposing the Magic of Design: A Practitioner s Guide to the Methods and Theory of Synthesis. Oxford University Press, New York, [8] Kozinets, R.V. Netnography: doing ethnographic research online. SAGE, Los Angeles, [9] Masten, D.L. and Plowman, T.M. Digital ethnography. Design Management Journal 14, 2 (2003), [10] Montgomery, D.C. Introduction to statistical quality control. Wiley, New York, 2007.