Alexa is my new BFF : A Case Study of the Amazon Echo s Social Functions and Roles.

Size: px

Start display at page:

Download "Alexa is my new BFF : A Case Study of the Amazon Echo s Social Functions and Roles."

Angela Thompson
6 years ago
Views:

1 Alexa is my new BFF : A Case Study of the Amazon Echo s Social Functions and Roles. Amanda Purington Jessie G. Taft Shruti H. Sannon Natalya N. Bazarova Samuel Hardman Taylor Department of Communication Cornell University Ithaca, NY Paste the appropriate copyright/license statement here. ACM now supports three different publication options: ACM copyright: ACM holds the copyright on the work. This is the historical approach. License: The author(s) retain copyright, but ACM receives an exclusive publication license. Open Access: The author(s) wish to pay for the work to be open access. The additional fee must be paid to ACM. This text field is large enough to hold the appropriate release statement assuming it is single-spaced in Verdana 7 point font. Please do not change the size of this text box. Each submission will be assigned a unique DOI string to be included here. Abstract Amazon s Echo and its conversational agent Alexa provide new opportunities to study how people perceive and interact with social devices. Drawing from user reviews of the Echo posted to Amazon.com, this case study explores the degree to which user reviews indicate personification of the device, sociability level of interactions, factors linked with personification, and influences on user satisfaction. Results indicate users experience some degree of uncertainty in how to refer to the device, with over half using the personified name Alexa but most referencing the device with object pronouns. About half include at least some personification. Degree of personification is linked with sociability of interactions, with greater personification co-occurring with more social interactions. Reviewers mentioning multiple member households are more likely to personify the device than reviewers mentioning the self, the elderly, or disabled populations. Even after controlling for technical issues, personification predicts user satisfaction with Echo. Author Keywords Conversational agent; Amazon Echo; personification; social robots.

2 I just got this toy. I love it. There is still much I haven't used it for yet, but the concept is awesome. Alexa understands very well, and does learn as she goes. I just need to get all the items for her to interface with my house. If you just want a good speaker, she is good for that as well. I was thinking about this or Google Home. The Google item just doesn't look neat. Kind of drab actually. I love the way technology is heading, and this is a neat toy to have if you are a tech fan like me. My entire family enjoys Alexia! This device is in the center of our home! She keeps our grocery list, many reminders, plays all of our favorite playlists, orders amazon items, wakes us up, tells us when its time to leave, adds events to our calendars, and much much more! ACM Classification Keywords H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. Introduction With recent advances in technology, social conversational agents are increasingly becoming integrated into daily life. The popularity of devices such as the Amazon Echo, equipped with social agent Alexa, have provided researchers and designers with new opportunities to study how people perceive and respond to such social devices. The potential of the Echo to take on a range of different roles and functions in multi-user interactions, especially in personal spaces such as the home, makes this device particularly relevant for study. However, there continues to be a dearth of empirical research on its social uses and implications. This study seeks to contribute how people perceive, interact with, and integrate this device into social life through a case study of Amazon customer reviews. In particular, we draw on the Computers as Social Actors (CASA) paradigm [8] to understand users experiences and their social construction of this technology. According to the CASA paradigm, people respond to technologies as though they were human, despite knowing that they are interacting with a machine [8]. Thus, people ascribe personalities to computers [7] and even apply politeness norms to these interactions [6]. Particularly relevant to conversational agents is the finding that peoples perceptions of a machinegenerated voice are influenced by its human-like personality traits. For example, participants prefer machine-generated voices that are extroverted or similar to their own over introverted or dissimilar voices [4]. Speech functionality is an essential feature of conversational agents that enables anthropomorphic interactions with assistive technologies, in addition to other design choices that encourage socialization and perceptions of these devices as social actors [9]. The Amazon Echo has several affordances that predispose users to personifying it and integrating it into social life. First, to operate the device, users must interact with Alexa, the conversational agent. Thus, the Echo is inherently a socially interactive device, since it requires social interaction to function [2]. Alexa is also personified, in that the agent is endowed with a name, gender, and personality. Such design choices encourage users to anthropomorphize the device: unlike the mindless response described by Nass, anthropomorphism is a conscious mechanism wherein people infer that a non-human entity has human-like characteristics and warrants human-like treatment [5;3]. Finally, the Echo is collocated with users, and can alter the dynamics of its surrounding environment. Though the above characteristics are designed to afford social functionalities and promote anthropomorphism, it is less known how interactions with such devices are constituted in real-world social settings, and the types of roles and functions they fulfill for users. To shed light on this problem, this paper employed a case study approach to examine a series of questions through analysis of users reviews posted on Amazon.com. First, how do users perceive and respond to this device? Second, what factors predict when users are more likely to personify the device? Finally, how are users evaluations of the device influenced by degree of personification, the ways users interact with it, and its technical qualities and integration with other services?

3 Echo Echo & Alexa Alexa Name Personification Table 1: Frequency of name (Echo/Alexa) personification levels Object Pronoun Personification 307 Both 48 Person 63 Table 2: Frequency pronoun (it/her) personification levels No personification Some personification All personification Overall Personification Table 3: Frequency of overall personification levels Understanding how people interact with Alexa can help determine the social impact of conversational agents, and how they can support social interactions. Methods Data Collection Amazon.com reviews of the full-sized Echo device were collected for two weeks in December of 2016, for a total of 851 reviews. Reviews from Amazon.com [1] provide an unsolicited evaluation of the Echo device and Alexa agent, and are a potentially rich source of descriptions of the technology's functions and roles in a variety of settings. Star ratings were also collected, and an estimate of the gender of the reviewer was made based on the username. No usernames or any other identifying information about reviewers was collected. Content Analysis The first pass through the data identified whether reviews were related to the actual experience of using Echo/Alexa. Reviews that were short or undescriptive (n=264) were excluded from further coding. The remaining usable reviews were coded using the categories described below. Twenty-seven percent of remaining reviews were coded by a second coder; intercoder reliability was substantial (Cohen s κ=0.81). Degree of personification. The extent to which the technology was personified was operationalized as a continuum based on whether the review text described the technology as a person (using the name Alexa or personal pronouns such as "her"), as an object (Echo, or the pronoun "it"), or both (i.e., when users referred to the device as both her and it ). Types of interactions. An open coding process was used to generate codes that reflected the ways people use and interact with Alexa/Echo. Reviewers' uses of Alexa/Echo fell into five main categories, from less sociable to more sociable interactions: (1) information, news/weather, and facts; (2) entertainment: playing music, audio books and games; telling jokes; (3) assistant: managing shopping, schedules, and timers/alarms; (4) conversation partner or companion, an unspecified entity that can listen and speak; and (5) friend, family member, roommate, or spouse. Integration. A major advertised use of the Echo/Alexa is its ability to integrate with other technologies and services [1]. To assess whether personification of Echo/Alexa is associated with integration with other technologies, reviews were also coded for mentions of other smart devices or services. These included other Amazon devices (Dot, Tap, or additional Echos), integration with services (Amazon Prime, streaming music, etc.), and smart or connected home devices (smart home hubs, lights, thermostats, speakers). Technical qualities and issues. Personification and satisfaction with the device may depend on its ability to perform its advertised tasks serving as a speaker with voice recognition technology and intelligent responses. We noted reviewers' mentions of speaker or sound quality and voice recognition capabilities. The appropriateness or effectiveness of Alexa's responses were also captured. Other issues related to general functioning of the device were also noted. Household characteristics. We coded if the reviewer mentioned (1) being part of a family or living with another, (2) having children or grandchildren, (3) being

Frequency of Interaction Type Info 135 Entertainment 274 Assistant 116 Companion 19 Friend 25 Table 4: Frequency of interaction types elderly or having an elderly person in the home, (4) being

4 Frequency of Interaction Type Info 135 Entertainment 274 Assistant 116 Companion 19 Friend 25 Table 4: Frequency of interaction types elderly or having an elderly person in the home, (4) being disabled or having a disabled person in the home, and (5) using the device in a non-home setting. Results Of the 851 coded reviews, 264 were marked as unrelated to use of Echo/Alexa (n=31), undetailed (n=229), or both (5) and were excluded from subsequent analyses. The remaining 587 reviews were very favorable, with a mean star rating of 4.32 (SD=1.17) on a one- to five-star scale. Reviewer gender was not discernable for most of these reviews (n=260); when possible to infer from the username, most reviewers appeared to be male (n=200). Perceptions and Interactions Nearly all of the reviews kept for analysis contained some language that could be coded for degree of personification (n=503). When reviewers mentioned the device by name (n=328), a slight majority used Alexa, a smaller proportion used Echo, and some used both names in a single review (Table 1). In contrast, of the reviews containing pronouns referencing the technology (n=418), the majority used object pronouns, with far fewer using solely person pronouns (Table 2). Some reviewers used both object and person pronouns to reference the technology in the same review. There is a small, positive correlation between degree of name and pronoun personification (r=0.21, n=587, p<.001). Combining these two variables such that exclusive use of Echo and/or object pronouns is the lowest level (no personification at all), exclusive use of Alexa and/or person pronouns is the highest level (all personification), and a mix of object and person references is the mid-level (some personification) yields a continuum of increasing degree of personification. Of the reviews containing language coded for degree of personification (n=503), just over half had no personification at all, just under a third contained some personification language, and just under twenty percent contained personified language exclusively (Table 3). Open coding user descriptions of interactions with the Amazon Echo yielded five categories of interactions, on a continuum from least social (providing information such as news or weather and facts) to most social (friend, family member, roommate, or spouse). Of the 587 reviews kept for analysis, 347 described how the user interacts with the technology. Most reviewers described a single type of interaction (n=189), some indicated two types (n=104), and few indicated three (n=52) or four (n=4) types of interactions. The most common interaction was entertainment (Table 4). Highest level of interaction sociability was recorded and used for subsequent analyses. Predictors of Personification A one-way Analysis of Variance (ANOVA) found a significant effect of name personification on the level of interaction sociability: F(3, 343)=12.82, p<.001. A similar result was found for pronoun personification (F(3, 343)=11.79, p<.001) and combined personification (F(3, 343)=20,84, p<.001) on the level of interaction sociability. Those users who use the name Alexa and person pronouns to describe the technology are more likely to describe their interactions with the device in social terms when compared with reviewers who use the name Echo and object pronouns. Of the 587 reviews kept for analysis, 447 (76.15%) made some reference to members of the household. The plurality of these made reference to the self

5 (n=258), and the next most frequent reference was an undetailed indication of other family members (n=124). Specific mentions of children in the household (n=50), being elderly or having an elderly person in the home (n=10), being disabled or having a disabled person in the home (n=4), or using the device in a non-home setting (n=3) were less common. Because some of the categories had so few instances, we collapsed these categories into a single variable. References to the self were retained, while the undetailed indication of other family members and mentions of children were combined and special populations (including elderly and the disabled) were combined. This yielded a single variable for household type: single, multi-member family, or special populations. A one-way ANOVA found a significant effect of household type on personification, such that households with children or other family members who interact with the Echo are more likely to personify the technology than single users or special populations: F(2, 411)=7.23, p=.001. Household type did not have a statistically significant effect on sociability of interaction: F(2, 272)=1.91, p=.15. Influences on Satisfaction Star ratings offer a straightforward metric of reviewer satisfaction with the Amazon Echo, and accompany each review. We explored how user evaluations are influenced by degree of personification, interaction type, technical uses or issues, and integration with other devices and services through a linear regression model (R 2 =.17, p<.001). The dependent variable of user evaluations was log-transformed to reduce left skewness. Reviewers who mentioned the technology s (lack of) general functionality (β= -.16, SE=.07, t(298)= -3.02, p=.003) or appropriateness of responses to their commands (β= -.25, SE=.06, t(298)= -4.34, p<.001) were less satisfied with the device. Degree of personification predicted satisfaction, with more personification predicting greater satisfaction (β=.15, SE=.03, t(298)=2.51, p=.012). All other variables included in the model were non-significant. Discussion Reviews of the Amazon Echo and agent Alexa posted on Amazon.com suggest users have some degree of uncertainty regarding how to refer to the technology, with over half using the personified name Alexa, but most referencing the technology with object pronouns. When name and pronoun personification are considered together, users are nearly evenly split between no personification and at least some personification, though under twenty percent commit to exclusive personification language. These results suggest the extent of personification varies among users, and it will be important to understand factors that can explain the process of personification and differences among users. Most user descriptions of interactions with Echo/Alexa suggest low to mid-level sociability of the interaction. Specifically, most users describe interacting with the technology for entertainment purposes (such as playing music) or use the device as an assistant (managing scheduling or shopping), interactions that involve a small degree of sociability. However, those reviewers who exclusively use the personified name Alexa and person pronouns to describe the technology in their review are more likely to characterize their interactions with the device in social terms than reviewers who use the name Echo and object pronouns. Thus, there is an association between sociable uses of the device and its personification, and higher social functions and sociable roles correspond to a greater degree of personification.

6 Future research needs to explore causality to understand what drives this relationship: Do interaction experiences determine degree of personification or does perceived personification drive interaction type? Users whose reviews indicate children or others are in the household interacting with Echo/Alexa are more likely to personify the device, using the name Alexa and person pronouns more often than reviewers mentioning the self as a user or special populations (e.g., elderly or disabled). This suggests the technology may play a more important social role in families, rather than serving as a companion for those who may need extra support or live alone. The unique role of a conversation agent in the family should be explored in future research. Personification of the device also plays a role in user satisfaction with the device. Personification of a conversational agent appears to predict increased levels of satisfaction, regardless of technological problems or function of the device. Simply put, people who love her, love her. The findings of this exploratory study offer insights into how people socially construct, anthropomorphize, and interact with the Amazon Echo as a social and conversational agent. Although users vary greatly in the extent of the device personification, our findings point to the promise of anthropomorphic technologies in that users find satisfaction with technologies they can personify and relate to socially. References 1. Amazon Echo. Amazon.com. Retrieved December 19, 2016 from Echo-Bluetooth-Speaker-with-WiFi- Alexa/dp/B00X4WHP5E 2. Terrence Fong, Illah Nourbakhsh, and Kerstin Dautenhahn A survey of socially interactive robots. Robotics Autonomous Sys 42, 3: Chad Gonnerman Examining the how and why of anthropomorphism. Metascience 17, 3: Kwan-Min Lee and Clifford Nass Socialpsychological origins of feelings of presence: Creating social presence with machine-generated voices. Media Psychol 7, 1: Clifford Nass and Youngme Moon Machines and mindlessness: Social responses to computers. J Soc Iss 56 1: Clifford Nass, Youngme Moon, and Paul Carney Are people polite to computers? Responses to computer-based interviewing systems. J Appl Soc Psychol 29, 5: Clifford Nass, Youngme Moon, B. J. Fogg, Byron Reeves, and Chris Dryer Can computer personalities be human personalities? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 95), Clifford Nass, Jonathan Steuer, Ellen Tauber, and Heidi Reeder Anthropomorphism, agency, and ethopoeia: Computers as social actors. In INTERACT'93 and CHI'93 conference companion on Human factors in computing systems, Sabine Payr Virtual butlers and real people: Styles and practices in long-term use of a companion. in R. Trappl (Ed)., Your Virtual Butler. Springer-Verlag, Berlin, pp

Designing for End-User Programming through Voice: Developing Study Methodology

Designing for End-User Programming through Voice: Developing Study Methodology Kate Howland Department of Informatics University of Sussex Brighton, BN1 9QJ, UK James Jackson Department of Informatics