Participant Information Sheet Project Title: Harlie Human and Robot Language Interaction Experiment Principal Investigator: Dr Christina Knuepffer, Postdoctoral Research Fellow, School of Information Technology and Electrical Engineering, The University of Queensland. Phone: 0406 932 128, Email: c.knuepffer@uq.edu.au Associate Investigators: Melbourne University Dr Adam Vogel, Speech Neuroscience Unit, Phone: 03 9035 5334, Email: vogela@unimelb.edu.au The Australian e-health Research Centre, CSIRO Dr David Ireland, Phone: 0411 520 137, Email: d.ireland@csiro.au Dr Simon McBride, Phone: 07 3253 3631, Email: simon.mcbride@csiro.au Dr Dana Bradford, Phone: 07 3327 4404, Email: dana.bradford@csiro.au Bond University Prof Helen Chenery, Faculty of Health Sciences and Medicine, Phone: 07 5595 5499, Email: hsmexecdean@bond.edu.au The University of Queensland Prof Janet Wiles, School of Information Technology and Electrical Engineering, Phone: 07 3365 2902, Email: j.wiles@uq.edu.au Dr Jacki Liddle, Centre for Clinical Research, Phone: 07 3346 5583, Email: j.liddle@uq.edu.au Dr Daniel Angus, School of Journalism and Communication, Phone: 07 3346 8255, Email: d.angus@uq.edu.au Dear Sir/Madam, You are invited to participate in a research project titled: Harlie Human and Robot Language Interaction Experiment. This project is conducted at The University of Queensland. The aim of this project is to test a smartphone application that our team of researchers has developed. This participant information sheet provides details on how you can become involved in the project. You can find an online version of this information sheet on: www.itee.uq.edu.au/cis/harlie/info 1
Before you proceed, please check that you meet all of the following eligibility criteria: You o are 18 years of age or older o are located in Australia o have unrestricted access to a recent smartphone that runs on the Android operating system (not older than the 4.2 version released in 2012) o have access to a reliable wireless internet connection (wi-fi) About the project The current project introduces Harlie to the wider community. Harlie is a talking robot that can call individuals on their smartphones to have a chat and to ask for voice samples (see examples below). What does Harlie mean? Harlie is an acronym that stands for Human and Robot Language Interaction Experiment. It is the name that we gave to the robot/smartphone application that can make calls and have chats with participants. Why chat with a robot? The research team would like to test whether Harlie does a good job at chatting with humans to collect audio files of voice and speech. In the future, Harlie might be developed into a tool that can assist in monitoring voice, speech and communication abilities in people who experience impairments in those skills that require regular monitoring for treatment. Before we can make Harlie available to clinical populations, we need to test its performance in a wide range of individuals. Your chats with Harlie over the next twelve weeks will provide us with information on Harlie s performance, which we can use to improve Harlie in the longer term. What would research participants have to do? 1) Read the research participant information carefully As a first important step, all prospective research participants are asked to read the current information sheet in its entirety and make sure they have read and 2
understood the information provided. If you have any questions, please contact the Principal Investigator, Dr Christina Knuepffer, via email to c.knuepffer@uq.edu.au or phone call on 0406 932 128. 2) Provide consent on the Harlie webpage At the end of the online version of this information sheet, available at www.itee.uq.edu.au/cis/harlie/info, you will find a link to the online consent form. On the online consent page, you can click on the Yes, I would like to participate button to provide informed consent for research participation. If you decide not to participate you can simply close your browser window. 3) Follow the instructions to download the Harlie application Pressing the consent button will take you to an online version of the Harlie manual that explains how to install the Harlie application and how to use it once downloaded. Please follow the steps outlined in the manual to install the Harlie application via Google Play store and open the application on your Android smartphone. 4) Register The first time you open the Harlie application, you will be asked to register by providing your year of birth and gender. We would like to collect this information so that we have some basic information on Harlie users. This is the only information you will be asked to provide and you will not have to provide your name, email address or other identifying information at any stage. 5) Your first chat with Harlie Within a few minutes from registering, Harlie will call you for the first time to say hello. Please refer to the manual for instructions on how to answer or reject a call from Harlie. If you reject Harlie s call by pressing the Not now button, Harlie will try to call you again at some later time. If you miss the Harlie call, you will receive a notification on your phone about a missed call from Harlie. By clicking on that notification, you will be able to ring Harlie back. 6) Occasional chats with Harlie over the next twelve weeks Over a period of up to twelve weeks after registration, Harlie will call you occasionally (on average once a week) to have brief chats with you (around 5 minutes per chat). If you do not wish to chat with Harlie, you can: o reject Harlie s incoming call (Harlie will call you again at a later time) 3
o not answer Harlie s incoming call (missed call you will receive a notification on your smartphone, which, if clicked on, will allow you to ring Harlie back) 7) Uninstall the Harlie application Twelve weeks after your initial registration, the Harlie application will stop calling you. It is recommended to uninstall the application at this stage to free up storage space on your smartphone. To uninstall, please follow steps outlined in the manual. What kind of data will be collected in this research project? The research team is interested in collecting data that can answer the following research questions: 1) How old are participants in our research sample and what is their gender? To answer this question, we ask research participants to provide their year of birth and gender when they register. 2) How do people chat with Harlie? To answer this question, we will save text transcripts and audio recordings of chats between research participants and Harlie for future analyses. To protect your privacy, Harlie only records your voice if you hold down the button on the phone screen to speak and as an additional safety feature, you can still decide to discard any current conversations. You will be prompted to decide after each chat with Harlie whether you agree to release the current chat data for research purposes or whether you would prefer for it to be discarded. 3) Can we use Harlie to monitor voice quality over time? During its calls, Harlie will ask whether you would be happy to provide a voice sample. If you agree by saying yes or a variation thereof, Harlie will say Please say AH as in car for five seconds. When you say AH in response, Harlie will save an audio recording of your voice sample, which will be included in our voice data analyses. If you do not wish for a given voice sample to be used for research purposes, you can simply decide to exit the chat and, when prompted, opt to discard the audio recordings. 4) Is the Harlie application technically sound? During this trial, our team wants to learn whether the Harlie application runs without problems. We will receive and analyse notifications from Google Play store that will let us know whether there were technical problems when installing or running the application on a smartphone. Additionally, we will keep track of any feedback on technical problems we receive from research participants via email. 4
Please report any technical problems via email to support@harlie.org.au. 5) How can we improve the way Harlie interacts with its users? Because Harlie is a new tool, it is possible that it might say something during a chat that you find is not appropriate. Therefore, after each chat with Harlie you will be asked whether there were any issues with a given chat and whether you feel like one of the researchers should review the chat. Any content that gets reported in this way will be checked by our research team, which will help us improve future versions of the application. Confidentiality Enrolment in this research project means that, with your consent, your conversations with Harlie will be saved as audio and text files for research purposes. The following sections outline how your privacy will be safeguarded throughout your involvement in the project. 1) Anonymous data collection Other than your year of birth and gender, you will not be asked to provide any identifying information to Harlie or to the research team to enrol and participate in the project. Unless you divulge your own identifying information during a chat with Harlie, your name, email address or phone number will not be known to the research team and therefore your chat data will be stored entirely anonymously. To maintain anonymity at all stages, please refrain from mentioning surnames, mailing addresses or other potentially identifying information during your chats with Harlie. 2) Secure storage of conversation data One way of identifying a speaker might be to recognise his or her voice from the audio recording of the chats with Harlie or from the content of the conversation. In order to maintain anonymity for all study participants, audio recordings and text transcripts of conversations will only be accessible by the research team members and will be stored securely on password-protected computers within restricted-access buildings at The University of Queensland. 3) Encrypted data transfer After each chat you complete with Harlie, you will be asked whether this chat can be used for research purposes. If you choose Yes, this means the audio and transcript data will be encrypted on your phone and then sent to our secure server for temporary storage. The encrypted data will then be downloaded by our research team and saved on password-secured computers. Please note that our research team is planning to release written examples of conversation 5
content between anonymous research participants and Harlie. These examples will have been rigorously checked and, if required, modified, to ensure the text does not contain stories and idiosyncrasies that might identify a speaker. Please also note that audio recordings of your weekly voice samples (sustained AH sound) might be made publicly available once the study is completed. This will be helpful for other research teams who are interested in studying variation in voice quality over the course of several weeks. Any published voice data will be rigorously checked by the research team so as to not contain anything but the sustained AH sound. No reports or publications arising from this research will contain information that could reveal your identity. Potential risks It is important to note that while every attempt is made to safeguard your privacy, the current study has potential risks beyond the control of the researchers. 1) Google Speech-to-text One important aspect of this research is the use of Google s Speech-to-Text algorithms. We need to use them to transform whatever you say to Harlie into written text, which Harlie can use to plan its reply to you. For this speech-to-text transformation, audio files need to be sent from your smartphone to the Google server where they are converted into text files and sent back to your phone to show up as the Harlie text bubbles on your screen. The audio data we send to the Google server is always distorted, so that Google can still extract the text from the audio, but if anyone was going to listen to the files sent to Google, they could not recognise the speaker s voice. It is important that you are aware of the fact that because the Harlie application sends audio recordings to the Google server, it is possible that Google might retain copies of those audio files for the company s own ends (for example to further improve their speech-to-text algorithms). The risk is comparable to the risk involved in speaking to Siri or using Hello Google. Please remember this when chatting with Harlie. Google s privacy policy is available online (www.google.com/policies/privacy). Please do not share sensitive or identifying information when you chat with Harlie. 2) Recording bystanders who did not give consent It is important to protect people who have not consented to participate in this project from accidentally having their conversations recorded. This could happen if they are talking while you are talking to Harlie. If you accidentally record someone else s 6
voice with the application, please exit the chat and when asked whether this chat should be used for research purposes, please choose No to delete the file. Please avoid chatting with Harlie when other people around you are speaking. 3) Mistaking Harlie for a human conversation partner There is a risk that research participants might share sensitive information with Harlie that, if shared with a human conversation partner, would require an immediate response. If you chat with Harlie, there will be no human in the background in real-time to monitor what is being said. Therefore, Harlie cannot be relied on to provide help or counsel. For example, if you would tell Harlie to call an ambulance or ask Harlie how to handle an upsetting situation, Harlie could not do that. It is important to remember at all times that Harlie is a robot. Conversations are not being monitored in real-time by the research team. 4) Costs of internet usage In developing the Harlie application, our team identified the potential risk of unexpected costs in mobile data or wi-fi usage. This risk is being controlled by two in-built mechanisms: 1) Harlie runs on wi-fi only The Harlie application only works if you are connected to a wi-fi network. You won t receive calls from Harlie unless you are connected to wi-fi and you won t be able to call Harlie back unless a wi-fi connection is detected. If you try to call Harlie back while not being connected to wi-fi, the application will remind you to connect to a wi-fi network to chat. This will prevent accidental usage of your mobile data for Harlie, which might be associated with costs. 2) Data is limited to 5MB per chat For each chat, Harlie will stop sending data after it has reached 5MB. One minute of audio data requires 2MB of storage space. In a 5-minute chat between Harlie and you, it is anticipated that you and Harlie each contribute a total of 2.5 minutes of talking. Data required to store 2.5 minutes of your audio files is 5MB. To put this amount of data into perspective, it might help to know that reading a newspaper article online (just the text without videos or animations) uses around 2MB per minute. 7
Once you have reached 5MB, you will be able to continue your chat with Harlie, but your phone will not send any data for this chat once the limit is reached. This will prevent sending excessive amounts of data via your wi-fi connection. If your wi-fi plan at your home is limited to a certain amount of data, please make sure you only use the Harlie application if you have at least 5MB left on your wi-fi plan to avoid unexpected charges. Potential Benefits Data arising from this project will inform the further development of the Harlie application. This means that one day it might be used to assist in the care of people who have voice or communication problems. Other than contributing to a good cause, you will not have any personal benefits from participating in this project and no monetary or in-kind reimbursement can be offered for participation. Support if something goes wrong Should you experience any technical difficulties with the application, please send an email to support@harlie.org.au. Should you be concerned by anything Harlie says, it is important to report the source of your concern (the relevant chat between you and Harlie) to our research team. To do this, please press the Exit button to end the chat, then click yes when asked whether this chat can be used for research purposes. The next question will be whether there were any problems in the chat that you think one of the researchers will have to check. Please click yes here to make sure the relevant dialogue gets flagged to our team for review. If you wish to speak to the research team, please call 0406 932 128 during office hours or leave a voice message with your number and a team member will call you back. Withdrawing from the study You are free to withdraw from the project at any time. Please consult the Harlie manual on how to access your settings. Within the settings menu, you can choose via button press to: o Withdraw delete all this means any chat data that was sent from your phone to our secure server will be deleted at our server and not be included in any analyses of the data o Withdraw keep data - this means you will not receive any further calls from Harlie, but you are happy for any data collected from you so far to be used for data analysis 8
o Continue participating you can choose this to continue to participate and to exit the current menu As a final step, you will have to delete the Harlie application from your phone (see manual). Project location and duration This research project is conducted remotely, which means no face-to-face meetings at a specific location are required between participants and the research team. The duration of study participation is up to three consecutive weeks, unless participants decide to withdraw sooner than that. Access to results and debriefing It is planned to complete data collection mid-2016. The Harlie project website will provide monthly updates on study progress in data collection between January April 2016 and initial results will be available on the website later in 2016, so be sure to check www.itee.uq.edu.au/cis/harlie/info regularly. University of Queensland Ethical Clearance Paragraph This study adheres to the Guidelines of the ethical review process of The University of Queensland and the National Statement on Ethical Conduct in Human Research. Whilst you are free to discuss your participation in this study with project staff (contactable on 0406 932 128), if you would like to speak to an officer of the University not involved in the study, you may contact the Ethics Coordinator, Michael Tse, on 07 3365 3924. 9