Measuring emotions: New research facilities at NHTV. Dr. Ondrej Mitas Senior lecturer, Tourism, NHTV

Size: px

Start display at page:

Download "Measuring emotions: New research facilities at NHTV. Dr. Ondrej Mitas Senior lecturer, Tourism, NHTV"

Suzan Fisher
5 years ago
Views:

1 Measuring emotions: New research facilities at NHTV Dr. Ondrej Mitas Senior lecturer, Tourism, NHTV

2 experiences are key central concept in tourism management one of three guiding research themes of NHTV

3 emotions make experiences valuable brief, intense reactions to specific things complex (mind, body, behavior) emotions are the value in tourism experiences fuel memory, (re)purchase decisions, loyalty, reputation, health

4 Emotion mangement Which emotions do you want which customers to feel at which touchpoint? Emotion measurement Which emotions do which customers actually feel when? Experience Marketing Using the right emotion to that triggers decisions Experience Design Intervening with people, interfaces, and setting to match the promised experience

5 Emotion mangement Do you plan long enough Which emotions do you want which into the future? customers to feel at which touchpoint? Emotion measurement Are you really open to Which emotions do which customers finding out? actually feel when? Experience Marketing Do you add value Using the right emotion beyond price? to that triggers decisions Experience Design Is there anything you Intervening with people, interfaces, can actually and setting change? to match the promised experience

6 Emotion mangement Which emotions do you want which customers to feel at which touchpoint? LEGO Serious Play workshops: Facilitation to inspire and formulate customer experience statements Contact: Xander Lub

right emotion to that triggers decisions Experience Design

7 CRADLE: Digital and blended experiences Storytelling: Creating emotion from your product DNA Experience Marketing Using the right emotion to that triggers decisions Experience Design Intervening with people, interfaces, and setting to match the promised experience

8 Emotion measurement Which emotions do which customers actually feel when?

9 Self-response questionnaires + inexpensive, flexible - validity problems Narrative interviews + precise, very flexible - validity problems Facial coding + time- and emotion-specific, highly accurate - clear view of the face and expression required Neuro and bio techniques + very time-specific, accurate, also unconscious/-expressed emotions - less emotion-specific, limited by context

10 A laboratory at the leading edge of emotion measurement A three-part laboratory in which partners can measure, design, and evaluate experiences Mobile emotion recording Measuring emotional reactions Virtual reality simulations

wearables and cameras Record emotional experiences

11 Part 1: Mobile emotion recording Measure GSR (sweating), heart rate, and facial expressions using wearables and cameras Record emotional experiences during leisure/tourism, as they happen Contact: Ondrej Mitas

completed soft selling training, further developing the experience of visiting the

12 Tourist Info Center as an experience A mobile emotion measurement study The Gent TIC facilitates quality experiences of this beautiful medieval city Employees recently completed soft selling training, further developing the experience of visiting the information center itself Thus, an interesting context for testing and refining experience measurement tools

expression analysis software students fitted with wristbands at beginning of day one student at a

13 Using the tools of the NHTV Experience Lab Empatica E4 wristbands on 7 student mystery visitors A variety of action and professional cameras pointed at students and TIC employee FaceReader facial expression analysis software students fitted with wristbands at beginning of day one student at a time sent on a mystery visit; face also recorded for realism, students then enjoyed day based on advice

Experience lab tools reveal the Gent TIC experience Facial expressions of visitors were neutral about half of the time, especially when looking at maps; positive about a quarter of the time,

14 Experience lab tools reveal the Gent TIC experience Facial expressions of visitors were neutral about half of the time, especially when looking at maps; positive about a quarter of the time, especially when starting and ending the conversation Facial expression of employee was most positive when talking with visitors Heart rate often showed a local peak during conversation, suggesting positive valence (Example participant below; orange line is heart rate) Enters office Begins conversation Ends conversation

15 Facial expression analyses show the importance of the beginning and end of the visit these are most positive 0.35 Valence (positivity) first quarter second quarter third quarter fourth quarter participant

16 Higher recommending visitors are more positive at beginning (TIC and Gent in general) Facial expression positivity of more and less Gent (the city in general) recommending visitors first quarter second quarter third quarter fourth quarter Facial expression positivity of more and less TIC recommending visitors recommend==8 recommend== first quarter second quarter third quarter fourth quarter recommend==8 recommend==10

picture of physical aspects of emotional responses

18 Part 2: Measuring emotional reactions Measure high-quality EEG (brain activity), GSR (sweating), and heart rate Accurate picture of physical aspects of emotional responses Neuroscience techniques for testing marketing Contact: Marcel Bastiaansen

19 What is Electroenceophalography (EEG)? Electrodes placed on the scalp in an elastic swim cap Amplifier strengthens the small changes in electrical potential due to brain activity Responses in this signal are averaged over many pictures and participants to show differences

20 When averaged over many pictures, differences due to emotion show up This difference shows valence (+/-) vs. emotionally neutral This difference shows strongly arousing vs. less arousing Time: 0 Time: 1 second We use facial coding or electrodes to determine which emotion Building AI algorithms at EUR to classify desire emotion Be careful with unsupported promises of commercial firms

photos, as well as usergenerated photo and video competitions (Fan van Zeeland) Strategic decisions

21 User generated and commercial photos in Zeeland An in-lab emotion reaction measurement study Zeeland destination marketing organization traditional marketing with TV commercials (Land in Zee) and stock photos, as well as usergenerated photo and video competitions (Fan van Zeeland) Strategic decisions among these approaches? Are TV commercials still effective, and are user-generated or stock photos more powerful?

22 EEG EEG Using the tools of the NHTV Experience Lab Consumer neuroscience approach using EEG Experimental design with different combinations of commercials and pictures Group 1 TV commercial Zeeland UG pictures of Zeeland Stock pictures of Zeeland UG pictures of Zeeland Stock pictures of Zeeland Group 2 TV commercial Honda UG pictures of Zeeland Stock pictures of Zeeland

23 Experience lab tools reveal the emotional impact of TV commercials Before Zeeland commercial After Zeeland commercial

simulated experiences Combine with EEG or wearables

24 Component 3: Interaction with technology Use VR and eye tracking to design and evaluate virtual or simulated experiences Combine with EEG or wearables for accurate emotion measurement Contact: Marnix van Gisbergen

amusement parks, a fundamental question is how people remember experiences

25 How we experience VR A virtual reality movie study (for PhD of Wim Strijbosch) For marketing and experience design in attractions such as amusement parks, a fundamental question is how people remember experiences Specifically, how do we pick experiences from our stream of consciousness to remember?

26 Using the tools of the NHTV Experience Lab Watching two versions (one suspenseful) of a VR film while EEG, skin, and heart rate signals are recorded Virtual reality stimulus Retelling the experience Self-report on retold experience 50 participants Virtual reality stimulus susp. Retelling the experience Self-report on retold experience

27 Experience lab tools reveal the VR experience

28 Experience lab tools reveal the VR experience Correlating self reported emotions to physiology to refine physical detection of emotions

29 Experience lab tools reveal the VR experience Comparing physical signals at key moments in film to determine which emotions mark beginning and end of experience

30 Manage customer experiences with us Excellent customer experiences require managing, measuring, and designing for emotions The future of emotion measurement is physical Our lab offers the most up-to-date emotion measurement and management process in leisure and tourism

31 To the demo Ondrej Mitas Wilco Boode

Measuring Behavior in Simulators Software tools and integrated solutions. Diana Demmer, MA Accountmanager NL Thursday, June 4, 2015

Measuring Behavior in Simulators Software tools and integrated solutions Diana Demmer, MA Accountmanager NL Thursday, June 4, 2015 Representatives today @ Marin Tobias Heffelaar, M.Sc. Project leader &