EYE TRACKING IN GEO DATA MANAGEMENT. Dr. Kristien Ooms; Ghent University

Transcription

1 EYE TRACKING IN GEO DATA MANAGEMENT Dr. Kristien Ooms; Ghent University

2 WHAT IS EYE TRACKING? Tracking the user s eye movements Sampling rate (times/second) Current location of eyes on screen/picture/etc. (x,y,t) raw data Time Type Trial L POR X [px] L POR Y [px] SMP 1 589,64 590, SMP 1 586,6 587, SMP 1 824,04 396, SMP 1 589,08 584, SMP 1 592,91 580, SMP 1 588,32 578, SMP 1 594,35 580, SMP 1 594,57 579, SMP 1 598,26 575, SMP 1 598,33 571, SMP 1 597,96 569, SMP 1 597,92 571, SMP 1 600,35 570, SMP 1 601,55 571, SMP 1 603,14 568,78 Metrics and measurements Deriving meaningful metrics from raw data - fixations, saccades, smooth pursuit Study design? Medium: paper, screen, etc.? Topic: VR, websites, simulators, maps, etc. Analysis: qualitative, quantitative, visual, statistical, etc. 2

3 A LITTLE BIT OF HISTORY Earliest research: Basis facts about eye movement discovered Phase with more applied focus, little research Buswell: How people look at pictures (1935) Tinker (1946) - Cannot learn much from eye tracking data - Limits of technology Fits et. al (1950) - Study eye movements of pilots in cockpit - First use in usability engineering Clear visualization of eye movements Yarbus (1967) Shows importance of eye movement recordings Source: schirillo/articles/buswell,% pdf Yarbus (1967) 4

4 A LITTLE BIT OF HISTORY Recent evolution 1970s: - Improvements in eye movement recording systems - Advances in psychological theory 1980s: Use of eye tracking in real time -Human-Computer interaction -Disabled users 1990s Solving usability problems -Internet, websites, s, video-conferencing, 5

5 WHAT IS EYE TRACKING? Video-based combined pupil and corneal reflection Gives point of regard (POR) measurements! - Point where user is looking - Gaze position - Pixel coordinates screen coordinates Corneal reflections (from infra-red light source) - Purkinje reflections or images - Eye rotations: relative positional difference with pupil center - Appropriate callibration: determining user s POR 6

6 WHAT IS EYE TRACKING? Video-based combined pupil and corneal reflection 7

7 DEMO Measurements: Points Of Regard at certain sampling rate - Calibration! - x, y: screen coordinates - Timestamp - Huge amount of raw data Deriving metrics: - Fixations, Saccades, (Smooth Pursuit) DESIGNING AND CONDUCTING USER RESEARCH 8

8 WHAT IS EYE TRACKING? Metrics: Fixations - Stable relative position pupil corneal reflection dispersion =??? (40px; 0.5 visual angle; ) - During certain period minimum duration =??? ( ms) Saccades: - Rapid eye movements - Reposition of fovea - Person does not see anything during saccade 9

9 WHAT IS EYE TRACKING? Metrics meaning? Link eye movements - attentive behavior - Can shift attention without movement of the eyes! - Central and peripheral vision - Attention precedes a saccade to a certain location - Complex task link is very tight - Need of peripheral vision - Need of attention 10

10 WHAT IS EYE TRACKING? Metrics meaning? Link eye movements - attentive behavior Data Interpretation Information processing is guided by higher level mental processes. When we construct our perception drawing on our past experiences and expectations The most basic sensation and perception. Entry Level sensory analysis 11

11 WHAT IS EYE TRACKING? References: Book of - Holmqvist et al. (2011) - Duchowski et al. (2007) Jacob & Karn (2003) - 20 different usability studies - Most commonly used metrics: Number of fixations, overall Gaze % (proportion of time) on each of the AOIs Fixation duration mean, overall Number of fixations on each of the AOI Gaze duration mean, on each of the AOI Fixation rate,overall (fixation/saccades) 12

12 WHAT IS EYE TRACKING? Related to Fixations (Overview by Poole & Ball, 2005) 13

13 METRICS & MEANING Related to Saccades (Overview by Poole & Ball, 2005) 14

14 STUDY DESIGN Other methods Qualitative vs. Quantitative Questionnaires Thinking aloud Response time measurements Sketching Scoring Mouse & keyboard logging Observation Interview EEG 15

15 STUDY DESIGN Software Setting up experiment Recording data Interpretation raw data Analyses Vendor specific Open Source Statistical Packages Spatial analyses 16

16 EYE TRACKING AT THE DEPARTMENT OF GEOGRAPHY, GHENT UNIVERSITY PROF. DR. PHILIPPE DE MAEYER PROF. DR. NICO VAN DE WEGHE PROF. DR. VEERLE VAN EETVELDE DR. KRISTIEN OOMS DR. RASHA DEEB LIEN DUPONT ANNELIES INCOUL PEPIJN VIAENE LIESELOT LAPON

17 LANDSCAPE PERCEPTION Lien Dupont Which elements in a landscape catch the attention and in which context are they most eye-catching? Important for the location of new infrastructures Observations of landscapes are influenced by Observer Landscape Representation 18

18 Experiment 3 Experiment 2 Experiment 1 LANDSCAPE PERCEPTION How do people observe landscapes in general? Influence of the photograph properties? Focal length, horizontal and vertical view angles Influence of the landscape characteristics? Degree of openness Degree of heterogeneity Influence of the social/professional background of the observer? Landscape experts versus novices Influence of type of landscape? Degree of urbanisation Landscape experts versus novices Predict viewing pattern? 19

19 LANDSCAPE PERCEPTION Study Design Experiment 1 a) Panoramic photograph b) Standard photograph Focal length Horizontal view angle Vertical view angle 50mm 70 20,9 50mm 31 20,9 c) Zoom 1 70mm 22,4 15 d) Zoom 2 100mm 15,8 10,5 e) Wide angle photograph 18mm 75,1 54,3 18 landscapes 90 photographs in total 23 participants (geographers) 20

20 Enclosed Semi-open Open Homogeneous Heterogeneous 90 photographs in total 21 landscape expert participants 23 novice participants 21

21 LANDSCAPE PERCEPTION Eye tracking technology Non-portable RED-system (SMI) Eye tracking experiments Random order 5 or 10 seconds per photograph Free-viewing Measured eye tracking metrics Fixations: number, duration (ms) Saccades: number, amplitude ( ), velocity ( /s) Derived products: focus maps 22

22 LANDSCAPE PERCEPTION Panoramic More fixations Shorter saccades More information extraction Shorter fixation duration Easier information extraction More saccades Larger saccades Faster saccades Open Less & longer fixations Less saccades Weaker visual exploration Homogeneous Less fixations Less & longer saccades Weaker visual exploration Stronger visual exploration 23

23 LANDSCAPE PERCEPTION More fixations & saccades Shorter fixations Expert Scan paths Novice Less fixations & saccades Longer fixations Longer scan path Focus maps Shorter scan path Larger visual span Smaller visual span Larger Vorornoi cells Voronoi cells Smaller Voronoi cells 24

24 Saliency map Focus map 1050 x 1680 matrices Correlation between focus maps and saliency maps? 25

25 INFLUENCE OF WIND TURBINES? Fanny Van Den Haute Sustainable energy >> wind turbines >> spatial planning Appropriate in the landscape? Visual impact? Research Questions How do people look at a landscape with wind turbines? Is there a difference before and after placement of the wind turbines? Is there a difference due to personal characteristics (expertise)? Does the type of landscape play any role in this? 26

26 INFLUENCE OF WIND TURBINES? Stimuli Panoramic photos Simulations in photoshop 5 different landscape types 60 pictures in total 7 seconds free viewing Participants 15 experts 29 non-experts 27

27 INFLUENCE OF WIND TURBINES? Wind turbine Time to first fixation % longest viewings Number of fixations Fixation durations Wind turbine vs. other vertical objects Simulation of wind turbines in same landscape Experts vs. non-experts Type of landscape 28

28 INFLUENCE OF WIND TURBINES? WIND TURBINES HAVE A VISUAL IMPACT EXPERTISE HAS NO INFLUENCE ON VIEWING PATTERN TYPE OF LANDSCAPE HAS INFLUENCE ON VIEWING PATTERN 29

29 EFFICIENCY OF THE TRIANGULAR MODEL? Pieter Laseure Evaluate added value of the Triangular Model to depict time intervals, compared to the traditional Linear Model 30

30 EFFICIENCY OF THE TRIANGULAR MODEL? LM 25 novice participant; some removed 3 expert participants 8 stimuli & questions for LM 8 stimuli & questions for TM Similar questions Mixed Alternate TM Quantitative analyses Response time Score Fixation duration Saccadic length Qualitative analyses 31

31 EFFICIENCY OF THE TRIANGULAR MODEL? Students response time Students saccadic length Students nr of fixations per second Students score Students fixation duration Participants preference and score attributed to the models GROUP nr AVG. SCORE LM AVG. SCORE TM PREFERENCE Students 25 5,48/10 8,3/10 TM (25/25) Experts 3 4,75/10 8/10 TM (3/3) 32

32 EFFICIENCY OF THE TRIANGULAR MODEL? 33

33 EFFICIENCY OF THE TRIANGULAR MODEL? Part. Gender SCANPAD STRING P01 P02 P03 P05 P06 M F M F F MMBACCDEDCCCCDDEEBBBBBCBCDEDDE EDDSWWRSSSSSSSSSSSSSSNSRWSSSSS SSSWWSSMNSSDEEDCCDDDEFDDRSXWS MLAABBBBCCDDDDDDDEDEEDDDWWXSSR RRSSSSSSSSWCDEEXWSXSSWXSSSSSSS WSSSSSSSNSRDEBDDRSSSSSNNSSSRRM MLRRNSSWXXXXWXDDEWSSSSSSNSNSSS SWNSSSSS MMHBABBCDDCCDERWSSSSSXXIDEBBBBC CCCDDDEESSSXXRSSSSSSSXDESRRWSSS SNSSSSSSSD MMLBCCCCDDDDEENXXWSSSSSSSSSSXW RCDDCBCBBRSSSRSWWRMRLLIRRWWR MMBBABBCDDDEEDEDEWWWWWXSSSSSS SRSSSSSWSSSXXWSSWN Scanpad String Similarities 34

34 EFFICIENCY OF THE TRIANGULAR MODEL? 35

35 READING PAPER VS. DIGITAL MAPS Annelies Incoul Paper versus digital maps Drawbacks of digital maps: Resolution Colour ranges Dimensions Same information displayed differently Eye tracking Register the users eye movements (Point of Regards, POR) Users cognitive process compare the users attentive behaviour 36

36 READING PAPER VS. DIGITAL MAPS Participants 32 Master students or researchers Department of Geography, Ghent University Similar domain knowledge in geography and cartography Familiar with the design of the Belgian topographic maps Stimuli 6 topographic maps on 1 : Regions in the Southern part of Belgium Two similar groups of participants Three paper and three digital maps (alternately) 37

37 READING PAPER VS. DIGITAL MAPS Task Visual search Locate three labels in the map image Questionnaire - Background information - Familiarity with the depicted regions - Search strategy Apparatus and Set-up Eye tracker: SMI RED system 120Hz 50 inch television screen Stand alone mode 38

38 READING PAPER VS. DIGITAL MAPS Creating the gridded visualisation Areas Of Interest (AOIs) Fixation counts and distribution Grid of 32 x 22 cells AOIs of 40 x 40 pixels 39

39 READING PAPER VS. DIGITAL MAPS Fixation count Mean search times (P = > 0.05) Fixations per second (P < 0.000) Digital maps were less difficult to interpret than paper maps Fixation duration Mean fixation duration (P = > 0.05) Shorter saccades digital maps 1 paper digital paper digital paper digital paper digital

40 DESIGN OF MAP LABELS Rasha Deeb Typography on maps Semiotics according to Bertin Bold, italic, shape (font), orientation, etc. Preference? Efficiency? Lettering system? Colour? 41

41 DESIGN OF MAP LABELS Research Questions Influence of complementary colors (background-label) on the users search efficiency? Is this further influenced by the user s characteristics (gender and expertise)? Are the users preference and search efficiency linked? The findings are compared to the traditionally black labels 42

42 DESIGN OF MAP LABELS 31 participants 15 experts - 7 females - 8 males 16 novices - 7 females - 9 males 43

43 STUDY DESIGN Color system Design conditions Display conditions HSV RGB CIE XYZ Color No. H S% V% R G B L* (D65) a* (D65) b* (D65) X Y Z 1 0, Black

44 DESIGN OF MAP LABELS Colour difference ΔE* ab= {(ΔL*) 2 +(Δa*) 2 +(Δb*) 2 } 1/2 where: ΔL*= L foreground * - L background *; Δa*= a foreground * -a background *; Δb*= b foreground* -b background *. Colour difference vs. average fixation count per second 45

45 DESIGN OF MAP LABELS Luminance difference ΔY= Y foreground Y background calculated from the measured Y-value in the XYZ-system luminance difference vs. the target fixation duration 46

46 LANDMARKS IN INDOOR NAVIGATION Pepijn Viaene What is a landmark? = a wayfinding tool a location or a direction view-action pair How to identify a landmark? Asking observers picture based object recognition, verbal protocols, verbal eye-catcher detection, Wizard of Oz Prototyping, picture based object description... Quantifying = object + saliency» Visual Semantic Structural 47

47 LANDMARKS IN INDOOR NAVIGATION Study Aim & Design thinking aloud [CTA] [CRTA] eye-mind hypothesis saliency = eye catching eye tracking [fixation locus] [duration] 48

48 LANDMARKS IN INDOOR NAVIGATION [CTA (x2)] [CRTA ] 13 recordings 1924 verbalisation segments 49

49 LANDMARKS IN INDOOR NAVIGATION [CTA (x2)] [CRTA ] 13 recordings 50

50 LANDMARKS IN INDOOR NAVIGATION Analysing the eye tracking data 51

51 LANDMARKS IN INDOOR NAVIGATION Results 41 % Referral to a landmark 59 % No referral to a landmark 52

52 LANDMARKS IN INDOOR NAVIGATION Conclusions: For the identification of (indoor) landmarks eye tracking can provide qualitative and complete data, in addition verbal protocols can clarify specific fixations. 53

53 MAPS, HOW DO USERS SEE THEM? Kristien Ooms Research Aims: How do map users Read Interpet Store Retrieve information on digital cartographic products? Advice for design (syntax, semiotics) of digital cartographic products: Guidelines Implement in online tools... 54

54 MAPS ARE VISUAL Eye Tracking Evaluate maps: UCD - Log users Point of Regard Location Duration in screen-coordinates (px) - Combination with other methods Reaction time measurements Thinking alound Sketch maps Questionnaires 55

55 borderdesign totaldesign original view PHD RESEARCH Basic map design Expert vs. novices Label placement 56

56 PHD RESEARCH Complex map design Expert vs novices Adaptations in symbology Mirroring of map objects... 57

57 PHD RESEARCH 3D gridded visualisation Average total fixation duration Average fixation duration per fixation 58

58 PHD RESEARCH Gridded visualisation: statistical comparison Statistical comparison (ANOVA) 59

59 PHD RESEARCH Scanpaths 60

60 PHD RESEARCH Thinking aloud Word segmentation (count in ) Based on frequency Based on theme 61

61 PHD RESEARCH Thinking aloud Full thought - 4 Levels of codes: Level 1: Map Level Orientate Execute - Evaluate Level 2: Item Level Gather Thougts Draw Correct - Evaluate Level 3: Confidence Level 4: Actions Confident Neutral Not Confident Check Correct Draw Erase Fill Colour Talk Take Pencil - Time ratio for each code: [0-1] Psychological Theories Coding Scheme Transcriptions (Raw Protocols) Psychological Model Task Analysis Proposed Codes Segmented Protocols T H E O R Y Coded Protocols USER DATA 62

62 PHD RESEARCH Sketch maps Order of drawing Scores on maps Questionnaire Stated confidence 63

63 MAPS ARE INTERACTIVE Maps on the Internet/Web Typical user interactions - Panning changing extent - Zooming changing scale & extent Influence on users cognitive processes? Benifical for user? e.g. memory, change blindness, Read Interpet Store Retrieve 64

64 EYE TRACKING & INTERACTIVITY? Level of experimental control High: simulations of interactions same stimuli high comparability easy to analyse Low: free interactions different stimuli low comparability difficult to analyse At certain timestamp: - different scale - different extent for each participant Less intrusion on cognitive processes Higher realism vs. ecological validity 65

65 LOGGING INTERACTIVITY? Mouse actions Panning Zooming Mouse down Mouse up Scroll wheel Time stamp & location (x and y in px) Eye Tracking software Existing tools APIs web mapping software Javascript, AJAX, PHP, SQL, DB (with proxy server) Desktop tool based on JAVA: JNativeHook based on Python: PyHook 66

66 EYE TRACKING & INTERACTIVITY? Georeferencing eye movement data Changing point of origin Applying map projection formula Spherical Mercator (inverse) λ = λ 0 + x R φ = 2 tan 1 exp y R π 2 67

67 EYE TRACKING & INTERACTIVITY? Three eye tracking systems SMI RED 250 Tobii T120 SR Research EyeLink 1000 Panning 68

68 EYE TRACKING & INTERACTIVITY? Three eye tracking systems Panning 69

69 PANNING IN GOOGLE MAPS Panning along a route Count intersections Zoom level 13 Alteration map - satellite view 70

70 PANNING IN GOOGLE MAPS Find Belgium Zoom level 7 Alteration map - satellite view Start: Fiji Start: Quttinirpaaq NP Canada 71

71 PANNING IN GOOGLE MAPS 72

72 PANNING IN GOOGLE MAPS 73

73 PANNING IN GOOGLE MAPS 74

74 PANNING IN GOOGLE MAPS mouse key down mouse key up SpaCoast SpaAarlon Fiji-Belgium Canada-Belgium 75

75 RESULTS 76

76 RESULTS 77

77 SIDE PROJECT: THE EYE TRIBE TRACKER Small-size & Low cost Easy transportable Use outside lab Do parallel tests Low cost = low accuracy, precision, reliability??? Compare with SMI RED

78 SIDE PROJECT: THE EYE TRIBE TRACKER 79

79 SIDE PROJECT: THE EYE TRIBE TRACKER Eye Tracking device Sampling Rate Recording software Processing software 80

80 SIDE PROJECT: THE EYE TRIBE TRACKER 81

81 SIDE PROJECT: THE EYE TRIBE TRACKER Recorded time intervals between samples at 60 Hz Time interval (ms) SMI 60Hz (%) ET 60 Hz (%) [18-20] ]20-30] 0 0 ]30-40] > Hz = once every ms Statistical comparison of the registered offset values dist distx disty M Med SD M Med SD M Med SD ET , SMI Med-test Mann-Whitney U test 82

82 FUTURE PLANS Zooming? In theory: same concept, only change in R value Logging change in zoom levels - Scroll wheel Other map projections? In theory: same concept, only change in map projection formula Example: Google Earth - Spherical General Perspective Azimuthal projection 83

83 FUTURE PLANS Evaluation of Neogeography maps Lieselot Lapon Evaluation of maps on different devices Touch-interactions Interdisciplinary Project Marketing Department 84

84 FUTURE PLANS Evaluation of the school s textbooks Use of landmarks in pedestrian navigation systems Indoor vs. outdoor Urban vs. rural Evaluation of the new 25K symbology Together with 1 : : Paper maps, over whole Belgium 85

85 WANT TO KNOW MORE? Mail me: Have a look at the website of Bibliography Resources of past events Upcoming events: ICA Commission on Use, Users, and Usability Issues - AAG Special Sessions on Cognition, Visualization, and User Issues, US - Training Workshop on Designing and Conducting User Studies in conjunction with the ICC+GIS, Bulgaria Join the mailing list to stay up-to-date 86

86 EYE TRACKING IN GEO DATA MANAGEMENT Dr. Kristien Ooms; Ghent University