Issues. Thursday September 21, Monday September 25, Monday October 30, Explanation Practical. Team formation (in preparation -

Transcription

1 Issues Thursday September 21, 2017 Explanation Practical Monday September 25, 2017 Team formation (in preparation - Monday October 30, 2017 Written Exam (extra) HCI & InfoVis 2017, fjv 1

2 September 18 th, 2017 LECTURE 4 HISTORY, MENTAL MODELS HCI & InfoVis 2017, fjv 2

3 Recapitulation Lecture 3 Principles of Reasoning Principles of Human Vision Principles of Gestalt Layers of Vision Principles of Human Hearing Principles of Human Touch Haptics 1 more Key principle of HCI Affordance HCI & InfoVis 2017, fjv 3

4 A self explanatory Gestalt What do you see HCI & InfoVis 2017, fjv 4

5 A self explanatory cognition Read this text aloud Perceptual Set, link between expectation and perception. Paris in the the spring HCI & InfoVis 2017, fjv 5

6 Perceptual Set It stresses the idea of perception as an active process involving selection, inference and interpretation. Perceptual set is a bias or readiness to perceive certain aspects of available sensory data and to ignore others. -filtering!- How proficient are we in filtering? HCI & InfoVis 2017, fjv 6

7 Example Perceptual Set I2 I3 I4 HCI & InfoVis 2017, fjv 7

8 Example Perceptual Set A I3 C HCI & InfoVis 2017, fjv 8

9 Example Perceptual Set I2 A I3 C I4 Minturn & Bruner, 1951 HCI & InfoVis 2017, fjv 9

10 Key Concepts of HCI Usability Cognetics Locus of Attention Affordance what sort of operations and manipulations can be done to object crucial is the Perceived Affordance Visibility (Transparency) mapping between controls and effects should be sensible and meaningful feedback Task orientation Fit, Analysis HCI & InfoVis 2017, fjv 10

11 Definition of Visibility The extent to which functions and their results are visible to the user as well as how easy it is to find and use these functions Interface components User Goals User Actions Results (controls) Functions of the system Feedback The User can reason - internalize knowledge in interface HCI & InfoVis 2017, fjv 11

12 Visibility, aka Transparency Phones vs. Cars What feedback do you get Important concept: constraining interfaces. restricting the way we can use them GUI: de-activation (greying-out) Categories of constraints: Physical - e.g. disk Logical - rely on knowledge of how world works Cultural - learned conventions ( arbitrary ) These categories are (implicitly) used in design; HCI & InfoVis 2017, fjv 12

13 Exercise What is the visibility of a file/copy operation In Linux In MacOS In Win 7/8 How would you rate the visibility of the Cut/Paste function in most software? HCI & InfoVis 2017, fjv 13

14 What came first, how did it look, who were the pioneers. Things you sould know about graphical user interfaces but were afraid to ask about... (SHORT) HISTORY OF THE GRAPHICAL USER INTERFACE HCI & InfoVis 2017, fjv 14

15 GUI related Landmarks Dynabook Alan Kay (1977) VisiCalc Frankston & Bricklin (1977) Hypertext (keywords & cross references) Memex: Vannevar Bush (1945) Hopscotch, Rayuela, J. Cortazar (1963) Ted Nelson, coined the term (1965) WWW: Tim Berners-Lee (1990) Ubiquitous Computing Mark Weiser (1991) # HCI & InfoVis 2017, fjv 15

16 Memex & Hypertext Memex = Memory Extension Collection of aids to assist in making associative relations In its organization predecessor of hypertext. v.bush Ted Nelson coined Hypertext (1965) Continues to exist as XANADU # t.nelson HCI & InfoVis 2017, fjv 16

17 Landmarks in GUI: DynaBook DynaBook (1977), Xerox Parc (Palo Alto Research Centre) User interface is learning environment Introduced the GUI & Hyper Media Notion of handheld dynamic Book, medium for thought? 2009 HCI & InfoVis 2017, fjv 17

18 Landmarks in GUI: STAR STAR, Xerox Parc (1981) 1 st Commercial PC with WYSIWYG Designed for business professionals Complete implementation of DeskTop Metaphor Network & Printing integration Not a commercial success!? First system based on usability engineering HCI & InfoVis 2017, fjv 18

19 Landmarks in GUI: Apple Apple Lisa (1983) predecessor of Apple-Mac GUI s (simple but complete) WYSIWYG: what you see is what you get. One-button mouse First computers still equipped with 5.25 FDD HCI & InfoVis 2017, fjv 19

20 Evolving from Apple Lisa GUI Apple Lisa GUI MacPlus GUI HCI & InfoVis 2017, fjv 20

21 ... Evolved to The MacOS X GUI interface components. Typically holds a lot of different metaphores... Is the desktop metaphore still valid? HCI & InfoVis 2017, fjv 21

22 How we support our reasoning and thinking on artifacts. MENTAL MODELS METAPHORS AND HCI & InfoVis 2017, fjv 22

23 Contents Knowledge representation Schemata Mental models Relation Mental model to User Relation Mental model to Designer Metaphors HCI & InfoVis 2017, fjv 23

24 Case Study Food has an expiry date Do not buy food past it expiry date Remove food after expiry date Credit cards have an expiry date Can not use Credit Card after expiry date Throw away expired Credit Card certificates have expiry date Passport has expiry date What to do after certificate/passport expires? Throw away: will block with certificate! s become unreadable and unusable! will complicate obtaining a new passport! HCI & InfoVis 2017, fjv 24

25 Case Study & Mental Models Mental Model of expiry date Seems not to be context sensitive Food, Credit Card: same behavior certificate/ Passport: different behavior whilst same mental model is appealed on! Design requires a good Mental model Designers use it Users use it Based on former knowledge How do we deal with knowledge HCI & InfoVis 2017, fjv 25

26 Knowledge and HCI - InfoVis One should realize: Understanding of the way that knowledge is represented, organized and retrieved is important. It can be used so that then perhaps it is possible to develop interfaces that facilitate thinking and problem solving. Prediction of behavior of the machine Understanding of visualization, Enable reasoning (comp. Prediction human behaviour) HCI & InfoVis 2017, fjv 26

27 Knowledge Representation Symbolic Representation Cognition depends on rule-like manipulation of symbolic structures Imagists Images Picture representation Propositionalists Propositions Proprosition representation Sub-Symbolic Representation Connectionists: Connectionism distributed representations HCI & InfoVis 2017, fjv 27

28 Imagists Analogical representations underlie mental processing Analogical representations are picture like images (of the real world) Buying bread Putting bread on the table No thought without images Images and Imagery play a role in thinking and reasoning HCI & InfoVis 2017, fjv 28

29 Propositionalists Propositional representations underlie mental processing Proposition representations are abstract language-like statements that make assertions The bread is on the table The table is in the garden Images are a by-product HCI & InfoVis 2017, fjv 29

30 Connectionists Distributed representations Distributed representations are Networks of Nodes Knowledge is implicit in connections of nodes Images and Propositions are complementary Exist at higher level of representation Sub-symbolic: properties of neural network of nodes HCI & InfoVis 2017, fjv 30

31 Connectionism Visualized Knowledge is in the nodes HCI & InfoVis 2017, fjv 31

32 Experiments Are Analogical, Propositional and Distributed representation distinct forms of mental representation? In the following experiments the mental ability to rotate an object is addressed. (I) The more disparate the view, the more time it takes (P) Complexity, i.e. # propositions, affects time for mental rotation HCI & InfoVis 2017, fjv 32

33 Which images are the same?(1) Shepperd & Metzler Experiments, 1971 HCI & InfoVis 2017, fjv 33

34 Which images are similar to 1 HCI & InfoVis 2017, fjv 34

35 Which images are the same?(2) HCI & InfoVis 2017, fjv 35

36 Mental Manipulation of Objects Experiments by Metzler & Shepard ( ) Case study of rotation Rotation is a mental process With/Without Hidden line removal Small/Large rotations between similar objects Not the 2D picture was rotated, rather the underlying 3D object Rate of rotation ~ 60 /s The farther apart, the longer it takes. Effect of hidden lines: complexity! HCI & InfoVis 2017, fjv 36

37 Mental Images Visual Memory Mental images are transitory maintained by cognitive effort Relatively simple images can be kept Cognitive load is low Operations can be done on mental Images Visual imagery same neural machinery as vision fmri show same activations in brain Mental & external imagery can be combined Cognitive load is high, very active visual reasoning HCI & InfoVis 2017, fjv 37

38 Visual Memory has limited capacity Change blindness Changes in scenes/screens/displays unnoticed Consquence of limited capacity of visual memory Locus of attention not on the change Visual momentum Continuity in design Same in filming scene continuity Important for Interfaces, context of application Risk of changing the mental model Cognitive load HCI & InfoVis 2017, fjv 38

39 Change Blindness The Simons & Levin study 1998 There are many more examples, but this one represents the original research. HCI & InfoVis 2017, fjv 39

40 Mental Models Scottish psychologist Kenneth Craik (1943) Mind constructs small scale models of reality to anticipate events, to reason and to underlie explanation A mental model is acquired through Interaction Explanation A mental model is constructed from Perception Imagination Interpretation of discourse A mental model represents Explicitly what is true Not what is false HCI & InfoVis 2017, fjv 40

41 Schemata: definition A schemata is a network of knowledge based on previous experience Schemata are static, as a sequence of events is played back Schemata can sometimes be represented by a metaphor A script is a special schema, a scenario for a defined situation HCI & InfoVis 2017, fjv 41

42 Schemata: example Walking into a restaurant with a group of people Look at menu outside Enter with the party Wait for the waiter Be seated, be asked for a drink Choose from the menu and order Taking the train to Amsterdam CS from Leiden CS Make sure there is enough budget on your OV-card Look at time-table for next train Go to the platform, Check-in Wait for the train to arrive [this schemata has been re-inforced recently] HCI & InfoVis 2017, fjv 42

43 Mental Conflicts The scene looks ordinary but Actions on stairs confuse state HCI & InfoVis 2017, fjv 43

44 Mental Conflicts can be Disturbing HCI & InfoVis 2017, fjv 44

45 Mental model: definition Mental models are models people have of themselves, others and the things with which they interact. People form mental models through experience, training and instruction (D. Norman). A mental model is a dynamic mental representation allowing people to: make predictions about future states make inferences and imagine situations not experienced before. HCI & InfoVis 2017, fjv 45

46 Application of Mental Model Our mental models of how bicycles work can simulate this to know it will not work HCI & InfoVis 2017, fjv 46

47 Mental Mismatch Mental model elevator, actions correspond Not corresponding to stairs HCI & InfoVis 2017, fjv New mental model, other actions Action coupling: embodied cognition 47

48 Tasking and Mental Model Object-Action Model User knows which actions can be applied on objects e.g. word processor: words are objects select, copy, change font, delete etc. are actions State-Transition Model How actions changes the state of the model e.g. ATM: card input / PIN input / service select / amount request / cash withdrawal / card output / sales slip Complex systems have both e.g. PowerPoint presentation draw / text edit / slide edit / slide presentation HCI & InfoVis 2017, fjv 48

49 Different Mental Models Functional Mental Model Knowing what to do : the how Referred as the how-to-use-it model Context sensitive For a driver a functional model of a car is sufficient Structural Mental Model Understanding a structure: the why Referred to as the how-it-works model Context free For a mechanic a structural model of a car is a necessity HCI & InfoVis 2017, fjv 49

50 Applied to Design System Model Systems hardware & software components Designer Model Conceptual model of the designer System Image User interface (screen design, interface) User Model (= Mental Model) Conceptual model of the user (*) HCI & InfoVis 2017, fjv 50

51 Users & Designers Metaphors Graphics Icons Glyphs Geons Language Documentation Tutorials HCI & InfoVis 2017, fjv 51

52 Metaphor To map familiar to unfamiliar knowledge: helps understanding and learning in new domain The transference of the relation between one set of objects to another set for the purpose of brief explanation (Webster s revised) l. metaphora, fr. to carry over, transfer; meta: beyond, over HCI & InfoVis 2017, fjv 52

53 Analogy Analogy suggests that the mode of operation of the analogon is completely the same as the original A resemblance of relations; an agreement or likeness between things in some circumstances or effects, when the things are otherwise entirely different. (Webster s revised) 1. Similarity in some respect between things that are otherwise dissimilar: "the operation of a computer presents an interesting analogy to the working of the brain (WordNet) 2. (logic) inference that if things agree in some respects they probably agree in others (WordNet) HCI & InfoVis 2017, fjv 53

54 Metaphors in daily life Lakoff & Johnson...the way we think, what we experience, and what we do every day is very much a matter of metaphor.'' In our language & thinking - argument is war he attacked every weak point... criticisms right on target... if you use that strategy soccer is war! In our language & thinking time is money this GUI really saves me time installing the latest version was a waste of time on what subject did you spend most of your time HCI & InfoVis 2017, fjv 54

55 Metaphore or Simile A simile, aka open comparison, form of metaphor that compares two different things to create a new meaning. always uses "like" or "as" within the phrase and is more explicit than a metaphor. e.g. The spy was close as a shadow. She runs like a cheetah. metaphor and simile used to enhance creative expression HCI & InfoVis 2017, fjv 55

56 Interface: Metaphors The book metaphor Strength and weakness The desktop metaphor History and examples The internet metaphor Strength and weakness The spatial metaphor In InfoVis The social metaphor In development (cf later) HCI & InfoVis 2017, fjv 56

57 Composite Metaphors While analyzing a metaphor, often more than one metaphor is found. Example: desktop Menu, is a metaphor from a restaurant Button is a metaphor of an electrical device Users develop mental model to deal with composite metaphors HCI & InfoVis 2017, fjv 57

58 The Book Metaphor (Early application of) CD-Rom in multi media Content page Page numbering Index Help navigating the data Disadvantages of the book metaphor Not capable of dynamic look-up Turning pages does not fit with fast search a bit limited in current use HCI & InfoVis 2017, fjv 58

59 Modern Book Metaphor HCI & InfoVis 2017, fjv 59

60 Internet Metaphor Creating metaphor has come after the fact Internet is a phone Internet is a highway Internet is surfing Good interface Metaphors can not be applied to preexisting technologies So, Start with a good Metaphor from the beginning Transition of mail Has an address Send, Received, Forwarded, Redirected human computer interaction 2014, fjv 60

61 The Desktop Metaphor Started at Xerox PARC Xerox Star (next slide) Bitmapped screens made it possible Not meant to be a real desktop Idea is to organize information in away to allow people to use it comparable to the way they use information on their desktops Allow windows to overlap Make the screen act as if there were objects on it HCI & InfoVis 2017, fjv 61

62 Archetype Desktop Metaphor human computer interaction 2010, fjv HCI & InfoVis 2017, fjv 62

63 Xerox Desktop Metaphor Xerox PARC (Palo Alto Research Centre) and the Xerox Star HCI & InfoVis 2017, fjv 63

64 Steve Jobs visits Xerox Parc HCI & InfoVis 2017, fjv 64

65 Desktop Metaphor Apple took it further Waste basket, etc Microsoft brought to extremes Microsoft Bob a recognized failure Win 2000, XP Linux adopted it A variety of desktops is available to user Not depending on Linux implementation HCI & InfoVis 2017, fjv 65

66 The Story of Microsoft BOB Making the MS-Windows environment more accessible Introduced in 1995 Announced as: MS Bob is a new family member Officially declared dead in 1997 Introduce a Living Room Metaphor Clicking on items starts the program Task oriented (learned): most used in the living room Letter writer, Check book, Calendar, Address book Friends of BOB Rover (dog): easy to work with, tries to be your best friend Scuzz (mouse): Seldom helps, does not care. HCI & InfoVis 2017, fjv 66

67 Living Room Metaphor HCI & InfoVis 2017, fjv 67

68 Bob s Heritage MS 97 Paperclip Assistant MS Wizard Assistants Desktop themes (only the outside) Science, Desert, Jungle HCI & InfoVis 2017, fjv 68

69 InfoVis: Landscape Metaphor HCI & InfoVis 2017, fjv 69

70 InfoVis: Map Metaphor Stresses the distances in the graph (right), Clusters the nodes that are closer together HCI & InfoVis 2017, fjv 70

71 InfoVis: Botany, Tree Metaphor HCI & InfoVis 2017, fjv 71

72 InfoVis: Metaphors Link to Visual strong concepts Landscape, has Elevations Vegetations Land-use Map, has Cartography Distance Boundary Tree, has Branches Leaves Height HCI & InfoVis 2017, fjv 72

73 Review #4 Transparency (visibility) Knowledge representation Mental Models, definitions Mental Models and Design Schemata, definitions & examples Metaphors, definitions & examples User Interfaces Information Visualization Relations to Historic Landmarks in HCI/GUI HCI & InfoVis 2017, fjv 73