LECTURE 5 COMPUTER PERIPHERALS INTERACTION MODELS

Transcription

1 September 20 th, 2018 LECTURE 5 COMPUTER PERIPHERALS INTERACTION MODELS 1 Our Mental Conflict... 2 HCI & InfoVis 2018, Lecture 5 1

2 Our Mental Conflict... 3 Recapitulation Lecture #4 Knowledge representation Mental Models, definitions Mental Models and Design Mental Models and Visual perception Schemata, definitions & examples Metaphors, definitions & examples Introduction of key principle: Affordance 4 HCI & InfoVis 2018, Lecture 5 2

3 Mismatched Metaphor (1) 5 Mismatched Metaphor (2) From the Hall of Shame The use of the trashcan to eject the floppy disk? The trashcan on the desktop? HCI &b InfoVis 2017, fjv 6 HCI & InfoVis 2018, Lecture 5 3

4 Functional ~ Structural Mental Model Physical sectioning of an object Reconstructing the sections to 3D object Metaphor: section is represented by image: Image has a thickness All domains in that image have same thickness Builds up a model of the object Mental Model: How does the biologist look at this model How does the CS look at this model 7 Example from the Workbench 8 HCI & InfoVis 2018, Lecture 5 4

5 Metaphor Example (1) King HaraldBlåtandwas a legendary 10 th century Viking king who unified Scandinavian countries (N + DK): succeeded people talk to each other taken up by ICT industry as metaphor for wireless technology uniting technology, communication and consumer electronics. Blåtand = Blue tooth 9 Metaphor Example (2) Harald Blåtand (HB) continued policy of his father King of Denmark, then Norway Brought Christianity Tall man with a dark skin Old Danish Blå = dark skinned Tan = great man tooth hygiene not known Left stones with Old Norge Runes 10 HCI & InfoVis 2018, Lecture 5 5

6 The Metaphor for the ipad? What is your metaphor for your ipad? My metaphor is brain (external) My tablet metaphor would be like Jamie's Torch. Left it in office once, I was bereft, daughter said its like I'd left her there! Mobile shopper ;-) Definitely "life partner" ;-) Ascrapbook It s my daily computer. Once or twice a week I ll unsheaththe MacBook Air for something specific Mine would be a window into the world or a bridge to communicate with others via social networks HCI &b InfoVis 2017, fjv 11 Example HCI & IV Poiise Practical HCI & InfoVis 2018, fjv 12 HCI & InfoVis 2018, Lecture 5 6

7 Key concepts in HCI & IV VISIBILITY 13 Key Concepts of HCI/InfoVis Usability - Measurement of Success Cognetics- Locus of Attention Affordance what sort of operations and manipulations can be done to an object crucial is the Perceived Affordance Visibility (Transparency) mapping between controls and effects should be sensible and meaningful feedback Task orientation Fit, Analysis 14 HCI & InfoVis 2018, Lecture 5 7

8 Definition of Visibility The extent to which functions and their results are visible to the user as well as how easy it is to find and use these functions Interface components User Goals User Actions Results (controls) Functions of the system Feedback The User can reason - internalize knowledge in interface 15 Visibility, aka Transparency Phones vs. Cars What feedback do you get Important concept: constraining interfaces. restricting the way we can use them GUI: de-activation (greying-out) Categories of constraints: Physical -e.g. disk Logical -rely on knowledge of how world works Cultural - learned conventions ( arbitrary ) These categories are (implicitly) used in design; 16 HCI & InfoVis 2018, Lecture 5 8

9 Exercise What is the visibility of a file/copy operation In Linux In MacOS In Win 10 How would you rate the visibility of the Cut/Paste function in most software? 17 You and Human Centered Interaction! Learned major principles: expert mode Assess an interactive computer task Usability, recognize its usability Cognetics, assess Focus, Cognitive Load Affordance, are affordances well communicated Visibility, assess feedback and reachability Screen Layout, principles of Gestalt Logics, reasoning Mental Model & Metaphor 18 HCI & InfoVis 2018, Lecture 5 9

10 tobe or not to be interactive... COMPUTER PERIPHERALS FOR INTERACTION WITH GRAPHICAL INTERFACES 19 Computer consists of Elements Elements affect the interaction Internals RAM, CPU, Disk(s), Network Input devices Text entry Pointing Scanner, Camera, Microphone, Gesture devices (camera) Output devices Screen: small & large, LCD, Projection Displays Printer, writing terminal, paper Sound: headphone, speaker system Haptic feedback device Virtual reality special interaction and display devices physical interaction e.g. sound, haptic, bio-sensing, sensors 20 HCI & InfoVis 2018, Lecture 5 10

11 Text input QWERTY Keyboard Most common text input device Allows rapid entry of text by experienced users Keypress Closes connection, causing a character code to be sent Usually connected by cable, but can be wireless Standardised layout non-alphanumeric keys are placed differently accented symbols needed for different scripts minor differences between UK and USA keyboards QWERTY arrangement not optimal for typing layout to prevent typewriters jamming! Alternative designs allow faster typing but large social base of QWERTY typists produces reluctance to change. 21 Alternatives to QWERTY Alphabetic keys arranged in alphabetic order not faster for trained typists not faster for beginners either! Dvorak common letters under dominant fingers biased towards right hand common combinations of letters alternate between hands 10-15% improvement in speed and reduction in fatigue large social base of QWERTY typists produce market pressures not to change Q W E R T Y U I O A S D F G H J K L Z X C V B N M,. SPACE P 22 HCI & InfoVis 2018, Lecture 5 11

12 Qwerty vs. Dvorak Q W E R T Y U I O A S D F G H J K L Z X C V B N M,. SPACE P 23 Error Preventing Keyboard Haptic keyboard errors cost more pressure Pressure sensative keyboard Uses spelling checker on the fly Uses prediction and Bag-of-Words Tactile interface AI added to interaction 24 HCI & InfoVis 2018, Lecture 5 12

13 Demonstration Keyboard 25 Mouse Most common pointing device Handheld pointing device very common easy to use Metaphor (by accident) Two characteristics planar movement buttons originally 1 to 3 buttons on top making a selection, indicating an option, initiating drawing etc.) 26 HCI & InfoVis 2018, Lecture 5 13

14 Introduction of Mouse device 1967 built by Douglas Engelbart(Stanford) X-Y position indicator for a display system 1970 US patent Hand in hand with development hypertext Links should be pointed at Quintessential in development of hypertext and internet Tail = mouse metaphore 27 Archetypal mouse A true prototype 28 HCI & InfoVis 2018, Lecture 5 14

15 Engelbart s workstation Numerical Keypad isn and then, there was the Mouse 30 HCI & InfoVis 2018, Lecture 5 15

16 Advanced pointing devices LCD Digitizer: Tablet & Display 31 Advanced pointing devices LCD Digitizer: Tablet & Display Input of Text and Graphics Range of additional interactive modes Includes handwritten input Pen has additional affordances 32 HCI & InfoVis 2018, Lecture 5 16

17 Other TEXT input - Dasher Intelligent processing of vowel-consonant combinations. AI techniques applied in support of the interaction Color is used to indicate different groups of characters/fonts Range of applications in different devices 33 Dasher Demonstration 34 HCI & InfoVis 2018, Lecture 5 17

18 Haptic devices Cyber grasp, force feedback virtual glove Wii Mote Force feedback device, used in virtual design, medical applications. 35 Haptic Force Feedback Adding immersion to VR using Haptics; here, Force Feedback Arm Typical multi-modal Approach 36 HCI & InfoVis 2018, Lecture 5 18

19 Demonstration Haptic I/O 37 Haptic/Tactile Feedback 38 HCI & InfoVis 2018, Lecture 5 19

20 Ubiquitous I/O: Gestures 39 Other I/O devices... Leap Motion Myo Gestures, Pointing Gestures, Grasping Gestures, Muscle-tension 40 HCI & InfoVis 2018, Lecture 5 20

21 Expression: Emotion in I/O Can a computer machine Sense emotion Act upon emotion Artificial Intelligence A computer produces an intelligent responsein a given situation; obtained by learning Affective Computing A computer produces an emotive response; Combitation of sensing and acting HCI & IV , fjv Producing Emotions Kismet, by Cyntia Breazeal, MIT 42 HCI & InfoVis 2018, Lecture 5 21

22 What did we see Action to Robot triggers response Response is an emotion Robot interacts directly Robot produces adequate behavior 43 Models for Interaction, theory PARADIGMS, INTERACTION MODELS, INTERACTION STYLES 44 HCI & InfoVis 2018, Lecture 5 22

23 Interacting with computers Understanding human computer interaction Understanding the human, i.e. the user Motivation, Intention Expectation i.e. Norman 7 Stages Model Understanding the computer, i.e. the elements Mapping expectations Input and Output devices, paper, sensors, etc. Capacity, possibilities? memory, processing, networks 45 Paradigms An example that serves as pattern or model. Greek paradeigma = example Model that forms basis of a theory or collection of theories. HCI & InfoVis 2018, Lecture 5 23

24 Paradigms and HCI Predominant theoretical frameworks or scientific world views e.g., Aristotelian, Newtonian, Einsteinian (relativistic) paradigms in physics Understanding HCI history is largely about understanding a series of paradigm shifts Not all listed here are necessarily paradigm shifts, but are at least candidates History will judge which are true shifts 47 Interaction Paradigms Batch processing Timesharing Networking Graphical display Microprocessor WWW Ubiquitous Computing Grid Computing Cloud Computing Multi Modal Computing 1950 s 1960 s 1970 s 1980 s 1990 s 1995 s This era 48 HCI & InfoVis 2018, Lecture 5 24

25 Paradigm shifts Batch processing Timesharing Networking Graphical display Microprocessor WWW Ubiquitous Computing Grid Computing Multi modal Wait on turn, get output Wait, intermediate output Share resources, own CPU Direct manipulation, Visibility Personal computing, WIMP Globalization information Sensor driven, environmental CPU driven, includes others Multimedia Small Devices Augmentation & Virtualization 49 Multimodality A mode is a human communication channel PC, Computer Internet Smart Phone, Tablet, etc. Emphasis on simultaneous use of multiple channels for input and output Multiple Displays Cell/Smart Phone, PC Sound devices + 50 HCI & InfoVis 2018, Lecture 5 25

26 Example Multimodal Interaction 51 Interactivity? Early interaction paradigms: batch processing punched card stacks or large data files prepared long wait. line printer output and if it is not right Now most computing is interactive rapid feedback = VISIBILITY (expected) the user in control (most of the time) doing rather than thinking 52 HCI & InfoVis 2018, Lecture 5 26

27 Important: Styles of Manipulation Office-direct manipulation user interacts with artificial world with artifacts Industrial indirect manipulation user interacts with real world through interface Issues.. feedback delays immediate feedback instruments interface system plant 53 Styles and Paradigms Simple paradigm, Single user interface Complex paradigm, Multi user interface Multi user, GRID Industry, science and home apps? Multi user, multi modal Industry, science and home apps!! Think of some examples Phone PC iphone Intelligent fridge (RFID tag technology) Etc.. 54 HCI & InfoVis 2018, Lecture 5 27

28 INTERACTION CONCEPTS I 55 Concepts of interaction domain the area of work under study (where) e.g. graphic design, infovis, interaction goal what you want to achieve (what) e.g. create a solid red triangle, glyph task how you go about doing it (how) ultimately in terms of operations or actions e.g. select fill tool, click over triangle Note traditional interaction use of terms differs a lot especially task/goal!!! 56 HCI & InfoVis 2018, Lecture 5 28

29 Interaction model Norman Task language: the language of the user Core language: the language of the system Seven stages 1. user establishes the goal 2. formulates intention 3. specifies actions at interface 4. executes action 5. perceives system state 6. interprets system state 7. evaluates system state with respect to goal Norman s model concentrates on user s view of the interface, i.e. the user image 57 Execution/Evaluation loop execution goal evaluation system user establishes the goal formulates intention specifies actions at interface executes action perceives system state interprets system state evaluates system state with respect to goal 58 HCI & InfoVis 2018, Lecture 5 29

30 Execution/Evaluation loop goal execution system evaluation user establishes the goal formulates intention specifies actions at interface executes action perceives system state interprets system state evaluates system state with respect to goal 59 Execution/Evaluation loop execution goal system user establishes the goal formulates intention specifies actions at interface executes action perceives system state interprets system state evaluates system state with respect to goal evaluation 60 HCI & InfoVis 2018, Lecture 5 30

31 7 Stages of (Inter)Action (Norman, 1986) Goals Intention Expectation Evaluation task language Action specification Interpretation core language Execution Mental activity Perception Physical activity 61 Application of 7 Stages Model Hampering Execution (gulf of execution) user s formulation of actions actions allowed by the system interface: reduce the gulf to be effective Hampering of Evaluation (gulf of evaluation) user s expectation of changed system state actual presentation of this state gulf should be as small as possible for effective evaluation 62 HCI & InfoVis 2018, Lecture 5 31

32 7 Stages Gulf of Evaluation/Execution Goals Gulf of Execution task language core language Intention Action specification Expectation Evaluation Interpretation Gulf of Evaluation Execution Mental activity Perception Physical activity 63 Errors using an Interface Slip understand system and goal correct formulation of action incorrect action Mistake may not even have right goal! Fixing things? Slip: better interface design (screens) Mistake: better understanding of system (metaphor) 64 HCI & InfoVis 2018, Lecture 5 32

33 Interaction framework Abowd& Beale Extension of Norman s 7 stages of interaction Abowd & Beale interaction framework has 4 parts user input system output task language, core language Each part has its own unique language S core O output I input U task Interaction translation between languages Problems in interaction = problems in translation 65 Using Abowd& Beale s model user intentions translated into actions at the interface translated into alterations of system state reflected in the output display interpreted by the user general framework for understanding interaction not restricted to electronic computer systems identifies all major components involved in interaction allows comparative assessment of systems an abstraction 66 HCI & InfoVis 2018, Lecture 5 33

34 Paradigm Shifts 67 Review #5 Computer Peripherals Interaction Hardware Paradigms, shifts Interaction models Norman Abowd& Beale Gulf of Execution Gulf of Evaluation Errors: slip, mistake 68 HCI & InfoVis 2018, Lecture 5 34