What s an input device. Input Technologies and Techniques. Input Device Properties. Property Sensed. What s an input device

Transcription

1 What s an input device Input Technologies and Techniques Input devices sense physical properties of people, places or things However, they do not operate in isolation, i.e. need visual feedback otherwise similar to a pen without paper What s an input device Evaluation and analysis of input devices Input Device States Interaction Modalities Must include: the physical sensor (positioning wheels) the feedback presented to the user (cursor) the ergonomic of the device (fits in hand) interplay between all the interaction techniques supported by a system (clicking, moving, selecting, etc.) Need an understanding of input technologies to design interaction techniques that match a user s natural workflow Hinckley, K., Input Technologies and Techniques, Chapter 7. Handbook of Human-Computer Interaction, ed. by Andrew Sears and Julie A. Jacko. Lawrence Erlbaum & Associates. 10/26/ Graduate Course in HCI 2 Input Device Properties Several properties characterize most devices: Property sensed Number of dimensions Indirect vs. direct Device acquisition time Gain Property Sensed Most devices sense Linear position (tablets sense position of pen) Motion (mice sense change in position) Force (Isometric joysticks, IBM Trackpoint) Angle or change in angle (rotary input) Absolute input device position sensing Relative input device motion sensing Relative device requires visual feedback but also can be inefficient due to clutching 10/26/ Graduate Course in HCI 3 10/26/ Graduate Course in HCI 4

2 Number of Dimensions Devices sense one or more input dimensions Two linear dimensions (mouse, x/y) Angular dimension (knob) 6 degree-of-freedom (magnetic tracker, senses 3- position dimensions and 3 orientation dimensions) Degrees of Freedom vs. Dimensions Usually confuse the idea of dimensions and degrees-of-freedom A pair of knobs is a 1D+1D device, mouse with scroll wheel is a 2D+1D multi-channel device Multiple degree-of-freedom devices sense three or more simultaneous dimensions of spatial position or orientation 10/26/ Graduate Course in HCI 5 the bat Ware, C. and Jessome, D. (1988), Using the Bat: A Six Dimensional Mouse for Object Placement. IEEE Computer Graphics and Applications. November 8-6, /26/ Graduate Course in HCI 6 Degrees of Freedom vs. Dimensions 3DOF Mouse Degrees of Freedom vs. Dimensions 4DOF Mouse A Two-Ball Mouse Affords Three Degrees of Freedom I. Scott MacKenzie, R. William Soukoreff, & Chris Pal CHI 97 Electronic Publications: Late-Breaking/Short Talks 10/26/ Graduate Course in HCI 7 Ravin Balakrishnan, Thomas Baudel, Gordon Kurtenbach, George W. Fitzmaurice. (1997). The Rockin'Mouse: Integral 3D manipulation on a plane. Proceedings of ACM CHI 1997 Conference on Human Factors in Computing Systems, p /26/ Graduate Course in HCI 8

3 Degrees of Freedom vs. Dimensions 5DOF Mouse Indirect vs. Direct Mouse is indirect, i.e. must move mouse to move pointer on screen Touch-screens or tablets are direct input devices, i.e. unified input and display surface Occlusion is typically a problem Hinckley, K., Sinclair, M., Hanson, E., Szeliski, R., Conway, M., The VideoMouse: A Camera-Based Multi-Degree-of-Freedom Input Device, ACM UIST'99 Symposium on User Interface Software & Technology, pp /26/ Graduate Course in HCI 9 10/26/ Graduate Course in HCI 10 Device Acquisition Time Acquisition time: average time to move hand to device Homing time: average time to return to a home position, i.e. mouse to keyboard For common desktop workflows, pointing and selecting dominate acquisition/homing time integration of pointing with keyboard may not improve overall performance Gain Control-to-display gain or C:D ratio distance moved by an input device/distance moved on the display Composite measurement taking into account device size and display size Gain has very little effect on the time to perform pointing movements not a commonly used metric 10/26/ Graduate Course in HCI 11 10/26/ Graduate Course in HCI 12

4 Pointing Devices Mouse: Invented in 67 Used for pointing Picks up changes in x,y As good as pointing with finger Integrated with buttons/wheels etc Trackball: Senses relative motion of partially exposed ball in 2DOF Engage different muscle groups than the mouse, but an alternative for those who experience discomfort Pointing Devices Isometric joystick: Force sensing Rate of cursor is proportional to the force exerted Returns to center when released Good when space is at a premium Isotonic joystick: Sense angle of deflection Different than isometric joystick J. Lipscomb and M. Pique (1993). Analog Input Device Physical Characteristics. SIGCHI Bulletin 25 (3): /26/ Graduate Course in HCI 13 10/26/ Graduate Course in HCI 14 Pointing Devices Touchpads: Small touch-sensitive devices found on laptops Use relative mode for cursor control Can operate in absolute mode by dragging finger and leaving it on edge of the pad Necessitates multiple clutchings Touchscreens/pen-operated devices: Fingers, or electromagnetic digitizers Parallax error, mismatch between sensed input position and apparent position Input Device States Disaccord between states of a GUI and states and events sensed by devices Difficult to support interface primitives such as click, drag, double-click, and right-click Useful to diagram device states Identifies relationship between events sensed by input device and interaction technique demands 10/26/ Graduate Course in HCI 15 10/26/ Graduate Course in HCI 16

5 Three-State Model of Graphical Input Input Device States Buxton's 3-state model for graphical input devices Expression of the operation of computer pointing devices in terms of state transitions Expressive vocabulary for exploring the relationship between pointing devices and the interaction techniques they afford 10/26/ Graduate Course in HCI 17 10/26/ Graduate Course in HCI 18 Input Device States Three-states: The states are Out of range (State 0), for clutching or repositioning a mouse on a mouse pad; Tracking (State 1) for moving a tracking symbol such as a cursor about a display Dragging (State 2) for moving an icon on the desktop or for grouping a set of objects or a range of text Input Device States Based on Buxton s model, the mouse & touchsensitive devices are a two-state device Seems simple and obvious but can add insight to the existing body of pointing device research can be extended to multi-button interaction, stylus input, and direct vs. indirect input 10/26/ Graduate Course in HCI 19 10/26/ Graduate Course in HCI 20

6 Activity: Input Device States Based on the previous state diagram, can you describe a limitation of touch-sensing input (PDA s) Input Device States Can fully capture core interaction by extending the 3-state model Do you know of a device that supports all 3 states 10/26/ Graduate Course in HCI 21 10/26/ Graduate Course in HCI 22 Evaluation and Analysis of Input Devices GOMS, Keystroke-Level Model Fitts Law Hick-Hyman Law Power Law of Practice Accot s Steering Law GOMS Models Goals, Operators, Methods, Selection rules Card, Moran, Newell, 1980/83 a priori assumptions and predictions wide coverage of various subtasks in HCI empirically proven learnable and usable in practice good approximation to actual experimental data 10/26/ Graduate Course in HCI 23 10/26/ Graduate Course in HCI 24

7 What is a GOMS model? Goals - the state the user wants to achieve i.e. find a website Operators - the cognitive processes & physical actions performed to attain those goals i.e. decide which search engine to use Methods - the procedures for accomplishing the goals i.e. drag mouse over field, type in keywords, press the go button Selection rules - determine which method to select when there is more than one available Assist with: Predicting usage patterns in editors Power key assignments to reduce typing keystrokes Predicting performance with hierarchical menus Goals Something user tries to accomplish i.e. copy a file, create a directory Often hierarchical May require sub-tasks to be accomplished Typically an action-object pair i.e. copy file Can be thought of in lay terms 10/26/ Graduate Course in HCI 25 10/26/ Graduate Course in HCI 26 Operators Actions user performs, or actions system allows user to take Operator is something that is simply executed e.g., press a key on keyboard Also action object pair e.g. press button, drag cursor Types of operators External operators Observable physical actions between user & system Mental operators Internal actions performed by users Non observable Contentious, hypothetical, guess? 10/26/ Graduate Course in HCI 27 10/26/ Graduate Course in HCI 28

8 Methods Well learned sequence of operators needed to achieve a goal External + mental operators If user can perform a task, it means they have a method for the task Method is routine cognitive/perceptual/motor skill No planning required to determine actions Selection Rules If more than one method, selection rule routes control to most appropriate method Often internal rules users follow to pick best method i.e., to delete a paragraph of text, there could be different methods depending on length of paragraph, with appropriate selection rules i.e. how many ways to delete a paragraph? Which is the most effective? 10/26/ Graduate Course in HCI 29 10/26/ Graduate Course in HCI 30 Four GOMS models Keystroke level model Some simplifying assumptions Pre-established operators for prediction CMN GOMS (Card Moran Newell) Psuedo code notation, strict structure NGOMSL (Natural GOMS Language) Program form, very general Breadth first expansion of top level goals into methods Only one to predict both performance and learning times CPM GOMS (Cognitive Perceptual Motor GOMS) Based on model-human processor Allows parallel performance of operators Keystroke Level Model Only address one aspect of task performance: time Predicts expert error-free task-completion time with the following inputs: a task or series of subtasks method used command language of the system motor-skill parameters of the user response-time parameters of the system Predict time an expert would take to execute the tasks Assuming no errors 10/26/ Graduate Course in HCI 31 10/26/ Graduate Course in HCI 32

9 KLM operators Six operators Keystroke Avg time determined by std typing tests Pointing Pointing with a mouse or other device on a display to select an object. Varies from seconds Homing Bring home hands on the keyboard or other device 0.4 seconds based on various studies Mental 1.35 seconds, experimentally determined Response Encoding Methods E.g., replace 5 letter word with another in a text editor Reach for mouse H mouse Point to word P word Select word K Home on keyboard H keyboard Call replace cmd M,K replace Type new 5 letter word 5K word T execute = T M + T P + 2T H + 7T k T execute = T K + T P + T H + T M + T R 10/26/ Graduate Course in HCI 33 10/26/ Graduate Course in HCI 34 Where does GOMS apply? Where users perform tasks at expert level users have mastered a skill users are not problem solving users know what to do, just act on the steps Using GOMS Qualitatively Used for designing training programs, help systems Focus design on problem areas Quantitatively Good predictions of performance time Maybe some predictions on learning 10/26/ Graduate Course in HCI 35 10/26/ Graduate Course in HCI 36

10 GOMS weaknesses The model applied to skilled users, not to beginners or intermediate model doesn't account for either learning of the system or its recall after a period of disuse skilled users occasionally make errors; however, the model doesn't account for errors. Model explicit about elementary perceptual and motor components. cognitive processes in skilled behaviour are treated in a less distinguished fashion Model does not address: mental workload functionality: which tasks should be performed by the system user fatigue Hick-Hyman Law Law for choice reaction-time, given in the form of a prediction equation Given a set of n stimuli (flashing objects), associated with n responses (selecting object), reaction time (RT) can be given as follows: RT = a + b log 2 (n) a, b empirically determined constants Examples include: selection time by phone operator when light behind button appears (Card et al., 1983) measuring and predicting time to select items in a hierarchical menu (Landauer & Nachbar, 1985) predicting text-entry rates on soft keyboards with non-qwerty layouts, since users have to visually scan the layout (MacKenzie et al., 1995, 1999) 10/26/ Graduate Course in HCI 37 10/26/ Graduate Course in HCI 38 A Quiz Designed to Give You Fitts Fitts.html Microsoft Toolbars offer the user the option of displaying a label below each tool. Name at least one reason why labeled tools can be accessed faster. (Assume, for this, that the user knows the tool and does not need the label just simply to identify the tool.) A Quiz Designed to Give You Fitts 1. The label becomes part of the target. The target is therefore bigger. Bigger targets, all else being equal, can always be acccessed faster. Fitt's Law. 2. When labels are not used, the tool icons crowd together. 10/26/ Graduate Course in HCI 39 10/26/ Graduate Course in HCI 40

11 A Quiz Designed to Give You Fitts You have a palette of tools in a graphics application that consists of a matrix of 16x16- pixel icons laid out as a 2x8 array that lies along the left-hand edge of the screen. Without moving the array from the left-hand side of the screen or changing the size of the icons, what steps can you take to decrease the time necessary to access the average tool? A Quiz Designed to Give You Fitts 1. Change the array to 1X16, so all the tools lie along the edge of the screen. 2. Ensure that the user can click on the very first row of pixels along the edge of the screen to select a tool. There should be no buffer zone. 10/26/ Graduate Course in HCI 41 10/26/ Graduate Course in HCI 42 A Quiz Designed to Give You Fitts Microsoft offers a Taskbar which can be oriented along the top, side or bottom of the screen, enabling users to get to hidden windows and applications. This Taskbar may either be hidden or constantly displayed. Describe at least two reasons why the method of triggering an auto-hidden Microsoft Taskbar is grossly inefficient. A Quiz Designed to Give You Fitts Screen edges are prime real estate. You don't waste an entire edge that could be housing a couple of dozen different fast-access icons just for one object, the Taskbar The auto-hidden Taskbar is entirely too easy to display by accident. Users are constantly triggering it when trying to access something that is close to, but not at, the edge The Taskbar would not have any of these problems, yet be even quicker to get to if it were located at any one of four corners of the display. Throw the mouse up and to the left, for example, and you'll have a taskbar displayed. Fast access without the false triggering 10/26/ Graduate Course in HCI 43 10/26/ Graduate Course in HCI 44

12 Fitts Law Robust and highly adopted model of human movement Originated as interest of applying information theory to the analysis and understanding of difficulty of movement tasks & human rate of information processing Used Shannon s law for information capacity C = B log 2 (S / N + 1)» S is the signal power and N is the noise power Based on the following analogies: Amplitude of aimed movement == electronic signal Spatial accuracy of movement == electronic noise 10/26/ Graduate Course in HCI 45 Fitts Law Described the analogy in two papers: a serial, or reciprocal, target acquisition task wherein subjects alternately tapped on targets of width W separated by amplitude A experiment using a discrete task, wherein subjects selected one of two targets in response to a stimulus light Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, Fitts, P. M., & Peterson, J. R. (1964). Information capacity of discrete motor responses. Journal of Experimental Psychology, 67, /26/ Graduate Course in HCI 46 Fitts Law Quantify a movement task's difficulty ID, the index of difficulty ID = log 2 (A / W + 1) (bits) A = amplitude, W = width of object Movement time to complete a task is predicted using a linear equation MT = a + b * ID (secs) a & b are empirically determined using linear regression Throughput (TP) or Index of Performance (IP) is TP = ID / MT (bits/sec) Fitts Law To determine a and b design a set of tasks with varying values for A and W (conditions) For each task condition multiple trials conducted and the time to execute each is recorded and stored electronically for statistical analysis Accuracy is also recorded either through the x-y coordinates of selection or through the error rate the percentage of trials selected with the cursor outside the target 10/26/ Graduate Course in HCI 47 10/26/ Graduate Course in HCI 48

13 Fitts Law Fitts Law Target 1 Target 2 Target 1 Target 2 Same ID Same Difficulty 10/26/ Graduate Course in HCI 49 Smaller ID Easier 10/26/ Graduate Course in HCI 50 Fitts Law Fitt s Law A (pixels) W (pixels) ID (bits) Device 'A' ER (%) MT (ms) Device 'B' ER (%) MT (ms) Target 1 Target Mean: Larger ID Harder 10/26/ Graduate Course in HCI 51 Example data sets for two devices from a Fitts' law experiment 10/26/ Graduate Course in HCI 52

14 Fitt s Law Fitt s Law If primary goal in Fitts law experiment is to determine performance between devices/interaction techniques, then throughput (TP) is best criterion TP = ID/MT If for a given device ID = 4.09 bits and task is executed in MT = 979 ms human rate of information processing for that task is 4.09 / = 4.18 bits/s or TP = 4.18 bits/s Mean throughput across all the A-W conditions for Device 'A' is TP = 2.40 / = 3.73 bits/s For Device 'B', TP = 2.40 / = 1.57 bits/s Using throughput we conclude users' performance with Device 'A' was about 3.73 / 1.57 = 2.4 times better than performance with Device 'B' 10/26/ Graduate Course in HCI 53 10/26/ Graduate Course in HCI 54 Setting it up MacKenzie, I. S. (1995). Movement time prediction in human-computer interfaces. In R. M. Baecker, W. A. S. Buxton, J. Grudin, & S. Greenberg (Eds.), Readings in human-computer interaction (2nd ed.) (pp ). Los Altos, CA: Kaufmann. [reprint of MacKenzie, 1992] Case Study #1: Text Entry Rates on Mobile Phones Can we predict text entry rate on mobiles using Fitts Law? Vary A,W values for at least 4 ID conditions Small A, small W Small A, large W Large A, small W Large A, large W 2-4 variations in between Clicking start position presents object to click on Record whether user missed Record time to click on stimulus Design with several repetitions and several blocks 10/26/ Graduate Course in HCI 55 10/26/ Graduate Course in HCI 56

15 Case Study #1: Text Entry Rates on Mobile Phones Two main approaches: Multi-tap: presses each key one or more times to specify the input character large overhead: 33 key presses 15 characters of text "on average" multi-tap method requires keystrokes per character q u i c k _ b r o w n _ f o x One-key disambiguation: Add linguistic knowledge to make best guess Can be ambiguous in some cases, have to correct q u i c k _ b r o w n _ f o x 10/26/ Graduate Course in HCI 57 Case Study #1: Text Entry Rates on Mobile Phones Text entry on a mobile phone, for example, consists of aiming for and acquiring (pressing) a series of keys "as quickly and as accurately as possible Time to press any key, given any previous key, can be readily predicted using Fitts' law For index finger input = MT = ID and for thumb input = MT = ID 10/26/ Graduate Course in HCI 58 Case Study #1: Text Entry Rates on Mobile Phones Case Study #1: Text Entry Rates on Mobile Phones Elements to build a textentry prediction model are: Information on position and size of keys (ruler) Letter assignment to keys (any phone) Relative probabilities of digrams (probabilities of letter pairs) in target language (sources) t-h or e-space have high P g-k or f-v have low P Space /26/ Graduate Course in HCI 59 A B C D Z A B C D Z Space Time to enter each i-j sequence is predicted using Fitts law giving MT ij, weighted by the probability of the digram in the target language P ij MT L = (P ij MT ij ) WPM = MT L (60 / 5) (avg 5 chars/word) Method Multi-tap - wait for timeout - timeout kill Predicted Expert Entry Rate (wpm) Index Finger One-key with disambiguation assumptions: - all words are in dictionary - when ambiguity arises the intended word is the most probable Thumb /26/ Graduate Course in HCI 60

16 Case Study: Using Fitts to redesign text entry Using Letter Frequency Nesbat, S. A System for Fast, Full-Text Entry for Small Electronic Devices, Proceedings of the Fifth International Conference on Multimodal Interfaces, ICMI 2003 (ACM-sponsored), Vancouver, November 5-7, MessagEase Onscreen Keyboard Example of an interface design which can be adapted to multiple devices /26/ Graduate Course in HCI 61 10/26/ Graduate Course in HCI 62 Nine Most Frequent Letters: Double Click Eight Less Frequent Letters: Two Taps E T N R O I A S H D C U P G B Q J 10/26/ Graduate Course in HCI 63 10/26/ Graduate Course in HCI 64

17 Remaining Nine Letters: Two Taps Adding Space, Shift, and Mode F M Y W V X K Z 10/26/ Graduate Course in HCI 65 10/26/ Graduate Course in HCI 66 Special Characters Soft Keyboard Design The same mapping used for letters Hard Key Soft Key 38 special characters entered by two taps; characters can be entered with combine. Most Frequent Letters Less Frequent Letters Double Click Two Clicks Single Tap Single Drag 10/26/ Graduate Course in HCI 67 10/26/ Graduate Course in HCI 68

18 Special Characters Entered with a single drag Optimization and Evaluation Exhaustively simulated all permutations of letters within each group The configuration with the max speed was selected 10/26/ Graduate Course in HCI 69 10/26/ Graduate Course in HCI 70 Fitts Law Digraph Probability Movement Time from one key to another: A B A B C D Z Space MT = a + b*log 2 (A/W+1) C D Z Space A W The probability P ij that letter j will follow letter i in a body of text: ΣΣP ij = 1 10/26/ Graduate Course in HCI 71 10/26/ Graduate Course in HCI 72

19 Performance Measure Hard Key Calculation of max theoretical entry speed: Movement Time MT = a + b log 2 (A/W+1) Total time (2 Clicks) CT = MT 1 + MT 2 Total time (Dble Click no movement) CT DC = 2a + b log 2 (A/W+1) Average Time CT av = ΣΣ(P ij CT ij ) Speed WPM = (1/ CT av ) (60/5) Performance Measurement Soft Key Most frequent characters Single tap: TL i = (1/4.9) log 2 [(D 0-i /W) + 1]; if D 0-i > 0, TL i = a; if D 0-i = 0 Less frequent characters Drag: TL jk = t 0-j + t down + t j-k + t up TL jk = (t 0-j + t down + t up ) + (t j-k + t up + t down ) (t down + t up ) TL jk = TL j + TL k a t 0-j: time to move to key j t j-k: time to move from key j to key k t down: time to move stylus down t up: time to move stylus up 10/26/ Graduate Course in HCI 73 10/26/ Graduate Course in HCI 74 Hard Key (Cell Phone) Comparison Example: Expanding targets 30 Theoretical 130% User Study 209% TO READ: McGuffin, M. & Balakrishnan, R., Acquisition of Expanding Targets. Proceedings of ACM Conference on Human Factors in Computing Systems (CHI) 2002, pages WPM WPM Multi-tap MessagEase 0 Multi-tap MessagEase 10/26/ Graduate Course in HCI 75 10/26/ Graduate Course in HCI 76

20 Example: Expanding targets Mac OS X dock Size of the interface widget (viewing region) changes dynamically Provide the user with a magnified target area at their focus of attention (area around the cursor) Expanding toolbar implemented in latest Apple OS X operating system Furnas Generalized fisheye views CHI 1986 Mackinlay, Robertson, Card The Perspective Wall CHI 1991 Bederson Fisheye Menus UIST 2000 Does this make acquisition easier? 10/26/ Graduate Course in HCI 77 10/26/ Graduate Course in HCI 78 Advantages and Disadvantages Advantages Icons are displayed in reduced size to solve the increasing number of commands and icons Larger amount of screen real estate devoted to the display of the underlying data Disadvantages Can reduce the user s ability to select the desired icon efficiently Fitts Law 3 different scenarios describing what Fitts Law is modeling Iterative corrections model Impulse variability model Optimized initial impulse model 10/26/ Graduate Course in HCI 79 10/26/ Graduate Course in HCI 80

21 Iterative Corrections Model States that the movement consists of many discrete submovements Each sub-movement takes the user closer to the target Sub-movements are triggered by feedback indicating the target has not been reached yet Impulse Variability Model Movement consists of initial impulse delivered by the muscles towards the target, flinging the limb towards the target Last part of movement time consists of limb coasting towards target Either type of explanation cannot explain the Fitt s Law completely given a range of tasks 10/26/ Graduate Course in HCI 81 10/26/ Graduate Course in HCI 82 Optimized Initial Impulse Model (1) What does Fitts Law really model? Most complete and successful explanation to the Fitts Law Open-loop W Combination of the iterative corrections and the impulse variability models movement is initiated towards the target task is completed if the movement lands at the target another movement is required if it lands outside the target same processes will be carried out until the target is reached Speed Undershoot Distance Closed-loop Overshoot 10/26/ Graduate Course in HCI 83 10/26/ Graduate Course in HCI 84

22 Expanding Targets Basic Idea: Big targets can be acquired faster, but take up more screen space So: keep targets small until user heads toward them Can this be used for devices with small viewing space? Click Me! Okay Cancel Experiment Goals The experiment was designed to answer the following questions for a typical expanding target selection task: 1. Can such a task be modeled by Fitts law? 2. If it can be modeled by Fitts law, is it possible to predict performance in such tasks from a base set of data where no expansion takes place? 3. Is movement time dependent on the final target width and not the initial one at onset of movement? 4. At what point should the target begin expanding? 5. Do different target expansion strategies affect performance? 10/26/ Graduate Course in HCI 85 10/26/ Graduate Course in HCI 86 Experimental Setup W Expansion: How? Experimental Setup Target Start Position A Animated Expansion 10/26/ Graduate Course in HCI 87 10/26/ Graduate Course in HCI 88

23 Experimental Setup Experimental Setup Expansion: How? Expansion: How? When? P = 0.25 Fade-in Expansion 10/26/ Graduate Course in HCI 89 10/26/ Graduate Course in HCI 90 Experimental Setup Experimental Setup Expansion: Expansion: How? When? P = 0.5 How? When? P = /26/ Graduate Course in HCI 91 10/26/ Graduate Course in HCI 92

24 Pilot Study 7 conditions: No expansion (to establish a, b values) Expanding targets Either animated growth or fade-in P is one of 0.25, 0.5, 0.75 (Expansion was always by a factor of 2) Pilot Study 7 conditions x 16 (A,W) values x 5 repetitions x 2 blocks x 3 participants = 3360 trials 10/26/ Graduate Course in HCI 93 10/26/ Graduate Course in HCI 94 Pilot Study: Results Pilot Study: Results Time (seconds) Time (seconds) A a + b log 2 ( + 1) W ID (index of difficulty) 10/26/ Graduate Course in HCI ID (index of difficulty) 10/26/ Graduate Course in HCI 96

25 Pilot Study: Results Pilot Study: Results A a + b log 2 ( + 1) W Time (seconds) ID (index of difficulty) 1 A a + b log 2 ( + 1) 2 W 10/26/ Graduate Course in HCI 97 Time (seconds) P = 0.25 static targets measured expanding targets lower bound expanding targets measured ID (index of difficulty) 10/26/ Graduate Course in HCI 98 Pilot Study: Results Pilot Study: Results Time (seconds) 1 P = 0.5 Time (seconds) 1 P = static targets measured expanding targets lower bound expanding targets measured ID (index of difficulty) 10/26/ Graduate Course in HCI static targets measured expanding targets lower bound expanding targets measured ID (index of difficulty) 10/26/ Graduate Course in HCI 100

26 Implications Pilot Study suggests the advantage of expansion doesn t depend on P So, set P = 0.9 and perform a more rigorous study Full Study 2 conditions: No expansion (to establish a, b values) Expanding targets, with Animated growth P = 0.9 Expansion factor of 2 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 102 Full Study Results 2 conditions x 13 (A,W) values x 5 repetitions x 5 blocks x 12 participants = 7800 trials Time (seconds) A, W values 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 104

27 Results Results Time (seconds) Time (seconds) ID (index of difficulty) 10/26/ Graduate Course in HCI ID (index of difficulty) 10/26/ Graduate Course in HCI 106 Results Results Time (seconds) 1.2 Time (seconds) 1.2 P = ID (index of difficulty) 10/26/ Graduate Course in HCI static targets measured expanding targets lower bound expanding targets measured ID (index of difficulty) 10/26/ Graduate Course in HCI 108

28 Implications For single-target selection task, Expansion yields a significant advantage, even when P=0.9 What about multiple targets? Implications for Design (1) Experimental results can influence the design of buttons, menus, or other selectable widgets Interface with multiple expanding targets does not need to predict cursor's trajectory to anticipate which widgets to expand Instead, just expand widgets as the cursor approaches them 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 110 Implications for Design (2) Expansion Strategies for adjacent widgets (e.g. toolbars) Expanding a widget around its center will cause overlap & occlusion with nearby widgets Expanding a group of widgets around a group s center Expand nearest widgets and move adjacent widgets away Expand nearest widgets, but allow some overlap as well as expand adjacent widgets so they are easier to see Summary Expanding targets acquisition of can be accurately modeled by Fitts Law User performance is aided by target expansion Targets that are always expanded can be acquired just as fast as targets that expand just as the user reaches them Implications of results can be applied towards the design of UI widgets for devices with limited viewing space 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 112

29 Improvement to Fitts : Bubble Cursor Bubble Cursor TO READ: Tovi Grossman, Ravin Balakrishnan. The Bubble Cursor: Enhancing target acquisition by dynamic resizing of the cursor s activation area, ACM CHI, 2005, p /26/ Graduate Course in HCI /26/ Graduate Course in HCI 114 Bubble Cursor Improvements by Decreasing A Drag-and-pop [Baudisch et al.] Object pointing [Guiard et al.] Modification to area cursor Design of Bubble Cursor Increasing W Area cursor [Kabbash & Buxton] Enhanced area cursor [Worden at al] Expanding targets [McGuffin & Balakrishnan] Decreasing A and Increasing W Semantic pointing [Blanch et al] Problem! Circular cursor resolves problem 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 116

30 Design of Bubble Cursor Design of Bubble Cursor Size problem 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 118 Design of Bubble Cursor Design of Bubble Cursor 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 120

31 Design of Bubble Cursor Results 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 122 Results Experiment 2 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 124

32 Results Models for Trajectory-Based HCI Tasks Trajectory tasks are becoming more common Navigating through nested menus Drawing curves Moving in 3D worlds Cannot be successfully modeled using Fitts law Steering through tunnel as paradigm to represent trajectory-based tasks Video 10/26/ Graduate Course in HCI 125 Beyond Fitts Law: Models for trajectory based HCI tasks. Proceedings of ACM CHI 1997 Conference 10/26/ Graduate Course in HCI 126 Beyond pointing: Trajectory based tasks Beyond pointing: Trajectory based tasks Experimental paradigm focused on is steering between boundaries (constrained motion) It appears that the time to produce trajectories sets the relative speed-accuracy ratio: the larger the amplitude, the less precise the result is. Want to derive and validate quantitative relationships between completion time and movement constraints in trajectory-based tasks 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 128

33 Beyond pointing: Trajectory based tasks EXPERIMENT 1: GOAL PASSING A steering task with constraints only at the ends of the movement Result: goal passing task follows same law as in Fitts tapping task Beyond pointing: Trajectory based tasks EXPERIMENT 2: INCREASING CONSTRAINTS What happens if you place more goals along the trajectory? Allows to formulate a hypothetical relationship of the steering task Result: model successful in describing the difficulty of the task 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 130 Beyond pointing: Trajectory based tasks 2 goals passing D D ID = log 2 ( + 1) W 3 goals passing D ID log ( W = ) 2 N+1 goals passing D ID = N log 2 ( + NW goals passing D ID =? W 1) D/2 D/2 D/N D/N D/N 10/26/ Graduate Course in HCI 131 D W W Fixed width tunnel D D ID =, MT = a + b W W Narrowing tunnel = D dx ID 0 W ( x ) ID = D/(W 2 -W 1 )*ln(w 2 /W 1 ) General Steering Law ds ID = c W ( s ) Steering Law W 1 W(x) 10/26/ Graduate Course in HCI 132 c dx ds D W(s) W W 2

34 Design Implications Interacting with current GUIs, one often implicitly performs various path steering tasks i.e. menu selection Each step in menu selection is a linear path steering task, similar to the one in Experiment 2 Crossing Based Interfaces: Motivation Pointing is most universal and best adapted interaction paradigm in human computer interfaces However some disadvantages exist: time-consuming if object pointed to is small pointing-driven widgets consume screen real estate double-clicking is not trivial for novice users, due to rapid succession of clicks (temporal dependence) 10/26/ Graduate Course in HCI 133 More than dotting the i's --- foundations for crossing-based interfaces. Johnny Accot, Shumin Zhai. Proceedings of the SIGCHI conference on Human factors in computing systems. April /26/ Graduate Course in HCI 134 Crossing based interfaces Crossing interfaces: example designs Macintosh menu bar affords orthogonal selection since height is infinite Windows provides collinear selection 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 136

35 Crossing interfaces: example designs Crossing interfaces: example designs 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 138 Crossing interfaces: example designs Guiard s Model Originated from the area of motor behavior referred to as bimanual control or laterality Both hands perform a different set of tasks Given this knowledge and handedness of people, interesting to evaluate how interaction accommodates best the division Results in descriptive model of bimanual skill, given by Guiard in 1987 paper Guiard, Y. (1987). Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behavior, 19, /26/ Graduate Course in HCI /26/ Graduate Course in HCI 140

36 Hand Guiard s Model Role and Action Guiard s Model Example: a right-handed artist sketches a design of car Non-preferred leads the preferred hand sets the spatial frame of reference for the preferred hand performs coarse movements acquires a template with left hand (non-preferred hand leads) template is manipulated over the workspace (coarse movement, sets the frame of reference) Preferred follows the non-preferred hand works within established frame of reference set by the nonpreferred hand performs fine movements Right hand picks stylus (preferred hand follows) and placed close to the template (works within frame of reference set by the non-preferred hand) Artist sketches (preferred hand makes precise movements) 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 142 Guiard s Model People naturally gravitate to using two hands Performance times are reduced Bimanual Control and Desktop Computer Affordances How does the distribution of keys on a keyboard facilitate task division between right/left hands? where does interaction with the mouse fit into the model? Can be used for interfaces that employ: Drawing designs Fabricating virtual objects Positioning Reshaping Right side bias toward power keys (executive keys + modifier keys, marked in red dots) 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 144

37 Bimanual Control and Desktop Computer Affordances Bimanual Control and Desktop Computer Affordances Dominance on right hand side, good for the 80s but how does this work with GUIs and pointing devices that are now commonplace? Right handed have to reach over with left or leave the mouse Task Delete Select an option in a window Leading Movement Left hand manipulate pointer with mouse and select text/object by double clicking or dragging Left hand manipulate pointer with mouse and click on an option Trailing/Overlapping Movement Right hand press DELETE (probably with little finger) Right hand press ENTER (Note: OK button is the default) Is there an advantage for left-handed users? Click on a link in a browser Open a file, open a folder, or launch a program Right hand navigate to link via PAGE UP and/or PAGE DOWN keys Left hand manipulate pointer with mouse and single click on icon Left hand manipulate pointer with mouse and select link by clicking on it Right hand press ENTER (Note: avoids error prone double-click operation) 10/26/ Graduate Course in HCI 145 Common tasks performed by a left-handed user manipulating mouse in the left hand 10/26/ Graduate Course in HCI 146 Bimanual Control and Desktop Computer Affordances Tasks described previously are faster for left-handed users than right-handed users When pointing is juxtaposed with power key activation (excluding SHIFT, ALT, & CONTROL), the desktop interface presents a left-hand bias Bimanual Control and Desktop Computer Affordances Scrolling typically accomplished by dragging elevator of scrollbar along the right-hand side of an application s window Takes up to 2 secs per trial and is obtrusive and nontransparent In perspective of Guiard s model of bimanual control, we can delegate scrolling to non dominant hand Task Characteristics Scrolling Selecting, editing, reading, drawing, etc. precedes/overlaps other tasks sets the frame of reference minimal precision needed (coarse) follows/overlaps scrolling works within frame of reference set by scrolling demands precision (fine) 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 148

38 Redesigning the Scrolling Interface Papers to Read Please read the list of papers given in the following slides. The videos are available online 10/26/ Graduate Course in HCI /26/ Graduate Course in HCI 150 Interactive Techniques Large Displays 1) Shahzad Malik, Abhishek Ranjan, Ravin Balakrishnan. (2005). Interacting with large displays from a distance with vision-tracked multi-finger gestural input. Proceedings of UIST the ACM Symposium on User Interface Software and Technology. p (uist2005_videohand.mov) 2) Clifton Forlines, Ravin Balakrishnan, Paul Beardsley, Jeroen van Baar, Ramesh Raskar. (2005). Zoom-and-Pick: Facilitating visual zooming and precision pointing with interactive handheld projectors. Proceedings of UIST the ACM Symposium on User Interface Software and Technology. p (uist2005_zoomandpick.mov) 3) Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., and Zierlinger, A. Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-operated Systems. In Proceedings of Interact 2003, Zurich Switzerland, August 2003, pp (2004-Baudisch-CHI04-DragAndPop.mpeg) 4) Anastasia Bezerianos, Ravin Balakrishnan. (2005). The Vacuum: Facilitating the manipulation of distant objects. Proceedings of CHI 2005 the ACM Conference on Human Factors in Computing Systems. p (chi2005_vacuum.mov) 5) Xiang Cao, Ravin Balakrishnan. (in press, 2006). Interacting with dynamically defined information spaces using a handheld projector and a pen. To appear in Proceedings of UIST 2006 the ACM Symposium on User Interface Software and Technology. (uist2006_handheldprojector.mov) 6) Baudisch, P., Cutrell, E., and Robertson, G. High-Density Cursor: A Visualization Technique that Helps Users Keep Track of Fast-Moving Mouse Cursors. In Proceedings of Interact 2003, Zurich Switzerland, August 2003, pp (2003-Baudisch-Interact03-HighDensityCursor.wmv) Interactive Techniques Small Displays 1) Baudisch, P. and Rosenholtz, R. Halo: A Technique for Visualizing Off- Screen Locations. In Proceedings of CHI 2003, Fort Lauderdale, FL, April 2003,pp (2003-Baudisch-CHI03-Halo.wmv) 2) Mackinlay, J., Good, L., Zellweger, P., Stefik, M., and Baudisch, P. City Lights: Contextual Views in Minimal Space. In Proceedings of CHI 2003 (Short paper), Fort Lauderdale, FL, April 2003,pp (2003-Good- Citylights.mpeg) 3) Lam, H. and Baudisch, P. Summary Thumbnails: Readable Overviews for Small Screen Web Browsers. In Proceedings of CHI 2005, Portland, OR, Apr 2005, pp (2005-Baudisch-CHI05-SummaryThumbnails.mov) 4) Irani, P, Gutwin, C. Yang, X-D, Improving selection of off-screen targets with hopping. CHI 2006: /26/ Graduate Course in HCI /26/ Graduate Course in HCI 152