Haptic Feedback on Mobile Touch Screens Applications and Applicability 12.11.2008 Sebastian Müller Haptic Communication and Interaction in Mobile Context University of Tampere
Outline Motivation ( technologies Theory (haptic sense, ( applications Practice (technologies, Design Issues Future Discussion 12.11.2008 2
Motivation Keyboards being replaced by virtual buttons worse entry speed and higher error rate mobile = flexible context appropriateness of feedback techniques additional information channel = additional capacity ( real physical metaphor ( more 12.11.2008 3
What information to deliver actions click, double-click, drag, rotate, zoom, scroll reactions events scales time, strength target objects GUI elements, on-screen keyboard 12.11.2008 4
Sensation space Haptic: ( x2 ) frequency, amplitude, length, space 5 times faster than vision [5] 2mm two-point vibrotactile discrimination threshold at fingertip much smaller position shift identification threshold [7] no privacy effect on touchscreens 12.11.2008 5
Touchscreen interaction Depending on technology Capacitive TS-> skin Depending on device size and layout Depending on context ( required Noise, movement, task (accuracy 12.11.2008 6
( 1/2 ) [8] Johnston Hoggan, Brewster & Investigating text input Palm Treo 750 (physical keyboard) and Samsung ( keyboard i718 (virtual over-, home- (F + J), click- and slip-events imitating physical properties Lab vs. Subway Physical vs. Virtual vs. Haptic ( feedback (Global vs. Local 12.11.2008 7
( 2/2 ) Hoggan, Brewster & Johnston Haptic feedback significantly improves performance, especially under mobile conditions Two specialized actuators as good as a physical keyboard under mobile condition 12.11.2008 8
What I thought 12.11.2008 9
What I found Snap-Crackle-Pop [4] off-the-shelf piezo actuators global feedback 4 applications numerical keypad, text selection, scrolling, drag and drop Result: potential to improve usability and user experience ( 2006 ) Nokia Research Center, Helsinki 12.11.2008 10
Nokia Haptikos two small piezo sensor pads 0.1mm movement As you have seen in Nokia 770 Funnily enough, although you think you re typing faster than normal because of the feedback, in actual fact you re not [...] There s just some sort of mental satisfaction that comes from typing with a tactile response. -- Roope Takala, Senior Program Manager at Nokia s research labs (after redferret.com [10]) 12.11.2008 11
Poupyrev & Maruyama (2003) [5] TouchEngine [12] ( 0.5mm ~) very thin actuators embeddable inside screen only the glass moves haptically augmented GUI elements various actuator sizes 12.11.2008 12
( 1/2 ) [2] Pen Haptic low cost multi-user non-contact feedback 3D tracking allows feedback on any surface drawable surfaces feedback based on location 12.11.2008 13
( 2/2 ) Haptic Pen Haptic Pen Demonstration via youtube.com 12.11.2008 14
T-Bars Finger gesture guide Replacing buttons Changes in intensity of tactile feedback Can be invisible Required size?? Direction? 12.11.2008 15
Design Issues - Hardware energy consumption material fatigue where to put actuators parallax problem [5] synchronization ( occurrence frequency, amplitude, (frequency of 12.11.2008 16
Design Issues - Wetware expressiveness limited (?) necessity to learn haptic icons no [...] interaction between cognitive load and [...]haptic feedback were found [1] subjectively useful/helpful confusing, annoying [1] Inter-subject perception difference Gender, culture, context 12.11.2008 17
Future Blackberry Thunder RIM keyboard United States Patent Application 0070247429 (Apple, Inc.) [11] [...] a touch sensitive electrode circuit board disposed beneath the surface cover having a plurality of holes disposed therein; an articulating frame disposed beneath the touch sensitive electrode circuit board having integral therewith a plurality of key edge ridges aligned with the holes in the touch sensitive electrode Articulating Frame Protrudes/Deforms Surface Cover at Key Edges During Typing No technical details! Same for Nokia International Patent Application PCT/EP2006/009377 12.11.2008 18
Discussion Haptic feed vs. haptic feedback Fingers exploring touchscreen A haptic feed piezo-style stylus Can movement positions be averaged? 12.11.2008 19
Summary Software-/Interaction-triggered actuators With the addition of this extra tactile feedback the performance of touchscreen keyboards can be brought close to the level of real, physical keyboards. [...] the perceived quality of the buttons is improved [8] Feedback moves from global to local No real-time 3D printer 12.11.2008 20
Thank you.* *Can somebody change being opponents with me on Dec. 10? 12.11.2008 21
( 1/3 ) References [1] @conference{leung2007eha, title={{evaluation of haptically augmented touchscreen gui elements under cognitive load}}, author={leung, R. and MacLean, K. and Bertelsen, M.B. and Saubhasik, M.}, booktitle={proceedings of the ninth international conference on Multimodal interfaces}, pages={374 381}, year={2007}, organization={acm New York, NY, USA}} [2] @conference{lee2004hpt, title={{haptic pen: a tactile feedback stylus for touch screens}}, author={lee, J.C. and Dietz, P.H. and Leigh, D. and Yerazunis, W.S. and Hudson, S.E.}, booktitle={proceedings of the 17th annual ACM symposium on UI software and technology}, pages={291--294}, year={2004}, organization={acm New York, NY, USA}} [3] @conference{doerrer2001nao, title={{a New Approach to Operating Machines with High Functionality}}, author={doerrer, C. and Werthsch{\"u}tzky, R.}, booktitle={proceedings of Eurohaptics 2001}, pages={105--107}} [4] @article{kaaresoja2006scp, title={{snap-crackle-pop: Tactile Feedback for Mobile Touch Screens}}, author={kaaresoja, T. and Brown, L.M. and Linjama, J.}, journal={proceedings of Eurohaptics 2006}, pages={565--566}, year={2006}}
( 2/3 ) References [5] @article{geldard1960snp, title={{some Neglected Possibilities of Communication}}, author={geldard, F.A.}, journal={science}, volume={131}, number={3413}, pages={1583--1588}, year={1960}} [6] @conference{poupyrev2003tis, title={{tactile interfaces for small touch screens}}, author={poupyrev, I. and Maruyama, S.}, booktitle={symposium on User Interface Software and Technology: Proceedings of the 16 th annual ACM symposium on User interface software and technology}, volume={2}, number={05}, pages={217--220}, year={2003}} [7] @book{barfield1995vea, title={{virtual Environments and Advanced Interface Design}}, author={barfield, W. and Furness, T.A.}, year={1995}, publisher={oxford University Press, USA}} [8] @article{hoggan2008iet, title={{investigating the effectiveness of tactile feedback for mobile touchscreens}}, author={hoggan, E. and Brewster, S.A. and Johnston, J.}, year={2008}, publisher={acm New York, NY, USA}}
( 3/3 ) References [9] @article{hall8tbt, title={{t-bars: Towards Tactile User Interfaces for Touchscreen Mobiles}}, author={hall, M. and Hoggan, E. and Brewster, SA}, journal={proc MobileHCI'08}} [10] Nokia perfects the clicky tactile touchscreen - iphone gnashes teeth, swears revenge http://www.redferret.net/?p=9533, accessed 06.11.2008 [11] United States Patent Application 0070247429 http://appft1.uspto.gov/netacgi/nph-parser?sect1=pto2§2=hitoff&u=%2fnetahtml %2FPTO%2Fsearch-adv.html&r=1&f=G&l=50&d=PG01&p=1&S1=20070247429 &OS=20070247429 &RS=20070247429, accessed 06.11.2008 [12] @conference{poupyrev2002ttd, title={{touchengine: a tactile display for handheld devices}}, author={poupyrev, I. and Rekimoto, J. and Maruyama, S.}, booktitle={conference on Human Factors in Computing Systems}, pages={644--645}, year={2002}, organization={acm Press New York, NY, USA}}