Haptic, vestibular and other physical input/output devices

Transcription

1 Human Touch Sensing - recap Haptic, vestibular and other physical input/output devices SGN-5406 Virtual Reality Autumn 2007 ismo.rakkolainen@tut.fi The human sensitive areas for touch: Hand, face Many types of sensors Also forces, balance Sometimes touch is vital: Braille writing for blind people Haptic and I/O devices Haptic interfaces Haptic interfaces VR Engine Software & Databases Input Devices Output Devices The User Task Haptic interface is a device that enables manual interaction with virtual environments or tele-operated remote systems Employed for tasks that are usually performed using hands in the real world The same device can be both an input and output device An input device with force feedback Information acquisition and object manipulation through touch The user is physically touching the device Sensing of shape, softness and texture An umbrella term covering all aspects of manual exploration and manipulation by humans and machines in real, virtual or teleoperated environment Allow users to manipulate, touch and feel objects simulated by virtual environments and teleoperator systems The Haptics Community, Haptic input devices Fitt s Law Passive input Events triggered by the system monitoring the participant Passive haptic input devices The force the user feels when using the input device is not controlled by the computer program Can still give user a tactual sensation Active input Events specifically triggered by the user Active haptic input devices Include both sensors and actuators Can present controlled forces to the user Allow user to feel virtual objects as well as control them Used by interaction designers Fitt s Law states that the time to acquire a target is a function of the distance to and size of the target Defines an index of difficulty For a haptic virtual environment or teleoperated system, you often want to show that you can minimize difficulty via haptic feedback 1

2 Basic haptic input devices Game pad, 3D Mice Keyboard Designed for writing text, not for VR Still used as a controlling device in simulations (e.g. games) Mouse Developed to give X and Y coordinates to a special cursor in a graphical user interface, also includes 1-3 buttons for functions Nowadays mice have a scroller, maybe also some extra buttons Sometimes wireless Oldest type of mouse includes a ball and two rotating sensors to capture ball rotation. Optical mouse includes a LED for positioning. Laser mice have best resolution Joystick Originally had only 8 directions, newer models can give out any value in 2-DOF space Includes a few buttons for actions Common in gaming since the 1980 s Very good for PC flight simulators and many other kinds of games GamePad An input device which includes a couple of small joysticks 4 DOF if using both at the same time Many buttons Game consoles + PCs May contain vibrating tactile feedback A sort of device like this could also be used in a CAVE for navigation Various 3D Mice Ring mouse, wand, fly mouse, cubic mouse, dragonfly, Trackball Three-dimensional probe Special use input devices Hand as an input device Drawing tablet Sensing chair Wheels and pedals for car simulations Simulators for controlling many kinds of special machines Full simulators simulating the whole environment for a certain machine Mock-up cockpits for flight simulators Car simulators Forest machine simulators Etc Hand is an important tool in daily life Hand is a complex input device Many possible arm, palm, joint and finger positions Pointing, selection, clicking Virtual keyboard, finger mouse Gestures Translation of sign language Grabbing Tele-surgery Tactile (touch) and kinesthetic (force) feedback 2

3 Data gloves Soft or exoskeleton data gloves are one type of haptic devices Can be as input or also (active) output devices Hand position and orientation (6 DOF): usually electromagnetic sensor on hand Joint positions, many degrees of freedom (up to 22 DOF) Wrist 2 DOF, finger joints 5 x 3 DOF, sideways movement of fingers 5 DOF Drawbacks Need for user-specific calibration A number of hand poses, can take several minutes People have different sizes of hands. Glove-embedded sensors overlap different finger locations for different users Complexity High cost Hard to put on and off Gloves, joint measurements Dataglove History Fiber optics Attenuation of light in the fiber measures bending of fingers (and fiber) Expensive, fragile Ultrasonics Measures the arrival of ultrasound from the hand Magnetics A small magnetic field in the joints -> orientation Electrical resistance Measures the resistance of stretch sensors (in Finnish venymäliuska ) etc., which gives the bending Hybrids VPL DataGlove 1987 Fiber optic, 10 DOF, 5 deg, 30 Hz + Polhemus position tracker Exos Dextrous Hand Master DOF, 8-bit, 200 Hz, + Polhemus Mattel/Nintendo PowerGlove A cheap toy, ultrasonic Not available anymore The Pinch Glove Calibration not necessary Conductive fiber patches Fingertips Back of the finger In the palm Gives on/off touch input Gestures detected as the establishing and breaking of electrical contacts between parts in the glove A multiplexing chip reducing number of wires Quite common, price about $2000 The Pinch Glove 3

4 Datagloves The 5DT Data Glove Virtual Technologies CyberGlove Precise stretch sensors, 0.5 deg. Pro-glove, 18/22 DOF, calibration, 13,000 CyberTouch 19,000 (vibrotactile in fingers and palm) CyberGrasp 49,000 CyberForce 70,000 Virtex.com Each finger has a fiber loop Light emitted and sensed Compact and light Users feel comfortable Optoelectric connector on the back of the hand LED fiber phototransistor Wireless communication Some other gloves: Essential Reality P5: 80 For gaming, includes tracking Nissho, SwRI Some Haptic Products Applications of active haptic interfaces All over 10,000 Reach-in haptic workstations FCS HapticMASTER Force dimension 6DOF Haption VIRTUOSE series SensAble PHANTOM series Absolutely required for some tasks Many medical procedures Sensing of forces, e.g., force feedback to simulate the feel of organs as they are manipulated Training / tele-operation Testing the ease of manual assembly of complex mechanisms before they are manufactured Making virtual environments accessible to visually impaired users Can improve user s sense of presence If the joystick is vibrated when player crosses a bridge (to simulate driving over planks) it can provide a landmark for navigation, and signal the vehicle s speed (vibration frequency) and weight (amplitude) Simulated automobile steering wheels provide the road "feel" for race car simulations Haptic cues to augment graphical user interfaces (e.g., Logitech IFeel mouse) Active haptic interfaces can improve performance by providing natural constraints In some VE systems selecting and repositioning of objects without haptic cues can be difficult, e.g., when trying to set a simulated coffee mug down on a simulated table top, not pushing it through May reduce the visual attention required of the user Can improve accuracy and rate of spatial input During a virtual pick-and-place task force feedback cut positioning errors in half Advantages of active haptic interfaces Some Aspects of Haptic Displays Active haptic interfaces can reduce information clutter Unlike speakers and video monitors, haptic displays don t generally clutter a user s environment with unnecessary information For example, a mobile phone in vibrating mode is not disturbing the environment, but can still give the right information to right person A good haptic interface represents a good match between the human haptic system and hardware for sensing and display The primary input-output variables of the interfaces are displacements and forces, including their spatial and temporal distributions In contact tasks involving finite impedances, either displacement or force can be viewed as the control variable Consistency among free-hand motions and contact task is best achieved by viewing the position and motion of the hand as the control variable Resulting net force vector and its distribution within the contact regions as the display variables Users may tolerate considerable drift errors in haptic displays Two-handed interfaces may offer significant advantages over one-handed haptic languages should be developed as efficient as possible Wide range of information transfer rates for different methods of manual communication The senses involved Haptic means physical contact or touch Kinesthesia, the sense of movement or strain from within the muscles Taction, the sense of touch Haptic displays are used much less than visual or aural displays Required on surgery simulators etc. Creating a satisfactory display device is very difficult The haptic/kinesthetic senses are difficult to fool The haptic/kinesthetic senses can t be simulated at distance 4

5 Haptic displays Primary methods of haptic interface Touch and forces can aid on feeling and making the objects more real By using haptics for one object in the virtual world, the rest of the world seems more real as well Two main types: Tactile, touch (skin-based) Spatial, temporal variations Hot, pressure, textures, softness, Kinesthetic, Force feedback Muscle/joint-based Hand, arm motion in probing Forces etc. can be felt Tactile displays provide information to the user in response of Touching Grasping Feeling surface textures Sensing temperature of an object End-effector displays Provide means to simulate grasping and probing objects Provide resistance and pressure to achieve these effects Primary methods of haptic interface Haptic displays Robotically operated shape displays Use robots to present physical objects to the user s fingertips Provide information about shape, texture and location 3D hardcopy Automated creation of physical plastic models based on 3D computer models Provides haptic and visual representation of an object Since this model is a static object, it functions only as an output system Haptic displays Haptic displays Most haptic displays interact with the user via hand or hand-held stylus Many of them focus especially on fingertips Passive haptic representations Static materials to represent something Often part of a device Most force displays focus on limbs as whole Manipulation arm, stair stepper, unicycle 5

6 Tactile displays, interface issues Components of tactile displays The purpose of most tactile displays is to provide information in response to user s Touching or grasping something Feeling object s surface texture Sensing the temperature of an object To generate these sensations are used Global pressure Multiple local pressures Vibration Heat transfer Bladder actuators Pockets that can be expanded and contracted By controlling the flow of air (pneumatic) By controlling the flow of liquid (hydraulic) Strategic placement of the pockets creates the sensation of pressure on different areas of the participant s hand and body Components of tactile displays Components of tactile displays Vibrator actuators Often used to display the sensation of pressure More robust and easier to control than bladders Can be integrated with glove input device or with a handheld prop Vibrating game pads a common implementation Low-frequency speakers (subwoofers) can also be used as a vibratory display Pin actuator Used for the display of surface textures Still being researched and has not been used in many applications Small, square pin arrays placed on each finger Pins aligned on a rotating cylinder Textures are detected by pressure variations across the fingertip over time C.f., Northrop Grumman s TerrainTable Components of tactile displays Components of tactile displays Pin actuator Temperature actuators Can very rapidly present temperature fluctuations Typically to fingertips Can become sufficiently hot or cold to damage human tissue Limits must be set, and not always show the real temperature of the simulated objects Seldom used in virtual reality systems 6

7 End-effector displays End-effector displays Force displays in which the user s extremities (hands and/or feet) have contact A device that the user can manipulate This device can become active and respond to user s actions with resistance and force Generally linked to mechanical tracking sensors A collection of various haptic devices: ndex.html A surgery simulator End-effector displays End-effector displays Example: UNC Nanomanipulator A force-feedback glove End-effector displays Components of end-effector displays Mechanical trackers Affecting user movement is generally provided by Electronic motors Additional motor usually for each degree of freedom Hydraulics Pneumatics 7

8 Features of end-effector displays Full Body Devices Often operates also as an input device Potentially providing resistance to input controls Mechanical movement sensors are generally incorporated directly into the system Mechanical tracking is generally very fast and accurate Typically operate with respect to single point in the virtual world Number of DOFs: 1-6 Tactile displays such as temperature actuators and vibrator actuators can be mounted within the end-effector Exoskeletal Surround (platforms etc.) Robotically operated shape displays Haptic displays comparison Use robots to place a representation of the virtual world where the user is reaching May be generic (corners and edges) May be specific (selection of switches) Usually uses a finger surrogate for fast tracking, and safety Benefits of Tactile displays helps in the fine manipulation of virtual objects (some) can be added to some end-effector displays can be world or body grounded (usually bodygrounded) body-grounded method is mobile generally not too expensive generally portable Haptic displays comparison Haptic displays interfaces Benefits of End-effector displays Can be world or body grounded (exoskeletal is bodygrounded) Exoskeletal method is mobile World-grounded method is not very encumbering Fast and accurate tracking is usually built-in Benefits of Robotically operated shape displays Can provide a very realistic haptic display World-grounded display Fast and accurate tracking is usually built into the display Works primarily with head-based visual displays Interface design must be useful for manipulation make it non-intimidating, usable for all mimic real world interaction And look out for physical perils! Especially for force feedback All users may not be strong men: children at risk! 8

9 Haptic rendering Haptics Rendering Pipeline 3D objects & properties loaded from the database 1. Collision detection Only colliding objects are passed on 2. Compute collision forces, smoothing, mapping Many force models Linear spring law F = kx (k: stiffness, x: indentation) 3. Haptic texturing (touch) Vibrations, temperature, etc. Needs to be fast, upto 1000 Hz Vestibular display Vestibular display Physically moving the user Helps the user to sense equilibrium, acceleration and orientation that he expects to feel Strong relationship between the vestibular and visual systems of humans Prevents simulator sickness Makes better immersion Typically used in specialized simulators Flight simulators etc. The whole simulator is moved by hydraulic cylinders Tilt, rotation, acceleration, moving Typically big, expensive devices E.g., flight simulators Very powerful, enhanced effect Max. 1 G (or sling structures) Gyroscopic systems Even zero gravity is possible (during a parabolic flight) Electrical Vestibular Simulation Locomotion platforms EVS Volta 1803: balance disturbances, when electricity into certain areas of head Tilt, sway Amplitude, frequency Can add to the perception of moving No known side effects Walking Running A running mat in a CAVE Bicycle simulators 9

10 Locomotion platforms CirculaFloor Omni-Directional Treadmill 1.3 x 1.3 m, x and y belts, 3400 rolls Moving motion foot pads on the floor following the tracked user. Creates an infinite floor! Omni-directional walking, keeping the walker in the center of the space ACM Siggraph 2004 Emerging Technologies Hiroo Iwata, University of Tsukuba It enables a revolution in entertainment or training simulators Olfactory Displays Olfactory Displays Artificial smell simulation Mostly at research, prototype phase, some niche products Mixing chemical compounds and concentrations Typically essential oils How to get rid of them? Digital signals written into software code trigger the aroma generator to emit precise amounts of the appropriate aroma Aromas can be used to enhance the experience and trigger fear, excitement and many other emotions Device could reside next to your monitor, directly in front of you or even be worn similar to a doctor's stethoscope Enhanced cosmetics, perfume, food advertising Aromatherapy MEMS & VR MEMS & VR MEMS = MicroElectroMechanical System IC processing + micromachining = Sensors, Motors, Structures, Electronics Input devices Acceleration, vibration, tilt Temperature, light Humidity, barometric pressure Microphones Magnetic field (compass, ), Output devices Photons! Tactile? Neural Small Cheap Low energy Advanced microdisplays Data glasses Sensors, communication Actuators 10