Haptic Interface Technologies

Transcription

1 Haptic Interface Technologies 02/14/2010 EECE596: Copyright since 1999, 5/01/2018 EECE596: Copyright since 1999, Sidney Fels Sidney Fels

2 Haptics: Overview Haptics: Introduction tactile proprioception Tactile Interface Technologies Force Feedback Technologies

3 Haptics: Introduction Sense of touch and kinesthesia tactile sensing proprioception Somatosensory system Bidirectional sense environment temperature, vibration, weight, etc. manipulate environment push, pull, pinch, hit, rotate, etc.

4 Haptics: Tactile Sensing Can sense: texture/vibration temperature of object or environment slip detection surface compliance elasticity viscosity electrical/thermal conductivity vibration (other than for texture) initial contact detection gauging force required for manipulation

5 Haptics: Tactile I/F Critical for performance in many tasks Massimino and Sheridan, 1993 tracking (Patrick, Sheridan and Massimino, 1990) target pointing (Akamatsu, 1994) degraded visual condition (Massimino and Sheridan, 1993) See Lederman s work for many experiments looking at attributes of tactile sensing Considered to be critical for virtual environments needed for immersion and sense of reality

6 Haptics: Tactile Sensing Four main sense organs Meissner s Corpuscles surface curvature, velocity, local shape, flutter, slip poor spatial resoln 43% Pacinian Corpuscles vibration, slip, acceleration Hz response frequency range 13% Markel s Disks skin curvature, local shape, pressure 25% Ruffini Endings skin stretch, local force 19%

7 Haptics: Tactile Sensing Skin properties: finger pad (Reynier and Hayward, 1993) spatial resolution, 0.15mm; two-point discrimination 1-3mm smooth glass feature, 2µm for single dot, 0.06µm for grating, 0.85µm for straight lines line orientation detection, 8.7mm separation (sequential); 13.1mm separation (simultaneous) 5.5msec separation of two 1msec stimulti increase stimuli duration -> reduce threshold Other results available for different properties strain, texture properties, temperature, etc. check Burdea and Coiffet (1994)

8 Haptics: Tactile Actuators

9 Haptics: Tactile Displays (commercial) CyberTouch (Virtual Technolgies) little vibrators on fingers 0-200Hz $14,000 US TouchMaster (Exos) voice coils on fingers Hz Tactool System (Xtensory) - sold as kits pins driven by shape memory alloys impulse sensation (30g) vibration (20Hz) $1,500 Displace Temp. Sensing System (CM Research) thimble thermoelectric heat pump (Peltier effect) $10,000

10 Haptics: Peltier Effect P - Bismuth N - Telluride

11 Haptics: CyberTouch

12 Haptics: Tactile Displays (commercial) Intelligent Systems Solutions air bladders + pnematics not available anymore Braille Displays Freedom of Speech (85-8 dot cells) Games Playstation controller fishing, driving, baseball

13 Haptics: Tactile Displays (research) Reading aids Sherrick, 1984; Shimizu, 1986; Barfield and Furness, 1995 tools for studying haptics Cholewiak and Sherrick, 1981; Schneider, 1988, Lederman, 1999 tools for rehabilitation Wise et al., 1990 tools for teleoperation

14 Haptics: Tactile Displays (research) Disney - TeslaTouch

15 Haptics: Tactile Displays (research) Armstrong Labortatory 5x6 array tactile stimulator, SMA wires solenoid attached to Phantom Harvard (Howe) 4x6 array of pins driven by SMA actuators 2.1mm spacing, 3mm height, 62msec rise/fall time, BW 6-7Hz studied palpation, shape display (single row) + force feedback display 3 subjects, 60 trials, tumor finding task, 4mm cylinder inside foam rubber error <1mm in 50%; <3mm in 95%; no shape info > 13mm

16 Haptics: Tactile Displays (research) voice coils + force feedback device (teleoperation) investigated 3 types of tasks detection of vibration is fundamental goal worn ball bearning vibration feedback helps discrimination vibrations indicate state of task piercing plastic membrane vibration feedback reduces force exerted and increases response time vibration not important to task peg-in-hole task no effect, but subjective improvement

17 Haptics: Tactile Displays (research) Hokkaido (Ino) displays for shearing, pressure and temperature Shear display table mounted pneumatic driver Temp. display

18 Haptics: Tactile Displays (research) MIT - Touch Lab (Srinivasan) linear and planar graspers used for psychophysical experiments use with Phantom libraries available for compliance, viscosity, mass, shape, texture, friction, walls and corners created force shading for creating feel of smoothly curved surfaces what about contact sounds?

19 Haptics: Tactile Displays (research) Karlsruhe Research Centre (Germany) three 24 needle print heads needles vibrated at 600Hz to simulate contact pressure actuated by sensors attached to forceps Many others: Queens, (Lederman) - spinning disks Sandia Labs, 2x3 array of electromagnetic actuators TiNi Alloy Comp., 5x6 array of tactor pins U of Salford, UK, vibration with piezo-electric actuator, Peltier effect heat-pump for temp., pneumatic bladders for contact force, glove mounted

20 Haptics: Kinesthetic Interfaces Awareness of position and movement and forces on body parts Force feedback has been shown to be important Teleoperation Hill and Salisbury, 1976; Hanaford, 1989 and Howe, 1992 Molecular docking Ouh-Young, Beard and Brooks, 1989 Grasping tasks Gomez, Burdea and Langrana, 1995 (reduced error and learning time by 50%)

21 Haptics: Kinesthetic Interfaces Where is it useful? Virtual reality/augmented reality medicine surgery diagnosis scientific visualization data manipulation interactive art situations where auditory and visual feedback are limited aids to disabled peripheral tasks 3D manipulation

22 Haptics: Kinesthetic Interfaces Basic idea: 1. Measure movement and forces exerted by user (fingers, hand, arm, body) 2. Calculate effect of forces on manipulated objects and resulting forces on user virtual or real objects 3. Present forces to the user s fingers, wrist, arms etc. as appropriate

23 Haptics: Kinesthetic Interfaces Devices are either earth grounded off-the-body exoskeletons

24 Haptics: Kinesthetic Technologies

25 Haptics: Kinesthetic Technologies Other technologies magnetic levitation tendons???

26 Haptics: Human Kinesthetic Properties In addition to cells mentioned we have: Golgi endings in joint ligaments joint torque Ruffini type endings in joint capsules capsule stretch Golgi tendon organs muscle tension muscle spindle organs muscle stretch and rate of change probably most important for kinesthetic sense nothing known for weight or effort inside central nervous system

27 Haptics: Human Kinesthetic Properties Somatosensory system is not symmetric force control and perceptual bandwidths differ (Brooks, 1990) deliver forces at 5-10Hz receiving position and force signals >20-30Hz JND for force sensing is about 7% rigid body to 415 N/cm force production: 16.5N distal finger to 192.3N shoulder joint output resoln : high at finger tip, low at shoulder

28 Haptics: Human Grasp Properties Some force outputs for different types of grasps Jacobus et al, 1992

29 Haptics: Human Grasp Properties Current technologies generally change way they work based on grasp types several ways to categorize grasps Schlesinger (1919) cylindrical, finger-tip, hook, palmar, spherical, lateral Napier (1956) distinguish between power and precision grasp MacKenzie (1990s) prehensile vs non-prehensile

30 Haptics: Human Grasp Properties Cutkosky and Howe, 1990

31 Haptics: Force-feedback Issues Should keep grasping/reflected forces to less than 15% of max (Wiker, 1989) comfort and fatigue issues I.e., index finger 7N, middle 6N, ring 4.5N

32 Haptics: Issues for Creating FF/B Devices 1. Sampling data from user lag, update rates etc. 2. Computing forces S/W libraries needed like graphics libraries 3. Presenting forces calculating control parameters in real-time 1000Hz update Impedance control (force) vs. Compliance control (position) overcome inertia

33 Haptics: Force-F/B Devices (comm) Phantom (SensAble Devices, Massey) thimble on finger single point force feedback only motors and cables and good engineering can use more than one at a time (if you have $$$)

34 Haptics: Force-F/B Devices (comm) Impulse Engine (Immersion Corp.) 5 dof 2 dof pivot insertion translation rotation of tool open-close motion of instrument servo-motor actuators laproscopic and endoscopic simulation

35 Haptics: Force-F/B Devices (comm) Laproscopic Impulse Engine

36 Haptics: Force-F/B Devices (comm) CyberGrasp (Virtual Technologies) motor actuated tendons attached to fingers 1 dof/finger used to be flappers on end for contact force mounted on CyberGlove

37 Cyberforce Exoskelton arm forces Tendons grasp forces

38 Haptics: Force-F/B Devices (comm) Exos 4 dof Force Feedback Master surgical simulator hand and arm Force Exoskeleton

39 Haptics: Force-F/B Devices (comm) SaFiRE (Exos) exoskeleton for hand applies forces to thumb, index finger and wrist 8 dof: 3 dof to thumb, 3 dof to index, 2 dof to wrist links grounded to forearm supply 3D Cartesian forces to fingertips and palm DC motors used with wires and gears

40 Haptics: Force-F/B Devices (comm) Hand Exoskeleton Haptic Display (Exos) based on SaFiRE hand exoskeleton 1 dof to thumb, 2 dof to index slip display on thumb and index boom mounted; 2 dof position sensing, 1 dof vertical FF/B

41 Haptics: Force-F/B Devices (comm) Per-Force Handcontroller and Finger Forcer Option (Cybernet Sys. Corp) 6 dof force feedback joystick finger force feedback added to top of joystick thimbles attached to finger tips

42 Haptics: Force-F/B Devices (comm) Haptic Master (Iwata in Tsukuba) 3D force and torque three sets of pantograph links

43 Haptics: Force-F/B Devices (comm) Interactor (Aura)

44 Haptics: Force-F/B Devices (research) Many issues remain to be solved for force feedback devices commercial FF/B displays only scratch the surface Extensive research on how to use these displays not a lot of practical uses yet Most research effort to develop FF/B displays has come from teleoperation applies to virtual environments other effort from psychophysical testing labs effort to explore application of FF/B displays done in well funded labs

45 Haptics: Force-F/B Devices (research) Ino, Hokkaido elbow joint force feedback metal hydride actuator temperature changes in alloy controls pressure of hydrogen gas pressure converted into propulsion 300g, cylinder is mm, 6g of hydride lift 10kg to 50mm with vel. of 9mm/sec noise free, no sudden impact force similar compliance to elbow joint report good results (obviously) probably not good for hand probably work for knee and other large joints

46 Haptics: Force-F/B Devices (research) MIT (Salisbury) continuing with Phantom changing tip creating software haptic rendering - calculating and generating forces still point interaction contact forces contact persistence impedance some curvature, texture and friction want general purpose s/w libraries to allow for haptic rendering Interval was working on some of this as well

47 Haptics: Force-F/B Devices (research) Institut des Systemes Intelligents et de Robotique, Université Pierre et Marie Curie (Hayward previously McGill) 9 dof Stylus - tendon driven single point Pantograph 2 dof translation - 10cm X 10cm FF/B mouse found acceleration best for shock and hard contacts for effective performance need wide frequency response high accuracy in presenting forces mechanical impedance controllable over wide range dynamic response up to Hz

48 Haptics: Force-F/B Devices (research) Pantograph Stylus

49 Haptics: Force-F/B Devices (research) Northwestern University (Colgate) 4 dof Force Reflecting Manipulandum translation + rotation in horizontal plane high impedance control stiff springs and hard walls no singularities in the work space study physics based simulation for haptic interfaces

50 Haptics: Force-F/B Devices (research) Rutgers (Burdea) Rutgers Master uses pneumatic microcylinders positioned in palm ends attached to tips of 3 fingers and thumb Experiment 1: manipulate ball without deforming it compared visual f/b (ball, bar graph and mono/stereo) audio f/b force f/b Results FF/B most significant non-redundant modality FF/B + audio best redundant combo.

51 Haptics: Force-F/B Devices (research) U. of North Carolina (Brooks) Argonne Remote Manipulator (ARM) 6 dof large workspace device hand grip display, joint action at shoulder and outward used mostly for molecular docking

52 Haptics: Force-F/B Devices (research) Others: Suzuki Motor Corp: SPICE 6 dof, mechanical + motors Tokyo Inst. Of Tech single point FF/B strings and motors

53 Haptics: Force-F/B Devices (research) CCRMA (Gillespie) piano/keyboard key feedback simulate touch of different keyboards 7 keys, DC motor driven, dynamic simulation of key action UBC (Salcudean) maglev joystick force feedback motor controlled pantograph with additional dof UBC (Maclean) 1 dof rotary devices Cobots (Peshkin and Colgate) COBOTS use mechanical constraints to set up forces

54 Haptics: Cobots

55 Haptics: Summary Need good haptic interfaces for: immersive VR 3D direct manipulation i/f effective teleoperation mediated communication Very active area of research

56 Haptics: Summary Current status of Tactile displays limited area (finger tip) don t scale up large physical dimensions - not practical no software models + psychophysical info available + devices capable of meeting resolution and B/W reqd. in specific domains + >20-30Hz BW + pin displays are close to human resolution + vibration available at wide range of frequencies not high enough for skillful manipulation + temperature displays have sufficient resolution

57 Haptics: Summary Force feedback displays generally application specific generally small workspace encumbering most use servo motor actuation stability problems backdrivability friction Specs for good force feedback force resolution: 12 bits, position resolution: in passive friction less than 1% of max. force minimum sampling rate: 2000Hz latency <1msec Phantom is closest to meeting all of these for single point

58 Haptics: Summary Still can t represent solid, immovable wall or rigid objects instability at solid edge add viscosity to whole environment reduce stiffness - everything is a sponge More problems sensor accuracy latency of computer actuator performance difference between actuator and sensor placement mechanical transmission difficulties non-linear system due to dynamics of user characteristics safety software models lacking more psychophysical testing needed

59 Haptics: Summary Currently no general purpose haptic interfaces no practical applications in common use Future: robotic graphics? New actuators? Non-linear, deformable manipulators