Week (date) (date) Lecture Readi d ngs Assignment due

Transcription

1 Haptics & VR Syllabus Week (date) Lecture Readings Assignment due Week1 (9.2) Week2 (9.9) Overview 1. Introduction Week3 (9.16) 2. Input Devices summary 1 Readings proposalp Week4 (9.23) 3. Output Devices summary 2 Haptic VR project proposal Week5 (9.30) 4. Computing Architecture for VR summary 3 Graphics assignment1 Week6 (10.7) 5. Modeling ( 김기권 ) summary 4 Graphics assignment2 Week7 (10.14) 6. VR Programming ( 김기권 ) summary 5 Graphics assignment3 Week8 (10.21) 7. Human Factors in VR ( 이환문 ) summary 6 Graphics assignment4 Week9 (10.28) 8. Traditional VR Applications summary 7 Graphics assignment7 Week10 (11.4) 9. Emerging Applications of VR ( 이환문 ) summary 8 Graphics assignment8 Week11 (11.11) final exam Week12 (11.18) Lab1 Seminar1 Week13 (11.25) Lab2 Seminar2 Week14 (12.2) Lab3 Seminar3 Week15 (12.9) Lab4 Seminar4 Week16 (12.16) 16) Haptic VR project presentation Haptic VR project code & document 1

2 Output Devices: Graphics, 3-D Sound, and Haptic Displays 2

3 Output Devices The human senses need specialized interfaces Graphics displays for visual feedback; 3-D audio hardware for localized sound; Haptic interfaces for force and touch feedback; Not interested in smell and taste feedback. 3

4 Output Devices Definition: A graphics display is a computer interface that presents synthetic world images to one or several users interacting with the virtual world. 4

5 Output Devices Graphics Displays Human stereo viewing; Personal displays; Large volume displays: Active glasses Workbenches; Microsoft Surface Caves; Walls; 5

6 Output Devices Human Visual System Vision is the dominant sensorial channel; Depth perception in mono images is based on occlusion (one objects blocks another from view; on shadows, textures and motion parallax (closer images appear to move more than distant ones) Vision is the most powerful human sensorial channel 6

7 Human Visual System-continued Depth p perception p in stereo is based on seteropsis when the brain registers and fuses two images; Image parallax means that the two eyes register different images (horizontal shift); The amount of shift depends on the inter- pupillary distance (IPD) (varies for each person in the range of mm); Works W k in the near field (to a few meters from the eye) pupillary : 동공의, 눈동자의 7

8 Output Devices Field of view: one eye: 150 horizontally, 120 vertically both eyes: 180 horizontally, 120 vertically Binocullar overlap: 120 horizontally Motion parallax is important in monoscopic depth perception Parallax: 두눈의시차 ( 視差 ) 8

9 Output Devices (same principle used in new 3D HDTVs) Left eye image Right eye image 9

10 Output Devices Implications i for Stereo Viewing i devices Need to opese present two images of the same evr environment; The two images can be presented at the same time on two displays (HMD); The two images can also be presented time- sequenced on one display (active glasses; shutter glasses??); The h two images can also be presented spatially-sequenced on one display (autostereoscopic displays). 10

11 3D display / iki/3d Types of 3D displays Stereoscopic Stereoscopy Based on the principles of stereopsis, described by Sir Charles Wheatstone in the 1830s, stereoscopic technology uses a separate device for each person viewing i the scene to provide a different image to the person's left and right eyes. Examples of this technology include anaglyph images and polarized glasses. Stereoscopic technologies generally involve special spectacles. Autostereoscopic Autostereoscopy An evolutionary development of stereoscopy, autostereoscopic display technologies use optical trickery at the display, rather than worn by the user, to ensure that each eye sees the appropriate image. They generally allow the user to move their head a certain amount without destroying the illusion of depth. Automultiscopic displays include view-dependent pixels with different intensities and colors based on the viewing angle; this means that a number of different views of the same scene can be seen by moving horizontally around the display. In most automultiscopic displays the change of view is accompanied by the breakdown of the illusion of depth, but some displays exist which can maintain the illusion as the view changes [2]. This category of display technology includes autostereograms. Computer-generated holography Computer Generated Holography The hologram is a familiar artifact of the late 20th century, and research into holographic displays has produced devices which are able to create a light field identical to that which would emanate from the original scene, with both horizontal and vertical parallax across a large range of viewing angles. The effect is similar to looking through a window at the scene being reproduced; this may make CGH the most convincing of the 3D display technologies, but as yet the large amounts of calculation required to generate a detailed hologram largely prevent its application outside of the laboratory. Some companies do produce holographic imaging equipment commercially. [3] Volumetric displays Volumetric display In addition there are volumetric displays, where some physical mechanism is used to display points of light within a volume. Such displays use voxels instead of pixels. Volumetric displays include multiplanar displays, which have multiple display planes stacked up; and rotating panel displays, where a rotating panel sweeps out a volume. Other technologies have been developed to project light dots in the air above a device. An infrared laser is focused on the destination in space, generating a small bubble of plasma which emits visible light. As of August 2008, the experiments only allow a rate of 100 dots per second. One of the issues which arise with this filme 3D display system is the use of technologies that could be harmful to human eyes. 11

12 Output Devices Definition: Personal Displays A graphics display that toutputs t a virtual scene destined to be viewed by a single user. Such image may be monoscopic or stereoscopic, monocular (for a single eye) or binocular (displayed on both eyes). 12

13 Output Devices Personal Displays Head Mounted Displays; 3-D DBinoculars (hand supported); Booms (floor supported); Virtual windows (floor supported); Auto-stereoscopic displays (desk supported). 13

14 Simplified HMD optics model The lower the HMD resolution and the higher the FOV, the greater is the number of arc-minutes of eye view corresponding to each pixel. (granularity of HMD: expressed in arc-minutes/pixel) 14

15 HMD integration in a VR system Consumer HMD Professional HMD 15

16 AMLCD (active matrix liquid crystal display), Resolution: 267x225 FOV: 30x23 degrees Equivalent to 62 in at 2 m Weight: 100 grams Can be worn over glasses Olympus Eye Trek Face Mounted Display (FMD 200) 16

17 Sony Olympus Olympus Eye Trek Head Mounted Display Optics uses free-form lens to compensate for aberrations; - an eccentric optical system to reduce size (eliminate 45 degree mirror) 17

18 Olympus Eye Trek Face Mounted Display Optics 18

19 Daeyang cy-visor Face Mounted Display LCOS display, Resolution: 800x600 FOV: 60x43 degrees Weight: 160 grams Can be worn over glasses Liquid Crystal on Silicon display (LCOS) 19

20 Daeyang cy-visor visor Face Mounted Display It is reflective needs external lighting 20

21 Organic LEDs (OLED) In an active-matrix OLED display, each individual pixel can be addressed independently via the associated TFT s and capacitors in the electronic back plane. That is, each pixel element can be selected to stay on during the entire frame time, or duration of the video. Since OLED is an emissive device, the display aperture factor is not critical, unlike LCD displays where light must pass through aperture. Therefore, there are no intrinsic limitations to the pixel count, resolution, or size of an active-matrix OLED display, leaving the possibilities for commercial use open to our imagination. Also, because of the TFT s in the active-matrix design, a defective pixel produces only a dark effect, which is considered to be much less objectionable than a bright point defect, like found in LCD s. 21

22 Organic LEDs (OLED) Robust Design - OLED s are tough enough to use in portable devices such as cellular phones, digital video cameras, DVD players, car audio equipment and PDA s. Viewing Angles Can be viewed up to 160 degrees, OLED screens provide a clear and distinct image, even in bright light. High Resolution High information applications including videos and graphics, active-matrix OLED provides the solution. Each pixel can be turned on or off independently to create multiple colors in a fluid and smooth edged display. Electronic Paper OLED s are paper-thin. Due to the exclusion of certain hardware goods that normal LCD s require, OLED s are as thin as a dime. Production Advantages Up to 20% to 50% cheaper than LCD processes. Plastics will make the OLED tougher and more rugged. The future quite possibly could consist of these OLED s being produced like newspapers, rather than computer chips. Video Capabilities They hold the ability to handle streamlined video, which could revolutionize the PDA and cellular phone market. Hardware Content Lighter and faster than LCD s s. Can be produced out of plastic and is bendable. Also, OLED s do not need lamps, polarizers, or diffusers. Power Usage Takes less power to run (2 to 10 volts). 22

23 5DT Head Mounted Display 800x600 pixels 40 o diagonal view Organic LED Frame sequential stereo 600 grams $4k 23

24 Samsung Emagin z800 OLED HMD Weight 8 oz PC connection - USB, RGB input SVGA resolution (800x600 pixels) stereo Tracking degrees pan 60 degrees pitch $1200 USD OLED (Organic light-emitting diode) 24

25 Sensics pisight panoramic OLED HMDs 25

26 Sensics pisight panoramic HMDs Uses Organic LED A series of micro-displays with special optics to generate a panoramic view Weight 2 lbs (1 Kg) SVGA input resolution (2400x1729 pixels) Field of view 179 horizontal by 58 vertical Binocular overlap 82 Cost? USD sensics.com 26

27 Professional HMDs Keiser ProView AMLCD display, Resolution: 1024x768 FOV: 28x21 degrees Weight: 992 grams Even with the improved resolution, consumer FMDs cannot match the professional-grade LCD-based HMDs 27

28 Professional HMDs N-Vision Datavisor CRT display, Resolution: 1280x FOV: 78x39 degrees Weight: 1587 grams 28

29 LCOS (Liquid Crystal on Silicon display) Virtual Binoculars ( 가상쌍안경 ) 29

30 Virtual Binoculars 30

31 Floor-supported displays Boom3C (courtesy of Fakespace Labs.) 31

32 21 LCD display, Resolution: SXGA (1600x1200) Weight: Counter- Balanced; No dead space but High latencies due to Third-party tracker Virtual Window 3-D Display (courtesy of Virtual Research Co.) 32

33 Output Devices Auto-stereoscopic displays do d not require use of special glasses; Passive auto-stereoscopic displays do not track user s head and thus restrict user s position; Active auto-stereoscopic t displays track the head motion and give more freedom of motion. 33

34 d D Passive Auto-stereoscopic 3-D Display (Dimension Technologies Co.) The relation between the backlighting distance d and the distance to the user D determines a strea viewing cone 34

35 18.1 LCD display, Resolution: 1280x1024 (mono) 640 x 1024 (stereo) Weight: kg Passive Auto-stereoscopic 3-D Display (courtesy of Dimension Technologies Co.) 35

36 SynthaGram technology (StereoGraphics Co.) The lenticular screen is slanted The flat panel display is presented 9 images obtained by 9 virtual cameras The final image is produced in software by sub-pixel sampling and re-arranging the final image pixels in intimate juxtaposition with the optical elements of the lenticular screen The resultant image has a higher resolution in the horizontal direction The resultant image is sharper 36

37 40 LCD display, Resolution: 1280x768 pixels 70 o horizontal viewing (7 to 15 feet) Weight: 33.2 kg SynthaGram 404 (courtesy of StereoGraphics Co. - $12,000) 37

38 20 LCD display, Resolution: 1600x1200 (mono) 100 o horizontal viewing (1.5 to 6 feet) Weight: 8.4 kg SynthaGram 204 (courtesy of StereoGraphics Co. - $3,000) 38

39 18 LCD display, Resolution: 1280x1024 (mono) 640 x 1024 (stereo) Weight: 17 kg Active auto-stereoscopic 3-D Display (courtesy of Dresden 3D Co.) mechanical adaptation 39

40 TFT (thin film transistor) Active tracking accommodates 25 degrees Change in view direction electronical adaptation: AAC (auto adaptation coder) 40

41 Multi-user Auto-stereoscopic display How it works! The display redirects the appropriate frames to the right and left eye so that each eye can only see the relevant frame. The tracker locates each eye and sends the information to the control box. The control box would then tell the LCD screen what pixels to display. Through the optics system in the screen, the image will get directed through the thin film transistor (TFT) directly to the appropriate eye. A split second later it would do the same to the other eye. Hence, creating a 3D image. 41

42 Multi-user Auto-stereoscopic display Multiple users can be tracked simultaneously and more pixels can be opened up at any given time allowing light beams to be directed simultaneously to more than one eye and more than one 3D user. Position ii finders already track pupils of multiple l viewers with very small delay. Good resolution but still shows some flicker. OLED s becoming mainstream can help eliminate flicker What needs to be done? Better displays (100Hz 120Hz) Complete the multi-user concept 42

43 Multiple users auto-stereoscopic prototype 19 display from NextGen Technology 43

44 Sharp autostereoscopic laptop Pentium 4, 15 diagonal display, 1024x768 resolution, 2D and 3 D mode, uses parallax barrier. 44

45 Autostereoscopic cell phones! Ocuity (UK) and NEC make 2.5 diameter autostereoscopiccell phones. InTouch mobile handset (TTPCom) 2.1 Transflective 2D/3D TFT-LCD 132xRGBx176 pixel display Automatic control of 2D to 3D switching function Running TTPCom WGE 3D stereo game demonstration 45

46 Holographic displays The image source is based on standard flat panel technology of which the image is seen upon a nine optical layer glass panel. Objects will appear to float in space. For the maximum 3D effect, the background seen through the display should be several feet behind the display and dark in color. _ icrystal _ hd.pdf 46

47 Holographic displays EON TouchLight Bare-hand 3D interaction virtual reality display system VR scene can be zoomed, panned and rotated with both hands Uses image processing techniques to combine the output of two video cameras placed behind a semi-transparent plane in front of the user. Incorporated IR cameras and image processing board 47

48 48

49 Output Devices Large Volume Displays Allow several co-located users to view a monoscopic or stereoscopic view of the virtual world; Can be classified as monitor-based large volume displays or projector-based large volume displays. Allow more freedom of motion vs. personal displays. 49

50 Output Devices Monitor-based Large Volume Displays Use active or passive glasses; Several users can look at a monitor; Can C have a single monitor, or multiple li l side-byside monitors; If side-by-side, image continuity becomes an issue. 50

51 Untracked and wireless Tracked and wireless Refreshing screen at double the normal scan rate or 120~140 scans/sec IR controller directs orthochromatic liquid crystal shutters Active glasses 51

52 Output Devices Active glasses vs. FMDs Some advantages: no cables if head position is not tracked; light and ergonomic (can be used over vision glasses); work well with large volume displays; allows full screen resolution 1280x1024. Some disadvantages: lose 2/3 of image light intensity through LCD filtering; require special lcrt stereo ready that t has twice the hardware refresh rate (Hz) 120 Hz or more; require direct line of sight for IR controller; different viewing metaphor through the window. 52

53 Wireless old model Active glasses Wireless new model Wired to the synchronizing jack of the graphics card I-O Display Systems Inc. $99 vs. $1000 for StereoGraphics wireless glasses 53

54 Wired/wireless i glasses need a stereo enabler when connected to a VGA card without a 3-pin mini DIN output jack 54

55 Passive glasses vs. Active glasses How far you are intending to view the pictures from requires a certain separation between the cameras. This separation is called stereo base or stereo base line and results from the ratio of the distance to the image to the distance between your eyes. The mean interpupillary distance (IPD) is 63 mm (about 2.5 inches) (ex) image on the computer monitor from a distance of 1000: view ratio 1000/

56 Passive glasses vs. Active glasses 56

57 Passive glasses vs. Active glasses 57

58 Passive glasses vs. active glasses 58

59 Through the window metaphor The projection factor is changes by a factor K which such that exaggeration factor k = r (u U) + U Where: r is the responsiveness factor (optimally 1.25); u is the current head distance from the screen; U is the default distance (say 30 cm). Unfortunately t tracker jitter is amplified as well Better to exaggerate the image responseto the user s head motion 59

60 Active glasses system 60

61 Tiled monitors-based display VC 3.1 on book CD Resolution is 3840 x 1024 and dimensions are 1,1111 x029m

62 Non-synchronized tiled image discontinuity Synchronized tiled image 62

63 Output Devices Projector-based Large-Volume Displays Old technology is CRT-based (analog) three projector tubes (R, G, B); Requires special fast green coatingtoavoidthe to avoid the fogging due to fast switching (at 120 Hz); Suffer from low luminosity problems ( lumens) 63

64 Output Devices Projector-based Large-Volume Vl Displays Technology makes transition from CRT-based (analog) to Digital Micro-mirror Device (DMD) ) (digital) projectors; Workbench-type displays (Fakespace Responsive Workbench, Barco Baron, V-desk, etc.) Cave-type display (CAVE, RAVE) Wall-type displays Domes 64

65 Output Devices Digital Micro-mirror Device Display Light intensities are much larger than for CRT-based projectors 300 lumens to 1000 or more lumens Thus ambient light does not hinder image quality 65

66 Tilted surface Viewing Cone Reflector mirror Floor CRT projector (not shown) The old Fakespace ImmersaDesk workbench 66

67 IR Controllers CRT Projector Mirrors Tilting mechanism Baron workbench (courtesy of BARCO Co.) 67

68 Baron Workbench-type display geometries V-desk 68

69 CRT Projector Screen Mirror CAVE 3-D large volume display (courtesy of Fakespace Co.) 69

70 CAVE 3-D large volume display (courtesy of Fakespace Co.) 70

71 New types of stereo displays Such as BARCO Trace Driven by Barco Galaxy Stereo DLP projectors 3000 Lumens; 800:1 contrast ratio WARP geometry distortion for edge matching; 1400 x 1050 pixel resolution Barco Galaxy WARP: Barco integrates Real-Time 70 inch diagonal screen active stereo glasses Geometry Distortion into Active-Stereo DLP Projector 71

72 Microsoft SURFACE one large display (projector) five infrared cameras tracks k user s finger contact with the surface PC included in the enclosure 72

73 RAVE ( Re-configurable Virtual Environment ) Modular construction that allows various viewing configuration, from flat wall, to angled theater, to CAVE; Vertical wall image 2.3 m X 2.4 m; Several CRT projectors (260 lumens, 1280x1024 resolution); Takes 30 minutes or less to reconfigure 73

74 74

75 Output Devices Wall-type displays Accommodate more users Using a single projector on a large wall means small image resolution; Thus tiled displays place smaller images side-by-side i so they need multiple projectors; Images need to have overlap, to assure continuity; However overlap from two projectors means intensity discontinuity y( (brighter images in the overlap areas) Projectors need to modulate intensities to dim their light for overlap pixels. 75

76 Pano-Wall display Three projectors; Approx. 7 x 2 m 2 pp 76

77 PanoWall display 77

78 Output Devices 78

79 Tiled composite image from four projectors 79

80 Tiled composite image from four projectors after adjustment 80

81 Wll Wall and ddome-type displays Advantages: Accommodate more users (tens to hundreds) Give users more freedom of motion; Disadvantages: Large cost (up to millions of dollars); Even with multiple projectors, resolution is much lower than for CRTs (because the area is large). Example PanoWall has 200,000 pixels/m 2 while a monitor has 18,200,000 pixels/m 2 To have equal numbers of pixels/unit itare more projectors (military) 81

82 82

83 Output Devices 3-D Audio Displays Definition: Sound ddisplays are computer interfaces that provide synthetic sound feedback to the user interacting with the virtual world. The sound can be monoaural (both ears hear the same sound) or binaural (each ear hears a different sound). 83

84 Output Devices 3-D Audio Displays 3-D audio should not be confused with stereo sound; Human hearing model; HRTF(head-related transfer functions)-based 3-D sound; Convolvotron; 3-D sound cards. 84

85 Stereo vs. 3-D sound. 85

86 Output Devices Human Hearing Model Vertical-Polar coordinate system azimuth, elevation, distance (range); azimuth cues; elevation cues; Effect of pinna (outer ear); HRTFs 86

87 Output Devices Head Related Transfer Function (HRTF) 87

88 Output Devices 88

89 3-D Sound Effect of pinna filtering of sound (elevation and azimuth cues) 89

90 Output Devices NASA again a pioneer in 3-D sound Put microphones in dummy heads; Played localized sound and measured signal; Determined i d the HRTF; Worked on first circuitry; 90

91 3D sound localization 91

92 The Convolvotron PC 3-D sound boards. 92

93 The Huron workstation ti. 93

94 Output Devices 3-D Audio Displays 94

95 Sweet Spot Possible to create the illusion of many more phantom speakers surrounding the user and create effective azimuth localization. 95

96 Cross-talk effect Sound from one speaker reaches both ears: [][ [ H l,l H l,r ][ ] = Y left Y right l,l l,r H r,l H r,r S left S right where H l,l is the HRTF between the left speaker and the left ear, H l,r is the HRTF between the right speaker and the left ear, Y left is the sound reaching the left ear Y right is the sound reaching the right ear S left is the sound coming from the left loudspeaker S right is the sound coming from the right loudspeaker 96

97 Cross-talk effect cancellation Sound from both speakers e s is adjusted such that: -1 [][ S [ H l,l H l,r ] [ Y ] = S left S right H r,l H r,r Y left Y right where Y left and Y right are known (the output if the convolving process) 97

98 Commercial 3D Sound Cards What they have to offer: Digital Output Multi-speaker compatibility (7.1 channel format allows for 8 speakers) Positional Audio offers 3D dimensions of sound 98

99 Two Main Audio APIs DirectSound 3D (DS3D) Microsoft s DirectX component for positional audio Games Sound dcard and DS3D (Engines to Software Signals) Aureal 3D (A3D) Two Versions was very similar il to plain DS3D 2.0 could more accurately simulate how sound sources in a complex environment behave An extension of DS3D (by itself, its just reverb) PC environmental reverb standard created by Creative Lab All sound cards can have EAX capability EAX (Environmental Audio extensions) 99

100 EAX (Environmental Audio extensions) Multi-Environments Supports rendering of multiple simultaneous audio environments in real-time, enabling the creation of exceptionally realistic acoustic environments in games containing multiple locations/rooms where differences in size, texture and/or shape are present. Environment Panning Makes spatializing and localizing environments in 3D possible, providing new 3D gaming effects never heard before. Environment Reflections Offers localization of early reflections and echoes, bringing more detail and realism to 3D gaming. Environment Filtering Accurately simulates sound propagation in open and closed env. 100

101 EAX (Environmental Audio extensions) Environment Morphing Allows for seamless transition of audio from one environment to the next. Extreme Effects New EAX effects rendered dby an extended dreverberation algorithm that surpasses the complexity of many studio-based reverb processors. Enhanced 3D Audio Performance * Enhanced 3D audio performance through the correction of every positional delay and elimination of undesired audio artifacts (comb filtering) * Increased positional accuracy, especially elevation (vertical plane) that results in smoother movements and rotations. * HRTF filters and cross-talk cancellation algorithms for headphones, 2-speaker, 4-speaker, 5.1-speaker, 6.1-speaker and 7.1-speaker 101

102 Creative Labs Sound dbl Blaster Audigy 4 Pro High Definition Audio Quality for Playback and Recording Playback of 64 audio channels, each at an arbitrary sample rate; 24-bit Analog-to-Digital conversion of analog inputs at 96 khz sample rate 24-bit Digital-to-Analog conversion of digital sources at 96 khz to analog 7.1 speaker output 24-bit Digital-to-Analog it t l conversion of stereo digital it sources at 192 khz to stereo output t 16-bit to 24-bit recording sampling rates: 8, , 16, 22.05, 24, 32, 44.1, 48 and 96 khz. Supports Sony/Philips Digital Interface (SPDIF) format of up to 24-bit/96 khz quality. Selectable sampling rate of 44.1, 48 or 96 khz Low latency multitrack recording with ASIO 2.0 support at 16-bit, 48 khz and 24-bit, 96 khz resolution. 102

103 Turtle Beach Montego DDL 7.1 Sound Card Optical S/PDIF In/Out for playback of pure digital audio at resolutions of up to 24 Bits at 96kHz (out) sample rate playback and 16 Bits at 48kHz sample rate for recording (in). Allows for pass-through of Dolby Digital and DTS multi-channel DVD sound to external A/V receivers or digital it speaker systems Selectable 2, 4, 6 or 8 output channels with 24 bit playback at up to 96 khz sampling rate Converts stereo sound sources to multi-channel format, so you can listen to with enhanced multispeaker surround sound environment Pure surround sound and virtual surround sound from eight analog outputs provides 7.1 surround sound Supports Game Surround Sound APIs such as EAX 2.0 and A3D PCI Interface with bus mastering and burst modes 103

104 Commercial 3D Sound Cards Name Creative Sound Blaster Audigy 4 Pro Chip/3Dsound engine/api CA10200 ICT DSP/CreativeWa re/a3d 1.0, EAX Advanced HD 4 in/out 7.1-analog out 5.1-digital out (DIN) 2-digital in/out (Coaxial) 2-digital in/out optical ac3/dts passthru SRP $299 Philips Acoustic ThunderBird d 5.1-analog out $100 Edge Avenger/QSound 2-digital in/out (Coaxial) /A3D 1.0/EAX ac3/dtz pass-thru 2.0 Turtle Beach EAX 1 and 2, 7.1-analog out $80 Montego DDL 7.1 A3D, I3DL2 and DirectSound 3D Optical S/PDIF In/Out for audio resolutions 24 bit (out) 16 (in) sample rates 96kHz (out) and 48kHz (in)

105 Output Devices Haptics Haptic Interfaces Comes from Greek Hapthai meaning the sense of touch; Groups touch feedback and force feedback 105

106 Output Devices Touch Feedback Relies Rli on sensors in and close to the skin; Conveys y information on contact surface geometry, roughness, slippage, temperature; Does not actively resist user contact motion; Easier to implement than force feedback. 106

107 Output Devices Force Feedback Relies on sensors on muscle tendons and bones/joints proprioception; Conveys information on contact surface compliance, object weight, inertia; Actively Ati l resist ituser contact tmotion; More difficult to implement than touch feedback (no commercial products until mid 90s). 107

108 Haptic Interfaces Human touch sensing mechanism Most touch sensors are on the hand (much less density on other parts of the body); Four F primary types of sensors: 40 % are Meissner s corpuscles detect movement across the skin velocity detectors 25% are Merkel s disks measure pressure and vibrations 13 % are Pacinian corpuscles deeper in skin (dermis) acceleration sensors. Most sensitive to vibrations of about 250 Hz 19% are Rufini corpuscles detect skin shear and temperature changes 108

109 Haptic Interfaces Skin touch sensors 109

110 Haptic Interfaces Sensorial adaptation Measure the decrease in electrical signals from the skin sensor over time, for a constant stimulus; If the sensor produces a constant electrical discharge for a constant mechanical stimulus It is called Slow Adapting (SA); If the rate of electrical discharge drops rapidly over time for a constant stimulus called Rapidly Adapting (RA) 110

111 Haptic Interfaces Spatial resolution Measure the receptive field size of a sensor; If the sensor has a large receptive field it has low spatial resolution (Pacinian and Ruffini)SAII SA-II, RA-II If the receptive field is small has high spatial resolution (Meissner and Merkel) SA- I, RA-I 111

112 Haptic Interfaces Two-point limen test: 2.5 mm fingertip, 11 mm for palm, 67 mm for thigh 112

113 Haptic Interfaces 113

114 Haptic Interfaces Human grasping g configurations 114

115 Haptic Interfaces Maximum and sustained force exertion Maximum force exerted during power grasp Averages 400 N (male) and 225 N (female); Looking at body location, force output Grows G from 50 N at PIP finger joint, to 100 N at shoulder; Sustained force feedback is much smaller than maximum, owing to fatigue and pain 1N 은 1kg 의질량을가진물체를 1 m s2의가속도로가속시키는데필요한힘이다. 115

116 Haptic Interfaces Fatigue measured as a function of % Maximum Voluntary Contraction (MVC) and rest cycle 116

117 Haptic Interfaces Haptic feedback actuators Need to maximize power/weight ratio; Need to have high power/volume ratio; Need to have high bandwidth; Need d to have high hdynamic range (fidelity); Need to be safe for the user - None of the current actuator technology satisfies all these requirements 117

118 Haptic Interfaces Actuator comparison based on P/W ratio 118

119 Output Devices Touch hf Feedback ki Interfaces Can be desk-top or wearable (gloves); Touch feedback mouse; CyberTouch glove; Temperature feedback actuators; 119

120 Haptic Interfaces The ifeel Mouse (0-125 Hz). 120

121 Haptic Interfaces 6 individually Controlled Vibrotactile actuators Hz frequency 1.2 N amplitude at 125 Hz CyberTouch hgl Glove (Virtex) 121

122 Output Devices VC 3.3 on book CD 122

123 Output Devices Temperature feedback Added simulation realism by simulating surface thermal feel ; No moving gp parts; Uses thermoelectric pumps made of solid-state materials sandwiched between heat source and heat sink ; Single pump can produce 65 C differentials; 123

124 Temperature feedback actuator 124

125 User comfort zone C If system fails Heat travels back Through the pump And can burn skin Temperature feedback actuator control 125

126 Output Devices Force Feedback Interfaces Need mechanical grounding to resist user motion; Can C be grounded don desk, wall, or on user body; More difficult to construct and more expensive than tactile feedback interfaces 126

127 Haptic Interfaces 127

128 Uses potentiometers to sense position in spherical coordinates; Uses electrical actuators to apply resistive torques; Logitech Force feedback joystick 128

129 129

130 The PHANToM used for 3D sculpting (courtesy of SensAble Technology Co.) 130

131 131

132 PHANToM Omni 132

133 PHANToM Comparison 133

134 PHANToM 1.6/

135 Omega Haptic Device 3 DOF Force Feedback Delta Haptic Device 3 DOF Force Feedback Delta Haptic Device 6 DOF Force Feedback 3 DOF End-effector replaceable Resolution mm Max M continuous force 12 N Stiffness 14.5 N/mm Connectivity USB 2.0/PCI Full gravity compensation Real-time safety (velocity monitoring and electromagn. brakes) 3 DOF (36 cm diam 30 cm) Resolution.03 mm Max continuous force 20 N Stiffness 14.5 N/mm Connectivity PCI Real-time safety (velocity monitoring and electromagnetic brakes) 6 DOF (36 cm diam 30 cm) 20 o /axis Resolution.03 mm, 0.04 o Max M continuous force 20 N Torques continuous 0.2 Nm Stiffness 14.5 N/mm Connectivity y PCI Real-time safety (velocity monitoring and electromagnetic brakes) 135

136 NOVINT FALCON 3 DOF Force Feedback 3 DOF (right-left, lf forward/backward, d up/down) mm rumble, vibrations 3D exploration and textures t Dynamic effects (inertia, weight, momentum) Cost less than $300 Resolution >0.06 mm Max continuous force 10 N Stiffness 5N/mm Connectivity USB khz control bandwidth 136

137 3 DOF cylindrical robot The Haptic Master Max force output 250 N Stiffness 50 N/mm Uses force-in, position-out arrangement 137

138 Exoskeleton over CyberGlove) Cables and pulleys 16 N/finger (continuous?); Weight 539 grams; remote electrical actuators in a control box. The CyberGrasp force feedback glove 138

139 The CyberGrasp force feedback glove VC 3.4 on book CD 139

140 CyberGrasp glove Electronic interface box Tether Wrist Tracker The CyberPack (courtesy of Virtex Co.) 140

141 6 DOF mechanical arm Wrist position and Force feedback No need for a tracker Allows simulation of weight and inertia, not possible with glove-only only interfaces CyberForce interface (introduced recently) 141

142 Haptic Interfaces CyberForce interface VC 3.5 on book CD 142

143 143