Introduction to Virtual Environments - Spring 2004 - Wernert/Arns Lecture 3.1 Human Perception (II): Olfactory, Haptics, Motor Capabilities, & Sensory Interaction Outline 1. Olfactory 2. Haptics 3. Motor Capabilities 4. Sensory Comparison & Interaction 1. Olfactory Perception 1.1 General compared to other senses, olfaction is poorly understood lack of effective displays and difficulty in producing broad range of stimuli make research difficult available research is highly specific; other research is secretive (funded by perfume companies) 1.2 Olfactory Memory and Response memory for olfactory stimuli is more stable with the passing of time than visual stimuli olfactory stimuli are tied very closely with emotions and specific events/scenarios e.g., pine smell at Christmastime, grandma's apple pie, fresh cut grass, etc. commercial examples: perfume sample cards, aromatherapy entertainment examples: scratch n' sniff cards, scratch n' sniff movies (Nickelodeon's Rugrats Go Wild) 1.3 Olfactory Perception Mechanisms Olfaction (smell) is the gathering/sensing information about the chemical events in our environment olfactory epithelium area high in nasal cavity that is sensitive to odor approx. 1 square inch in total size approx. 10 million olfactory receptor cells olfactory bulb - smell center of the brain chemical stimuli for olfaction must be volatile (able to be vaporized) soluable in water and lipids must be chemicals we're capable of perceiving (we can perceive a large number of organic compounds) olfaction is an important adjunct to our common perception of taste olfactory bulb is associated with and connected to the limbic system (the "emotional brain") 1 of 7
1.4 Olfactory Perception Theories Lock and key - specific receptor sites fit the geometric shape of each type of molecule we can smell no research to support this level of specificity Receptors are sensitive to different, broader categories of stimuli; a particular odor will cause a variety of receptors to trigger yielding the percept seems to fit better with available evidence What are the odor "primaries"? (similar to red, green, blue for additive color and salty, sweet, sour, bitter for taste) Cater:caprylic (goaty), fragrant, acid, burnt Ackerman: minty,floral, ethereal (e.g., pears), musky, resinous (e.g., camphor), foul, acrid Henning: putrid, ethereal, resinous, burned, spicy, fragrant lack of agreement hinders research into effective displays DigiScents ismell (c. 2000, now defunct) based on approx. 200 primaries 1.5 Properties of Olfactory Perception olfactory events (odor sources) may be near or far; however, our directional sensitivity is generally poor our sensitivity is relatively high - we can detect odors in sparse conentrations (low threshold) range of sensitivity is large - we can perceive in very small or very large quantities temporal sensitivity is poor and response times are slow may need between 20-60 seconds between stimuli to resolve different smells ("cleansing of the palatte") no real control over stimulus decay rate(without significant air circulation or respiration control) odor combinations in some cases, they blend to form a new odor in other cases, they remain separately identifiable olfactory adaptation - desensitization to an odor through continued exposure olfactory masking - covering up one odor with another in larger concentrations 1.6 Olfactory Differences in population 2 of 7
sensitivity decreases as we reach old age women are more sensitive than men olfactory dysfunction Anosmia - Absence of smell sensation - about 50% of people over 80 Hyposmia - Decreased sensation Dysosmia - Distortion of smell sensation Cacosmia - Sensation of a bad or foul smell Parosmia - Sensation of smell in the absence of appropriate stimulus 2. Haptics 2.1 General Haptic - from the Greek, relates to physical contact or touch, includes tactile/cutaneous and kinesthetic/force perception Separation of kinesthetic and tactile perception in humans is nearly impossible; combining them in a computer display is difficult Haptic sense is a bidirectional sensory channel if you touch an object, you also affect it (unlike seeing or listening) combination of output and input latency is especially noticable Unlike visual and audio senses, it usually cannot be stimulated from a distance kinesthetic/force feedback require direct physical contact some cutaneous senses may be remotely stimulated: wind (fans), heat (heat lamps), etc. capable of multiple simultaneous channels of feedback 2.2 Tactile/Cutaneous Perception ability to detect mechanical, thermal, and electrical stimuli at the surface of the skin two types of skin - each has different type and concentrations of receptors skin with hair (~90%) skin without hair (glabrous, ~10%) ~2 sq. meters on average adult Mechanoreception receptor types (all present in glabrous skin of fingertips) slowly adapting type 1 - detects object shape slowly adapting type 2 - detects lateral skin stretch rapidly adapting - detects transient dynamic events Pacinan - detect vibration tactile stimulations step function - held for at least one second impulse function - shorter durations periodic function - vibrations sensitivity spatial just noticable difference (JND) up to 70 mm on back, 30 mm on forearm, 2 mm on fingertip precision - can detect a grating 0.06 microns high spatial and temporal summation - results in a fusion of tactile information to rapidly identify objects by touch Thermoreceptors 3 of 7
sensitivity to temperature varies greatly by part of the body (forearm, back, forehead are most sensitive) function of the area of exposure adaptation, rate of change issues 2.3 Kinesthetic/Force Perception Terminology kinesthesia - the perception of movement, position, and torque of the limbs and other body parts (e.g., swing a tennis racket) passive kinesthetic perception - limbs are moved by an external force active kinesthetic perception - movement is self induced force perception - sensitivity to the muscular opposition to mechanical forces (e.g. pushing on a door) proprioception - stimulation from within the body; an individual's ability to sense their own body posture, even when no forces are acting upon it. 75 joints in entire body; 44 in the hands alone homunculus perception of joint angles is more accurate in joints closer to the body sholder JND: 0.8 degrees wrist, elbow JND: 2.0 degrees Hand features hands can detect force step of 0.5 Newtons maximum force of human finger: ~ 40 newtons for precise manipulation, rarely exceeds 10 newtons asymetrical input/output 5-10 Hz motion control of hand requires feedback of 20-30 Hz Forces controlable forces increase from 3.7 lbs (most distal joint of fingers) to 23 lbs from the most proximal joint (shoulder) 140 lbs/in required to simulate stiffness of rigid object pressure resolution is greater as a function of area 4 of 7
(users will accept 20 Nt/cm as a solid, immovable wall) Frequency can perceive from 300-400 Hz (chattering) up to 5000-10,000 Hz (fine vibrations) to generate perception of smooth force, need 1000 Hz input Stages of force interaction initial dynamic surface contact quasi-static interaction with hard surface final dynamic release of the surface each can be measured with crispness, hardness, and cleanness 3. Motor Capabilities 3.1 Finger and Hand movements finger forces - sustained forces of 4-7 Newtons, momentary forces of 30-50 Nt hand force commands - issued at rate of 5-10 Hz perception of forces in the hand - as high as 1000 Hz hand strength and size - varies widely across the population finding proper size and force for wide range of users is difficult key-pressing - skilled typists can produce ~450 strokes per minute 3.2 Aimed Movements reaching, pointing, grasping, touching duration is dependent upon the distance and size of the target (Fitts Law) several postulated models of aimed movement - most contain an initial impulse phase followed by current-control phase based on sensory feedback stereo cues only helpful for local manipulation; depth compression effect the further we get to the extent of the arm (more when we discuss the design of VEs) 3.3 Locomotion methods of moving ourselves through the real world walking, running, skipping, crawling, hopping, swiming, climbing, cycling involve rythmic, repetitive actions influenced by sensory feedback (e.g. optic flow, force feedback from the floor, etc.) involve the kinesthetic, vestibular, and visual senses (more when we discuss the design of VEs) 3.4 Other Interesting/Relevant Motor Capabilities for VEs writing drawing sculpting tapping playing musical instuments sign language eye movements speech 4. Sensory Comparison and Interaction 5 of 7
Vision, Audition, Vestibular, Olfactory, Haptic 4.1 Comparing Senses relative reaction time/response rate relative acuity or sensitivity directionality, attentiveness stimulus production - can it be remotely induced or does it require direct contact emotional/memory evocations information throughput number of dimensions number of simultaneous/distinguishable channels sensitivity to environmentals 4.2 Combining the Senses temporal synchronization positional synchronization; collocation relative fidelity dissonance or mismatch 4.2.1 Sensory Adaption humans and human senses are highly adaptable e.g., mapping from horizontal mouse motion to vertical point motion humans can learn to overcome mismatches between stimuli; however, designers should use this capability knowingly and sparingly should provide readaption time where appropriate (e.g., driving simulation) 4.2.2 Sensory Combination Effects Ventriloquist effect - dominance of visual cues (and prior knowledge/experience) over audio localization visual cues have also shown dominance over haptic cues Transference of Object Permanence - making one object in the environment seem very real (e.g., using haptics) makes the rest of the world seem more real. Vestibular/visual cues - small/limited motions coupled with proper graphics and visual flow give a strong sense of broader range of physical motion 4.3 Georgia Tech study of Sensory Interaction Evaluating the Importance of Multi-Sensory Input on Learning and the Sense of Presence in Virtual Environments, H.Q. Dinh, N. Walker, C. Song, A. Kobayashi, and L. Hodges. IEEE Virtual Reality (VRAIS) 1999 Current project page: http://www.cs.stevens-tech.edu/~quynh/smellorama.html VR Equipment: HMD + Treadmill olfactory input from oxygen mask with tupperware containers of odors tactile input from fan and heat lamp 6 of 7
Virtual Environment: Office - interior and exterior balcony Experiment design Senses included: graphics - high resolution (local lights, high-res textured) or low res (ambient light, low-res textures) frame rate fixed at 20 FPS for both audio - present or not - copier machine, fan, etc. olfactory - present or not - flowers, coffee machine tactile - present or not - fan, heat lamp (outdoors) 16 possible test conditions 322 Ga. Tech students - 18 per condition tasks simple presence measurement: scale 0-100 presence questionnaire: 13 questions spatial layout questionnaire: 4 questions object location questionnaire: 5 objects Results presence visual fidelity did not have any effect on simple sense of presence audio and tactile feedback had positive, statistically significant correlation with presence olfactory feedback had positive correlation, but not statistically significant no significant interactive effects layout test no significant effects of interactions bathroom question (toilet flushing sound) did show significant effect (70% vs. 59%) object location significant effects of tactile feedback and olfactory feedback overall no significant interaction effects on presence - leads to an additive model surprising result regarding visual fidelity -> depending on objective, resources may best be invested on senses other than visuals. 7 of 7