From Encoding Sound to Encoding Touch

Transcription

1 From Encoding Sound to Encoding Touch Toktam Mahmoodi King s College London, UK ETSI STQ Workshop, May 2017

2 Immersing a person into the real environment with Very efficient codec & High fidelity devices at both ends High speed broadband services True immersion into remote environment is only possible if you can physically interact with the object/person. Telepresence

3 While the movement is communicated through action/reaction, the Sense of touch is received via sensory system of the skin. Kinaesthetic Position and Force refers to the movement & activation of muscles Tactile Sense of touch of the skin includes pressure, temperature, and texture. Full Immersion

4 @ Disney Research While principles of hearing and vision are established knowledge, touch is the less explored human perception. Among different information that shapes the sense of touch, we devise our attention to texture, given its complexity. Past works in [1] and [2] have shown that the high-frequency vibrations are the main identification of texture for human. We explore the idea of similarities between tactile and audio signals in time and frequency domain. Tactile Perception

5 Hearing: Sounds are the vibrations of air with the frequency and intensity of the vibrations determining the pitch and volume. When oscillations occur in air, the generated sound waves are collected by pinna (a part of the outer ear) before they are transmitted to the middle ear through the ear canal. Merkel Touch: human skin has eight types of mechanoreceptors controlling the sense of touch. We focuses on the glabrous skin of hands including four of the mechanoreceptors: Merkel cells, Ruffini endings, Meissner corpuscles and Pacinian corpuscles. Pacinian corpuscles are responsible for perceiving vibrational stimuli that are main cues for human to discriminate the texture of different material. Best stimulus Example Freq. range (Hz) Most Sensitive Freq. (Hz) cell Pressure, edges, corner, points Reading Braille Tactile & Sound Ruffini ending Stretch Holding large objects Meissner corpuscle Lateral motion Sensing Slippage of objects Pacinian corpuscle Highfrequency vibration Sensing texture / /

6 For comparing audio & tactile, first a good representation is needed. Perceptual characteristics of sound (pitch, loudness, timbre, duration), can be represented by a joint expression in both time and frequency domains. Subjective experimentation has shown acceleration signals are the most appropriate and effective way to represent the sense of touch. Audio and tactile signals are similar temporally and spectrally on the condition that we represent tactile signals as acceleration signals. Representation

7 Bandwidth The frequency range that humans normally can hear is from 20 Hz to 20 khz as long as the sound intensity is above a certain level. The sampling frequency must be at least 40 khz (Nyquist Rate) for perfect reconstruction. the Pacinian corpuscles sense vibrations of up to only 1 khz (which is yet the high-frequency vibrational signal) while the lowfrequency is generally lower than 5 Hz, relating to the movement of human joints and muscles. Dimensions Audio signals are low in dimensionality. Pacinian corpuscles are capable of sensing multi-dimensional vibrations. However, it has been shown that human are not quite sensitive to directions of the vibrations [3]. As a result, we need to transform three-axis vibrational tactile signal into one-axis signal which are Differences convenient for modelling and Irendering in haptic communication systems.

8 Requirements of Bitrate & Delay In data compression, there is always a trade-off between fidelity and bitrate. Due to different sensitivity and characteristics of ear and skin, the requirements of fidelity and bitrate for audio and tactile codecs are not the same. Past empirical results show that ms delay is not observable for ordinary hearing (excluding e.g. professional musicians) whereas the detection threshold of time-delay for tactile stimuli is approximately 41 ms [3]. Differences II

9 Experimentation Setup The setup is smoothly moving a pen attached with the accelerometer forward and backward between two points on the surface, which is to imitate human s natural movements to feel the texture. It is used it to record four types of materials, i.e. Aluminium Mesh, Desk wood, Paper, Tissue. Number of processing are done as follows: Filter, in order to remove gravity of earth and the vibrations beyond human s perception (1000Hz), a bandpass filter with lower band as 20Hz and upper band as 1000 Hz is applied. Resample, the accelerometer can record data with rate up to 3200 Hz, but the data needs to be resampled. Synthesis, since the vibrators in glove can only produce 1-dimensional vibrations, the 3-dimensional data recorded is reduced to 1- dimensional data using DFT321. Measurements

10 Measurements

11 Measurements

12 Measurements

13 Measurements

14 Audio and tactile signals are similar temporally and spectrally on the condition that we represent tactile signals as acceleration signals. As a result, it is theoretically possible to transform the audio codecs to tactile codecs regardless of the differences. Our study starts with PCM and DPCM and examine effect of various codec parameters on its efficiency in encoding texture. DPCM shows reasonable performance with different sampling frequency, etc. Concluding remarks

15 [1] K.J. Kuchenbecker, J. Fiene, and G. Niemeyer, Improving contact realism through event-based haptic feedback. IEEE Transactions on Visualization and Computer Graphics (2): pp [2] W. McMahan, J.M. Romano, A.M.A. Rahuman, and K.J. Kuchenbecker. High frequency acceleration feedback significantly increases the realism of haptically rendered textured surfaces. in IEEE Haptics Symposium IEEE. [3] S. Okamoto, M. Konyo, S. Saga, and S. Tadokoro. Identification of cutaneous detection thresholds against time-delay stimuli for tactile displays.. IEEE International Conference on Robotics and Automation [4] X Liu, M. Dohler, T. Mahmoodi, H. Liu, Challenges and Opportunities for Designing Tactile Codecs from Audio Codecs EUCNC available online: codec.pdf References

16 King s College London