Building a Cognitive Model of Tactile Sensations Based on Vibrotactile Stimuli

Transcription

1 Building a Cognitive Model of Tactile Sensations Based on Vibrotactile Stimuli Yuichi Muramatsu and Mihoko Niitsuma Department of Precision Mechanics Chuo University Tokyo, Japan Abstract We investigated the correspondence relationship between vibrotactile stimuli and tactile mental imagery using a vibrotactile glove proposed in a previous study as an interface for vibrotactile stimuli [1]. The mental impressions of the vibrotactile stimuli obtained by the human subjects were measured using the semantic differential method. Analysis of variance, multiple comparisons, and factor analyses were used to extract and evaluate the distinctive imageries induced by passive touch, based on the answers of the subjects to certain questions. To build a cognitive model of tactile sensations, we investigated the correspondence relationships between tactile sensations and vibrotactile stimuli. Keywords human perception; vibrotactile glove; tactile sensation; semantic differential I. INTRODUCTION The mental image created in the human mind when a physical object is touched or held is important because, although subjective, it is the basis of the perceived value of the object. More advanced interactions with virtual objects would be possible if tactile sensations could be experienced without actual contact with the objects [2]. Advanced technologies that virtually present information about an object in three dimensions using visual sensations have been developed and are now familiar. On the other hand, the presentation of such information using tactile sensations is relatively unfamiliar, although it has been studied [3-5]. Thus, a tactile interface is needed to present tactile information about objects in virtual space. Humans use their hands to interact with, manipulate, and evaluate surrounding objects, we therefore chose hands as the medium for presenting tactile sensations of virtual objects. Moreover, humans perceive tactile sensations of an object by direct contact, during which their hands are constantly moving. Hence, to recognize objects with their hands, they require not only sensations perceived while touching the objects, but also information received from the movement and posture of their hands [6]. More tactile information can be received through an active touch than through a passive one, an active touch is more effective for object recognition [7]. Tactile stimuli should therefore be changed in response to the movement of the fingers and hands (i.e., active touch) in virtual tactile sensation applications. The PHANTOM is a well-known haptic interface that is Trygve Thomessen PPM AS Trondheim, Norway often used in virtual spaces. It is, however, large and heavy and must be used in a fixed position, which limits the workspace of the user. Although useful for expressing the weight and hardness of virtual objects, it does not easily express the size and roughness [8]. Many researchers used vibrotactile stimuli for tactile interfaces such as a tactile display [9-14]. To produce tactile sensations in this study, we used vibrotactile stimuli generated by vibration motors, which have the advantages of ease of use, high portability, and lightweight interface design. We designed a tactile interface to generate vibration stimuli in response to the movement of the hand(s). Before discussing the tactile sensations of virtual objects, we will consider how humans perceive tactile sensations of a real object. The perception of tactile sensations by humans is as shown in Fig. 1 [15]. When a haptic sense was perceived and converted into tactile sensation by a human, he uses his memory and can verbally express the tactile sensation, and the exact expression was used to analyze the human impression of the sensation. Then, we extracted the words that expressed the tactile sensations. Okamoto et al. assumed that tactile perception comprised psychophysical and affective layers [16]. The psychophysical layer determines the perception of the physical properties of materials such as surface roughness and elasticity. The affective layer is mentally mediated and determines the perception of qualities such as richness and cleanliness. The Fig. 1. Mental imagery of tactile sensation [15] The authors thank to the Norwegian Research Council to support this project through the VRI program and JGC-S Scholarship foundation.

2 psychophysical layer is considered to be fundamental to the expression of tactile sensations. Therefore, in designing vibration stimuli to generate tactile sensations, we focused on the psychophysical layer. The studies on tactile structure revealed that the psychophysical layer has five dimensions, namely fine roughness, macro roughness, hardness, temperature, and friction (Fig. 2). Therefore, if a tactile interface could produce these five tactile sensations, it would effectively provide a user with the appropriate tactile perceptions for experiencing virtual objects. We previously designed a glove-type vibrotactile interface that produced tactile sensations of virtual objects when the user bent his/her fingers [1]. Since vibration stimuli generate tactile sensations that are different from the sensations of touching real objects, we investigated how vibration stimuli could be generated to evoke certain tactile sensations. To study the relationship between vibration stimuli and human impressions of the stimuli, we presented the stimuli to users of the glove, both passively and actively. In previous investigations, we examined the impressions of glove users of the stimuli when varying the duty ratio of the pulse-width modulation (PWM) and the activation time of the vibration stimuli. Finally, we evaluated the tactile mental imageries of the users for both passive and active touches, and observed a definite relationship between the vibration stimuli and the impressions of the users. However, designing the vibration stimuli requires a more precise understanding of the relationship between a small change in the vibration stimuli and the consequent change in the tactile mental imagery. We have also investigated the correspondence relationships between vibration stimuli and tactile sensations for passive touch using higher resolution vibration stimuli [17]. More specifically, the investigation involved variation of the duty ratio of the PWM and the activation time. In the present study, we investigated the correspondence relationships based on a PWM frequency and a cycle time of the vibration stimulus. This required the development of a high-resolution model for the generation of vibration stimuli when the glove is used for active touch. We also discuss the capability of the proposed vibrotactile interface to produce the five fundamental psychophysical dimensions. II. SYSTEM CONFIGURATION AND VIBRATION STIMULI A. Vibrotactile Glove The proposed vibrotactile glove is shown in Fig. 3. Diskshaped vibration motors (FM64F; Tokyo Parts Industrial Co., Ltd.) were used as the actuators of the lightweight vibrotactile interface. The vibration motors were installed on the fingernails because the latter are very sensitive [7], [18]. Bending sensors were also installed to track the movements of the fingers. Each vibration motor and bending sensor was controlled through a microcomputer. The vibration motors on the glove were controlled by commands from a vibration generator. The vibrotactile glove communicates with the vibration generator via Bluetooth. The proposed vibrotactile glove has a total weight of 187 g and its circuit board measures 80 mm 55 mm 30 mm. B. System Configuration The system configuration for producing tactile sensations of virtual objects is shown in Fig. 4. The components in bold rectangles were used for the experiment. The system comprises a server and the vibrotactile glove. The server consists of a tactile information generator and a vibration generator. The tactile information generator is used as the database of the correspondence relationships between the vibration stimuli and the tactile sensations, and supplies the relationship information to the vibration generator. The vibration generator produces the appropriate vibration stimuli based on data obtained from the five bending sensors and the desired tactile sensations. The data from the five bending Fig. 3. Vibrotactile glove Fig. 2. Five psychophysical dimensions of tactile perception Fig. 4. System configuration

3 sensors are gathered by a microcomputer and sent to the server. The vibration stimulus in the vibration generator is specified based on this data and information produced by the tactile information generator. The vibration motors generate a vibration stimulus when a command is sent to the microcomputer. The dashed lines in Fig. 4 represent wireless communications among the components. C. Design of Stimulus for Producing Tactile Sensation The vibration stimuli were designed based on the voltage impressed on the vibration motors. The duty ratio, PWM frequency, cycle time, and activation time during a cycle time were regarded as design parameters of the impressed voltage. The relationships among these parameters are presented in Table I. The activation time is the period during which a vibration is activated during a cycle. A method for designing the vibration stimuli to produce the proper tactile sensations was required. We therefore investigated the relationships between tactile sensations and each design parameter of the impressed voltage to develop a model for generating the vibration stimuli that produce the desired tactile sensations. III. MENTAL IMAGERY OF VIBROTACTILE STIMULI A. Experimental Method We conducted an experiment to investigate the relationship between vibration stimuli and tactile sensations for passive touch. We used four parameters of the impressed voltage, namely the duty ratio, activation time during a cycle, PWM frequency, and cycle time. Four approaches were applied in designing the vibration stimuli: (1) using various duty ratios and constant activation time, PWM frequency, and cycle time; (2) using various activation time ratios and constant duty ratio, PWM frequency, and cycle time; (3) using various PWM frequencies and constant duty ratio, activation time, and cycle time; and (4) using various cycle times and constant duty ratio, activation time, and PWM frequency. The produced vibration stimuli are listed in Table II. The intensity of a vibration stimulus could be set to one of eleven levels by changing the PWM duty ratio for a given impressed voltage of 5 V. The activation time of the vibration stimulus could also be set to one of eleven levels. The frequency of the vibration stimulus could be set to one of seven levels by changing the PWM frequency. The cycle time of the vibration stimulus could also be set to one of seven levels. In this experiment, (1) while using various duty ratios, the activation time ratio was 1.0, and the PWM frequency was 145 Hz; (2) while using various activation time ratios, the duty ratio was 0.7, the PWM frequency was 145 Hz, and the cycle time was 250 ms; (3) while using various PWM frequencies, the duty ratio was 0.7 and the activation time ratio was 1.0; TABLE IV. Design parameters of impressed voltage Length of Activation time Frequency PWM Duty ratio PWM frequency Vibration cycle Activation time during a cycle Cycle time and (4) while using various cycle times, the duty ratio was 0.7, the activation time ratio was 0.5 and the PWM frequency was 145 Hz. The vibration stimuli were produced using the levels listed in Table II, with the exception of level 0. B. Experimental procedure Twenty-five subjects comprising 24 males and one female, all in their 20s, participated in the experiment. The hand fitted with the vibrotactile glove was hidden from the subject to avoid biasing the mental imagery. The impressions of the vibration stimuli obtained by the subjects were determined by the semantic differential (SD) method. Each subject was required to score each of the twelve pairs of adjectives listed in Table III on a scale of one to seven. Each adjective pair described a type of tactile sensation. The tactile sensations were categorized into groups by factor analysis. By analysis of variance (ANOVA) and multiple comparisons, we determined which pair of adjectives corresponded to each vibration stimulus. The tactile sensations that were identified to belong to the same or different groups were determined by factor analysis. A tactile sensation was considered to correspond to a vibration stimulus if the average score of the pair of adjectives used for the tactile sensation changed in tandem with the PWM duty ratio, vibration TABLE III. Vibration stimuli Level Duty Activation PWM frequency Cycle ratio time ratio [Hz] time [ms] (Activation time during a cycle) = (Cycle time) (Activation time ratio) TABLE III. Pairs of adjectives hard/soft (hardness) cold/warm (temperature) rough/smooth (fine roughness) even/uneven (macro roughness) moist/dry (friction) sticky/slippery (friction) heavy/light (weight) elastic/inelastic (elasticity) large/small (size) like/dislike natural/artificial attractive/repulsive Psychophysical layer Affective layer

4 activation time, PWM frequency, or vibration cycle time. We then focused on only the pairs of adjectives in the psychophysical layer. C. Experimental Results and Discussion We focused on the psychophysical adjective pairs with significant differences in the results of the analysis of variance and multiple comparisons. The Italic words indicate factors that were extracted by factor analysis. The naming of each factor was based on the sensation that was most closely related to the vibration stimuli. The experimentally determined correspondence relationships between the duty ratio of the PWM and the tactile sensations and between the vibration activation time and the tactile sensations [17]. The result of the correspondence relationships between the duty ratio of PWM and tactile sensations are shown in Fig. 5 and Table IV. We observed correspondence relationships between the PWM duty ratio and the tactile imagery of virtual objects perceived by passive touch for tactile sensations such as hardness (hardness, friction (sticky/slippery), elasticity, or size), roughness (macro roughness), and weight. The result of the correspondence relationships between the vibration activation time and tactile sensations are shown in Fig. 6 and Table V. The correspondence relationships between the vibration activation time and the tactile imagery were also TABLE IV. Factor analysis results for varied duty ratio [17] Factor Likability Hardness Roughness Weight like/dislike attractive/repulsive natural/artificial moist/dry sticky/slippery hard/soft large/small elastic/inelastic even/uneven rough/smooth cold/warm heavy/light Blanks indicate that the absolute value was less than 0.1. observed for other tactile sensations such as smoothness (fine roughness or macro roughness), weight, friction (sticky/slippery), elasticity, and hardness (hardness or size). Fig. 7 shows the responses of the subjects to the SD questions when the PWM frequency was varied. The ANOVA results showed significant differences among the impressions of each subject when presented with the adjective pairs for fine roughness (F(5, 144) = , p <.05), macro roughness (a) Hardness (a) Temperature (b) Temperature (b) Macro roughness (c) Weight Fig. 5. SD question results for varied duty ratio [17] (c) Elasticity Fig. 6. SD question results for varied activation time [17]

5 TABLE V. Factor analysis results for varied activation times [17] Factor Likabil Smooth Fric Elas Hard Weight ity ness tion ticity ness like/dislike attractive/ repulsive even/uneven rough/ smooth natural/ artificial heavy/light cold/warm sticky/ slippery elastic/ inelastic hard/soft large/small moist/dry Blanks indicate that the absolute value was less than 0.1. (F(5, 144) = , p <.05), and friction (sticky/slippery; F(5, 144) = 8.258, p <.05). In addition, using Tukey s method for multiple comparisons, we found that all the impressions of the subjects of the extracted adjective pairs were significantly different for the some different levels of the PWM frequency of the vibration, as shown in Fig. 7. The asterisks indicate significant differences in the vibration levels. The SD results show that, when the PWM frequency was varied, a corresponding change was observed in the tactile imagery of the fine roughness, macro roughness, and friction (sticky/slippery) of the virtual objects. In particular, fine roughness and macro roughness were observed to be strongly related to the vibration PWM frequency as indicated by the significant differences for several cases (Figs. 7(b) and (c)). The results of the factor analysis when the PWM frequency was changed are presented in Table VI. All the tactile sensations that exhibited significant differences (fine roughness, macro roughness, and friction (sticky/slippery)) are included under the smoothness factor. Hence, a sensation that TABLE VI. Factor analysis results for varied PWM freqency Factor Smoothness Likability Temperature Weight even/uneven rough/smooth sticky/slippery like/dislike natural/artificial attractive/repulsive elastic/inelastic cold/warm moist/dry heavy/light hard/soft large/small Blanks indicate that the absolute value was less than 0.1. (b) (c) (a) Temperature Fine Roughness Macro roughness (d) Friction Fig. 7. Responses to SD questions where PWM frequency was varied belongs to the smoothness factor can be changed by varying the appropriate PWM frequency. Therefore, the proposed vibrotactile glove enables the user to perceive the tactile sensation of smoothness of virtual objects. When the cycle time was varied, no correspondence relationship between the vibrotactile stimuli and tactile sensations was observed. Furthermore, no correspondence relationship was observed between the vibrotactile stimuli and the tactile sensations of temperature (cold/warm) a fundamental dimension of the psychophysical layer when the duty ratio of the PWM, activation time, and PWM frequency (F(5, 144) = 2.794, p <.05) were varied (Figs. 5(b), 6(a), and 7(a)). The experimental results prove that the proposed vibrotactile glove has the potential for presenting hardness and

6 fine roughness by varying the duty ratio of the PWM; for presenting hardness, fine or macro roughness, and friction by varying the vibration activation time; and for presenting fine roughness, macro roughness, or friction by varying the PWM frequency to a user as fundamental sensations in the psychophysical layer. Temperature is the only sensation in the psychophysical layer that cannot be presented by the proposed vibrotactile glove. Furthermore, the proposed vibrotactile glove has the potential for presenting weight by varying the duty ratio of the PWM, and for presenting both weight and elasticity by varying the vibration activation time. The structure of the psychophysical layer dimensions that can be presented by the proposed vibrotactile glove is shown in Fig. 8. IV. CONCLUSIONS We observed the relationships between the vibration PWM frequency and the tactile imagery of sensations such as smoothness. We could not observe correspondence relationship between the vibrotactile stimuli and the sensations when the cycle time was varied. The experimental results show that the proposed vibrotactile glove can present hardness, fine roughness, macro roughness, and friction stimuli to a user as fundamental sensations in the psychophysical layer. It can also present weight and elasticity stimuli to a user in the psychophysical layer. However, the glove requires modification to present temperature stimuli, which is the outstanding class of basic sensations in the psychophysical layer. Through the experiments, we found that the produced vibrotactile stimuli could not present fine roughness and macro roughness respectively, although these sensations exhibited high relationships with the PWM frequency and activation time. However, if a user could actively touch an object, s/he would be able to interpret the vibrotactile stimuli based on his/her motions. We therefore expect that a user would be able to distinguish two kinds of sensations from the vibrotactile stimuli by an active touch. In a future work, we will investigate active touch by varying the four variable parameters. The design of the vibration stimuli will be based on the physical quantities of real objects. REFERENCES [1] Y. Muramatsu, M. Niitsuma, and T. Thomessen, Perception of Tactile Sensation Using Vibrotactile Glove Interface, IEEE Int. Conf. on Cognitive Infocommunications, pp , [2] P. Baranyi and A. Csapo, Definition and Synergies of Cognitive Infocommunications, Acta Polytechnica Hungarica, Vol. 9, No. 1, pp , [3] Y. Kunii, Y. Nishino, T. Kitada, and H. Hashimoto, Development of 20 DOF glove type haptic interface device-sensor Glove II, Advanced Intelligent Mechatronics 97, [4] P. Galambos, A. Roka, P. Baranyi, and P. Korondi, Contrast Vision- Based Grasp Force Feedback in Telemanipulation, Advanced Intelligent Mechatronics 2010, pp , [5] A. Csapo and P. Baranyi, An interaction-based model for auditory substitution of tactile percepts, Intelligent Engineering Systems 2010, pp , [6] C. Pagano, C. Carello, and M. Turvey, Exteroception and exproprioception by dynamic touch are different functions of the inertia tensor, Perception & Psychophysics, pp , [7] H. Ando and T. Maeda, Haptics Display Method using Finger Trace Illusion, Journal of the Robotics Society of Japan, Vol. 30 No. 5, pp , 2012 (in Japanese). [8] J. Martinez, D. Martinez, J. P. Molina, and A. Garcia, Comparison of Force and Vibrotactile Feedback with Direct Stimulation for Texture Recognition, Int. Conf. on Cyberworlds, pp , [9] K. Sakamoto, Y. Shimizu, K. Mito, and M. Takanokura, Tremor and Vibratory Perception in a Living Body Functional Evaluation of Mechanical Vibration, Society of Biomechanisms Japan, ISBN , 2009 (in Japanese). [10] T. Sakurai, M. Konyo, and S. Tadokoro, Presenting Sharp Surface Shapes Using Overlapped Vibrotactile Stimuli, IEEE/RSJ Int. Conf. onintelligent Robots and Systems, pp , [11] S. Asano, S. Okamoto, Y. Matsuura, H. Nagano, and Y. Yamada, Vibrotactile Display Approach that Modifies Roughness Sensations of Real Textures, IEEE Int. Symposium on Robot and Human Interactive Communication, pp , [12] K. O. Sofia and L. A. Jones, Mechanical and Psychophysical Studies of Surface Wave Propagation during Vibrotactile Stimulation, IEEE Transactions on HAPTICS, pp , [13] Y. E. Song, M. Niitsuma, T. Kubota, H. Hashimoto and H. I. Son, Mobile Multimodal Human-Robot Interface for Virtual Collaboration, IEEE Int. Conf. on Cognitive Infocommunications, pp , [14] J. Kang, J. Lee, H. Kim, K. Cho, S. Wang, and J, Ryu, Smooth Vibrotactile Flow Generation Using Two Piezoelectric Actuators, IEEE Transactions on HAPTICS, pp , [15] M. Nakatani, Y. Kakehi, and H. Shirado, Shokkan wo tukuru, Iwanami Shoten, ISBN X, 2011 (in Japanese). [16] S. Okamoto, H. Nagano, and Y. Yamada, Psychophysical Dimensions of Tactile Perception of Textures, IEEE Transactions on HAPTICS, pp , [17] Y. Muramatsu and M. Niitsuma, Correspondence Relationships between Vibrotactile Stimuli and Tactile Sensations Determined by Semantic Differential, IEEE Int. Symposium on Robot and Human Interactive Communication, pp , [18] P. Galambos, Vibrotactile Feedback for Haptics and Telemanipulation: Survey, Concept and Experiment, Acta Polytechnica Hungarica, Vol. 9 No. 1, pp , Fig. 8. Psychophysical dimensions provided by the glove