UPDRS tests for Diagnosis of Parkinson s Disease Employing Virtual-Touchpad

Transcription

1 2010 Workshops on Database and Expert Systems Applications UPDRS tests for Diagnosis of Parkinson s Disease Employing Virtual-Touchpad Adam Kupryjanow 1, Bartosz Kunka 2, Bożena Kostek 3 Faculty of Electronics, Telecommunications and Informatics Gdansk University of Technology Narutowicza 11/12, Gdansk, Poland 1 adamq@sound.eti.pg.gda.pl 2 kuneck@sound.eti.pg.gda.pl 3 bozenka@sound.eti.pg.gda.pl Abstract This paper presents a new approach to diagnosing Parkinson s disease. The progression of the disease can be measured by the UPDRS (Unified Parkinson Disease Rating Scale) scale which is used to evaluate behavioral and motor symptoms of Parkinson's disease. Hitherto the evaluation of the advancement of the disease in the UPDRS scale is made by a specialist through medical observation. The authors suggest a partial automation of this process, i.e. using a multimodal interface called Virtual-Touchpad (VTP) may support the medical diagnosis. The VTP is based on processing the image of hand acquired by a camera. Therefore, a patient is not constrained by any equipment. Two types of UPDRS tests that can be supported by VTP are presented in the paper. In the Finger Taps test the patient taps thumb with the index finger in rapid succession. The Rapid Alternating Movement of Hands test consists in pronation-supination movements of hands. VTP captures the image of the patient s hand while performing the test and, based on this image, it assigns hand gesture to the given class. The hand movement velocity is measured and used in the diagnostic process. Keywords-diagnosis of Parkinson s Disease, multimodal interface, virtual touchpad I. INTRODUCTION Current development of multimodal interfaces contributes to the design of alternative methods of humancomputer communication. The interface presented in this paper is based on image processing. A patient can be diagnosed remotely through acquiring hand image as a collection of digital data. This approach combines advanced image processing methods with typical medical procedures such as diagnosis of the patient, resulting in a telemedicine service. Telemedicine (medicine at a distance) is a branch of clinical medicine where medical information is transferred through interactive audiovisual media for the purpose of consulting or performing remote medical procedures or examinations. Telemedicine can be associated with three different types of services: store-and-forward, remote monitoring and interactive services. Store-and-forward telemedicine consists in acquiring medical data (i.e. medical images, biosignals) and then transmitting these data to a physician or a medical specialist at a convenient time for the assessment. This type of telemedicine does not require the presence of the patient and the doctor at the same time. Remote monitoring, also known as self-monitoring enables medical professionals to monitor a patient remotely using various technological devices. This method is primarily used for managing chronic diseases or specific conditions, such as heart disease, diabetes etc. Interactive telemedicine services provide real-time patient-doctor interactions. Phone conversations or online communication could be regarded as examples of interactive telemedicine services [1]. Parkinson s Disease affects nearly 1% of people older than 60 years [2]. It is a well-known fact that symptoms and disease progress may vary for individual patients. Therefore every patient should be diagnosed and treated by a neurologist. This means that both PD diagnosis and the treatment are patient-dependent. During the clinical examinations patient s state is assed using various subjective tests. Every specialist is using his own judgment during the examination. Therefore results of the test for one patient made by different doctors could be different. More details about PD diagnosis could be found in the literature [2] [3] [4]. Virtual-Touchpad is an interface which can support a diagnosis of Parkinson s disease and its progress. The patient can perform the initial phase of the medical examination at home and send the results electronically to the physician. The doctor can analyze these results in a convenient time and decide whether to recall the patient to the hospital or continue the remote monitoring of patient s health. Thus, the VTP can be a supporting device of storeand-forward telemedicine. The paper is organized as follows: in the section II Related Works, systems developed for PD patients tests were described. In the section III Virtual-Touchpad hardware and software configurations of the proposed multimodal interface is presented. Section IV UPDRS tests explains how the proposed interface could be used in the UPDRS test. The presented solution is concluded in the last section of the paper. II. RELATED WORKS R. Okuno et al. developed finger tap measurement system [5]. The system was composed of 3-axis accelerometers, touch sensors, AD converter and a personal computer. The fingerstalls are worn on the subject s index finger and /10 $ IEEE DOI /DEXA

2 thumb, respectively. The hand is fixed on the desk. The plane that the index finger flexed or extended was parallel to the desk. The outputs of the sensors are recorded using PC. The system works correctly with right and left hand [5]. physician s observations. Thus, it may be said that this is the decision-making support for the MD (medical doctor). Figure 1. Accelerometers and touch sensors on fingers [5]. The other approach to measurement and evaluation of finger tapping is a system presented in Fig. 2. It consists of a magnetic sensor and an ordinary personal computer (PC). The user performs finger tapping movements with two magnetic sensor coils attached to the distal parts. The magnetic sensor output voltages reflect the distance between the two coils. The measured voltages are converted into values representing the distance between two fingertips based on a nonlinear calibration model in the PC. Further, the features of the movements measured are computed for evaluation of the finger taps [6]. Figure 2. Conceptual scheme of the system based on magnetic sensors [6]. There exists an approach based on force estimation in finger tapping. Relationships between the fingerpad deformation of the thumb, index finger and the fingerpad force are approximated. A special device consisting of the magnetic sensor, force sensor and a multitelemeter system was made for such a purpose. Figure 3 presents a concept of this experiment [7]. All the above methods consist in using additional hardware such as special sensors attached to the thumb and the index finger. The method described in this paper is easier to perform and more reliable than the methods mentioned above because it does not require an additional equipment that is attached to the patient s hand. The entire process associated with the diagnosis of Parkinson's disease runs in real (home/hospital) conditions, therefore the examination findings may be more reliable. Moreover the automatic evaluation process may be compared with the Figure 3. Experimental apparatus for identification of fingertip forces [7]. III. VIRTUAL-TOUCHPAD Virtual-Touchpad (VTP) is a multimodal interface developed at the Multimedia Systems Department (MSD) of Gdansk University of Technology (GUT). It belongs to the group of HCI interfaces (Human-Computer Interaction) because it enables to control the computer mouse and cooperate with dedicated applications. VTP is an interface which consists of two layers: hardware and software. A. Hardware layer of the VTP description According to the assumptions which take into account that the interface engineered should not be expensive, the hardware layer is not technically very sophisticated. Moreover the VTP is a mobile interface. The most important component is an ordinary webcam with a dedicated stand and a computational unit, which may be the average PC computer. The chosen camera was designed for portable computers, making it lightweight and convenient to use. Its lens focal length is 12 mm which allows placing the camera at such a height that unwanted objects are out of its field of view. The camera stand was designed specifically for this interface. The arm of the stand, on which the camera is mounted, enables setting the camera distance from the desk. Depending on the size of the active area in which the user is able to move his/her hand, the camera is placed at a suitable height. Fig.4 presents a user working with Virtual-Touchpad [8]. It should be noted that the image processing results are directly associated with the webcam parameters. The most important parameter in this context is the frame rate, i.e. number of frames per second captured by the camera (fps - frames per second). The camera used in the interface has 30 fps but the software layer processes 15 frames effectively. Such time resolution makes possible real-time operation of the system. 133

3 Figure 4. Virtual-Touchpad in use. B. Software layer of the VTP description The functional scheme of the interface is presented in Fig. 5. Software layer consists in two stages: image processing (hand detection and hand classification) and the image analysis. As a result of the image processing a gesture of hand detected in the image is classified. Steps of the algorithm associated with the image processing can be described as follows. The hand detection algorithm is based on the background subtraction combined with contour finding algorithms. For the background subtraction the static model of the scene is calculated for every pixel of the image as the mean value of the pixel and the mean value of the differences of this pixel for the first 2 s of the recording. A pixel is defined as the foreground if its intensity is out of the range of values that represents the background model. To minimize distortions provided by the shadow of the hand, the shadow removal algorithm was employed. This algorithm is a typical solution based on thresholding operation in the YCrCb color space where a comparison of thresholds is performed according to the foreground pixels obtained by the background subtraction algorithm and the background model [9]. To minimize the discontinuity of the detected foreground object, the morphologic closing operation is performed. Then the contour finding algorithm searches for every object with the region larger than 5000 pixels. The first one that is found is defined as hand. During the analysis of the hand gesture, the mask of the hand is used for the classification. Gesture classification is performed every time the hand is detected in the movie image. The classification algorithm utilizes the SVM classifier. Four SVM classifiers were trained separately for every gesture. Gesture recognition is obtained by the analysis of the outputs of the individual SVM classifiers. Every classifier returns the decision (gesture belongs to the given class or not) and the probability of this decision. The probability is obtained using the libsvm implementation of the probability model. Probabilities of the last five positive decisions for every classifier are saved in the buffer. The final decision is obtained by calculation of the highest mean probability from the buffer. To reduce the number of the false positive detections the mean value of the highest probability is compared to the threshold default set to 95 %. If the mean probability is less than this threshold no gesture is classified. It is worth noting that the hand detection stage is supported by data obtained in the calibration process which was conducted at the beginning of using the interface. Overall, the VTP calibration process consists in getting patterns of hand gestures made by the user in a given sequence. During the calibration a sequence of two types of image processing operations is performed for each gesture. The first stage, called transition stage, consists in preparing a given hand gesture in the camera window preview. The image of the hand gesture is not analyzed. Then, in the second stage, called learning, parameters used in training SVM classifier are calculated while processing the hand gestures images. Learning stage requires 35 frames for each gesture. Features used for the training stage are related to the number of pixels that belong to hand placed on the half line beginning in the centre of gravity of the hand mask and calculated for various angles of the half line. The angle is changed from 0 to 360 degrees with the hop of 2 degrees. The second operation of the algorithm is the image analysis. This stage consists in monitoring the following parameters: velocity of gesture changes, periodicity of gesture repetition and time performance of each gesture. The description of the image analysis is contained in Section III (b). Figure 5. Functional scheme of the VTP. 134

4 IV. UPDRS TESTS Parkinson s disease (PD) is a neurodegenerative disease which affects middle aged and elderly people. Tremor, rigidity, akinesia, and gait disturbance are the most common symptoms of this disease. The major treatment is pharmacotherapy, particularly L-dopa. Quantitative evaluation of the symptoms is important for dosage of the drugs. Unified Parkinson Disease Rating Scale (UPDRS) is currently a standard and widely used scale for PD diagnosing. The UPDRS scale was developed to incorporate elements from existing scales and provide a comprehensive but efficient and flexible means to monitor PD-related disability [10] [11] [12]. The actual version of this scale enables to conduct 42 various tests diagnosing Parkinson s disease. The composition of these tests includes, among others cognitive assessment, e.g. evaluation of thought disorder, depression, as well as motor dysfunction, such as: speech, swallowing, walking, tremor, facial expression, leg agility, posture and finger tapping, hand movements and rapid alternating movements of hands. The described interface could be employed in the last mentioned UPDRS test, however authors decided to apply it effectively in two tests described below. Virtual-Touchpad employed in UPDRS tests Employing the VTP in tests for diagnosis Parkinson s disease seems quite a demanding task. Nevertheless, applying advanced methods of the image processing and SVM classifiers may ensure a correct and stable operation of the developed interface. A collection of hand gestures which are important in the UPDRS tests and recognized by VTP is presented in Fig. 6. The first group consists of test gestures (a) and refers to the test UPDRS No. 23, The Finger Taps (b). The second test, corresponding to the UPDRS No. 25 test, i.e. The Rapid Alternating Movement of Hands, is based on gestures shown in Fig. 6c and 6d. Both tests consist in performing gestures that are associated with a given test alternately. a) b) c) d) As mentioned in Section III (b) it is possible to estimate three parameters which are directly associated with hand observations according to UPDRS tests. The Finger Taps and The Rapid Alternating Movement of Hands tests are evaluated by the same criteria. Hand motor activity, like all other UPDRS tests, is assessed on a five-score scale. When a subject performs the given hand gesture correctly, he/she receives 0 score in the test. The description of five-score scale of two UPDRS tests is included in Table I. TABLE I. SCORES DESCRIPTION OF UPDRS TESTS NO 23 AND 25 [13] UPDRS score Description 0 Normal. 1 Mild slowing and/or reduction in amplitude. Moderately impaired. Definite and early 2 fatiguing. May have occasional arrests in movement. Severely impaired. Frequent hesitation in 3 initiating movements or arrests in ongoing movement. 4 Can barely perform the task. The first parameter which can be determined using the VTP and based on the analysis of hand motor activity is velocity. The velocity is defined as the number of gestures performed in time unit. It implies that the velocity is represented by the sum of activity states of two gestures. Functions in Fig. 7 show examples of hand motor activity independently for each gesture: i.e. pronation and supination. The graphical interpretation of the velocity based on graphs from Fig. 7 indicates that the subject performed six gestures in time unit. The second parameter, which may be designed based on data obtained from the VTP, is the so-called periodicity. This parameter is related to the repetition of gesture sequence in a given period of time. The subject whose hand activity is visualized in Fig. 7 performed this test task irregularly. The last, third parameter is the time performance of a particular gesture. Assuming that this parameter is set for gesture 1, the formula of determining this parameter can be written as follows. The desired value of this parameter should be close to 0.5. T1 C1 = (1) T + T C 1 time performance of gesture 1 T 1 total time of activity states of gesture 1 [s] T 2 total time of activity states of gesture 2 [s] 1 2 Figure 6. Two groups of recognized gestures: a), b) open-closed hand; c), d) pronation-supination of hand. 135

5 1 Figure 7. Hand motor activity in the UPDRS test. V. CONCLUSIONS This paper presents a new approach to support the diagnosis of Parkinson's disease based on the UPDRS test and the image analysis. The multimodal interface called Virtual-Touchpad was developed at the MSD. The presented study focuses on the methodology of diagnosing patients using the VTP as well as on the proposal of the interpretation of data obtained from this interface. Currently, the authors are preparing tests to be conducted with Parkinson's disease patients based on the existing cooperation with neurologists from St. Adalbert Hospital in Gdansk. These tests may allow for specifying rules which will be used in classifiers utilizing UPDRS scores. The physician s role will be to evaluate the proposed methodology in clinical conditions and also to verify employed rules. ACKNOWLEDGMENT Research funded within the project No. POIG /08, entitled "Elaboration of a series of multimodal interfaces and their implementation to educational, medical, security and industrial applications". The project is subsidized partially by the European regional development fund and by the Polish State budget". REFERENCES [1] R. Rudowski, Medical Informatics, Warsaw, Poland, PWN 2003 (in Polish). [2] A.Samii, J.Nutt, B.Ransom, Parkinson's disease, The Lancet, vol. 9423, 2004, pp [3] J. Jankovic, Parkinson's disease: clinical features and diagnosis. J. Neurol. Neurosurg. Psychiatr, 2008, vol. 79, no. 4, pp [4] B. Christe, P. Burkhard, A. Pegna, E. Mayer, and C. Hauert, Clinical assessment of motor function: a processes oriented instrument based on a speed-accuracy trade-off paradigm, Behavioural Neurology, vol. 18, pp [5] R. Okuno, M. Yokoe, K. Akazawa, K. Abe and S. Sakoda, Finger Taps Movement Acceleration Measurement System for Quantitative Diagnosis of Parkinson's disease, Proc. 2 8th IEEE EMBS Annual International Conference, pp [6] K. Shima, T. Tsuji, E. Kan, A. Kandori, M. Yokoe, S. Sakoda, Measurement and Evaluation of Finger Tapping Movements Using Magnetic Sensors, Proc. 30 th Annual International IEEE EMBS Conference, 2008, pp [7] K. Shima, Y. Tamura, T. Tsuji, A. Kandori, M. Yokoe S. Sakoda, Estimation of Human Finger Tapping Forces Based on a Fingerpad- Stiffness Model, 31 st Annual International Conference of the IEEE EMBS Minneapolis, Minnesota, USA, 2009, pp [8] A. Kupryjanow, B. Kunka, A. Czyżewski, Virtual Touchpad video based multimodal interface, 2 nd International Conference on Information Technology ICIT 2010, Gdańsk, June [9] Prati A., Mikić I., Trivedi M. M., Cucchiara R., Detecting moving shadows: formulation, algorithm and evaluation, [10] S. Fahn, R. L. Elton, UPDRS Development Committee members, Unified Parkinson s disease rating scale. In Resent Developments in Parkinson s disease, vol. 2, S. Fahn, C. D. Maesden, M.Goldstein, D. B. Calne, Ed. Florham Park, New Jersey: Macmillan Healthcare Information, 1987, pp [11] Movement Disorder Society, The Unified Parkinson s Disease Rating Scale (UPDRS): Status and Recommendations, vol. 18, No. 7, 2003, pp [12] P. Żwan, K. Kaszuba and B. Kostek, Monitoring Parkinson s disease patients employing biometric sensors and rule-based data processing, RSCTC 2010, Warsaw, 2010 (in print). [13] (2010) The Unified Parkinson s Disease Rating Scale. Available: RS.pdf 136