Kalman Filter Improvement for Gyroscopic Mouse Movement Smoothing

Kalman Filter Improvement for Gyroscopic Mouse Movement Smoothing Karel Slavicek, Vladimir Schindler, Martin Sykora, and Otto Dostal Abstract The Kalman filter is formally an algorithm used to produce estimation of a random variable based on measurement containing noise observed over time. This paper discusses Kalman filter capabilities to smooth noisy data obtained from electromechanical gyroscopes and accelerometers. Optimal design and slight modification of this algorithm is discussed. T Keywords Gyroscope, Accelerometer, Kalman Filter. I. INTRODUCTION AND MOTIVATION HE work is part of an international project Lifewear solved under Eureka cluster program ITEA2. One of the tasks of this project is to utilize devices wearable on a human body to control external equipment e.g., videoconferencing or presentation units. The main intended applications for such a model of remote devices control (and the main motivation for the project) is to offer good tool for surgeons to navigate through digital X-ray viewers during surgery [4] [6]. To achieve this task, we have designed a sensor unit containing electromechanical gyroscopes and accelerometers. This unit has reasonable small dimensions to be able to affix it on a human body like wristwatches and is used to study properties of inertial sensors. The electronic design is briefly introduced in section II. The data produced by the sensor unit contain two types of noise: noise induced by the electronic (both properties of the electro-mechanical nature of the electronic and noise introduced by digitization) and noise induced by human i.e., noise caused by shivering of human hand. The goal is to smooth the data produced by sensor unit movement to make gestures more distinguishable. A gesture usable for control of some external equipment consists of one or more simple movements of the sensor unit (e.g., movement from left to right or from bottom to up). Noise in the measured data can be easily understood as change in the movement and interpreted as another gesture. This problem will substantially decrease the ability to properly distinguish human gestures and can introduce rather many misinterpretations of the gesture. The dual nature of the noise source makes legacy Kalman filter not fully usable for noise suppression. The aim of this work is to offer modification of Kalman filter which can accommodate both types of signal noise. II. ELECTRONIC DESIGN A prototype sensor unit for gaining the human gestures was designed in terms of this project. The main components of this unit are gyroscopes and accelerometers. The unit processes the output of these basic sensors in an ATMEL processor and sends the difference of the sensor unit position via a Bluetooth interface. The protocol used for this purpose is the same as in computer mouse. A. Accelerometers Accelerometers with variable capacitance are used in this project. The operation of this sensor is based on variable capacitance of three-electrode air capacitor. The sensor utilizes the dependence of the electric capacity on distance of the capacitor electrodes. One out of the capacitor electrodes is movable and its movement depends on acceleration. The principle is easy to see from the figure 1. Karel Slavicek is with the Institute of Computer Science Masaryk University, Botanicka 68a, 60200 Brno, Czech Republic (corresponding author s phone: +420 549496831; e-mail: karel@ics.muni.cz). Vladimir Schindler is with the Institute of Computer Science Masaryk University, Botanicka 68a, 60200 Brno, Czech Republic and currently is a Ph.D. student at the Department of Telecommunications Faculty of Electrical Engineering and Communication Brno University of Technology, Technicka 12, 61600 Brno, Czech Republic (e-mail: xschin02@stud.feec.vutbr.cz). Martin Sykora is Ph.D. student at the Department of Telecommunications Faculty of Electrical Engineering and Communication Brno University of Technology, Technicka 12, 61600 Brno, Czech Republic (e-mail: msykora@gity.cz). Otto Dostal is with the Institute of Computer Science Masaryk University, Botanicka 68a, 60200 Brno, Czech Republic (e-mail: otto@ics.muni.cz). Fig. 1 The basic principle of used accelerometer The mechanical design of the accelerometer has to ensure linear and perceptive transformation of the acceleration into mechanical movement of the middle electrode. This http://dx.doi.org/10.15242/ IJCCIE.E1113079 92

electromechanical structure can measure acceleration in one direction. Adding of two more structures turned by 90o brings ability to measure acceleration in three orthogonal directions and we have a so called 3D accelerometer. An ADXL345 accelerometer from Analog Devices inc. is used in this project. ADXL345 is a small and thin 3D accelerometer with 13-bit resolution and measurement range ± 16 g. It is supplied in 3mm x 5mm 14-pin package. Digital output is formatted as 16-bit value and is accessible through either a SPI (3- or 4-wire) or I 2 C digital interface. B. Gyroscopes Electronic gyroscopes are usually based on utilization of the Coriolis effect. The principle of micro electro mechanical gyroscope is easy to be seen from figure 2. Fig. 3 The heart of sensor unit is based on Atmel ATMega328 Fig. 2 Simplified structure of MEMS gyroscope The gyroscope is based on mass structure of known weight, which is attached to a frame via set of springs. The mass structure is oscillating in direction perpendicular to the gyroscope rotation. The Coriolis effect causes compression of outer springs and movement of sense fingers which compose one electrode of a capacitor. The change in capacitance is proportional to the angle of the frame rotation. An LPR530AL gyroscope produced by STMicroelectronics group is used in this project. The LPR530AL is a small and tiny 2D (pitch and roll) gyroscope packaged It is supplied in a 5mm x 5mm LGA-16 package and offers two analog (voltage) outputs. C. The Overall Sensor Unit Design First attempts were made on modified off the shelf prototype board (see figure 4). Real measurement and research of human gestures usable to control external equipment we made on a newly designed sensor unit. The unit has shape of a computer mouse and behaves very similarly. The PCB of this unit is depicted on figure 5 and the overall sensor unit design is on figure 6. The heart of the sensor unit is Atmel ATMega328 CPU (see figure 4) surrounded by ADXL345 3D digital accelerometer, LPR530 and LY530 analog gyroscopes and three mouse-like buttons. The unit communicates with external equipment via Bluetooth. This unit allows to study human gestures suitable to control external equipment. This unit can work as a computer mouse. Unlike legacy computer mouse it has not to be related to a fixed surface (mousepad). This simple fact introduces a lot of uncommon properties. E.g., the user can turn the unit upside down. Gyroscopes provide information about orientation of the sensor unit and accelerometers detect the velocity and direction of the movement. The output from accelerometer is recalculated with respect to the present orientation of the unit by the internal ATMega328 CPU. Fig. 4 First attempts were made on modified off the shelf prototype board The output of the sensor unit is differential like the output of the legacy computer mouse and means the change of position in X and Y directions. The differential output mode seems to be the only option for sensor unit recognizing human gestures as in this mode it is easily adjustable. It is common to upscale the position offset in case the user moves the sensor unit (or mouse or his/her finger on touchpad) quickly and to downscale it if the user s movement is slow. III. KALMAN FILTER The data produced by the sensor unit contain two types of noise: noise induced by the electronic (both properties of the electro-mechanical nature of the electronic and noise introduced by digitization) and noise induced by human i.e., http://dx.doi.org/10.15242/ IJCCIE.E1113079 93

noise caused by shivering of human hand. The goal is to smooth the data produced by sensor unit movement to make gestures more distinguishable. A gesture usable for control of some external equipment consists of one or more simple movements of the sensor unit (e.g., movement from left to right or from bottom to up). Noise in the measured data can be easily understood as change in the movement and interpreted as another gesture. This problem will substantially decrease the ability to properly distinguish human gestures and can introduce rather many misinterpretations of the gesture. Fig. 5 PCB of sensor unit used for real measurement and research of human gestures The Kalman filter [1] - [3] is a very powerful mathematical tool commonly used to smooth noisy data. It is an algorithm that uses a series of measurements observed over time and produces estimates of unknown variables that tend to be more precise than those based just on the measurement. More formally, the Kalman filter operates recursively on the noisy input data stream to produce a statistically optimal estimate of the underlying system state. The filter is named after Rudolf E. Kálmán, one of the primary developers of its theory. measurement is expressed by Kalman gain. With a high gain, the filter places more weight on the measurements, with a low gain, the filter follows the model predictions more closely, smoothing out noise but decreasing the responsiveness. In the extreme cases, a gain of one causes the filter to ignore the state estimate entirely, while a gain of zero causes the measurements to be ignored. This process is repeated every time step. This means that the Kalman filter works recursively and requires only the last system s state estimation to calculate a new state. From a theoretical standpoint, the main assumption of the Kalman filter is that the underlying system is a linear dynamical system and that both noise introduced by the nature of the process studied and noise introduced by inaccuracy of the measurement have a Gaussian distribution. The Kalman filter obviously used to smooth the movement of computer mouse is very simple. The cursor position is given just by two coordinates (x,y) and the mouse provides us with the difference (u,v) in every time step. In the next step the cursor will be at position (x+u,y+v). Properties of the proposed improvement of Kalman filter can be easily demonstrated on example of a simple right to left and back movement of the discussed sensor unit. The discussed sensor unit can be used to control some external equipment by sensing of human hand movement in left-right and up-down direction. The front-to-back direction is not intended to be utilized because of natural human behavior. The front-to-back direction is much harder to use as it doesn t correspond to natural human gestures and form physiologic point of view this direction is also more difficult to achieve as it requests more concentration of the person. If the differential output of the sensor unit would be used just in this direct approach, then the cursor movement will look like the curves on the figures 7 and 8. Figure 7 shows movement in horizontal direction and figure 8 show movement in vertical direction. Even if the person intends to move the sensor unit in left-right direction only, some movement in updown direction usually also appears. Fig. 6 The overall sensor unit design The Kalman filter predicts a new state of observed based on the current state and the new measurement. Both the predicted and the measured states have different weights based on the estimated level of uncertainty. The uncertainty is represented by a covariance matrix. The resulted state estimation is a weighted average of the previous state and current observation or measurement. The new state estimate then lies between the previously predicted state and the new measurement. The Kalman filter tends to suppress rapid state changes a little bit and so this way smooth measured data. The ratio of weight of the prediction and weight of the Fig. 7 Original data from the sensor unit horizontal direction http://dx.doi.org/10.15242/ IJCCIE.E1113079 94

K T BPB R 1 T PB (6) The estimation of the covariance matrix Pˆ is done by Pˆ I K( B( P( (7) And the prediction of new value of the covariance matrix P is done by Fig. 8 Original data from the sensor unit vertical direction The Kalman filter used to smooth the computer mouse trajectory is the following: Lets x( be the real cursor position at the step (time) k. Let be the cursor position reported by the computer mouse. Let A and B be the matrixes expressing relation between the cursor position in current and next steps and relation between the mouse output and the cursor position respectively. Let v( and w( be the noise. Then P( k 1) A( Pˆ( A( T Q( (8) The Kalman algorithm the repeats these two steps, that means prediction of new values of the x( and update of the prediction induced by measurement. This straightforward application of the Kalman can smooth the cursor movement implied by the corresponding movement of the discussed sensor unit as shown on the figures 9 and 10. x A. x( k 1) w( y B. x( v( (1) (2) Where we assume that v( and w( are independent white noises with Gaussian distributions p( w) N(0, Q) p( v) N(0, R) Where Q and R are the covariance matrixes of noise of the cursor movement and noise of the measurement performed by the sensor unit respectively. The aim of the Kalman filter is to produce estimation x ˆ( of the cursor position x( such that the cursor movement is smooth, without unwanted jitter and fluctuations. The algorithm works in two steps. The first step is prediction (3) Fig. 9 Data from the sensor unit smoothed by legacy Kalman filter horizontal direction x( k 1) A.ˆ( x k ) (4) The second step is improvement of the prediction given by the last measurement. The improvement value z( given by xˆ x K( [ B. x( ] (5) Is used for this purpose. Let denote by P the covariance matrix of measured values and estimated values x ˆ(. The Kalman gain, that means the amount of influence of the measurement on the estimated value, is calculated as Fig. 10 Data from the sensor unit smoothed by legacy Kalman filter horizontal direction This level of smoothing is still not enough for proper recognition of human gestures. The main sources of problems are the points, where the curve relatively sharp turns its direction. These points make very difficult (if not completely impossible) to distinguish between direct movement and lancet http://dx.doi.org/10.15242/ IJCCIE.E1113079 95

movement of the sensor unit because both types of movement represent different gestures and probably will have different meaning in the process of external equipment control. For this reason, the goal was to modify the Kalman filter in such a way that these fake point of lance would be filtered out. IV. SMOOTHED KALMAN FILTER To achieve this better smoothing of the movement indicated by the movement of sensor unit we propose a slight modification of the Kalman algorithm described in this section. The basic idea is to average several consecutive measurements and unit position estimations. To achieve this task we propose to consider five consecutive values one more estimated value from cursor trajectory and the same number of consecutive values from the data provided by sensor unit. In the case of the discussed sensor unit, the matrixes A and B are identity matrixes. This fact simplifies all of the computations (4) (8): x ( k 1) 1/ 5 1/ 5 i5 i1 i4 i0 k i) k i) x K( [ x ] 1 K( P( P R( Pˆ( I K( ) P( P( k 1) Pˆ( Q( (12) x ( k 1) x K( [ x ] 1 K( P( P R( Pˆ( I K( ) P( P( k 1) Pˆ( Q( (9) Fig. 11 Data from the sensor unit smoothed by proposed smoothed Kalman filter vertical direction. The proposed smoothed Kalman filter uses prediction x ( k ) in the following form: ( k 1) 1/ 5 i 4 i0 k i) x (10) The contribution of measurement is replaced by an averaging one: i 5 i1 y 1/ 5 k i) (11) The whole smoothed Kalman filter can be summarized into the following set of equations (we still keep the simplified version valid for the case of cursor control made by the earlier discussed sensor unit or general computer mouse: It is easy to see that we have done only minor modification of the original well known Kalman filter. Currently, we are working on optimization of this smoothing principle in terms of the depth of the averaging factor and of course in terms of a weighted averaging where we can make more emphasize on the recent values and use the historical ones only as a correction factor. Also we are searching for some objective measure of smoothness of the estimated values. Fig. 12 Data from the sensor unit smoothed by proposed smoothed Kalman filter horizontal direction This modification of the Kalman filtering algorithm suppresses trembling of the cursor movement yet more than the legacy one. The trajectory of the same sensor unit movement smoothed by this modified Kalman filter is demonstrated on the figures 10 and 11. The smoothed Kalman filter as proposed is section IV can provide slightly better result comparing to the original version as it is easy to learn by comparing the figures 9 and 11. V. CONCLUSION The Kalman filter proved to be a viable and flexible framework. It can be easily modified to accommodate special conditions like additional sources of the noise. At the same http://dx.doi.org/10.15242/ IJCCIE.E1113079 96

time, the sensor technology has matured and now is ready to be utilized in a new type of applications namely it can completely rearrange current approach of control of many commonly used equipment e.g., videoconferencing and presentation devices. ACKNOWLEDGMENT This work was supported by Eureka project LF13020 Lifewear and project LW13006 BaaS solved under cluster ITEA2. REFERENCES [1] M. S. Grewal, A. P. Andrews, Kalman Filtering, Prentice-Hall 1993, ISBN 0-13-211335-X [2] M. I. Robeiro, Kalma and Extended Kalma Filters, Lisboa 2004, available at http://users.isr.ist.utl.pt. [3] G. Welch, G. Bishop, AnIntroduction to Kalman Filter, UNC Chapel Hill 2006, available at http://www.cs.unc.edu. [4] M. Javornik, O. Dostal, K. Slavicek, "Regional Medical Imaging System". World Academy of Science, Engineering and Technology, Francie. ISSN 2010-376X, 2011, vol. 7, no. 79, s. 389-393.. [5] M. Javornik, O. Dostal, K. Slavicek, M. Drdla, "MEDIMED - Regional Centre for Medicine Image Data Processing". In Knowledge Discovery and Data Mining. 2010. USA : IEEE Computer Society, 2010. ISBN 978-0-7695-3923-2, s. 310-313. 2010, Phuket, Thailand. [6] M. Javornik, O. Dostal, K. Slavicek, M. Petrenko, "MEDIMED- Regional Centre for Archiving and Interhospital Exchange of Medicine Multimedia Data", Proceedings of the Second IASTED International Conference on Communications, Internet, and Information Technology. Scottsdale, Arizona, USA : International Association of Science and Technology for Development- IASTED, 2003. ISBN 0-88986-398-9, s. 609-614. 2003, Scottsdale Arizona USA. http://dx.doi.org/10.15242/ IJCCIE.E1113079 97