DELIVERABLE REPORT 1 DESCRIPTION OF THE TASK 2 DESCRIPTION OF DELIVERABLE 3 IMPLEMENTATION OF WORK. Project acronym: INPUT Project number:

Transcription

1 DELIVERABLE REPORT Project acronym: INPUT Project number: Deliverable D8.1, Computer-based rehabilitation game Dissemination type: R Dissemination level: PU Planned delivery date: Actual delivery date: Reporting Period: 1 WP8, Task 8.1, Developing computer-based rehabilitation game, UMCG 1 DESCRIPTION OF THE TASK The game to be developed in this work package uses an avatar that is controlled by the same algorithms that control a prosthetic hand in a simultaneous and proportional way. The avatar will have as many DOF controllable as the prosthesis, although the avatar movements do not necessarily have to be the same as those of a prosthesis (e.g., a model airplane accelerating/decelerating, turning right and left and going up and down could reflect hand open/close, wrist pronation/supination and wrist flexion/extension of a prosthesis). The avatar can move in all directions, because movement directions can be activated simultaneously. The proportional aspect of the control allows changing the movement speed of the avatar. Therefore, in such a way, the control of the prosthesis could be trained in a fun and motivating way. In this task, the game will be developed and implemented on a computer. The game will be optimized using behavioural experiments in an iterative process. During the iterative process decisions have to be made about with which game implementations to proceed. The decisions about the game that are taken further will be based on a test that contains a certain level of the game that has to be performed by people who have trained with the game and a control group. In this task the game will be developed with able-bodied users controlling the game with their myosignals. 2 DESCRIPTION OF DELIVERABLE A computer-based rehabilitation game in which the avatar is controlled with algorithms used in controlling a prosthetic hand simultaneously and proportionally. 3 IMPLEMENTATION OF WORK Two computer-based rehabilitation games have been implemented which are controlled using the electromyogram (EMG) recorded from the arm of the user in the same fashion as a pattern-recognition controlled prosthetic hand using 4 and 8 electrodes respectively. The first game, named Myo-replicator, is embedded in the system training procedure, which is the procedure in which the pattern-recognition based control algorithms are adapted to the user. The correct execution of this procedure is critical, since the control capacity of the entire system is :42:00 1/5

2 established based on its results. Usually when this procedure is carried out, the user receives no direct feedback on his/her performance. This means that the user does not know if he/she is performing it correctly until after the procedure is done, when the accuracy of the pattern-recognition classifier is calculated. The concept of Myo-replicator was to provide direct feedback by turning the system training procedure into a game. In Myo-replicator the user controlled 4 coloured dots and the goal was to make the dots converge to a circle in the middle of the screen. See figure 1. Figure 1. Two instances of the game interface for Myo-replicator which shows the movement the user has to replicate (left hand side, Open Hand ) and the game view in the middle. A high-score (upper left corner of the game view) encourages the players to perform the task at the best of their abilities. In the top game interface, the player is succeeding in the game, since 2 of the dots are in the circle (the blue dot covers the green dot) and the red and yellow dots are moving towards the middle. In the bottom game interface, the player is failing, since the dots are far away from the goal circle :42:00 2/5

3 A reference movement was recorded from the user right before the game session. After the reference movement was recorded, the user should try to replicate the reference movement as to succeed in Myo-replicator. When successfully replicating the reference movement the dots converge to the targetcircle in the middle. If the user initially cannot match the reference movement the dots will disperse from the middle so the user knows that they are not performing the movement in the correct manner. This gives the user a chance to adapt their movement as to match the reference movement. The user will know that they have adapted in the correct way when the dots start converging on the circle. For example; if the user cannot get the red dot to converge to the middle, he/she can adapt the contraction of the muscle that corresponds to the red dot (EMG electrodes were colour coded in accordance with the dots) and be successful in replicating the reference movement. Myo-replicator has a high-score which is calculated based on the amount of time the dots were in the circle. In the second game, named Myobox, the user controls a ball (avatar) and the goal is to collect boxes. This game is also controlled using the EMG of the user, but instead of matching a reference movement, each direction of the ball is mapped to an electrode. For example; the EMG from electrode 1 (placed on the volar side of the forearm) will move the avatar upwards, electrode 2 (placed slightly more ulnar than electrode 1) will move the avatar up and to the right, electrode 3 (placed on the ulnar side) will move the avatar to the right etc. The direction of the ball will be calculated based on the total activation of all electrodes. If there is equal EMG activation on the electrodes corresponding to the opposite directions such as up and down or left and right, the avatar will not move. If there is activation on the electrodes corresponding to directions up and left, the avatar will move upwards and to the left at the same time. The purpose of this game is to train users to improve their myocontrol e.g. their control of the EMG. The idea is that after playing the game for a sufficient amount of time, the user will have learned to control the ball in 8 different directions by producing 8 unique EMG patterns robustly. These EMG patterns can then be used to control the prosthesis. See figure 2. Figure 2. The game interface for Myobox. The player avatar is the orange ball on the left hand side of the interface and the goal of the game is to collect the orange boxes (middle and right hand part of the interface). 4 RESULTS :42:00 3/5

4 Myo-replicator was used in an experiment in which it was compared with the conventional system training. The experimental group playing Myo-replicator and the conventional group achieved comparable results. See the poster depicted below in Figure 3 for detailed results and discussions thereof. The poster was presented at the TIPS conference in Glasgow, September See figure 3 below. Figure 3. Poster describing the experiment performed for evaluating the Myo-replicator game :42:00 4/5

5 Several insights were gained by performing the experiment using Myo-replicator. The use of a reference movement as a basis for feedback was problematic since it was difficult to evaluate if a certain reference movement was good (in terms of being distinct from other movements and easily replicable) before all movements had been performed. Also, implementing the game as a part of the system training procedure was problematic since the system training was not long enough for the user to learn the mapping between the EMG and the game control. The conclusion that was drawn from this experiment was that it should be investigated whether a game employed before system training with a more intuitive, or fixed, mapping between the EMG and game control could improve training. Based on these insights it was decided that no more work should be put in Myo-replicator and a new game should be developed. This game is the Myobox game. Myobox will be used in an experiment in the first half of The experiment will have 4 experimental groups performing different kinds of training for controlling a pattern-recognition based prosthetic arm. Group one will train by playing Myobox, group two will train using the conventional training method, group three will receive the same training as group two but with extended feedback and group four will serve as the sham training group that will perform a non-related EMG control task. Each group will receive training for one hour per day for five consecutive days. To summarize; two computer-based rehabilitation games in which the avatars are controlled using the same control signal and algorithms as prosthetic hands simultaneously and proportionally have been developed. After experimental testing it was revealed that the first game, Myo-replicator, did not provide suitable feedback for the users. Based on the findings of the experiment with the Myoreplicator another game was developed that did not have the limitations of the Myo-replicator game. This Myobox game will be used in the remainder of the project. The level of learning that can be established with this game will be examined in an experiment in the near-future. 5 SUBCONTRACTING All of the work was done within the UMCG by Morten Bak Kristoffersen with help from his supervisors. No subcontracting occurred :42:00 5/5