INVESTIGATING PERCEIVED OWNERSHIP IN RUBBER AND THIRD HAND ILLUSIONS USING AUGMENTED REFLECTION TECHNOLOGY. Lavell Müller

Transcription

1 INVESTIGATING PERCEIVED OWNERSHIP IN RUBBER AND THIRD HAND ILLUSIONS USING AUGMENTED REFLECTION TECHNOLOGY Lavell Müller A dissertation submitted for the degree of Master of Sciences At the University of Otago, Dunedin, New Zealand 15 July 2013

2 Abstract Neuroplasticity can be explained as a change in the brains wiring which is due to the changes in behaviour, environment and neural processes, as well as changes following bodily injury (Pascual-Leone et al., 2011). In the Rubber Hand Illusion a participant is shown a rubber hand being stroked while their real hand is hidden and stroked behind a screen. The participant then perceives the rubber hand to belong to their body in place of their real hand. This was an early example of Neuroplasticity demonstrated by Botvinick and Cohen (1998). The Rubber Hand Illusion was investigated in the first experiment of this research, along with a video mediated version of this experiment. The data were examined to investigate perceived ownership toward the rubber hand in both experiments. The results suggest that in both experiments, the RHI and the video mediated Rubber Hand Illusion; there was a sense of perceived ownership toward the rubber hand. The second part of this research investigated a study that suggests it is not necessary for the hand of the participant to be hidden during the Rubber Hand Illusion. The Third Hand Illusion suggest that a participant could be convinced into perceived ownership of a supernumerary limb. The Third Hand Illusion was the second experiment of this research, performed with two conditions using Augmented Reflection Technology, which is a system developed to investigate Neuroplasticity. In the first condition (athi) the participants were shown their hands alongside a pre-recorded rubber hand on the screen. The second condition (mthi) showed the participant their real right hand, a mirror image of their right hand to represent their left hand and a pre-recorded rubber hand. The two conditions investigated whether there was a sense of perceived ownership during the experiment and compared to see in which condition the sensation was stronger. The results suggest perceived ownership in the mthi condition, with no sense of perceived ownership in the athi condition. The study showed that there was still a sense of perceived ownership towards the rubber hand in both experiments performed through the ART system. i

3 Acknowledgments Firstly, I thank God for giving me the strength and patience to complete this research and for giving me the ability to push through all the struggles faced throughout my studies. I could never have done this without the faith I have in Him. I am thankful to Assoc. Prof. Holger Regenbrecht for his support, wisdom, guidance, and advice throughout this research. Without his support it would have been impossible to complete this Thesis. Thanks to Simon Hoermann for tolerating my niggling questions and his valuable help and advice during this study. Thanks to Mark McDowall at the Dunedin Limb Centre in the Dunedin Hospital who sponsored the prosthetic limbs used in this research. I would like to thank all of the people who participated in both experiments carried out for this research, their time was really appreciated. Also, I want to thank my family and friends for their continued support and motivation while completing my studies. Special thanks go to my father, mother and sister for their support and love throughout my journey as a student. Last, but not least, I would like to thank the Information Science Department and all members of our Human-Computer Interaction research group, who in our weekly meetings, contributed with their thoughts and inspirations to this research. Especially in alphabetical order, Abdulaziz Alshaer, Cameron Teoh, Constantin Brosda, Jack Wu, Johnny Collins, Julian Münster, Mariusz Nowostawski, Max Allen, Mohammed Alghamdi, Mustafa Alsaeed, Reece Arnott, Tobias Langlotz and Tomek Kolasa. ii

4 Table of Contents Abstract... i Acknowledgments... ii Table of Contents... iii List of Tables... vi Table of Figures... vii Definition of terms... x Abbreviations... x 1. Introduction The problem and sub problems Scope and Limitations Importance of Study Structure Review of Related Literature Introduction Neuroplasticity Referred Sensation The Rubber Hand Illusion The Third Hand Illusion Summary of Important Findings Hypotheses Rubber Hand Illusion: Vision and Video Introduction iii

5 3.2 Research Variables Independent Variables Dependent Variables Confounding Variables System Setup Hardware Software Research Method Experiment Design Participants and Task Questionnaires Environment Experiment Procedure Data Collection Assumptions Results Normal Distribution Mean Comparison Perceived Ownership Discussion Third Hand Illusion: Video and ART Introduction Research Variables Independent Variables Dependent Variables System Setup iv

6 4.3.1 Hardware Software Research Method Experiment Design Participants and Task Questionnaires Experiment Procedure Data Collection Assumptions Results Normal Distribution Mean Comparison Perceived Ownership Discussion Conclusion and Future Work Conclusion Future Work References Appendix A Appendix B Appendix C v

7 List of Tables Table 3.1 Test of Normality: Rubber Hand Illusion (RHI and vrhi) Table 3.2 Mauchly's Test of Sphericity Table 3.3 Tests of Within-Subjects Effects Table 3.4 Pairwise Comparison Table 3.5 Statements for RHI Condition Table 3.6 RHI One Sample Statistic Table 3.7 RHI One-Sample Test Table 3.8 Statements for vrhi Condition Table 3.9 vrhi One Sample Statistics Table 3.10 vrhi One Sample Test Table 4.1 of Normality: Third Hand Illusion (athi and mthi) Table 4.2 Mauchly's Test of Sphericity Table 4.3 Tests of Within-Subjects Effects Table 4.4 Pairwise Comparison Table 4.5 Statements for athi Condition Table 4.6 athi One Sample Statistics Table 4.7 athi One Sample Test Table 4.8 Statements for mthi Condition Table 4.9 mthi One Sample Statistics Table 4.10 mthi One Sample Test vi

8 Table of Figures Figure 1.1The Optical Mirror Box (Ramachandran & Hirstein, 1998)... 2 Figure 1.2 An example of the Rubber Hand Illusion (Botvinick, 2004)... 3 Figure 1.3 The Augmented Mirror Box... 4 Figure 2.1 A demonstration of the three conditions used (Takasugi et al., 2011)... 9 Figure 2.2 An example of the LED attached to the rubber hand (Rorden et al., 1999) Figure 2.3The three conditions used in the research of IJsselsteijn et al. (2006) Figure 2.4 Virtual arm being stabbed (Hägni et al., 2008) Figure 2.5 Synchronous stimulation of the real and virtual hand (Slater et al., 2008) Figure 2.6 The virtual hand and the virtual cursor (Yuan & Steed, 2010) Figure 2.7 Two rubber hands representing the real hand (Ehrsson, 2009) Figure 2.8 The MIRAGE System Setup (Newport et al., 2010) Figure 2.9 Synchronous touch and how the touch is perceived (Davies & White, 2011) Figure 2.10 A knife threat made to the rubber hand (Guterstam et al., 2011) Figure 2.11 Control conditions used in Experiment 1, 3 and 4 (Guterstam et al., 2011) Figure 3.1 The hardware used for the Rubber Hand Illusion Figure 3.2 The Video Mediated RHI Figure 3.3 A. Vision (Video Mediated) B. Vision (Direct) Figure 3.4 Mean Distribution for RHI Condition Figure 3.5 Mean Distribution for vrhi Condition Figure 3.6 Mean and Std. Error Mean In RHI Condition Figure 3.7 Mean and Std. Error Mean In vrhi Condition Figure 4.1 Setup for THI (without third box) Figure 4.2 Setup for THI (with third screen) Figure 4.3 The athi Condition Setup Figure 4.4 The mthi Condition Setup Figure 4.5 The three planes shown alongside each other Figure 4.6 Left: Game View. Right: The light source in Scene View Figure 4.7 The application controls shown at start-up Figure 4.8 An example of the Experiment Setup Figure 4.9 Mean Distribution for athi Condition Figure 4.10 Mean Distribution for mthi Condition vii

9 Figure 4.11 Mean and Std. Error Mean in athi Condition Figure 4.12 Mean and Std. Error Mean In mthi Condition viii

10 Ars Celare Verum... ix

11 Definition of terms Complex Regional Pain Syndrome: A condition causing intense pain in an affected body part; caused by minor or no apparent injury. Mirror Box Therapy: Therapy developed to treat patients with limb impairments such as PLP, CRPS or movement disorders after stroke Regenbrecht, McGregor, et al. (2011). Neuroplasticity: This can be defined as the brains ability to change itself due to change in behaviour, environment or by physical stimulation (Doidge, 2010). Phantom Limb Pain: A condition whereby as subject suffers from pain in a limb which no longer exists, that has either been amputated or lost in accident Rubber Hand: A limb prosthesis used as replacement for a person missing a hand Abbreviations AMB: ART: athi: CRPS MRI: mthi: MVF: OMB: PLP RHI: RS: SCR: SPL: THI: vrhi: Augmented Mirror Box Augmented Reflection Technology ART mediated Third Hand Illusion Complex Regional Pain Syndrome Magnetic Resonance Imaging ART mediated Mirrored Third Hand Illusion Mirror Visual Feedback Optical Mirror Box Phantom Limb Pain Rubber Hand Illusion Referred Sensation Skin Conductance Response Supernumerary Phantom Limb Third Hand Illusion Video Mediated Rubber Hand Illusion x

12 1. Introduction A previous misunderstanding taught to generations of medical students in the field of neurology was that the perception of one s own body was hardwired and inflexible (Ramachandran & Altschuler, 2009). Neuroplasticity can be defined as a change in the neural pathways which were due to the changes in behaviour, environment and neural processes, as well as changes resulting from bodily injury (Pascual-Leone, Amedi, Fregni, & Merabet, 2005). According to H. Regenbrecht (personal communication, June 21, 2012), this can be described as a rewiring of the brain, changing the way the brain perceives the body; whether over a long period of time (long term Neuroplasticity) or short period (immediate Neuroplasticity). Ramachandran suggested that change is possible and evident as he demonstrates when treating conditions such as Phantom Limb Pain (PLP) or Complex Regional Pain Syndrome (CRPS). PLP is a condition whereby as subject suffers from pain in a limb which no longer exists, that has either been amputated medically or in an accident. Complex Regional Pain Syndrome (CRPS) is described as intense pain in an affected body part; caused by minor or no apparent injury. Based on these principles of Neuroplasticity, Ramachandran introduced Mirror Visual Feedback (MVF) and the Optical Mirror Box (OMB, Figure 1.1); to treat patients with disorders such as Phantom Limb Pain (Ramachandran, Rogers-Ramachandran, & Cobb, 1995). He suggests that the brain can be rewired into believing that the missing limb was present with help of the OMB, aiding in the treatment of this chronic pain for this now visible phantom limb. 1

13 Figure 1.1The Optical Mirror Box (Ramachandran & Hirstein, 1998) The first Rubber Hand Illusion (RHI) was performed by Botvinick and Cohen (1998) to see whether a person s perceptions could be altered, convincing them to take ownership of a rubber hand, if their real hand was not visible. This was done by hiding the real left hand of a participant and making only the rubber left hand visible to them as shown in Figure 1.2. They would then stroke both the rubber hand and the real hand synchronously with a brush for a period of time. At the end, the participant was asked some questions about the perceived ownership of the rubber hand. The research found that the participant took ownership of the rubber hand while the real hand was not visible (Botvinick & Cohen, 1998). Following this, the Third Hand Illusion (THI) was carried out by Guterstam, Petkova, and Ehrsson (2011) to investigate whether a person would take ownership of a rubber hand alongside their existing two hands. This was investigated by performing the original rubber hand illusion, but not hiding the real hand during the experiment. They stroked both the real hand and rubber hand in plain sight of the participant during the experiment. Their results showed that there was a sense of perceived ownership towards the rubber hand, even though the real hand was shown in plain sight of the participant alongside the other two real hands. 2

14 Figure 1.2 An example of the Rubber Hand Illusion (Botvinick, 2004) What if a person with Phantom Limb Pain (PLP) is given a new limb prosthesis that they need to take ownership of? Or a person with limb paralysis is given a new limb to use alongside their existing limb? Can we alter their perception of their own body, convincing their brain into ownership of this new limb? In 2011, the Augmented Reflection Technology (ART) system (Figure 1.3); was developed by Regenbrecht Regenbrecht, McGregor, et al. (2011) to investigate Neuroplasticity. This System consisted of an Augmented Mirror Box (AMB), which is a computer mediated variation of the OMB introduced by Ramachandran and the ART Software. This system allowed for therapy and treatment of patients with PLP, CRPS and movement disorders after stroke (Regenbrecht, McGregor, et al., 2011). Different from the OMB developed by Ramachandran which allows the patient to view the mirrored limb directly in front of them, the AMB is computer mediated, allowing for a number of different manipulations and image augmentations to be performed, that cannot be accomplished by the OMB (Regenbrecht, Franz, McGregor, Dixon, & Hoermann, 2011a). The user places their hands in the AMB and observes their hands on a screen in front of them at a 90 degree angle and separated from the actual location; this is known as decoupling, shown in Figure

15 Figure 1.3 The Augmented Mirror Box The aim of the research described in this thesis was to investigate whether a sense of perceived ownership still occurs in the Third Hand Illusion (THI) when using the ART system. There were two experiments carried out in this research, the first experiment investigated perceived ownership in the original Rubber Hand Illusion (RHI) and the Video Mediated Rubber Hand Illusion (vrhi). The second experiment investigated perceived ownership in the ART mediated Third Hand Illusion (athi) and the ART mediated Mirrored Third Hand Illusion (mthi). These were both explained in further detail in Chapter 3 and Chapter 4, respectively. 1.1 The problem and sub problems Botvinick and Cohen (1998) suggested that a person could be convinced into taking ownership of a rubber hand, provided that the real hand was hidden and the rubber hand was placed in the correct position and location that the real hand would normally be located. Working off these findings, Guterstam et al. (2011) further claimed that it was even possible to take ownership of a rubber hand as a third hand, as long as it was in the correct position and location relevant to the body and real hands. They stated that if the rubber hand was placed parallel to the real hands at the correct distance and orientation during the experiment, the 4

16 participant would take ownership. The research reported in this thesis, investigated whether there is still a sense of perceived ownership even if the conditions were video and ART mediated. Secondly, this research addresses the assumptions made by previous researchers questioning whether position, location and orientation were necessary or relevant in order to achieve this sense of perceived ownership. 1.2 Scope and Limitations The research was carried out using the ART system, which could present a number of opportunities such as investigating therapeutic and scientific applications of the RHI and THI. The system allowed us to investigate the assumptions whether distance and position is relevant for perceived ownership and referred sensation. The participants of the study were taken from student volunteers in the University of Otago. These volunteers were English speaking and have good English writing skills. The research was carried out under certain lab conditions with ethical approval and consent for the participants. Decoupling the hands could become a limitation in the research if the previous assumptions about distance and position were accurate; or it could be that the position and angle of the screen cause discomfort to the participant having to look up during the experiment. Since both the RHI and THI were video mediated through a screen instead of directly visible to the subject as in the original experiments, this could become a limitation if the video quality is not satisfactory to display the rubber hand as real. 1.3 Importance of Study The importance of the study was to show that there was still perceived ownership towards the rubber hand in both experiments performed through the ART system, furthermore to replicate the Rubber Hand Illusion and Third Hand Illusion as closely as possible. This could allow new forms of rehabilitation therapies using the ART System to improve the quality of life for people with limb impairments. Therapy could involve treating patients with PLP by referring ownership to the rubber hand in place of the phantom limb. Another possible therapy for rehabilitation could be referring ownership to a third rubber hand of patients with CRPS. The findings of this research may encourage new scientific research in the field of Neuroscience. 5

17 1.4 Structure This thesis is structured in 5 Chapters. Chapter 2 discusses a review of the related literature. An introduction to Neuroplasticity and Referred Sensation is presented. This is followed by a review of the system setups used in the Rubber Hand Illusion and Third Hand Illusion. By examining the current systems, limitations of these studies will be identified; exposing a gap in the literature which will assist in refining the experimental design of this research. Chapter 3 and 4 describe the empirical studies of the RHI and THI performed in Experiment 1 and Experiment 2 of this research respectively. These chapters identify the research variables needed to test the hypotheses of this thesis. The system setup is described, followed by a detailed explanation of the research method. The results of the studies were then reported and discussed. Chapter 5 will give an outline of the main points of the thesis. The results were summarised into a conclusion and potential future works were presented. 2. Review of Related Literature 2.1 Introduction The research examined two studies, firstly the Rubber Hand Illusion (RHI) which investigated perceived ownership towards a rubber hand when one of the real hands was not visible during the experiment. The second study was the Third Hand Illusion (THI) which investigated perceived ownership towards a rubber hand when both of the real hands were visible during the experiment. The researcher reviewed the previous methods of repeating these studies and investigated new methods to duplicate the results and effects of the RHI and the THI. This chapter looks at the related literature and discusses past as well as recent work in this research area. A brief background and explanation of Neuroplasticity and Referred Sensation is given, followed by a review of the RHI and THI literature. This is followed by a discussion looking at the concept of perceived ownership in the two studies. The different methodologies of investigating perceived ownership were presented, as well as the different systems in which similar results of perceived ownership can be experienced. The chapter is concluded with a 6

18 summary of findings and a brief discussion of our research in relation to the literature reviewed. 2.2 Neuroplasticity Neuroplasticity is defined as the brains ability to change itself due to change in behaviour, environment or by physical stimulation (Doidge, 2010). This phenomenon is a growing area of research for neurologists and psychotherapists (Regenbrecht, McGregor, et al., 2011). After more than a century of the prevailing idea that connections in the brain were fixed during infancy, with no new connections being formed later in life, these notions have been changed (Ramachandran, 2005). One of the earliest examples of this rewiring of the brain or Neuroplasticity is demonstrated by Merzenich et al. (1984) where they removed a finger(s) from adult owl monkeys, and then they would define areas in the brain that represented each finger 2 to 3 months after surgical removal. In all the monkeys studied, the areas in the brain representing the fingers next to the ones removed expanded topographically, to occupy most or all of the areas previously representing the removed finger(s). Following the findings of Merzenich et al. (1984) it was found that this change in the brain also takes place in humans, with remarkable changes in the way one s own body is perceived (Blakesle & Ramachandran, 1998; Ramachandran, 2003; Ramachandran & Hirstein, 1998; Ramachandran & Rogers-Ramachandran, 1996; Ramachandran et al., 1995; Ramachandran, Rogers-Ramachandran, Stewart, & Pons, 1992). The researchers further demonstrate Neuroplasticity, by showing that it was possible to revive paralysed or inactive phantom arms. They did this by a simple method of using Mirror Visual Feedback (MVF) to deliver the illusion that the phantom arm was moving in reply to the brains command (Ramachandran et al., 1995). From these findings, Ramachandran (2005) later suggested that there is significant concealed plasticity in the adult brain that needs to be further investigated. 2.3 Referred Sensation One of first notable examples of referred sensation was demonstrated using MVF, by Ramachandran et al. (1995), which was briefly mentioned in the previous section. The researchers performed a rehabilitation experiment on arm amputees, asking them to place 7

19 their intact arm alongside a vertical mirror in front of them. The participants were positioned in a way so that it appeared as if the amputated arm was now present when looking into the mirror. The participants were then asked to perform certain movements while looking at the reflection in the mirror. Some of the patients who suffered from Phantom Limb Pain reported that there was a reduction in pain in the phantom arm; others reported that they felt a sensation of touch on the phantom when they saw or felt the healthy arm being touched (Ramachandran et al., 1995). Following this research, Ramachandran and Rogers-Ramachandran (1996) using MVF, further examined referred sensation or sensation felt on sites of skin not actually stimulated. The findings from this investigation suggested that effects of referred sensation were exclusive to phantom limbs and that the study was unable to elicit referred sensation in healthy limbs. This assumption was supported by the researchers previous findings, saying that referred sensation was reported in 4 out of 10 arm amputee patients (Ramachandran & Hirstein, 1998) and also that this referred sensation onto the phantom limb was triggered when stimulating the face of the arm amputee (Ramachandran et al., 1992). A number of further studies demonstrated referred sensation in patients with different symptoms, for example post stroke patients (Longo & Haggard, 2010) or patients with spinalcord injuries affecting their limbs (Moore et al., 2000). Referred sensation was also evident in patients with CRPS, as demonstrated by McCabe, Haigh, Halligan, and Blake (2003) who reported tactile RS in 5 out of 16 patients, while their eyes were shut. This sensation was not present when they had direct vision of the limb that was being stimulated. Similarly, when Krämer, Seddigh, Lorimer Moseley, and Birklein (2008) used a mirror to overlay the replicated image of the stimulated limb over the affected one, RS was not evoked in patients that did not suffer from CRPS. 8

20 Figure 2.1 A demonstration of the three conditions used (Takasugi et al., 2011) Contrary to these finding, Takasugi et al. (2011) evoked a sense of RS using MVF, on healthy participants in two separate within-subject experiments, whose setups were shown in Figure 2.1. In the first experiment with 21 participants, containing two conditions, in the control condition; the participant s one hand was hidden behind a mirror while they were shown their other hand being stimulated in the reflection. In the experiment condition, a reflection of the assistant s hand being stimulated was shown to the participant instead. For the second experiment, 16 participants were shown a reflection of a rubber hand instead of the assistant s hand. In both experiments, all participants experienced a stronger sense of RS in the experiment condition compared to the control condition. Furthermore, all but one participant experienced a sense of perceived ownership associated with the visual appearance; towards the hand in the mirror image. These finding lead us onto the following section in this chapter, which looks at the Rubber Hand Illusion and the sense of perceived ownership elicited during this experiment. 2.4 The Rubber Hand Illusion Following the early research by Ramachandran et al. (1995), Botvinick and Cohen (1998), created a Rubber Hand Illusion (RHI) whereby they performed an experiment to see if it was possible to persuade the human mind into taking ownership of a rubber hand, by hiding the participant s real left hand and stroking both the hidden hand and rubber hand with a brush. In 9

21 all 10 participants, a sense of perceived ownership was reported towards the rubber hand. They proved successful with this experiment and this phenomenon has been repeated a number of times hereafter in the same way as described in their paper. This phenomenon of ownership is sometimes attributed to the brains neural plasticity, which is described by Ramachandran (1998), as the brains ability to change its perception about one s own body. According to Ramachandran and Hirstein (1998) they claimed that it was even possible for a person to perceive a shoe as part of the human body if the hand was placed out of view from the participant while both the hand and shoe were stroked with a brush simultaneously. To be sure the participants were not just using a figure of speech, by saying they felt the sensation in the shoe; the researcher hit the shoe with a rubber hammer, unexpectedly. Bizarrely, they saw the participants flinch as well as register a strong increase in skin conductance when measuring their SCR. Similar results were achieved by Rorden, Heutink, Greenfield, and Robertson (1999) when they found that a patient was able to detect tapping on a rubber hand better on their affected real hand if an led light was attached to the rubber hand (Figure 2.2). The patient could only feel the stimulation on the hand when the tapping device was visible to him, which suggests that visual feedback can help recover tactile sensation. Figure 2.2 An example of the LED attached to the rubber hand (Rorden et al., 1999) Four years later Armel and Ramachandran (2003) reported that subjects took ownership of a table when their real hand was hidden and both the table and the real hand where tapped and stroked in synchrony. The researcher said that if the table was suddenly injured while stroking and tapping, the subjects displayed a strong skin conductance response. Later, Botvinick (2004) re-examined the rubber hand illusion and discussed the reason for this sense of perceived ownership and looked at whether it was just a visual observation and 10

22 response, or an actual feeling of ownership by the body. He said that it was due to the link between what people see, feel and proprioception, which is the ability of the brain to sense the position, location, orientation and movement of the one s own body and its parts. This proposed that there is a correlation between what we see and what we feel. As we see the rubber hand being stroked and feel it on our real hand, the brain links the feeling with the visual feedback and corrects the proprioception of the hand. Ensuing these findings Ehrsson, Holmes, and Passingham (2005), found that even in the absence of visual input, the sense of perceived ownership was still present with synchronous tactile and proprioceptive stimulation when the subject was blindfolded. Contrary to this, Tsakiris and Haggard (2005) suggest that the rubber hand illusion does not create these phenomenon of ownership or referred sensation due to stimulation or conditions of the experiment. They suggest that it is due to the way the body perceives itself and the way in which it adjusts perception of the body. In their experiments, the researchers perform experiments with rubber hands as well as other neutral objects in congruent and incongruent positions, to investigate whether it is the perception of one s own body that creates the phenomena or is it due to visual and tactile stimulation. The findings suggest that it is the congruent position of the object, rather than the visual or tactile stimulation which causes these phenomena to occur. Similarly, IJsselsteijn, de Kort, and Haans (2006) suggests that there is still a sense of perceived ownership towards a rubber hand, even it is video mediated in a virtual reality condition or mixed reality condition, as long as the rubber hand in these conditions is presented to the participant in the correct position and location as seen in Figure 2.3. In the virtual reality condition, the rubber hand and synchronous stimulation were both projected onto the table in front of the participant. In the mixed reality condition, the rubber hand is projected onto the table, with the projection of the rubber hand being physically stimulated directly in front of the participant. 11

23 Figure 2.3 The three conditions used in the research of IJsselsteijn et al. (2006) A. Unmediated original RHI. B. Projected hand and stimulation. C. Physical stimulation on projected hand The results of this research suggested a stronger sense of perceived ownership in the experiment conditions when compared to the results of the original Rubber Hand Illusion condition. Furthermore, in support of the two previous experiments, Costantini and Haggard (2007) investigated whether the Rubber Hand Illusion was affected if the position or the rubber hand or real hand was changed. Secondly, would the Rubber Hand Illusion be affected if the stimulation was altered on either the rubber hand or real hand? The researchers found that the sense of perceived ownership diminished as the degree of spatial disparity between visual and tactile feedback increased. Another variable to consider was a suggestion by Haans, IJsselsteijn, and de Kort (2008) which was: does shape and texture of the object affect the Rubber Hand Illusion or does it still produce the same results? They performed 2 experiments; the first was with a hand shaped object with natural skin texture compared to a hand shaped object with non-natural skin texture. The second experiment was with a non-hand shaped object with natural skin texture compared to non-hand shaped object with non-natural skin texture. 12

24 The results reported that there was a stronger RHI within the hand shaped objects compared to the non-hand shaped objects. However, in contrast, the findings suggest that natural skin texture on a hand-shaped object provided a stronger RHI than on non-hand shaped object. This meant that the sense of perceived ownership was closer related to the shape of the object, rather than the skin texture of that object. This brings into question the previous research mentioned by Ramachandran and Hirstein (1998), which suggest that it was even possible to take ownership of a shoe or desktop. If there was a sense of perceived ownership, how strong was this phenomenon and how strong would it have to be, to propose a sense of perceived ownership towards the said objects? Later, Hägni et al. (2008) asked participants to observe a virtual pair of arms and hands in two conditions. In condition one, the participants were just told to observe virtual arms intercepting virtual balls, in condition two the participants were asked to observe the virtual arms and imagine that they were their own arms while they intercepted the virtual balls. While the participants observed the virtual arms, the right arm was unexpectedly stabbed and began bleeding (Figure 2.4). The results showed an increased SCR in both conditions but suggested that the participants in condition two showed significantly higher skin conductance than the participants in condition one. They suggest that simple visual feedback paired with mental imagery may convince the brain to significantly integrate external objects into one s own body image. Does this mean that the sense of perceived ownership is only possible if the brain is primed before the Rubber Hand Illusion? Another question is whether the priming of the mind is necessary, or does it just amplify the effect of perceived ownership and referred sensation? 13

25 Figure 2.4 Virtual arm being stabbed (Hägni et al., 2008) Additional investigations into this virtual hand illusion was carried out by Raz, Weiss, and Reiner (2008) based on the original RHI by Botvinick and Cohen (1998) to investigate whether haptic feedback as well as passive or active movement of the virtual arm during conditions, would increase the effects of perceived ownership. Firstly, they found that there was a stronger sense of perceived ownership during synchronous conditions when both the real hand and the virtual hand moved precisely at the same time or were stimulated by the same stick. Secondly, the researchers found that there was a greater sense of perceived ownership when there was active mode with haptic feedback given to both the real and virtual hand in synchrony. Does this suggest that the virtual hand needs to behave like a real hand in order to be perceived as one s own body? Or does this movement just amplify the sensation of perceived ownership and RS? An additional investigation into this virtual hand illusion was later carried out by Slater, Perez-Marcos, Ehrsson, and Sanchez-Vives (2008). They performed the virtual hand illusion by stroking the patient s real right hand and an aligned 3D stereo virtual arm projected horizontally out of their shoulder in synchrony (Figure 2.5). Their results found that in the group of 21 male participants when the hand was stimulated in synchrony, there was a sense of perceived ownership. Contrasting the first group, when a group of 20 men participated in a control experiment, where the stimulation on the virtual arm and real arm was not in synchrony, there was no sense of perceived ownership. This would support the findings 14

26 earlier suggested by Raz et al. (2008), that the sense of perceived ownership is greater when the virtual arm and real arm were seen and felt to be happening in synchrony. Figure 2.5 Synchronous stimulation of the real and virtual hand (Slater et al., 2008) Yuan and Steed (2010) investigated whether this sense of perceived ownership was still present when immersing the user in a virtual reality environment. The user was asked to complete two tasks while seated at a real table, wearing a head mounted display, showing the virtual environment. The first task was to use a wand tracker to point and match picture locations on the wall in the virtual environment, with the location of pictures on the real wall in the room. In the second task, the user was asked to pick up a ball and place it into one of three holes, highlighted on the table they were sitting at. Once the hole is highlighted and the ball is placed on the virtual table, as the user proceeds to place the ball in the hole, the virtual desk lamp falls onto the virtual arm or a virtual cursor arrow representing the participants arm (Figure 2.6). The results found a significant higher SCR on the real arm when the lamp was dropped on top of it, compared to when the lamp was dropped on top of the cursor. This suggests that there was a sense of perceived ownership towards the virtual arm while being immersed in the virtual environment. This brings up the question of whether the virtual body has to resemble the human body or could this illusion be possible by just having any virtual object stimulated in synchrony with the real arm? 15

27 Figure 2.6 The virtual hand and the virtual cursor (Yuan & Steed, 2010) Bekrater-Bodmann, Foell, and Flor (2011) suggest that since chronic pain such as PLP and CRPS often go together with illusory perceptions of the affected limb, these changes in body perception also indicate underlying alterations of the body representation to one s self. They propose that by using simple methods, such as MVF, the body representation in patients with chronic pain, can be corrected in some cases (Bekrater-Bodmann et al., 2011). In addition to this, the researchers recommend that these simple methods could even bring a reduction of illusory sensations as well as a reduction in perceived pain level. This is supported by Hänsell, Lenggenhagerl, Känell, Curatolol, and Blankel (2011), which found that when they showed a participant a mannequin representing their body while stroking the back of the participant on the back, the participants pain acceptance level was significantly higher compared to when they showed the participant a white sheet of cardboard to represent their body. Does this suggest that pain could possibly be referred onto virtual representations of the body? Or would it even be possible to train patients with PLP or CRPS to have a higher pain tolerance or even no sensation of pain at all? Further investigations need to be made into these findings, to examine the level of referred sensation possible as well as how strong this sense of perceived ownership is towards these fake representations of body parts. 16

28 2.5 The Third Hand Illusion The previous literature discusses the rubber hand illusion and how it allows a person with two healthy hands to take ownership of a rubber or virtual hand. This is done by placing a person s real hand out of sight and showing them a rubber or virtual hand. The rubber or virtual hand is then stimulated in synchrony with the real hand and sometimes a threat is made to the rubber or virtual hand, which increases the sense of perceived ownership. Would it be possible then, for a person to take ownership of all three hands instead of just the two? Could the real hands and the rubber or virtual hand be part of one body? While the amount of related literature investigating perceived ownership of supernumerary limbs is not vast, the researcher will examine several studies which suggest that ownership of supernumerary limbs is possible. Halligan, Marshall, and Wade (1993) were pioneers in the field of research investigating supernumerary limbs. The researchers examined a patient who suffered from stroke, but after the stroke had a Supernumerary Phantom Limb (SPL) added to the body. This third hand was not perceived to anyone except the patient. The patient either identified the left arm or the right arm as the SPL when asked to identify it to the researcher. The patient was adamant about owning three arms, without being able to explain how owning three arms was even possible. This is one of the earliest examples of perceived ownership of a SPL, even without a physical third arm being present. The sense of ownership is so strong, that when the patient looked at his own limbs, he would reassign one of them to be the third arm, with full belief that it was. Sometimes the patient would suggest that the third arm has been amputated but still present. Attempting to further investigate the illusion of the SPL caused by stroke more than a decade later,, Khateb et al. (2009) used Magnetic Resonance Imaging (MRI) to investigate the multimodal nature of this SPL, which the patient claimed to be able to see, use, and move purposely. When the patient was asked to imagine scratching her left cheek with her left hand, her right pre-motor and motor regions were activated. When she performed the same task with the SPL, additional left middle areas located at the back of the brain were activated. Moreover, comparison of responses induced by left cheek versus right cheek scratching with the SPL demonstrated significant activation in areas which were concerned with bodily sensations. 17

29 Schaefer, Heinze, and Rotte (2009) investigated the THI and how strong the sense of perceived ownership was towards the third arm by examining the somatosensory cortex with neuro-magnetic source localization. They performed the THI by connecting an artificial hand and arm to the body, and giving subjects the visual impression that they had a supernumerary third arm, while stimulating the hand. The results showed that the participants not only regarded the artificial arm as their own, but felt to have three arms. This is how a simple illusion evoked feelings of ownership of a third arm (Schaefer et al., 2009). Ehrsson (2009) further investigated this THI by performing a modified version of the RHI. He placed two rubber right hands on a table in front of twenty participants, while their real right hand was hidden under the table. The two rubber hands were side by side, 10cm apart and 10cm above the real hand (Figure 2.7). A special double paintbrush and a regular paintbrush were used to synchronously brush the two rubber hands and the participant's hidden hand on the matching. After 2 minutes of brushing, most of the people described sensing the paintbrushes touching both rubber hands. People also often described how both rubber hands felt like their own right hand at the same time. These findings were supported by the results of the SCR, which showed a greater sense of perceived ownership towards the rubber hands when they were stroked in synchrony with the real hand. Figure 2.7 Two rubber hands representing the real hand (Ehrsson, 2009) 18

30 The findings by Newport, Pearce, and Preston (2010) not only suggest that it was possible to own a third hand, but they further demonstrate that the illusion also affects the subsequent reaching error. Participants were shown two moving virtual left hands representing their real hidden left hand on a mirror in front of them; reflected from a screen above the mirror (Figure 2.8). When only one was in synchrony during active touch, ownership was only claimed for that hand and the accuracy of reaching was in relation to control of the synchronous hand. When both virtual hands were in synchrony, ownership was claimed over both, but only one was controlled. This would suggest that fake limbs can be incorporated as part of the body, but the body schema can accommodate only one (Newport et al., 2010). Figure 2.8 The MIRAGE System Setup (Newport et al., 2010) Davies and White (2011) suggest that the sensation of owning a third limb is even possible without there even being visual feedback given to the participant. They perform an experiment where the participant is made to close their eyes and their hand is guided by the experimenter to stroke their own right cheek, while the experimenter would stroke the left cheek either synchronously or asynchronously as seen in Figure 2.9. The results found that 19 out of 21 participants reported the illusion of self-touch, and many of these participants (13 out of 19) reported that it appeared as if they were touching the right side of the face with the right hand and the left side of the face with a third hand (Davies & White, 2011). 19

31 Figure 2.9 Synchronous touch and how the touch is perceived (Davies & White, 2011) Guterstam et al. (2011) went a step further to investigate this Third Hand Illusion by performing a set of experiments based on the RHI to see whether perceived ownership and referred sensation was still possible, if the rubber hand was not hidden during the illusion. Would the participant take ownership of both real hands and the rubber hand if they were all visible at the same time? The study consisted of data collected from 154 participants across 5 different experiments. All experiments we carried out in a sound proof laboratory. The rubber hand was placed next to the participant s real right hand with a distance of 12.5cm between the index finger of the artificial limb and the index finger of the participant s right hand, the only exception being in experiment 5. The first experiment investigated the supernumerary hand illusion. The rubber right hand was parallel to the real right hand, with three conditions: synchronous brushing, asynchronous brushing and synchronous brushing of a rubber hand rotated 180 degrees. The second experiment explored physiological confirmation for the supernumerary hand illusion. The rubber right hand was parallel and corresponding to the real right hand, with two sessions divided into four one minute long periods of synchronous or asynchronous brushing. This was followed by a threat made with a knife towards the real hand or rubber hand (Figure 2.10), while the SCR was measured. 20

32 Figure 2.10 A knife threat made to the rubber hand (Guterstam et al., 2011) The third experiment investigated the type of limb that could be owned as a supernumerary limb. There were three 2 minute conditions with synchronous brushing of a rubber right hand, rubber left hand and rubber left foot which were parallel to the real right hand (Figure 2.11). The fourth experiment explored physiological confirmation for the type of limb that can be owned as a supernumerary limb. There were three sessions divided into four one minute long periods of synchronous brushing. The rubber right hand was parallel and congruent to the real right hand, a rubber hand rotated 180 degrees, a rubber left hand and rubber left foot which were parallel to the real right hand. This was followed by a threat made with a knife towards the real hand or rubber hand, while the Skin SCR was measured. Figure 2.11 Control conditions used in Experiment 1, 3 and 4 (Guterstam et al., 2011) Experiment five investigated the unique qualities of the supernumerary hand illusion. The original rubber hand illusion and third hand illusion compared to see whether the two experiments were qualitatively different from each other. The two conditions were synchronous brushing for two minutes of the real hand and rubber hand, with the real hand visible (Condition 1 RHI) or not visible (Condition 2 THI). 21

33 In Experiment 1 the results show that the illusion of owning a supernumerary limb requires synchronous visuo-tactile stimulation and a rubber hand orientated in parallel with the real hand. The participants reported a greater feeling of ownership in the synchronous condition compared to the asynchronous condition. The observation made in Experiment 2 was a significantly greater SCR the rubber hand was threatened in the synchronous condition than in the asynchronous condition. The results of Experiment 3 demonstrate that only worked when a rubber right hand was used in conjunction with the real right hand, as it was found that the illusion was significantly reduced when a rubber left hand or foot was used. There was a greater threat-evoked SCR in Experiment 4 for the illusion condition (when a rubber right hand was used) than in the control conditions using the rotated rubber hand, rubber left hand or a rubber right foot. The findings of Experiment 5 suggest a weaker sense of perceived ownership towards the rubber hand in the Third Hand Illusion than the Rubber Hand Illusion, but a weaker sense of disownership towards the right hand and a significantly stronger feeling of owning two right hands in the Third Hand Illusion compared to the Rubber Hand Illusion. 2.6 Summary of Important Findings These findings have important implications for investigating the ART system used in our research. The findings suggest that it might not be possible to carry out any of the experiments proposed, since the rubber hand used would not be congruent to that of the real hands; meaning the position, location and orientation would not allow for perceived ownership or RS to occur. The purpose of this thesis was to: Demonstrate perceived ownership in the decoupled, video mediated RHI (vrhi) using the ART system. Demonstrate perceived ownership in the decoupled, ART mediated THI (arhi) using the ART system. 22

34 2.7 Hypotheses The review of the related literature provides support for the proposed research as well as support for the methods of testing perceived ownership. These methods for examining perceived ownership can be used to investigate the hypotheses which claim a sense of perceived ownership could still be experienced while using a video mediated display device. Hypothesis 1: The ART system can create a sense of perceived ownership during the decoupled, video mediated Rubber Hand Illusion. Hypothesis 2: The ART system can create a sense of perceived ownership during the decoupled, ART mediated Third Hand Illusion. 23

35 3. Rubber Hand Illusion: Vision and Video 3.1 Introduction In the previous chapter, the related work was discussed. A number of systems to investigate perceived ownership were examined; in which similar results of perceived ownership were experienced. In this study, a within subject design was used to investigate perceived ownership of a rubber hand in the original (RHI) and video mediated Rubber Hand Illusion (vrhi). The findings reported in this chapter suggest that there was a sense of perceived ownership experienced by users in both the RHI and vrhi. This chapter describes the dependent and independent variables for this study, followed by a detailed description of the system setup with regards to the hardware and software used. The research method is then discussed, giving a description of the experiment design, measurement tools and the procedure that were used to investigate perceived ownership. Thereafter an explanation will be given about the data collection process as well as the type of data collected in this study. The results of this study were then reported and assumptions for these findings will be stated and briefly discussed. 3.2 Research Variables The independent and dependent variables were defined. The confounding variables in this study were then identified Independent Variables The independent variable in this study was vision. There were two display methods used and they were defined as vision with two states: Vision (Direct). This was when the subject did not use any digital display device; they would just have a direct field of vision as in the original Rubber Hand Illusion. 24

36 Vision (Video Mediated). This was when the Rubber Hand Illusion was performed through the ART System to alter the subject s visual perceptions. Their perception was altered by changing the angle and position of the subject s field of vision. The subject was made to place their hands in the ART System and view them on a screen above the system Dependent Variables The dependent variable in this study was perceived ownership. This is defined as follows: Perceived Ownership: This is the sense of ownership a person feels toward a rubber hand, after feeling their own hidden hand being synchronously touched in the corresponding location of the rubber hand Confounding Variables A number of confounding variables have been identified and possible solutions were proposed to control these variables: Decoupling Effect: A number of studies suggested that in order for the illusion of perceived ownership to work, the rubber hand needs to be congruent (Tsakiris & Haggard, 2005) or parallel (Guterstam et al., 2011) to the real hand. The decoupling effect is when the representations of the hands were decoupled from their physical location. This research investigated whether the sense of perceived ownership was still present during decoupling. Order of Conditions: Since this research used a within-subject design, there could have been a learning effect during the experiment which could have affected the dependent variable of the study. The order of conditions was therefore randomised and balanced. This meant that the direct vision condition and the video mediated vision condition were performed in the first session an equal number of times. 25

37 Rubber Hand Realism: Studies suggest that the appearance of the rubber hand or object representing the real hand is important. Haans et al. (2008) suggested that the strongest sense of perceived ownership is achieved by using a hand shaped object that has a natural skin colour and texture. Contrary to this, Ramachandran and Hirstein (1998) suggest that it is possible to take ownership of a shoe in place of the hidden hand as part of one s own body. They later suggested that it is even possible to take ownership of a table as part of one s own body (Armel & Ramachandran, 2003). To avoid disownership, four rubber prosthetic hands were obtained from Mark McDowall at the Dunedin Limb Centre in the Dunedin Hospital (Pictures of Prosthetic Hands, Appendix A). Participant Demographics: A number of demographic issues could become confounding variables in this study. Age, gender or familiarity with the study could become confounding variables. Male participants might not take ownership of a female prosthetic. Or an older person might not take ownership of a younger looking prosthetic. To solve this, two male and two female prosthetic hands were obtained. One male prosthetic resembled that of an older person, while the other male prosthetic resembled that of a younger person. This was the same for the female prosthetics. 3.3 System Setup The ART system which was used for this research consisted of both hardware and software components. The system was developed by Regenbrecht, Franz, McGregor, Dixon, and Hoermann (2011b) to test the hypotheses of Ramachandran et al. (1995) about limb presence and perception as well as theories about pain management. The system projects a real-time video of a person s healthy limb(s) through a computer and camera system. The augmented characteristic of the technology, which they referred to as Augmented Reflection Technology (ART), enables the user to alter perceptions and beliefs using mixed reality environments; hence enabling both users and researcher to reflect outside the box (Regenbrecht, Franz, et al., 2011a) Hardware The hardware described and developed by Regenbrecht, Franz, et al. (2011a) consisted of a standard desktop computer running on the Windows Operating System, connected to the ART system. The system consisted of two screens, one for the experimenter and the other for the participant. There were three black fibre board boxes (dimensions 37cm x 37cm x 37cm) 26

38 placed beside each other. The front openings of the boxes had a black curtain with a split at the bottom, so as to allow the participants to place their hands inside the boxes. For this research, the back openings of these boxes were left unsealed. The base board of these boxes were covered in black cloth to reduce the reflection of light within the boxes. Within two of the boxes there were two cameras, one in each box, connected via USB (v2.0) to the computer system. A pair of wide angle; Logitech Quickcam Pro 9000 cameras, with approximately 80 degrees of diagonal field-of-view, were used in the study. These cameras were mounted to the ceiling of the curtain side, facing downward into the boxes. The ceiling of the box is covered with 4 x 4 Light Emitting Diodes (LEDs) for consistent and appropriate lighting. The LED arrangement was determined by experimentation to optimize brightness, diffuse illumination and to provide sufficient shadowing effects for background subtraction and depth cuing purposes, this includes perceived texture depth of the hands (Regenbrecht, Franz, et al., 2011a). Figure 3.1 The hardware used for the Rubber Hand Illusion A wide screen monitor was placed on top of the two boxes for viewing by the participant, while the second standard monitor was placed alongside the boxes, only visible to the researcher and a standard keyboard and mouse to control the application (Regenbrecht, Franz, et al., 2011a). The system hardware can be seen in Figure Software For this experiment the software used to control the incoming video stream was the same as that described in the paper by Regenbrecht, Franz, et al. (2011a). The software allows for background subtraction from the incoming video stream. The content and environment could be controlled and manipulated from a Graphical User Interface (GUI). 27

39 3.4 Research Method Experiment Design This study used 22 participants in a within subject, repeated measure experimental design. There were two conditions in this experiment; one condition was the original Rubber Hand Illusion (RHI), while the other condition was the video mediated Rubber Hand Illusion (vrhi) using the ART System. The study consisted of two sessions, one for each of the conditions. Each of the participants performed both conditions of the experiment, but the order of the within subject conditions were randomised. The order of randomisation can be found in Appendix A. If the participant performed the vrhi condition for the first session using the ART System, they would not use the system in RHI condition in the second session; and vice versa. The reason for choosing a within subject, repeated measure design; was to compare the difference between the two conditions. This would also mitigate the learning effect. The order of conditions was balanced so that both conditions were in the first session an equal number of times during the 22 experiments of this study. This meant that the RHI condition was performed in the first session of 11 experiments and vrhi was performed in the first session of the other 11 experiments Participants and Task In this experiment there were a total of 23 volunteers used in this study, 13 males and 10 females. All of the volunteers used in this study were students from the University of Otago. The first participant of the study was used as a pilot study subject and was excluded from the final results of the experiment. The age of the 22 participants ranged from 18 to 28 years (M = 22, SD = 3.16). Of the 22 participants, 18 of them did not have prior knowledge about the experiment, while 3 of them had heard of the experiment before. Although when the 3 participants were asked about how much knowledge they had about the experiments, their knowledge was limited. All of the participants were recruited from different departments within the University of Otago. During each phase of the experiment, the participants were asked to place their hands within the boxes on the far left and far right with the rubber hand in the centre box, while watching their real right hand and the rubber left hand either in front of them or on the screen above the 28

40 boxes as seen in Figure 3.2. There were no hand movements and there was no physical task that needed to be performed Questionnaires Figure 3.2 The Video Mediated RHI For this research there were three types of questionnaires used. The first questionnaire was the Demographic Survey, the second was the Perceived Ownership Questionnaire used in the original experiment by Botvinick and Cohen (1998) to investigate perceived ownership, and the final questionnaire used was a Post Study survey about how the user felt after the experiment. See Appendix B for all the questionnaires used. Demographic Survey A demographic questionnaire was used to collect general information about the participant such as age, gender, handedness, whether they had corrected or normal vision, whether they had any disabilities that could affect the study and whether they had participated in any studies like this before. 29

41 The nominal data gathered from this questionnaire was used to identify and control the effects of any confounding variables, as well as provide information assisting in analysing the effects that familiarity with the rubber hand illusion has on the perceived ownership. Perceived Ownership Questionnaire The perceived ownership questionnaire was developed by Botvinick and Cohen (1998) to measure the perceived ownership of a rubber hand. They did this by hiding the left hand of a participant behind a screen and placing a rubber hand in the position at which the left hand would normally be in front of the participant. The rubber hand and the hidden real hand were then stroked and tapped in synchrony. The questionnaire was used to measure visual feedback, tactile feedback, and posture (proprioception) during the RHI. This questionnaire has now been used as a standard measurement tool to measure perceived ownership during the rubber hand illusion (Ehrsson et al., 2005; Haans et al., 2008; IJsselsteijn et al., 2006; Pavani, Spence, & Driver, 2000). Post Study Questionnaire The post study questionnaire consisted of three questions and they were used to measure the user experience at end of the experiment. This was to determine whether there was an effect on user experience depending on which experiment was run first, since this experiment was run in conjunction with two other experiments involving limb perception and the ART System. The experiments were run over 3 x 30 minute experiments with a total time of 1.5 hours. The questions asked were whether the participant felt bored, sleepy or uninterested. This was developed to determine whether the duration of the experiment was disproportionate or moderate, as well as to investigate whether participants found any interest in the research at the end of the experiment Environment The experiments were carried out in an air conditioned room. The room contained a desk which the ART System was placed on top of. The system mentioned is described in sections and of this work. In front of the system there was an adjustable chair so that the participant could sit in a comfortable position with their hands inside each box and the screen clearly visible to them, with the experiment facilitator seated behind the system. The room 30

42 used for this experiment was the Multimedia Systems Research Laboratory, located in the Information Science Annexe Building at the University of Otago Experiment Procedure The first step in the procedure was to recruit participants for the study, this was done through lists and approaching students at the University of Otago. There was no specific age or gender targeted for this experiment, so anyone who responded to the or request to participate were allowed to take part in this study. These experiments were scheduled to take place at least four or five times a day from Monday to Friday for one week. Each experiment would take approximately twenty minutes to complete and every participant was given a voucher for participating in this study. Once the participants arrived at the venue, they were welcomed by the experimenter and were seated and told the nature of the study and that they were allowed to withdraw at any time. The participant was then given the information sheet, task description and a consent form if they wanted to continue with the study (Appendix A). Once they had completed the consent form, it was placed into a box away from the rest of the paperwork, so that the participant would remain anonymous. A demographic sheet was then handed to the participant in order to obtain basic details. The demographic sheet was checked for completion and the sheet was placed into a separate folder from the consent form. The experimenter would then explain the first session to the participant, describing either the video mediated or the non-video mediated condition (Figure 3.3). Once the participants understood the task, they were placed in front of the ART System and positioned at a comfortable height and distance. At this stage, the participant was asked to remove all jewellery or wrist watches. 31

43 Figure 3.3 A. Vision (Video Mediated) B. Vision (Direct) The experimenter would then go to the other side of the ART System where they could access the system controls and begin the experiment. Depending on the session, the experimenter would either turn on the screen so that the participant would see their real right hand and a rubber left hand displayed on the screen or tell the participant to look into the ART boxes and watch the real right hand and rubber left hand in front of them. Each finger on both the real left hand and rubber left hand would then be stroked from left to right starting on the index finger, for the duration of five minutes with the frequency of 1 finger per 4 seconds and 1 second interval to change fingers. This stroking was done as synchronously as possible in order for the participant to feel the stroking at the same time that they saw the stroking on the rubber hand. At the end of the five minutes, after the stroking was complete, the participant was ask to move back to the desk they were seated at and asked to fill out a Perceived Ownership Questionnaire for that session. Once they had completed the questionnaire, they were then asked to sit back in front of the ART System. Depending on which condition was performed in the first session, the system was then made ready for the opposite condition in the second session. The procedure was based on the procedure used in the original experiment of Botvinick and Cohen (1998). The same procedure was carried out again in the second session, but depending on the session, the experimenter would either turn on the screen so that the participant would see their real right hand and a rubber left hand displayed on the screen or tell the participant to look into the ART boxes and watch the real right hand and rubber left hand in front of them. After completing the Perceived Ownership Questionnaire, the participant was asked to fill out a Post Study Questionnaire to measure the participant s user experience during the entire 32

44 study for each condition retrospectively. The participants were then debriefed, which involved explaining the purpose of the research as well as telling them more about the experiment and how it works. They were then informed about the next steps of the research and they were asked whether they would like to be notified about the results of the research. At this stage, participants could ask any questions they had about the experiment. The participants were then given a voucher for participating in the study and shown out of the building. The results were then captured into Microsoft Excel to be analysed later Data Collection There were several types of data collected in this study such as Demographic Data, Perceived Ownership Data and User Experience Data. The data of all questionnaires was entered manually into Microsoft Excel 2010 by the researcher then later transferred into IBM SPSS Statistics 20. The raw data from the questionnaires were kept on paper. To ensure anonymity, the consent forms of each participant were kept separately from the questionnaires. To identify the participants, a simple numbering scheme was used, P for Participant and the number of the experiment that was taking place, E.g. P1 - represented the first participant for the first experiment. This ensured that it was possible to identify each participant while still allowing them to remain anonymous Assumptions The following assumptions have been made for the purpose of achieving external validity: The participants have little or no knowledge about the experiment or the nature of how it works. If the participants know too much information about the experiment, this could affect the sense of perceived ownership. The participants will answer all questions about their perception honestly. Some participants might have a sense of perceived ownership towards the rubber hand, but they may feel shy or embarrassed to say that they felt a sense of ownership towards the rubber hand. It was assumed that the participants were honest about their perceived ownership during the experiment. 33

45 The tool used to measure perceived ownership is sufficient. The only tool used to measure perceived ownership was the questionnaire developed (Botvinick & Cohen, 1998). Some studies used skin conductance meters to measure SCR along with this questionnaire to measure perceived ownership. It was also assumed that the participants were a good representation of typical computer users. The reason for these assumptions was to not limit the scope of participants for the experiment. 34

46 3.5 Results This study was to investigate two types of rubber hand illusions, the original Rubber Hand Illusion (RHI) and the video mediated Rubber Hand Illusion (vrhi), to see whether there was a sense of perceived ownership in both conditions. This was done by using two setups under controlled lab conditions, allowing the participants to use each setup in each session and give feedback by completing a standardised questionnaire. The questionnaire was used to measure perceived ownership of the rubber hand at the end of each session. This section of the report presents the results of the research divided into perceived ownership in the RHI and the perceived ownership in the vrhi. The data was analysed using IBM SPSS Statistics 20 and Microsoft Excel 2010, to capture, sort and perform the relevant statistical tests, such as Kolmogorov-Smirnov to test for Normality of the data, ANOVA Repeated Measures Test to investigate whether there was a significant difference between each condition and the one sample t-test to investigate the hypothesis regarding perceived ownership. After receiving the physical copies of the participant s feedback, the data was manually inserted into Excel and sorted accordingly. The one sample t-test was used to analyse the data for each condition, to investigate whether there was perceived ownership in both sessions and if the strength of perceived ownership was significant. This was done only after testing that the data in each group was normally distributed. The result of this test, known as the p-value, will show whether the sense of perceived ownership was significant at a 95% confidence interval by producing a p-value less than or equal to 0.05 or not significant with a p-value greater than All the data that was collected from the participants was used in the analysis, there were no outliers. The tables containing all the data can be found in Appendix C of this Thesis Normal Distribution To test whether the data was normally distributed, Kolmogorov-Smirnov test was performed. If the test was non-significant (p >.05), this tells us that the distribution of the sample was not significantly different from a normal distribution, i.e. the data was normally distributed (Field, 2009). However if the test value was significant (p <.05) then the distribution in question was significantly different from a normal distribution, i.e. the data was not normally distributed (Field, 2009). 35

47 The descriptive statistic results shown in Table 3.1 indicate that in both the Kolmogorov- Smirnov and Shapiro-Wilk test for both conditions the data was normally distributed with a p- value larger than.05. Table 3.1 Test of Normality: Rubber Hand Illusion (RHI and vrhi) Conditions Kolmogorov-Smirnov a Shapiro-Wilk Statistic df Sig. Statistic df Sig. RHI Condition * vrhi Condition * *. This is a lower bound of the true significance. a. Lilliefors Significance Correction A graphical representation of the data distribution for each condition is given in the form of a histogram with the normal distribution overlaid. If the curve is a symmetrical, that does not seem too pointy or flat, this indicates a normally distributed data set (Field, 2009). Figure 3.4 for the RHI Condition and Figure 3.5 for the vrhi Condition display this normal distribution respectively. 36

48 Figure 3.4 Mean Distribution for RHI Condition Figure 3.5 Mean Distribution for vrhi Condition 37

49 Further descriptive statistics testing normality of the data can be found in Appendix C Mean Comparison To investigate whether there was a significant difference between each condition; a One-Way Repeated Measure ANOVA was performed. Included in this test was Mauchly s test of Sphericity, which tests that the variances of the differences between conditions were equal (Field, 2009). Also, the results of the ANOVA for the Within-Subject Effects and a Pairwise Comparison were reported. The results from Mauchly s Test of Sphericity were reported in Table 3.2. In order for Sphericity to be an issue, you need at least three conditions. Therefore, as there was only one repeated measure variable with two levels, it was assumed that Sphericity had been met. Hence the significance value is indicated as a dot and the estimate computed for Mauchly s W is 1, representing perfect Sphericity. Table 3.2 Mauchly's Test of Sphericity Within Subjects Effect Mauchly's W Approx. df Sig. Epsilon Chi-Square Greenhouse-Geisser Huynh-Feldt Lower-bound Condition Comparison We observe the results of the Test of Within-Subject Effects, taking note of the Condition Comparison and Error for Condition Comparison, if Sphericity is assumed, highlighted in Table

50 Table 3.3 Tests of Within-Subjects Effects Source Type III Sum of df Mean F Sig. Squares Square Sphericity Assumed ConditionComparison Geisser Huynh-Feldt Lower-bound Sphericity Assumed Error(ConditionComparison) Greenhouse- Greenhouse- Geisser Huynh-Feldt Lower-bound Mauchly s Test of Sphericity indicated that Sphericity is met. Since Sphericity was assumed, the results show that there was no significant difference between the RHI Condition and the vrhi Condition, F (1, 21) = 3.41, p >

51 A further post hoc test was carried out to investigate whether there was any significant difference between conditions. The Pairwise Comparison compares the difference between group means. The standard error, the significance value and a confidence interval for difference between means is displayed in Table 3.4. Table 3.4 Pairwise Comparison (I) Condition (J) Condition Mean Difference (I-J) Std. Error Sig. a 95% Confidence Interval for Difference a Lower Bound Upper Bound RHI vrhi vrhi RHI Based on estimated marginal means a. Adjustment for multiple comparisons: Bonferroni. The results of the Pairwise Comparison further support the earlier findings that there was no significant difference between the RHI Condition and the vrhi Condition, p > Perceived Ownership To investigate perceived ownership a one sample t-test was used to analyse the data for each condition, to investigate whether the perceived ownership in each session was significant. A number of statements such as, It seemed as if I were feeling the touch of the paintbrush in the location where I saw the rubber hand touched, It seemed as though the touch I felt was caused by the paintbrush touching the rubber hand or, I felt as if the rubber hand were my hand were said to be gauges to determine ownership depending on how strongly the participant agrees or disagrees with the statement (Botvinick & Cohen, 1998). To investigate ownership and how strong the sense of ownership was in this experiment, the researcher qualified a sense of ownership towards the rubber hand when the participants indicated a value equal to or higher than the midpoint value of 4 for each of these three statements. A mean value near or equal to 4 indicated a slight sense of ownership, whereas a mean value near or equal to 7 indicated a stronger sense of ownership. The t-test also compared whether there was a significant difference between the midpoint and mean values, for each statement. 40

52 RHI Perceived Ownership This section shows the results of the perceived ownership questions, analysed in the first three statements on the Questionnaire, during the RHI Condition of the study. Perceived drift and ownership of supernumerary limbs were reported in the statements following the first three. Each statement can be seen in Table 3.5. Table 3.5 Statements for RHI Condition Ownership towards the Rubber Hand Ownership towards the Rubber Hand Ownership towards the Rubber Hand Perceived Drift Ownership of Supernumerary Limb Ownership of Supernumerary Limb V1 V2 V3 V4 V5 V6 It seemed as if I were feeling the touch of the paintbrush in the location where I saw the rubber hand touched. It seemed as though the touch I felt was caused by the paintbrush touching the rubber hand. I felt as if the rubber hand were my hand. It felt as if my (real) hand were drifting towards the right (towards the rubber hand). It seemed as if I might have more than one left hand or arm. It seemed as if the touch I was feeling came from somewhere between my own hand and the rubber hand. Control Statement V7 It felt as if my (real) hand were turning rubbery. Perceived Drift Control Statement V8 V9 It appeared (visually) as if the rubber hand were drifting towards the left (towards my hand). The rubber hand began to resemble my own (real) hand, in terms of shape, skin tone, freckles or some other visual feature. 41

53 Table 3.6 RHI One Sample Statistic Statement N Mean Std. Deviation Std. Error Mean V V V V V V V V V In Table 3.6, The Mean, Std. Deviation and Std. Error Mean were presented, followed by a graph showing the mean values and Std. error means for each statement in Figure

54 7 RHI Condition Statements 2 1 V1 V2 V3 V4 V5 V6 V7 V8 V9 Figure 3.6 Mean and Std. Error Mean In RHI Condition It can be seen from the data in Figure 3.6 that mean values for the first three statements concerning perceived ownership were all above the midpoint value of 4 (V1: M = 6.27, V2: M = 5.32 and V3: M = 4.91). This indicated that there was a sense of perceived ownership towards the rubber hand in the Rubber Hand Illusion condition. Further analysis seen in Table 3.7 was performed to investigate whether the mean values for this condition were significantly different from the midpoint. The mean values for the statements concerning perceived ownership of a supernumerary limb (V5 and V6) were below the midpoint value of 4, which suggests that the participants perceived having only one left hand and one right hand. There was no perceived drift of the rubber hand or the real hand, which was indicated in statement V4 and V8 with mean values below the midpoint. In the control statements, V7 and V9, the participants real hand did not feel rubbery and the rubber hand did not resemble the participants own hand. This was indicated with mean values below the midpoint. 43

55 Table 3.7 RHI One-Sample Test Test Value = 4 t df Sig. (2-tailed) Mean Difference 95% Confidence Interval of the Difference Lower Upper V V V V V V V V V The results of the one sample t-test were shown in Table 3.7, comparing the mean value to the midpoint value of 4 with a confidence interval of 95% which is a p value = Any value below the mid-point disagrees with the given statement and any value at or above midpoint agrees with the given statement. The t-test results also indicate whether the mean value was significantly different from the midpoint value of 4. For the statements concerning perceived ownership, the mean values were significantly different from the midpoint value of 4 in the RHI condition (V1: p <.05, V2: p <.05 and V3: p<.05). 44

56 The results for the statements concerning ownership of a supernumerary limb also proved to be significantly different from the midpoint value (V5: p <.05 and V6: p <.05). One of the statements concerning drift was significantly different from the midpoint (V8: p <.05) whereas the other statement concerning drift (V4: p =.459) was not statistically significant. The results for both control statements were not statistically significant (V7: p =.756 and V9: p =.438) vrhi Perceived Ownership This section shows the results of the perceived ownership questionnaire, analysed in the first three statements on the Questionnaire, during the vrhi Condition of the study. Perceived drift and ownership of supernumerary limbs were reported in the statements following the first three. Each statement can be seen in Table 3.8. Table 3.8 Statements for vrhi Condition Ownership towards the Rubber Hand Ownership towards the Rubber Hand Ownership towards the Rubber Hand Perceived Drift Ownership of Supernumerary Limb Ownership of Supernumerary Limb A1 A2 A3 A4 A5 A6 It seemed as if I were feeling the touch of the paintbrush in the location where I saw the rubber hand touched. It seemed as though the touch I felt was caused by the paintbrush touching the rubber hand. I felt as if the rubber hand were my hand. It felt as if my (real) hand were drifting towards the right (towards the rubber hand). It seemed as if I might have more than one left hand or arm. It seemed as if the touch I was feeling came from somewhere between my own hand and the rubber hand. Control Statement A7 It felt as if my (real) hand were turning rubbery. Perceived Drift Control Statement A8 A9 It appeared (visually) as if the rubber hand were drifting towards the left (towards my hand). The rubber hand began to resemble my own (real) hand, in terms of shape, skin tone, freckles or some other visual feature. 45

57 Table 3.9 vrhi One Sample Statistics Statement N Mean Std. Deviation Std. Error Mean A A A A A A A A A In Table 3.9, The Mean, Std. Deviation and Std. Error Mean were presented, followed by a graph showing the mean values and Std. error means for each statement in Figure

58 7 ART Condition Statement 2 1 A1 A2 A3 A4 A5 A6 A7 A8 A9 Figure 3.7 Mean and Std. Error Mean In vrhi Condition It can be seen from the data (Figure 3.7) the mean values for the first three statements concerning perceived ownership were all above the midpoint value of 4 (A1: M = 6.27, A2: M = 5.09 and A3: M = 4.23). This indicates that there was a sense of perceived ownership towards the rubber hand in the vrhi condition. Further analysis seen in Table 3.10 was performed to investigate whether the mean values for this condition were significantly different from the midpoint. The mean values for the statements concerning perceived ownership of a supernumerary limb (A5 and A6) were below the midpoint value of 4, which suggests that the participants perceived having only one left hand and one right hand. There was no perceived drift of the rubber hand or the real hand, which was indicated in statement A4 and A8 with mean values below the midpoint. In the control statements, A7 and A9, the participants real hand did not feel rubbery and the rubber hand did not resemble the participants own hand. This was indicated with mean values below the midpoint. 47

59 Table 3.10 vrhi One Sample Test Test Value = 4 t df Sig. (2-tailed) Mean Difference 95% Confidence Interval of the Difference Lower Upper A A A A A A A A A The results of the one sample t-test are shown in Table 3.10, comparing the mean value to the midpoint value of 4 with a confidence interval of 95% which is a p value = Any value below the mid-point disagrees with the given statement and any value at or above midpoint agrees with the given statement. The t-test results also indicate whether the mean value was significantly different from the midpoint value of 4. For the first two statements concerning perceived ownership, the mean values were statistically different from the midpoint value of 4 in the vrhi condition (A1: p <.05 and A2: p <.05). The result for the third statement concerning ownership (A3: p =.554) was not statistically significant. 48

60 The results for the statements concerning ownership of a supernumerary limb also proved to be significantly different from the midpoint value (A5: p <.05 and A6: p <.05). Both of the statements concerning drift were significantly different from the midpoint (V4: p <.05 and V8: p <.05). The results for both control statements were not statistically significant (A7: p =.115 and A9: p =.498). 3.6 Discussion The results of this study suggest that there was a sense of perceived ownership in both the original Rubber Hand Illusion and in the video mediated Rubber Hand Illusion, even though the sensation was not as strong in the latter. This supports Hypothesis 1 which suggests that ART can generate a sense of perceived ownership during the Rubber Hand Illusion. These results support the findings of IJsselsteijn et al. (2006) which suggest that there was still a sense of perceived ownership towards a rubber hand, even if it was video mediated in a virtual reality condition or mixed reality condition. The findings were in contrast to that of Tsakiris and Haggard (2005) and IJsselsteijn et al. (2006) suggesting that position and location of the hand affects whether the Rubber Hand Illusion was possible or not. Costantini and Haggard (2007) suggested the sense of perceived ownership diminished as the degree of spatial disparity between visual and tactile feedback increased, which corresponded with the findings of this study. Although there was a sense of perceived ownership in both conditions, the sensation of ownership diminishes in the video mediated condition, which displays the hands as decoupled. Nonetheless, this study successfully demonstrates perceived ownership in the decoupled, video mediated RHI using the ART system. However, participants did not experience the sensation of owning a supernumerary limb and did not experience the sensation of drift in both the Rubber Hand Illusion and the video mediated Rubber Hand Illusion. A possible explanation for this could be; since the participant was only shown two hands, they perceived to have only one left hand and one right hand. It was assumed that if they were shown three hands, the participants would take ownership of all three. A further study investigating perceived ownership of a supernumerary limb was carried out in the following chapter (Chapter 4) to support this assumption. 49

61 4. Third Hand Illusion: Video and ART 4.1 Introduction In the previous chapter, we investigated and reported on perceived ownership during the original (RHI) and video mediated Rubber Hand Illusion (vrhi). The results suggested a sense of perceived ownership in both conditions. As mentioned previously, this chapter will investigate perceived ownership of three hands using the ART System. This study was based on the research and findings of Guterstam et al. (2011) which suggest that it is possible to perceive ownership of a third hand. Since the ART system succeeded in creating a sense of perceived ownership in the vrhi, this chapter will investigate whether the same phenomenon will occur in the Third Hand Illusion using the ART system. Chapter 4 describes the dependent and independent variables for this study, followed by a description of the changes made to the setup of ART system and detailed description of the software developed for this study. The research method is then discussed, giving a description of the experiment design, measurement tools and the procedure that were used to investigate perceived ownership. Thereafter an explanation will be given about the data collection process as well as the type of data collected in this study. The results of this study were then reported and assumptions for these findings will be stated and briefly discussed. Two factors of the original research were excluded from this study. The knife threat posed to the participant was not included in this study and there was no SCR measured in this study. A questionnaire was used to measure whether there was a sense of perceived ownership and if the strength of perceived ownership was significant. 50

62 4.2 Research Variables The independent and dependent variables were defined. The confounding variables in this study were the same as identified in Chapter Independent Variables The independent variable in this study was vision. There were two display methods used and they were defined as vision with two states: Vision (athi). ART mediated Third Hand Illusion (athi). This was when the subject saw a video stream of the rubber hand alongside their real hands on the screen, with no physical rubber hand present in the hardware setup. Vision (mthi). ART mediated, mirrored Third Hand Illusion (mthi). This was when the participant was shown their real right hand, a mirror image of their right hand in place of their left hand and a video stream of the rubber hand, with no physical rubber hand present in the hardware setup Dependent Variables The dependent variable in this study was perceived ownership. This is defined as follows: Perceived Ownership: This is the sense of ownership a person feels toward a rubber hand, after feeling their own hidden hand being synchronously touched in the corresponding location of the rubber hand. 4.3 System Setup The ART system which was used for in the previous study was used once more in this study, with some changes made to the setup. These changes will be described under Section in this Chapter. To investigate perceived ownership during the Third Hand Illusion, the software used in the previous study did not meet the requirements to create the conditions necessary for this study. A new software application was developed in order to create the conditions needed to carry out this investigation; this software is described under Section in this Chapter. 51

63 4.3.1 Hardware The hardware described by Regenbrecht Regenbrecht, Franz, et al. (2011a) was still used to perform this study, with some changes made to the setup. For this study a third screen was added to the setup and only two of the three boxes were used next to each other (Figure 4.1). Figure 4.1 Setup for THI (without third box) Within each of these boxes there were two cameras, one in each box, as described in the previous chapter. As mentioned before, in addition to the two existing monitors, a second wide screen monitor was placed alongside the boxes. This was so the experimenter could see exactly what the participant was seeing. Both these control monitors were out of the participant s field of view. This was achieved by rotating both the screens to face the opposite direction of the participant (Figure 4.2). The system was still controlled by a standard keyboard and mouse as in the previous study. 52

64 Figure 4.2 Setup for THI (with third screen) As shown in Figure 4.2, the setup now allowed for the experimenter to see exactly what the participant was seeing, while controlling the application on the third screen. The wide screen monitor was placed next to the box which contained the participant s hand that was going to be stroked. This was necessary in order to allow the experimenter to stroke the real hand, in synchrony with the brush strokes being made on the rubber hand in the video Software For this study, the software described in Chapter 3 could not be used. New software needed to be developed in order to create the two conditions needed to carry out the investigation. The functionality required to create these conditions was discussed; which resulted in the software requirements defined below. 53

65 Software Requirements For the athi condition, the software needed to allow the user to view the video streams of their right hand, the rubber left hand in the centre being stroked and their own left hand being stroked. This is shown in Figure 4.3. Figure 4.3 The athi Condition Setup In Figure 4.3, it can be seen in the athi condition that the participant was shown their real right on the right of the screen, with the video stream of the rubber hand being stroked in the centre and the real left hand being stroked on the right. For the mthi condition, the participant needed to show the user the video streams of their right hand, the rubber left hand in the centre being stroked and a mirrored imaged of their right hand in place of the left hand. This is shown in Figure

66 Figure 4.4 The mthi Condition Setup Figure 4.4 shows mthi condition that the participant was shown their real right on the right of the screen, with the video stream of the rubber hand being stroked in the centre and a mirror image of their right hand in place of their left hand. This condition was used to investigate perceived ownership when the user was made to believe none of their hands were being stroked, by altering the participant s visual perception to contrast what they felt on their hand Software Components The ART System 5.0 software to meet these requirements was developed using the Unity 3d Game Engine and Visual Studio All components of the software were developed, including the plugins and dynamic link libraries (dll s) used in the software. The plugins were written in C++ using the OpenCV library in Visual Studio A plugin had to be written for each camera stream, because if one plugin was written to deal with both camera streams, the processing time increased noticeably. Since Unity 3d requires its plugins to be compiled as dll files, the two plugin programs created in Visual Studio had to be 55

67 compiled as dll s and imported into Unity. Two functions where created, one to begin camera capture and the other to release or stop camera capture, for each camera stream. Once these programs had been built and compiled these dll files were imported into Unity, the functions of the dll could then be accessed and used in the software. To create the software in Unity a new project was created, to accomplish this Unity 3d Pro version was required. In this project, a scene was created which could be described as the physical environment of the software application. Planes which can be described as a Graphical User Interface (GUI), where then added to this scene (Figure 4.5). Some features of these planes included moving them around, resizing or rotating them. Each of these planes that were created also had a set of properties for the object selected. Figure 4.5 The three planes shown alongside each other 56

68 These planes needed to be rotated and moved so that they faced perpendicular to the Main Camera/Game view; which was the view that the participant would see during the study. A light source was needed to see what was going on, so a directional light source needed to be added to the scene as well. The game view and light source can be seen in Figure 4.6. Figure 4.6 Left: Game View. Right: The light source in Scene View Once the plugins have been imported into unity, the functions can be used inside of a script. The script was attached to the plane was written in C#; it allowed each function of the plugin to be called on that plane. Scripts were created and attached to each of the three planes created earlier. The script contained a number of functions, such as a start function which is called on initialisation of the application. Everything that was in the start function is run at the beginning of the program. A function call was created to close the camera once the plane was no longer in use and disabled. This was because the original function call that was used turned off the camera on all planes if the same script was attached on all planes. So an example of this would be if you were watching a camera stream shown on the first and second plane, if you made a function call to close the camera, the stream would stop on both planes. The other functions used were to retrieve, update and apply the video texture onto the planes. 57

69 Each of the three planes that were created was given three sub planes; the script was then attached to each of these sub planes. The reason for these three sub planes was to display three textures. The first sub-plane contained a movie texture, which would display the prerecorded video of the rubber hand being stroked. The second sub-plane contained the script to display the stream from the camera in the left box. The third sub-plane contained the script to display the stream from the camera in the right box. The video was made using the stream from one of the cameras. Keyboard controls where added to pause (pause), play (p) and stop (stop) the video at any time when the application was running. The last task in developing the software involved adding controls to GUI of the application that would give the experimenter the option to create the conditions needed for the study. The controls that were added to the application allowed the experimenter to switch between the left camera stream, right camera stream and the pre-recorded video on each plane. In addition to this, the functionality was added to mirror the camera stream on each of the planes, which was necessary for one of the conditions of the study. These controls can be seen in Figure 4.7. Figure 4.7 The application controls shown at start-up A detailed technical report documenting the development of this application is yet to be published. 58

70 4.4 Research Method Experiment Design This study used 32 participants in a within subject, repeated measure experimental design. The environment used during this study was the same as in Chapter 3. There were two conditions in this experiment; the ART mediated Third Hand Illusion (athi) used the camera streams of the left and right hand alongside a pre-recorded video of the rubber hand as the third hand. The other condition was the ART mediated, mirrored Third Hand Illusion (mthi). This used the camera stream of the right hand with a mirrored stream of the right hand in place of the left hand and a pre-recorded video of the rubber hand as the third hand. The study consisted of two sessions, one for each of the conditions. Each of the participants performed both conditions of the experiment, but the order of the within subject conditions were randomised. If the participant performed the mthi condition for the first session, they would perform the athi condition in the second session and vice versa. The reason for choosing a within subject, repeated measure design; was to investigate and compare the difference between the two conditions. This would also mitigate the learning effect. The order of conditions was balanced so that both conditions were in the first session an equal number of times during the 32 experiments of this study. This meant that the athi Condition was performed in the first session of 16 experiments and mthi condition was performed in the first session of the other 16 experiments Participants and Task In this experiment there were a total of 34 volunteers used in this study, 15 males and 19 females. All of the volunteers used in this study were students from the University of Otago. The first two participant of the study were used as a pilot study subjects and were excluded from the final results of the experiment. The age of the 31 participants ranged from 19 to 36 years (M = 25, SD = 4.51). One of the participants chose not to reveal their age. Of the 32 participants, 16 of them did not have prior knowledge about the experiment, while 4 of them had heard of the experiment before and 2 of them had participated in a similar experiment. When the 6 participants were asked about how much knowledge they had about the experiments, their knowledge was limited. All of the participants were recruited from different departments within the University of Otago. 59

71 During each phase of the experiment, the participants were asked to place their hands within the boxes on the left and right, respectively, while watching the hands on the screen above the boxes (Figure 4.8). The participants were shown the athi condition and mthi condition in random order. There were no hand movements and there was no physical task that needed to be performed Questionnaires Figure 4.8 An example of the Experiment Setup For this research there were three types of questionnaires used. The first questionnaire was the Demographic Survey used in previous study and described in Chapter 3, the second was the Perceived Ownership Questionnaire used in the original experiment by Guterstam et al. (2011) to investigate perceived ownership of a third hand, and the final questionnaire used was a Post Study Survey about how the user felt after the experiment also used in previous study and described in Chapter 3. See Appendix B for all the questionnaires used. 60

72 Perceived Ownership Questionnaire The original perceived ownership questionnaire was developed by Botvinick and Cohen (1998) to measure the perceived ownership of a rubber hand by investigating visual feedback, tactile feedback, and posture (proprioception). The questionnaire used by Guterstam et al. (2011) was based on the original questionnaire and modified to measure perceived ownership of a third hand during the Third Hand Illusion (THI). The questionnaire in this study was based on the one used by Guterstam et al. (2011) to try and duplicate their THI research as closely as possible. There were only two changes made to the questionnaire. The first change was referring to the left hand instead of the right hand in this study. This was because in the study performed by Guterstam et al. (2011) they used a rubber right hand alongside the real right hand, in this study a rubber left hand was displayed alongside the real left hand or mirrored right hand. The second change was using a 7 Point Likert Scale like the one used in original perceived ownership questionnaire developed by Botvinick and Cohen (1998), instead of the 9 Point Likert scale used by Guterstam et al. (2011) Experiment Procedure The experiment procedure was similar to the one followed in Chapter 3, with a few minor changes made. The procedure was the same up to the point at which the participant was seated in front of the system. A description of the procedure is given from this point onwards: Once the participant understood the task, they were placed in front of the ART System and positioned at a comfortable height and distance. At this stage, the participant was asked to remove all jewellery or wrist watches. The experimenter would then go to the other side of the ART System where they could access the system controls and begin the experiment. Depending on the session, the experimenter would setup the athi condition, in which the participant would see their real right hand, a rubber left hand and their real left hand displayed on the screen. Or the mthi condition, where the participant would see the real right hand, a rubber left hand and a mirrored image of their right hand, in place of their left hand. The experimenter would then start the video of the rubber hand being stroked. The participant would either see each finger on both the real left hand and rubber left hand being stroked from left to right starting on the index finger, for the duration of five minutes. Or they would see 61

73 each finger on the rubber left hand being stroked from left to right starting on the index finger, for the duration of five minutes. This stroking was done as synchronously as possible in order for the participant to feel the stroking at the same time that they saw the stroking on the rubber hand. At the end of the five minutes, after the stroking was complete, the participant was ask to move back to the desk they were seated at and asked to fill out a Perceived Ownership Questionnaire for that session. Once they had completed the questionnaire, they were then asked to sit back in front of the ART System. Depending on which condition was performed in the first session, the system was then made ready for the opposite condition in the second session. The procedure was then carried out again in the second session. At the end of the five minute session, the participant was ask to move back to the desk they were seated at and asked to fill out a perceived ownership questionnaire for that session. After completing the Perceived Ownership Questionnaire, the participant was asked to fill out a Post Study Questionnaire to measure the participant s user experience during the entire study for each condition retrospectively. The participants were then debriefed about the next steps of the research and they were asked whether they would like to be notified about the results of the research. The participants were then given a block of chocolate for participating in the study and shown out of the building. The results where then captured into Microsoft Excel to be analysed later Data Collection There were several types of data collected in this study such as Demographic Data, Perceived Ownership Data and User Experience Data. The data of all questionnaires was entered manually into Microsoft Excel 2010 by the researcher then later transferred into IBM SPSS Statistics 20. The raw data from the questionnaires were kept on paper. To ensure anonymity, the consent forms of each participant were kept separately from the questionnaires. To identify the participants, a simple numbering scheme was used, P for Participant and the number of the experiment that was taking place, E.g. P1 - represented the first participant for the first experiment. This ensured that it was possible to identify each participant while still allowing them to remain anonymous. 62

74 4.4.6 Assumptions The following assumptions have been made for the purpose of achieving external validity: The participants have little or no knowledge about the experiment or the nature of how it works. If the participants know too much information about the experiment, this could affect the sense of perceived ownership. The participants will answer all questions about their perception honestly. Some participants might have a sense of perceived ownership towards the rubber hand, but they may feel shy or embarrassed to say that they felt a sense of ownership towards the rubber hand. It was assumed that the participants were honest about their perceived ownership during the experiment. The tool used to measure perceived ownership is sufficient. The only tool used to measure perceived ownership was based on the questionnaire used by Guterstam et al. (2011). As mention before, this study did not use skin conductance meters to measure SCR to measure perceived ownership. It was also assumed that the participants were a good representation of typical computer users. The reason for these assumptions was to not limit the scope of participants for the experiment. 63

75 4.5 Results This study was to investigate two types of third hand illusions, the ART mediate Third Hand Illusion (athi) and the ART mediated, mirrored Third Hand Illusion (mthi), to see whether there was a sense of perceived ownership in both conditions. This was done by using two setups under controlled lab conditions, allowing the participants to use each setup in each session and give feedback by completing a standardised questionnaire. The questionnaire was used to measure perceived ownership of the rubber hand at the end of each session. This section of the report presents the results of the research divided into perceived ownership in the athi and the perceived ownership in the mthi. The data was analysed using IBM SPSS Statistics 20 and Microsoft Excel 2010, to capture, sort and perform the relevant statistical tests, such as Kolmogorov-Smirnov to test for Normality of the data, ANOVA Repeated Measures Test to investigate whether there was a significant difference between each condition and the one sample t-test to investigate the hypothesis regarding perceived ownership. After receiving the physical copies of the participant s feedback, the data was manually inserted into Excel and sorted accordingly. The one sample t-test was used to analyse the data for each condition, to investigate whether there was perceived ownership in both sessions and if the strength of perceived ownership was significant. This was done only after testing that the data in each group was normally distributed. The result of this test, known as the p-value, will show whether the sense of perceived ownership was significant at a 95% confidence interval by producing a p-value less than or equal to 0.05 or not significant with a p-value greater than All the data that was collected from the participants was used in the analysis, there were no outliers. The tables containing all the data can be found in Appendix C of this Thesis Normal Distribution To test whether the data was normally distributed, Kolmogorov-Smirnov test was performed. If the test was non-significant (p >.05), this tells us that the distribution of the sample was not significantly different from a normal distribution, i.e. the data was normally distributed (Field, 2009). However if the test value was significant (p <.05) then the distribution in question was significantly different from a normal distribution, i.e. the data was not normally distributed (Field, 2009). 64

76 The descriptive statistic results shown in Table 4.1 indicate that in the Kolmogorov-Smirnov and Shapiro-Wilk test for athi condition, the data was not normally distributed with a p- value less than.05. In the mthi condition, the data was normally distributed, with a p-value larger than.05. Table 4.1 of Normality: Third Hand Illusion (athi and mthi) Kolmogorov-Smirnova Shapiro-Wilk Statistic df Sig. Statistic df Sig. athi Condition mthi Condition * * This is a lower bound of the true significance. a Lilliefors Significance Correction A graphical representation of data distribution for each condition is given in the form of a histogram with the normal distribution overlaid. If the curve is a symmetrical, that does not seem too pointy or flat, this indicates a normally distributed data set (Field, 2009). Figure 4.9 for the athi Condition and Figure 4.10 for the mthi Condition display the data distribution respectively. 65

77 Figure 4.9 Mean Distribution for athi Condition Figure 4.10 Mean Distribution for mthi Condition 66

78 Further descriptive statistics testing normality of the data can be found in Appendix C Mean Comparison To investigate whether there was a significant difference between each condition; a One-Way Repeated Measure ANOVA was performed. Included in this test was Mauchly s test of Sphericity, which tests that the variances of the differences between conditions were equal (Field, 2009). Also, the results of the ANOVA for the Within-Subject Effects and a Pairwise Comparison were reported. The results from Mauchly s Test of Sphericity were reported in Table 4.2. In order for Sphericity to be an issue, you need at least three conditions Field (2009). Therefore, as there was only one repeated measure variable with two levels, it is assumed that Sphericity has been met. Hence the significance value is indicated as a dot and the estimate computed for Mauchly s W is 1, representing perfect Sphericity. Table 4.2 Mauchly's Test of Sphericity Within Subjects Effect Mauchly's W Approx. df Sig. Epsilonb Chi-Square Greenhouse-Geisser Huynh-Feldt Lower-bound Condition Comparison We observe the results of the Test of Within-Subject Effects, taking note of the Condition Comparison and Error for Condition Comparison, if Sphericity is assumed, highlighted in Table

79 Table 4.3 Tests of Within-Subjects Effects Source Type III Sum of df Mean F Sig. Squares Square Sphericity Assumed ConditionComparison Geisser Huynh-Feldt Lower-bound Sphericity Assumed Error(ConditionComparison) Greenhouse- Greenhouse- Geisser Huynh-Feldt Lower-bound Mauchly s Test of Sphericity indicated that Sphericity is met. Since Sphericity was assumed, the results show that there was a significant difference between the athi Condition and the mthi Condition, F (1, 31) = 2.64, p <

80 A further post hoc test was carried out to investigate whether there was any significant difference between conditions. The Pairwise Comparison compares the difference between group means. The standard error, the significance value and a confidence interval for difference between means is displayed in Table 4.4. Table 4.4 Pairwise Comparison (I) Condition (J) Condition Mean Difference (I-J) Std. Error Sig. b 95% Confidence Interval for Difference b Lower Bound Upper Bound athi mthi * mthi athi.406 * Based on estimated marginal means b. Adjustment for multiple comparisons: Bonferroni. The results of the Pairwise Comparison further support the earlier findings that there was a significant difference between the athi Condition and the mthi Condition, p < Perceived Ownership To investigate perceived ownership a one sample t-test was used to analyse the data for each condition, to investigate whether the perceived ownership in each session was significant. Guterstam et al. (2011) suggest the statements, I felt the touch of the brush on the rubber hand and I felt as if the rubber hand was my hand were used to measure perceived ownership of the rubber hand and the statements, I felt the touch of the brush on my (real) hand and I could no longer feel that my (real) hand belonged to my body were said to measure disownership of the real hand. They further suggest the statements, I could feel the touch of the brush on both hands at the same time and It felt as if I had two right hands were used to measure perceived ownership of the supernumerary limb. To investigate ownership and how strong the sense of ownership was in this experiment, the researcher qualified a sense of ownership towards the rubber hand when the participants indicated a value equal to or higher than the midpoint value of 4 for each of these three statements. A mean value near or equal to 4 indicated a slight sense of ownership, whereas a mean value near or equal to 7 indicated a stronger sense of ownership. The t-test also 69

81 compared whether there was a significant difference between the midpoint and mean values, for each statement athi Perceived Ownership This section shows the results of the perceived ownership questions, analysed in the first two statements on the Questionnaire, during the athi Condition of the study. Perceived disownership of the real hand and ownership of a supernumerary limb were reported in the statements following the first two. There was also four control statements. Each statement can be seen in Table 4.5. Table 4.5 Statements for athi Condition Ownership of the rubber hand Ownership of the rubber hand Disownership of the real hand Disownership of the real hand Control Statement Ownership of Supernumerary Limb Ownership of Supernumerary Limb Control Statement Control Statement Control Statement athi_ Q1 athi_ Q2 athi_ Q3 athi_ Q4 athi_ Q5 athi_ Q6 athi_ Q7 athi_ Q8 athi_ Q9 athi_ Q10 I felt the touch of the brush on the rubber hand. I felt as if the rubber hand was my hand. I felt the touch of the brush on my (real) hand. I could no longer feel that my (real) hand belonged to my body. I felt the touch of the brush on both hands, but never at the same time. I could feel the touch of the brush on both hands at the same time. It felt as if I had two left hands. It felt as if my (real) hand were turning rubbery. It seemed as if the touch I was feeling came from somewhere between my own hand and the rubber hand. It appeared (visually) as if the rubber hand was drifting towards the left (towards my real hand). 70

82 Table 4.6 athi One Sample Statistics Statement N Mean Std. Deviation Std. Error Mean athi_q athi_q athi_q athi_q athi_q athi_q athi_q athi_q athi_q athi_q In Table 4.6, The Mean, Std. Deviation and Std. Error Mean were presented, followed by a graph showing the mean values and Std. error means for each statement in Figure

83 athi Condition Statements Figure 4.11 Mean and Std. Error Mean in athi Condition It can be seen from the Figure 4.11 that mean values for the first two statements concerning perceived ownership were below the midpoint value of 4 (athi_q1 and athi _Q2). This indicates that there was no perceived ownership towards the rubber hand in the athi condition. Further analysis seen in Table 4.7 was performed to investigate whether the mean values for this condition were significantly different from the midpoint. The mean values for the statements concerning perceived disownership of real hand (athi_q3: M = 6.66 and athi_q4: M = 1.97) suggests that the participants still perceived their real hand as belonging to their body. The mean values for the statements concerning perceived ownership of a supernumerary limb (athi_q6 and athi_q7) were below the midpoint value of 4, which suggests that the participants perceived having only one left hand and one right hand. In the control statements, athi_q5, athi_q8, athi_q9 and athi_q10, the mean values were all below the midpoint suggesting that the participants did not agree with any of the control statements. 72

84 Table 4.7 athi One Sample Test Test Value = 4 t df Sig. (2-tailed) Mean Difference 95% Confidence Interval of the Difference Lower Upper athi_q athi_q athi_q athi_q athi_q athi_q athi_q athi_q athi_q athi_q The results of the one sample t-test were shown in Table 4.7, comparing the mean value to the midpoint value of 4 with a confidence interval of 95% which is a p value = Any value below the mid-point disagrees with the given statement and any value at or above midpoint agrees with the given statement. The t-test results also indicate whether the mean value was significantly different from the midpoint value of 4. For the statements concerning perceived ownership, the mean values were significantly different from the midpoint value of 4 in the athi condition (athi_q1: p <.05 and athi_q2: p <.05). 73

85 The results for the statements concerning disownership of the real hand proved to be significantly different from the midpoint value (athi_q4: p <.05 and athi_q5: p <.05). The results for the statements concerning ownership of a supernumerary limb proved to be significantly different from the midpoint value (athi_q6: p <.05 and athi_q7: p <.05). The results for all control statements proved to be significantly different from the midpoint value (athi_q5 = p <.05, athi_q8 = p <.05, athi_q9 = p <.05 and athi_q10 = p <.05) mthi Perceived Ownership This section shows the results of the perceived ownership questions, analysed in the first two statements on the Questionnaire, during the mthi Condition of the study. Perceived disownership of the real hand and ownership a supernumerary limb were reported in the statements following the first two. There was also four control statements. Each statement can be seen in Table 4.8. Table 4.8 Statements for mthi Condition Ownership of the rubber hand Ownership of the rubber hand Disownership of the real hand Disownership of the real hand Control Statement Ownership of Supernumerary Limb Ownership of Supernumerary Limb Control Statement Control Statement Control Statement mthi_ Q1 mthi_ Q2 mthi_ Q3 mthi_ Q4 mthi_ Q5 mthi_ Q6 mthi_ Q7 mthi_ Q8 mthi_ Q9 mthi_ Q10 I felt the touch of the brush on the rubber hand. I felt as if the rubber hand was my hand. I felt the touch of the brush on my (real) hand. I could no longer feel that my (real) hand belonged to my body. I felt the touch of the brush on both hands, but never at the same time. I could feel the touch of the brush on both hands at the same time. It felt as if I had two left hands. It felt as if my (real) hand were turning rubbery. It seemed as if the touch I was feeling came from somewhere between my own hand and the rubber hand. It appeared (visually) as if the rubber hand was drifting towards the left (towards my real hand). 74

86 Table 4.9 mthi One Sample Statistics Statement N Mean Std. Deviation Std. Error Mean mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q In Table 4.9, The Mean, Std. Deviation and Std. Error Mean were presented, followed by a graph showing the mean values and Std. error means for each statement in Figure

87 mthi Condition Statement Figure 4.12 Mean and Std. Error Mean In mthi Condition It can be seen from the Figure 4.12 the mean values for the first two statements concerning perceived ownership were above the midpoint value of 4: mthi_q1 = 4.53, mthi _Q2 = This indicates that there was perceived ownership towards the rubber hand in the mthi condition. Further analysis seen in Table 4.10 was performed to investigate whether the mean values for this condition were significantly different from the midpoint. The mean values for the statements concerning perceived disownership of real hand (mthi_q3 = 6.59 and mthi_q4 = 2.44) suggests that the participants still perceived their real hand as belonging to their body. The mean values for the statements concerning perceived ownership of a supernumerary limb (athi_q6 = 3.19 and athi_q7 = 2.19) were below the midpoint value of 4, which suggests that the participants perceived having only one left hand and one right hand. In the control statements, athi_q5, athi_q8, athi_q9 and athi_q10, the mean values were all below the midpoint suggesting that the participants did not agree with any of the control statements. 76

88 Table 4.10 mthi One Sample Test Test Value = 4 t df Sig. (2-tailed) Mean Difference 95% Confidence Interval of the Difference Lower Upper mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q The results of the one sample t-test were shown in Table 4.10, comparing the mean value to the midpoint value of 4 with a confidence interval of 95% which is a p value = Any value below the mid-point disagrees with the given statement and any value at or above midpoint agrees with the given statement. The t-test results also indicate whether the mean value was significantly different from the midpoint value of 4. For the statements concerning perceived ownership, the mean values in the mthi condition (mthi_q1: p >.179 and mthi_q2: p >.179) were not statistically significant. 77

89 4.6 Discussion The results of this study suggest that there was a sense of perceived ownership toward the rubber hand in the mthi condition and no perceived ownership in the athi condition. This does not support Hypothesis 2 which suggests that ART system can create a sense of perceived ownership during the Third Hand Illusion. The findings of the current study do not support the previous research by Guterstam et al. (2011). This study was unable to demonstrate perceived ownership towards a rubber hand, when the rubber hand was being stroked alongside the participants visible left and right hands. Although these results differ from the results of Guterstam et al. (2011) which this study tried to replicate, they were consistent with those of Newport et al. (2010). The researchers suggest that fake limbs can be incorporated as part of the body, but the body schema can accommodate only one of each real hand. Although there was a sense of perceived ownership in the mthi condition towards the rubber hand, the participants reported ownership of their own hand and the rubber hand, but not a supernumerary limb. The findings of this study did not demonstrate perceived ownership in the decoupled, ART mediated Third Hand Illusion. This study did demonstrate perceived ownership towards the rubber hand and left hand in the ART mediated mirrored Third Hand Illusion, however the participants did not experience the sensation of owning a second left hand, just a third hand. Assumptions related to the results and findings throughout this research will be discussed in the following chapter of this Thesis. 78

90 5. Conclusion and Future Work This chapter will discuss the findings and assumptions made regarding this research. A short overview of the limitations during this study will be reported, followed by a final conclusion and thoughts for future work. 5.1 Conclusion The motivation for this research was to investigate perceived ownership during the vrhi and athi. The related literature was vague in reporting as to whether perceived ownership was possible in either of these conditions and failed on answering several questions within the previous studies. This study sought to answer two of these questions. Could the RHI and THI be replicated using the ART system? Will the sensation of perceived ownership be present if the illusion was performed through the ART System? The purpose of this study was to demonstrate perceived ownership in the decoupled, video mediated RHI (vrhi) using the ART system and demonstrate perceived ownership in the decoupled, ART mediated THI (athi) using the ART system. The first study of this research investigated vision as the independent variable with two states; direct vision and video mediated vision. Direct vision represented the original RHI and video mediated vision represented the RHI mediated through the ART system. The dependent variable in this study was perceived ownership. There was no significant difference found when comparing the means of the RHI and vrhi. Both conditions of the study were successful in demonstrating perceived ownership. Albeit the sense of perceived ownership was not as strong as in the vrhi compared to original RHI, the results of the study were found to be statistically significant. This suggests that it was possible to still maintain ownership of the rubber hand even when it was video mediated and decoupled. The display quality of the screen could have caused the rubber hand to look less real in the vrhi condition, causing a weaker sense of perceived ownership. The participants did not report their real hand as turning rubbery or the rubber hand resembling their own hand in anyway. This could suggest that participants perceived the rubber hand as a real hand and belonging to their own body. At the conclusion of the study, the findings suggested that the ART system was successful in duplicating the RHI and demonstrating perceived ownership in the vrhi. 79

91 The findings of this study contribute to the existing knowledge of the RHI by providing evidence demonstrating Neuroplastic change in the participants during the study. The method used in this study to stimulate this Neuroplastic change could be used to allow new forms of therapies using the ART system, such as treating patients with PLP by referring ownership to the rubber hand in place of the phantom limb as stated in Chapter 1. The findings provided evidence to continue investigation to see whether it was possible to demonstrate perceived ownership in the athi condition. The second study investigated vision as the independent variable with two states; ART mediated, Third Hand Illusion and ART mediated, mirrored Third Hand Illusion. The dependent variable again was perceived ownership. There was a significant difference found when comparing the means of the athi and mthi. The athi condition of the study was not successful in demonstrating perceived ownership of the rubber hand, whereas the mthi condition was successful in demonstrating perceived ownership. Even though the results of the study were found to be statistically significant, the mthi condition did not succeed in perceived ownership of a supernumerary limb. This suggests that it was possible to maintain ownership of the rubber hand and the real left hand, but not two left hands. This study demonstrated perceived ownership of a rubber hand and left hand, but not as a supernumerary limb, in the mthi condition. Participants verbally reported to have felt the brushing on the real left hand but did not see it, assuming they just imagined the feeling because they saw the rubber hand being brushed. The study was not successful in duplicating the THI or demonstrating perceived ownership in this study. In addition to the display quality of the screen being a possible factor affecting the study, synchrony of brush strokes could have been a factor as well. Since a pre-recorded video was used, complete synchrony of brush strokes were not possible. As the participant would see both the rubber hand and real hand being stroked, it would be easier to notice if the strokes were not in sync. Again, participants did not report their real hand as turning rubbery or the rubber hand resembling their own hand. In view of these findings suggesting perceived ownership of the rubber hand during the mthi; the methods used in referring sensation to the rubber hand may be applied to treat patients with CRPS. A number of limitations where noted during this study but decoupling of the hands was not one of the limitations encountered. Previous studies suggested that the hands needed to be in the correct position, location and orientation for RHI to be successful. One of the first 80

92 limitations noted was the angle at which the screen was positioned above the boxes. Participants verbally commented about the discomfort of having to look up at the screen. Especially in the second study, this required them to look at the screen for both conditions. Another limitation was faced when the first study was run in collaboration with two other studies in the lab. This presented a number of problems. When the study was run after either one of the other studies, which both involved using the participant s hands and the ART system, a number of participants reported feeling either bored or sleepy. This could have affected the perception of the participants, who would have to sit at the system for more than an hour staring up at the screen. As mentioned earlier, this caused a number of participants discomfort. Another problem was mitigating the learning effect, since the participants would be familiar with the system and how it worked if this study followed either of the other two studies. In one of the sessions where this study was performed last, a participant was unsure if there were really two brushes used, since they saw the other two illusions prior to this study. The main limitation was time. If there was more time to perform either of the studies, the results may perhaps be stronger. As demonstrated in the original rubber hand illusion by Botvinick and Cohen (1998); if the rubber hand and real hand were stroked for a longer period, the sensation of ownership towards the rubber hand becomes stronger. The participants hands were only stroked for a maximum of five minutes for both studies in this research. With more time, there would be a larger sample size. With a wide population distribution, chance variation is ruled out; providing results with greater significance. To conclude, this research supports the following hypothesis: Hypothesis 1: The ART system can create a sense of perceived ownership during the decoupled, video mediated Rubber Hand Illusion. The results of this research did not provide evidence to support the following hypothesis: Hypothesis 2: The ART system can create a sense of perceived ownership during the decoupled, ART mediated Third Hand Illusion. 81

93 5.2 Future Work The findings of this thesis suggest a number of possible avenues of research that could be carried out using the ART system. One of these avenues is to investigate and identify the capabilities of the system as therapeutic technology; for people with limb impairments which originate in the brain such as PLP, CRPS or people who have suffered a stroke could be treated using neuroplastic change. As neuroplastic change was demonstrated in this thesis, a number of possible therapies could be investigated. If successful in creating a system that could be used to treat these people; their quality of life may be improved. The different hardware setups of this system could be tested to investigate methods to make the system more portable and user friendly. A portable system would be ideal for people to use for home rehabilitation therapy. Therapists could use this system to treat people in rural areas, who would not normally be able to travel to physiotherapy clinics. If the system is to be used as a therapeutic technology, investigation needs to be carried out for suitable display methods. As the current setup was reported to cause discomfort when participants were made to stare upward at the screen, a setup that would have a screen at eye-level could be further examined. In regards to the display method, high definition screens could be another setup to consider. This could get rid of the problem of picture quality affecting the visual appearance of the rubber hand. If the problem was not the screen, it may be the cameras used that could have attributed to the lack of ownership in the athi condition. If the cameras did not display the frames fast enough to match the strokes of the pre-recorded video and vice versa, this would be noticeable by the participant. Better cameras and video rendering methods could be investigated in a similar setup. Since the factor of the knife threat was not included in the second study of this Thesis as in the original THI; this could have been another factor that influenced perceived ownership in the original experiment. Further research to compare a knife vs. no knife needs to be investigated in the ART system. It could be possible to create a virtual knife threat in the system or a pre-recorded video of a knife could be used. The virtual knife threat vs. the real knife threat may even be two conditions to compare. Other threats may be made to investigate whether there is perceived ownership and if the strength of perceived ownership is increased with a range of threats towards the rubber hand. This could be measured with SCR which was not included in this research, but could be a possible avenue to investigate. The possibilities of future work with the ART system are numerous, as the mind has no limits to creativity. 82

94 References Armel, K. C., & Ramachandran, V. S. (2003). Projecting sensations to external objects: evidence from skin conductance response. Proceedings of the Royal Society of London. Series B: Biological Sciences, 270(1523), Bekrater-Bodmann, R., Foell, J., & Flor, H. (2011). Relationship between bodily illusions and pain syndromes. Pain, 1(3), Blakesle, S., & Ramachandran, V. (1998). Phantoms in the Brain: Probing the Mysteries of the Human Mind: New York, Harper Collins. Botvinick, M. (2004). Probing the neural basis of body ownership. Science, 305(5685), Botvinick, M., & Cohen, J. (1998). Rubber hands' feel'touch that eyes see. Nature, 391(6669), Costantini, M., & Haggard, P. (2007). The rubber hand illusion: sensitivity and reference frame for body ownership. Consciousness and cognition, 16(2), Davies, A. M. A., & White, R. C. (2011). Touching my face with my supernumerary hand: A cheeky illusion. Perception, 40, Doidge, N. (2010). The brain that changes itself: Stories of personal triumph from the frontiers of brain science: Penguin Books. Ehrsson, H. H. (2009). How many arms make a pair? Perceptual illusion of having an additional limb. Perception, 38(2), 310. Ehrsson, H. H., Holmes, N. P., & Passingham, R. E. (2005). Touching a rubber hand: feeling of body ownership is associated with activity in multisensory brain areas. The Journal of Neuroscience, 25(45), Field, A. (2009). Discovering statistics using SPSS: Sage Publications Limited. Guterstam, A., Petkova, V. I., & Ehrsson, H. H. (2011). The illusion of owning a third arm. PloS one, 6(2), e Haans, A., IJsselsteijn, W. A., & de Kort, Y. A. W. (2008). The effect of similarities in skin texture and hand shape on perceived ownership of a fake limb. Body Image, 5(4),

95 Hägni, K., Eng, K., Hepp-Reymond, M. C., Holper, L., Keisker, B., Siekierka, E., & Kiper, D. C. (2008). Observing virtual arms that you imagine were yours increases the galvanic skin response to an unexpected threat. PloS one, 3(8), e3082. Halligan, P. W., Marshall, J. C., & Wade, D. T. (1993). Three arms: a case study of supernumerary phantom limb after right hemisphere stroke. Journal of Neurology, Neurosurgery & Psychiatry, 56(2), Hänsell, A., Lenggenhagerl, B., Känell, R., Curatolol, M., & Blankel, O. (2011). Seeing and identifying with a virtual body decreases pain perception. European Journal of Pain, 15(8), IJsselsteijn, W. A., de Kort, Y. A. W., & Haans, A. (2006). Is this my hand I see before me? The rubber hand illusion in reality, virtual reality, and mixed reality. Presence: Teleoperators and Virtual Environments, 15(4), Khateb, A., Simon, S. R., Dieguez, S., Lazeyras, F., Momjian Mayor, I., Blanke, O.,... Annoni, J. M. (2009). Seeing the phantom: A functional magnetic resonance imaging study of a supernumerary phantom limb. Annals of neurology, 65(6), Krämer, H. H., Seddigh, S., Lorimer Moseley, G., & Birklein, F. (2008). Dysynchiria is not a common feature of neuropathic pain. European Journal of Pain, 12(1), Longo, M. R., & Haggard, P. (2010). An implicit body representation underlying human position sense. Proceedings of the National Academy of Sciences, 107(26), McCabe, C., Haigh, R., Halligan, P., & Blake, D. (2003). Referred sensations in patients with complex regional pain syndrome type 1. Rheumatology, 42(9), Merzenich, M. M., Nelson, R. J., Stryker, M. P., Cynader, M. S., Schoppmann, A., & Zook, J. M. (1984). Somatosensory cortical map changes following digit amputation in adult monkeys. The Journal of comparative neurology, 224(4), Moore, C. I., Stern, C. E., Dunbar, C., Kostyk, S. K., Gehi, A., & Corkin, S. (2000). Referred phantom sensations and cortical reorganization after spinal cord injury in humans. Proceedings of the National Academy of Sciences, 97(26), Newport, R., Pearce, R., & Preston, C. (2010). Fake hands in action: embodiment and control of supernumerary limbs. Experimental brain research, 204(3), Pascual-Leone, A., Amedi, A., Fregni, F., & Merabet, L. B. (2005). The plastic human brain cortex. Annu. Rev. Neurosci., 28,

96 Pascual-Leone, A., Freitas, C., Oberman, L., Horvath, J. C., Halko, M., Eldaief, M.,... Westover, B. (2011). Characterizing Brain Cortical Plasticity and Network Dynamics Across the Age-Span in Health and Disease with TMS-EEG and TMS-fMRI. Brain topography, Pavani, F., Spence, C., & Driver, J. (2000). Visual capture of touch: Out-of-the-body experiences with rubber gloves. Psychological Science, 11(5), Ramachandran, V. S. (1998). Consciousness and body image: lessons from phantom limbs, Capgras syndrome and pain asymbolia. Philosophical Transactions of the Royal Society B: Biological Sciences, 353(1377), Ramachandran, V. S. (2003). The emerging mind: Profile Books London. Ramachandran, V. S. (2005). Plasticity and functional recovery in neurology. Clinical Medicine, Journal of the Royal College of Physicians, 5(4), Ramachandran, V. S., & Altschuler, E. L. (2009). The use of visual feedback, in particular mirror visual feedback, in restoring brain function. Brain, 132(7), Ramachandran, V. S., & Hirstein, W. (1998). The perception of phantom limbs. The DO Hebb lecture. Brain, 121(9), Ramachandran, V. S., & Rogers-Ramachandran, D. (1996). Synaesthesia in phantom limbs induced with mirrors. Proceedings of the Royal Society of London. Series B: Biological Sciences, 263(1369), Ramachandran, V. S., Rogers-Ramachandran, D., & Cobb, S. (1995). Touching the phantom limb. Nature, 377(6549), Ramachandran, V. S., Rogers-Ramachandran, D., Stewart, M., & Pons, T. P. (1992). Perceptual correlates of massive cortical reorganization. SCIENCE-NEW YORK THEN WASHINGTON-, 258, Raz, L., Weiss, P., & Reiner, M. (2008). The virtual hand illusion and body ownership. Haptics: Perception, Devices and Scenarios, Regenbrecht, H., Franz, E., McGregor, G., Dixon, B., & Hoermann, S. (2011a). Beyond the looking glass: Fooling the brain with the Augmented Mirror Box. Presence: Teleoperators and Virtual Environments, 20(6), Regenbrecht, H., Franz, E., McGregor, G., Dixon, B., & Hoermann, S. (2011b). From mirror therapy to augmentation. Regenbrecht, H., McGregor, G., Ott, C., Hoermann, S., Schubert, T., Hale, L., & Franz, E. (2011). Out of reach? A novel AR interface approach for motor rehabilitation. Paper 85

97 presented at the Mixed and Augmented Reality (ISMAR), th IEEE International Symposium. Rorden, C., Heutink, J., Greenfield, E., & Robertson, I. H. (1999). When a rubber hand'feels' what the real hand cannot. Neuroreport, 10(1), Schaefer, M., Heinze, H. J., & Rotte, M. (2009). My third arm: Shifts in topography of the somatosensory homunculus predict feeling of an artificial supernumerary arm. Human brain mapping, 30(5), Slater, M., Perez-Marcos, D., Ehrsson, H. H., & Sanchez-Vives, M. V. (2008). Towards a digital body: the virtual arm illusion. Frontiers in Human Neuroscience, 2. Takasugi, J., Matsuzawa, D., Murayama, T., Nakazawa, K., Numata, K., & Shimizu, E. (2011). Referred sensations induced by a mirror box in healthy subjects. Psychological research, 75(1), Tsakiris, M., & Haggard, P. (2005). The rubber hand illusion revisited: visuotactile integration and self-attribution. Journal of Experimental Psychology: Human Perception and Performance; Journal of Experimental Psychology: Human Perception and Performance, 31(1),

98 Appendix A Order Of Randomisation Information Sheet Consent Form Task Description Prosthetic Hands 87

99 Order Of Randomisation: Experiment 1 Experiment No Session Condition Experiment 1 Experiment 2 Experiment 3 Experiment 4 Experiment 5 Experiment 6 Experiment 7 Experiment 8 Experiment 9 Experiment 10 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Vision ART ART Vision Vision ART ART Vision Vision ART ART Vision Vision ART Vision ART ART Vision ART Vision 88

100 Order Of Randomisation: Experiment 1 Experiment No Session Condition Experiment 11 Experiment 12 Experiment 13 Experiment 14 Experiment 15 Experiment 16 Experiment 17 Experiment 18 Experiment 19 Experiment 20 Experiment 21 Experiment 22 Experiment 23 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 Condition 1 Condition 2 ART Vision Vision ART Vision ART ART Vision Vision ART ART Vision Vision ART ART Vision Vision ART Vision ART Vision ART ART Vision ART Vision 89

101 Rubber Hand Experiment: Investigating Ownership of a Rubber Hand in a Video Mediated Environment Participant Information Thank you for showing interest in this joint project between the departments of Information Science and Psychology. Please read this information sheet carefully before deciding whether or not to participate. If you decide to participate we thank you. If you decide not to participate there will not be any disadvantage to you of any kind and we thank you for considering our request. What is the aim of this project? The aim of this project is to test the reliability and usability of a new Augmented Reality system called Augmented Reflection Technology (ART). Our ART prototype system will project a realistic rubber hand in place of your real hand. The augmented aspect of the technology enables us to investigate perceptions and usability. What will the participant s be asked to do? Should you agree to take part in this experiment, you will first be asked to place your two hands in front of you. One hand will be visible while the other hand will be placed out of sight, behind a black box. A physical representation of a hand will be placed in front of you to represent the hidden hand. Your hands will then be stroked with a paint brush and after 10 minutes, the stroking will stop and you will be asked to answer a questionnaire about your perception of the experiment. In the second part of the task you will be asked to put one hand into a black box with a camera inside and the other hand will be placed outside of the box while a representation of the hand is placed into another black box also containing a camera so you can watch your hands on a computer screen in front of you. Your hands will then be stroked again for 10 minutes. At the end of the 10 minutes, you will be asked to fill in a questionnaire about your perception of the experiment. Can participants change their minds and withdraw from the project? You may withdraw from participation in the project without any disadvantage to yourself of any kind. What data or information will be collected and what use will be made of it? 90

102 The responses to the questionnaires will be recorded. Only the researchers Holger Regenbrecht, Simon Hoermann, Mansoor Alghamdi and Lavell Muller will have access to the data. Results of this project may be published but any data included will in no way be linked to any specific participant (anonymous). What if participants have any questions? If you have any questions about this project, either now or in the future, please feel free to contact Lavell Muller, MSc Candidate University of Otago, New Zealand, Department of Information Science Tel Assoc. Prof. Holger Regenbrecht, Project supervisor University of Otago, New Zealand, Department of Information Science Tel Please keep this sheet if you like. 91

103 Rubber Hand Experiment: Investigating Ownership of A Rubber Hand in A Video Mediated Environment Consent Form for Participants I, (please print your name) I have read the Information Sheet concerning this project and understand what it is about. All my questions have been answered to my satisfaction. I understand that I am free to request further information at any stage. My participation in the project is entirely voluntary I understand that I may withdraw from the experiment at any time without any disadvantage, including the withdrawal of any information I have provided. All data will be destroyed at the conclusion of the project but any raw data on which the results of the project depend may be retained in secure storage for five years, after which it will be destroyed The results of the project may be published and available in the library, but every attempt will be made by the researcher to preserve my anonymity. 92

104 Furthermore: I may / may not (please circle one) be quoted directly If quoted directly, I wish to remain anonymous / use a pseudonym (please circle one) If quoted directly, I hereby grant copyright permission to the researcher for the purpose of publication: yes / no (please circle one) The researcher will confirm my consent for individual quotes On this basis I agree to participate as a subject in this project Signature: Date: 93

105 Rubber Hand Experiment: Investigating Ownership of A Rubber Hand in A Video Mediated Environment Task Description There will be two sessions during the experiment and the order of the sessions will be randomised. For one session you will be asked to place your two hands in front of you. One hand will be visible while the other hand will be placed out of sight, behind a black box. A rubber hand will be placed in front of you to represent the hidden hand. Your hands will then be stroked with a dry paint brush and after 10 minutes, the stroking will stop and you will be asked to answer a questionnaire about your perception of the experiment. For the other session you will be asked to put one hand into a black box with a camera inside and the other hand will be placed outside of the box while a representation of the hand is placed into another black box also containing a camera so you can watch your hands on a computer screen in front of you. Your hands will then be stroked again for 10 minutes. At the end of the 10 minutes, you will be asked to fill in a questionnaire about your perception of the experiment. Please take off any jewellery on your hands and fingers, including a wrist watch and rings and put it besides the two boxes on the table. Sit in front of the boxes, adjust the chair to a comfortable height and position. Look into the black boxes and make sure that these were empty. Ask any questions you want if you don t understand the task, nature or procedure of the experiment. Please feel free to vocalise your thoughts about the task at any time. Thank you Lavell Muller 94

106 Order Of Randomisation: Experiment 2 Participant Number Session One Session Two P1 Condition 1 Condition 2 P2 Condition 1 Condition 2 P3 Condition 1 Condition 2 P4 Condition 1 Condition 2 P5 Condition 1 Condition 2 P6 Condition 1 Condition 2 P7 Condition 2 Condition 1 P8 Condition 1 Condition 2 P9 Condition 2 Condition 1 P10 Condition 2 Condition 1 P11 Condition 1 Condition 2 P12 Condition 2 Condition 1 P13 Condition 1 Condition 2 P14 Condition 2 Condition 1 P15 Condition 2 Condition 1 P16 Condition 2 Condition 1 P17 Condition 1 Condition 2 P18 Condition 2 Condition 1 P19 Condition 2 Condition 1 P20 Condition 2 Condition 1 P21 Condition 1 Condition 2 P22 Condition 1 Condition 2 P23 Condition 2 Condition 1 P24 Condition 1 Condition 2 P25 Condition 1 Condition 2 P26 Condition 2 Condition 1 P27 Condition 1 Condition 2 P28 Condition 1 Condition 2 P29 Condition 2 Condition 1 P30 Condition 2 Condition 1 P31 Condition 1 Condition 2 P32 Condition 2 Condition 1 P33 Condition 1 Condition 2 P34 Condition 1 Condition 2 Legend P = Condition 1 = Condition 2 = Participant athi mthi 95

107 Third Hand Experiment: Investigating Ownership of a Third Hand in an Augmented Environment Participant Information Thank you for showing interest in this joint project between the departments of Information Science and Psychology. Please read this information sheet carefully before deciding whether or not to participate. If you decide to participate we thank you. If you decide not to participate there will not be any disadvantage to you of any kind and we thank you for considering our request. What is the aim of this project? The aim of this project is to test the reliability and usability of a new Augmented Reality system called Augmented Reflection Technology (ART). Our ART prototype system will project a realistic rubber hand in place of your real hand. The augmented aspect of the technology enables us to investigate perceptions and usability. What will the participant s be asked to do? Should you agree to take part in this two part experiment, in one session you will be asked to place your two hands in front of you inside two black boxes. Both hands will be visible on the screen in front of you alongside a physical representation of a third hand in the form of a rubber hand. You will see your hand and the rubber hand being stroked with a paint brush and after 5 minutes, the stroking will stop and you will be asked to answer a questionnaire about your perception of the experiment. In the other session of the experiment you will be asked to place your two hands in front of you inside two black boxes. Both hands will be visible on the screen in front of you alongside a physical representation of a third hand in the form of a rubber hand. Your hand and the rubber hand will then be stroked with a paint brush but you will not see your hand being stroked. After 5 minutes, the stroking will stop and you will be asked to answer a questionnaire about your perception of the experiment. The order of these sessions will be randomised. Can participants change their minds and withdraw from the project? You may withdraw from participation in the project without any disadvantage to yourself of any kind. 96

108 What data or information will be collected and what use will be made of it? The responses to the questionnaires will be recorded. Only the researchers Holger Regenbrecht, Simon Hoermann, Mansoor Alghamdi and Lavell Muller will have access to the data. Results of this project may be published but any data included will in no way be linked to any specific participant (anonymous). What if participants have any questions? If you have any questions about this project, either now or in the future, please feel free to contact Lavell Muller, MSc Candidate University of Otago, New Zealand, Department of Information Science Tel Simon Hoermann, PhD Candidate University of Otago, New Zealand, Department of Information Science Tel Assoc. Prof. Holger Regenbrecht, Project supervisor University of Otago, New Zealand, Department of Information Science holger@infoscience.otago.ac.nz Tel Please keep this sheet if you like. 97

109 Third Hand Experiment: Investigating Ownership of a Third Hand in a Video Mediated Environment Consent Form For Participants I, (please print your name) I have read the Information Sheet concerning this project and understand what it is about. All my questions have been answered to my satisfaction. I understand that I am free to request further information at any stage. My participation in the project is entirely voluntary I understand that I may withdraw from the experiment at any time without any disadvantage, including the withdrawal of any information I have provided. All data will be destroyed at the conclusion of the project but any raw data on which the results of the project depend may be retained in secure storage for five years, after which it will be destroyed The results of the project may be published and available in the library, but every attempt will be made by the researcher to preserve my anonymity. 98

110 Furthermore: I may / may not (please circle one) be quoted directly If quoted directly, I wish to remain anonymous / use a pseudonym (please circle one) If quoted directly, I hereby grant copyright permission to the researcher for the purpose of publication: yes / no (please circle one) The researcher will confirm my consent for individual quotes On this basis I agree to participate as a subject in this project Signature: Date: 99

111 Third Hand Experiment: Investigating Ownership of a Third Hand in an Augmented Environment Task Description There will be two sessions during the experiment and the order of the sessions will be randomised. Should you agree to take part in this two part experiment, in one session you will be asked to place your two hands in front of you inside two black boxes. Both hands will be visible on the screen in front of you alongside a physical representation of a third hand in the form of a rubber hand. You will see your hand and the rubber hand being stroked with a paint brush and after 5 minutes, the stroking will stop and you will be asked to answer a questionnaire about your perception of the experiment. In the other session of the experiment you will be asked to place your two hands in front of you inside two black boxes. Both hands will be visible on the screen in front of you alongside a physical representation of a third hand in the form of a rubber hand. Your hand and the rubber hand will then be stroked with a paint brush but you will not see your hand being stroked. After 5 minutes, the stroking will stop and you will be asked to answer a questionnaire about your perception of the experiment. The order of these sessions will be randomised. Please take off any jewellery on your hands and fingers, including a wrist watch and rings and put it besides the two boxes on the table. Sit in front of the boxes; adjust the chair to a comfortable height and position. Look into the black boxes and make sure that these were empty. Ask any questions you want if you don t understand the task, nature or procedure of the experiment. Please feel free to vocalise your thoughts about the task at any time. Thank you Lavell Muller 100

112 101

113 Appendix B Demographic Survey Post Condition (Perceived Ownership) Questionnaire Post Study Questionnaire 102

114 Participant #: Date/time: Condition Sequence: Rubber Hand Experiment: Investigating Ownership of A Rubber Hand in A Video Mediated Environment Participant Demographic Survey 1. How old were you? years Rather not say 2. What is your gender? Female Male 103

115 3. Which is your dominant hand? Left Hand Right Hand Left or Right Hand depending on task (Cross-dominance or Mixed-handedness) Left and Right hand equally for all tasks (ambidextrous) Neither - low level of dexterity (ambilevous or ambisinister) 4. Have you participated in a similar experiment like this before? No No, but know what it is all about. Yes. 104

116 5. Do you have normal or corrected to normal (e.g. glasses, contact lenses) vision? Yes No If No please specify issue: Rather not say 6. Do you have a disability of the hand, arm, shoulder, neck, back or other that could affect this experiment? No Yes If Yes please specify: Rather not say 105

117 Participant #: Date/time: Condition #: Rubber Hand Experiment: Investigating Ownership of A Rubber Hand in A Video Mediated Environment Post-Condition Questionnaire Please read each statement and indicate how strongly you agree or disagree with the statement by circling a number on the scale. Please write comments to elaborate on your answers if need be. 1. It seemed as if I were feeling the touch of the paintbrush in the location where I saw the rubber hand touched: Strongly Disagree Strongly Agree Comments: 106

118 2. It seemed as though the touch I felt was caused by the paintbrush touching the rubber hand: Strongly Disagree Strongly Agree Comments: 3. I felt as if the rubber hand were my hand: Strongly Disagree Strongly Agree Comments: 4. It felt as if my (real) hand were drifting towards the right (towards the rubber hand): Strongly Disagree Strongly Agree Comments: 107

119 5. It seemed as if I might have more than one left hand or arm: Strongly Disagree Strongly Agree Comments: 6. It seemed as if the touch I was feeling came from somewhere between my own hand and the rubber hand: Strongly Disagree Strongly Agree Comments: 7. It felt as if my (real) hand were turning rubbery : Strongly Disagree Strongly Agree Comments: 108

120 8. It appeared (visually) as if the rubber hand were drifting towards the left (towards my hand): Strongly Disagree Strongly Agree Comments: 9. The rubber hand began to resemble my own (real) hand, in terms of shape, skin tone, freckles or some other visual feature: Strongly Disagree Strongly Agree Comments: 109

121 Participant #: Date/time: Rubber Hand Experiment: Investigating Ownership of a Rubber Hand in a Video Mediated Environment Post-Study Questionnaire Please read each statement and indicate how strongly you agree or disagree with the statement by circling a number on the scale. Please write comments to elaborate on your answers if need be. 1. I felt bored: Strongly Disagree Strongly Agree Comments: 2. I felt sleepy: Strongly Disagree Strongly Agree Comments: 3. I felt uninterested: Strongly Disagree Strongly Agree Comments: 110

122 Comments on study: Comments on system (hardware, software, experiment ): 111

123 Participant #: Date/time: Condition Sequence: Third Hand Experiment: Investigating Ownership of a Third Hand in an Augmented Environment Participant Demographic Survey 1. How old were you? years Rather not say 2. What is your gender? Female Male 112

124 3. Which is your dominant hand? Left Hand Right Hand Left or Right Hand depending on task (Cross-dominance or Mixed-handedness) Left and Right hand equally for all tasks (ambidextrous) Neither - low level of dexterity (ambilevous or ambisinister) 4. Have you participated in a similar experiment like this before? No No, but know what it is all about. Yes. 113

125 5. Do you have normal or corrected to normal (e.g. glasses, contact lenses) vision? Yes No If No please specify issue: Rather not say 6. Do you have a disability of the hand, arm, shoulder, neck, back or other that could affect this experiment? No Yes If Yes please specify: Rather not say 114

126 Participant #: Date/time: Condition #: Third Hand Experiment: Investigating Ownership of a Third Hand in an Augmented Environment Post-Condition Questionnaire Please read each statement and indicate how strongly you agree or disagree with the statement by circling a number on the scale. Please write comments to elaborate on your answers if need be. During the experiment there were times when I experienced that: 1. I felt the touch of the brush on the rubber hand: Strongly Disagree Strongly Agree Comments: 115

127 2. I felt as if the rubber hand was my hand: Strongly Disagree Strongly Agree Comments: 3. I felt the touch of the brush on my (real) hand: Strongly Disagree Strongly Agree Comments: 4. I could no longer feel that my (real) hand belonged to my body Strongly Disagree Strongly Agree Comments: 116

128 5. I felt the touch of the brush on both hands, but never at the same time: Strongly Disagree Strongly Agree Comments: During the experiment there were times when I experienced that: 6. I could feel the touch of the brush on both hands at the same time: Strongly Disagree Strongly Agree Comments: 7. It felt as if I had two left hands: Strongly Disagree Strongly Agree Comments: 117

129 8. It felt as if my (real) hand were turning rubbery Strongly Disagree Strongly Agree Comments: 9. It seemed as if the touch I was feeling came from somewhere between my own hand and the rubber hand: Strongly Disagree Strongly Agree Comments: 10. It appeared (visually) as if the rubber hand was drifting towards the left (towards my real hand): Strongly Disagree Strongly Agree Comments: 118

130 Participant #: Date/time: Third Hand Experiment: Investigating Ownership of a Third Hand in an Augmented Environment Post-Study Questionnaire Please read each statement and indicate how strongly you agree or disagree with the statement by circling a number on the scale. Please write comments to elaborate on your answers if need be. 1. I felt bored: Strongly Disagree Strongly Agree Comments: 2. I felt sleepy: Strongly Disagree Strongly Agree Comments: 3. I felt uninterested: Strongly Disagree Strongly Agree Comments: 119

131 Comments on study: Comments on system (hardware, software, experiment ): 120

132 Appendix C Excel Data Normal Distribution Statistics Mean Comparison Statistics Perceived Ownership Statistics 121

133 RHI vs. vrhi Demographics Exp. No 1 Exp. No 2 Exp. No 3 Exp. No 4 Exp. No 5 Exp. No 6 Exp. No 7 Exp. No 8 Exp. No 9 Experiment Order L L M S L L S L M S M L L M S L S L M S S M S (M) (M) M S Age Gender M M F F M M M M M Exp. Knowledge no no no yes no no no no no Exp. No 10 Exp. No 11 Exp. No 12 Exp. No 13 Exp. No 14 Exp. No 15 Exp. No 16 Exp. No 17 Exp. No 18 Experiment Order S S L M S L M L S M M S L M S S M L L L M S L M L S M Age Gender M F F M M M M F F Exp. Knowledge no no yes no no no no no no Exp. No 19 Exp. No 20 Exp. No 21 Exp. No 22 Exp. No 23 Exp. No 24 Exp. No 25 Exp. No 26 Experiment Order M L M S M M S L S M S M S S L S L S (L) (L) L (L) M M Age Gender F F M M F M F M Exp. Knowledge no no no no yes no no no Legend: L Lavell's Experiment Mean Age SD. In Age M Mansoor's Experiment S Simon's Experiment Tot. Mal Tot. Fem ( ) Did Not Participate 13 10

134 RHI vs. vrhi Perceived Ownership Participant Session Condition Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 P1 S1 Vision P1 S2 ART P2 S1 ART P2 S2 Vision P3 S1 Vision P3 S2 ART P4 S1 ART P4 S2 Vision P5 S1 Vision P5 S2 ART P6 S1 ART P6 S2 Vision P7 S1 Vision P7 S2 ART P8 S1 Vision P8 S2 ART P9 S1 ART P9 S2 Vision P10 S1 ART P10 S2 Vision P11 S1 ART P11 S2 Vision P12 S1 Vision P12 S2 ART

135 RHI vs. vrhi Perceived Ownership Participant Session Condition Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 P13 S1 Vision P13 S2 ART P14 S1 ART P14 S2 Vision P15 S1 Vision P15 S2 ART P16 S1 ART P16 S2 Vision P17 S1 Vision P17 S2 ART P18 S1 ART P18 S2 Vision P19 S1 Vision P19 S2 ART P20 S1 Vision P20 S2 ART P21 S1 Vision P21 S2 ART P22 S1 ART P22 S2 Vision P23 S1 ART P23 S2 Vision

136 RHI Perceived Ownership Means Statements N Mean Std. Deviation Std. Error Mean V V V V V V V V V RHI Condition Statements 2 1 V1 V2 V3 V4 V5 V6 V7 V8 V9 125

137 arhi Perceived Ownership Means Statement N Mean Std. Deviation Std. Error Mean A A A A A A A A A arhi Condition Statement 2 1 A1 A2 A3 A4 A5 A6 A7 A8 A9 126

138 RHI vs. vrhi Post Study Data Participant # in Series I felt bored I felt sleepy I felt uninterested P P P P P P P P P P P P P P P P P P P P P P All Participants Excluding Pilot RHI vs. vrhi Post Study Bar Graph 7 Post Study Questionnaire Average of I felt sleepy Average of I felt bored Average of I felt uninterested 127

139 RHI vs. vrhi Normal Distribution Tests of Normality Kolmogorov-Smirnov a Shapiro-Wilk Statistic df Sig. Statistic df Sig. mean_for_rhi mean_for_vrhi * * * This is a lower bound of the true significance. a Lilliefors Significance Correction 128

140 Mean Box Plot for RHI Condition 129

141 130

142 Mean Box Plot for vrhi Condition 131

143 RHI vs. vrhi Normal Distribution: Descriptive Statistics Statistic Std. Error mean_for_rhi Mean % Confidence Interval for Mean Lower Bound Upper Bound % Trimmed Mean Median Variance.776 Std. Deviation Minimum 2.33 Maximum 5.78 Range 3.44 Interquartile Range.92 Skewness Kurtosis

144 RHI vs. vrhi Normal Distribution: Descriptive Statistics Statistic Std. Error mean_for_vrhi Mean % Confidence Interval for Mean Lower Bound Upper Bound % Trimmed Mean Median Variance Std. Deviation Minimum 2.00 Maximum 6.00 Range 4.00 Interquartile Range 1.56 Skewness Kurtosis

145 134

146 135

147 RHI vs. vrhi Mean Comparison Statistics: Descriptive Statistics Descriptive Statistics Mean Std. Deviation N RHI vrhi Mauchly's Test of Sphericity a Measure: MEASURE_1 Within Subjects Mauchly's W Approx. Chi- df Sig. Epsilon b Effect Square Greenhouse- Huynh- Lower- Geisser Feldt bound ConditionComp Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables is proportional to an identity matrix. a. Design: Intercept Within Subjects Design: ConditionComp b. May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests were displayed in the Tests of Within-Subjects Effects table. 136

148 Tests of Within-Subjects Effects Measure: MEASURE_1 Source Type III Sum of Squares df Mean Square F Sig. Sphericity Assumed ConditionComp Greenhouse-Geisser Huynh-Feldt Lower-bound Sphericity Assumed Error(ConditionComp) Greenhouse-Geisser Huynh-Feldt Lower-bound Pairwise Comparisons Measure: MEASURE_1 (I) ConditionComp (J) ConditionComp Mean Difference (I- J) Std. Error Sig. a 95% Confidence Interval for Difference a Lower Bound Upper Bound RHI vrhi vrhi RHI Based on estimated marginal means a. Adjustment for multiple comparisons: Bonferroni. 137

149 RHI t-test One-Sample Statistics N Mean Std. Deviation Std. Error Mean V V V V V V V V V

150 One-Sample Test Test Value = 4 t df Sig. (2-tailed) Mean Difference 95% Confidence Interval of the Difference Lower Upper V V V V V V V V V

151 vrhi t-test One-Sample Statistics N Mean Std. Deviation Std. Error Mean A A A A A A A A A

152 One-Sample Test Test Value = 4 t df Sig. (2-tailed) Mean Difference 95% Confidence Interval of the Difference Lower Upper A A A A A A A A A

153 Demographics - Third Hand Illusion Exp. No 1 Exp. No 2 Exp. No 3 Exp. No 4 Exp. No 5 Exp. No 6 Exp. No 7 Exp. No 8 Exp. No 9 Age n/a 26 Gender F F M F F M F F F Exp. Knowledge No No No No No, but No No, but No No Exp. No 10 Exp. No 11 Exp. No 12 Exp. No 13 Exp. No 14 Exp. No 15 Exp. No 16 Exp. No 17 Exp. No 18 Age Gender F F F F M F F M F Exp. Knowledge No No No, but No No Yes No No No Exp. No 19 Exp. No 20 Exp. No 21 Exp. No 22 Exp. No 23 Exp. No 24 Exp. No 25 Exp. No 26 Exp. No 27 Age Gender F F M F F M M M F Exp. Knowledge No No No No No No No No No Exp. No 28 Exp. No 29 Exp. No 30 Exp. No 31 Exp. No 32 Exp. No 33 Exp. No 34 Age Gender M M M M M M M Exp. Knowledge Yes No No No No No, but No Mean Age SD. In Age Tot. Mal Tot. Fem

154 athi vs. mthi Perceived Ownership Participant Condition Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Condition Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 1 c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c

155 athi vs. mthi Perceived Ownership Participant Condition Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Condition Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 16 c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c c

156 athi Perceived Ownership Means Statements N Mean Std. Deviation Std. Error Mean athi_q athi_q athi_q athi_q athi_q athi_q athi_q athi_q athi_q athi_q athi Condition Statements 145

157 mthi Perceived Ownership Means Statement N Mean Std. Deviation Std. Error Mean mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi_q mthi Condition Statement 146

158 RHI vs. vrhi Post Study Data Participant I felt I felt I felt bored sleepy uninterested P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P All Participants Excluding Pilot 147

159 athi vs. mthi Post Study Bar Graph 7 Post Study Questionnaire Average of I felt sleepy Average of I felt bored Average of I felt uninterested 148

160 athi vs. mthi Normal Distribution Tests of Normality Kolmogorov-Smirnov a Shapiro-Wilk Statistic df Sig. Statistic df Sig. athimean mthimean * * This is a lower bound of the true significance. a Lilliefors Significance Correction 149

161 Mean Box Plot for athi Condition 150

162 151

163 Mean Box Plot for mthi Condition 152

164 athi vs. mthi Normal Distribution: Descriptive Statistics Descriptives Statistic Std. Error Mean % Confidence Interval for Mean Lower Bound Upper Bound % Trimmed Mean Median Variance athimean Std. Deviation Minimum 1.60 Maximum 5.20 Range 3.60 Interquartile Range 2.13 Skewness Kurtosis

165 Mean % Confidence Interval for Mean Lower Bound Upper Bound % Trimmed Mean Median Variance mthimean Std. Deviation Minimum 1.60 Maximum 4.90 Range 3.30 Interquartile Range 1.75 Skewness Kurtosis

166 155

167 156