Feedback on Touch Screen User Interfaces. Visual Auditory Haptic

Transcription

1 Feedback on Touch Screen User Interfaces Visual Auditory Haptic

2 Mobili-T Case Study The goal of the Mobili-T project is to design a mobile swallowing therapy device for patients with swallowing impairments. A simplified form of the therapy consists of multiple swallows and a swallow that is held for a period of time. The ideal is to reduce the complexity of the necessary equipment to a portable device such as a touch screen tablet and the adhesive pad placed on the chin for the swallowing muscle measurements.

3 Visual Feedback Visual feedback is anything that would be seen on the screen of the device that occurs when the user interacts with the touch screen.

4 Auditory Feedback Auditory feedback is the sound emitted by the device when the user interacts with the device.

5 Haptic Feedback Haptic feedback is about touch. For the most part, it is simple vibrations made by the device.

6 Multi-Modal Feedback Any combination of the three main types of feedback.

7 Multi-Modal Feedback Four combinations were studied. Visual Visual + Auditory Visual + Haptic Visual + Auditory + Haptic

8 Study The task consists of using a hand grip and doing two types of exercises: 1) The first exercise is holding the hand grip closed for a length of time while observing the feedback. 2) The second exercise is opening and closing the hand grip a number of times and observing the feedback.

9 Study Grip and Hold Participant hold hand grip closed until the circle on the screen touches the edge of the screen. The following slides are an example of the simulated Visual and Auditory Feedback received by participants

10 Visual + Auditory Grip and Hold

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30 Thank you! Please hand the device back to the PI

31 Study Grip Strength Participant opens and closes the hand grip until all the red circles on the screen become green. The following slides are an example of the simulated Visual and Auditory Feedback received by participants

32 Visual + Auditory Grip Strength

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47 Thank you! Please hand the device back to the PI

48 Design Solutions Reach the Target Goal: To reach the target at the far end of the screen. Purpose: Increase the concentration of the patient by focusing on the movement of the item on the screen. Swallow Hold: As patient holds swallow, the small circle moves left to right toward the target. Previous Attempts: The faded circles with numbers indicate previous swallow hold attempts in this session.

49 1 2

50 1 2

51 1 2

52 1 2

53 1 2

54 1 2

55 1 2

56 1 2

57 1 2

58 1 2

59 1 2

60 1 2

61 1 2

62 1 2

63 1 2

64 1 2

65 1 2

66 1 2

67 1 2

68 1 2

69 1 2

70 1 2

71 1 2

72 1 2

73

74 Design Solutions Reach the Target Goal: To reach the target at the far end of the screen. Purpose: Increase the concentration of the patient by focusing on the movement of the item on the screen. Multiple Swallows: Each swallow moves the circle to the intermediate goals before the final target.

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105 Design Solutions Fill the Circle Goal: To fill the circle on the screen. Purpose: Patient to want to complete the therapy session in order to fill up the image on the screen. Swallow Hold: During a swallow hold, the circle is filled in a continuous clock wise direction. Previous Attempts: The faded circles and numbers indicate previous swallow hold attempts in this session.

106 1 2

107 1 2

108 1 2

109 1 2

110 1 2

111 1 2

112 1 2

113 1 2

114 1 2

115 1 2

116 1 2

117 1 2

118 1 2

119

120

121

122

123

124

125 Design Solutions Fill the Circle Goal: To fill the circle on the screen. Purpose: Patient to want to complete the therapy session in order to fill up the image on the screen. Multiple Swallows: At each swallow, the circle will fill up in sections.

126

127

128

129

130

131

132

133

134 Design Solutions Sweep the Screen Goal: Change the colour of the entire screen by sweeping a vertical bar across the screen. Purpose: Maximize the concentration of the patient and to want to complete the therapy session. Swallow Hold: As the swallow is held, the vertical bar moves left to right changing the colour of the screen. Previous Attempts: The faded circles with numbers indicate previous swallow hold attempts in this session.

135 1 2

136 1 2

137 1 2

138 1 2

139 1 2

140 1 2

141 1 2

142 1 2

143 1 2

144

145 Design Solutions Sweep the Screen Goal: Change the colour of the entire screen by sweeping a vertical bar across the screen. Purpose: Maximize the concentration of the patient and to want to complete the therapy session. Multiple Swallows: At each individual swallow, each section of the screen will change colour.

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160 Future Research Currently, the design solutions are simple and very basic. Complexity may be added to the solutions make them more game-like and engaging for the patients. Reach the Target: Basketball moving across a basketball court or beach ball bouncing along a beach. Fill the Circle: The circle becomes a clock face or a pizza. Sweep the Screen: An image of a landscape or friends and family is revealed. Or each swallow reveals a panel of a comic.

161 Feedback on Touch Screen User Interfaces Thank You