Questionnaire Design with an HCI focus

Transcription

1 Questionnaire Design with an HCI focus from A. Ant Ozok Chapter 58 Georgia Gwinnett College School of Science and Technology Dr. Jim Rowan

2 Surveys! economical way to collect large amounts of data for comparison economical in terms of expense economical in terms of time time to create time to execute time to evaluate cost to evaluate

3 Surveys! criticized are the scales used interpreted in the same way from subject to subject do subjects interpret I strongly agree and I often do in the same way?

4 Surveys! they serve HCI well in terms of eliciting qualitative and quantitative user responses concerning user satisfaction user preferences user opinions

5 Surveys! categories of information gathered user evaluation how does an existing product meet user expectations? user opinion what do users think might be useful in the future? other population demographics frequency of use

6 Survey implementation! sample size large enough? representative of the underlying population?! two forms discussed: paper and pencil electronic

7 Paper and Pencil Survey Examples

8 Survey implementation! electronic stand alone computer? greater control over participants greater control over submissions web-based? greater audience but what is the population? is it representative? how do you prevent one subject taking the survey multiple times? does it matter?

9 Survey: Web Based! three forms survey provider (e.g. survey monkey) can have restricted access so that you can only take the survey once google forms more free form less control over access use of one s own hardware and software

10 Survey: Web Based! long surveys result in decreased response rates! limit to 20 questions per screen (seems way too many to me!)

11 HCI survey types! Opinion surveys! Evaluation surveys! Other survey types

12 HCI survey types: Opinion surveys! What do users think in general?! Not centered on a particular product! Can be both quantitative and qualitative

13 HCI survey types: Opinion surveys! Fall into 3 categories Opinions on a medium Opinions on an event Opinions on a procedure

14 HCI survey types: Opinion surveys! Fall into 3 categories Opinions on a medium interface design user satisfaction usability user preferences examples: are you generally satisfied with web access on your cell phone? what additional capabilities would you like to see?

15 HCI survey types: Opinions on a medium

16 HCI survey types: Opinion surveys! Fall into 3 categories Opinions on an event (within HCI) Occur more rarely Examples: Do you find interface gestures more useful than point and click? Do you find command based interfaces easier to use than GUIs?

17 HCI survey types: Opinions on an event

18 HCI survey types: Opinion surveys! Fall into 3 categories Opinions on a procedure user opinions of an HCI task not about user opinions of a particular medium itself but more about user opinions of how the medium is used Example: Are you satisfied about how long it takes to purchase a product on a particular site?

19 HCI survey types: Opinions on a procedure

20 HCI survey types: Evaluation surveys! unlike opinion surveys, evaluation surveys are usually administered right after participants have completed a procedure! used to evaluate system usability user satisfaction user preference

21 ! done to find HCI survey types: Evaluation surveys the exact nature and location of problems points open to improvement! questions like was the site difficult to navigate? did you enjoy the overall experience? would you visit this web-site again?

22 HCI survey types: Evaluation surveys! can be qualitative or quantitative! quantitative can give statistical backing to user evaluations! qualitative!provide new ideas to improve usability/user-preference-related design issues!

23 HCI survey types: Evaluation surveys

24 HCI survey types: Other survey type: Demographics! demographics don t usually stand on their own are essential parts of opinion and evaluation surveys determine how generalizable the results are

25 HCI survey types: Demographics! age! sex! education! occupation! school class (sophomore?)! major

26 HCI survey types: Demographics: Challenges! Data is much more personal in nature! Can raise privacy issues! Providing a sense of privacy (through informed consent) can improve participant enthusiasm

27 HCI survey types: Demographics: Example

28 Survey application domains! Sales and Marketing! Medicine! Education! Information Technology Research

29 Survey design methodology:! Survey design techniques! Survey evaluation techniques

30 Survey design methodology: Survey design techniques! Survey design techniques issues of content issues of the scale used! Scale use: required for statistical evaluation of opinions it converts qualitative information into quantitative without a scale Do you enjoy using this app? becomes a qualitative question

31 Survey design methodology: Survey design techniques! Not using a scale? Then it is qualitative! No-scale options: open ended questions radio buttons Don t forget to add Other (please specify)

32 Survey design methodology: Survey design techniques! some measure amounts! some measure frequency! some ask yes or no questions! don t forget:! not applicable! not available

33 Survey design methodology:! Survey evaluation techniques run a pilot this checks out the process before you waste time and subjects on faulty questions involves issues of: survey reliability survey reliability

34 Example satisfaction survey

35 Example satisfaction survey

36