Questionnaire Design with an HCI focus from A. Ant Ozok Chapter 58 Georgia Gwinnett College School of Science and Technology Dr. Jim Rowan
Surveys! economical way to collect large amounts of data for comparison economical in terms of expense economical in terms of time time to create time to execute time to evaluate cost to evaluate
Surveys! criticized are the scales used interpreted in the same way from subject to subject do subjects interpret I strongly agree and I often do in the same way?
Surveys! they serve HCI well in terms of eliciting qualitative and quantitative user responses concerning user satisfaction user preferences user opinions
Surveys! categories of information gathered user evaluation how does an existing product meet user expectations? user opinion what do users think might be useful in the future? other population demographics frequency of use
Survey implementation! sample size large enough? representative of the underlying population?! two forms discussed: paper and pencil electronic
Paper and Pencil Survey Examples
Survey implementation! electronic stand alone computer? greater control over participants greater control over submissions web-based? greater audience but what is the population? is it representative? how do you prevent one subject taking the survey multiple times? does it matter?
Survey: Web Based! three forms survey provider (e.g. survey monkey) can have restricted access so that you can only take the survey once google forms more free form less control over access use of one s own hardware and software
Survey: Web Based! long surveys result in decreased response rates! limit to 20 questions per screen (seems way too many to me!)
HCI survey types! Opinion surveys! Evaluation surveys! Other survey types
HCI survey types: Opinion surveys! What do users think in general?! Not centered on a particular product! Can be both quantitative and qualitative
HCI survey types: Opinion surveys! Fall into 3 categories Opinions on a medium Opinions on an event Opinions on a procedure
HCI survey types: Opinion surveys! Fall into 3 categories Opinions on a medium interface design user satisfaction usability user preferences examples: are you generally satisfied with web access on your cell phone? what additional capabilities would you like to see?
HCI survey types: Opinions on a medium
HCI survey types: Opinion surveys! Fall into 3 categories Opinions on an event (within HCI) Occur more rarely Examples: Do you find interface gestures more useful than point and click? Do you find command based interfaces easier to use than GUIs?
HCI survey types: Opinions on an event
HCI survey types: Opinion surveys! Fall into 3 categories Opinions on a procedure user opinions of an HCI task not about user opinions of a particular medium itself but more about user opinions of how the medium is used Example: Are you satisfied about how long it takes to purchase a product on a particular site?
HCI survey types: Opinions on a procedure
HCI survey types: Evaluation surveys! unlike opinion surveys, evaluation surveys are usually administered right after participants have completed a procedure! used to evaluate system usability user satisfaction user preference
! done to find HCI survey types: Evaluation surveys the exact nature and location of problems points open to improvement! questions like was the site difficult to navigate? did you enjoy the overall experience? would you visit this web-site again?
HCI survey types: Evaluation surveys! can be qualitative or quantitative! quantitative can give statistical backing to user evaluations! qualitative!provide new ideas to improve usability/user-preference-related design issues!
HCI survey types: Evaluation surveys
HCI survey types: Other survey type: Demographics! demographics don t usually stand on their own are essential parts of opinion and evaluation surveys determine how generalizable the results are
HCI survey types: Demographics! age! sex! education! occupation! school class (sophomore?)! major
HCI survey types: Demographics: Challenges! Data is much more personal in nature! Can raise privacy issues! Providing a sense of privacy (through informed consent) can improve participant enthusiasm
HCI survey types: Demographics: Example
Survey application domains! Sales and Marketing! Medicine! Education! Information Technology Research
Survey design methodology:! Survey design techniques! Survey evaluation techniques
Survey design methodology: Survey design techniques! Survey design techniques issues of content issues of the scale used! Scale use: required for statistical evaluation of opinions it converts qualitative information into quantitative without a scale Do you enjoy using this app? becomes a qualitative question
Survey design methodology: Survey design techniques! Not using a scale? Then it is qualitative! No-scale options: open ended questions radio buttons Don t forget to add Other (please specify)
Survey design methodology: Survey design techniques! some measure amounts! some measure frequency! some ask yes or no questions! don t forget:! not applicable! not available
Survey design methodology:! Survey evaluation techniques run a pilot this checks out the process before you waste time and subjects on faulty questions involves issues of: survey reliability survey reliability
Example satisfaction survey
Example satisfaction survey