While working on project 1, I had a few questions about how to to complete usability studies. I sent an email asking/commenting the following:
"In your usability studies, how do you get the subjects to take the interview, survey, etc as serious as you do? Most people who take surveys fill it out without much thought. One time I created a survey to determine if a particular tool was useful; it seemed as if the participants just went through the motions, especially if the questions only required them to select from a list (i.e. agree..strongly agree). Is there a certain method of questioning or interviewing that is more effective for getting honest answers or well thought out answers?"
The response to my email was to do the following:
1. Build rapport with the people you are interviewing. Make sure they feel like their opinions are valued and put them at ease so they feel they can be completely honest.
2. Wording of questions is critical. Make sure you ask really open ended questions without leading.
I did some browsing around also and what I found in a document that someone posted on the web was helpful in providing more insight as well as confirming the email response. The author of the document posted on the web gave some do's and don'ts of usuability study design. For example, give the participant an options to end the study if they feel uncomfortable. Make sure the user understand that there is no right or wrong answer. Also, have them sign a form explaining that he results are anonymous; this helps them feel that they can be honest in their answers. The author also gave techniques on what to do during the study as far as what to watch for. For example, be sure to note non-verbal communication. Some advice on the types of questions were to give open ended questions as well as task oriented questions.
The author also gave pointers on when to perform usability studies, the best person to conduct the study, etc.
http://www.highedweb.org/2007/presentations/PST6.pdf
This blog is for the HCI 596 course being taught at Iowa State University through it's Human-Computer Interaction program.
I was luck that I took an “evaluation” course last spring that helped me tremendously creating evaluations and usability study. I am not sure if I could have done well in those areas without taking this course.
ReplyDeleteThe course taught me how to create effective surveys that people would take and provide valuable answers. We also learned to evaluate evaluations too. You talked about how participants just took survey because of making closed-ended questions (Likert-scale). I totally understand what you mean. What I learned in the course is to always mix question types. Closed-ended and open-ended question type should be used to mix up the survey. It forces participants to reflect back to what they just answered. In survey results, you get a good balance of open/closed-ended results that will help you determine what users are thinking. We also learned how to create interview questions and conduct interviews. We also learned to create good usability test too so that results I corrected were valid.
I am sure we can find more resources about how to conduct interview, survey, and usability study.
You can check it out the books we used.
Books:
Jeffrey Rubin and Dana Chisnell, “Handbook of Usability Testing”
Jody Fitzpatrick, “Program Evaluation”