Typically, usability testing involves a testing laboratory, video cameras, and software to record actions. The user is placed in a foreign environment. It has been recently suggested that this isolation and sterilization of the user-environment may not be producing natural test results. In her interview with Dr. Lawrence Shapiro on the Brain Science Podcast website (www.brainsciencepodcast.com), Dr. Beverly Campbell explains that, "Embodied cognition is an approach to the study of cognition that regards cognition as something that involves, not just the brain, but also the body and its environment."
Standard cognition would explain the brain as more of a computer, processing stimulae as it comes in from the senses. Shapiro says, "Embodied cognition, in contrast, imagines not that the brain can be isolated from the body and the environment, but thinks of the body as in some sense shaping, or constraining, or involved in the very processing of the kinds of information that an organism needs to interact successfully with the world. So, it’s no longer possible, according to people in embodied cognition, to think about cognition as being the middle stage in the sense-think-act cycle. Instead, thinking involves active exploration—use of the body with things in the environment."
I decided to try this theory by doing my next two usability tests not only on-site, but literally at the desks of the users who typically use the software. I was presenting a beta version of the software to them, a clickable prototype, actually. But I ran into a couple problems with this scenario. The first problem was attempting to get the software to the user's machine. In other words, preparation is difficult unless you take over the user's machine, which is probably not appropriate. The next issue was the noise factor. The test is loud and the heavy users of our products are not the execs that have offices. We bothered the coworkers and they were good sports, but this was an inescapable distraction. Observation by the product owner was done over remote software and this also had to be setup on the user's machine. After all this preparation and distraction, it's a wonder they didn't toss me out on my butt. They were cordial, though, and I learned quite a bit. The key might be to try to setup a test environment that is "like" a user environment or at least to streamline the setup process.
But Shapiro says, "the last thing you want to do is place the subject in a kind of artificial environment where you’re removing all the kinds of what cognitive scientists think of as distractions, but what we embodied cognition people think of as necessary props in the environment that the subject uses to collect or simplify the sorts of information that the subject needs to be performing his or her cognitive tasks." So, what to do? I tend to believe the theory of embodied cognition but it puts us in a testing conundrum. I like the idea of testing in a user's own habitat so I'm going to continue working toward that. I think it can be done effectively with the right tools and process.
BRAIN SCIENCE PODCAST With Ginger Campbell, MD
Episode #73
Interview with Lawrence Shapiro, PhD, Professor of Philosophy and
Author of Embodied Cognition
http://www.brainsciencepodcast.com/storage/transcripts/year-5/73-bsp-Shapiro.pdf
This blog is for the HCI 596 course being taught at Iowa State University through it's Human-Computer Interaction program.
One way of looking at it is, what do you hope to learn from the study? If your study isn't one that addresses users' actual work tasks and is more of an evaluation of discoverability of features, etc. then the absence of the natural work environment may not be a problem. On the other hand, if the tasks you are evaluating are highly dependent upon the user's cognitive faculties for a task they would do at work, then evaluating _in situ_ would be ideal. In such a case, you would want to minimize the amount of things you are importing into the users environment (e.g. instead of the stakeholder viewing the live footage, send them a video of it). The application should also, ideally, be installed where the user would install it if they had acquired it through normal channels (and even more ideally) you wouldn't be present yourself--you would perform a remote capture of data and talk through the footage later.
ReplyDeleteOther options include "replicating" the user's work environment from photos within the lab. You also have the option of using the technique I described in class, where you have a user walk through a task and showing pictures of all of the objects they have access to, ask them what would happen if that object was not available (finding out which objects are critical for the task and which aren't--this is more done on the design side than the evaluation side... but if you have that information, you can take it into account when setting up a testing environment). Of course, there's also the idea that the work may not be done individually--the resource they use to augment their cognition may be other employees.... but, again, it all comes down to what your research question is and what you would gain from each method.
These are very good points. I got a little caught up in the minutia here and my users would probably be just fine in the lab since I'm really making sure I'm presenting them with what they need and that it's usable. A mobile test is much different I'm just about to embark on that one.
ReplyDelete