Methods: Think Alouds
2015-04-30
Think-aloud protocols involve participants thinking aloud as they are performing a set of specified tasks. Users are asked to say whatever they are looking at, thinking, doing, and feeling as they go about their task. This enables observers to see first-hand the process of task completion (rather than only its final product). read more
Think Alouds are designed to help researchers "read minds." This is especially helpful when evaluating a person's mental model of a product because it reveals assumptions and expectations about how the product works. In design, we use existing mental models to help inform how we communicate a conceptual model to create a change in user behavior.
Goals
Listen to a person while they work (accomplish a task).
Learn about what frustrates, confuses, and pleases a person.
Reveal how a person thinks about the problems you appear to be solving.
Use data to help iterate on the design approach.
Steps
1.) Start by introducing the think-aloud protocol:
I realize it's probably a little strange that we're asking you to share your personal thoughts about something out loud, but this will help us to get insight into your experience and see things through your eyes. You might ask us questions about how things work or what they are, and please don't be surprised but we'll probably turn those questions back to you to learn more about what you think should happen and why.
2.) Set up the scenario for each screen (static comp or interactive prototype) to help the participant understand at a high level how things work and why they are important.
3.) Encourage the participant to think aloud and share questions/concerns verbally.
Note: People will usually ask questions, but you don't need to provide them with answers. Instead, ask:
"What do you think would happen?
What would you want to happen?
Would that trigger any other event, like an email notification?
Good Questions
What's going through your mind right now?
What would you try to accomplish on this page? (Why?)
How do you expect that to work?
How close was that to what you expected?
What additional information would you want to see? (Why?)
How do you think you would solve this problem?
Note: Avoid starting questions with "Why?" instead; surface the Why as a follow-up.
Tips
What people don't say. Think Alouds aren't only about what people say out loud; they are about facial expressions, attitudes, periods of silence, and patterns (what do people notice first). An example: In a study we did about project management, every participant smiled, and then they saw that "assignee" had become "assignees." A single "s" made a world of difference.
Measurement errors. Think Alouds can produce errors by slowing the thought process and increasing mindfulness. Avoid the term "usability test" with participants because it establishes the idea that people must do things the "right" way. The "wrong" way is incredibly valuable!
Embrace silence or "remember the Ma.” Try to give yourself 20 - 30 seconds of pause between a person asking you a question and then responding with, "How do you think that might work?" It is more than likely that the participant will begin to fill up the void with valuable conversation, but it also provides them with a comfortable amount of time to be thoughtful.
Struggle is helpful, too. Don't rescue participants who are struggling. It's uncomfortable not to jump in and start solving the problem or explaining how things work. Instead, learn what happens when people get lost and how/if they recover.
Hold a debriefing session
Ask the team to list out their top 5 - 10 insights.
Organize insights into priorities.
Write down insights on sticky notes and sort priorities into lists/phases.
Moderators
Be aware of your body language and what it communicates to other people.
Balance praise (I struggle with smiling too much).
If you are doing a remote session, don't put two people in one room on one screen. Everyone should dial into the session individually to set an equal experience.
Good reading
Barnum, Carol M. Usability Testing Essentials
Rubin, Jeffery, and Dana Chisnell. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests
