Think Aloud Sneak Attack

Posted by Chrissie Brodigan on March 2, 2016

There’s an infamous story and that often comes up in product debates to help justify not listening or reacting to customer requests. It’s inaccurately attributed to Henry Ford:

“If I had asked people what they wanted, they would have said faster horses.”

Customer requests are an important input in product development and the role of research is to listen to and surface the how, why, when, and where, behind them. We’re looking for patterns, use cases, and the human motivations and goals driving the conversation.

Customer Request: We need Branch Permissions

At GitHub, in 2015, customers told us they needed a complex feature called “branch permissions.” Competitor products offered branch permissions, and our customers pointed out that GitHub wasn’t meeting their permissioning needs — our product felt like it was falling behind in an increasingly busy marketplace.

If you like creating puzzles, then designing user permissions can be incredibly fun, but permissions systems are also incredibly complex to grow, iterate on, and maintain.

When designing for a large audience you need to be deliberate with decisions, especially when it comes to adding something new— it’s always much harder to take something away. This is especially true with permissions systems.

People can become very efficient using bad design and on-demand features can slowly pile up into a terrible-but-functional mess. From a research perspective Branch Permissions presented our team with a challenge to surface the human motivations and goals behind the request itself. We accomplished our task with a 25 minute interview, leading up to what we really cared about, which was a five minute Think Aloud.

Fork v. Branch: Choosing a collaboration model

GitHub has two models for collaborating with peers on a codebase:

Choosing a collaboration model:

  1. Fork and pull — Make a complete copy of the parent project, edit it, and propose your change upstream with a pull request.

  2. Branch and merge — Make a partial copy of the parent project, edit an area, and propose your change back to the Master branch with a pull request.

Think Aloud: Feature Prioritization

For the exercise. I spent about 20–25 minutes talking to each participant about their collaboration model: How did you choose to fork and pull vs. branch (and the inverse). Often collaboration models were inherited.

These “warm up” questions led to the core think aloud activity, which presented a list of features in no particular order (branch permissions was among them).

Participants were asked to order these features with the most important at the top and least at the bottom, and to think out loud to explain how they were making their choices. The were also encouraged to add to the list is something was missing.

However, here’s where we had a sneak attack planned …

When participants noticed a featured titled, “Undo button.” They were stumped. A few hit their hands to their foreheads, asking:

“Where is there an undo button, that would have helped us out so much!”

And, I’d ask:

“Can you tell me where would you look for undo button in the product’s UI?”

“Can you describe a time when you could have used this undo button?”

Carefully capturing the details of the users’ motivations and goals across a series of interviews, this data (among other inputs) helped our team pursue an opinionated direction, building protected branches and required status checks.

3 Tips for designing a sneak attack:

  1. Include items that don’t exist, but sound like they might.
  2. Listen to people define what they think the “feature” is.
  3. Ask how, where, when, & why they would use the feature.