Designing to change human behaviors

Designing for “big leaps” may yield little-to-no change in behaviors.

This is a story about designing, measuring, and changing human behaviors with new product features. Measurement is essential, because it helps you to validate that change is positive (or at least neutral) and that it doesn’t alienate users or result in a loss of business.

Design for three behaviors

In order to change human behaviors with changes to existing features, the rollout of new features, or sometimes even with an all-new product, you must design clear and intuitive paths forward that are easy to adopt. When product experiences fall short, people develop workarounds and can become pretty good at being efficient with bad design.

If it works well enough, why change it? Change will take more time.

There are plenty of frameworks for thinking about designing to change behaviors, here is the model I’ve found the most useful:

Current- Core behaviors engrained into regular daily actions; may include workarounds where product falls short of expectations and needs.
New – Potential behaviors and changes to workflows that people are open to adopting if the changes help them to achieve current goals.
Aspirational – Future behaviors that emerge alongside both new features and product and may support new goals people have developed as a result.

Designing an experience that expects people will move from current to aspirational behaviors immediately requires people be willing to take a huge leap and in many cases be inconvenienced by a rough landing in the process.

Leapers have trust in the user experience, product, and company, as well as time to learn something new. In my experience, the leap from current to aspirational can result in breaking changes, which are risky, inconvenient, and overall kind of crazy to folks who have a lot of empathy for their customers.

Of course, you might succeed designing for the leap, especially if you’re lucky, however you’re guaranteed to learn more about the people using your product and make better product by encouraging people to adopt a new incremental behavior on the way towards aspirational.

In fact, the aspirational behavior you have in mind at the beginning of a project will ~~probably~~ (most definitely) change based on what you learn observing new behaviors people adopt. This brings me to one of my favorite examples –Git Large File Storage (Git LFS) on GitHub.

Releasing a really large feature

In April 2015, a fantastic team at GitHub released Git LFS to an enthusiastic audience at Git Merge and on Twitter.

Git LFS has a few moving and complementary parts, but this also makes designing a cohesive user experience tricky:

Git LFS client for managing large files locally.
Add-on plan for hosting and bandwidth on GitHub.com (free/paid tiers).
Open source project for bug reporting, latest releases, community and contributions.

Documentation for each one of these three pieces lives in a unique-but-separate location, which makes putting them altogether into a single story difficult.

In addition to the moving pieces, there were quite a few things that kept us up at night:

Pricing and billing – Were the tiers fair and attainable and the upgrade path clear? Did we plan enough storage and bandwidth for open source projects?
Infrastructure – Were we prepared and responsible? Would the site go down?
Support – Did we have answers to anticipated questions prepared for our Support team and links to helpful documentation? Would we get a rush of questions that would delay customer response time?

Early Access Program

Our team rolled Git LFS out slowly through an Early Access Program, a strategy grounded in OODA loop thinking (observe, orient, decide, act).

Git LFS requires that people change both their:

Workflows for managing large files (often using third-party software).
Mental model of how they understand git works, because it’s well-established that binaries slow git down and GitHub has historically had stringent size restrictions.

Together, changing current workflows and changing how a person understands git and large binaries presented us with a set of challenges. We had a hunch widespread adoption would be challenging, but we also knew from prior research that a strong, vocal minority of customers wanted this feature.

To validate hypotheses and measure how/if people changed behaviors to use Git LFS we ran the EAP as a controlled experiment. The premise of an early access program is that people self-select to receive the feature early, which means that we were working with a biased group. However, to help mitigate bias and ensure a diverse sample we used randomization to create two groups:

Group 1. “the treatment” received access to GitHub’s hosting plans (we turned this ability on for them and they were notified by email).
Group 2. “the control” did not receive access to GitHub’s hosting plans during the experiment timeframe (note: this group will still receive early access before we ship to genpop, just not during the experiment timeframe).

Key insight

At a high-level, both groups behaved similarly over a 6-8 week period –adoption takes time. People needed to get the client, experiment with a new workflow, and if they liked it enough upgrade to a paid plan (possibly replacing other third party storage solutions).

To round out our quantitative and qualitative insights and get clearer on our blind spots (why people weren’t adopting the add-on plan), we sent out two exit surveys to the treatment group. They exit surveys had different questions based on whether the program member adopted the feature or did not, and the surveys solicited great feedback, especially from people who didn’t use the feature. These insights are critical to helping inform ways we’ll continue to improve the user experience from installation to documentation to pricing and much more.

Missed opportunity, lesson learned

We asked people to leap from their current behaviors to our aspirational behaviors, which is basically like saying:

Hi there, would you mind switching over to a new belief system about how git works, install and use this client from the command line, and create a new workflow to replace other storage solutions you’ve been using both as an individual and as part of a team? We’ll give you about two months, let’s do this!

The missed opportunity and lesson learned is that we didn’t design an incremental step to create a new behavior. A possible example of something we could have done would be to encourage people to start uploading large binaries on the site in Issues and Pull Requests (currently, you can only add image file types). File uploads would have introduced people to a change with GitHub policy fewer file size restrictions, more file types, and add-on plans.

The adoption of new behaviors takes time and trust especially when you’re working with other people in team workflows, which also makes it really exciting product research to be part of!

I’m very appreciative to @technoweenie and @tclem who were willing to leap into research with me.