Effective User Testing

• To confirm your hypotheses
• To find out where you're wrong
• Kill Your Darlings
• Testing is the core of lean startup's build-measure-learn cycle

• Don't have any buttons or links that don't work
• Don't have an ugly OR a flashy design
• Don't start with a tour of the app
• Don't ask them to play with it (unless it's a game)
• Don't help them when they look confused

• Do scenario-based task testing
• Do record and/or take notes
• Do look at these user testing tips from the Center for Plain Language
  • http://www.slideshare.net/whitneyq/3-usability-techniques/36

• BEFORE: Write everything out in advance
  • Product box, pre / post surveys, task scenarios
• DURING: Shut up, watch, record and take notes
  • Don't tell them what to do
  • Answer questions with questions
  • Let them make mistakes. Can they recover?
  • Look for friction and confusion
• AFTER: Analyze
  • categorize and tally mistakes, confusions, hesitation, ...
  • focus on most critical and common problems

• A very few questions about things that you hypothesize best describe your ideal user
  • E.g., for a recipe app, "how many times a week do you cook?"
  • Avoid subjective questions like "are you a frequent traveler?"
  • Demographic information (age, gender, ...) only if relevant

• Written setup for a problem situation. Include realistic local details
  • "It's 4:55. You're just finishing a math class in Lunt. Your 394 team is meeting in Tech at 5:30. You're hungry. You'd like to see if any place on campus open right now is serving something vegan."
• No instructions on how to use the app
• User Testing Scenarios and Tasks

• A page of questions (at most) to see how users perceived your app
  • What do you think the point of this app is? Who is it intended for?
  • What were the 3 best and worst things about this app?

• Avoid the "counterfeit yes"
• Pass the Mom Test
• A good post-test question: "Would you be willing to test our next version?"

• Things users clicked on, asked about, even though not obviously relevant to task
• Could be a sign of what you should really be building
• Could be an evil attractor

• Who you test matters at least as much what you test
• Don't just find some random friends
• Testers have to align your value proposition
• Don't test a recipe app on people who never cook...
• ... unless your app is meant for people who don't cook
• Look in the places where your target users are more likely to be

Thanks to Hakim El Hattab for RevealJS:
https://github.com/hakimel/reveal.js/