• Why are people abandoning the donation form?
• Why don’t more people give recurring gifts?
• What are the main things people are looking for on
• Do people notice the promotion we’re doing?
• How should we organize our new website (or
• Good targets
- People somewhat involved with your organization
• Event participants
- The general public
• Generally avoid*
- Major volunteers
- Friends & family
1. Make sure you pick something that regular people
use, you will get regular people at Starbucks.
2. Test internally for practice
3. Test internally to make sure it is truly a 5 minute
4. Take 1 person with you. 1 person stays with laptop,
1 person solicits.
BEFORE YOU GO
1. Make sure Starbucks is ok with it. Talk to the
manager on duty first, have a pitch ready for that
conversation so you can easily explain what you
want to do.
2. Bring your charger, mouse, test internet connection
(if needed), etc.
3. Grab a table - go early or avoid rush times.
BEFORE YOU START
1. Talk to the people that are sitting around, you are buying
their NEXT cup of coffee, avoid the people in line.
2. Put your best foot forward, pick someone that will get a
3. Work on your pitch, it has to be faster than an elevator
pitch. Think of your pitch like a tweet: short, sweet and
to the point.
4. Make it obvious that you are a non-profit, lead with that,
wear the gear, beat people over the head with it; people
are more likely to help.
5. It helps to stress that this will take 5 minutes or less.
WHEN YOU ARE READY
1. Don’t record names
2. Do get a wide cross section of users (age,
gender, race, etc)
3. Test the savvy level of the user, make sure they
aren’t extremely savvy or extremely novice.
4. Give them their task
6. Try not to help
7. Ask follow up
WHEN YOU HAVE A USER
• If you find a massive gotcha, STOP. No need to keep
testing. Go back and fix it and try again another day.
• Once you have 3-5 users you probably have some really
good feedback, no reason to keep going. Maybe switch
to a new thing to test.
• This whole process might take 1.5-4 hrs
• You will most likely learn things that will blow your mind.
Things you never would have thought of.
WHEN THE TEST IS OVER
• Write down each 2nd level menu item on
its own note card
- In this example
• Write down
– News Archive
– Wayne’s blog
– Press releases
– Media contacts
• Don’t write
1. Get a participant and some table space
2. Give them
- your note cards
- some blank cards or post-it notes
• (No more than you have space in your top level
3. Let them sort the pre-written note cards into
4. Ask them to label each group (using the
blank note cards)
5. Record the results
6. Repeat at least 10 times
- Be sure to stay quiet while the participant is working
- Ask them to think aloud and listen to their commentary
- Take a photo when each person is done to capture their
- Pre-write the headers (closed card sort)
- Conduct the card sort online
POLL: WHO HAS ABANDONED A SURVEY THEY’VE
• 11 out of 15 people surveyed had never done any
- Surveys were the way most people do user
• 15 of 15 people surveyed had abandoned a survey
half way through because it was too long or too
OUR USER RESEARCH
• Keep it short
- Usually have 5 – 15 questions
- Should take less than 5 minutes to complete
• Always have a general comment field
- Let them speak to you, not just answer questions
• Have someone proofread your survey
- Make sure none of the questions are confusing
• Be neutral
- You want information, not vanity metrics
• You don’t need to survey everyone
- How many responses do you need to answer your questions?
• You can use statistics, or
• General questions, homogeneous audience ~100 responses
- Assume a response rate of 10 – 20% for people somewhat involved with
• Think of each question as testing a hypothesis. It
can help to word them that way
- Less useful: We should ask how many times per year they visit our
- Better: I suspect that most people visit either only once per year or
more than 4 times. This will affect how we price and market our
membership programs. If we find a lot of people who visit 3-4 times
a year, we might decide to adjust our programs
SURVEY ANALYSIS TIPS
• Avoid “neutral” options
- Very likely
- Somewhat likely
- Somewhat unlikely
- Very unlikely
• (But you can use “not applicable”, if applicable)
• Experiment with forced rankings
• If you’re doing a large survey, consider how many open-ended
questions you want to analyze
SURVEY ANALYSIS TIPS
THIS PRESENTATION BROUGHT TO YOU BY
BLACKBAUD USER EXPERIENCE
• We want to learn from you so we can design
experiences that delight
• Participate in Blackbaud Discovery!
- Help us with our research
- Have your voice heard for the products you use
• Sign up to participate at
• Stop by the Discovery booth in the expo hall