Prove it! A/B Testing for Nonprofits


Published on

A practical guide to hacking, tracking and cracking your way to donor insights, engagement and funds raised.

1 Comment
  • Web Design, Web Development and SEO Services Noida, Delhi/NCR (India)

    Web It Experts is a Noida, Delhi/NCR (India) based Web designing and website development company. We provide Services for SEO, Website promotion, Software development, Mobile applications, Web hosting services.

    Website redesign services, Software Development Company Noida, Delhi- NCR, India, SEO Company Noida, Delhi- NCR, India, Web Development Noida, Delhi- NCR, India, Website Maintenance Noida, Delhi- NCR, India, Cheap Website Design in India, USA, UK, CANADA, Australia Website solutions in Noida, Delhi- NCR, India

    Web It Experts is an India based e-commerce web design firm specialise in Web design, Web development, Web hosting, Zencart templates design.
    We are creative website design, Web Development, SEO, Custom Web Designing, SEO services provider company based in NOIDA/Delhi-NCR, our best Rates in Web Designing and Web Development.

    Our Services:

    Web Designing

    Web Development

    Search Engine Optimization

    Web Hosting

    Website :

    Email :
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • $690 raised online. 20 writers. 18 variations. “Hey” subject line revelation. “I will be outspent” raised $2.6 M.
  • Not “expert”.Focused on small & medium sized nonprofits.Resources (time).Current vs. future value.Knowing WHO your donors are (what they like, where they are, what motivates them, why they get involved, etc.)
  • Order matters!
  • Surveys: Ask them!donors want less mail, less communications, etc. but only give when they get those things.Feedback: very personalized responses.Analytics: we saw one client’s Board Member page had unusually high traffic. Other tests (hacks and tracks).
  • Surveys: Ask them!donors want less mail, less communications, etc. but only give when they get those things.Feedback: very personalized responses.Analytics: we saw one client’s Board Member page had unusually high traffic. Other tests (hacks and tracks).
  • Hindsight is not testing.If red buttons have higher rate, what changes? If blue does, what changes?You don’t need to run a test to know that people give when they are asked.
  • Confounding variables:
  • Confounding variables:
  • The biggest common weakness of small and medium sized nonprofits is how they capture and manage data.
  • Freebie: follow up email boosted giving 57%!
  • Hack.
  • Hack.
  • Crack.
  • Hack and Crack.
  • Hack and Crack.
  • Qualaroo: pop up to get information from people on pages/sites
  • <?xml version="1.0" encoding="ISO-8859-1" ?><poll url=""> <!-- This snippet was inserted via the Poll Everywhere Mac Presenter --> <!-- The presence of this snippet is used to indicate that a poll will be shown during the slideshow --> <!-- TIP: You can draw a solid, filled rectangle on your slide and the Mac Presenter will automatically display your poll in that area. --> <!-- The Mac Presenter application must also be running and logged in for this to work. --> <!-- To remove this, simply delete it from the notes yourself or use the Mac Presenter to remove it for you. --> <title>Sign up for emails...</title></poll>
  • Prove it! A/B Testing for Nonprofits

    1. Prove It! A Practical Guideto A/B Testing for Nonprofits @NetTuesday // @bradyjosephson // #ntvan
    2. 4 More Years.
    3. Culture of Learning. @NetTuesday // @bradyjosephson // #ntvan
    4. Tonight.• 3 Forms of Testing • Tweet – #ntvan• 4 Steps to Testing – @NetTuesday – 6 Sources for Insights – @bradyjosephson – 5 Things to Avoid• 1 Case Studies & 3 • Poll Other Examples – @poll – Text to: 37607• 7 Resources for Testing• 1 Live Test • Me• ? Questions – – – ng/ @NetTuesday // @bradyjosephson // #ntvan
    5. Hacking. Tracking. Cracking.3 FORMS OF TESTING @NetTuesday // @bradyjosephson // #ntvan
    6. Hack. @NetTuesday // @bradyjosephson // #ntvan
    7. Track. @NetTuesday // @bradyjosephson // #ntvan
    8. Crack. @NetTuesday // @bradyjosephson // #ntvan
    9. Insights. Hypothesis. Test. Analyze.4 STEPS TO TESTING @NetTuesday // @bradyjosephson // #ntvan
    10. Insights.Things that provoke curiosity and make you wantto test.6 Sources for Insights• Survey.• Feedback.• Analytics.• Focus Group.• Social Media.• Your Brain. @NetTuesday // @bradyjosephson // #ntvan
    11. Insights. // Dissonance.Text REDCROSS to 30333 @NetTuesday // @bradyjosephson // #ntvan
    12. Hypothesis.Tries to answer an important question with anexpected result.• If True? – Would we do differently?• If False? – What would we do differently?• Needed? – Is this question already answered? @NetTuesday // @bradyjosephson // #ntvan
    13. Test.The execution to get the data/results you needto make a decision.Choose the (right) metric.State what is significant.Prepare your data.Control your variables. @NetTuesday // @bradyjosephson // #ntvan
    14. Analyze.Looking at the unfiltered data for a winner, whatyou learned and what it means moving forward.Which one won?What else did you learn?How confident are you in the result?What are the next steps? – Run another test? – Change something in your strategy? @NetTuesday // @bradyjosephson // #ntvan
    15. Confounding Variables. Small Sample Size. Lack of Purpose. Wrong Metric. Skewed Data.5 THINGS TO AVOID @NetTuesday // @bradyjosephson // #ntvan
    16. Confounding Variables.Too many variables making it hard to know whatcaused what.• Try to change as few things as possible and iterate more• The more moving pieces you have the less you can trust the outcome @NetTuesday // @bradyjosephson // #ntvan
    17. Sample Size.Smaller the size the less confident you can be inthe result.• If you only have small sample sizes, run more tests• Smaller sample sizes allow for more custom/manual work @NetTuesday // @bradyjosephson // #ntvan
    18. No Purpose.What changes with the outcome of this test orfinding?• Think of a decision you recently made. Now think of what information you wish you had to make that decision. That’s purpose. @NetTuesday // @bradyjosephson // #ntvan
    19. Wrong Metric.Choosing to measure the wrong thing to “prove”the winner.• Email open rates measure subject lines (firstly) and timing (secondly) so unless you are testing those things it is a useless metric for a test. @NetTuesday // @bradyjosephson // #ntvan
    20. Skewed Data.Bad data gets compounded. Start with gooddata and you have a chance.• Start capturing good/clean data right now• Don’t try to capture everything all at once• Just because you can track it doesn’t mean that you should @NetTuesday // @bradyjosephson // #ntvan
    21. Email Appeals. Email Opens. Email Signups. Facebook Ads.1 CASE STUDY & 3 OTHEREXAMPLES @NetTuesday // @bradyjosephson // #ntvan
    22. Email Appeals.• Insight. – Current site revolves around projects, we are suggesting it should revolve around stories.• Question. – If we can prove more connection to stories, not projects, it will help with site and communications revision.• Hypothesis. – Appeals centered around stories, not projects, will produce a greater response rate measured by gifts made and greater than 20%. @NetTuesday // @bradyjosephson // #ntvan
    23. Email Appeals.• Test. – Send two similar emails to randomized group with slightly different funding focus.• Analyze. – See which email has the greater response rate as determined by gifts made. @NetTuesday // @bradyjosephson // #ntvan
    24. Email Appeals. // Test.• Randomized Group. – Excel and random function• Set Up Goals. – Connect MailChimp and Google analytics• Complications. – Small sample size – Additional emails @NetTuesday // @bradyjosephson // #ntvan
    25. Email Appeals. // Test.Choose the (right) metric. – Number of gifts.State what is significant. – 20% difference.Prepare your data. – Randomized in Excel.Control your variables. – Length, design, landing page, timing all the same. @NetTuesday // @bradyjosephson // #ntvan
    26. Email Appeals. // Test. @NetTuesday // @bradyjosephson // #ntvan
    27. Email Appeals. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    28. Email Appeals. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    29. Email Opens. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    30. Email Signups. // Test. @NetTuesday // @bradyjosephson // #ntvan
    31. Email Signups. // Test. @NetTuesday // @bradyjosephson // #ntvan
    32. Email Signups. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    33. Facebook Ads. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    34. UnBounce. MailChimp. Google. KISS Metrics. HubSpot. Wufoo. Excel.7 RESOURCES FOR TESTING @NetTuesday // @bradyjosephson // #ntvan
    35. Resources. Forms Landing Data Your Computer Analytics (and everything)Search “Google” on Marketing LTV & Awesome Blog Emails @NetTuesday // @bradyjosephson // #ntvan
    36. On You.1 LIVE TEST @NetTuesday // @bradyjosephson // #ntvan
    37. Live Test. // A.If you sign up to get emailsfrom me today, I’ll send youa free copy of our eBook“Tool Kit for Charities”.To sign up:• Text 599154 to 37607 OR• Tweet @poll with 599154 @NetTuesday // @bradyjosephson // #ntvan
    38. Live Test. // B.If you sign up to get emailsfrom me today, you’ll beentered for a chance to wina $25 gift card to Starbucks.To sign up:• Text 599156 to 37607 OR• Tweet @poll with 599156 @NetTuesday // @bradyjosephson // #ntvan
    39. Live Test. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    40. From You.? QUESTIONS & THANKS @NetTuesday // @bradyjosephson // #ntvan
    41. From me. @bradyjosephson // recharity.caTHANKS! @NetTuesday // @bradyjosephson // #ntvan