Prove It! A Practical Guideto A/B Testing for Nonprofits               @NetTuesday // @bradyjosephson // #ntvan
4 More Years.
Culture of Learning.          @NetTuesday // @bradyjosephson // #ntvan
Tonight.• 3 Forms of Testing         • Tweet                                 – #ntvan• 4 Steps to Testing             – @N...
Hacking. Tracking. Cracking.3 FORMS OF TESTING                               @NetTuesday // @bradyjosephson // #ntvan
Hack.  @NetTuesday // @bradyjosephson // #ntvan
Track.   @NetTuesday // @bradyjosephson // #ntvan
Crack.   @NetTuesday // @bradyjosephson // #ntvan
Insights. Hypothesis. Test. Analyze.4 STEPS TO TESTING                                  @NetTuesday // @bradyjosephson // ...
Insights.Things that provoke curiosity and make you wantto test.6 Sources for Insights• Survey.• Feedback.• Analytics.• Fo...
Insights. // Dissonance.Text REDCROSS to 30333                         @NetTuesday // @bradyjosephson // #ntvan
Hypothesis.Tries to answer an important question with anexpected result.• If True?  – Would we do differently?• If False? ...
Test.The execution to get the data/results you needto make a decision.Choose the (right) metric.State what is significant....
Analyze.Looking at the unfiltered data for a winner, whatyou learned and what it means moving forward.Which one won?What e...
Confounding Variables. Small Sample Size. Lack of Purpose. Wrong Metric. Skewed Data.5 THINGS TO AVOID                    ...
Confounding Variables.Too many variables making it hard to know whatcaused what.• Try to change as few things as possible ...
Sample Size.Smaller the size the less confident you can be inthe result.• If you only have small sample sizes, run more  t...
No Purpose.What changes with the outcome of this test orfinding?• Think of a decision you recently made. Now  think of wha...
Wrong Metric.Choosing to measure the wrong thing to “prove”the winner.• Email open rates measure subject lines (firstly)  ...
Skewed Data.Bad data gets compounded. Start with gooddata and you have a chance.• Start capturing good/clean data right no...
Email Appeals. Email Opens. Email Signups. Facebook Ads.1 CASE STUDY & 3 OTHEREXAMPLES                                @Net...
Email Appeals.• Insight.  – Current site revolves around projects, we are    suggesting it should revolve around stories.•...
Email Appeals.• Test.  – Send two similar emails to randomized group    with slightly different funding focus.• Analyze.  ...
Email Appeals. // Test.• Randomized Group.  – Excel and random function• Set Up Goals.  – Connect MailChimp and Google ana...
Email Appeals. // Test.Choose the (right) metric.  – Number of gifts.State what is significant.  – 20% difference.Prepare ...
Email Appeals. // Test.           @NetTuesday // @bradyjosephson // #ntvan
Email Appeals. // Analyze.             @NetTuesday // @bradyjosephson // #ntvan
Email Appeals. // Analyze.             @NetTuesday // @bradyjosephson // #ntvan
Email Opens. // Analyze.            @NetTuesday // @bradyjosephson // #ntvan
Email Signups. // Test.           @NetTuesday // @bradyjosephson // #ntvan
Email Signups. // Test.           @NetTuesday // @bradyjosephson // #ntvan
Email Signups. // Analyze.             @NetTuesday // @bradyjosephson // #ntvan
Facebook Ads. // Analyze.            @NetTuesday // @bradyjosephson // #ntvan
UnBounce. MailChimp. Google. KISS Metrics. HubSpot. Wufoo. Excel.7 RESOURCES FOR TESTING                                  ...
Resources.  Forms                                                     Landing Pageswufoo.com                          Data...
On You.1 LIVE TEST              @NetTuesday // @bradyjosephson // #ntvan
Live Test. // A.If you sign up to get emailsfrom me today, I’ll send youa free copy of our eBook“Tool Kit for Charities”.T...
Live Test. // B.If you sign up to get emailsfrom me today, you’ll beentered for a chance to wina $25 gift card to Starbuck...
Live Test. // Analyze.           @NetTuesday // @bradyjosephson // #ntvan
From You.? QUESTIONS & THANKS             @NetTuesday // @bradyjosephson // #ntvan
From me. @bradyjosephson // recharity.caTHANKS!                               @NetTuesday // @bradyjosephson // #ntvan
Upcoming SlideShare
Loading in...5
×

Prove it! A/B Testing for Nonprofits

2,275

Published on

A practical guide to hacking, tracking and cracking your way to donor insights, engagement and funds raised.

1 Comment
3 Likes
Statistics
Notes
  • Web Design, Web Development and SEO Services Noida, Delhi/NCR (India)


    Web It Experts is a Noida, Delhi/NCR (India) based Web designing and website development company. We provide Services for SEO, Website promotion, Software development, Mobile applications, Web hosting services.

    Website redesign services, Software Development Company Noida, Delhi- NCR, India, SEO Company Noida, Delhi- NCR, India, Web Development Noida, Delhi- NCR, India, Website Maintenance Noida, Delhi- NCR, India, Cheap Website Design in India, USA, UK, CANADA, Australia Website solutions in Noida, Delhi- NCR, India

    Web It Experts is an India based e-commerce web design firm specialise in Web design, Web development, Web hosting, Zencart templates design.
    ----------------------------------------------------------------------------------------------
    We are creative website design, Web Development, SEO, Custom Web Designing, SEO services provider company based in NOIDA/Delhi-NCR, our best Rates in Web Designing and Web Development.

    Our Services:

    Web Designing

    Web Development

    Search Engine Optimization

    Web Hosting

    Website : www.webitexperts.com


    Email : sales@webitexperts.com
    ----------------------------------------------
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
2,275
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
13
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide
  • $690 raised online. 20 writers. 18 variations. “Hey” subject line revelation. “I will be outspent” raised $2.6 M.
  • Not “expert”.Focused on small & medium sized nonprofits.Resources (time).Current vs. future value.Knowing WHO your donors are (what they like, where they are, what motivates them, why they get involved, etc.)
  • Order matters!
  • Surveys: Ask them!donors want less mail, less communications, etc. but only give when they get those things.Feedback: very personalized responses.Analytics: we saw one client’s Board Member page had unusually high traffic. Other tests (hacks and tracks).
  • Surveys: Ask them!donors want less mail, less communications, etc. but only give when they get those things.Feedback: very personalized responses.Analytics: we saw one client’s Board Member page had unusually high traffic. Other tests (hacks and tracks).
  • Hindsight is not testing.If red buttons have higher rate, what changes? If blue does, what changes?You don’t need to run a test to know that people give when they are asked.
  • Confounding variables:
  • Confounding variables:
  • The biggest common weakness of small and medium sized nonprofits is how they capture and manage data.
  • Freebie: follow up email boosted giving 57%!
  • Hack.
  • Hack.
  • Crack.
  • Hack and Crack.
  • Hack and Crack.
  • Qualaroo: pop up to get information from people on pages/sites
  • <?xml version="1.0" encoding="ISO-8859-1" ?><poll url="http://www.polleverywhere.com/multiple_choice_polls/MTg1NDY1NDY4OQ"> <!-- This snippet was inserted via the Poll Everywhere Mac Presenter --> <!-- The presence of this snippet is used to indicate that a poll will be shown during the slideshow --> <!-- TIP: You can draw a solid, filled rectangle on your slide and the Mac Presenter will automatically display your poll in that area. --> <!-- The Mac Presenter application must also be running and logged in for this to work. --> <!-- To remove this, simply delete it from the notes yourself or use the Mac Presenter to remove it for you. --> <title>Sign up for emails...</title></poll>
  • Prove it! A/B Testing for Nonprofits

    1. 1. Prove It! A Practical Guideto A/B Testing for Nonprofits @NetTuesday // @bradyjosephson // #ntvan
    2. 2. 4 More Years.
    3. 3. Culture of Learning. @NetTuesday // @bradyjosephson // #ntvan
    4. 4. Tonight.• 3 Forms of Testing • Tweet – #ntvan• 4 Steps to Testing – @NetTuesday – 6 Sources for Insights – @bradyjosephson – 5 Things to Avoid• 1 Case Studies & 3 • Poll Other Examples – @poll – Text to: 37607• 7 Resources for Testing• 1 Live Test • Me• ? Questions – brady@charityexpress.com – recharity.ca – recharity.ca/category/testi ng/ @NetTuesday // @bradyjosephson // #ntvan
    5. 5. Hacking. Tracking. Cracking.3 FORMS OF TESTING @NetTuesday // @bradyjosephson // #ntvan
    6. 6. Hack. @NetTuesday // @bradyjosephson // #ntvan
    7. 7. Track. @NetTuesday // @bradyjosephson // #ntvan
    8. 8. Crack. @NetTuesday // @bradyjosephson // #ntvan
    9. 9. Insights. Hypothesis. Test. Analyze.4 STEPS TO TESTING @NetTuesday // @bradyjosephson // #ntvan
    10. 10. Insights.Things that provoke curiosity and make you wantto test.6 Sources for Insights• Survey.• Feedback.• Analytics.• Focus Group.• Social Media.• Your Brain. @NetTuesday // @bradyjosephson // #ntvan
    11. 11. Insights. // Dissonance.Text REDCROSS to 30333 @NetTuesday // @bradyjosephson // #ntvan
    12. 12. Hypothesis.Tries to answer an important question with anexpected result.• If True? – Would we do differently?• If False? – What would we do differently?• Needed? – Is this question already answered? @NetTuesday // @bradyjosephson // #ntvan
    13. 13. Test.The execution to get the data/results you needto make a decision.Choose the (right) metric.State what is significant.Prepare your data.Control your variables. @NetTuesday // @bradyjosephson // #ntvan
    14. 14. Analyze.Looking at the unfiltered data for a winner, whatyou learned and what it means moving forward.Which one won?What else did you learn?How confident are you in the result?What are the next steps? – Run another test? – Change something in your strategy? @NetTuesday // @bradyjosephson // #ntvan
    15. 15. Confounding Variables. Small Sample Size. Lack of Purpose. Wrong Metric. Skewed Data.5 THINGS TO AVOID @NetTuesday // @bradyjosephson // #ntvan
    16. 16. Confounding Variables.Too many variables making it hard to know whatcaused what.• Try to change as few things as possible and iterate more• The more moving pieces you have the less you can trust the outcome @NetTuesday // @bradyjosephson // #ntvan
    17. 17. Sample Size.Smaller the size the less confident you can be inthe result.• If you only have small sample sizes, run more tests• Smaller sample sizes allow for more custom/manual work @NetTuesday // @bradyjosephson // #ntvan
    18. 18. No Purpose.What changes with the outcome of this test orfinding?• Think of a decision you recently made. Now think of what information you wish you had to make that decision. That’s purpose. @NetTuesday // @bradyjosephson // #ntvan
    19. 19. Wrong Metric.Choosing to measure the wrong thing to “prove”the winner.• Email open rates measure subject lines (firstly) and timing (secondly) so unless you are testing those things it is a useless metric for a test. @NetTuesday // @bradyjosephson // #ntvan
    20. 20. Skewed Data.Bad data gets compounded. Start with gooddata and you have a chance.• Start capturing good/clean data right now• Don’t try to capture everything all at once• Just because you can track it doesn’t mean that you should @NetTuesday // @bradyjosephson // #ntvan
    21. 21. Email Appeals. Email Opens. Email Signups. Facebook Ads.1 CASE STUDY & 3 OTHEREXAMPLES @NetTuesday // @bradyjosephson // #ntvan
    22. 22. Email Appeals.• Insight. – Current site revolves around projects, we are suggesting it should revolve around stories.• Question. – If we can prove more connection to stories, not projects, it will help with site and communications revision.• Hypothesis. – Appeals centered around stories, not projects, will produce a greater response rate measured by gifts made and greater than 20%. @NetTuesday // @bradyjosephson // #ntvan
    23. 23. Email Appeals.• Test. – Send two similar emails to randomized group with slightly different funding focus.• Analyze. – See which email has the greater response rate as determined by gifts made. @NetTuesday // @bradyjosephson // #ntvan
    24. 24. Email Appeals. // Test.• Randomized Group. – Excel and random function• Set Up Goals. – Connect MailChimp and Google analytics• Complications. – Small sample size – Additional emails @NetTuesday // @bradyjosephson // #ntvan
    25. 25. Email Appeals. // Test.Choose the (right) metric. – Number of gifts.State what is significant. – 20% difference.Prepare your data. – Randomized in Excel.Control your variables. – Length, design, landing page, timing all the same. @NetTuesday // @bradyjosephson // #ntvan
    26. 26. Email Appeals. // Test. @NetTuesday // @bradyjosephson // #ntvan
    27. 27. Email Appeals. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    28. 28. Email Appeals. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    29. 29. Email Opens. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    30. 30. Email Signups. // Test. @NetTuesday // @bradyjosephson // #ntvan
    31. 31. Email Signups. // Test. @NetTuesday // @bradyjosephson // #ntvan
    32. 32. Email Signups. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    33. 33. Facebook Ads. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    34. 34. UnBounce. MailChimp. Google. KISS Metrics. HubSpot. Wufoo. Excel.7 RESOURCES FOR TESTING @NetTuesday // @bradyjosephson // #ntvan
    35. 35. Resources. Forms Landing Pageswufoo.com Data unbounce.com Your Computer Analytics (and everything)Search “Google” on bing.com Marketing hubspot.com LTV & Awesome Blog Emails www.wufoo.com mailchimp.com @NetTuesday // @bradyjosephson // #ntvan
    36. 36. On You.1 LIVE TEST @NetTuesday // @bradyjosephson // #ntvan
    37. 37. Live Test. // A.If you sign up to get emailsfrom me today, I’ll send youa free copy of our eBook“Tool Kit for Charities”.To sign up:• Text 599154 to 37607 OR• Tweet @poll with 599154 @NetTuesday // @bradyjosephson // #ntvan
    38. 38. Live Test. // B.If you sign up to get emailsfrom me today, you’ll beentered for a chance to wina $25 gift card to Starbucks.To sign up:• Text 599156 to 37607 OR• Tweet @poll with 599156 @NetTuesday // @bradyjosephson // #ntvan
    39. 39. Live Test. // Analyze. @NetTuesday // @bradyjosephson // #ntvan
    40. 40. From You.? QUESTIONS & THANKS @NetTuesday // @bradyjosephson // #ntvan
    41. 41. From me. @bradyjosephson // recharity.caTHANKS! @NetTuesday // @bradyjosephson // #ntvan
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×