Metrics That MatterPresented by Chris Geoghegan
Who Am I?•   Chris Geoghegan•   Product Manager at Peer Giving Solutions•   Build software for charities•   Advise fundrai...
Who Am I?•   Chris Geoghegan•   Product Website: http://peergiving.com•   Charity Blog: http://ideas.peergiving.com•   Twi...
Who Are You?
Vanity    Actionable      Testing   People ThenMetrics    Metrics     Frameworks     Metrics
“There are three kinds of lies:lies, damned lies, and statistics”                          metrics           ~ Mark Twain
Not a single thing on this page helps me make good decisions.
Don’t be this guy!VANITY METRICS
VANITY METRICS ARE DANGEROUS
SO WHAT THEN?
Actionable MetricsActionable – Accessible – Auditable            ~ Eric Ries
Metrics that matter are actionable.If they aren’t answering your most important    questions then don’t waste your time.1....
Metrics that matter are accessible.1. How easy is for any decision maker to gain   access to current reports?2. How easy i...
Metrics that matter are auditable.Think Scientific Method. Can the metrics be   trusted, verified as accurate, and   repro...
Metrics that matter track people.“Metrics are People too.” ~ Eric RiesAre you tracking the behavior of smaller, logical   ...
Exercise: Identifying Your Key Metrics1. Determine the 3-5 most important macro   metrics for your organization.2. Test th...
Tools1. Google Analytics: It’s extremely likely that   your top 5 metrics are going to be either   goals or events.2. KISS...
Metric-BasedDecision FrameworksFunnels, Cohorts, A/B Testing
Funnels• A series of events that represent a progression  towards some goal• Identifies how many people move from the  pre...
AARRR!
AARRR: User Lifecycle Funnel    100    90    80    70    60    50%    40    30    20    10     0          Acquisition   Ac...
What’s Most Important Now?            How do people find you?    100    90    80                    Do people have a good ...
Exercise: What’s Your Lifecycle?
Cohorts•   A group of people•   Share a common characteristic or experience•   Exposed to similar conditions•   Behaviour ...
Examples of Cohorts•   Before & After Tests•   Time Periods•   Marketing Sources•   Product Versions•   Priming•   Demogra...
Exercise: Cohorts• Based on some of the suggestions in the  previous slides what are some relevant  cohorts for your organ...
Special Cohort: A/B Testing• The Gold standard of web metrics• Usually comparing a hypothesis against an  original• Big ch...
Tools• Google Analytics  – Funnels, but a bit limiting  – A/B with “Content Experiments”  – “Advanced Segments” can provid...
Q: WHAT DO I TEST?
A: TALK TO PEOPLE
How to Talk to People•   Usability Tests•   Interviews•   Surveys•   Focus Groups
QUALITATIVE THEN QUANTITATIVE
Exercise: Put it All Together1. Talk to people2. Develop a hypothesis based on these   conversations3. Identify the releva...
Resources•       Google Analytics    –       Setting up goals: http://analytics.blogspot.ca/2009/05/how-to-setup-goals-in-...
Thank You!•   Chris Geoghegan•   Product Website: http://peergiving.com•   Charity Blog: http://ideas.peergiving.com•   Tw...
Metrics that Matter
Upcoming SlideShare
Loading in …5
×

Metrics that Matter

578 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
578
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • They can’t be trustedAt best they show correlation“Yeah, but…” aboundsFail you in moments of crisisFall Back: Highest Paid Person’s Opinion
  • Without this you will ultimately fall back on intuition, HiPPO, and lies damned lies.
  • Metrics that Matter

    1. 1. Metrics That MatterPresented by Chris Geoghegan
    2. 2. Who Am I?• Chris Geoghegan• Product Manager at Peer Giving Solutions• Build software for charities• Advise fundraising consultants• 13+ Years of Experience as Knowledge Worker
    3. 3. Who Am I?• Chris Geoghegan• Product Website: http://peergiving.com• Charity Blog: http://ideas.peergiving.com• Twitter: chrisgeoghegan• Email: chris.geoghegan@peergiving.com
    4. 4. Who Are You?
    5. 5. Vanity Actionable Testing People ThenMetrics Metrics Frameworks Metrics
    6. 6. “There are three kinds of lies:lies, damned lies, and statistics” metrics ~ Mark Twain
    7. 7. Not a single thing on this page helps me make good decisions.
    8. 8. Don’t be this guy!VANITY METRICS
    9. 9. VANITY METRICS ARE DANGEROUS
    10. 10. SO WHAT THEN?
    11. 11. Actionable MetricsActionable – Accessible – Auditable ~ Eric Ries
    12. 12. Metrics that matter are actionable.If they aren’t answering your most important questions then don’t waste your time.1. How do you gain or lose revenue?2. How do you gain or lose customers/supporters?3. What are key benefits that people are coming to you for?HINT: Track the MACRO not the micro
    13. 13. Metrics that matter are accessible.1. How easy is for any decision maker to gain access to current reports?2. How easy is for any decision maker to understand and act on the metrics?
    14. 14. Metrics that matter are auditable.Think Scientific Method. Can the metrics be trusted, verified as accurate, and reproducible?
    15. 15. Metrics that matter track people.“Metrics are People too.” ~ Eric RiesAre you tracking the behavior of smaller, logical groups of people? Or possibly even individuals?
    16. 16. Exercise: Identifying Your Key Metrics1. Determine the 3-5 most important macro metrics for your organization.2. Test them against the questions in the previous slides to make sure they are actionable and not vanity metrics.3. Map those metrics to events that can be tracked. (i.e. visiting a specific page, clicking a link, filling out a form)4. Start Tracking
    17. 17. Tools1. Google Analytics: It’s extremely likely that your top 5 metrics are going to be either goals or events.2. KISS Metrics or MixPanel: Google Analytics alternatives.
    18. 18. Metric-BasedDecision FrameworksFunnels, Cohorts, A/B Testing
    19. 19. Funnels• A series of events that represent a progression towards some goal• Identifies how many people move from the previous event to the next one• Funnels can represent the macro and the micro• Macro Example: User Lifecycle• Micro Example: Donation form/wizard
    20. 20. AARRR!
    21. 21. AARRR: User Lifecycle Funnel 100 90 80 70 60 50% 40 30 20 10 0 Acquisition Activation Retention Revenue Referral
    22. 22. What’s Most Important Now? How do people find you? 100 90 80 Do people have a good first experience? 70 60 Do people come back? 50% 40 30 How do you raise funds? 20 10 Do users tell others? 0 Acquisition Activation Retention Revenue Referral
    23. 23. Exercise: What’s Your Lifecycle?
    24. 24. Cohorts• A group of people• Share a common characteristic or experience• Exposed to similar conditions• Behaviour is tracked• Compared to other cohorts
    25. 25. Examples of Cohorts• Before & After Tests• Time Periods• Marketing Sources• Product Versions• Priming• Demographic Information
    26. 26. Exercise: Cohorts• Based on some of the suggestions in the previous slides what are some relevant cohorts for your organization you could begin tracking?
    27. 27. Special Cohort: A/B Testing• The Gold standard of web metrics• Usually comparing a hypothesis against an original• Big changes, one at a time• Statistical significance is important! http://getdatadriven.com/ab-significance-test• Ballpark = 100 conversions
    28. 28. Tools• Google Analytics – Funnels, but a bit limiting – A/B with “Content Experiments” – “Advanced Segments” can provide some help• KISS Metrics & Mix Panel are optimized for this sort of thing
    29. 29. Q: WHAT DO I TEST?
    30. 30. A: TALK TO PEOPLE
    31. 31. How to Talk to People• Usability Tests• Interviews• Surveys• Focus Groups
    32. 32. QUALITATIVE THEN QUANTITATIVE
    33. 33. Exercise: Put it All Together1. Talk to people2. Develop a hypothesis based on these conversations3. Identify the relevant macro-actionable metrics4. Design an experiment using cohorts and funnels5. Start tracking
    34. 34. Resources• Google Analytics – Setting up goals: http://analytics.blogspot.ca/2009/05/how-to-setup-goals-in-google- analytics.html – Setting up funnels: http://blog.kissmetrics.com/conversion-funnel-survival-guide/ – Setting up Cohorts: http://blog.rjmetrics.com/the-best-method-for-cohort-analysis-in- google-analytics/ – A/B Testing: http://support.google.com/analytics/bin/answer.py?hl=en&answer=1745147• KISS Metrics & Mix Panel – Their documentation site explains how to setup events, metrics, funnels, cohorts, and run A/B tests – http://support.kissmetrics.com/ – https://mixpanel.com/docs• Usability Testing &Interviews – http://www.amazon.ca/Rocket-Surgery-Made-Easy-Do-It-Yourself/dp/0321657292 – http://www.huffingtonpost.com/michael-b-fishbein/effective-customer-development- interviews_b_2189455.html
    35. 35. Thank You!• Chris Geoghegan• Product Website: http://peergiving.com• Charity Blog: http://ideas.peergiving.com• Twitter:chrisgeoghegan• Email:chris.geoghegan@peergiving.com

    ×