Successfully reported this slideshow.
Your SlideShare is downloading. ×

The problem with impact measurement in Civic Tech (Matt Stempeck and Micah L. Sifry - Civic Hall)

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
1.6 Ben Kröse
1.6 Ben Kröse
Loading in …3
×

Check these out next

1 of 22 Ad

The problem with impact measurement in Civic Tech (Matt Stempeck and Micah L. Sifry - Civic Hall)

Download to read offline

This was presented by Matt Stempeck and Micah L. Sifry from Civic Hall at the Impacts of Civic Technology Conference (TICTeC 2018) in Lisbon on 18th April 2018. You can find out more information about the conference here: http://tictec.mysociety.org/2018

This was presented by Matt Stempeck and Micah L. Sifry from Civic Hall at the Impacts of Civic Technology Conference (TICTeC 2018) in Lisbon on 18th April 2018. You can find out more information about the conference here: http://tictec.mysociety.org/2018

Advertisement
Advertisement

More Related Content

More from mysociety (20)

Advertisement

Recently uploaded (20)

Advertisement

The problem with impact measurement in Civic Tech (Matt Stempeck and Micah L. Sifry - Civic Hall)

  1. 1. The problem with impact measurement in Civic Tech Matt Stempeck & Micah Sifry #impactproblems
  2. 2. The 10 problems with impact measurement in Civic Tech Matt Stempeck & Micah Sifry #impactproblems
  3. 3. The 10 problems with impact measurement 1. We're all using different metrics 2. Sharing is irregular 3. Most projects don't reach most people 4. Different constituencies want different impacts 5. We don't evaluate relative to investment made 6. We don't evalue relative to the macro environment 7. Quantitative metrics can miss the objective 8. Qualitative cases are too often biased 9. Causality is hard to prove in social environments 10. Our externalities may eat us all
  4. 4. Working backwards Outcomes -> Proxies for progress -> Prove our causality
  5. 5. Working backwards Outcomes -> Proxies for progress -> Prove our causality
  6. 6. 1. We're all using different metrics API calls Users Time spent Money raised "Wins" Money saved Votes Installs Shares Media
  7. 7. 1. http://www.networkimpact.org/civictecheval/
  8. 8. 2. Sharing is irregular Bi-annual or quarterly reports with metrics and stories "Impact" page with stories Couldn't find 2015 and 2016 reports
  9. 9. 3. Most projects don't reach most people Platform Reach Percent of US population Percent of world population Change.org 237,426,190 users 72.8% 3.2% Jumo 1,000,000 users when acquired in 2011 3.2% 0.01% ⚡ ⚡ ⚡ Everyblock 145,000 users when acquired in 2009 0.05% 0.002% Civic Hall 14,518 event guests in 2017 0.004% 0.0002%
  10. 10. 3. Most projects don't reach most people Platform Reach Percent of US population Percent of world population Change.org 237,426,190 users 72.8% 3.2% Jumo 1,000,000 users when acquired in 2011 3.2% 0.01% ⚡ ⚡ ⚡ Everyblock 145,000 users when acquired in 2009 0.05% 0.002% Civic Hall 14,518 event guests in 2017 0.004% 0.0002% Matt's Twitter 6,000 0.002% 0.00008%
  11. 11. 4. Different constituencies want different impacts
  12. 12. 4. Different constituencies want different impacts
  13. 13. 4. Different constituencies want different impacts
  14. 14. 5. We don't evaluate relative to investment made INPUTS OUTCOMES THE WORK
  15. 15. 6. We don't evaluate relative to the environment
  16. 16. 7. Quantitative metrics can miss the objective 2 + 2 =
  17. 17. 8. Qualitative cases are too often biased
  18. 18. 9. Causality is hard to prove in social environments y x
  19. 19. 10. Our externalities may eat us all
  20. 20. Some imperfect proxies for outside researchers
  21. 21. This is one reason we are building the civic tech field guide
  22. 22. Track impact by platform bit.ly/organizecivictech Micah Sifry micah@civichall.org Matt Stempeck stempeck@gmail.com

×