SlideShare a Scribd company logo
1 of 36
Download to read offline
ALISA ZOMER / AZOMER@MIT.EDU
@AZOMER / MIT GOV/LAB
MITGOVLAB.ORG / @MITGOVLAB
[design sprint
workshop]
TICTEC 2019
ENGAGEMENT
METRICS FOR
SOCIAL IMPACT
Nepal (Alisa Zomer)
Philippines (Nina McMurry)
today, we are
design sprinting
1. CRASH course on theory + process
2. LEARN from POPVOX + Grassroot
3. SPRINT in small groups
4. SHARE in the report back
Nepal (Alisa Zomer)
SOCIAL CAPITAL
// shared knowledge,
norms, rules, expectations
EMPOWERMENT
political efficacy // the belief
that you can influence politics
COLLECTIVE
ACTION
// group of people taking
voluntary action
SOCIAL
MOVEMENTS
// networks with collective
identity; “sustained campaign
of claim-making”; in/formal
Designing for
Empowerment with
Better Impact Metrics
Erhardt Graeff, Olin College of Engineering
@erhardt

egraeff@olin.edu
erhardtgraeff.com
The Impacts of Civic Technology Conference (TICTeC)
OECD, Paris, France, 19–20 March 2019
impact?
How do we measure the impact of civic
technologies in terms of empowerment
and civic learning? And what is the
ethical responsibility of technologists
as stewards of democracy?
erhardtgraeff.com
Empowerment: accruing the
knowledge, skills, attitudes, and
habits necessary to have voice
and influence
2016 Stats
• 1M+ users
• 300+ local government partners
• 931K+ issues reported
• 818K+ issues fixed
external

political
efficacy
internal

political
efficacy
platform

political
efficacy
Questions use a 7-point
Likert scale: 



1 - Strongly Disagree

2 - Disagree

3 - Disagree Somewhat

4 - Neither Agree nor Disagree

5 - Agree Somewhat

6 - Agree

7 - Strongly Agree
example questions
internal political efficacy (IPE)
I consider myself well-qualified to participate in local affairs.
external political efficacy (EPE)
People like me DON'T have any say about what my local
government does.
platform political efficacy (PPE)
SeeClickFix and its partner platforms help make my local
government pay attention to what the people want.
study design
Invited by email all users “active” on a SeeClickFix
platform in the past year to take the survey and take
the same survey again one month later
Correlated survey responses with platform activity
Invited Survey 1 Survey 2
115493 9166 3482
key findings
• Women and those
identifying as other
gendered have lower
efficacy on average
• High IPE people are
likely also early adopters
• Engagement is related
to efficacy and platform
efficacy may predict
future engagement
• Closing issues and
response times really do
matter for a user’s EPE
and PPE
response from SCF staff
• Long-held assumptions
made “visible and
tangible” by findings
• Want to find more ways
to show users their
impact, but must
overcome local gov
resistance to more
transparency
• See opportunities for
ongoing measurement
of political efficacy, but
also see opportunities
for doing deep,
qualitative research in
key cities
a simple idea.started with
In 2011
a simple idea.
The problem:
Staffers overwhelmed
Don’t know if they are hearing from real constituents
Public doesn’t understand the legislative process
What if we…?
Listed all bills online
Required constituents to register with their real address
Delivered input in an easy-to-process format
Made the whole process transparent
POP·VOX
from “vox populi” (voice of the people)
Let’s call it
Original mission (2011):
To connect people and government, empower effective participation,
and create a transparent record that informs policymaking and fosters
accountable, responsive governing.
A mission-driven “civic startup”
To create technology that informs and
empowers people and makes
government work better for everyone.
Updated mission (2019)
that technology can help us be better
informed, more connected, and make
government work for everyone…
We believe
INFRASTRUCTURE for governing
Message Delivery as a Service

serving advocacy vendors
‣ Ensures messages
delivered electronically
‣ Lays groundwork for
better analytics tools
within lawmaker offices
Launched August 2017
Secure dashboards for
legislative staffers
‣ Provides workflow tools
and opportunities for
staffers to connect and
interact
‣ Resource hub to make
work by other good
government groups more
accessible
In beta
‣ Allows local officials to post
agenda items and receive
public input
‣ Provides electronic public
notice system
‣ Familiarizes constituents with
“who represents me” and
local issues
Civic engagement infrastructure
for local governments
Coming Q3-Q4 2019
Civic network
‣ Constituents “follow”
lawmakers, jurisdictions,
and organizations
‣ Provides non-ad social
network for sharing
information on issues and
connecting with others
‣ Allows for constituent-
only public posts
Coming Q2-Q3 2019
Nonpartisan platform for civic
engagement and governing
‣ Lists all pending bills
‣ Organizations create
profile, post positions
supporting or opposing
bills
‣ Constituents create
profile, post positions
supporting or opposing
bills
‣ Messages delivered to
lawmakers
Launched August 2017
LOCAL Civic engagement infrastructure for local governments
LOCAL
LOCAL (Planned) 2019 Pilot Jurisdictions
San Mateo County, CA
Fayetteville, AR
Madison County, TN
Chicago Public Schools

Chicago, IL Philadelphia, PA

(partnership with
OpenAccess PHL)
Oakland, CA
Sebastopol, CA
‣ An observable, transparent
network (allowing anyone to
view activity but limiting
interactions to constituents)
‣ Prioritizing individual control
of data, without algorithmic
manipulation, and require real
names at sign up with
verification
LOCAL
The Goals
‣ Increase awareness and understanding of local issues
‣ Increase the number and diversity of participants engaging on
local issues
‣ Improve resident satisfaction with interactions with officials
‣ Increase awareness of “who represents me” and “how local
government works”
Improve tone, truth, and trust in online civic engagement
LOCAL
Design Constraints
‣ Must stay within basic POPVOX structure
‣ Most changes will also impact broader POPVOX structure (state
and federal)
‣ We do not currently collect demographic data – should we?
‣ Must work for varied jurisdictions: urban and rural; large city vs.
small city council
LOCAL
Problem Statement
LOCAL
How might we measure participants’ understanding of
government processes, comfort engaging, and sense that their
voice matters?
Grassroot
TICTeC 2019
Background
● Grassroot was founded in South Africa in late 2015
● We enable communities in South Africa to take collective
action and give effect to participatory democracy by
building and deploying simple technology
● Our team is half development, half field-work, working
with partner communities as we build, deploy and iterate
● We’ve reached over 350k people acting 2k+ times / month
Platform
● Our platform runs on USSD, Android, responsive web and
now WhatsApp, all syncing to a common back-end
● We cross the digital divide, to allow anyone on any phone,
enabling them to call meetings, ask for volunteers, sign
petitions, take polls, and issue press alerts
● Our ‘net promoter scores’ are similar to Amazon and
Netflix, with 80% of growth by word of mouth
Platform Screenshots & Description (1/2)
● USSD allows even feature phones, or smart
phones without data, to set up a community
group, recruit others, call meetings, etc
● Simple ‘join codes’ allow people to sign up
for alerts in large numbers
● Easy to use menu flows heavily user tested,
and available in multiple (5) languages
Platform Screenshots (2/2)
● Web app for complex
functions, e.g., votes,
petitions, involving
tens of thousands
● WhatsApp bot under
development, allows
signing petitions,
getting info on sexual
assault clinics, etc.
Goal
● We focus on two problems:
1. It’s too costly and difficult for the poor to organize
2. It’s too hard for local leaders to access information they can act
on and connect to each other in a meaningful way
● Grassroot tries to address those problems by creating
simple, mobile-first tools that increase the density and
effectiveness of collective action
Design Constraints
● Our traditional definition of impact has been:
● How much more often are people acting together? I.e., how much
has the density of self-organizing among the poor increased?
● How much more often are they getting results? I.e., how much
has the effectiveness of that organizing increased?
● The first we can measure using data and basic surveys
(‘lean data’), but we are finding it is a very poor proxy for
the second (i.e., our theory of change is broken)
Problem Statement
● We are pretty sure that our existing tools increase the
density of collective action but not the effectiveness
● That breaks our theory of change, so we are trying new
ideas to create impact, e.g., via short content in
messaging groups connecting organizers
● Need to know quickly if we are having impact, but hard to
measure quickly the impact, esp of changes in capacity
Design prompt
● How might we measure the ways in which social
movement organizations grow and build capacity?
● (Constraints: structurally unresponsive government and
organizations with early activity but subsequent decline)
Design Prompts
• POPVOX / How might we measure participants’
understanding of government processes, comfort
engaging, and sense that their voice matters?
• Grassroot / How might we measure the ways in which
social movement organizations grow and build capacity?
(constraints: unresponsive government and organizations
with strong initial momentum and steep decline)
[thank you for sprinting]
Alisa Zomer / azomer@mit.edu / @azomer
Erhardt Graeff / egraeff@olin.edu / @erhardt
Marci Harris / marci@popvox.com / @marcidale
Luke Jordan / luke.jordan@grassroot.org.za / @grassrootza

More Related Content

What's hot

Networked engagement (draft)(april 2013)
Networked engagement (draft)(april 2013)Networked engagement (draft)(april 2013)
Networked engagement (draft)(april 2013)
Catherine Howe
 
Solace digital leadership Masterclass (June 2014)
Solace digital leadership Masterclass (June 2014)Solace digital leadership Masterclass (June 2014)
Solace digital leadership Masterclass (June 2014)
Catherine Howe
 
Collaborative manifesto copy
Collaborative manifesto copyCollaborative manifesto copy
Collaborative manifesto copy
Catherine Howe
 
Digital civic space may 2013
Digital civic space may 2013Digital civic space may 2013
Digital civic space may 2013
Catherine Howe
 
eDemocracy2012 Simon Delakorda Political_informatics-how_should_civil_society...
eDemocracy2012 Simon Delakorda Political_informatics-how_should_civil_society...eDemocracy2012 Simon Delakorda Political_informatics-how_should_civil_society...
eDemocracy2012 Simon Delakorda Political_informatics-how_should_civil_society...
e-Democracy Conference
 

What's hot (17)

The Civic Tech timeline: a recent history (Matt Stempeck and Micah L. Sifry, ...
The Civic Tech timeline: a recent history (Matt Stempeck and Micah L. Sifry, ...The Civic Tech timeline: a recent history (Matt Stempeck and Micah L. Sifry, ...
The Civic Tech timeline: a recent history (Matt Stempeck and Micah L. Sifry, ...
 
Bright spots slideshare - final
Bright spots slideshare - finalBright spots slideshare - final
Bright spots slideshare - final
 
Making localism a reality
Making localism a realityMaking localism a reality
Making localism a reality
 
i-teams: The teams and funds making innovation happen in governments around t...
i-teams: The teams and funds making innovation happen in governments around t...i-teams: The teams and funds making innovation happen in governments around t...
i-teams: The teams and funds making innovation happen in governments around t...
 
lessons in Gov 2.0: building strategy from the inside out
lessons in Gov 2.0: building strategy from the inside outlessons in Gov 2.0: building strategy from the inside out
lessons in Gov 2.0: building strategy from the inside out
 
The Collaboration Project: Building Open, Participatory and Collaborative Gov...
The Collaboration Project: Building Open, Participatory and Collaborative Gov...The Collaboration Project: Building Open, Participatory and Collaborative Gov...
The Collaboration Project: Building Open, Participatory and Collaborative Gov...
 
Networked engagement (draft)(april 2013)
Networked engagement (draft)(april 2013)Networked engagement (draft)(april 2013)
Networked engagement (draft)(april 2013)
 
Scrutiny in the spotlight: Networked Councillors and the Scrutiny process
Scrutiny in the spotlight:  Networked Councillors and the Scrutiny processScrutiny in the spotlight:  Networked Councillors and the Scrutiny process
Scrutiny in the spotlight: Networked Councillors and the Scrutiny process
 
Solace digital leadership Masterclass (June 2014)
Solace digital leadership Masterclass (June 2014)Solace digital leadership Masterclass (June 2014)
Solace digital leadership Masterclass (June 2014)
 
Collaborative manifesto copy
Collaborative manifesto copyCollaborative manifesto copy
Collaborative manifesto copy
 
Adso presentation january 2013
Adso presentation january 2013Adso presentation january 2013
Adso presentation january 2013
 
Digital civic space may 2013
Digital civic space may 2013Digital civic space may 2013
Digital civic space may 2013
 
Policing 2.0 (15.10.10)(final)
Policing 2.0 (15.10.10)(final)Policing 2.0 (15.10.10)(final)
Policing 2.0 (15.10.10)(final)
 
Flavia Marzano Open Expo Europe April 21st 2020 Digital and Open Transformation
Flavia Marzano Open Expo Europe April 21st 2020 Digital and Open TransformationFlavia Marzano Open Expo Europe April 21st 2020 Digital and Open Transformation
Flavia Marzano Open Expo Europe April 21st 2020 Digital and Open Transformation
 
eDemocracy2012 Simon Delakorda Political_informatics-how_should_civil_society...
eDemocracy2012 Simon Delakorda Political_informatics-how_should_civil_society...eDemocracy2012 Simon Delakorda Political_informatics-how_should_civil_society...
eDemocracy2012 Simon Delakorda Political_informatics-how_should_civil_society...
 
2012_Philadelphia End of Year Report
2012_Philadelphia End of Year Report2012_Philadelphia End of Year Report
2012_Philadelphia End of Year Report
 
Finding a Foothold: How Nonprofit News Ventures Seek Sustainability
Finding a Foothold: How Nonprofit News Ventures Seek SustainabilityFinding a Foothold: How Nonprofit News Ventures Seek Sustainability
Finding a Foothold: How Nonprofit News Ventures Seek Sustainability
 

Similar to [Design Sprint Workshop] Engagement Metrics for Social Impact: Alisa Zomer (MIT GOV/LAB, US), Erhardt Graeff (Olin College of Engineering, US), Luke Jordan (Grassroot, South Africa) & Marci Harris (POPVOX, US)

Crowd-programmed initiatives (Dr Adrian Flint, Uni Bristol, and Chris Meyer z...
Crowd-programmed initiatives (Dr Adrian Flint, Uni Bristol, and Chris Meyer z...Crowd-programmed initiatives (Dr Adrian Flint, Uni Bristol, and Chris Meyer z...
Crowd-programmed initiatives (Dr Adrian Flint, Uni Bristol, and Chris Meyer z...
ALNAP
 
New+Voices-Better-Communities-Leveraging-Technology-LGMA_2014
New+Voices-Better-Communities-Leveraging-Technology-LGMA_2014New+Voices-Better-Communities-Leveraging-Technology-LGMA_2014
New+Voices-Better-Communities-Leveraging-Technology-LGMA_2014
Sarah Bishop
 
Building a Sustainable Citizen-Centric Smart City Approach
Building a Sustainable Citizen-Centric Smart City ApproachBuilding a Sustainable Citizen-Centric Smart City Approach
Building a Sustainable Citizen-Centric Smart City Approach
Dr. Mazlan Abbas
 

Similar to [Design Sprint Workshop] Engagement Metrics for Social Impact: Alisa Zomer (MIT GOV/LAB, US), Erhardt Graeff (Olin College of Engineering, US), Luke Jordan (Grassroot, South Africa) & Marci Harris (POPVOX, US) (20)

Meeting cuts&big society_challenge
Meeting cuts&big society_challengeMeeting cuts&big society_challenge
Meeting cuts&big society_challenge
 
Impacts of Open Data Standards on Transparency Tools - Khairil Yusof (Sinar P...
Impacts of Open Data Standards on Transparency Tools - Khairil Yusof (Sinar P...Impacts of Open Data Standards on Transparency Tools - Khairil Yusof (Sinar P...
Impacts of Open Data Standards on Transparency Tools - Khairil Yusof (Sinar P...
 
Activeshehri
ActiveshehriActiveshehri
Activeshehri
 
The State of Civic Tech Impact in the US
The State of Civic Tech Impact in the USThe State of Civic Tech Impact in the US
The State of Civic Tech Impact in the US
 
An introduction to Intensive Engagement
An introduction to Intensive EngagementAn introduction to Intensive Engagement
An introduction to Intensive Engagement
 
Using online tools queensland slides
Using online tools   queensland slidesUsing online tools   queensland slides
Using online tools queensland slides
 
Crowd-programmed initiatives (Dr Adrian Flint, Uni Bristol, and Chris Meyer z...
Crowd-programmed initiatives (Dr Adrian Flint, Uni Bristol, and Chris Meyer z...Crowd-programmed initiatives (Dr Adrian Flint, Uni Bristol, and Chris Meyer z...
Crowd-programmed initiatives (Dr Adrian Flint, Uni Bristol, and Chris Meyer z...
 
navigating the new social: Gov 2.0 and community engagement
navigating the new social: Gov 2.0 and community engagementnavigating the new social: Gov 2.0 and community engagement
navigating the new social: Gov 2.0 and community engagement
 
New+Voices-Better-Communities-Leveraging-Technology-LGMA_2014
New+Voices-Better-Communities-Leveraging-Technology-LGMA_2014New+Voices-Better-Communities-Leveraging-Technology-LGMA_2014
New+Voices-Better-Communities-Leveraging-Technology-LGMA_2014
 
Government Citizen Engagement Survival Guide
Government Citizen Engagement Survival GuideGovernment Citizen Engagement Survival Guide
Government Citizen Engagement Survival Guide
 
CTDC 21st Century Solutions
CTDC 21st Century SolutionsCTDC 21st Century Solutions
CTDC 21st Century Solutions
 
Martinez treasury 4 11
Martinez treasury 4 11Martinez treasury 4 11
Martinez treasury 4 11
 
Five years of Citizen Budget: an impact assessment (Christian Medina-Ramirez ...
Five years of Citizen Budget: an impact assessment (Christian Medina-Ramirez ...Five years of Citizen Budget: an impact assessment (Christian Medina-Ramirez ...
Five years of Citizen Budget: an impact assessment (Christian Medina-Ramirez ...
 
Open Data - context of Open Government in Edmonton
Open Data - context of Open Government in EdmontonOpen Data - context of Open Government in Edmonton
Open Data - context of Open Government in Edmonton
 
WEPOLITICS. the political social network
WEPOLITICS. the political social networkWEPOLITICS. the political social network
WEPOLITICS. the political social network
 
Making The Connection Part 2 (Government and Citizens)
Making The Connection Part 2 (Government and Citizens)Making The Connection Part 2 (Government and Citizens)
Making The Connection Part 2 (Government and Citizens)
 
civ.works: The Comprehensive Platform for Participatory Democracy and Budgeting.
civ.works: The Comprehensive Platform for Participatory Democracy and Budgeting.civ.works: The Comprehensive Platform for Participatory Democracy and Budgeting.
civ.works: The Comprehensive Platform for Participatory Democracy and Budgeting.
 
Building a Sustainable Citizen-Centric Smart City Approach
Building a Sustainable Citizen-Centric Smart City ApproachBuilding a Sustainable Citizen-Centric Smart City Approach
Building a Sustainable Citizen-Centric Smart City Approach
 
civ.works: A powerful, private social platform for civic engagement.
civ.works: A powerful, private social platform for civic engagement.civ.works: A powerful, private social platform for civic engagement.
civ.works: A powerful, private social platform for civic engagement.
 
eSwarajIntro
eSwarajIntroeSwarajIntro
eSwarajIntro
 

More from mysociety

More from mysociety (20)

Regulating Access to Information - Alex Parsons, mySociety (UK)
Regulating Access to Information - Alex Parsons, mySociety (UK)Regulating Access to Information - Alex Parsons, mySociety (UK)
Regulating Access to Information - Alex Parsons, mySociety (UK)
 
Watch this space (and pay for it): Alaveteli-driven exposure of the misuse of...
Watch this space (and pay for it): Alaveteli-driven exposure of the misuse of...Watch this space (and pay for it): Alaveteli-driven exposure of the misuse of...
Watch this space (and pay for it): Alaveteli-driven exposure of the misuse of...
 
What are the effects of OpenStreetMapping on the mappers themselves? - Aishwo...
What are the effects of OpenStreetMapping on the mappers themselves? - Aishwo...What are the effects of OpenStreetMapping on the mappers themselves? - Aishwo...
What are the effects of OpenStreetMapping on the mappers themselves? - Aishwo...
 
#PlanTech and the geospatial ecosystem - Ben Fowkes (Delib)
#PlanTech and the geospatial ecosystem - Ben Fowkes (Delib)#PlanTech and the geospatial ecosystem - Ben Fowkes (Delib)
#PlanTech and the geospatial ecosystem - Ben Fowkes (Delib)
 
Open data for local self governance: learnings from five Ukrainian cities - N...
Open data for local self governance: learnings from five Ukrainian cities - N...Open data for local self governance: learnings from five Ukrainian cities - N...
Open data for local self governance: learnings from five Ukrainian cities - N...
 
Digital Champions: community led development monitoring in Tanzania - Janet C...
Digital Champions: community led development monitoring in Tanzania - Janet C...Digital Champions: community led development monitoring in Tanzania - Janet C...
Digital Champions: community led development monitoring in Tanzania - Janet C...
 
Don’t build it: a practical guide for those building Civic Tech - Luke Jordan...
Don’t build it: a practical guide for those building Civic Tech - Luke Jordan...Don’t build it: a practical guide for those building Civic Tech - Luke Jordan...
Don’t build it: a practical guide for those building Civic Tech - Luke Jordan...
 
It takes two: when citizens and Congress Members deliberate online - Samantha...
It takes two: when citizens and Congress Members deliberate online - Samantha...It takes two: when citizens and Congress Members deliberate online - Samantha...
It takes two: when citizens and Congress Members deliberate online - Samantha...
 
Understanding the small hurdles that block community engagement, with behavio...
Understanding the small hurdles that block community engagement, with behavio...Understanding the small hurdles that block community engagement, with behavio...
Understanding the small hurdles that block community engagement, with behavio...
 
Our COVID consultation journey: from a small initiative to the desk of the pr...
Our COVID consultation journey: from a small initiative to the desk of the pr...Our COVID consultation journey: from a small initiative to the desk of the pr...
Our COVID consultation journey: from a small initiative to the desk of the pr...
 
Keeping track of open data in times of political change - David Zamora (Open ...
Keeping track of open data in times of political change - David Zamora (Open ...Keeping track of open data in times of political change - David Zamora (Open ...
Keeping track of open data in times of political change - David Zamora (Open ...
 
Civic tech vs. illicit pharmacies - Ibraheem Saleem (Code for Pakistan)
Civic tech vs. illicit pharmacies - Ibraheem Saleem (Code for Pakistan)Civic tech vs. illicit pharmacies - Ibraheem Saleem (Code for Pakistan)
Civic tech vs. illicit pharmacies - Ibraheem Saleem (Code for Pakistan)
 
Find that Charity: a tool to help find charities and improve charity data - D...
Find that Charity: a tool to help find charities and improve charity data - D...Find that Charity: a tool to help find charities and improve charity data - D...
Find that Charity: a tool to help find charities and improve charity data - D...
 
Civic tech for smartphone beginners: is the future binary? - Arran Leonard (I...
Civic tech for smartphone beginners: is the future binary? - Arran Leonard (I...Civic tech for smartphone beginners: is the future binary? - Arran Leonard (I...
Civic tech for smartphone beginners: is the future binary? - Arran Leonard (I...
 
How to monitor emergency procurement with open data: lessons from 12 countrie...
How to monitor emergency procurement with open data: lessons from 12 countrie...How to monitor emergency procurement with open data: lessons from 12 countrie...
How to monitor emergency procurement with open data: lessons from 12 countrie...
 
Accountability through petitions: Elections 2019 - Gaston Wright & Leandro As...
Accountability through petitions: Elections 2019 - Gaston Wright & Leandro As...Accountability through petitions: Elections 2019 - Gaston Wright & Leandro As...
Accountability through petitions: Elections 2019 - Gaston Wright & Leandro As...
 
Openly available air quality data: not just blue-sky thinking - Sruti Modekur...
Openly available air quality data: not just blue-sky thinking - Sruti Modekur...Openly available air quality data: not just blue-sky thinking - Sruti Modekur...
Openly available air quality data: not just blue-sky thinking - Sruti Modekur...
 
How to last in civic tech (especially now) - Matthew Stempeck & Micah L. Sifr...
How to last in civic tech (especially now) - Matthew Stempeck & Micah L. Sifr...How to last in civic tech (especially now) - Matthew Stempeck & Micah L. Sifr...
How to last in civic tech (especially now) - Matthew Stempeck & Micah L. Sifr...
 
Future of tech and democracy at the city of Reykjavík - Sigurlaug Anna Jóhann...
Future of tech and democracy at the city of Reykjavík - Sigurlaug Anna Jóhann...Future of tech and democracy at the city of Reykjavík - Sigurlaug Anna Jóhann...
Future of tech and democracy at the city of Reykjavík - Sigurlaug Anna Jóhann...
 
Lessons learned from building democracy’s database - Stacy Henderson (Cicero,...
Lessons learned from building democracy’s database - Stacy Henderson (Cicero,...Lessons learned from building democracy’s database - Stacy Henderson (Cicero,...
Lessons learned from building democracy’s database - Stacy Henderson (Cicero,...
 

Recently uploaded

Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
UK Journal
 

Recently uploaded (20)

Using IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & IrelandUsing IESVE for Room Loads Analysis - UK & Ireland
Using IESVE for Room Loads Analysis - UK & Ireland
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
 
Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024Long journey of Ruby Standard library at RubyKaigi 2024
Long journey of Ruby Standard library at RubyKaigi 2024
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptxBT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
BT & Neo4j _ How Knowledge Graphs help BT deliver Digital Transformation.pptx
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 

[Design Sprint Workshop] Engagement Metrics for Social Impact: Alisa Zomer (MIT GOV/LAB, US), Erhardt Graeff (Olin College of Engineering, US), Luke Jordan (Grassroot, South Africa) & Marci Harris (POPVOX, US)

  • 1. ALISA ZOMER / AZOMER@MIT.EDU @AZOMER / MIT GOV/LAB MITGOVLAB.ORG / @MITGOVLAB [design sprint workshop] TICTEC 2019 ENGAGEMENT METRICS FOR SOCIAL IMPACT Nepal (Alisa Zomer)
  • 2. Philippines (Nina McMurry) today, we are design sprinting 1. CRASH course on theory + process 2. LEARN from POPVOX + Grassroot 3. SPRINT in small groups 4. SHARE in the report back Nepal (Alisa Zomer)
  • 3. SOCIAL CAPITAL // shared knowledge, norms, rules, expectations EMPOWERMENT political efficacy // the belief that you can influence politics COLLECTIVE ACTION // group of people taking voluntary action SOCIAL MOVEMENTS // networks with collective identity; “sustained campaign of claim-making”; in/formal
  • 4. Designing for Empowerment with Better Impact Metrics Erhardt Graeff, Olin College of Engineering @erhardt
 egraeff@olin.edu erhardtgraeff.com The Impacts of Civic Technology Conference (TICTeC) OECD, Paris, France, 19–20 March 2019
  • 6. How do we measure the impact of civic technologies in terms of empowerment and civic learning? And what is the ethical responsibility of technologists as stewards of democracy? erhardtgraeff.com
  • 7. Empowerment: accruing the knowledge, skills, attitudes, and habits necessary to have voice and influence
  • 8. 2016 Stats • 1M+ users • 300+ local government partners • 931K+ issues reported • 818K+ issues fixed
  • 10. Questions use a 7-point Likert scale: 
 
 1 - Strongly Disagree
 2 - Disagree
 3 - Disagree Somewhat
 4 - Neither Agree nor Disagree
 5 - Agree Somewhat
 6 - Agree
 7 - Strongly Agree example questions internal political efficacy (IPE) I consider myself well-qualified to participate in local affairs. external political efficacy (EPE) People like me DON'T have any say about what my local government does. platform political efficacy (PPE) SeeClickFix and its partner platforms help make my local government pay attention to what the people want.
  • 11. study design Invited by email all users “active” on a SeeClickFix platform in the past year to take the survey and take the same survey again one month later Correlated survey responses with platform activity Invited Survey 1 Survey 2 115493 9166 3482
  • 12. key findings • Women and those identifying as other gendered have lower efficacy on average • High IPE people are likely also early adopters • Engagement is related to efficacy and platform efficacy may predict future engagement • Closing issues and response times really do matter for a user’s EPE and PPE
  • 13. response from SCF staff • Long-held assumptions made “visible and tangible” by findings • Want to find more ways to show users their impact, but must overcome local gov resistance to more transparency • See opportunities for ongoing measurement of political efficacy, but also see opportunities for doing deep, qualitative research in key cities
  • 14. a simple idea.started with In 2011
  • 15. a simple idea. The problem: Staffers overwhelmed Don’t know if they are hearing from real constituents Public doesn’t understand the legislative process What if we…? Listed all bills online Required constituents to register with their real address Delivered input in an easy-to-process format Made the whole process transparent POP·VOX from “vox populi” (voice of the people) Let’s call it
  • 16. Original mission (2011): To connect people and government, empower effective participation, and create a transparent record that informs policymaking and fosters accountable, responsive governing. A mission-driven “civic startup” To create technology that informs and empowers people and makes government work better for everyone. Updated mission (2019)
  • 17. that technology can help us be better informed, more connected, and make government work for everyone… We believe
  • 18. INFRASTRUCTURE for governing Message Delivery as a Service
 serving advocacy vendors ‣ Ensures messages delivered electronically ‣ Lays groundwork for better analytics tools within lawmaker offices Launched August 2017 Secure dashboards for legislative staffers ‣ Provides workflow tools and opportunities for staffers to connect and interact ‣ Resource hub to make work by other good government groups more accessible In beta ‣ Allows local officials to post agenda items and receive public input ‣ Provides electronic public notice system ‣ Familiarizes constituents with “who represents me” and local issues Civic engagement infrastructure for local governments Coming Q3-Q4 2019 Civic network ‣ Constituents “follow” lawmakers, jurisdictions, and organizations ‣ Provides non-ad social network for sharing information on issues and connecting with others ‣ Allows for constituent- only public posts Coming Q2-Q3 2019 Nonpartisan platform for civic engagement and governing ‣ Lists all pending bills ‣ Organizations create profile, post positions supporting or opposing bills ‣ Constituents create profile, post positions supporting or opposing bills ‣ Messages delivered to lawmakers Launched August 2017
  • 19. LOCAL Civic engagement infrastructure for local governments
  • 20. LOCAL
  • 21. LOCAL (Planned) 2019 Pilot Jurisdictions San Mateo County, CA Fayetteville, AR Madison County, TN Chicago Public Schools
 Chicago, IL Philadelphia, PA
 (partnership with OpenAccess PHL) Oakland, CA Sebastopol, CA
  • 22. ‣ An observable, transparent network (allowing anyone to view activity but limiting interactions to constituents) ‣ Prioritizing individual control of data, without algorithmic manipulation, and require real names at sign up with verification LOCAL
  • 23. The Goals ‣ Increase awareness and understanding of local issues ‣ Increase the number and diversity of participants engaging on local issues ‣ Improve resident satisfaction with interactions with officials ‣ Increase awareness of “who represents me” and “how local government works” Improve tone, truth, and trust in online civic engagement LOCAL
  • 24. Design Constraints ‣ Must stay within basic POPVOX structure ‣ Most changes will also impact broader POPVOX structure (state and federal) ‣ We do not currently collect demographic data – should we? ‣ Must work for varied jurisdictions: urban and rural; large city vs. small city council LOCAL
  • 25. Problem Statement LOCAL How might we measure participants’ understanding of government processes, comfort engaging, and sense that their voice matters?
  • 27. Background ● Grassroot was founded in South Africa in late 2015 ● We enable communities in South Africa to take collective action and give effect to participatory democracy by building and deploying simple technology ● Our team is half development, half field-work, working with partner communities as we build, deploy and iterate ● We’ve reached over 350k people acting 2k+ times / month
  • 28. Platform ● Our platform runs on USSD, Android, responsive web and now WhatsApp, all syncing to a common back-end ● We cross the digital divide, to allow anyone on any phone, enabling them to call meetings, ask for volunteers, sign petitions, take polls, and issue press alerts ● Our ‘net promoter scores’ are similar to Amazon and Netflix, with 80% of growth by word of mouth
  • 29. Platform Screenshots & Description (1/2) ● USSD allows even feature phones, or smart phones without data, to set up a community group, recruit others, call meetings, etc ● Simple ‘join codes’ allow people to sign up for alerts in large numbers ● Easy to use menu flows heavily user tested, and available in multiple (5) languages
  • 30. Platform Screenshots (2/2) ● Web app for complex functions, e.g., votes, petitions, involving tens of thousands ● WhatsApp bot under development, allows signing petitions, getting info on sexual assault clinics, etc.
  • 31. Goal ● We focus on two problems: 1. It’s too costly and difficult for the poor to organize 2. It’s too hard for local leaders to access information they can act on and connect to each other in a meaningful way ● Grassroot tries to address those problems by creating simple, mobile-first tools that increase the density and effectiveness of collective action
  • 32. Design Constraints ● Our traditional definition of impact has been: ● How much more often are people acting together? I.e., how much has the density of self-organizing among the poor increased? ● How much more often are they getting results? I.e., how much has the effectiveness of that organizing increased? ● The first we can measure using data and basic surveys (‘lean data’), but we are finding it is a very poor proxy for the second (i.e., our theory of change is broken)
  • 33. Problem Statement ● We are pretty sure that our existing tools increase the density of collective action but not the effectiveness ● That breaks our theory of change, so we are trying new ideas to create impact, e.g., via short content in messaging groups connecting organizers ● Need to know quickly if we are having impact, but hard to measure quickly the impact, esp of changes in capacity
  • 34. Design prompt ● How might we measure the ways in which social movement organizations grow and build capacity? ● (Constraints: structurally unresponsive government and organizations with early activity but subsequent decline)
  • 35. Design Prompts • POPVOX / How might we measure participants’ understanding of government processes, comfort engaging, and sense that their voice matters? • Grassroot / How might we measure the ways in which social movement organizations grow and build capacity? (constraints: unresponsive government and organizations with strong initial momentum and steep decline)
  • 36. [thank you for sprinting] Alisa Zomer / azomer@mit.edu / @azomer Erhardt Graeff / egraeff@olin.edu / @erhardt Marci Harris / marci@popvox.com / @marcidale Luke Jordan / luke.jordan@grassroot.org.za / @grassrootza