Making the strategy impactful
Simon Knight
User:Sjgknight
Vice chair, Wikimedia UK
Strategy process
Two senses of ‘making the strategy impactful’
1. We wanted a strategy to help us frame, measure,
and evaluate our activities
2. We also wanted the process of ‘making the
strategy’ to be impactful, we wanted to engage
stakeholders and not reinvent the wheel
Strategy process
Two pressures:
• Stability:
– Framing our work, building a narrative
– Need for clear targets against our strategy
– Reporting needs (e.g. FDC, charity law)
• Flexibility:
– Avoided priority setting for activities at
this stage
– Less concern for investment on particular
goals – this will come with time,
community engagement, & consideration
of impact
By 663highland (663highland) [GFDL (http://www.gnu.org/copyleft/fdl.html), CC-BY-SA-3.0
(http://creativecommons.org/licenses/by-sa/3.0/) or CC-BY-2.5
(http://creativecommons.org/licenses/by/2.5)], via Wikimedia Commons
https://upload.wikimedia.org/wikipedia/commons/e/ed/Horyu-ji09s3200.jpg
Model
Basic model
• We do things for a reason (we have goals)
• We run activities to meet those goals
• And to justify the activities, and assess their
success, we track activity outputs against our
goals (we measure)
Taken by: fir0002 | flagstaffotos.com.au
This file is published under the following Creative Commons license: Attribution
NonCommercial Unported 3.0
Strategy process
• Looked to:
– Other chapters (challenges)
– Other charities (but…)
– Other Open Knowledge groups
– The WMF (why didn’t we just use
that?)
WMF Goals
• To stabilize Wikimedia’s technical, financial and
organizational infrastructure
• To increase participation
• To improve quality
• To increase reach
• To encourage innovation
Then sets of concrete changes
Goals
We have five Strategic Goals that fall within our Mission:
• G1 Develop Open Knowledge
• G2 As a volunteer-led organisation, ensuring effective
use of the resources available to us:
– G2.1 Develop, involve and engage WMUK volunteers
– G2.2 Use effective and high quality governance and resource
management processes
• G3 Reduce barriers to accessing Open Knowledge
• G4 Encourage and support technological innovation
• G5 Develop, support, and engage with other Wikimedia
communities
E.g. G1 Develop Open Knowledge
• G1.1 The quantity of Open Knowledge
continues to increase
• G1.2 The quality of Open Knowledge
continues to improve
• G1.3 We are perceived as the go-to
organisation by UK GLAM, educational, and
other organisations who need support or
advice for the development of Open
Knowledge.
E.g. G1 Develop Open Knowledge
G1.1 The quantity of Open Knowledge continues
to increase
• Top down – which activities do this?
• Bottom up – which outcome is this activity
aimed at?
Outcomes without activities, or activities
without outcomes – a problem!
E.g. G1 Develop Open Knowledge
G1.1 The quantity of Open Knowledge continues
to increase
• General monitoring, but no goal
• Need to be clear on outcomes – n. of articles
(increase in open knowledge) v longevity of
diverse editors (G3/5 - thriving community),
we care about them all!
Easier reporting…?
• Highlighted need for impact methods – e.g.
volunteer questionnaire
• Need for tools – “increased access” via project
metrics (Kiwix, digital literacy, etc.) and
QRPedia scans, poor indicators
• Challenges e.g. on advocacy (e.g. ‘legislative
change for open content’) indicators
Lessons
Charting the territory
Chapters dialogue raised point: Diff orgs
are exploring new territory, do we share
a perspective?
By NASA [Public domain], via Wikimedia Commons
https://upload.wikimedia.org/wikipedia/commons/1/1d/Satellite_image_of_Cape_p
eninsula.jpg
Charting the territory
Chapters dialogue raised point: Diff orgs
are exploring new territory, do we share a
perspective?
But, can we share our mapping methods,
cartography relies on shared meaning
By U.S. Government; Uploader Zondor at en.wikipedia [Public domain], via Wikimedia Commons
https://upload.wikimedia.org/wikipedia/commons/d/d3/Sample-national-park-service-pictographs.gif
five Strategic Goals
• G1 Develop Open Knowledge
– Organisations different in each country
• G2 As a volunteer-led organisation, ensuring effective use of
the resources available to us inc developing volunteers (G2.1)
and using high quality governance (G2.2)
– Governance ‘standards’ different, volunteer needs (and groups)
also vary
• G3 Reduce barriers to accessing Open Knowledge
– Legal context and open-knowledge eco-system crucial
• G4 Encourage and support technological innovation
– Ditto
• G5 Develop, support, and engage with other Wikimedia
communities
– Bi-relational context matters
From Strategy to Impact
Measurement
Simon Knight
User:Sjgknight
Vice Chair Wikimedia UK
Model
Outcome indicators
• Input – Money, volunteer/staff time, materials
• Activity - An editathon
• Outputs - Attendees, articles edited, new articles
• Outcomes - Community growth, quality and
quantity of content increases
• Impact – Develop open knowledge
Outcome indicators
• Input teacher/student time
• Activity – a lesson on subject of ‘bias’
• Output – test score on ‘bias’ test
• Outcome – Learning about bias
• Impact – Learner confident about subject of bias
Inputs outputs challenge
Challenge:
• Tempting to measure “number of events doing
x”– but this is an input/activity
• Or set targets to increase “number of
members” – but what does this indicate?
• Outputs indicate – e.g. “n. of new articles”, “n.
of women leading events”
Outcomes and Evaluation
• Outcomes – what we want to achieve, what
our impact is
– Measured via output indicators
• Evaluation – an assessment of how effectively
we are addressing our outcomes
– Assessed via a variety of means, including
outputs. Thinking about how we run a programme
more effectively and in differing contexts
Mission
WMF Goals
• To stabilize Wikimedia’s technical, financial and
organizational infrastructure
• To increase participation
• To improve quality
• To increase reach
• To encourage innovation
Goals
We have five Strategic Goals that fall within our Mission:
• G1 Develop Open Knowledge
• G2 As a volunteer-led organisation, ensuring effective
use of the resources available to us:
– G2.1 Develop, involve and engage WMUK volunteers
– G2.2 Use effective and high quality governance and resource
management processes
• G3 Reduce barriers to accessing Open Knowledge
• G4 Encourage and support technological innovation
• G5 Develop, support, and engage with other Wikimedia
communities
E.g. G1 Develop Open Knowledge
• G1.1 The quantity of Open Knowledge
continues to increase
• G1.2 The quality of Open Knowledge
continues to improve
• G1.3 We are perceived as the go-to
organisation by UK GLAM, educational, and
other organisations who need support or
advice for the development of Open
Knowledge.
E.g. G1 Develop Open Knowledge
G1.1 The quantity of Open Knowledge continues
to increase
• Top down – which activities do this?
• Bottom up – which outcome is this activity
aimed at?
Outcomes without activities, or activities
without outcomes – a problem!
E.g. G1 Develop Open Knowledge
G1.1 The quantity of Open Knowledge continues
to increase
• Plausibility of controlled trials
• Top level v activity target setting
• Need to be clear on outcomes too – n. of
articles (increase in open knowledge) v
longevity of diverse editors (G3/5 - thriving
community)
• Highlighted need for impact methods – e.g.
volunteer questionnaire
• Need for tools – “increased access” via project
metrics (Kiwix, digital literacy, etc.) and
QRPedia scans, poor indicators
• Challenges e.g. on advocacy (e.g. ‘legislative
change for open content’) indicators
Lessons
Measuring impact –
co-ordination, and localisation
Making the most of tech?
Simon Knight
User:Sjgknight
Vice chair, Wikimedia UK
Basic model
• We do things for a reason (we have goals)
• We run activities to meet those goals
• And to justify the activities, and assess their
success, we evaluate them against our goals
(we measure)
Tenuous analogy time
• Think about a modern smart phone (device, OS, plug)
• Phone – strategy
• OS – activities
• Plug – impact
• What we do is tied to the framing, the capabilities,
restrictions, flexibility of the strategy.
• It also requires that we’re having impact – or else the
strategy gets old and tired, and our activities cease (the
phone dies)
• We should be free to remix as much as possible
• And I wonder to what extent we need shared
phones/OS/plugs…
Public domain
http://wikitravel.org/shared/File:Universal-plug-adaptor.jpg
WMMetrics, Wiki-Metrics, Metric-
Wikis, Wiki-trust wwwwwwwwww
• WMF – Wiki Metrics
• WMF – WM Metrics
• Other tools to appropriate – Wiki-trust, wiki-
todo, social-tools (snuggle), etc.
• Keeping track, sharing good practice, localising
– how do we do it?
e.g. WM-Metrics (France)
• List all files in category uploaded between
timestamp t1 and t2
• """Wikipedia article statistics.""“ (not yet t1
and t2)
• New articles
• File use
• Category use
five Strategic Goals
• G1 Develop Open Knowledge
– Organisations different in each country
• G2 As a volunteer-led organisation, ensuring effective use of
the resources available to us inc developing volunteers (G2.1)
and using high quality governance (G2.2)
– Governance ‘standards’ different, volunteer needs (and groups)
also vary
• G3 Reduce barriers to accessing Open Knowledge
– Legal context and open-knowledge eco-system crucial
• G4 Encourage and support technological innovation
– Ditto
• G5 Develop, support, and engage with other Wikimedia
communities
– Bi-relational context matters

Wikimedia Strategy - making it impactful, measuring impact, and thinking about tech and localisation

  • 1.
    Making the strategyimpactful Simon Knight User:Sjgknight Vice chair, Wikimedia UK
  • 2.
    Strategy process Two sensesof ‘making the strategy impactful’ 1. We wanted a strategy to help us frame, measure, and evaluate our activities 2. We also wanted the process of ‘making the strategy’ to be impactful, we wanted to engage stakeholders and not reinvent the wheel
  • 3.
    Strategy process Two pressures: •Stability: – Framing our work, building a narrative – Need for clear targets against our strategy – Reporting needs (e.g. FDC, charity law) • Flexibility: – Avoided priority setting for activities at this stage – Less concern for investment on particular goals – this will come with time, community engagement, & consideration of impact By 663highland (663highland) [GFDL (http://www.gnu.org/copyleft/fdl.html), CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/) or CC-BY-2.5 (http://creativecommons.org/licenses/by/2.5)], via Wikimedia Commons https://upload.wikimedia.org/wikipedia/commons/e/ed/Horyu-ji09s3200.jpg
  • 4.
  • 5.
    Basic model • Wedo things for a reason (we have goals) • We run activities to meet those goals • And to justify the activities, and assess their success, we track activity outputs against our goals (we measure)
  • 6.
    Taken by: fir0002| flagstaffotos.com.au This file is published under the following Creative Commons license: Attribution NonCommercial Unported 3.0 Strategy process • Looked to: – Other chapters (challenges) – Other charities (but…) – Other Open Knowledge groups – The WMF (why didn’t we just use that?)
  • 7.
    WMF Goals • Tostabilize Wikimedia’s technical, financial and organizational infrastructure • To increase participation • To improve quality • To increase reach • To encourage innovation Then sets of concrete changes
  • 8.
    Goals We have fiveStrategic Goals that fall within our Mission: • G1 Develop Open Knowledge • G2 As a volunteer-led organisation, ensuring effective use of the resources available to us: – G2.1 Develop, involve and engage WMUK volunteers – G2.2 Use effective and high quality governance and resource management processes • G3 Reduce barriers to accessing Open Knowledge • G4 Encourage and support technological innovation • G5 Develop, support, and engage with other Wikimedia communities
  • 9.
    E.g. G1 DevelopOpen Knowledge • G1.1 The quantity of Open Knowledge continues to increase • G1.2 The quality of Open Knowledge continues to improve • G1.3 We are perceived as the go-to organisation by UK GLAM, educational, and other organisations who need support or advice for the development of Open Knowledge.
  • 10.
    E.g. G1 DevelopOpen Knowledge G1.1 The quantity of Open Knowledge continues to increase • Top down – which activities do this? • Bottom up – which outcome is this activity aimed at? Outcomes without activities, or activities without outcomes – a problem!
  • 11.
    E.g. G1 DevelopOpen Knowledge G1.1 The quantity of Open Knowledge continues to increase • General monitoring, but no goal • Need to be clear on outcomes – n. of articles (increase in open knowledge) v longevity of diverse editors (G3/5 - thriving community), we care about them all!
  • 13.
  • 14.
    • Highlighted needfor impact methods – e.g. volunteer questionnaire • Need for tools – “increased access” via project metrics (Kiwix, digital literacy, etc.) and QRPedia scans, poor indicators • Challenges e.g. on advocacy (e.g. ‘legislative change for open content’) indicators Lessons
  • 15.
    Charting the territory Chaptersdialogue raised point: Diff orgs are exploring new territory, do we share a perspective? By NASA [Public domain], via Wikimedia Commons https://upload.wikimedia.org/wikipedia/commons/1/1d/Satellite_image_of_Cape_p eninsula.jpg
  • 16.
    Charting the territory Chaptersdialogue raised point: Diff orgs are exploring new territory, do we share a perspective? But, can we share our mapping methods, cartography relies on shared meaning By U.S. Government; Uploader Zondor at en.wikipedia [Public domain], via Wikimedia Commons https://upload.wikimedia.org/wikipedia/commons/d/d3/Sample-national-park-service-pictographs.gif
  • 17.
    five Strategic Goals •G1 Develop Open Knowledge – Organisations different in each country • G2 As a volunteer-led organisation, ensuring effective use of the resources available to us inc developing volunteers (G2.1) and using high quality governance (G2.2) – Governance ‘standards’ different, volunteer needs (and groups) also vary • G3 Reduce barriers to accessing Open Knowledge – Legal context and open-knowledge eco-system crucial • G4 Encourage and support technological innovation – Ditto • G5 Develop, support, and engage with other Wikimedia communities – Bi-relational context matters
  • 18.
    From Strategy toImpact Measurement Simon Knight User:Sjgknight Vice Chair Wikimedia UK
  • 19.
  • 20.
    Outcome indicators • Input– Money, volunteer/staff time, materials • Activity - An editathon • Outputs - Attendees, articles edited, new articles • Outcomes - Community growth, quality and quantity of content increases • Impact – Develop open knowledge
  • 21.
    Outcome indicators • Inputteacher/student time • Activity – a lesson on subject of ‘bias’ • Output – test score on ‘bias’ test • Outcome – Learning about bias • Impact – Learner confident about subject of bias
  • 22.
    Inputs outputs challenge Challenge: •Tempting to measure “number of events doing x”– but this is an input/activity • Or set targets to increase “number of members” – but what does this indicate? • Outputs indicate – e.g. “n. of new articles”, “n. of women leading events”
  • 23.
    Outcomes and Evaluation •Outcomes – what we want to achieve, what our impact is – Measured via output indicators • Evaluation – an assessment of how effectively we are addressing our outcomes – Assessed via a variety of means, including outputs. Thinking about how we run a programme more effectively and in differing contexts
  • 24.
  • 25.
    WMF Goals • Tostabilize Wikimedia’s technical, financial and organizational infrastructure • To increase participation • To improve quality • To increase reach • To encourage innovation
  • 26.
    Goals We have fiveStrategic Goals that fall within our Mission: • G1 Develop Open Knowledge • G2 As a volunteer-led organisation, ensuring effective use of the resources available to us: – G2.1 Develop, involve and engage WMUK volunteers – G2.2 Use effective and high quality governance and resource management processes • G3 Reduce barriers to accessing Open Knowledge • G4 Encourage and support technological innovation • G5 Develop, support, and engage with other Wikimedia communities
  • 27.
    E.g. G1 DevelopOpen Knowledge • G1.1 The quantity of Open Knowledge continues to increase • G1.2 The quality of Open Knowledge continues to improve • G1.3 We are perceived as the go-to organisation by UK GLAM, educational, and other organisations who need support or advice for the development of Open Knowledge.
  • 28.
    E.g. G1 DevelopOpen Knowledge G1.1 The quantity of Open Knowledge continues to increase • Top down – which activities do this? • Bottom up – which outcome is this activity aimed at? Outcomes without activities, or activities without outcomes – a problem!
  • 29.
    E.g. G1 DevelopOpen Knowledge G1.1 The quantity of Open Knowledge continues to increase • Plausibility of controlled trials • Top level v activity target setting • Need to be clear on outcomes too – n. of articles (increase in open knowledge) v longevity of diverse editors (G3/5 - thriving community)
  • 31.
    • Highlighted needfor impact methods – e.g. volunteer questionnaire • Need for tools – “increased access” via project metrics (Kiwix, digital literacy, etc.) and QRPedia scans, poor indicators • Challenges e.g. on advocacy (e.g. ‘legislative change for open content’) indicators Lessons
  • 32.
    Measuring impact – co-ordination,and localisation Making the most of tech? Simon Knight User:Sjgknight Vice chair, Wikimedia UK
  • 33.
    Basic model • Wedo things for a reason (we have goals) • We run activities to meet those goals • And to justify the activities, and assess their success, we evaluate them against our goals (we measure)
  • 34.
    Tenuous analogy time •Think about a modern smart phone (device, OS, plug) • Phone – strategy • OS – activities • Plug – impact • What we do is tied to the framing, the capabilities, restrictions, flexibility of the strategy. • It also requires that we’re having impact – or else the strategy gets old and tired, and our activities cease (the phone dies) • We should be free to remix as much as possible • And I wonder to what extent we need shared phones/OS/plugs… Public domain http://wikitravel.org/shared/File:Universal-plug-adaptor.jpg
  • 35.
    WMMetrics, Wiki-Metrics, Metric- Wikis,Wiki-trust wwwwwwwwww • WMF – Wiki Metrics • WMF – WM Metrics • Other tools to appropriate – Wiki-trust, wiki- todo, social-tools (snuggle), etc. • Keeping track, sharing good practice, localising – how do we do it?
  • 36.
    e.g. WM-Metrics (France) •List all files in category uploaded between timestamp t1 and t2 • """Wikipedia article statistics.""“ (not yet t1 and t2) • New articles • File use • Category use
  • 37.
    five Strategic Goals •G1 Develop Open Knowledge – Organisations different in each country • G2 As a volunteer-led organisation, ensuring effective use of the resources available to us inc developing volunteers (G2.1) and using high quality governance (G2.2) – Governance ‘standards’ different, volunteer needs (and groups) also vary • G3 Reduce barriers to accessing Open Knowledge – Legal context and open-knowledge eco-system crucial • G4 Encourage and support technological innovation – Ditto • G5 Develop, support, and engage with other Wikimedia communities – Bi-relational context matters