Last month I created an assessment for an international foundation on the impact of their video grants. The assessment (below) includes analysis and visualization of how 37 grantees used video in their advocacy work, and the effects of that use.
For this project I first analyzed grantee self-reports on how they were using video. I then developed a model comparing grantee activities and outcomes and finally visualized this analysis in the form of an easily skimmable “report card” for each grantee. This public version includes 10 representative reports from among the original 37.
by Mary Joyce with John Kennedy for the Open Society Foundation | December 2015 (v1.1) | Part 1
Image: Flickr/Very Quiet
VIDEO IMPACT ASSESSMENT
Closing the Impact Gap
Executive Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . slide 3
Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .slide 4
Recommendations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .slide 7
Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .slide 9
Grantee Report Cards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . slide 14
Appendices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .slide 54
This public version is anonymized to remove information identifying the nonproﬁt organizations being assessed and the foundation for which
the assessment was prepared. The purpose of this anonymization is to protect the privacy of the entities involved and instead put the focus
on the method of analysis and its presentation. Redacted content is covered with gray boxes.
About the Public Version
* Assessment Part 1 includes a 51% non-random sample of 37 video-producing grantees.Image: Flickr/Very Quiet
•This assessment of 2009-2015 grantees* revealed a gap between video content
and video impact.
•Despite strengths in storytelling, production, and distribution, grantees often fail to
beneﬁt from their videos.
•Strategy is often rudimentary.While the vast majority of grantees understand the
need for a goal and audience, most videos lack theories of change, precise audience
targeting, and systematic measurement of impact.
•To meet these challenges, the Media and Narratives Division should adopt a
population beneﬁt strategy that would focus funding priorities and methods of
proposal evaluation and grantee support on videos that aim to achieve concrete
beneﬁts for marginalized populations.
Impact is Little Understood & Poorly Measured
Lack of Consensus on Impact
•Impact claims were diverse, indicating little consensus on what
constitutes video impact.
•27% of grantees claimed that the video beneﬁted the
population on whose behalf their organization advocates, such as
improved health services for Roma or kinder community
treatment of HIV+ women.
•14% claimed the video beneﬁted their organization, such as
helping them raise money or get invitations to speak at
•32% of grantees claimed an effect on their audience, such as
raising awareness or increasing understanding, though not a direct
beneﬁt to themselves or their population.
•The remaining 27% made no claim of video effect or beneﬁt,
instead describing the scope of distribution or the quality of the
Audience Effect w/o Beneﬁt
No Effect or Beneﬁt Claim
Shortage of Evidence
•Many impact claims were unsupported by observable evidence, and none were measured systematically.
•Of the 27 grantees who claimed an effect or beneﬁt from their video, only 63% supported that claim with any kind of observable
evidence, such as audience self-reports of opinion change or an audience member taking a desired action.
•The other 37% of that group did not support their impact claims with any evidence or confused evidence of distribution, such as
online view counts, with evidence of impact (effects and beneﬁts).
•0% of grantees engaged in systematic measurement of impact. Evidence was ad hoc and anecdotal.
Note: See slide 55 for definitions of terms.
Strong Production/Distribution Hindered by Weak Strategy
GreatVideo Content and Production
•Grantees are telling compelling stories and producing high quality videos.
•The overall quality of story and production is very high.
•Selection of more compelling visuals and editing (to make them shorter and more conducive to online sharing) was a common
suggestion for improvement.
•Yet poor strategy often means that even excellent videos have little impact.
Distribution Shines Despite Lack of Focus
•Grantees are creative and energetic in their distribution, yet lack focus on speciﬁc audiences.
•Grantees used a range of online and ofﬂine methods to distribute their videos. They used social media, webpages, and even
temporary grafﬁti. They showed videos at workshops, conferences, trainings, public debates, and at meetings with decision-makers.
•Their shortcoming was a lack of focus on audience. Of the 30 grantees with sufﬁcient data to evaluate distribution, 70% did not
connect their distribution methods to speciﬁc audiences they were targeting.
•As a result, distribution was enthusiastic, but at times haphazard.
The Strategic Deﬁcit
•Behind the impact deﬁcit lies a strategy deﬁcit. Grantees have mastered only the basics.
•95% of grantees identiﬁed a goal and 86% identiﬁed one or more audiences, indicating widespread understanding of the most
basic strategic principles of video advocacy.
•Yet 32% are having problems identifying precise audiences, instead trying to reach the “general population” or “open public.”
•Only 7 grantees (14%) were able to make an evidence-backed claim that their video created an audience effect that beneﬁted
either themselves or their population.
•Outside of these 7, there was little evidence that grantees had in mind a video theory of change* that linked the audience effect
of their video to beneﬁts for the marginalized population.
* Deﬁnition on slide 55, model on slide 56.
Adopt a Population Beneﬁt Strategy forVideo
What is a Population Beneﬁt Strategy forVideo?
•This new Media and Narratives Division strategy would inform funding priorities, proposal evaluation criteria, and grantee support.
1.Funded videos would focus on achieving concrete improvements in the welfare of marginalized populations.
2.Funded videos would have a credible theory of change that describes the role the video will play in achieving that beneﬁt.
3.Grantees would receive high- and low-touch support to gain the capacity to create video theories of change.
Prioritize Advocacy Over Promotion
•If population beneﬁt is the goal, reconsider funding of promotional videos.
•The videos in this assessment included both advocacy videos, which make an argument for the improvement in the welfare of marginalized
populations, and promotional videos, which tout past successes of an organization.
•Promotional videos help grantees become more sustainable by acting as a fundraising tool, but their connection to population beneﬁt is indirect.
•Advocacy videos provide a more direct effect on marginalized populations because that is their explicit goal.
Focus on theVideo’s Theory of Change
•Proposals should outline a series of causal steps that link video use to population beneﬁt.
•Grantees may need help seeing the big picture, as video impact and theory of change are still unclear to many.
Provide a Suite of Support Services
•Different grantees will need different levels of support to get to video theory of change.
•For those with strong strategic thinking, more focused proposal feedback and visual aids (slides 12, 56) may be sufﬁcient.
•For those with weaker strategic thinking, technical assistance would be necessary.
•Current strategic skills are summarized in the Grantee Report Cards, beginning on slide 14.
Consider a Broader Assessment
•This assessment is based on a 51% non-random sample of grantees who self-selected to report on their video impact.
•Part 2 would analyze the impact of the other 49%, and give a more complete picture of grantee video impact.
Image: Flickr/Tom Blackwell
Image: Flickr/Very Quiet
Data & Model
Grantees wrote the following 3 self-report statements about each video
they submitted. All 3 statements were analyzed for this assessment.
1) Description (Please provide a 50-100 word description of the video.)
2) Use Statement (How did your organization use this video in your
3) Impact Statement (Did your video have a positive impact on your
cause? If yes, what was that impact? If not, what prevented that impact?)
A total of 72 videos were submitted to the
Health Rights Online Film Festival and
analyzed for this assessment.
2015 Health Rights Online Film Festival
An online festival of grantee videos (ﬁlmfest.health-rights.org) provided the perfect opportunity for an assessment of
video advocacy. Of the 73 grantees invited to submit a video, 37 (51%) responded and are included in this
assessment. They submitted not only videos, but also self-reports on both how they used those videos in advocacy
and the impact they realized from that use. Both the videos submitted and the self-reports describing those videos
were analyzed for this assessment.
by John Kennedy
Self-report texts analyzed by Mary Joyce
Mary Joyce used grounded content analysis to analyze all 72 sets of video statements submitted by grantees. This
qualitative method involved reading and annotating the 217 texts to identify patterns (image above). John Kennedy
viewed the 72 videos associated with these texts to make his assessment.
Desired effect on the
audience and how this
effect will lead to
b e n e ﬁ t s f o r t h e
Individuals whose change of
mind or action would have a
beneﬁcial effect on the
Narrative structures and
characterization used to
persuade the audience.
Technical elements of the
video, including sound,
lighting, and editing.
Means of ensuring the
audience sees the video.
Extent to which the desired
audience effect and population
b e n e ﬁ t s o c c u r r e d , a s
demonstrated by observable
For each grantee, analyses of both the video and texts are presented in an additive model of video advocacy
impact. In this model, early choices on Goal identiﬁcation and Audience speciﬁcation lead to decisions about the
Story a video will tell and the Production that will best express that story. The video is then Distributed
according to the audience the grantee wishes to reach. Impact is evaluated by comparing the videos effect on the
audience to the video’s initial goal.
Analysis: Mary Mary Mary MaryJohnJohn
cc Mary Joyce, 2015
Limitations of the Assessment
• Selection Bias: The grantees self-selected to submit videos to the Health Rights Online
Film Festival and to write the self-reports about those videos. As a result, the 37
grantees in this assessment represent a 51% non-random sample of the population of
2009-2015 video-producer grantees.
• Positive Skew: This self-selection likely resulted in a positive skew on the data. Those
who participated are likely more actively involved in video advocacy than grantees who
did not participate.
• Self-Reporting: The texts used in this assessment are written by the grantees
themselves. One would expect a tendency to portray their own video work in a
• Repurposed Data: The texts were produced as part of an application for the ﬁlm
festival’s Impact Award. Grantees were asked to speak about their video’s impact in
general terms. They were not asked to respond directly to the elements in the model
presented on the previous slide.
1. African Palliative Care Association (APCA)
2. Association for Development and Social Inclusion
3. Association for Emancipation, Solidarity and Equality of
Women of Republic of Madeconia (ESE)
4. Association for Supporting the Marginalized Workers
5. Center for Interethnic Dialogue andTolerance Amalipe
6. Centre For Advocacy And Research / Center for Health
and Social Justice (CHSJ)
7. Centro de Estudios para la Equidad y Gobernanza en
los Sistemas de Salud (CEGSS)
8. Ceva de Spus
9. Children of Georgia
10.Citizen Association SOLEM
11.English Collective of Prostitutes
13.Eurasian Harm Reduction Network
14.Fundación Salud por Derecho
15.GeorgianYoung Lawyers Association (GYLA)
17.Health Education and Research Association (HERA)
18.Healthy Options Project Skopje (HOPS)
19.Hungarian Civil Liberties Union (HCLU)
20.International Committee on the Rights of Sex Workers
in Europe (ICRSE)
21.Justice and Women CharitableTrust
22.Kenya Hospices and Palliative care Association
23.Kenya Legal and Ethical Issues Network on HIV and
24.Keystone Human Services Moldova
26.New Zealand Prostitutes Collective (NZPC)
27.Pivot Legal Society
29.Red Latinoamericana por el Acceso a Medicamentos
31.Sex Worker Education and AdvocacyTaskforce
32.Sexual and Health Rights of Marginalized Communities
33.Sonke Gender Justice
34.Transgender Equality Network Ireland (TENI)
36.Uganda National Health Users'/Consumers'
37.Ukrainian Community Advisory Board (UCAB) Patients
Report cards are arranged in this section in the above order.
Of the 37 grantees reviewed in the original
version, a representative selection of ten are
included here. They illustrate common
challenges in making advocacy videos.
1. Filmmaking and Strategy are Difference Skills
2.The Importance of a Clear Goal
3.The Problem with “Raising Awareness”
4.The Importance of Active Distribution
5.The Problem withTargeting the “Public”
6. Impact Without Evidence
7. When Goal and Impact are Disconnected
8. Media Coverage ≠ Impact
9.VideoViews ≠ Impact
10. Simple Strategy Wins the Day
THE REPORT CARDS
The assessment of each grantee’s video advocacy is presented in a report card, with above elements.
The grantee is scored on its
mastery of each element of the
model, as revealed through the
videos and self-report texts
included in the assessment.
Grantee quotes from the self-
reports are used whenever
possible to explain the score.
Provides advice directly to the
grantee, based on the model
Videos included in the
assessment are listed, with links.
A still from one of the videos
included in this grantee’s
“[P]eople with intellectual or
psychosocial disabilities who
a l s o h a v e p h y s i c a l
disabilities... are conﬁned... for
two main reasons: there is...
societal prejudice against
them... and their physical
e nv i r o n m e n t s d o n o t
accommodate” them. This is
great problem analysis. The
next step is to ﬁgure out
who has the power to
address either reason and
decide what action you want
them to take. Then you’ll
have your goal.
T h e “ p u b l i c ” i s
identiﬁed, but this is
an audience too
v a g u e t o b e
are excellent. The
i n t e r v i e w s w i t h
spokespeople feel a
little jarring - as they
interrupt the ﬂow of
the portraits. Does
Judy ever visit these
institutions? I'm just
looking for a visual
way to link her to the
very strong video.
Very active and diverse
distribution methods -
“Blog on OSF website,
O S M H I F a c e b o o k ,
Yo u T u b e , V i m e o ,
International Film Festivals'
screenings and public
d i s c u s s i o n s , s h a r i n g
between NGOs, TV
broadcasting “ yet without
an audience identiﬁed (or
an action desired of that
audience) it’s hard to
know if these methods
The impact described -
public discussions...” -
seems to indicate your
confusion as to what
the ﬁlm was meant to
achieve and which
e f f e c t s w o u l d b e
e v i d e n c e o f t h a t
these are methods of
distribution, not impacts
(effects) of distribution.
Such great ﬁlming
and visual choices.
This is a fantastic video. Yet there’s a disconnect between ﬁlmmaking and strategy. The ﬁlmmaker is clearly talented at production, storytelling, and distribution.
But without clarity on goal or audience it’s very hard to make an impact... and this is what happened to this ﬁlm. The next time you work with an NGO, make
sure they tell you exactly who they are trying to inﬂuence and what successful inﬂuence would look like to them. They may not be clear on the answers, so you
may need to press them. The result, however, will be a more effective advocacy video.
Videos in Assessment: New Day (2015): video.health-rights.org/index.php/list/product/view/1/55.html
* Incomplete data
GRAL FILMFilmmaking and Strategy are Different Skills1
This talented ﬁlmmaker was at a loss when it came to strategy.
CENTER FOR INTERETHNIC DIALOGUE AND
TOLERANCE AMALIPE (AMALIPE)
“We use the movie to
make RHSP popular and
to advocatte [sic.] for its
sustainability” - The
sustainability goal is clear,
precise, and beneﬁcial.
Focus on that. Being
“popular” is not precise
and its beneﬁt is also
This is a great example of
how a clear goal leads to
clear audiences, but unclear
ones are a dead end.
institutions” that could
provide funding are a clear
a u d i e n c e f o r t h e
s u s t a i n a b i l i t y g o a l .
Unsurprisingly, there is no
goal for the vague goal of
p o p u l a r i t y. A n d t h e
“advocacy work at local
level” - what is that about?
It looks more like a
about your work
than an advocacy
video as a tool.
Really well ﬁlmed.
There’s little information on
how you showed the video
to the funding institution.
Maybe this was because you
thought a face-to-face
meeting was boring or
unworthy of mention. But
distribution strategy does not
need to be high-tech. It just
needs to put your video
together with its intended
audience. And distribution
for popularity? And the
advocacy work? Need more
Impact assessment should link
back to the goal using evidence
to make an argument, but that
didn’t happen here. There’s no
mention of whether the video
helped you get funding. You say
that the video helped you
popularize your program, but
provide no evidence. You
mention again advocacy at the
local level, but this is a means
of achieving impact, not the
“Community Monitoring” is a very well ﬁlmed and edited video. However, it looks more like a mini documentary about your work than an advocacy tool. It’s
hard to tell from the incomplete information, but it seems that you made this strategic analysis harder than it had to be. You needed funds. You created a video
illustrating your past work to convince “relevant national institutions” to continue funding that work. That’s a completely acceptable video strategy! It just needs a
means of measurement. Did you get funded? Did you receive more money than you had before you were able to use video as a promotional tool?
Measurement of impact just means using observable evidence to evaluate the effect your video on your audience. If you keep your strategy simple it will be
easier to both create video strategies and measure their impact.
Videos in Assessment: Videos in Assessment: The White Swallows III (2015) video.health-rights.org/index.php/list/product/view/1/284.html; Community Monitoring (2015) video.health-rights.org/index.php/list/
* Incomplete data
2 The Importance of a Clear Goal
A clear goal is the focus and foundation of video strategy.
“To raise awareness
issues.” - While a
common goal of
raising awareness is
not enough. What
do you want to
happen after people
are aware? Sketch
the full theory of
change to keep
“The video has a
positive impact on
a u d i e n c e - t h e
audience is convinced”
- There is an awareness
that the audience is an
impor tant par t of
video strategy, but
there is no actual
identiﬁcation of who
the audience is.
Very good storytelling.
Continue to push your
team - what are the
best visuals and most
(“All Boys” is my
favorite video in the
assessment - MJ)
The video was actively
distributed on TV, social
media, and through public
events. You were also
attentive to evidence of
audience exposure, such as
comments and likes.
However, without a clear
idea of who should be
seeing the ﬁlm it’s hard to
tell whether the right
distribution methods were
I appreciate your honesty in
saying you felt you couldn’t
measure impact directly. This is
because you did not identify a
speciﬁc audience, so you didn’t
know where to measure. If you
select an audience, you will be
able to measure impact because
you will know where to seek
evidence of effect. Remember
that you cannot say a video had
a “positive impact” if you have
no evidence of audience effect.
Nice simple ﬁlming and
Your ﬁlm has a clear and excellent message and the ﬁlming is simple and well-done. Now it’s time to build on that by working on the other elements of your
strategy so you can get a bigger impact next time. The next time you create a video, create your strategy ﬁrst. Here are the basics of linking all the elements
above into a clear strategy. The purpose of any advocacy video is to persuade an audience, either to take an action or to change their mind about something.
The action/mental change they are persuaded to undertake is the goal. The argument the video makes to persuade them is the story. How you will make sure
the audience sees the video is distribution. When you measure impact you are looking for evidence as to whether the audience took the action (easier to
observe) or changed their minds (harder to observe). If you follow this strategic process you will get more value out of your future videos - and the ones you’ve
already made. Good luck!
Videos in Assessment: Institution or community? (2015) video.health-rights.org/index.php/list/product/view/1/318.html; Returning Home (2015) video.health-rights.org/index.php/list/product/view/1/320.html; I wish
all boys from residential institutions could live in a family (2013) video.health-rights.org/index.php/list/product/view/1/319.html
* Incomplete data
KEYSTONE HUMAN SERVICES3 The Problem with “Raising Awareness”
“Raising awareness” is too vague a goal for precise and effective strategy.
“[T]o raise awareness around
sex workers human rights and
health issues, as well as the
need for change in law” - You
identify the direct effects of the
video (raising awareness) and
the ultimate impact you are
aiming for (change in the law),
but you don’t link them. Will
raising awareness lead directly
to changing the law? If not,
what are the intermediate steps
and how will you move through
“Change in the law”
i m p l i e s a
g o v e r n m e n t a l
audience, though none
is speciﬁed. The
individuals who would
be targeted to sign
the e-petition are also
Very clear, well
You write that you cannot
measure impact yet, but since
you do not even note the
number of signatures on the e-
petition, I suspect the number is
lower than you would have
wished, rather than being
uncountable. You also “hope”
the video “reach[es] people and
bring[s] about change,” which is
oddly passive given that it is the
aim of your own campaign to
achieve these ends. Don’t give
up. You can make it happen!
Online advocacy is not a silver bullet. It just implies a new set of tools. Making an (admittedly high quality) digital video does not imply virality. Placing that video
online does not guarantee engagement in an online tactic like e-petition signing. In fact, because there is so much content online vying for our attention, active
promotion to clearly speciﬁed audiences is critically important. Although I sense that this effort might have been discouraging, you’ll get a better result if you
engage in more intentional video strategy. Here are the basics: The purpose of any advocacy video is to persuade an audience to take action (or change their
mind). The action they are persuaded to do is the goal. The argument the video makes to persuade them to take that action is the story. How you will make
sure the audience sees the video is distribution. And when you measure impact you are looking for evidence as to whether the audience took the action (or
changed their mind, though that is harder to measure). Good luck!
Videos in Assessment: Decriminalise Sex Work Now!(2015): video.health-rights.org/index.php/list/product/view/1/41.html
* Incomplete data
SEX WORKER EDUCATION AND ADVOCACY
The video was posted on
its own page on your
website, but this really isn’t
enough. There is no
m e n t i o n o f a c t i v e
outreach (for example, via
listservs and social media)
to drive trafﬁc to the
page. You also do not
specify the audiences you
were trying to reach with
G o o d s i m p l e
4 The Importance of Active Distribution
Simply posting a video online isn’t enough to engage an audience.
CEVA DE SPUS
“[T]o change the
mentality of the public
towards people with
t o p r o v e [ t h e i r ]
abilities" - Ambitious,
but I’d like to see the
video linked to a more
s p e c i ﬁ c a n d
measurable action from
the audience. Perhaps
t h e r e a c t i o n yo u
w a n t e d f r o m t h e
U n i v e r s i t y
“[T]he open public” -
Trying to target the
public is a common
mistake. It’s too broad
t o b e t a r g e t e d
makers... civil society
or ganizations and
donors” - These are a
Wonderful videos - so
clear why we should all
have equal rights. Think
about more visuals. I
know these were
created as simple
portraits - but I also
know that Zoli is very
skilled at capturing
small moments which
show the personality of
Good ﬁlming, Zoli!
"[U]ploaded on social
media and website...
different events... in press
releases.... projections at
different events, round
Diver se and active
beneﬁt from a bit more
l i n k a g e b e t w e e n
distribution method and
audience . Which
audiences were you
hoping to reach with
e a c h m e t h o d o f
“The west University started
an accessibility project, they
make efforts to fulﬁll all of our
demands... For sure this video
had a small contribution to
the declaration of Ministry of
Labor regarding starting the
Though I wish there were
some evidence behind the
claim about the MoL, the
p o l i c y c h a n g e a t t h e
University is a great example
about how focused pressure
on speciﬁc institutions can be
a starting place for broader
We are both huge fans of this video series. You make it so obvious why no one should be institutionalized permanently. You’ve used video to realize concrete
advances towards broad social change. Next time you make a video, be as speciﬁc as you can about who you are trying to persuade and what you want to
persuade them to do or think. This will prevent you from trying to target overly broad audiences like “the open public.”
Videos in Assessment: There is life outside the walls (2013) video.health-rights.org/index.php/list/product/view/1/334.html; Andrei (2012) video.health-rights.org/index.php/list/product/view/1/335.html; Raluca (2011)
5 The Problem withTargeting the “Public”
This grantee made the common mistake of selecting an audience too broad to target effectively.
A very clear goal: to
help NGO’s in other
countries learn from
N Z P C ’ s b e s t
practices as they
m a k e s i m i l a r
effor ts. Clearly
connects the effect
of the video on the
to the desired impact
( i m p l e m e n t i n g
“ [ S ] e x w o r k e r
o r g a n i s a t i o n s
internationally” - A
audience given the
The message is clear
as a report about
the collective, but
less of an advocacy
“We believe the video has
had a positive impact as sex
workers in other countries
are using the information
within it to build support
for decriminalisation” - This
could be true, but there’s
no evidence provided to
support your claim. (How
do you know it is being
used?) This is the only place
where you video strategy
falls short. The rest is
Clear sound and visuals.
You are a great example of the strategic use of a promotional video. It is very rare for a video promoting an organization’s past work to have a clear and logical
goal, audience, and distribution method, but you’ve done it. Congratulations! The place for you to focus now is on measuring your impact. You could easily get a
sense of the impact of your video by following up with the people who ordered the DVD to see how they used it or by following up with anyone who contacted
your through the video website.
Videos in Assessment: Decriminalisation of Sex Work in New Zealand (2015): video.health-rights.org/index.php/list/product/view/1/314.html
NEW ZEALAND PROSTITUTES COLLECTIVE
Your distribution methods
makes a lot of sense given
your audience and goal.
The video is available via
DVD (perhaps for NGOs
w i t h o u t i n t e r n e t
connection) as well as on
a purpose-built website. I
also like that the video
was actively distributed to
other NGOs, rather than
simply letting it sit on a
6 Impact Without Evidence
This grantee’s strategy was really strong until the impact assessment stage, when they failed to measure the effect of their work.
ENGLISH COLLECTIVE OF PROSTITUTES
“1. to publicize our legal
victories... 2. To build the
m o v e m e n t f o r
decriminalisation.“ - The
value of the ﬁrst goal is
unclear. What future
beneﬁt will result from
publicizing these past
victories? The second
goal is a bit vague. What
exactly does “build the
movement” mean here?
How will the video
”[M]any individuals and
organisations” - Who
exactly? Why is it
important that they
h e a r a b o u t yo u r
victory? What role do
t h e y p l ay i n t h e
movement you are
It's a very good
video - but we
need sex worker
their voices we
can't be 100% sure
this is what they
are asking for.
“[S]howing in December
2014” - Who was in the
a u d i e n c e t h a t yo u
wanted to persuade?
“sent the ﬁlm to our
network” - To what end?
What actions were you
hoping for as a result?
“We...have been invited
t o s p e a k a t ﬁ l m
showings on a number
of occasions.” - This
seems more by chance
than by design. There’s a
goal and impact.
Good ﬁlming - both
interior and exterior
(protest) shots are
I know you are awesome activists and I love both the spirit and the method of your activism, so I am going to be a little tough on you (as in touch love) regarding
your video strategy. I get the sense that there wasn’t really a clear vision of what beneﬁt you were hoping to get out of this video. As a result, the audiences
targeted, the distribution method used to reach those audiences, and the impacts recorded seem rather haphazard. Bottom line - advocacy video is about
persuading and audience to take an action (often mediated by a change in state of mind) and impact is about measuring the extent to which that action was
taken. If you are speciﬁc about the action you want from the beginning, when you set your goal, the rest of the strategic process will be a lot easier.
Videos in Assessment: SohoTrot (2015): video.health-rights.org/index.php/list/product/view/1/312.html
7 When Goal and Impact are Disconnected
A big goal (movement-building) leads to minimal beneﬁts (speaking invites) without clear causal steps connecting the two.
EURASIAN NETWORK OF PEOPLE WHO
USE DRUGS (ENPUD),ALLIANCE UKRAINE
“ F o r d r a w i n g
attention to the
problem of clients of
OST program in
Donbass“ - Focus
more on the action
y o u w a n t t h e
audience to take.
Whose attention? To
“ [ D ] r a w i n g
a t t e n t i o n ,” b u t
whose attention? In
t h e a b s e n c e o f
realizing impact is
You’ve done a really good
job eliciting stories from
the protagonists. Their
discussions of their
families and emotional
lives humanize them. I
would have liked to see
someone who is a
recovering addict give a
tour of the city he knew
as an addict.
“A lot of famous mass-
media, such as BBC... key
events of EEC region” -
Though getting coverage
from a media partner
with a wide audience is
impressive, this is only
meaningful if the BBC’s
audience is the audience
you want to reach to
achieve your goal. Also,
remember that reaching
an audience is evidence
of distr ibution, not
evidence of impact.
It’s possible that the
media exposure resulted
in actions helpful to the
OST clients, but there’s
n o e v i d e n c e .
Remember that views
are not impact, action is.
Next time, think about
who has the power to
help and then target
The ﬁlming is of good
quality, but the visual
choice (the candle, for
example) could relate
to the subject matter
Your passion about this topic really comes through and the suffering of the ﬁlm participants is clear. For a crisis as challenging as this one,
ﬁlmmakers need to connect to other partners who are capable of carrying out more direct (in-person) advocacy with audiences capable of
providing practical assistance to those in need. If ﬁlmmakers are not currently connected to these actors, OSF could likely assist in making those
connections. Massive humanitarian crises cannot be addresses by ﬁlmmakers working alone. They need help.
Videos in Assessment: Slow Death in Ukraine (2015) video.health-rights.org/index.php/list/product/view/1/248.html; Live or Die in Donbas? (2015) video.health-rights.org/index.php/list/product/view/1/249.html;The
First CrimeanVictims(2015) video.health-rights.org/index.php/list/product/view/1/268.html;The Right to be Dependent - Irina (2011) video.health-rights.org/index.php/list/product/view/1/67.html
* Incomplete data
8 Media Coverage ≠ Impact
Media coverage is a mechanism of distribution, not a measure of impact.
OST = opioid substitutions therapy
“ [ D ] e m a n d s f o r
decriminalisation” - This is
a good campaign goal (or
even or ganizational
mission), but I’d like to see
a goal speciﬁc to the
video. What role should
t h e v i d e o p l a y i n
convincing politicians to
make this change? Do
t h e y n e e d s p e c i ﬁ c
information? A more
general emotional appeal
to empathy to lay the
groundwork for later
other networks and
organisations” - I’d
like to see more
clarity about why
e a c h a u d i e n c e
needs to see the
The fact that sex
workers were willing
to be ﬁlmed without
anonymization is very
rare. Yet the power
of their self-advocacy
is muted by the lack
of a narrative arc.
Online methods seems to
be the primary means of
distribution, including an
online training and social
media. Again, there
doesn’t seem to be much
clarity on who needs to
see the video, thus also a
lack of clarity on how to
reach that audience.
"I don't know. The video was
seen 3,500 times (July 2015)
so we assume it has allowed
many people to make
themselves familiar with our
message." - I appreciate your
h o n e s t y a b o u t y o u r
uncertainty in measuring
impact. I also like that you are
looking for obser vable
evidence This is the right
direction. In the future, note
that views are a measure of
distribution, not impact. What
did you want people to do
after they received your
A nice use of natural
light. Good looking
interviews - but think
about visuals that can
also tell a story.
Your video contains truly excellent self-advocacy from a range of sex workers across the European region. Next time you create a video, create your strategy
ﬁrst. The purpose of any advocacy video is to persuade, so a video strategy needs to identify who needs to be persuaded (audience), what they need to be
persuaded to do (goal), the argument the video will make to persuade them (story), how you will make sure the audience sees the video (distribution), and how
you will know whether the audience persuasion took place (impact measurement). In particular, try to narrow down your goal to a speciﬁc change in state of
mind or action needed from a speciﬁc audience, which will further your larger goal of decriminalization. This focus will help you develop a distribution plan linked
more directly to speciﬁc audiences and you will have a clearer idea of whose reaction to the ﬁlm you need to measure. Remember that it’s easier to measure
action than changes in state of mind, so see if you can link the change you want from your audience into an observable action.
Videos in Assessment: Sex workers are the solution not the problem - Sex work, HIV and human rights (2014): video.health-rights.org/index.php/list/product/view/1/303.html
INTERNATIONAL COMMITTEE ONTHE
RIGHTS OF SEX WORKERS IN EUROPE (ICRSE)9 VideoViews ≠ Impact
Though they are easy to measure, video views are evidence of distribution, not of impact.
“[T]o raise surrpport
[sic.]” among “families
[of] women living with
Hiv/Aids” - A modest
goal but one that is
clearly tied to the video.
“[T]o raise community
awareness to counter
stigma” - Clearly links the
effect of the video
(raising awareness) to
b e n e ﬁ c i a l i m p a c t s
(reducing stigma, raising
suppor t). Linking
awareness raising to a
broader purpose is so
rare - great job!
“ [ C ] o m m u n i t y
members... families [of]
women living with Hiv/
Aids”” - The audiences
are speciﬁc and are
clearly tied to the
Keep trying to ﬁnd
people who are
willing to have their
faces shown on
camera. I know this is
a sensitive subject,
but there will be
who want to tell
“The video was viewed by
over 1000 community
m e m b e r s i n t h e i r
homesteads where trained
community volunteer s
facilitated discussions on
issues raised by women in
the video.” - An excellent
e x a m p l e o f l i n k i n g
distribution method to
audience in order to address
a speciﬁc goal. Also excellent
use of quantitative data as
evidence. Despite the fact
that the video is digital,
distribution is all ofﬂine. This
is exactly what what needed
to reach your rural audience.
"The women involved in making
the video and in facilitating the
discussions reported qualitative
changes with families showing
increasing support to women
living with Hiv/Aids.” - Using
qualitative data (oral reports),
this impact links back directly to
the initial goal of the video.
Again, it is rare to see this type
of clear linkage. It would be
nice to see some kind of impact
assessment of the awareness-
raising/stigma-reducing goal, and
there is some confusion
between distribution (views)
and impact (action), but this is
really good work.
Good visuals. Think
about shortening the
video - a shorter video
will often be more
Congratulations! Your video has the strongest strategy of all the grantees in this assessment! Your goals, audience, distribution, and impact are all clearly linked.
The use of evidence (views, oral reports) to support your impact claims is also excellent. You should be teaching other NGOs how to create video strategies!
Videos in Assessment: TheTriple Burden (2015): video.health-rights.org/index.php/list/product/view/1/302.html
JUSTICE AND WOMEN CHARITABLETRUST10 Simple Strategy Wins the Day
This grantee’s simple strategy with clear causal linkages led to a beneﬁcial outcome clearly connected to their use of video.
• AdvocacyVideo: Recording of moving images designed to beneﬁt a population or individual whose interests
are marginalized or under-represented.
• Audience: Individuals whose change of mind or action would have a beneﬁcial effect.
• Beneﬁt: Concrete improvement in quality of life.
• Distribution: Means of ensuring the audience sees the video.
• Goal: Desired outcome. Video goals should reference both the direct effect on the audience and how this
effect will lead to beneﬁts for the grantee’s population of interest.
• Impact: Extent to which the desired audience effect and population beneﬁts occurred, as demonstrated by
• OST: Opioid substitution therapy, a form of harm reduction in which users of opiates, such as heroin, are given
substitution drugs, such as methadone, to manage their addictions.
• Production:Technical elements of the video, including sound, lighting, and editing.
• PromotionalVideo: Recording of moving images touting past successes, often for the purposes of gaining
funds for future work.
• Story: Narrative structures and characterization used to persuade the audience to undertake an action or
change of mind.
• Strategy: A plan for achieving a goal.
• Theory of Change: A series of causal steps that explains why it is logical that the actions in the strategy will
achieve the goal. While the strategy describes the actions of the advocate, the theory of change takes into
account the broader context in which the advocate is operating.
VIDEOTHEORY OF CHANGE
cc Mary Joyce, 2015
Desired beneﬁt for
Desired effect on
Realized effect on
Realized beneﬁt for
Start goal-setting here.
End evaluation here. Campaign Video
cc Mary Joyce, 2015
Desired beneﬁt for marginalized population
“Stop Sending TB Patients to Prison” by KELIN
Desired effect on audience
Realized effect on audience
Realized beneﬁt for marginalized population
VIDEOTHEORY OF CHANGE
Image: Flickr/Tom Blackwell
Image: Flickr/Very Quiet
Mary Joyce | firstname.lastname@example.org
Video Strategy Analysis, Findings and Recommendations, InformationVisualization and Models
John Kennedy |Video Content and Production Analysis | email@example.com