1.
by Mary Joyce with John Kennedy for the Open Society Foundation | December 2015 (v1.1) | Part 1
Image: Flickr/Very Quiet
VIDEO IMPACT ASSESSMENT
Closing the Impact Gap
Public Version

2.
CONTENTS
Executive Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . slide 3
Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .slide 4
Recommendations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .slide 7
Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .slide 9
Grantee Report Cards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . slide 14
Appendices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .slide 54
This public version is anonymized to remove information identifying the nonprofit organizations being assessed and the foundation for which
the assessment was prepared. The purpose of this anonymization is to protect the privacy of the entities involved and instead put the focus
on the method of analysis and its presentation. Redacted content is covered with gray boxes.
About the Public Version

3.
* Assessment Part 1 includes a 51% non-random sample of 37 video-producing grantees.Image: Flickr/Very Quiet
EXECUTIVE SUMMARY
•This assessment of 2009-2015 grantees* revealed a gap between video content
and video impact.
•Despite strengths in storytelling, production, and distribution, grantees often fail to
benefit from their videos.
•Strategy is often rudimentary.While the vast majority of grantees understand the
need for a goal and audience, most videos lack theories of change, precise audience
targeting, and systematic measurement of impact.
•To meet these challenges, the Media and Narratives Division should adopt a
population benefit strategy that would focus funding priorities and methods of
proposal evaluation and grantee support on videos that aim to achieve concrete
benefits for marginalized populations.

4.
Image: Flickr/Very Quiet
FINDINGS

5.
FINDINGS
Impact is Little Understood & Poorly Measured
Lack of Consensus on Impact
•Impact claims were diverse, indicating little consensus on what
constitutes video impact.
•27% of grantees claimed that the video benefited the
population on whose behalf their organization advocates, such as
improved health services for Roma or kinder community
treatment of HIV+ women.
•14% claimed the video benefited their organization, such as
helping them raise money or get invitations to speak at
conferences.
•32% of grantees claimed an effect on their audience, such as
raising awareness or increasing understanding, though not a direct
benefit to themselves or their population.
•The remaining 27% made no claim of video effect or benefit,
instead describing the scope of distribution or the quality of the
video’s content.
Population Benefit
Organizational Benefit
Audience Effect w/o Benefit
No Effect or Benefit Claim
n=37
32%
14%
Impact Claims
Shortage of Evidence
•Many impact claims were unsupported by observable evidence, and none were measured systematically.
•Of the 27 grantees who claimed an effect or benefit from their video, only 63% supported that claim with any kind of observable
evidence, such as audience self-reports of opinion change or an audience member taking a desired action.
•The other 37% of that group did not support their impact claims with any evidence or confused evidence of distribution, such as
online view counts, with evidence of impact (effects and benefits).
•0% of grantees engaged in systematic measurement of impact. Evidence was ad hoc and anecdotal.
Note: See slide 55 for definitions of terms.

6.
FINDINGS
Strong Production/Distribution Hindered by Weak Strategy
GreatVideo Content and Production
•Grantees are telling compelling stories and producing high quality videos.
•The overall quality of story and production is very high.
•Selection of more compelling visuals and editing (to make them shorter and more conducive to online sharing) was a common
suggestion for improvement.
•Yet poor strategy often means that even excellent videos have little impact.
Distribution Shines Despite Lack of Focus
•Grantees are creative and energetic in their distribution, yet lack focus on specific audiences.
•Grantees used a range of online and offline methods to distribute their videos. They used social media, webpages, and even
temporary graffiti. They showed videos at workshops, conferences, trainings, public debates, and at meetings with decision-makers.
•Their shortcoming was a lack of focus on audience. Of the 30 grantees with sufficient data to evaluate distribution, 70% did not
connect their distribution methods to specific audiences they were targeting.
•As a result, distribution was enthusiastic, but at times haphazard.
The Strategic Deficit
•Behind the impact deficit lies a strategy deficit. Grantees have mastered only the basics.
•95% of grantees identified a goal and 86% identified one or more audiences, indicating widespread understanding of the most
basic strategic principles of video advocacy.
•Yet 32% are having problems identifying precise audiences, instead trying to reach the “general population” or “open public.”
•Only 7 grantees (14%) were able to make an evidence-backed claim that their video created an audience effect that benefited
either themselves or their population.
•Outside of these 7, there was little evidence that grantees had in mind a video theory of change* that linked the audience effect
of their video to benefits for the marginalized population.
* Definition on slide 55, model on slide 56.

7.
Image: Flickr/Very Quiet
RECOMMENDATIONS

8.
RECOMMENDATIONS
Adopt a Population Benefit Strategy forVideo
What is a Population Benefit Strategy forVideo?
•This new Media and Narratives Division strategy would inform funding priorities, proposal evaluation criteria, and grantee support.
1.Funded videos would focus on achieving concrete improvements in the welfare of marginalized populations.
2.Funded videos would have a credible theory of change that describes the role the video will play in achieving that benefit.
3.Grantees would receive high- and low-touch support to gain the capacity to create video theories of change.
Prioritize Advocacy Over Promotion
•If population benefit is the goal, reconsider funding of promotional videos.
•The videos in this assessment included both advocacy videos, which make an argument for the improvement in the welfare of marginalized
populations, and promotional videos, which tout past successes of an organization.
•Promotional videos help grantees become more sustainable by acting as a fundraising tool, but their connection to population benefit is indirect.
•Advocacy videos provide a more direct effect on marginalized populations because that is their explicit goal.
Focus on theVideo’s Theory of Change
•Proposals should outline a series of causal steps that link video use to population benefit.
•Grantees may need help seeing the big picture, as video impact and theory of change are still unclear to many.
Provide a Suite of Support Services
•Different grantees will need different levels of support to get to video theory of change.
•For those with strong strategic thinking, more focused proposal feedback and visual aids (slides 12, 56) may be sufficient.
•For those with weaker strategic thinking, technical assistance would be necessary.
•Current strategic skills are summarized in the Grantee Report Cards, beginning on slide 14.
Consider a Broader Assessment
•This assessment is based on a 51% non-random sample of grantees who self-selected to report on their video impact.
•Part 2 would analyze the impact of the other 49%, and give a more complete picture of grantee video impact.

9.
IMPACT
Image: Flickr/Tom Blackwell
Findings
Image: Flickr/Very Quiet
METHODS
Data & Model

10.
DATA
THE DATA
Grantee Self-Reports
Grantees wrote the following 3 self-report statements about each video
they submitted. All 3 statements were analyzed for this assessment.
1) Description (Please provide a 50-100 word description of the video.)
2) Use Statement (How did your organization use this video in your
advocacy work?)
3) Impact Statement (Did your video have a positive impact on your
cause? If yes, what was that impact? If not, what prevented that impact?)
TextsVideos
Grantee-CreatedVideos
A total of 72 videos were submitted to the
Health Rights Online Film Festival and
analyzed for this assessment.
2015 Health Rights Online Film Festival
An online festival of grantee videos (filmfest.health-rights.org) provided the perfect opportunity for an assessment of
video advocacy. Of the 73 grantees invited to submit a video, 37 (51%) responded and are included in this
assessment. They submitted not only videos, but also self-reports on both how they used those videos in advocacy
and the impact they realized from that use. Both the videos submitted and the self-reports describing those videos
were analyzed for this assessment.
ANALYSTS
Videos analyzed
by John Kennedy
Self-report texts analyzed by Mary Joyce

11.
THE ANALYSIS
Mary Joyce used grounded content analysis to analyze all 72 sets of video statements submitted by grantees. This
qualitative method involved reading and annotating the 217 texts to identify patterns (image above). John Kennedy
viewed the 72 videos associated with these texts to make his assessment.

12.
Goal
Desired effect on the
audience and how this
effect will lead to
b e n e fi t s f o r t h e
marginalized population.
Audience
Individuals whose change of
mind or action would have a
beneficial effect on the
marginalized population.
Story
Narrative structures and
characterization used to
persuade the audience.
THE MODEL
Production
Technical elements of the
video, including sound,
lighting, and editing.
Distribution
Means of ensuring the
audience sees the video.
Impact
Extent to which the desired
audience effect and population
b e n e fi t s o c c u r r e d , a s
demonstrated by observable
evidence.
For each grantee, analyses of both the video and texts are presented in an additive model of video advocacy
impact. In this model, early choices on Goal identification and Audience specification lead to decisions about the
Story a video will tell and the Production that will best express that story. The video is then Distributed
according to the audience the grantee wishes to reach. Impact is evaluated by comparing the videos effect on the
audience to the video’s initial goal.
Analysis: Mary Mary Mary MaryJohnJohn
cc
cc Mary Joyce, 2015

13.
CAVEATS
Limitations of the Assessment
• Selection Bias: The grantees self-selected to submit videos to the Health Rights Online
Film Festival and to write the self-reports about those videos. As a result, the 37
grantees in this assessment represent a 51% non-random sample of the population of
2009-2015 video-producer grantees.
• Positive Skew: This self-selection likely resulted in a positive skew on the data. Those
who participated are likely more actively involved in video advocacy than grantees who
did not participate.
• Self-Reporting: The texts used in this assessment are written by the grantees
themselves. One would expect a tendency to portray their own video work in a
positive light.
• Repurposed Data: The texts were produced as part of an application for the film
festival’s Impact Award. Grantees were asked to speak about their video’s impact in
general terms. They were not asked to respond directly to the elements in the model
presented on the previous slide.

14.
IMPACT
Image: Flickr/Tom Blackwell
Findings
Image: Flickr/Very Quiet
1
GRANTEE REPORT CARDS

15.
1. African Palliative Care Association (APCA)
2. Association for Development and Social Inclusion
(ADIS)
3. Association for Emancipation, Solidarity and Equality of
Women of Republic of Madeconia (ESE)
4. Association for Supporting the Marginalized Workers
(Star-Star)
5. Center for Interethnic Dialogue andTolerance Amalipe
"AMALIPE"
6. Centre For Advocacy And Research / Center for Health
and Social Justice (CHSJ)
7. Centro de Estudios para la Equidad y Gobernanza en
los Sistemas de Salud (CEGSS)
8. Ceva de Spus
9. Children of Georgia
10.Citizen Association SOLEM
11.English Collective of Prostitutes
12.ENPUD,Alliance Ukraine
13.Eurasian Harm Reduction Network
14.Fundación Salud por Derecho
15.GeorgianYoung Lawyers Association (GYLA)
16.Gral Film
17.Health Education and Research Association (HERA)
18.Healthy Options Project Skopje (HOPS)
19.Hungarian Civil Liberties Union (HCLU)
20.International Committee on the Rights of Sex Workers
in Europe (ICRSE)
21.Justice and Women CharitableTrust
22.Kenya Hospices and Palliative care Association
(KEHPCA)
23.Kenya Legal and Ethical Issues Network on HIV and
AIDS (KELIN)
24.Keystone Human Services Moldova
25.KHAM
26.New Zealand Prostitutes Collective (NZPC)
27.Pivot Legal Society
28.Rattapallax "DAVIDA"
29.Red Latinoamericana por el Acceso a Medicamentos
30.Roma SOS
31.Sex Worker Education and AdvocacyTaskforce
(SWEAT)
32.Sexual and Health Rights of Marginalized Communities
Coalition
33.Sonke Gender Justice
34.Transgender Equality Network Ireland (TENI)
35.Transgender Europe
36.Uganda National Health Users'/Consumers'
Organization (UNHCO)
37.Ukrainian Community Advisory Board (UCAB) Patients
of Ukraine
THE GRANTEES
Report cards are arranged in this section in the above order.
Of the 37 grantees reviewed in the original
version, a representative selection of ten are
included here. They illustrate common
challenges in making advocacy videos.
1. Filmmaking and Strategy are Difference Skills
2.The Importance of a Clear Goal
3.The Problem with “Raising Awareness”
4.The Importance of Active Distribution
5.The Problem withTargeting the “Public”
6. Impact Without Evidence
7. When Goal and Impact are Disconnected
8. Media Coverage ≠ Impact
9.VideoViews ≠ Impact
10. Simple Strategy Wins the Day

16.
THE REPORT CARDS
The assessment of each grantee’s video advocacy is presented in a report card, with above elements.
Scores
The grantee is scored on its
mastery of each element of the
model, as revealed through the
videos and self-report texts
included in the assessment.
Quotes
Grantee quotes from the self-
reports are used whenever
possible to explain the score.
Recommendation
Provides advice directly to the
grantee, based on the model
analysis.
Videos
Videos included in the
assessment are listed, with links.
Image
A still from one of the videos
included in this grantee’s
assessment

17.
Goal: I*
“[P]eople with intellectual or
psychosocial disabilities who
a l s o h a v e p h y s i c a l
disabilities... are confined... for
two main reasons: there is...
societal prejudice against
them... and their physical
e nv i r o n m e n t s d o n o t
accommodate” them. This is
great problem analysis. The
next step is to figure out
who has the power to
address either reason and
decide what action you want
them to take. Then you’ll
have your goal.
Audience: C
T h e “ p u b l i c ” i s
identified, but this is
an audience too
v a g u e t o b e
effectively targeted.
Story: A
Fantastic.The portraits
are excellent. The
i n t e r v i e w s w i t h
spokespeople feel a
little jarring - as they
interrupt the flow of
the portraits. Does
Judy ever visit these
institutions? I'm just
looking for a visual
way to link her to the
very strong video.
Distribution: A-
Very active and diverse
distribution methods -
“Blog on OSF website,
O S M H I F a c e b o o k ,
Yo u T u b e , V i m e o ,
International Film Festivals'
screenings and public
d i s c u s s i o n s , s h a r i n g
between NGOs, TV
broadcasting “ yet without
an audience identified (or
an action desired of that
audience) it’s hard to
know if these methods
were well-selected.
Impact: C-
The impact described -
“Public awareness,
public discussions...” -
seems to indicate your
confusion as to what
the film was meant to
achieve and which
e f f e c t s w o u l d b e
e v i d e n c e o f t h a t
achievement. Also,
these are methods of
distribution, not impacts
(effects) of distribution.
Production:A
Such great filming
and visual choices.
Recommendation
This is a fantastic video. Yet there’s a disconnect between filmmaking and strategy. The filmmaker is clearly talented at production, storytelling, and distribution.
But without clarity on goal or audience it’s very hard to make an impact... and this is what happened to this film. The next time you work with an NGO, make
sure they tell you exactly who they are trying to influence and what successful influence would look like to them. They may not be clear on the answers, so you
may need to press them. The result, however, will be a more effective advocacy video.
Videos in Assessment: New Day (2015): video.health-rights.org/index.php/list/product/view/1/55.html
* Incomplete data
GRAL FILMFilmmaking and Strategy are Different Skills1
This talented filmmaker was at a loss when it came to strategy.

18.
CENTER FOR INTERETHNIC DIALOGUE AND
TOLERANCE AMALIPE (AMALIPE)
Goal: A-
“We use the movie to
make RHSP popular and
to advocatte [sic.] for its
sustainability” - The
sustainability goal is clear,
precise, and beneficial.
Focus on that. Being
“popular” is not precise
and its benefit is also
unclear.
Audience: B+
This is a great example of
how a clear goal leads to
clear audiences, but unclear
ones are a dead end.
“[R]elevant national
institutions” that could
provide funding are a clear
a u d i e n c e f o r t h e
s u s t a i n a b i l i t y g o a l .
Unsurprisingly, there is no
goal for the vague goal of
p o p u l a r i t y. A n d t h e
“advocacy work at local
level” - what is that about?
Story: A-
It looks more like a
mini documentary
about your work
than an advocacy
video as a tool.
Production: A
Really well filmed.
Distribution: I*
There’s little information on
how you showed the video
to the funding institution.
Maybe this was because you
thought a face-to-face
meeting was boring or
unworthy of mention. But
distribution strategy does not
need to be high-tech. It just
needs to put your video
together with its intended
audience. And distribution
for popularity? And the
advocacy work? Need more
info!
Impact: C
Impact assessment should link
back to the goal using evidence
to make an argument, but that
didn’t happen here. There’s no
mention of whether the video
helped you get funding. You say
that the video helped you
popularize your program, but
provide no evidence. You
mention again advocacy at the
local level, but this is a means
of achieving impact, not the
impact itself.
Recommendation
“Community Monitoring” is a very well filmed and edited video. However, it looks more like a mini documentary about your work than an advocacy tool. It’s
hard to tell from the incomplete information, but it seems that you made this strategic analysis harder than it had to be. You needed funds. You created a video
illustrating your past work to convince “relevant national institutions” to continue funding that work. That’s a completely acceptable video strategy! It just needs a
means of measurement. Did you get funded? Did you receive more money than you had before you were able to use video as a promotional tool?
Measurement of impact just means using observable evidence to evaluate the effect your video on your audience. If you keep your strategy simple it will be
easier to both create video strategies and measure their impact.
Videos in Assessment: Videos in Assessment: The White Swallows III (2015) video.health-rights.org/index.php/list/product/view/1/284.html; Community Monitoring (2015) video.health-rights.org/index.php/list/
product/view/1/285.html
* Incomplete data
2 The Importance of a Clear Goal
A clear goal is the focus and foundation of video strategy.

19.
Goal: B
“To raise awareness
toward disability
issues.” - While a
common goal of
advocacy videos,
raising awareness is
not enough. What
do you want to
happen after people
are aware? Sketch
the full theory of
change to keep
yourself focused.
Audience: I*
“The video has a
positive impact on
a u d i e n c e - t h e
audience is convinced”
- There is an awareness
that the audience is an
impor tant par t of
video strategy, but
there is no actual
identification of who
the audience is.
Story: A
Very good storytelling.
Continue to push your
team - what are the
best visuals and most
important messages.
(“All Boys” is my
favorite video in the
assessment - MJ)
Distribution: A-
The video was actively
distributed on TV, social
media, and through public
events. You were also
attentive to evidence of
audience exposure, such as
comments and likes.
However, without a clear
idea of who should be
seeing the film it’s hard to
tell whether the right
distribution methods were
used.
Impact: B-
I appreciate your honesty in
saying you felt you couldn’t
measure impact directly. This is
because you did not identify a
specific audience, so you didn’t
know where to measure. If you
select an audience, you will be
able to measure impact because
you will know where to seek
evidence of effect. Remember
that you cannot say a video had
a “positive impact” if you have
no evidence of audience effect.
Production: A-
Nice simple filming and
titles.
Grantee Recommendation
Your film has a clear and excellent message and the filming is simple and well-done. Now it’s time to build on that by working on the other elements of your
strategy so you can get a bigger impact next time. The next time you create a video, create your strategy first. Here are the basics of linking all the elements
above into a clear strategy. The purpose of any advocacy video is to persuade an audience, either to take an action or to change their mind about something.
The action/mental change they are persuaded to undertake is the goal. The argument the video makes to persuade them is the story. How you will make sure
the audience sees the video is distribution. When you measure impact you are looking for evidence as to whether the audience took the action (easier to
observe) or changed their minds (harder to observe). If you follow this strategic process you will get more value out of your future videos - and the ones you’ve
already made. Good luck!
Videos in Assessment: Institution or community? (2015) video.health-rights.org/index.php/list/product/view/1/318.html; Returning Home (2015) video.health-rights.org/index.php/list/product/view/1/320.html; I wish
all boys from residential institutions could live in a family (2013) video.health-rights.org/index.php/list/product/view/1/319.html
* Incomplete data
KEYSTONE HUMAN SERVICES3 The Problem with “Raising Awareness”
“Raising awareness” is too vague a goal for precise and effective strategy.

20.
Goal: B
“[T]o raise awareness around
sex workers human rights and
health issues, as well as the
need for change in law” - You
identify the direct effects of the
video (raising awareness) and
the ultimate impact you are
aiming for (change in the law),
but you don’t link them. Will
raising awareness lead directly
to changing the law? If not,
what are the intermediate steps
and how will you move through
them?
Audience: I*
“Change in the law”
i m p l i e s a
g o v e r n m e n t a l
audience, though none
is specified. The
individuals who would
be targeted to sign
the e-petition are also
no specified.
Story: A
Very clear, well
done.
Impact: C-
You write that you cannot
measure impact yet, but since
you do not even note the
number of signatures on the e-
petition, I suspect the number is
lower than you would have
wished, rather than being
uncountable. You also “hope”
the video “reach[es] people and
bring[s] about change,” which is
oddly passive given that it is the
aim of your own campaign to
achieve these ends. Don’t give
up. You can make it happen!
Recommendation
Online advocacy is not a silver bullet. It just implies a new set of tools. Making an (admittedly high quality) digital video does not imply virality. Placing that video
online does not guarantee engagement in an online tactic like e-petition signing. In fact, because there is so much content online vying for our attention, active
promotion to clearly specified audiences is critically important. Although I sense that this effort might have been discouraging, you’ll get a better result if you
engage in more intentional video strategy. Here are the basics: The purpose of any advocacy video is to persuade an audience to take action (or change their
mind). The action they are persuaded to do is the goal. The argument the video makes to persuade them to take that action is the story. How you will make
sure the audience sees the video is distribution. And when you measure impact you are looking for evidence as to whether the audience took the action (or
changed their mind, though that is harder to measure). Good luck!
Videos in Assessment: Decriminalise Sex Work Now!(2015): video.health-rights.org/index.php/list/product/view/1/41.html
* Incomplete data
SEX WORKER EDUCATION AND ADVOCACY
TASKFORCE (SWEAT)
Distribution: B
The video was posted on
its own page on your
website, but this really isn’t
enough. There is no
m e n t i o n o f a c t i v e
outreach (for example, via
listservs and social media)
to drive traffic to the
page. You also do not
specify the audiences you
were trying to reach with
each method.
Production: A
G o o d s i m p l e
animation.
4 The Importance of Active Distribution
Simply posting a video online isn’t enough to engage an audience.

21.
CEVA DE SPUS
Goal: A-
“[T]o change the
mentality of the public
towards people with
intellectual disabilities...
t o p r o v e [ t h e i r ]
abilities" - Ambitious,
but I’d like to see the
video linked to a more
s p e c i fi c a n d
measurable action from
the audience. Perhaps
t h e r e a c t i o n yo u
w a n t e d f r o m t h e
U n i v e r s i t y
administrators you
mention later?
Audience: B
“[T]he open public” -
Trying to target the
public is a common
mistake. It’s too broad
t o b e t a r g e t e d
effectively. “[D]ecision
makers... civil society
or ganizations and
donors” - These are a
lot clearer.
Story: A-
Wonderful videos - so
clear why we should all
have equal rights. Think
about more visuals. I
know these were
created as simple
portraits - but I also
know that Zoli is very
skilled at capturing
small moments which
show the personality of
a person.
Production: A
Good filming, Zoli!
Distribution: A
"[U]ploaded on social
media and website...
different events... in press
releases.... projections at
different events, round
tables, conferences."
Diver se and active
distribution. Would
benefit from a bit more
l i n k a g e b e t w e e n
distribution method and
audience . Which
audiences were you
hoping to reach with
e a c h m e t h o d o f
distribution?
Impact: A
“The west University started
an accessibility project, they
make efforts to fulfill all of our
demands... For sure this video
had a small contribution to
the declaration of Ministry of
Labor regarding starting the
deinstitutionalisation process”
Though I wish there were
some evidence behind the
claim about the MoL, the
p o l i c y c h a n g e a t t h e
University is a great example
about how focused pressure
on specific institutions can be
a starting place for broader
social change.
Recommendation
We are both huge fans of this video series. You make it so obvious why no one should be institutionalized permanently. You’ve used video to realize concrete
advances towards broad social change. Next time you make a video, be as specific as you can about who you are trying to persuade and what you want to
persuade them to do or think. This will prevent you from trying to target overly broad audiences like “the open public.”
Videos in Assessment: There is life outside the walls (2013) video.health-rights.org/index.php/list/product/view/1/334.html; Andrei (2012) video.health-rights.org/index.php/list/product/view/1/335.html; Raluca (2011)
video.health-rights.org/index.php/list/product/view/1/336.html
5 The Problem withTargeting the “Public”
This grantee made the common mistake of selecting an audience too broad to target effectively.

22.
Goal: A+
A very clear goal: to
help NGO’s in other
countries learn from
N Z P C ’ s b e s t
practices as they
m a k e s i m i l a r
decriminalization
effor ts. Clearly
connects the effect
of the video on the
audience (learning)
to the desired impact
( i m p l e m e n t i n g
decriminalization
internationally).
Audience: A+
“ [ S ] e x w o r k e r
o r g a n i s a t i o n s
internationally” - A
strategically logical
audience given the
goal.
Story: B+
The message is clear
as a report about
the collective, but
less of an advocacy
message.
Impact: B
“We believe the video has
had a positive impact as sex
workers in other countries
are using the information
within it to build support
for decriminalisation” - This
could be true, but there’s
no evidence provided to
support your claim. (How
do you know it is being
used?) This is the only place
where you video strategy
falls short. The rest is
excellent.
Production: B+
Clear sound and visuals.
Recommendation
You are a great example of the strategic use of a promotional video. It is very rare for a video promoting an organization’s past work to have a clear and logical
goal, audience, and distribution method, but you’ve done it. Congratulations! The place for you to focus now is on measuring your impact. You could easily get a
sense of the impact of your video by following up with the people who ordered the DVD to see how they used it or by following up with anyone who contacted
your through the video website.
Videos in Assessment: Decriminalisation of Sex Work in New Zealand (2015): video.health-rights.org/index.php/list/product/view/1/314.html
NEW ZEALAND PROSTITUTES COLLECTIVE
(NZPC)
Distribution: A+
Your distribution methods
makes a lot of sense given
your audience and goal.
The video is available via
DVD (perhaps for NGOs
w i t h o u t i n t e r n e t
connection) as well as on
a purpose-built website. I
also like that the video
was actively distributed to
other NGOs, rather than
simply letting it sit on a
website.
6 Impact Without Evidence
This grantee’s strategy was really strong until the impact assessment stage, when they failed to measure the effect of their work.

23.
ENGLISH COLLECTIVE OF PROSTITUTES
Goal: B
“1. to publicize our legal
victories... 2. To build the
m o v e m e n t f o r
decriminalisation.“ - The
value of the first goal is
unclear. What future
benefit will result from
publicizing these past
victories? The second
goal is a bit vague. What
exactly does “build the
movement” mean here?
How will the video
accomplish this?
Audience: C+
”[M]any individuals and
organisations” - Who
exactly? Why is it
important that they
h e a r a b o u t yo u r
victory? What role do
t h e y p l ay i n t h e
movement you are
building?
Story: B
It's a very good
video - but we
need sex worker
voices. Without
their voices we
can't be 100% sure
this is what they
are asking for.
Distribution: B
“[S]howing in December
2014” - Who was in the
a u d i e n c e t h a t yo u
wanted to persuade?
“sent the film to our
network” - To what end?
What actions were you
hoping for as a result?
Impact: B-
“We...have been invited
t o s p e a k a t fi l m
showings on a number
of occasions.” - This
beneficial outcome
seems more by chance
than by design. There’s a
disconnect between
goal and impact.
Production: A
Good filming - both
interior and exterior
(protest) shots are
captured clearly.
Recommendation
I know you are awesome activists and I love both the spirit and the method of your activism, so I am going to be a little tough on you (as in touch love) regarding
your video strategy. I get the sense that there wasn’t really a clear vision of what benefit you were hoping to get out of this video. As a result, the audiences
targeted, the distribution method used to reach those audiences, and the impacts recorded seem rather haphazard. Bottom line - advocacy video is about
persuading and audience to take an action (often mediated by a change in state of mind) and impact is about measuring the extent to which that action was
taken. If you are specific about the action you want from the beginning, when you set your goal, the rest of the strategic process will be a lot easier.
Videos in Assessment: SohoTrot (2015): video.health-rights.org/index.php/list/product/view/1/312.html
7 When Goal and Impact are Disconnected
A big goal (movement-building) leads to minimal benefits (speaking invites) without clear causal steps connecting the two.

24.
EURASIAN NETWORK OF PEOPLE WHO
USE DRUGS (ENPUD),ALLIANCE UKRAINE
Goal: B
“ F o r d r a w i n g
attention to the
problem of clients of
OST program in
Donbass“ - Focus
more on the action
y o u w a n t t h e
audience to take.
Whose attention? To
what end?
Audience: I*
“ [ D ] r a w i n g
a t t e n t i o n ,” b u t
whose attention? In
t h e a b s e n c e o f
audience analysis,
realizing impact is
very difficult.
Story: A
You’ve done a really good
job eliciting stories from
the protagonists. Their
discussions of their
families and emotional
lives humanize them. I
would have liked to see
someone who is a
recovering addict give a
tour of the city he knew
as an addict.
Distribution: B+
“A lot of famous mass-
media, such as BBC... key
events of EEC region” -
Though getting coverage
from a media partner
with a wide audience is
impressive, this is only
meaningful if the BBC’s
audience is the audience
you want to reach to
achieve your goal. Also,
remember that reaching
an audience is evidence
of distr ibution, not
evidence of impact.
Impact: B
It’s possible that the
media exposure resulted
in actions helpful to the
OST clients, but there’s
n o e v i d e n c e .
Remember that views
are not impact, action is.
Next time, think about
who has the power to
help and then target
them specifically.
Production: A-
The filming is of good
quality, but the visual
choice (the candle, for
example) could relate
to the subject matter
more directly.
Recommendation
Your passion about this topic really comes through and the suffering of the film participants is clear. For a crisis as challenging as this one,
filmmakers need to connect to other partners who are capable of carrying out more direct (in-person) advocacy with audiences capable of
providing practical assistance to those in need. If filmmakers are not currently connected to these actors, OSF could likely assist in making those
connections. Massive humanitarian crises cannot be addresses by filmmakers working alone. They need help.
Videos in Assessment: Slow Death in Ukraine (2015) video.health-rights.org/index.php/list/product/view/1/248.html; Live or Die in Donbas? (2015) video.health-rights.org/index.php/list/product/view/1/249.html;The
First CrimeanVictims(2015) video.health-rights.org/index.php/list/product/view/1/268.html;The Right to be Dependent - Irina (2011) video.health-rights.org/index.php/list/product/view/1/67.html
* Incomplete data
8 Media Coverage ≠ Impact
Media coverage is a mechanism of distribution, not a measure of impact.
OST = opioid substitutions therapy

25.
Goal: A-
“ [ D ] e m a n d s f o r
decriminalisation” - This is
a good campaign goal (or
even or ganizational
mission), but I’d like to see
a goal specific to the
video. What role should
t h e v i d e o p l a y i n
convincing politicians to
make this change? Do
t h e y n e e d s p e c i fi c
information? A more
general emotional appeal
to empathy to lay the
groundwork for later
policy arguments?
Audience: B
“[O]ur contacts...
other networks and
organisations” - I’d
like to see more
clarity about why
e a c h a u d i e n c e
needs to see the
video.
Story: B+
The fact that sex
workers were willing
to be filmed without
anonymization is very
rare. Yet the power
of their self-advocacy
is muted by the lack
of a narrative arc.
Distribution: B+
Online methods seems to
be the primary means of
distribution, including an
online training and social
media. Again, there
doesn’t seem to be much
clarity on who needs to
see the video, thus also a
lack of clarity on how to
reach that audience.
Impact: B-
"I don't know. The video was
seen 3,500 times (July 2015)
so we assume it has allowed
many people to make
themselves familiar with our
message." - I appreciate your
h o n e s t y a b o u t y o u r
uncertainty in measuring
impact. I also like that you are
looking for obser vable
evidence This is the right
direction. In the future, note
that views are a measure of
distribution, not impact. What
did you want people to do
after they received your
message?
Production: A-
A nice use of natural
light. Good looking
interviews - but think
about visuals that can
also tell a story.
Recommendation
Your video contains truly excellent self-advocacy from a range of sex workers across the European region. Next time you create a video, create your strategy
first. The purpose of any advocacy video is to persuade, so a video strategy needs to identify who needs to be persuaded (audience), what they need to be
persuaded to do (goal), the argument the video will make to persuade them (story), how you will make sure the audience sees the video (distribution), and how
you will know whether the audience persuasion took place (impact measurement). In particular, try to narrow down your goal to a specific change in state of
mind or action needed from a specific audience, which will further your larger goal of decriminalization. This focus will help you develop a distribution plan linked
more directly to specific audiences and you will have a clearer idea of whose reaction to the film you need to measure. Remember that it’s easier to measure
action than changes in state of mind, so see if you can link the change you want from your audience into an observable action.
Videos in Assessment: Sex workers are the solution not the problem - Sex work, HIV and human rights (2014): video.health-rights.org/index.php/list/product/view/1/303.html
INTERNATIONAL COMMITTEE ONTHE
RIGHTS OF SEX WORKERS IN EUROPE (ICRSE)9 VideoViews ≠ Impact
Though they are easy to measure, video views are evidence of distribution, not of impact.

26.
Goal: A+
“[T]o raise surrpport
[sic.]” among “families
[of] women living with
Hiv/Aids” - A modest
goal but one that is
clearly tied to the video.
“[T]o raise community
awareness to counter
stigma” - Clearly links the
effect of the video
(raising awareness) to
b e n e fi c i a l i m p a c t s
(reducing stigma, raising
suppor t). Linking
awareness raising to a
broader purpose is so
rare - great job!
Audience: A+
“ [ C ] o m m u n i t y
members... families [of]
women living with Hiv/
Aids”” - The audiences
are specific and are
clearly tied to the
goal. Excellent!
Story: B+
Keep trying to find
people who are
willing to have their
faces shown on
camera. I know this is
a sensitive subject,
but there will be
some champions
who want to tell
their story.
Distribution: A+
“The video was viewed by
over 1000 community
m e m b e r s i n t h e i r
homesteads where trained
community volunteer s
facilitated discussions on
issues raised by women in
the video.” - An excellent
e x a m p l e o f l i n k i n g
distribution method to
audience in order to address
a specific goal. Also excellent
use of quantitative data as
evidence. Despite the fact
that the video is digital,
distribution is all offline. This
is exactly what what needed
to reach your rural audience.
Impact: A
"The women involved in making
the video and in facilitating the
discussions reported qualitative
changes with families showing
increasing support to women
living with Hiv/Aids.” - Using
qualitative data (oral reports),
this impact links back directly to
the initial goal of the video.
Again, it is rare to see this type
of clear linkage. It would be
nice to see some kind of impact
assessment of the awareness-
raising/stigma-reducing goal, and
there is some confusion
between distribution (views)
and impact (action), but this is
really good work.
Production: A-
Good visuals. Think
about shortening the
video - a shorter video
will often be more
powerful.
Recommendation
Congratulations! Your video has the strongest strategy of all the grantees in this assessment! Your goals, audience, distribution, and impact are all clearly linked.
The use of evidence (views, oral reports) to support your impact claims is also excellent. You should be teaching other NGOs how to create video strategies!
Videos in Assessment: TheTriple Burden (2015): video.health-rights.org/index.php/list/product/view/1/302.html
JUSTICE AND WOMEN CHARITABLETRUST10 Simple Strategy Wins the Day
This grantee’s simple strategy with clear causal linkages led to a beneficial outcome clearly connected to their use of video.

27.
IMPACT
Image: Flickr/Tom Blackwell
Findings
Image: Flickr/Very Quiet
APPENDICES

28.
• AdvocacyVideo: Recording of moving images designed to benefit a population or individual whose interests
are marginalized or under-represented.
• Audience: Individuals whose change of mind or action would have a beneficial effect.
• Benefit: Concrete improvement in quality of life.
• Distribution: Means of ensuring the audience sees the video.
• Goal: Desired outcome. Video goals should reference both the direct effect on the audience and how this
effect will lead to benefits for the grantee’s population of interest.
• Impact: Extent to which the desired audience effect and population benefits occurred, as demonstrated by
observable evidence.
• OST: Opioid substitution therapy, a form of harm reduction in which users of opiates, such as heroin, are given
substitution drugs, such as methadone, to manage their addictions.
• Production:Technical elements of the video, including sound, lighting, and editing.
• PromotionalVideo: Recording of moving images touting past successes, often for the purposes of gaining
funds for future work.
• Story: Narrative structures and characterization used to persuade the audience to undertake an action or
change of mind.
• Strategy: A plan for achieving a goal.
• Theory of Change: A series of causal steps that explains why it is logical that the actions in the strategy will
achieve the goal. While the strategy describes the actions of the advocate, the theory of change takes into
account the broader context in which the advocate is operating.
GLOSSARY
Video AdvocacyTerminology

29.
VIDEOTHEORY OF CHANGE
Basic Model
Desired
Realized
Marginalized
Population
Audience
cc Mary Joyce, 2015
Goals &
Planning
Impact and
Evaluation
Campaign
Goal
Desired benefit for
marginalized
population
1
Video
Goal
Desired effect on
audience
2
34
Video
Impact
Realized effect on
audience
Campaign
Impact
Realized benefit for
marginalized
population
Start goal-setting here.
End evaluation here. Campaign Video
Compare
to evaluate
Compare
to evaluate
Audience
action
ID audience
that can
help population
cc

30.
Desired
Realized
Marginalized
Population
Audience
cc Mary Joyce, 2015
Campaign Goal:
Desired benefit for marginalized population
1 2
34
“Stop Sending TB Patients to Prison” by KELIN
Video Goal:
Desired effect on audience
Video Impact:
Realized effect on audience
Campaign Impact:
Realized benefit for marginalized population
VIDEOTHEORY OF CHANGE
Example: KELIN
cc

31.
IMPACT
Image: Flickr/Tom Blackwell
Findings
Image: Flickr/Very Quiet
1
THANK YOU!
Mary Joyce | contact@maryjoyce.com
Video Strategy Analysis, Findings and Recommendations, InformationVisualization and Models
John Kennedy |Video Content and Production Analysis | 505050john@gmail.com