Inspiring consumers to adopt a vegan lifestyle is complicated by many opposing influences. For many people, veganism is a long-term journey. Activists must continue on a steady and prolonged course of interactions with consumers to keep vegan messaging present in consumers’ minds and dietary choices. This information was presented by Leslie Barcus, Executive Director of VegFund as part of the new Activist Learning Series. This particular webinar addresses why evaluating behavior change is complex and presents ideas about how activists can gain insights into the success of their advocacy efforts. For more resources like this please visit: https://vegfund.org/resources/

1. The Basics of Project
Evaluation for Effective
Outreach
VegFund Activist Learning Series
November 6, 2018
Leslie Barcus
Executive Director, VegFund
With Special Guest: Pablo Moleman, Proveg International, Netherlands

2. Upcoming webinars scheduled through 2018:
Marketing to Non-Vegans
Tuesday, November 13, 2:00 p.m. to 3:30 p.m. EST / 11:00 a.m. to 12:30 p.m. PST
Presenter: Stephanie Redcross West, Founder and Managing Director, Vegan Mainstream
Learn how to reach and engage non-vegans in your advocacy work!
The Science of Making People Care
Wednesday, November 28, 2:00 p.m. to 3:30 p.m. EST / 11:00 a.m. to 12:30 p.m. PST
Presenters: Ann Searight Christiano, Professor and Frank Karel Chair in Public Communications, University of Florida
College of Journalism and Communications
The scientific evidence is mounting: people don’t act on information; they act on what they care about most.
GDPR: What It Means for Organizations Collecting Consumer Information and DataWednesday, December 5, 2:00
p.m. to 3:30 p.m. EST / 11:00 a.m. to 12:30 p.m. PST
Presenter: Jennifer Mailander, Associate General Counsel, Privacy and Compliance, comScore, Inc.
When Bad Luck Happens to Good Vegfests
Friday, December 14, 2:00 p.m. to 3:30 p.m. EST / 11:00 a.m. to 12:30 p.m. PST
Panel discussion with Richard Rogers, Philly VegFest; Helene Greenberg, Triangle Vegfest; Katelin Rupp, Indy
VegFest; and Jenna Bardoff, Solutionary Events

3. ~The practice of evaluation is
difficult in the best of
circumstances~
Source: A friend who practices evaluation

4. Theory of Change
Action
Reach people and
“convince” them to
consider new ways
of being
We end up with
results or a change
in the world that we
desire to see

5. ‘Theory of Change’
ACTIVITIES:
Film screening
Food sample
Veggie
challenge
Vegfest
OUTPUT:
Individuals
newly exposed
to new,
knowledge and
practices
Ultimate Desired
Result
People Stop
Eating Animals
Longer-term
OUTCOME:
Observed changed
attitudes, leading to
changed behaviors
(practices)
Theory of Change Components

6. ‘Theory of Change’
ACTIVITY:
Host a film
screening
OUTPUT:
People exposed
to new
information &
ideas
(Knowledge)
Ultimate
Long-run
Result:
People Stop
Eating Animals
LONG-TERM
OUTCOME:
Individuals
begin to modify
their dietary
choices
(Practice)
Assumptions:
• Screening organized
and advertised
• People attend the
screening
• Attendees think about
the screening visuals
and script content
Assumptions:
• They believe the
information
• They know how to act
on the information
• Their family & friends
support changes
• Vegan recidivism will
decline
Assumptions:
• They retain
new knowledge
beliefs, or
attitudes, feel
comfortable
with the change
and will go and
stay vegan
Assumptions:
• 100 people will come
who are not vegan
• 50 people will come
who are vegan
Adding Assumptions

7. ‘Theory of Change’
ACTIVITY:
Host a film
screening
OUTPUT:
People
exposed to
new
information
(Knowledge)
Ultimate,
Long-run
Result:
People Stop
Eating Animals
OUTCOME:
Individuals
recall film
messages
and modify
their dietary
choices
(Behavior)
Measurement:
• How many
screenings
held?
• My hours spent
on this activity
• The cost to hold
the screening
Measurement:
• # of people who
were sent
information about
the screening date
• # of people who
came
• # of people who
stayed for the
entire
documentary
Measurement:
• What proportion of
target population
recall the
information from
the screening 3-
months later?
Measurement:
• How many people
ate meat 9 months
after the
screening?
Adding Measurement

8. Defining Evaluation
Evaluation is a systematic method for
collecting, analyzing, and using
information (or data) to answer questions
about outputs and outcomes defined
within the design (assumptions, activities,
inputs) of projects, policies and programs.
If all works to our design and plan, we
hope to observe causality to our desired
results.

9. What Evaluation (hopes to) Answer:
✓ Did our assumptions about outputs and outcomes hold?
✓ At what scale?
✓ Were we cost efficient and cost effective?
✓ Did we observe our anticipated result(s)?
✓ Did our activity show causality for the result?

10. The Nature of Evaluation
1. Evaluation work is an actual
academic and professional practice
and not just a casual past-time.
2. Vegan outreach and the
evaluation thereof involves the
design and implementation of events
for people who are subject to many
influences making the activist’s
attempts at evaluation even more
difficult.

11. Vegan activism is about human behavior change

12. …and is human behavior is profoundly influenced by
society

13. We seek to change people’s KNOWLEDGE

14. We seek to change people’s ATTITUDES

15. …..with the desired outcome that people change their PRACTICES/BEHAVIORS

16. We should design our actions for the baseline of knowledge, attitudes
and practices (KAP’s) of our audiences.
Action
Reach and
“convince”
people to
act
That change
or results we
see in the
world
Considering
KAPs, we
DESIGN
appropriate
interventions
Think about your action with the desired result in mind,
but shape your action to the realities of your target audience

17. Effective activism requires its
own set of KAPs
The Vegan Activist: the other key human element in
vegan outreach!

18. Logical Thought #1
If we ignore the KAPs of the
people we want to reach, do
not design appropriate
interventions for them and fail
to examine our own KAPs as
activists, then we should not
anticipate our desired results.

19. Logical Thought #2
Vegan outreach and the
evaluation thereof is for
patient people.

20. Humans don’t change quickly, on average
Diffusion of Innovation Theory

21. Another look at innovation diffusion
Diffusion of Innovation Theory

22. Stages of Change
Source: Prochaska and
DiClemente/Transtheoretical
Model
With each vegan
outreach event or
message, your
non-vegan
audiences are
hopefully building
their knowledge
about veganism
and its benefits
and changing their
attitudes about
vegan living
Your non-vegan
audiences are
building their practice
to live and maintain a
vegan lifestyle with
each vegan
intervention or
message
Vegan
recidivism

23. Logical Thought #3
This all sounds gloomy….why did
I attend this webinar? Is there
any hope to evaluate my efforts?

24. Effective outcomes
We can observe at least
incrementally changed
outcomes in a person with
respect to her knowledge,
attitudes, or practices
resulting from an exposure
to some intervention.
Results
Seeing evidence for sustained
behavior change that results in
the achievement of a higher goal
is a long-term, costly and difficult
endeavor.
Gathering Evidence For Effectiveness

25. Single Interventions
• Documentary screenings
• Food sampling
• Paid per view
• Etc.
By which we mean some form
of short-term exposure in a
single day or a short time
period.
Programs
A longer-term veggie challenge,
encompassing ongoing mentorship, the
provision of recipes, email support, access to
vegan-meet up groups, shopping
suggestions, cooking skill development, etc.
with the measurement of progress toward
dietary change at different time intervals and
at some subsequent time period in the future
after the completion of the challenge
Single Interventions vs. Programs

26. Four Levels of (Quick and Dirty) Audience Evaluation
(from a training model)
1.(Immediate) Reaction to an intervention or message
2. Learning - the resulting change in knowledge and change in
attitudes measured immediately at the end of the intervention
3. Behavior Change - transfer of and observed knowledge,
practice, and/or attitudes from the intervention to changes
in behavior (typically this step is measured roughly 3-6 months
after the intervention.
4. Program Impact - the final results that occurred because
of participation.
More
difficult
Easiest to
measure!
Most
difficult

27. An example of capturing reaction of a film screening
“Thank you for joining us for the screening of the documentary
XX today. Even if the images and messages you just saw in
the film may have been difficult for you to watch, how do you
feel about what you learned from the film? Please circle one.”
I learned a lot and want to explore more about vegan living!
I am thinking about what I learned, including if I could ever live a
vegan lifestyle.
😳 Wait, was I supposed to learn something?
😤 I got nothing out of that film!

28. Examples of tools for capturing audience knowledge learned, new
attitudes and potential adopted practices

29. Let’s take a look at the evaluation of a
program that considers outputs,
outcomes (behavior/practice change)
and results (impact)

30. VeggieChallenge
International

31. 31
personalized
challenge
personalized
start-date

32. 32

33. Optimizing for omnivore engagement
Omnivore
40%
Vegetarian
4 – 9%
Vegan
0,5 – 1,1%
omnivore vegan
Reducetarian
50 – 56%
2 animals
per year
0 animals
per year
27 animals
per year
13 animals
per year

34. Optimizing for omnivore engagement
34

35. Coaching
Optimizing diet change at every stage
of the veg*n journey
35
● Personal support over e-mail
● Peer support through Facebook group
● Personalized content and statistics
● Recipe database + easy weekly menus
● Product database + weekly discounts
● Restaurant database
● Health information
● Biweekly livestream with professional
dietitian
Hi! How I h yo ?

36. Our vegan making machine
Events
Social
media
Screen
ings
Activist
s
Donors
Market
eers
Leaflet
s

37. Optimizing diet change
37

38. 38

39. Online advertising
39
2017 budget: € 17.291
● Total leads: 12.717
● Avg. € 1,36 per lead
● Avg. € 0,18 per animal
saved/year
96.184 animals saved!

40. Challenges
- sample size
- response bias
- social desirability bias
- long-term effect?
Possible solutions
- gamification/direct feedback
- long term measurements
- measure attitudes as well as diet change
- other metrics

41. Impact Report 2017
41

42. 42
● Allows for highly specific targeting and impact measurement
Online advertising
Health Taste Environment

43. Logical Thought #4
Neither single interventions (a film screening) nor
a longer-term program (a vegan challenge with
multiple touch point over time) may yield the full
results (impact) we desire; however, we each
step may keep an aspirational vegan on a
journey of behavior change that can save animals
lives.

44. The Primary Take Away Ideas of this Webinar
✓ For the majority of people, the committed adoption of vegan living will involve a lifetime
journey of learning, trying, failing and trying again!
✓ Effective outreach begins with an activist’s understanding about the existing knowledge,
attitude and practices of her audience with regard to vegan living.
✓ As a vegan activist, your knowledge, attitudes and practices about how to conduct
effective outreach is a vital component to your success and will affect the outcomes and
results designed within your Theory of Change.
✓ Measuring the long-term, sustained behavior change of our audiences and the results of
their choices is a long-term, expensive endeavor and measuring behavior change and
causation of results at scale will be cost prohibitive. But, there are reactions and learning
we can observe about our audiences, collaborate with others who share our goals and
look to external market-based indicators (what is the market share of non-dairy milks, for
example) to appreciate the desired change we want to see in the world.

45. Evaluation Resources, available on
Amazon.com

46. Thank you for joining us!
1. We would deeply appreciate your
feedback today
2. Other webinars:
https://vegfund.org/news/six-new-webinar
s-to-step-up-your-activism/