The document discusses logic models, which are visual representations of programs outlining activities, outputs, and expected outcomes. It distinguishes between outputs (what a program does) and outcomes (the changes resulting from the program), elaborating on the importance of defining short-term, intermediate, and long-term outcomes. It also highlights the significance of understanding confirmatory and exploratory outcomes in evaluation and the contextual factors that can impact program success.

1. Let’s Talk About
Logic Models
Isaac D. Castillo
@isaac_outcomes
December 12, 2023

2. 12/12/2023 2
Castillo – Logic Models
Isaac Castillo
Senior Advisor for Learning and Evaluation at KABOOM!
Venture Philanthropy Partners
DC Promise Neighborhood Initiaive
Child Trends
LAYC
Cosmos Coporation
President of the Eastern Evaluation Research Society

3. 12/12/2023 3
Castillo – Logic Models
What is a Logic Model?
• Visual representation of a program
• What a program does and what you hope to achieve
• Can be simple or complex, can be linear or not
• There is no ‘best’ format, just what works best for you

4. 12/12/2023 4
Castillo – Logic Models
History of Logic Models

5. 12/12/2023 5
Castillo – Logic Models
United Way Logic Model

6. 12/12/2023 6
Castillo – Logic Models
Kellogg Foundation Logic Model

7. 12/12/2023 7
Castillo – Logic Models
Modern Day Logic Models
Inputs Activities Outputs
Short Term
Outcomes
Intermediate
Outcomes
Long Term
Outcomes

8. 12/12/2023 8
Castillo – Logic Models
Outputs vs Outcomes
OUTPUTS OUTCOMES
• What actually happened
when you offered your
activities.
• Who received your services
and in what amount.
• Changes in knowledge,
attitudes, behaviors, or
conditions among your
participants (those you serve).
• What changes actually
happened among your
participants as a result of your
activities.

9. 12/12/2023 9
Castillo – Logic Models
Outputs
• Output measures assess what you do and who you serve. Examples include:
• Served 100 youth during summer camp
• Provided 2,250 hours of tutoring during the academic year
• 9 out of 10 youth attended at least 75 % of available art instruction classes
offered
• Outputs DO:
• Tell you about whether your program delivered services as intended.
• Outputs DO NOT:
• Tell you if participants benefited from your program
• Serve as indicators of program success or effectiveness

10. 12/12/2023 10
Castillo – Logic Models
Outcomes
• Outcome measures assess changes in your service population. Examples include:
• 75 % of youth increased their knowledge of local history during the summer camp
• 50% of youth increased math grades by one grade level during the academic year
• 25% fewer youth reported being involved in bullying over the last year
• Outcomes DO:
• Tell you if participants benefited from your program
• Serve as indicators of program success or effectiveness
• Outcomes DO NOT:
• Tell you about whether your program was implemented well (or provide clues
about how your program improved participant outcomes)

11. 12/12/2023 11
Castillo – Logic Models

12. 12/12/2023 12
Castillo – Logic Models
Where do people get stuck with outputs?
• Do I track number served or dosage?
• Both if possible.
• Do I include targets or goals in the logic model?
• It depends.
• Examples:
• # of children served (or participating in program)
• % of classes / sessions attended (or attendance rate)
• # or % that attended at least 75% of sessions (regular participants).
• X# served or X% attendance – this is a target.

13. 12/12/2023 13
Castillo – Logic Models
Examples of Outputs
OUTPUTS OUTCOMES
• # of youth served
• % of classes / sessions attended (or
attendance rate)
• # or % that attended at least 75% of
sessions (regular participants).
ACTIVITIES

14. 12/12/2023 14
Castillo – Logic Models
Examples of Outputs
OUTPUTS OUTCOMES
• # of teens served
• % of classes / sessions attended (or attendance
rate)
ACTIVITIES
ACTIVITY 1
ACTIVITY 2
ACTIVITY 3
• # of adults served
• % of classes / sessions attended (or attendance
rate)
• # of young children served
• % of classes / sessions attended (or attendance
rate)

15. 12/12/2023 15
Castillo – Logic Models
Examples of Outputs
OUTPUTS OUTCOMES
ACTIVITIES
ACTIVITY 1
ACTIVITY 2
ACTIVITY 3
• # of youth served
• % of classes / sessions attended (or
attendance rate)
• # or % that attended at least 75% of
sessions (regular participants).

16. 12/12/2023 16
Castillo – Logic Models
Where do people get stuck with outcomes?
Short Term
Outcomes
Intermediate
Outcomes
Long Term
Outcomes
• There is no commonly accepted definition of each of these or agreed upon
timing.
• What constitutes short, intermediate, or long? Are long term outcomes stuff that
happens after program is completed? Are short term outcomes things that
happen within the first 50% of a program?
• One program’s long term outcomes can be another program’s short term
outcomes.

17. 12/12/2023 17
outputs
LEVEL 1
OUTCOMES
LEVEL 2
OUTCOMES
LEVEL 3
OUTCOMES
LEVEL 4
OUTCOMES
Castillo – Logic Models
Linkages between outcomes

18. 12/12/2023 18
Castillo – Logic Models
Outcomes to Larger Change
SHOWING UP
(PROGRAM
ATTENDANCE)
CHANGES IN
KNOWLEDGE
CHANGES IN
ATTITUDES
CHANGES IN
BEHAVIORS
CHANGES IN
CONDITIONS

19. 12/12/2023 19
Castillo – Logic Models
One last way to think about outcomes
Short Term
Outcomes
Intermediate
Outcomes
Long Term
Outcomes
Confirmatory
Outcomes
Exploratory
Outcomes
Aspirational
Outcomes

20. 12/12/2023 20
Castillo – Logic Models
One last way to think about outcomes
Confirmatory
Outcomes
Exploratory
Outcomes
Aspirational
Outcomes
• Outcomes you expect will
change because of program
participation.
• You will hold yourself
responsible for confirmatory
outcomes for all (or the
majority) of your regular
participants.
• You will devote adequate
resources and time to
achieving confirmatory
outcomes.
• You will measure your
confirmatory outcomes
• Outcomes potentially, but not
definitely, improved by your program.
• You think (have a theory) that you
may have an effect on exploratory
outcomes.
• But you will not hold yourself
responsible for exploratory
outcomes.
• You may or may not measure
exploratory outcomes.
• If you do not measure your
exploratory outcomes, you should
have a theory (or prior research) that
supports why you think the
exploratory outcomes are likely to
happen.
• Outcomes that might result
from your work but are more
similar to goals / hopes than
prior types of outcomes.
• The chain of events to get to
aspirational outcomes is long.
• Organization will not hold
itself for achieving aspirational
outcomes, nor will resources
be used to measure them.

21. 12/12/2023 21
Castillo – Logic Models
Confirmatory Outcomes
• Confirmatory outcomes: outcomes you expect will change as a result of program
participation.
• You will hold yourself responsible for confirmatory outcomes for all (or the
majority) of your regular participants.
• You will devote adequate resources and time to achieving confirmatory
outcomes.
• You will measure your confirmatory outcomes

22. 12/12/2023 22
Castillo – Logic Models
Exploratory Outcomes
• Exploratory outcomes : outcomes potentially, but not definitely, improved by
your program.
• You think (have a theory) that you may have an effect on exploratory outcomes.
•
• But you will not hold yourself responsible for exploratory outcomes.
• You may or may not measure exploratory outcomes.
• If you do not measure your exploratory outcomes, you should have a theory (or
prior research) that supports why you think the exploratory outcomes are likely
to happen.

23. 12/12/2023 23
Castillo – Logic Models
Aspirational Outcomes
• Outcomes that might result from your work but are more similar to goals / hopes
than prior types of outcomes.
• The chain of events to get to aspirational outcomes is long.
• Organization will not hold itself for achieving aspirational outcomes, nor will
resources be used to measure them.

24. 12/12/2023 24
Castillo – Logic Models
Confirmatory vs Exploratory Outcomes Example
• A 24-week program designed to teach teens about the dangers of tobacco use
and decrease (or prevent) smoking of cigarettes
• Confirmatory outcome: decrease in the self-reported number of cigarettes
smoked during the past 7 days
• Exploratory outcome: decrease in the self-reported use of alcohol, marijuana, or
other drugs during the past 7 days
• The program SHOULD decrease use of cigarettes. And if the program decreases
cigarette use, then participants MIGHT decrease use of other substances as well.

25. 12/12/2023 25
Castillo – Logic Models
What does this look like?
Confirmatory
Outcomes
Exploratory
Outcomes
Aspirational
Outcomes
Activities Outputs

26. 12/12/2023 26
Castillo – Logic Models
Arrows or No Arrows?
• Arrows can be useful to demonstrate flow or connection.
• Most older / traditional style logic models use arrows, but
there were widely used versions that do not have arrows.
• Leaving arrows out is ok if lots of things are interconnected or
there is no desire to separate out connections.
• Arrows should have a purpose – don’t put them in there to
just have them. They should mean something.

27. 12/12/2023 27
Castillo – Logic Models
Logic
Model with
no arrows
Source: ToE TIG Week: Vacuums, Reviews, and Outputs: Teaching
Logic Models by Ayesha Boyce, January 11, 2022, AEA365 Blog,
https://aea365.org/blog/toe-tig-week-vacuums-reviews-and-out
puts-teaching-logic-models-by-ayesha-boyce/

28. 12/12/2023 28
Castillo – Logic Models
Logic Model with arrows
Source: Designing a logic model by Incept and INWPCP Prevention Alliance, https://www.incept.org.au/planning-evaluation/evaluation-planning-methods/designing-a-logic-model

29. 12/12/2023 29
Castillo – Logic Models
Arrows as references
Youth Matched with
a Promotor
Youth more likely to
get HS diploma or
GED
Solomon, 2021
Youth experience
fewer unwanted
births
Urban Institute, 2016

30. 12/12/2023 30
Castillo – Logic Models
Contextual Factors and Assumptions
• Conditions or other things that could affect how your logic
model succeeds (or doesn’t) or changes.
• Could affect any part of your logic model, or even your data
collection.

31. 12/12/2023 31
Castillo – Logic Models
Complicated Logic Model
Short Term
Outcomes
Intermediate
Outcomes
Long Term
Outcomes
Activities Outputs
Inputs
(Resources)
Assumptions
Contextual Factors

32. 12/12/2023 32
Castillo – Logic Models
Participant Satisfaction and Program Quality
• Not an OUTPUT or OUTCOME, but is very useful in helping to improve
service delivery.
• Frequently used to explain other things:
• Why is attendance so low?
• Why does the program not seem to be ‘working’?
• Participant satisfaction can be high, but the program may not be successful.
• The part of the program people ‘like’ may not be effective.
• Just because something is liked, it doesn’t mean it will create lasting and useful
change.
• But still important to track to identify why/how program can be improved.

33. #3DLM 33
10/28/2020
http://bmafunders.org/wp-content/uploads/2014/03/under-construction-laplazita-impact-model.pdf

34. #3DLM 34
10/28/2020
Medicine Wheel Logic Model
https://bit.ly/2IGHszo

35. #3DLM 35
10/28/2020
https://bit.ly/2v4QapA
Eco Logic Model