LOGIC MODELS,
PROGRAM
EVALUATION,
AND OTHER
FRIGHTENING
TOPICS
EVALUATION?
We are not using all of
the tools available to us.
THINK DIFFERENTLY
ABOUT EVALUATION
time sequences | locations | ideas | events
* Comfort of
interviewee
* Be sincere
* Be conversational
* Open-ended
questions
* Listen
(The phone just
records;
it doesn’t listen.)
Video
Or
Audio
first person
voice
document process
& outcomes
Useful Resource:
"16 tips for making video interviews come alive" by Lindsay Oberst
http://www.socialbrite.org/2011/08/09/tips-for-telling-stories-on-camera/
www.storify.com
www.placestories.com
LOGIC MODEL
the graphic depiction
of the relationship
between your activities
and their
intended effects.
Headache
(SITUATION)
Get Pills
(INPUT)
Take Pills
(OUTPUT)
Feel Better
(OUTCOME)
Get Everybody
on the Same
Page
Build understanding &
promote consensus about
what the program is, and
how it will work.
Make your underlying
beliefs explicit.
Summarize complex
programs to communicate
with stakeholders and
funders.
A logic model
can . . .
EVALUATION HINGES ON ASSUMPTIONS
WHY IS IT IMPORTANT THAT WE
THINK ABOUT ASSUMPTIONS?
As you left the house today
and came to this training,
what were some of your
assumptions about the day?
WHO is going to do WHAT,
WHEN, WHY, and
TO WHAT STANDARD?
Reactions
Learning
Actions
Participation
Social, Economic,
& Environmental
Improvements
Hierarchy of Effects
Bennett and Rockwell, 1995,
“Targeting Outcomes of Programs”
Number and characteristics of people reached;
frequency and intensity of contact
Degree of satisfaction with program; level of interest;
feelings toward activities, educational methods
Changes in knowledge,
attitudes, skills, aspirations
Changes in behaviors
and practices
University of Wisconsin Extension
Limitations
of Logic
Models
They represent reality, but are not reality
(Human relationships formulas).
Programs/projects are not linear
(Nothing ever goes exactly as planned).
They focus on expected outcomes,
not on actual outcomes. (positive or negative
unintended consequences)
They have a tendency to assess what is easiest
to measure rather than what is most valuable.
There can be causal attribution issues
(Variables may not be isolated and many
factors are influencing outcomes).
They don’t address whether you are doing
the right thing, only what you did.
Theory of Change
IF > THEN, IF >THEN, IF > THEN, IF > THEN
"I think you should be more explicit here in step two."
Cartoon by Sidney Harris
We measure what we value
and value what we measure.
Quality of Life vs. Standard of Living
Never a
number
without
a story;
never a
story
without a
number
Skills Incentives Resources
Action
Plan
Confusion=
Vision Incentives Resources
Action
Plan
Anxiety=
Vision Skills Resources
Action
Plan
Resistance=
Vision Skills Incentives
Action
Plan
Frustration=
Vision Skills Incentives Resources Treadmill=
Adapted from Knoster, T. (1991) Presentation at TASH Conference, Washington DC
(Adapted by Knoster from Enterprise Group Ltd.)
Vision Skills Incentives Resources
Action
Plan
Change=
Handout
Know where to focus your efforts
Some Things to Remember
Do no harm, and protect
each other from making mistakes.
Don’t take your partners for granted.
There are
incredibly
smart people
who will help you
if you ask.
You just need to ask.
NEED FIVE MINUTES?
Thinking
About the
Fundamental
Nature of
Higher
Education
0.0000000000
000000000000
000000000001
Don Novello
“Father Guido Sarducci's
Five Minute University”
http://youtu.be/kO8x8eoU3L4
"Virtually every feature of traditional formal
education was created between 1850 and 1919 to
support the Industrial Age."
- Cathy Davidson, Duke University
Alice Loher demonstrates how to load film onto a projector to a Library School class, 1948 (University of Illinois Archives)
“. . . educational technology has moved from being something
that supported classroom teaching and later distance education,
to a force for radical change in our educational systems. . ."
- Tony Bates, distance education guru
“About 500 years ago, the primary mode of
teaching in the university was to come in with
blank sheets of paper and listen to the professor
recite from a manuscript so you could make your
own copy of the book."
- David Wiley, Brigham Young University
image: http://d.lib.ncsu.edu/collections/catalog/0020104
Image: http://unlearningschool.tumblr.com
SCHOOL
LIFE
Higher
Education
Social
Media
Membership in intellectual and social affinity groups X X
Access to resources and experts X X
Engaging in intellectual discussions X X
Accumulate and develop skills for employment X X
Association with professional community X X
Establish social and professional network X X
Enhance personal and professional reputation X X
Share enthusiasm for common interests X X
Build skills X X
“Educators are no longer the gatekeepers of knowledge.”
- Dr. Sarah Smith-Robbins
Higher Education and Social Media
“Educators are no longer the gatekeepers of knowledge.”
- Dr. Sarah Smith-Robbins
Higher Education and Social Media
Higher Education Social Media
Indicator of Success Official endorsement of
completion via diploma
Evidence of work
completed and place in
community
Navigation Guidance through
experiences and thought
processes
Self-directed exploration
of ideas, discussions, and
sources
Investment Money Time
Priority Priority set by Institution Priority set by Individual
Rules Rigid Flexible
Education vs Everyday
Analog  Digital
Tethered  Mobile
Isolated  Connected
Generic  Personal
Consumption  Creating
Closed  Open
Then vs Now
Analog  Digital
Tethered  Mobile
Isolated  Connected
Generic  Personal
Consumption  Creating
Closed  Open
Licensed under Creative Commons By-SA
David Wiley
Education vs Everyday
Analog  Digital
Tethered  Mobile
Isolated  Connected
Generic  Personal
Consumption  Creating
Closed  Open
Civic
Engagement
CONNECTIVISM
Stephen Downes
George Siemens
KNOWLEDGE AND LEARNING CAN
BE DESCRIBED AND EXPLAINED
USING NETWORK PRINCIPLES
To ‘teach’ is
to model
and
demonstrate
To ‘learn’ is
to practice
and reflect
‘an authentic
community
of practice’
What does this remind you of?
Connectivist Pedagogy (Downes)
Role of the Teacher
• To practice one’s
work in an open
manner; to work
transparently
• To ‘work’ is to
engage in a
community
• To be openly
reflective, eg., to write
about the work
Role of the Learner
• To attach oneself to
an authentic
environment
• To observe and
emulate successful
practice
• To be ‘reflective’, ie.,
to engage in
conversation about the
practice
Connectivist Pedagogy (Downes)
Image licensed under Creative Commons by Nongbri Family Pix: http://www.flickr.com/photos/aznongbri/3349618967/ Retrieved from http://dangerouslyirrelevant.org
Image licensed under Creative Commons by craigCloutier: http://www.flickr.com/photos/craigcloutier/3045377851/
Quote source: Learning and Understanding (2002) Principle 2
HOW DO YOU LEARN
NEW THINGS?
HOW DID YOU LEARN NEW
THINGS 20 YEARS AGO?
WHAT IS DIFFERENT?
Each of us
has our own
“Personal
Message
Shield”
http://www.slideshare.net/socialogilvy/the-insidious-plot-to-socialize-the-enterprise
“My contention is that
creativity now is as important
in education as literacy, and
we should treat it with the
same status… we're now
running national education
systems where mistakes are
the worst thing you can make.
And the result is that we are
educating people out of their
creative capacities.”
- Sir Ken Robinson
Photo: http://www.flickr.com/photos/ter-burg/3569194255/ Thanks to: http://www.slideshare.net/trib
“Creativity comes from looking for the unexpected
and stepping outside your own experience.”
- Masaru Ibuka, co-founder of Sony
"In the times of rapid
change, learners inherit
the Earth, while the
learned find themselves
beautifully equipped to
deal with a world that
no longer exists."
- Eric Hoffer
washingtonrebel.typepad.com
ccSteveWheeler,UniversityofPlymouth,2010
Institutional Civic Engagement
Student Community Service or Service-Learning
Institutions
shouldn’t
say one thing
and do another.
No Vision
No Vision Imposed Vision
No Vision Imposed Vision Shared Vision
Teaching
&
Learning
Human
Resources
Material
Resources
Financial
Resources
http://energy.caeds.eng.uml.edu/peru-07/index2.htm
http://energy.caeds.eng.uml.edu/peru-07/index2.htm
http://energy.caeds.eng.uml.edu/peru-07/index2.htm
Civic engagement evaluation

Civic engagement evaluation