CSU Extension, Engagement and the Logic model
Upcoming SlideShare
Loading in...5
×
 

CSU Extension, Engagement and the Logic model

on

  • 2,292 views

Presentation delivered to graduate class Principles of Extension. ...

Presentation delivered to graduate class Principles of Extension.

Much of the material generated in this lecture were from the extension, logic model, scholarship of engagement were taken from the University of Wisconsin-Extension, Program Development and Evaluation program.

http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

Statistics

Views

Total Views
2,292
Views on SlideShare
2,292
Embed Views
0

Actions

Likes
0
Downloads
21
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    CSU Extension, Engagement and the Logic model CSU Extension, Engagement and the Logic model Presentation Transcript

    • Extension and Engagement for the21stCentury Land Grant Institution
      The Logic Model and the Scholarship of Engagement
      Steven E. Newman, Ph.D., A.A.F.
      Greenhouse Crops Extension Specialist
      and Professor of Floriculture
    • Traditional Extension Model
      Outreach and Service
      Client based
      Outreach delivery
      Continuing education
      Laboratory testing
      Field days
      Youth development
      Ag consultation
      Food and nutrition
    • The Scholarship of Engagement
      What is the scholarship of engagement?
      Is it service?
      Service is often seen as somehow outside the "real" work of scholars.
    • The Scholarship of Engagement
      What is the scholarship of engagement?
      Is it service?
      Service is often seen as somehow outside the "real" work of scholars.
      Faculty members can extend their intellectual curiosity into their service activities bringing together their teaching, research, and service in a synergistic way, to the benefit of each aspect of their work and the benefit of those with whom they work.
    • The Scholarship of Engagement
      What is the scholarship of engagement?
      Is it service?
      Is it legitimate scholarship?
    • The Scholarship of Engagement
      What is the scholarship of engagement?
      Is it service?
      Is it legitimate scholarship?
      To make faculty service more legitimate, the institution must treat outreach and service activities as scholarly activities in the same way that research always has been and teaching is increasingly being treated.
      When faculty and administrators finally embrace a scholarship of engagement and acknowledge the important role of service in both the internal and external functioning and health of the campus, then faculty can begin to experience integrated academic lives.
    • The Scholarship of Engagement
      Engagement provides a conduit of information to flow from units not normally considered part of the outreach mission of an institution.
      This must include:
      Stakeholders
      Consumers
      All faculty
      Students
    • Evaluation of Engagement
    • The Logic Model
    • The Logic Model
      Five Components:
      Resources or inputs
      Program activities
      Outputs
      Outcomes
      Impacts
    • A logic model is…
      A depiction of a program showing what the program will do and what it is to accomplish.
      A series of “if-then” relationships that, if implemented as intended, lead to the desired outcomes
      The core of program planning and evaluation
      University of Wisconsin-Extension, Program Development and Evaluation
    • Simplest form
      INPUTS
      OUTPUTS
      OUTCOMES
      University of Wisconsin-Extension, Program Development and Evaluation
    • Logic models can be applied to: a small program, a process (i.e. a team working together), a large, multi-component program, or even to an organization or business.
      University of Wisconsin-Extension, Program Development and Evaluation
    • LOGIC
      the principles of reasoning
      reasonable
      the relationship of elements to each other and a whole
      MODEL
      small object representing another, often larger object (represents reality, isn’t reality)
      preliminary pattern serving as a plan
      tentative description of a system or theory that accounts for all its known properties
      The American Heritage Dictionary, 2nd Ed
      University of Wisconsin-Extension, Program Development and Evaluation
    • “If you don’t know where you are going, how are you gonna’ know when you get there?”
      Yogi Berra
      Where are you going?
      How will you get there?
      What will show that you’ve arrived?
      University of Wisconsin-Extension, Program Development and Evaluation
    • Many people say a logic model is a road map
      University of Wisconsin-Extension, Program Development and Evaluation
    • Logic model may also be called…
      Theory of change
      Program action
      Model of change
      Conceptual map
      Outcome map
      Program logic
      University of Wisconsin-Extension, Program Development and Evaluation
    • What gets measured gets done
      If you don’t measure results, you can’t tell success from failure
      If you can’t see success, you can’t reward it
      If you can’t reward success, you’re probably rewarding failure
      If you can’t see success, you can’t learn from it
      If you can’t recognize failure, you can’t correct it.
      If you can demonstrate results, you can win public support.
      Reinventing Government, Osborne and Gaebler, 1992
      Accountability era
      University of Wisconsin-Extension, Program Development and Evaluation
    • What logic model is not…
      A theory
      Reality
      An evaluation model or method
      It is a framework for describing the relationships between investments, activities, and results.
      It provides a common approach for integrating planning, implementation, evaluation and reporting.
      University of Wisconsin-Extension, Program Development and Evaluation
    • A bit of history
      Dates to late 1960’s
      Current accountability demands; logic model in widespread use
      Public Sector - GPRA
      Non-Profit Sector
      Private Sector
      International Agencies
      Evaluation
      University of Wisconsin-Extension, Program Development and Evaluation
    • Why the hype? What’s the benefit?
      Focus on and be accountable for what matters – OUTCOMES
      Provides common language
      Makes assumptions EXPLICIT
      Supports continuous improvement
      Promotes communications
      University of Wisconsin-Extension, Program Development and Evaluation
    • Logic modelling is a way of thinking…not just a pretty graphic
      “We build the road and the road builds us.”
      -Sri Lankan saying
      University of Wisconsin-Extension, Program Development and Evaluation
    • Everyday example
      H
      E
      A
      D
      A
      C
      H
      E
      Feel better
      Getpills
      Takepills
      Situation
      INPUTS
      OUTPUTS
      OUTCOMES
      University of Wisconsin-Extension, Program Development and Evaluation
    • Everyday example
      H
      U
      N
      G
      R
      Y
      Feel better
      Getfood
      Eat food
      University of Wisconsin-Extension, Program Development and Evaluation
    • Every day logic model –
      Family Vacation
      Family Members
      Drive to state park
      Family members learn about each other; family bonds; family has a good time
      Budget
      Set up camp
      Car
      Cook, play, talk, laugh, hike
      Camping Equipment
      OUTPUTS
      OUTCOMES
      INPUTS
      University of Wisconsin-Extension, Program Development and Evaluation
    • Assumptions
      Assumptions underlie much of what we do. It is often these underlying assumptions that hinder success or produce less-than-expected results. One benefit of logic modeling is that it helps us make our assumptions explicit.
      University of Wisconsin-Extension, Program Development and Evaluation
    • Assumptions
      The beliefs we have about the program, the participants, and how the program will work. Includes ideas about:
      the problem or existing situation
      program operations
      expected outcomes and benefits
      the participants and how they learn, behave, their motivations
      resources
      staff
      external environment: influences
      the knowledge base
      etc.
      University of Wisconsin-Extension, Program Development and Evaluation
    • Assumptions
      As you left the house today and came to this workshop, what were some of your assumptions about the day?
      Why is it important that we think about assumptions?
      University of Wisconsin-Extension, Program Development and Evaluation
    • A youth financial literacy program
      Teens establish sound financial habits
      Teens make better decisions about the use of money
      Teens gain knowledge and skills in money management
      A high school financial planning program – 7 unit curriculum - is developed and delivered in high schools
      Partners invest resources
      University of Wisconsin-Extension, Program Development and Evaluation
    • Business Counseling Example
      Improved business performance
      These owners gain knowledge and change practices resulting in
      A variety of educational activities are provided to business owners who participate
      Agency invests time and resources
      University of Wisconsin-Extension, Program Development and Evaluation
    • Parent Education Program – Logic model
      SITUATION: During a county needs assessment, majority of parents reported that they were having difficulty parenting and felt stressed as a result
      OUTCOMES
      INPUTS
      OUTPUTS
      Parents increase knowledge of child dev
      Assess parent ed programs
      Parents identify appropriate actions to take
      Reduced stress
      Staff
      Parents
      of 3-10
      year
      olds
      attend
      Parents better understanding their own parenting style
      Improved child-parent relations
      Design- deliver evidence-based program of 8 sessions
      Money
      Parents use effective parenting practices
      Partners
      Parents gain skills in new ways to parent
      Research
      Facilitate support groups
      Parents gain confidence in their abilities
      University of Wisconsin-Extension, Program Development and Evaluation
    • Example: Water quality
      University of Wisconsin-Extension, Program Development and Evaluation
    • Logic model of a training workshop
      Situation: Funder requires grantees to include a logic model in their funding request; grantees have limited understanding of logic models and are unable to fulfill the funding requirement
      OUTCOMES
      -Participants will increase knowledge of logic models
      -Participants will increase ability to create a useful logic model of program
      -Participants will Increase confidence in using logic models
      INPUTS
      OUTPUTS
      Fulfill requirement of funder
      Create meaningful logic models
      Use logic models in own work
      3 hour training
      • Interactive activities
      • Group work
      • Practice
      • Q and A
      Trainer
      Funds
      Equipment
      Research base
      Training curriculum
      Grantees
      Improved planning
      Improved evaluation
      Accountable here
      University of Wisconsin-Extension, Program Development and Evaluation
    • If-then relationships
      Underlying a logic model is a series of ‘if-then’ relationships that express the program’s theory of change
      IF then
      IF then
      IF then
      IF then
      IF then
      University of Wisconsin-Extension, Program Development and Evaluation
    • Theory of change
      “A theory of change is a description of how and why a set of activities – be they part of a highly focused program or a comprehensive initiative – are expected to lead to early, intermediate, and long-term outcomes over a specified period.”
      (Anderson, 2000)
      University of Wisconsin-Extension, Program Development and Evaluation
    • Logical chain of connections showing what the program is to accomplish
      INPUTS
      OUTPUTS
      OUTCOMES
      Activities
      Participation
      Short
      Medium
      Long-term
      Program investments
      What we do
      Who we reach
      What we invest
      What results
      University of Wisconsin-Extension, Program Development and Evaluation
    • How will activities lead to desired outcomes? A series of if-then relationships
      Tutoring Program Example
      IF then
      IF then
      IF then
      IF then
      IF then
      They will move to next grade level on time
      We invest time and money
      We can provide tutoring 3 hrs/week for 1 school year to 50 children
      They will get better grades
      They will learn and improve their skills
      Students struggling academical-ly can be tutored
      University of Wisconsin-Extension, Program Development and Evaluation
    • Don’t forget the arrows
      Arrows and feedback loops show the links between inputs, outputs and outcomes
      Arrows depict the underlying causal connections
      University of Wisconsin-Extension, Program Development and Evaluation
    • Simplest form of logic model
      INPUTS
      OUTPUTS
      OUTCOMES
      University of Wisconsin-Extension, Program Development and Evaluation
    • A bit more detail
      INPUTS
      OUTPUTS
      OUTCOMES
      Activities
      Participation
      Short
      Medium
      Long-term
      Program investments
      What we invest
      What we do
      Who we reach
      What results
      SO WHAT??
      What is the VALUE?
      University of Wisconsin-Extension, Program Development and Evaluation
    • Fully detailed logic model
      University of Wisconsin-Extension, Program Development and Evaluation
    • Defining the Situation: Critical first step in logic model development
      What problematic condition exists that demands a programmatic response?
      Why does it exist?
      For whom does it exist?
      Who has a stake in the problem?
      What can be changed?
      If incorrectly understood and diagnosed, everything that flows from it will be wrong.
      Factors affecting problems: protective factors; risk factors
      Review research, evidence, knowledge-base
      Traps:
      Assuming we know cause: symptoms vs. root causes.
      Framing a problem as a need where need is actually a program or service. “Communities need leadership training” Precludes discussion of nature of the problem: what is the problem? Whose problem? Leads one to value provision of the service as the result – is the service provided or not?
      University of Wisconsin-Extension, Program Development and Evaluation
    • University of Wisconsin-Extension, Program Development and Evaluation
    • Satisfaction
      University of Wisconsin-Extension, Program Development and Evaluation
    • C H A I N OF O U T C O M E S
      University of Wisconsin-Extension, Program Development and Evaluation
    • Tend not be included in a logic model graphic:
      Situational statement
      Priorities
      List of assumptions
      List of external factors
      Evaluation methods
      University of Wisconsin-Extension, Program Development and Evaluation
    • Actions
      Changes in behaviors and practices
      Learning
      Changes in knowledge, attitudes, skills, aspirations
      Reactions
      Degree of satisfaction with program; level of interest; feelings toward activities, educational methods
      Participation
      Number and characteristics of people reached; frequency and intensity of contact
      Hierarchy of effects
      Social-economic-environmental improvements
      Source: Bennett and Rockwell, 1995, Targeting Outcomes of Programs
      University of Wisconsin-Extension, Program Development and Evaluation
    • Language: What do you mean by…
      Goal = Impact
      Impact = Long-term outcome
      Objectives (participant focused) = Outcomes
      Activities = Outputs
      Outputs may signify “tangible” accomplishments as a result of activities; products
      University of Wisconsin-Extension, Program Development and Evaluation
    • Goal – outcome definition
      Goal represents a general, big-picture statement of desired results. “We find that it is useful to think of goals as the answer to the question ‘What are issues that you would like the program to address?’ (e.g., the goal of the program is to address existing community laws and norms about ATOD use) and outcomes as the answer to: ‘What changes do you want to occur because of your program?’ (e.g., the outcome of the program will be to increase the number of community residents who believe teenaged smoking is dangerous).”
      (Western CAPT)
      University of Wisconsin-Extension, Program Development and Evaluation
    • Outputs vs.outcomes
      Example: Number of patients discharged from state mental hospital is an output. Percentage of discharged who are capable of living independently is an outcome
      Not how many worms the bird feeds its young, but how well the fledgling flies
      (United Way of America, 1999)
      University of Wisconsin-Extension, Program Development and Evaluation
    • Logic Model…
      Represents intention, is not reality
      Focuses on expected outcomes
      Challenge of causal attribution
      Many factors influence process and outcomes
      Doesn’t address: Are we doing the right thing?
      Limitations
      University of Wisconsin-Extension, Program Development and Evaluation
    • From Poister, 2003
      University of Wisconsin-Extension, Program Development and Evaluation
    • Logic model with indicators for Outputs and Outcomes
      Outputs
      Outcomes
      Farmers practice new techniques
      Farm profitability increases
      Program implemented
      Targeted farmers
      Farmers learn
      Number of workshops held
      Quality of workshops
      Number and percent reporting increased profits; amount of increase
      Number and percent of farmers attending
      Number and percent who increase knowledge
      Number and percent who practice new techniques
      University of Wisconsin-Extension, Program Development and Evaluation
    • Typical activity indicators to track
      Amount of products, services delivered
      #/type of customers/clients served
      Timeliness of service provision
      Accessibility and convenience of service
      Location; hours of operation; staff availability
      Accuracy, adequacy, relevance of assistance
      Courteousness
      Customer satisfaction
      # of clients served
      # of consultations
      # of workshops held
      # of attendees
      # of referrals
      Quality of service
      University of Wisconsin-Extension, Program Development and Evaluation
    • Methods of data collection
      SOURCES OF INFORMATION
      Existing data
      Program records, attendance logs, etc
      Pictures, charts, maps, pictorial records
      Program participants
      Others: key informants, nonparticipants, proponents, critics, staff, collaborators, funders, etc.
      DATA COLLECTION METHODS
      Survey
      Interview
      Test
      Observation
      Group techniques
      Case study
      Photography
      Document review
      Expert or peer review
      University of Wisconsin-Extension, Program Development and Evaluation
    • Engagement
      Engagement is the partnership of university knowledge and resources with those of the public and private sectors to enrich scholarship, research, and creative activity; enhance curriculum, teaching and learning; prepare educated, engaged citizens; strengthen democratic values and civic responsibility; address critical societal issues; and contribute to the public good.
    • Benchmarks of Engagement
      Evidence of Institutional Commitment to Engagement
      Evidence of Institutional Resource Commitments to Engagement
      Evidence that Students are Involved in Engagement and Outreach Activities
      Evidence that Faculty and Staff are Engaged with External Constituents
    • Benchmarks of Engagement
      Evidence that Institutions are Engaged with their Communities
      Evidence of Assessing the Impact and Outcomes of Engagement
      Evidence of Resource/Revenue Opportunities Generated through Engagement
    • Assessment of Engagement
      Faculty Evaluation
      Clear goals
    • Assessment of Engagement
      Faculty Evaluation
      Clear goals
      Adequate preparation
    • Assessment of Engagement
      Faculty Evaluation
      Clear goals
      Adequate preparation
      Appropriate methods
    • Assessment of Engagement
      Faculty Evaluation
      Clear goals
      Adequate preparation
      Appropriate methods
      Significant results
    • Assessment of Engagement
      Faculty Evaluation
      Clear goals
      Adequate preparation
      Appropriate methods
      Significant results
      Effective presentation
    • Assessment of Engagement
      Faculty Evaluation
      Clear goals
      Adequate preparation
      Appropriate methods
      Significant results
      Effective presentation
      Reflective critique
    • What is required of you?
      Background of significant scholarship
    • What is required of you?
      Background of significant scholarship
      Diagnostic skills
    • What is required of you?
      Background of significant scholarship
      Diagnostic skills
      Use or development of creative and focused methodologies
    • What is required of you?
      Background of significant scholarship
      Diagnostic skills
      Use or development of creative and focused methodologies
      Information organization and media skills
    • What is required of you?
      Background of significant scholarship
      Diagnostic skills
      Use or development of creative and focused methodologies
      Information organization and media skills, and
      Written and oral skills in interpreting as well as presenting information