Transportation and
  Measurement

Presented by:
Melissa Levy
Yellow Wood Associates

April 26, 2012
You Get What You Measure®


A values-based facilitated group process
developed by Yellow Wood Associates in St.
Albans, Vermont, that integrates systems thinking
into strategic planning, identifies key leverage
indicators and aligns actions to achieve goals.




 © 2012, Yellow Wood Associates, Inc.
Selected Applications

 National Good Food Network to develop a strategic plan to
  strengthen regional food systems
 U.S. Forest Service, Cooperative Forestry Program
 Vermont Environmental Consortium (non-profit) Board to
  develop and monitor implementation of a strategic plan
 Northwest Area Foundation to develop a framework for
  evaluation of grantee projects
 Advantage Valley Entrepreneurship Development System
  Collaborative to evaluate a three year initiative funded by the
  Kellogg Foundation
 Community Foundation of Greater South Wood County to
  engage nonprofits and funders in workforce development


 © 2012, Yellow Wood Associates, Inc.
Measurement Dilemmas


No measures at all
Misleading measures
Process measures only
Measures without context




© 2012, Yellow Wood Associates, Inc.
Strategic Planning Dilemmas


False assumptions: e.g. We all understand the
 goal to mean the same thing
Arbitrary priorities: e.g. Whatever gets the most
 dots wins
Unspoken and untested assumptions – limits our
 sense of what’s possible
Lack of engagement and participatory learning –
 planning is for managers
Seeing measurement as an afterthought rather
 than a powerful tool for formative evaluation

 © 2012, Yellow Wood Associates, Inc.
Systems Thinking


Systems thinking is a discipline for seeing
wholes. It is a framework for seeing
interrelationships rather than things, for
seeing patterns of change rather than static
“snapshots.”
            - Peter Senge



 © 2012, Yellow Wood Associates, Inc.
Why Measure?

Measurement:
 Fuels continuous learning through reflection
 Creates new information and new patterns of information
  flow
 Captures results of risk and experimentation
 Supports constructive self-organizing behavior
 Can lead to new and unprecedented conversations,
  particularly with information gatekeepers




 © 2012, Yellow Wood Associates, Inc.
EXERCISE:
Everyday Measures

List a few things that you count or measure as part
of your daily life (not connected to work).


                    Think about how you measure them
                    and what kinds of decisions or actions
                    you take based upon the
                    measurements.



 © 2012, Yellow Wood Associates, Inc.
Process Overview




 © 2012, Yellow Wood Associates, Inc.
Goal


A goal is a condition that you wish to achieve. A
goal is not an action. It is not about doing or
making; it is about being.

                    Achieving a goal requires a change in
                    the way your organization or your
                    community looks, feels, and
                    acts.



 © 2012, Yellow Wood Associates, Inc.
Goal

Goal: From Spartanburg County
Foundation




   “In the year 2030, 40% of our residents
     age 25+ have a baccalaureate degree.”


 © 2012, Yellow Wood Associates, Inc.
Indicator

An indicator is something that must be changed, or
a condition that must be achieved, in order to claim
that progress is being made toward a goal.

                    Since goals are generally quite broad,
                    there are many possible indicators that
                    could suggest progress toward the
                    goal.

                    What needs to change?
                    In what direction?

 © 2012, Yellow Wood Associates, Inc.
What Makes a Good Indicator?


If the goal is:
Everyone in our community can find
rewarding work.
• Useful indicator: The number and diversity
   of work opportunities in our community are
   increasing.
• Less useful: Number of jobs in our
   community.
 © 2012, Yellow Wood Associates, Inc.
Indicator


• More early identification of students who have the
  ability to achieve academically.
• More parent involvement K-12 (16)
• Increased high school completion (diploma/GED).
• Increased community/parent knowledge, re:
  planning for our children to go to college.
• More sensitivity on the part of educated
  professionals, re: economic disadvantage, social
  class, family background, children’s family problems.



 © 2012, Yellow Wood Associates, Inc.
Indicator


• Indicators are contributed by each
  stakeholder at the table.
• The relationship between each indicator
  and every other is decided:
    – Strong relationship
    – Weak relationship
    – No relationship


 © 2012, Yellow Wood Associates, Inc.
“In the year 2030, 40% of our residents age 25+ have a
                baccalaureate degree.”

                                               1 – More emphasis on parent
                                               involvement in elementary
10 – More sensitivity on the part of           school. (si-1, di-1, so-4, do-2)
educated professionals, re:
economic disadvantage, social
class, family background,
children’s family problems. (si-1,
di-1, so-0, do-0) KRI#2                                         2 – More early identification
                                                                of students who have the
                                                                ability to achieve
                                                                academically. (si-4, di-2, so-
                                                                0, do-5) KRI #1
9 – Increased community/parent
knowledge, re: planning for our
children to go to college. (si-3, di-3,
so-2, do-1) KRI#2                                                      3- Eliminate socio-
                                                                       economic cultural
                                                                       prejudices in school.
                                                                       (si-1, di-3, so-3, do-2)


  8 – Increased high
  school completion
  (diploma/GED). (si-4, di-                                      4- More emphasis on
  0, so-2, do-1)                                                 quality early childhood
                                                                 education for all birth –
                                                                 5years. (si-2, di-1, so-2, do-
                                                                 3)


     7 Reduction of
     crime county-wide                                        5 – More action to break
     from youth to adult.                                     gangs. (si-4, di-2, so-1, do-2)
     (si-2, di-4, so-1, do-
     1)


                          6 – More parent involvement K-12
                          (16) (si-0, di-1, so-8, do-1) KLI
  © 2012, Yellow Wood Associates, Inc.
Key Leverage Indicator




    More parent involvement K-12 (16)




 © 2012, Yellow Wood Associates, Inc.
Indicator Analysis


The result
Key leverage indicators
Key results indicators




 © 2012, Yellow Wood Associates, Inc.
Assumption


An assumption is a hypothesis about some aspect of the
way the world works that we believe to be true.
Assumptions can be difficult to recognize because they are
often deeply imbedded in the way we think about the world.

                    The tendency to treat our assumptions as
                    unquestionably true allows them to shape our
                    world and become barriers to innovation and
                    creativity.




 © 2012, Yellow Wood Associates, Inc.
Measure

A measure provides a way to
actually count or value the status of an indicator.
For example, things may be measured in terms of “number
of,” “percent of,” “quality of,” “frequency of,” or “rating of.”




 © 2012, Yellow Wood Associates, Inc.
Steps to Developing a
Measure

1. Define the terms in your indicator so that
   everyone knows what they mean.

2. Determine the baseline. This is the first
   time you measure it.

3. Determine the unit. What are you
   measuring? People, percentages, miles,
   etc.


 © 2012, Yellow Wood Associates, Inc.
Using the Results of
Measurement

Rule #1:
If you don’t know how you are going to use information,
don’t collect it. Consider using information to influence
decision-makers, educate stakeholders, improve
interventions.
Rule #2:
Collect information that will be most relevant and useful in
achieving your goal. The more closely your measures are
tied to your interventions, the more useful they will be.
Don’t try to measure too much.


 © 2012, Yellow Wood Associates, Inc.
Intervention

An intervention is something you DO in order to
achieve your goal. The intervention should be
defined broadly enough to involve people in a
variety of different tasks and provide opportunities
for participants who don’t normally work together
to do so.




 © 2012, Yellow Wood Associates, Inc.
Information and Direct Action


• Information action – Letting someone
  know your situation.

• Direct action – Fixing the problem.

• An action plan should have both.



 © 2012, Yellow Wood Associates, Inc.
Choosing Sample Measures

    1.           What has to happen for it
                 to “count” as progress
                 toward the goal?

    2.           What are the behavioral
                 changes that indicate
                 progress?

    3.           What is the baseline that
                 meet the definitions?



 © 2012, Yellow Wood Associates, Inc.
Choosing Sample Measures

 Framing Measure:

 Measure:

 Unit of Measure:

 Baseline:




 © 2012, Yellow Wood Associates, Inc.
Measuring as Action

 The act of measuring helps us learn what we need to
  know to make progress toward the goal.

 Measurement can create progress.




 © 2012, Yellow Wood Associates, Inc.
Indicators to Measures
to Actions

                            Key Leverage Indicators

                                        Definitions

                                        Measures

                                         Actions

                                    Measurement Plan
 © 2012, Yellow Wood Associates, Inc.
The Power of Measurement

 Allows us to identify and test our assumptions about the
  way the world works.

 Focuses our efforts and helps us align our resources.

 Helps us tell our stories to potential supporters.




 © 2012, Yellow Wood Associates, Inc.
Strengths of This Process


 Community priorities – bottom up planning.

 Iterative annual process.

 Using experts to inform the process

 Engaging a variety of stakeholders

 Progress toward goals

 Tradeoffs between multiple goals

 Measures inform what happens next.


 © 2012, Yellow Wood Associates, Inc.
For More Information

    Melissa Levy
    Yellow Wood Associates, Inc.
    228 N. Main Street
    St. Albans, VT 05478
    802-524-6141
    melissa@yellowwood.org
    www.yellowwood.org




 Yellow Wood Associates, Inc. is a woman-owned firm providing services in rural community economic development since 1985.
 You Get What You Measure® is a trademarked service developed by Yellow Wood. For more information, email yellowwood@yellowwood.org

Levy rpo2012

  • 1.
    Transportation and Measurement Presented by: Melissa Levy Yellow Wood Associates April 26, 2012
  • 2.
    You Get WhatYou Measure® A values-based facilitated group process developed by Yellow Wood Associates in St. Albans, Vermont, that integrates systems thinking into strategic planning, identifies key leverage indicators and aligns actions to achieve goals. © 2012, Yellow Wood Associates, Inc.
  • 3.
    Selected Applications  NationalGood Food Network to develop a strategic plan to strengthen regional food systems  U.S. Forest Service, Cooperative Forestry Program  Vermont Environmental Consortium (non-profit) Board to develop and monitor implementation of a strategic plan  Northwest Area Foundation to develop a framework for evaluation of grantee projects  Advantage Valley Entrepreneurship Development System Collaborative to evaluate a three year initiative funded by the Kellogg Foundation  Community Foundation of Greater South Wood County to engage nonprofits and funders in workforce development © 2012, Yellow Wood Associates, Inc.
  • 4.
    Measurement Dilemmas No measuresat all Misleading measures Process measures only Measures without context © 2012, Yellow Wood Associates, Inc.
  • 5.
    Strategic Planning Dilemmas Falseassumptions: e.g. We all understand the goal to mean the same thing Arbitrary priorities: e.g. Whatever gets the most dots wins Unspoken and untested assumptions – limits our sense of what’s possible Lack of engagement and participatory learning – planning is for managers Seeing measurement as an afterthought rather than a powerful tool for formative evaluation © 2012, Yellow Wood Associates, Inc.
  • 6.
    Systems Thinking Systems thinkingis a discipline for seeing wholes. It is a framework for seeing interrelationships rather than things, for seeing patterns of change rather than static “snapshots.” - Peter Senge © 2012, Yellow Wood Associates, Inc.
  • 7.
    Why Measure? Measurement:  Fuelscontinuous learning through reflection  Creates new information and new patterns of information flow  Captures results of risk and experimentation  Supports constructive self-organizing behavior  Can lead to new and unprecedented conversations, particularly with information gatekeepers © 2012, Yellow Wood Associates, Inc.
  • 8.
    EXERCISE: Everyday Measures List afew things that you count or measure as part of your daily life (not connected to work). Think about how you measure them and what kinds of decisions or actions you take based upon the measurements. © 2012, Yellow Wood Associates, Inc.
  • 9.
    Process Overview ©2012, Yellow Wood Associates, Inc.
  • 10.
    Goal A goal isa condition that you wish to achieve. A goal is not an action. It is not about doing or making; it is about being. Achieving a goal requires a change in the way your organization or your community looks, feels, and acts. © 2012, Yellow Wood Associates, Inc.
  • 11.
    Goal Goal: From SpartanburgCounty Foundation “In the year 2030, 40% of our residents age 25+ have a baccalaureate degree.” © 2012, Yellow Wood Associates, Inc.
  • 12.
    Indicator An indicator issomething that must be changed, or a condition that must be achieved, in order to claim that progress is being made toward a goal. Since goals are generally quite broad, there are many possible indicators that could suggest progress toward the goal. What needs to change? In what direction? © 2012, Yellow Wood Associates, Inc.
  • 13.
    What Makes aGood Indicator? If the goal is: Everyone in our community can find rewarding work. • Useful indicator: The number and diversity of work opportunities in our community are increasing. • Less useful: Number of jobs in our community. © 2012, Yellow Wood Associates, Inc.
  • 14.
    Indicator • More earlyidentification of students who have the ability to achieve academically. • More parent involvement K-12 (16) • Increased high school completion (diploma/GED). • Increased community/parent knowledge, re: planning for our children to go to college. • More sensitivity on the part of educated professionals, re: economic disadvantage, social class, family background, children’s family problems. © 2012, Yellow Wood Associates, Inc.
  • 15.
    Indicator • Indicators arecontributed by each stakeholder at the table. • The relationship between each indicator and every other is decided: – Strong relationship – Weak relationship – No relationship © 2012, Yellow Wood Associates, Inc.
  • 16.
    “In the year2030, 40% of our residents age 25+ have a baccalaureate degree.” 1 – More emphasis on parent involvement in elementary 10 – More sensitivity on the part of school. (si-1, di-1, so-4, do-2) educated professionals, re: economic disadvantage, social class, family background, children’s family problems. (si-1, di-1, so-0, do-0) KRI#2 2 – More early identification of students who have the ability to achieve academically. (si-4, di-2, so- 0, do-5) KRI #1 9 – Increased community/parent knowledge, re: planning for our children to go to college. (si-3, di-3, so-2, do-1) KRI#2 3- Eliminate socio- economic cultural prejudices in school. (si-1, di-3, so-3, do-2) 8 – Increased high school completion (diploma/GED). (si-4, di- 4- More emphasis on 0, so-2, do-1) quality early childhood education for all birth – 5years. (si-2, di-1, so-2, do- 3) 7 Reduction of crime county-wide 5 – More action to break from youth to adult. gangs. (si-4, di-2, so-1, do-2) (si-2, di-4, so-1, do- 1) 6 – More parent involvement K-12 (16) (si-0, di-1, so-8, do-1) KLI © 2012, Yellow Wood Associates, Inc.
  • 17.
    Key Leverage Indicator More parent involvement K-12 (16) © 2012, Yellow Wood Associates, Inc.
  • 18.
    Indicator Analysis The result Keyleverage indicators Key results indicators © 2012, Yellow Wood Associates, Inc.
  • 19.
    Assumption An assumption isa hypothesis about some aspect of the way the world works that we believe to be true. Assumptions can be difficult to recognize because they are often deeply imbedded in the way we think about the world. The tendency to treat our assumptions as unquestionably true allows them to shape our world and become barriers to innovation and creativity. © 2012, Yellow Wood Associates, Inc.
  • 20.
    Measure A measure providesa way to actually count or value the status of an indicator. For example, things may be measured in terms of “number of,” “percent of,” “quality of,” “frequency of,” or “rating of.” © 2012, Yellow Wood Associates, Inc.
  • 21.
    Steps to Developinga Measure 1. Define the terms in your indicator so that everyone knows what they mean. 2. Determine the baseline. This is the first time you measure it. 3. Determine the unit. What are you measuring? People, percentages, miles, etc. © 2012, Yellow Wood Associates, Inc.
  • 22.
    Using the Resultsof Measurement Rule #1: If you don’t know how you are going to use information, don’t collect it. Consider using information to influence decision-makers, educate stakeholders, improve interventions. Rule #2: Collect information that will be most relevant and useful in achieving your goal. The more closely your measures are tied to your interventions, the more useful they will be. Don’t try to measure too much. © 2012, Yellow Wood Associates, Inc.
  • 23.
    Intervention An intervention issomething you DO in order to achieve your goal. The intervention should be defined broadly enough to involve people in a variety of different tasks and provide opportunities for participants who don’t normally work together to do so. © 2012, Yellow Wood Associates, Inc.
  • 24.
    Information and DirectAction • Information action – Letting someone know your situation. • Direct action – Fixing the problem. • An action plan should have both. © 2012, Yellow Wood Associates, Inc.
  • 25.
    Choosing Sample Measures 1. What has to happen for it to “count” as progress toward the goal? 2. What are the behavioral changes that indicate progress? 3. What is the baseline that meet the definitions? © 2012, Yellow Wood Associates, Inc.
  • 26.
    Choosing Sample Measures Framing Measure:  Measure:  Unit of Measure:  Baseline: © 2012, Yellow Wood Associates, Inc.
  • 27.
    Measuring as Action The act of measuring helps us learn what we need to know to make progress toward the goal.  Measurement can create progress. © 2012, Yellow Wood Associates, Inc.
  • 28.
    Indicators to Measures toActions Key Leverage Indicators Definitions Measures Actions Measurement Plan © 2012, Yellow Wood Associates, Inc.
  • 29.
    The Power ofMeasurement  Allows us to identify and test our assumptions about the way the world works.  Focuses our efforts and helps us align our resources.  Helps us tell our stories to potential supporters. © 2012, Yellow Wood Associates, Inc.
  • 30.
    Strengths of ThisProcess  Community priorities – bottom up planning.  Iterative annual process.  Using experts to inform the process  Engaging a variety of stakeholders  Progress toward goals  Tradeoffs between multiple goals  Measures inform what happens next. © 2012, Yellow Wood Associates, Inc.
  • 31.
    For More Information Melissa Levy Yellow Wood Associates, Inc. 228 N. Main Street St. Albans, VT 05478 802-524-6141 melissa@yellowwood.org www.yellowwood.org Yellow Wood Associates, Inc. is a woman-owned firm providing services in rural community economic development since 1985. You Get What You Measure® is a trademarked service developed by Yellow Wood. For more information, email yellowwood@yellowwood.org