SlideShare a Scribd company logo
Challenges and
Successes
Associated with
Introducing
Systems Concepts
to an Existing
Evaluation
Ginger Fitzhugh
American Evaluation Association
October 26, 2012
WaterBotics scale-up evaluation
                Stevens
               Institute
                of Tech
WaterBotics scale-up evaluation
                     Stevens
                    Institute
       League for    of Tech    NGCP
       Innovation
WaterBotics scale-up evaluation
                         Stevens
                        Institute
          League for     of Tech    NGCP
          Innovation




         Formal
         Hub site




     Formal Educators


        Students
WaterBotics scale-up evaluation
                         Stevens
                        Institute
          League for     of Tech          NGCP
          Innovation




         Formal                         Informal
         Hub site                       Hub site




     Formal Educators               Informal Educators


        Students                       Students
WaterBotics scale-up evaluation
                     Stevens
                    Institute
       League for    of Tech     NGCP
       Innovation




       Formal                   Informal
       Hub site                 Hub site
A system is an
interconnected
set
of elements
that is
coherently
organized in a
way that
achieves
something.
From D. Meadows (2008).
Thinking in Systems. White
River Junction, VT: Chelsea
Gun Publishing.
Q: How can I add systems?




Limited Time   Limited Scope   Limited Budget
A: Focus on three concepts
1. Boundaries
2. Relationships
3. Perspectives
Concept 1: Boundaries
Demarcations that create a region or entity1




1Courtesy   of Beverly Parsons, InSites
Concept 2: Relationships
Connections/exchanges among bounded system
parts1




1Courtesy   of Beverly Parsons, InSites
Concept 3:
Perspectives
Mental models,
world views,
purposes1




1Courtesy   of Beverly Parsons,
InSites
Questions That Matter




                        Parsons, B. and
                        Jessup, P. (2009).
                        “Questions that
                        Matter: A Tool for
                        Working in
                        Complex
                        Situations”. Ft.
                        Collins, CO:
                        InSites. Available
                        at insites.org.
Boundary Questions
General Systems            Specific Interview Questions
Questions
How stable/permeable       How do you see informal settings being
are the boundaries?        similar to or different than formal settings?
What shifts are needed in What skills and abilities have the hub sites
resources to reach the    lacked or struggled with in implementing
desired goal?             WaterBotics? How has the leadership team
                          addressed these issues?
How are the boundaries     What external factors have significantly
changing between the       affected the project this past year? How did
intervention and its       they affect the project?
context?
Relationship Questions
General Systems            Specific Interview Questions
Questions
What are the              In what ways have you involved the hub
relationships/connections sites and partners in planning the work as
among the stakeholders? the project was implemented?
What feedback loops are    How often did you communicate with hub
built into the project?    sites leaders during this past year? Who
                           initiated the communication? What kinds of
                           things did you discuss?
How might changes in       What new relationships/connections are
relationships affect the   needed to make this project even more
short- and long-term       successful in meeting its goals?
outcomes or general
direction of the work?
Perspective Questions
General Systems            Specific Interview Questions
Questions
What are stakeholders’     We’re interested in what your vision of the
perspectives re: project   WaterBotics project is now that you have
goals? How are they        begun implementation. Imagine that it is
similar or different?      four years from now and WaterBotics has
                           been very successful. Could you describe
                           what that looks like?
Are stakeholders missing   Do you seek input from hub site leaders
or avoiding an important   and/or educators? If so, how do you do
perspective?               that?
Where is energy being      What has been most exciting about the
created? Where is it       work thus far? At this point in the project,
stagnating?                what has been most frustrating or
                           challenging?
System Dynamics
        Low
                                               More open


                                                           More diverse
   Agreement




                                                                      More differences
                      Closed


                               Less diverse



                                              Few differences
        High




               High                   Certainty                           Low
Mapping Project Subsystems
        Low




                                           Informal educators &
   Agreement



                                           organizations




                      Teachers & schools
        High




               High                 Certainty                     Low
Learning
Lessons
Set up doable
boundaries around
evaluation

Develop relationships
with clients around
systems thinking

Look at existing
evaluation with fresh
perspective
Visit www.inSites.org to learn more about ECLIPS

                 Ginger Fitzhugh
             gfitzhugh@eraeval.org
                www.eraeval.org

More Related Content

Similar to Challenges and Successes Associated with Introducing Systems Concepts to an Existing Evaluation

Setting Up for Collaboration: Top Four Things to Keep in Mind
Setting Up for Collaboration: Top Four Things to Keep in MindSetting Up for Collaboration: Top Four Things to Keep in Mind
Setting Up for Collaboration: Top Four Things to Keep in Mind
Christopher Wilson
 
The ROI of Your Digital Strategy
The ROI of Your Digital StrategyThe ROI of Your Digital Strategy
The ROI of Your Digital StrategyBeth Kanter
 
Making Networks Work
Making Networks WorkMaking Networks Work
Making Networks Work
Robert Jake Jacobs
 
RACE in Instructional Technology
RACE in Instructional TechnologyRACE in Instructional Technology
RACE in Instructional TechnologyDionne Curbeam
 
User Experience as an Organizational Development Tool
User Experience as an Organizational Development ToolUser Experience as an Organizational Development Tool
User Experience as an Organizational Development Tool
Donovan Chandler
 
COF Presentation: Leading the Networked Foundation
COF Presentation:  Leading the Networked FoundationCOF Presentation:  Leading the Networked Foundation
COF Presentation: Leading the Networked FoundationBeth Kanter
 
Innovation and Collaboration Pulse Surveys - Dave Litwiller - April 19 2020
Innovation and Collaboration Pulse Surveys - Dave Litwiller - April 19 2020Innovation and Collaboration Pulse Surveys - Dave Litwiller - April 19 2020
Innovation and Collaboration Pulse Surveys - Dave Litwiller - April 19 2020
Dave Litwiller
 
Successfully transitioning your organization to a network mindset complete (1)
Successfully transitioning your organization to a network mindset complete (1)Successfully transitioning your organization to a network mindset complete (1)
Successfully transitioning your organization to a network mindset complete (1)
Leadership Learning Community
 
Final final collaborating on outcomes 30 5 12
Final final collaborating on outcomes 30 5 12Final final collaborating on outcomes 30 5 12
Final final collaborating on outcomes 30 5 12Kerry McCarroll
 
KM Chicago: Organisational Network Analysis
KM Chicago: Organisational Network AnalysisKM Chicago: Organisational Network Analysis
KM Chicago: Organisational Network Analysis
KM Chicago
 
VolunteerMatch: Measuring the Networked Nonprofit
VolunteerMatch:  Measuring the Networked NonprofitVolunteerMatch:  Measuring the Networked Nonprofit
VolunteerMatch: Measuring the Networked NonprofitBeth Kanter
 
Uconn Coiro Assessment 2008
Uconn Coiro Assessment 2008Uconn Coiro Assessment 2008
Uconn Coiro Assessment 2008
Julie Coiro
 
Allstate Foundation
Allstate FoundationAllstate Foundation
Allstate FoundationBeth Kanter
 
How to Ask for Technical Help? Evidence-based Guidelines for Writing Question...
How to Ask for Technical Help? Evidence-based Guidelines for Writing Question...How to Ask for Technical Help? Evidence-based Guidelines for Writing Question...
How to Ask for Technical Help? Evidence-based Guidelines for Writing Question...
Fabio Calefato
 
Skills & ideas for #ProblemGamblingKTE
Skills & ideas for #ProblemGamblingKTE Skills & ideas for #ProblemGamblingKTE
Skills & ideas for #ProblemGamblingKTE
Anne Bergen
 
SNCR Webinar - Jan 4, 2013
SNCR Webinar - Jan 4, 2013SNCR Webinar - Jan 4, 2013
SNCR Webinar - Jan 4, 2013Beth Kanter
 
Cultivating Collaboration: lessons from cohousing studios
Cultivating Collaboration: lessons from cohousing studiosCultivating Collaboration: lessons from cohousing studios
Cultivating Collaboration: lessons from cohousing studios
Nancy Cheng
 
Boston Book Tour: Measuring the Networked Nonprofit
Boston Book Tour: Measuring the Networked NonprofitBoston Book Tour: Measuring the Networked Nonprofit
Boston Book Tour: Measuring the Networked Nonprofit
Beth Kanter
 

Similar to Challenges and Successes Associated with Introducing Systems Concepts to an Existing Evaluation (20)

Setting Up for Collaboration: Top Four Things to Keep in Mind
Setting Up for Collaboration: Top Four Things to Keep in MindSetting Up for Collaboration: Top Four Things to Keep in Mind
Setting Up for Collaboration: Top Four Things to Keep in Mind
 
The ROI of Your Digital Strategy
The ROI of Your Digital StrategyThe ROI of Your Digital Strategy
The ROI of Your Digital Strategy
 
Making Networks Work
Making Networks WorkMaking Networks Work
Making Networks Work
 
Learning session 1 2 developing powerful partnerships
Learning session 1 2 developing powerful partnershipsLearning session 1 2 developing powerful partnerships
Learning session 1 2 developing powerful partnerships
 
RACE in Instructional Technology
RACE in Instructional TechnologyRACE in Instructional Technology
RACE in Instructional Technology
 
User Experience as an Organizational Development Tool
User Experience as an Organizational Development ToolUser Experience as an Organizational Development Tool
User Experience as an Organizational Development Tool
 
COF Presentation: Leading the Networked Foundation
COF Presentation:  Leading the Networked FoundationCOF Presentation:  Leading the Networked Foundation
COF Presentation: Leading the Networked Foundation
 
Innovation and Collaboration Pulse Surveys - Dave Litwiller - April 19 2020
Innovation and Collaboration Pulse Surveys - Dave Litwiller - April 19 2020Innovation and Collaboration Pulse Surveys - Dave Litwiller - April 19 2020
Innovation and Collaboration Pulse Surveys - Dave Litwiller - April 19 2020
 
Successfully transitioning your organization to a network mindset complete (1)
Successfully transitioning your organization to a network mindset complete (1)Successfully transitioning your organization to a network mindset complete (1)
Successfully transitioning your organization to a network mindset complete (1)
 
Final final collaborating on outcomes 30 5 12
Final final collaborating on outcomes 30 5 12Final final collaborating on outcomes 30 5 12
Final final collaborating on outcomes 30 5 12
 
KM Chicago: Organisational Network Analysis
KM Chicago: Organisational Network AnalysisKM Chicago: Organisational Network Analysis
KM Chicago: Organisational Network Analysis
 
VolunteerMatch: Measuring the Networked Nonprofit
VolunteerMatch:  Measuring the Networked NonprofitVolunteerMatch:  Measuring the Networked Nonprofit
VolunteerMatch: Measuring the Networked Nonprofit
 
Uconn Coiro Assessment 2008
Uconn Coiro Assessment 2008Uconn Coiro Assessment 2008
Uconn Coiro Assessment 2008
 
Allstate Foundation
Allstate FoundationAllstate Foundation
Allstate Foundation
 
Stanford
StanfordStanford
Stanford
 
How to Ask for Technical Help? Evidence-based Guidelines for Writing Question...
How to Ask for Technical Help? Evidence-based Guidelines for Writing Question...How to Ask for Technical Help? Evidence-based Guidelines for Writing Question...
How to Ask for Technical Help? Evidence-based Guidelines for Writing Question...
 
Skills & ideas for #ProblemGamblingKTE
Skills & ideas for #ProblemGamblingKTE Skills & ideas for #ProblemGamblingKTE
Skills & ideas for #ProblemGamblingKTE
 
SNCR Webinar - Jan 4, 2013
SNCR Webinar - Jan 4, 2013SNCR Webinar - Jan 4, 2013
SNCR Webinar - Jan 4, 2013
 
Cultivating Collaboration: lessons from cohousing studios
Cultivating Collaboration: lessons from cohousing studiosCultivating Collaboration: lessons from cohousing studios
Cultivating Collaboration: lessons from cohousing studios
 
Boston Book Tour: Measuring the Networked Nonprofit
Boston Book Tour: Measuring the Networked NonprofitBoston Book Tour: Measuring the Networked Nonprofit
Boston Book Tour: Measuring the Networked Nonprofit
 

Recently uploaded

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
siemaillard
 
Unit 2- Research Aptitude (UGC NET Paper I).pdf
Unit 2- Research Aptitude (UGC NET Paper I).pdfUnit 2- Research Aptitude (UGC NET Paper I).pdf
Unit 2- Research Aptitude (UGC NET Paper I).pdf
Thiyagu K
 
The Roman Empire A Historical Colossus.pdf
The Roman Empire A Historical Colossus.pdfThe Roman Empire A Historical Colossus.pdf
The Roman Empire A Historical Colossus.pdf
kaushalkr1407
 
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
Nguyen Thanh Tu Collection
 
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
MysoreMuleSoftMeetup
 
The Accursed House by Émile Gaboriau.pptx
The Accursed House by Émile Gaboriau.pptxThe Accursed House by Émile Gaboriau.pptx
The Accursed House by Émile Gaboriau.pptx
DhatriParmar
 
Operation Blue Star - Saka Neela Tara
Operation Blue Star   -  Saka Neela TaraOperation Blue Star   -  Saka Neela Tara
Operation Blue Star - Saka Neela Tara
Balvir Singh
 
Digital Tools and AI for Teaching Learning and Research
Digital Tools and AI for Teaching Learning and ResearchDigital Tools and AI for Teaching Learning and Research
Digital Tools and AI for Teaching Learning and Research
Vikramjit Singh
 
Introduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp NetworkIntroduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp Network
TechSoup
 
CACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdfCACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdf
camakaiclarkmusic
 
How to Make a Field invisible in Odoo 17
How to Make a Field invisible in Odoo 17How to Make a Field invisible in Odoo 17
How to Make a Field invisible in Odoo 17
Celine George
 
Thesis Statement for students diagnonsed withADHD.ppt
Thesis Statement for students diagnonsed withADHD.pptThesis Statement for students diagnonsed withADHD.ppt
Thesis Statement for students diagnonsed withADHD.ppt
EverAndrsGuerraGuerr
 
The approach at University of Liverpool.pptx
The approach at University of Liverpool.pptxThe approach at University of Liverpool.pptx
The approach at University of Liverpool.pptx
Jisc
 
TESDA TM1 REVIEWER FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
TESDA TM1 REVIEWER  FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...TESDA TM1 REVIEWER  FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
TESDA TM1 REVIEWER FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
EugeneSaldivar
 
A Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in EducationA Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in Education
Peter Windle
 
Synthetic Fiber Construction in lab .pptx
Synthetic Fiber Construction in lab .pptxSynthetic Fiber Construction in lab .pptx
Synthetic Fiber Construction in lab .pptx
Pavel ( NSTU)
 
Lapbook sobre os Regimes Totalitários.pdf
Lapbook sobre os Regimes Totalitários.pdfLapbook sobre os Regimes Totalitários.pdf
Lapbook sobre os Regimes Totalitários.pdf
Jean Carlos Nunes Paixão
 
2024.06.01 Introducing a competency framework for languag learning materials ...
2024.06.01 Introducing a competency framework for languag learning materials ...2024.06.01 Introducing a competency framework for languag learning materials ...
2024.06.01 Introducing a competency framework for languag learning materials ...
Sandy Millin
 
Guidance_and_Counselling.pdf B.Ed. 4th Semester
Guidance_and_Counselling.pdf B.Ed. 4th SemesterGuidance_and_Counselling.pdf B.Ed. 4th Semester
Guidance_and_Counselling.pdf B.Ed. 4th Semester
Atul Kumar Singh
 
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXXPhrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
MIRIAMSALINAS13
 

Recently uploaded (20)

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
 
Unit 2- Research Aptitude (UGC NET Paper I).pdf
Unit 2- Research Aptitude (UGC NET Paper I).pdfUnit 2- Research Aptitude (UGC NET Paper I).pdf
Unit 2- Research Aptitude (UGC NET Paper I).pdf
 
The Roman Empire A Historical Colossus.pdf
The Roman Empire A Historical Colossus.pdfThe Roman Empire A Historical Colossus.pdf
The Roman Empire A Historical Colossus.pdf
 
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
BÀI TẬP BỔ TRỢ TIẾNG ANH GLOBAL SUCCESS LỚP 3 - CẢ NĂM (CÓ FILE NGHE VÀ ĐÁP Á...
 
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
Mule 4.6 & Java 17 Upgrade | MuleSoft Mysore Meetup #46
 
The Accursed House by Émile Gaboriau.pptx
The Accursed House by Émile Gaboriau.pptxThe Accursed House by Émile Gaboriau.pptx
The Accursed House by Émile Gaboriau.pptx
 
Operation Blue Star - Saka Neela Tara
Operation Blue Star   -  Saka Neela TaraOperation Blue Star   -  Saka Neela Tara
Operation Blue Star - Saka Neela Tara
 
Digital Tools and AI for Teaching Learning and Research
Digital Tools and AI for Teaching Learning and ResearchDigital Tools and AI for Teaching Learning and Research
Digital Tools and AI for Teaching Learning and Research
 
Introduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp NetworkIntroduction to AI for Nonprofits with Tapp Network
Introduction to AI for Nonprofits with Tapp Network
 
CACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdfCACJapan - GROUP Presentation 1- Wk 4.pdf
CACJapan - GROUP Presentation 1- Wk 4.pdf
 
How to Make a Field invisible in Odoo 17
How to Make a Field invisible in Odoo 17How to Make a Field invisible in Odoo 17
How to Make a Field invisible in Odoo 17
 
Thesis Statement for students diagnonsed withADHD.ppt
Thesis Statement for students diagnonsed withADHD.pptThesis Statement for students diagnonsed withADHD.ppt
Thesis Statement for students diagnonsed withADHD.ppt
 
The approach at University of Liverpool.pptx
The approach at University of Liverpool.pptxThe approach at University of Liverpool.pptx
The approach at University of Liverpool.pptx
 
TESDA TM1 REVIEWER FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
TESDA TM1 REVIEWER  FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...TESDA TM1 REVIEWER  FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
TESDA TM1 REVIEWER FOR NATIONAL ASSESSMENT WRITTEN AND ORAL QUESTIONS WITH A...
 
A Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in EducationA Strategic Approach: GenAI in Education
A Strategic Approach: GenAI in Education
 
Synthetic Fiber Construction in lab .pptx
Synthetic Fiber Construction in lab .pptxSynthetic Fiber Construction in lab .pptx
Synthetic Fiber Construction in lab .pptx
 
Lapbook sobre os Regimes Totalitários.pdf
Lapbook sobre os Regimes Totalitários.pdfLapbook sobre os Regimes Totalitários.pdf
Lapbook sobre os Regimes Totalitários.pdf
 
2024.06.01 Introducing a competency framework for languag learning materials ...
2024.06.01 Introducing a competency framework for languag learning materials ...2024.06.01 Introducing a competency framework for languag learning materials ...
2024.06.01 Introducing a competency framework for languag learning materials ...
 
Guidance_and_Counselling.pdf B.Ed. 4th Semester
Guidance_and_Counselling.pdf B.Ed. 4th SemesterGuidance_and_Counselling.pdf B.Ed. 4th Semester
Guidance_and_Counselling.pdf B.Ed. 4th Semester
 
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXXPhrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
Phrasal Verbs.XXXXXXXXXXXXXXXXXXXXXXXXXX
 

Challenges and Successes Associated with Introducing Systems Concepts to an Existing Evaluation

  • 1. Challenges and Successes Associated with Introducing Systems Concepts to an Existing Evaluation Ginger Fitzhugh American Evaluation Association October 26, 2012
  • 2. WaterBotics scale-up evaluation Stevens Institute of Tech
  • 3. WaterBotics scale-up evaluation Stevens Institute League for of Tech NGCP Innovation
  • 4. WaterBotics scale-up evaluation Stevens Institute League for of Tech NGCP Innovation Formal Hub site Formal Educators Students
  • 5. WaterBotics scale-up evaluation Stevens Institute League for of Tech NGCP Innovation Formal Informal Hub site Hub site Formal Educators Informal Educators Students Students
  • 6. WaterBotics scale-up evaluation Stevens Institute League for of Tech NGCP Innovation Formal Informal Hub site Hub site
  • 7. A system is an interconnected set of elements that is coherently organized in a way that achieves something. From D. Meadows (2008). Thinking in Systems. White River Junction, VT: Chelsea Gun Publishing.
  • 8. Q: How can I add systems? Limited Time Limited Scope Limited Budget
  • 9. A: Focus on three concepts 1. Boundaries 2. Relationships 3. Perspectives
  • 10. Concept 1: Boundaries Demarcations that create a region or entity1 1Courtesy of Beverly Parsons, InSites
  • 11. Concept 2: Relationships Connections/exchanges among bounded system parts1 1Courtesy of Beverly Parsons, InSites
  • 12. Concept 3: Perspectives Mental models, world views, purposes1 1Courtesy of Beverly Parsons, InSites
  • 13. Questions That Matter Parsons, B. and Jessup, P. (2009). “Questions that Matter: A Tool for Working in Complex Situations”. Ft. Collins, CO: InSites. Available at insites.org.
  • 14. Boundary Questions General Systems Specific Interview Questions Questions How stable/permeable How do you see informal settings being are the boundaries? similar to or different than formal settings? What shifts are needed in What skills and abilities have the hub sites resources to reach the lacked or struggled with in implementing desired goal? WaterBotics? How has the leadership team addressed these issues? How are the boundaries What external factors have significantly changing between the affected the project this past year? How did intervention and its they affect the project? context?
  • 15. Relationship Questions General Systems Specific Interview Questions Questions What are the In what ways have you involved the hub relationships/connections sites and partners in planning the work as among the stakeholders? the project was implemented? What feedback loops are How often did you communicate with hub built into the project? sites leaders during this past year? Who initiated the communication? What kinds of things did you discuss? How might changes in What new relationships/connections are relationships affect the needed to make this project even more short- and long-term successful in meeting its goals? outcomes or general direction of the work?
  • 16. Perspective Questions General Systems Specific Interview Questions Questions What are stakeholders’ We’re interested in what your vision of the perspectives re: project WaterBotics project is now that you have goals? How are they begun implementation. Imagine that it is similar or different? four years from now and WaterBotics has been very successful. Could you describe what that looks like? Are stakeholders missing Do you seek input from hub site leaders or avoiding an important and/or educators? If so, how do you do perspective? that? Where is energy being What has been most exciting about the created? Where is it work thus far? At this point in the project, stagnating? what has been most frustrating or challenging?
  • 17. System Dynamics Low More open More diverse Agreement More differences Closed Less diverse Few differences High High Certainty Low
  • 18. Mapping Project Subsystems Low Informal educators & Agreement organizations Teachers & schools High High Certainty Low
  • 19. Learning Lessons Set up doable boundaries around evaluation Develop relationships with clients around systems thinking Look at existing evaluation with fresh perspective
  • 20. Visit www.inSites.org to learn more about ECLIPS Ginger Fitzhugh gfitzhugh@eraeval.org www.eraeval.org

Editor's Notes

  1. The second concept is relationships. Relationships are about interactions within different parts of the system and across different systems. We typically think about relationships as being between people or organizations, but relationships can also be between cause and effect. Relationship questions evaluators might want to ask include:What are the connections between different parts of the system? Are the parts and people of the system connected that need to be? What feedback loops are built into the system?Social network analysis is a tool for analyzing these relationships and how they change over time.
  2. This slide shows some examples of interview questions we used related to boundaries. You may recall the earlier slide showing the project is implemented in formal and informal settings. We wanted to know what the similarities and differences were between these two contexts.We also wanted to know the degree to which resources such as professional development or other kinds of support were being provided, and whether they were crossing the boundaries as needed.Finally, it was important to try to understand what factors outside the project system were affecting it.
  3. We asked a lot of questions to try to understand the different perspectives of the leadership team members, hub site leaders, and educators, and perspectives of those from formal and informal spheres. We asked the interviewees to describe their vision for the project and analyzed the results to see where there were similarities and differences.We asked members of the leadership team if they asked for input from other stakeholders in the system.Finally, we asked a few questions aimed at understanding where there was energy in the project and where it was lacking.
  4. You can map the concepts of boundaries, relationships and perspectives onto this diagram, which shows the relationship between the certainty or predictability of a system and the level of agreement within it. Systems with closed boundaries, less diverse relationships, and few differences in perspective also have a high degree of certainty and agreement, and are therefore would be placed in the lower left area of the graph. Systems with open or very permeable boundaries, diverse relationships, and many different perspectives have a low degree of certainty or agreement, and therefore would be placed in the upper right area of the graph. It’s important to note that placement on the graph does not constitute judgment about the quality of the system.