Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
MFLN Evaluation and
Reporting
Developmental Evaluation
“DE supports innovation development to
guide adaptation to emergent and dynamic
realities in comp...
Core DE Question
What is getting developed and what are
the implications of what gets
developed?
Complexity theory of chan...
Key Characteristics of DE
Focus on development
Focus on development
Development is when people are
changing what they are doing, and the very
nature of the standards are...
Complex and dynamic
environment
Multiple leadership
nodes
Multiple paths
forward
Rapid and continual
feedback
Ask evaluative questions
Apply evaluative logic
Gather and report evaluative data
Evaluation ...
Developmental Evaluator
Works collaboratively with innovators
to conceptualize, design, and test new
approaches
in a long-...
Evaluation for MFLN
• Ongoing
development
• Innovation in
learning
– Internal
practice
– External
product
• Reflection and...
DE for MFLN
MFLN: Focus on development
Programming has been established…
– Identify guiding principles that inform ongoing
development...
Cooperative agreement
environment
+
eXtension Initiative
Cooperative Agreement
“An opportunity to provide responsive and
timely educational programming through
being actively nimb...
Constant evaluative
feedback and discussion
Monthly reports (internal,
DoD)
Quarterly reports (NIFA)
Annual reports (inter...
Internal evaluator
Member of leadership team
Elucidates innovation and adaptation
processes
Tracks implications and result...
DE for MFLN: What It Looks
Like
Continuous
adaptation
DE for MFLN: What It Looks
Like
Continuous
adaptationSocial, political, economic, technological, and
demographic patterns
DE for MFLN: What It Looks
Like
Continuous
adaptationSocial, political, economic, technological, and
demographic patterns
...
DE for MFLN: What It Looks
Like
Continuous
adaptationSocial, political, economic, technological, and
demographic patterns
...
DE for MFLN: What It Looks
Like
Continuous
adaptationSocial, political, economic, technological, and
demographic patterns
...
DE for MFLN: What It Looks
Like
Continuous
adaptationSocial, political, economic, technological, and
demographic patterns
...
DE for MFLN: What It Looks
Like
(not this. . . .)
DE for MFLN: What It Looks
Like
(but this. .
Inspired by Jeff Conklin,
cognexus.org
Time
Complex development situations are ones in
which this…
25
Michael Quinn Patton...
Michael Quinn Patton
AEA eStudy webinar 2014
26
And this…
Turns out to be this…
Time
27
Michael Quinn Patton
AEA eStudy webinar 2014
Michael Quinn Patton
AEA eStudy webinar 2014
28
…looks like this
DE for MFLN: What It Looks
Like
Inductive
DE for MFLN: What It Looks
Like
Inductive
Chaotic and messy
DE for MFLN: What It Looks
Like
Inductive
Chaotic and messy
Embrace forks in the road
DE for MFLN: What It Looks
Like
Inductive
Chaotic and messy
Embrace forks in the road
Developmental moments
Track what’s g...
DE for MFLN: What It Looks
Like
Everyone is an
evaluator
DE Successes for MFLN
DE Successes for MFLN
Very responsive to our partners’
programmatic requests and innovations
DE Successes for MFLN
Very responsive to our partners’
programmatic requests and innovations
You. Here. Now.
DE Successes for MFLN
Very responsive to our partners’
programmatic requests and innovations
You. Here. Now.
Programming
DE Successes for MFLN
Very responsive to our partners’
programmatic requests and innovations
You. Here. Now.
Programming
“...
DE Challenges for MFLN
Innovation is challenging, slow, and can be
contentious
DE Challenges for MFLN
Innovation is challenging, slow, and can be
contentious
Moving from program improvement mode to
red...
DE Challenges for MFLN
Innovation is challenging, slow, and can be
contentious
Moving from program improvement mode to
red...
DE Challenges for MFLN
Innovation is challenging, slow, and can be
contentious
Moving from program improvement mode to
red...
DE Challenges for MFLN
Innovation is challenging, slow, and can be
contentious
Moving from program improvement mode to
red...
DE Challenges for MFLN
Innovation is challenging, slow, and can be
contentious
Moving from program improvement mode to
red...
DE Challenges for MFLN
Innovation is challenging, slow, and can be
contentious
Moving from program improvement mode to
red...
DE Features for MFLN: Nodal
Evaluations
Webinars
Social media
Focus groups
Deliverables/plans of work
• Why as much as wha...
DE Features for MFLN:
Reporting
From Concentration Areas
• PIs
– Monthly via Google forms: narrative updates,
pubs, presen...
DE Features for MFLN:
Reporting
To CAs:
• Monthly social media from Sprout Social
• Quarterly webinar data reports
Interna...
DE Features for MFLN: Reflective
Discussion
CAs:
• Monthly social media specialists meetings
– NetLit, processes, evaluati...
DE Features for MFLN: Learning
and Working Paradigms
Leadership
Reflection
Transparency
Collaboration
Action
Innovation
MFLN Evaluation and Reporting Launch
MFLN Evaluation and Reporting Launch
Upcoming SlideShare
Loading in …5
×

MFLN Evaluation and Reporting Launch

317 views

Published on

Evaluation and reporting slides for the 2015 MFLN Launch Meeting, Alexandria, Va.

Published in: Education
  • Be the first to comment

  • Be the first to like this

MFLN Evaluation and Reporting Launch

  1. 1. MFLN Evaluation and Reporting
  2. 2. Developmental Evaluation “DE supports innovation development to guide adaptation to emergent and dynamic realities in complex environments.” — MQP Goal: support project, program, and organizational development with timely feedback
  3. 3. Core DE Question What is getting developed and what are the implications of what gets developed? Complexity theory of change: Bring people together who are knowledgeable and committed and they will self-organize, take action, and work together to create movement, innovation, and change.
  4. 4. Key Characteristics of DE
  5. 5. Focus on development
  6. 6. Focus on development Development is when people are changing what they are doing, and the very nature of the standards are also changing. Standards are adapting to changing conditions. NOT the same as continuous improvement (formative eval or accountability) – Quality improvement is helping programs meet standards that have been set.
  7. 7. Complex and dynamic environment Multiple leadership nodes Multiple paths forward
  8. 8. Rapid and continual feedback Ask evaluative questions Apply evaluative logic Gather and report evaluative data Evaluation becomes part of
  9. 9. Developmental Evaluator Works collaboratively with innovators to conceptualize, design, and test new approaches in a long-term, ongoing process of adaptation, intentional change, and development.
  10. 10. Evaluation for MFLN • Ongoing development • Innovation in learning – Internal practice – External product • Reflection and action
  11. 11. DE for MFLN
  12. 12. MFLN: Focus on development Programming has been established… – Identify guiding principles that inform ongoing development – Establishes track record, reliability, and reputation ... BUT is not necessarily fixed – Innovate, innovate, innovate – Internal adaptability to meet changing external standards and conditions
  13. 13. Cooperative agreement environment + eXtension Initiative
  14. 14. Cooperative Agreement “An opportunity to provide responsive and timely educational programming through being actively nimble, flexible, innovative, and creative in true partnership with our funders as they identify organizational priorities and ask us to engage in the construction of appropriate and necessary deliverables that meet the on- time needs of the target audience. The MFLN and funding partners are seen as true and honest equals in expertise, providing valued and accepted feedback bi- directionally. This often results in
  15. 15. Constant evaluative feedback and discussion Monthly reports (internal, DoD) Quarterly reports (NIFA) Annual reports (internal, DoD, NIFA) Webinar evaluation reports (CAs) Quarterly webinar reports (internal, DoD) Weekly leadership team meetings Instant messaging Hangouts E-mail Google docs Twitter Phone
  16. 16. Internal evaluator Member of leadership team Elucidates innovation and adaptation processes Tracks implications and results Facilitates data-based decision-making
  17. 17. DE for MFLN: What It Looks Like Continuous adaptation
  18. 18. DE for MFLN: What It Looks Like Continuous adaptationSocial, political, economic, technological, and demographic patterns
  19. 19. DE for MFLN: What It Looks Like Continuous adaptationSocial, political, economic, technological, and demographic patterns Emergent developments/needs of our target audiences
  20. 20. DE for MFLN: What It Looks Like Continuous adaptationSocial, political, economic, technological, and demographic patterns Emergent developments/needs of our target audiences Cooperative agreement environment
  21. 21. DE for MFLN: What It Looks Like Continuous adaptationSocial, political, economic, technological, and demographic patterns Emergent developments/needs of our target audiences Cooperative agreement environment eXtension
  22. 22. DE for MFLN: What It Looks Like Continuous adaptationSocial, political, economic, technological, and demographic patterns Emergent developments/needs of our target audiences Cooperative agreement environment eXtension DoD
  23. 23. DE for MFLN: What It Looks Like (not this. . . .)
  24. 24. DE for MFLN: What It Looks Like (but this. .
  25. 25. Inspired by Jeff Conklin, cognexus.org Time Complex development situations are ones in which this… 25 Michael Quinn Patton AEA eStudy webinar 2014
  26. 26. Michael Quinn Patton AEA eStudy webinar 2014 26 And this…
  27. 27. Turns out to be this… Time 27 Michael Quinn Patton AEA eStudy webinar 2014
  28. 28. Michael Quinn Patton AEA eStudy webinar 2014 28 …looks like this
  29. 29. DE for MFLN: What It Looks Like Inductive
  30. 30. DE for MFLN: What It Looks Like Inductive Chaotic and messy
  31. 31. DE for MFLN: What It Looks Like Inductive Chaotic and messy Embrace forks in the road
  32. 32. DE for MFLN: What It Looks Like Inductive Chaotic and messy Embrace forks in the road Developmental moments Track what’s going on Understand what’s going on Adapt to what’s going on
  33. 33. DE for MFLN: What It Looks Like Everyone is an evaluator
  34. 34. DE Successes for MFLN
  35. 35. DE Successes for MFLN Very responsive to our partners’ programmatic requests and innovations
  36. 36. DE Successes for MFLN Very responsive to our partners’ programmatic requests and innovations You. Here. Now.
  37. 37. DE Successes for MFLN Very responsive to our partners’ programmatic requests and innovations You. Here. Now. Programming
  38. 38. DE Successes for MFLN Very responsive to our partners’ programmatic requests and innovations You. Here. Now. Programming “Walk the talk”: model use of social media, collaborative learning, personal learning networks
  39. 39. DE Challenges for MFLN Innovation is challenging, slow, and can be contentious
  40. 40. DE Challenges for MFLN Innovation is challenging, slow, and can be contentious Moving from program improvement mode to redesign based on evaluation findings
  41. 41. DE Challenges for MFLN Innovation is challenging, slow, and can be contentious Moving from program improvement mode to redesign based on evaluation findings IRB challenges
  42. 42. DE Challenges for MFLN Innovation is challenging, slow, and can be contentious Moving from program improvement mode to redesign based on evaluation findings IRB challenges Daily, evaluative thinking
  43. 43. DE Challenges for MFLN Innovation is challenging, slow, and can be contentious Moving from program improvement mode to redesign based on evaluation findings IRB challenges Daily, evaluative thinking Platform delivery limitations
  44. 44. DE Challenges for MFLN Innovation is challenging, slow, and can be contentious Moving from program improvement mode to redesign based on evaluation findings IRB challenges Daily, evaluative thinking Platform delivery limitations Military culture
  45. 45. DE Challenges for MFLN Innovation is challenging, slow, and can be contentious Moving from program improvement mode to redesign based on evaluation findings IRB challenges Daily, evaluative thinking Platform delivery limitations Military culture Scaling up
  46. 46. DE Features for MFLN: Nodal Evaluations Webinars Social media Focus groups Deliverables/plans of work • Why as much as what Documentation • Change, innovation, decision-making
  47. 47. DE Features for MFLN: Reporting From Concentration Areas • PIs – Monthly via Google forms: narrative updates, pubs, presentations, challenges • Social Media Specialists – Monthly via Google forms: blogs, AaE, CEUs – Webinar evaluation reports
  48. 48. DE Features for MFLN: Reporting To CAs: • Monthly social media from Sprout Social • Quarterly webinar data reports Internal/External: • Monthly (internal/DoD) • Quarterly (NIFA) • Quarterly webinar (internal/DoD) • Annual (internal/D0D/NIFA)
  49. 49. DE Features for MFLN: Reflective Discussion CAs: • Monthly social media specialists meetings – NetLit, processes, evaluation, communications Leadership: • Weekly meetings PIs: • Monthly meetings
  50. 50. DE Features for MFLN: Learning and Working Paradigms Leadership Reflection Transparency Collaboration Action Innovation

×