Edmonton may2 abridged

348 views

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
348
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • References:Compton and Taut 2007 – defining ECBCousins2008 and Fleischer2008Preskill and Boyle 2008 (both)Cousins 2008
  • Edmonton may2 abridged

    1. 1. draft<br />A Logic Modelfor Evaluation Capacity Development <br />Abridged version of slides presented at the <br />Canadian Evaluation Society 2011 Conference <br />Edmonton, Alberta<br />May 2nd<br />by <br />Michael Obrecht and Robyn Sachs<br />with acknowledgement of the contribution of <br />Wendy Ryan<br />
    2. 2. Note on the Abridged Version<br />This version of the presentation focuses on the draft logic model and our ideas for next steps <br />At the conference we had also presented slides on the evaluation capacity development project that inspired our work<br />
    3. 3. Why a Logic Model?<br />A simple logic model for evaluation capacity development should be useful when …<br />Assessing needs for development of evaluation capacity<br />Discussing approaches to improving evaluation capacity <br />Examining progress in capacity development<br />A logic model will complement other types of model that are already documented in the evaluation literature<br />
    4. 4. Drafting the Logic Model<br />We brainstormed on Inputs, Activities, Outputs, and Outcomes by reflecting on our experiences with evaluation capacity development in Indiaand Canada <br />We searched the literature for existing models on evaluation capacity<br />We found: “A Multidisciplinary Model of Evaluation Capacity Development” by HalliePreskill and Shanelle Boyle in the American Journal of Evaluation, 2008<br />
    5. 5. Work by Others<br />The Preskill-Boyle model (2008)<br />Shows the elements of a learning organization <br />Thorough and complete<br />Scholarly and evidence-based, with 52 supporting references<br />Non-linear - Does not illustrate sequential development of evaluation capacity<br />
    6. 6. Other Points from the Literature …<br />Specifying the goals for capacity development up front helps orient activities<br />Organizational culture, attitude, and readiness will affect evaluation capacity development<br />Evaluation capacity development may lead to organizational learning and change<br />It can be especially challenging to develop evaluation capacity in a government environment<br />
    7. 7. … continued<br />Evaluation capacity development is a long, slow process<br />It involves consideration of not just education and experience, but also motivation and personality <br />Soft-skills are critical <br />A balancing of formal training, mentorship, and hands-on experience is required<br />Two or more years of capacity development are needed before someone is ready to lead an evaluation<br />
    8. 8. First Draft of a Logic Model<br />Initial efforts were focused on getting the broad categories clear<br />Working definitions of Inputs, Activities, Outputs and Outcomes were flexible<br />Our exposition of the draft model will focus on Inputs and Activities<br />
    9. 9. Outcomes<br /> Inputs<br /> Activities<br /> Outputs<br />I<br /><ul><li>Identify
    10. 10. Engage
    11. 11. Assess
    12. 12. Train
    13. 13. Mentor
    14. 14. Conduct
    15. 15. Portfolios
    16. 16. Experience
    17. 17. Careers
    18. 18. Staff
    19. 19. Managers
    20. 20. Capabilities
    21. 21. Frameworks
    22. 22. Evaluations</li></ul>People<br />I<br /><ul><li>Plan
    23. 23. Govern
    24. 24. Monitor
    25. 25. Inspire</li></ul>I<br /><ul><li>Plans
    26. 26. Policies
    27. 27. Assurance
    28. 28. Receptivity</li></ul>I<br />Sustainable <br />evaluation practice<br /><ul><li>Stability
    29. 29. Use
    30. 30. Demand</li></ul>Structures<br />Structures<br />I<br />I<br />I<br />Context<br /><ul><li>Adapt
    31. 31. Flexiblity
    32. 32. Adaptation</li></li></ul><li>Availability<br /> Outcomes<br /> Inputs<br /> Activities<br /> Outputs<br />Skills and experience <br />I<br /><ul><li>Identify
    33. 33. Engage
    34. 34. Assess
    35. 35. Train
    36. 36. Mentor
    37. 37. Conduct
    38. 38. Portfolios
    39. 39. Experience
    40. 40. Careers
    41. 41. Staff
    42. 42. Managers
    43. 43. Capabilities
    44. 44. Frameworks
    45. 45. Evaluations</li></ul>People<br />Motivation<br />Interests<br />I<br /><ul><li>Plan
    46. 46. Govern
    47. 47. Monitor
    48. 48. Inspire</li></ul>I<br /><ul><li>Plans
    49. 49. Policies
    50. 50. Assurance
    51. 51. Receptivity</li></ul>I<br />Sustainable <br />evaluation practice<br /><ul><li>Stability
    52. 52. Use
    53. 53. Demand</li></ul>Structures<br />Structures<br />I<br />I<br />I<br />Context<br /><ul><li>Adapt
    54. 54. Flexibility
    55. 55. Adaptation</li></li></ul><li> Outcomes<br /> Inputs<br /> Activities<br /> Outputs<br />I<br /><ul><li>Identify
    56. 56. Engage
    57. 57. Assess
    58. 58. Train
    59. 59. Mentor
    60. 60. Conduct
    61. 61. Portfolios
    62. 62. Experience
    63. 63. Careers
    64. 64. Staff
    65. 65. Managers
    66. 66. Capabilities
    67. 67. Frameworks
    68. 68. Evaluations</li></ul>People<br />Resources<br />Reporting Relationships<br />I<br /><ul><li>Plan
    69. 69. Govern
    70. 70. Monitor
    71. 71. Inspire</li></ul>I<br /><ul><li>Plans
    72. 72. Policies
    73. 73. Assurance
    74. 74. Receptivity</li></ul>I<br />Sustainable <br />evaluation practice<br /><ul><li>Stability
    75. 75. Use
    76. 76. Demand</li></ul>Senior Management support<br />Structures<br />Structures<br />Demand<br />I<br />I<br />I<br />Context<br /><ul><li>Adapt
    77. 77. Flexibility
    78. 78. Adaptation</li></ul>Understanding<br />
    79. 79. Outcomes<br /> Inputs<br /> Activities<br /> Outputs<br />I<br /><ul><li>Identify
    80. 80. Engage
    81. 81. Assess
    82. 82. Train
    83. 83. Mentor
    84. 84. Conduct
    85. 85. Portfolios
    86. 86. Experience
    87. 87. Careers
    88. 88. Staff
    89. 89. Managers
    90. 90. Capabilities
    91. 91. Frameworks
    92. 92. Evaluations</li></ul>People<br />I<br /><ul><li>Plan
    93. 93. Govern
    94. 94. Monitor
    95. 95. Inspire</li></ul>I<br /><ul><li>Plans
    96. 96. Policies
    97. 97. Assurance
    98. 98. Receptivity</li></ul>I<br />Sustainable <br />evaluation practice<br /><ul><li>Stability
    99. 99. Use
    100. 100. Demand</li></ul>Structures<br />Structures<br />Culture<br />Legislation<br />I<br />I<br />I<br />Context<br /><ul><li>Adapt
    101. 101. Flexibility
    102. 102. Adaptation</li></ul>Politics<br />
    103. 103. Identify need for personnel<br /> Outcomes<br /> Inputs<br /> Activities<br /> Outputs<br />Reassign or recruit<br />I<br /><ul><li>Identify
    104. 104. Engage
    105. 105. Assess
    106. 106. Train
    107. 107. Mentor
    108. 108. Conduct
    109. 109. Portfolios
    110. 110. Experience
    111. 111. Careers
    112. 112. Staff
    113. 113. Managers
    114. 114. Capabilities
    115. 115. Frameworks
    116. 116. Evaluations</li></ul>People<br />Determine developmental needs<br />Provide training and mentoring<br />Conduct evaluative work<br />I<br /><ul><li>Plan
    117. 117. Govern
    118. 118. Monitor
    119. 119. Inspire</li></ul>I<br /><ul><li>Plans
    120. 120. Policies
    121. 121. Assurance
    122. 122. Receptivity</li></ul>I<br />Sustainable <br />evaluation practice<br /><ul><li>Stability
    123. 123. Use
    124. 124. Demand</li></ul>Structures<br />Structures<br />I<br />I<br />I<br />Context<br /><ul><li>Adapt
    125. 125. Flexibility
    126. 126. Adaptation</li></li></ul><li> Outcomes<br /> Inputs<br /> Activities<br /> Outputs<br />I<br /><ul><li>Identify
    127. 127. Engage
    128. 128. Assess
    129. 129. Train
    130. 130. Mentor
    131. 131. Conduct
    132. 132. Portfolios
    133. 133. Experience
    134. 134. Careers
    135. 135. Staff
    136. 136. Managers
    137. 137. Capabilities
    138. 138. Frameworks
    139. 139. Evaluations</li></ul>People<br />Overall plan for the ECD<br />Evaluation and monitoring policy<br />I<br /><ul><li>Plan
    140. 140. Govern
    141. 141. Monitor
    142. 142. Inspire</li></ul>I<br /><ul><li>Plans
    143. 143. Policies
    144. 144. Assurance
    145. 145. Receptivity</li></ul>G<br />I<br />Sustainable <br />evaluation practice<br /><ul><li>Stability
    146. 146. Use
    147. 147. Demand</li></ul>Evaluation planning<br />Structures<br />Structures<br />Evaluation quality control<br />Support innovation and creativity<br />I<br />I<br />I<br />Context<br /><ul><li>Adapt
    148. 148. Flexibility
    149. 149. Adaptation</li></li></ul><li> Outcomes<br /> Inputs<br /> Activities<br /> Outputs<br />I<br /><ul><li>Identify
    150. 150. Engage
    151. 151. Assess
    152. 152. Train
    153. 153. Mentor
    154. 154. Conduct
    155. 155. Portfolios
    156. 156. Experience
    157. 157. Careers
    158. 158. Staff
    159. 159. Managers
    160. 160. Capabilities
    161. 161. Frameworks
    162. 162. Evaluations</li></ul>People<br />I<br /><ul><li>Plan
    163. 163. Govern
    164. 164. Monitor
    165. 165. Inspire</li></ul>I<br /><ul><li>Plans
    166. 166. Policies
    167. 167. Assurance
    168. 168. Receptivity</li></ul>G<br />I<br />Sustainable <br />evaluation practice<br /><ul><li>Stability
    169. 169. Use
    170. 170. Demand</li></ul>Structures<br />Structures<br />Identify response to critical cultural and other environmental variables<br />I<br />I<br />I<br />Context<br /><ul><li>Adapt
    171. 171. Flexibility
    172. 172. Adaptation</li></li></ul><li>Next Steps<br />Further reading of the literature<br />Work on details of the model<br />Draw in more of the logic links<br />Interview practitioners for feedback on the draft<br />Possible data collection through surveys<br />
    173. 173. Recommended Readings<br />Two issues of the Canadian Journal of Program Evaluation, focused on evaluation capacity:<br />1) Volume 23, Number 3, 2008 <br />2) Special issue, 2011<br />and<br />Preskill, H., and Boyle, S. A Multidisciplinary Model of Evaluation Capacity Development. American Journal of Evaluation (Online). September 2008.<br />
    174. 174. Thank You<br />We welcome your feedback!<br />For further information:<br />Michael Obrecht m@obrecht.ca<br />Robyn Sachs sachs.robyn@gmail.com<br />

    ×