Managing Sound In-house                   Evaluations                                  Elga Nikolova     Copyright e-NOVA ...
Our perspective todayWe will focus on Program evaluation Goal-oriented evaluation Practices that work, rather than theo...
Why evaluation?   To identify gaps   To find out if a program is working well   To make sure clients are satisfied   T...
Evaluation   IS A DECISION-MAKING TOOLOCASI Professional             Managing Sound In-house Evaluations e-               ...
The manager’s role in the evaluation     Capacity                               Possibility     Resources                 ...
BUILDING CAPACITY FOR   EVALUATIONOCASI Professional             Managing Sound In-house Evaluations e-                   ...
Capacity building blocks   Champion   Competence   Cost   Resources   Common language   Responsibilities   Schedule...
Five things to keep in mind about costs You will have to invest time even if you hire an  external evaluator Make time f...
Three tips about evaluation resources • Invest in basic technology: SurveyMonkey,   Skype • Develop staff competence for e...
The importance of common language • What do these terms mean?                                          Deliverable OCASI P...
The need for common evaluation                  language What terms do we use? What terms do our funders and stakeholder...
MAKING EVALUATION POSSIBLE   PROGRAM DESIGN AND PROGRAM EVALUABILITYOCASI Professional             Managing Sound In-house...
Fundamental requirements for evaluation  The target population is clearly defined  The program has objectives, specified...
If you do not know where you are going,how will you know if you are getting there?OCASI Professional             Managing ...
Theory of changeThe Turkey Dinner Project                                 Purpose: to grow our friendship                 ...
The Turkey Dinner Logic Model                                         Why am I doing this? (Purpose and Objective)        ...
ACTION   FROM DATA TO KNOWLEDGE TO PRACTICEOCASI Professional             Managing Sound In-house Evaluations e-          ...
Five stepping stones from data to                decision-making Determine what data to collect, when and  how Select a ...
Collecting good data How do you know that change is happening?       Attitudes      Knowledge      Behaviour      Act...
Choosing a comparison framework There is no evaluation without comparison Your choice of comparisonframeworks:         ...
Tools: SurveyMonkey PROS                                       CONS  Stores data                               One accou...
House tools   Observation templates   Excel spreadsheets   Forms   Process descriptions: how do you store,    analyse ...
Implement your evaluation Evaluation is a project: use common project  management sense Align evaluation timing with the...
You have paid for it: use it Once in a while, look at your findings Share success and challenges Have evaluation result...
More questions?                             Elga Nikolova                       elga@e-novaconsulting.com                 ...
Upcoming SlideShare
Loading in...5
×

A9 managing sound in house evaluations

301

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
301
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "A9 managing sound in house evaluations"

  1. 1. Managing Sound In-house Evaluations Elga Nikolova Copyright e-NOVA Consulting, 2012.This work is the intellectual property of the author. Permissions granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the authorOCASI Professional Managing Sound In-house Evaluations e- 1Development Conference, 2012 NOVA Consulting
  2. 2. Our perspective todayWe will focus on Program evaluation Goal-oriented evaluation Practices that work, rather than theory Managing vs. doingOCASI Professional Managing Sound In-house Evaluations 2Development Conference, 2012 e-NOVA Consulting
  3. 3. Why evaluation? To identify gaps To find out if a program is working well To make sure clients are satisfied To measure the impact of programs and services To improve programs and services To ensure that programs contribute to the organization’s vision and goals To justify requests for funding To meet funding requirements To assist with planning To find the value we are getting for money spent on a programOCASI Professional Managing Sound In-house Evaluations 3Development Conference, 2012 e-NOVA Consulting
  4. 4. Evaluation IS A DECISION-MAKING TOOLOCASI Professional Managing Sound In-house Evaluations e- 4Development Conference, 2012 NOVA Consulting
  5. 5. The manager’s role in the evaluation Capacity Possibility Resources Ac t i o n Program set up (program Tools and evaluability) processes Review and analysis of data Decision-makingOCASI Professional Managing Sound In-house Evaluations e- 5Development Conference, 2012 NOVA Consulting
  6. 6. BUILDING CAPACITY FOR EVALUATIONOCASI Professional Managing Sound In-house Evaluations e- 6Development Conference, 2012 NOVA Consulting
  7. 7. Capacity building blocks Champion Competence Cost Resources Common language Responsibilities ScheduleOCASI Professional Managing Sound In-house Evaluations 7Development Conference, 2012 e-NOVA Consulting
  8. 8. Five things to keep in mind about costs You will have to invest time even if you hire an external evaluator Make time for your staff to participate Technology and good process may take time to establish but they save time down the road Budget time to review and analyze the collected data Beware of the hidden costsOCASI Professional Managing Sound In-house Evaluations 8Development Conference, 2012 e-NOVA Consulting
  9. 9. Three tips about evaluation resources • Invest in basic technology: SurveyMonkey, Skype • Develop staff competence for evaluation • Use external consultants smartly OCASI Professional Managing Sound In-house Evaluations 9 Development Conference, 2012 e-NOVA Consulting
  10. 10. The importance of common language • What do these terms mean? Deliverable OCASI Professional Managing Sound In-house Evaluations 10 Development Conference, 2012 e-NOVA Consulting
  11. 11. The need for common evaluation language What terms do we use? What terms do our funders and stakeholders use? When we say the same word, do we mean the same thing?OCASI Professional Managing Sound In-house Evaluations 11Development Conference, 2012 e-NOVA Consulting
  12. 12. MAKING EVALUATION POSSIBLE PROGRAM DESIGN AND PROGRAM EVALUABILITYOCASI Professional Managing Sound In-house Evaluations e- 12Development Conference, 2012 NOVA Consulting
  13. 13. Fundamental requirements for evaluation  The target population is clearly defined  The program has objectives, specified deliverables and outcomes  There is a logical connection between needs, objectives, actions, deliverables and outcomes: theory of change and program logic model OCASI Professional Managing Sound In-house Evaluations 13 Development Conference, 2012 e-NOVA Consulting
  14. 14. If you do not know where you are going,how will you know if you are getting there?OCASI Professional Managing Sound In-house Evaluations 14Development Conference, 2012 e-NOVA Consulting
  15. 15. Theory of changeThe Turkey Dinner Project Purpose: to grow our friendship ??? Objective: serve a great Thanksgiving dinner to make friends feel happy and appreciated OCASI Professional Managing Sound In-house Evaluations 15 Development Conference, 2012 e-NOVA Consulting
  16. 16. The Turkey Dinner Logic Model Why am I doing this? (Purpose and Objective) To make friends feel happy and appreciated To treat my friends to a nice, memorable Thanksgiving Dinner Inputs Outputs OutcomesOne turkey Activities Product (what?) Process /quality So What? (what?) One roast turkey outcomes (how?)Cooking Stuff and roast Table arranged for 16ingredients Turkey juicy and turkey Short-term Long-term tenderCooking skills Lay table Music appropriate Friends enjoy Plan seating for the moment. Participation (Who?) dinner Stronger Select DVDs 16 guests attend Everyone involvedSeasonal Friends feel friendshipsdecorations Serve dinner dinner in conversation happy during Maintain Guests stuff holidaysNice music conversation themselves on Friends feel turkey appreciated Assumptions: good food makes friends happy, all External Factors: Weather, friends’ availability, friends’ friends like music, all friends like turkey, host will have mood upon arrival, guests’ like/dislike for each other enough time to prepareOCASI Professional Managing Sound In-house Evaluations e- 16Development Conference, 2012 NOVA Consulting
  17. 17. ACTION FROM DATA TO KNOWLEDGE TO PRACTICEOCASI Professional Managing Sound In-house Evaluations e- 17Development Conference, 2012 NOVA Consulting
  18. 18. Five stepping stones from data to decision-making Determine what data to collect, when and how Select a comparison framework Have tools and processes in place Implement your evaluation Once in a while look at the findingsOCASI Professional Managing Sound In-house Evaluations 18Development Conference, 2012 e-NOVA Consulting
  19. 19. Collecting good data How do you know that change is happening?  Attitudes Knowledge Behaviour Action Involve clients and staff Use observations Count Keep it small Collect data about process and qualityOCASI Professional Managing Sound In-house Evaluations 19Development Conference, 2012 e-NOVA Consulting
  20. 20. Choosing a comparison framework There is no evaluation without comparison Your choice of comparisonframeworks:  Baseline data  Standards set by funders  Previous experience  Literature  Similar programs in the community Agree on a comparison frameworkfrom the start of the program Your comparison framework could changeOCASI Professional Managing Sound In-house Evaluations 20Development Conference, 2012 e-NOVA Consulting
  21. 21. Tools: SurveyMonkey PROS CONS  Stores data  One account, one login  Provides sample questions  Requires some survey design  Makes data entry easy knowledge  Can keep track of separate client  Has a learning curve groups  Has to be managed  Makes analysis easy  Makes presentation easy  Free for small questionnairesOCASI Professional Managing Sound In-house Evaluations 21Development Conference, 2012 e-NOVA Consulting
  22. 22. House tools Observation templates Excel spreadsheets Forms Process descriptions: how do you store, analyse and report data?OCASI Professional Managing Sound In-house Evaluations 22Development Conference, 2012 e-NOVA Consulting
  23. 23. Implement your evaluation Evaluation is a project: use common project management sense Align evaluation timing with the program cycle Give your participants a chance to speak to a third partyOCASI Professional Managing Sound In-house Evaluations 23Development Conference, 2012 e-NOVA Consulting
  24. 24. You have paid for it: use it Once in a while, look at your findings Share success and challenges Have evaluation results ready for reporting and proposal writing timesOCASI Professional Managing Sound In-house Evaluations 24Development Conference, 2012 e-NOVA Consulting
  25. 25. More questions? Elga Nikolova elga@e-novaconsulting.com 416 732 7979OCASI Professional Managing Sound In-house Evaluations 25Development Conference, 2012 e-NOVA Consulting

×