GENEVA EVALUATION NETWORK WORKSHOP CONFERENCE EVALUATIONOrganized by Laetitia Lienart & Glenn O’Neil Geneva, 16 March 2011
WHY IS IT IMPORTANT TO EVALUATE CONFERENCES? Institutional Memory ESSENTIAL FOR… Continuous Accountability Learning/ Improvement
WHAT ARE THE OBJECTIVES OF CONFERENCE EVALUATIONS? ASSESS IMMEDIATE IMPACTSPROCESS OUTCOMES (lasting changes) At individual Governance Reactions level Learnings/ At organizational Programme Benefits level Logistics & Applications At country level onsite support Information/ Communication
RELEVANT DATA COLLECTION METHODSFace-to-face or phone individual interviews (structured & semi-structured)Focus group interviewsOnline surveysPrinted surveysStructured observations of key sessions and conference areasReview of conference programme and online resourcesReview of statistical data on conference registration, scholarship recipients, abstracts, etcReview of statistical data and evaluation findings from previous conference to allow comparison over time
RELEVANT DATA COLLECTION METHODS (cont.)Use of rapporteurs to follow sessions addressing key topics. Their feedback can be also used to measure some indicators (e.g. number of sessions presenting new findings).Use of conference “instant” feedback systems.Use of network analysis and mapping.Analysis of the conference media coverage.Review of posts and comments left by delegates and non-attendees on the conference blog, Facebook page and Twitter.
Focus on IMPACT ASSESSMENT Assessing conference impact(s) is feasible but needs to beplanned and budgeted for at the planning stage (incl. in ToRs) Methods: follow-up survey (online/face-to-face), action plans Ex: AIDS 2008 follow-up survey (1,5 year after) 1,195 AIDS 2008 delegates completed the survey About 2/3 had learnt something new and had changed some aspects of their work practice thanks to the new knowledge gained at the conference Almost half reported that AIDS 2008 had directly influenced their organizations’ HIV work Almost 4 in 10 were aware of AIDS 2008’s influences on HIV work, policies or advocacy in their countries 75% had kept in contact with at least 1 person met at AIDS 2008, mainly to exchange knowledge, lessons learnt and/or suggested solutions (86%)
USE OF EVALUATION FINDINGSEvaluation findings should be “very usable” as conferences are often repeated annually or bi-annually.Importance of “buy-in” of conference organizers.Sharing of evaluation plan with conference organizers and committees/working groups*.Evaluation reports: the quality of content and format is crucial to attract readers and convince them that evaluation results are reliable and useable.Dissemination of evaluation results: timely, use a variety of channels depending on the target audience.Use of follow-up mechanisms** with conference organizers and relevant stakeholders.Review progress on evaluation findings in the lead-up to the next conference.
KEY LESSONS LEARNT1. Over-positive feedback (new strategy to be tested in 2011).2. Evaluation report more used as an accountability & marketing tool rather than for learning purposes.3. Unwillingness of conference organizers to devote adequate human & financial resources to evaluation.4. Impact assessment remains a challenge (difficult to measure the extent to which changes are attributable to the conference, especially policies, norms & guidelines).5. Data disaggregation is important to make evaluation results more accurate and useful*.6. Conference evaluation provides unique opportunity to see how findings are integrated (or not) into future conferences.
Further informationProceedings (slides & handouts) of a 1-day workshop onconference evaluation held in Nov 2010 are available onrequest (email Laetitia.Lienart@iasociety.org)Feel free to join the Conference Evaluation Google Group:http://groups.google.com/group/conference_evaluationGlenn’s blog has more resources on conference evaluation,see category “event evaluation:http://intelligentmeasurement.wordpress.com/category/event-evaluation/