Evaluating conferences and events: new approaches and initiatives - summary of new ideas and examples

1. Evaluating conferences and
events: new approaches and
initiatives
10th EES Biennial Conference
5 October 2012
Helsinki, Finland
Glenn O’Neil
oneil@owlre.com
www.owlre.com

2. The two case studies
• Both conferences undertake evaluation
• The conference evaluation differs in the
number and type of evaluation tools used, in
addition to aims and objectives

3. Approaches and ideas from two case
studies
The International AIDS Conference:
biannual, over 20,000 participants
The Lift technology conference:
annual, some 700 participants

4. Commonly used evaluation tools for
event/conference evaluation
• Face-to-face or phone individual interviews
• Focus groups
• Online/printed surveys
• Structured observations of key sessions and
conference areas
• Review of online resources/activities
• Review of statistical data on conference
registration
• Comparison with previous conferences

5. The question of impact
Some main questions posed by conference
organisers and funders are:
• What is the impact of a conference on
participants?
• What does it change in the way participants do
their jobs?

6. International AIDS Conference & impact
•Methods used: follow-up survey (online/face-to-face)
and action plans
Ex: AIDS 2008 follow-up survey (1.5 year after)
 1,195 AIDS 2008 delegates completed the survey
 About 2/3 had learnt something new and had changed some
aspects of their work practice
 Almost half reported that AIDS 2008 had directly influenced
their organizations’ HIV work
 Almost 4 in 10 were aware of AIDS 2008’s influences on HIV
work, policies or advocacy in their countries
 75% had kept in contact with at least 1 person met
Source: http://www.aids2008.org/admin/images/upload/AIDS_2008_follow_up_survey_report.pdf

7. Lift Conference and impact
•Methods used: follow-up survey (online) and
testimonies

8. Lift Conference and impact

9. Other emerging evaluation tools
• Use of conference “instant” feedback
systems
• Use of the “mystery shopper” approach
• Use of network analysis and mapping
• Analysis of conference media coverage
• Analysis of social media activity

10. Use of evaluation findings
• Evaluation findings should be “very usable” as
conferences are often repeated
• Importance of “buy-in” of conference organizers
• The quality of content and format of evaluation
reports is crucial to attract readers and convince
them that evaluation results are reliable and
useable
• Dissemination of evaluation results should be
timely, and use a variety of channels depending on
the target audience
• Use of follow-up mechanisms with conference
organizers and relevant stakeholders is
recommended

11. Lessons from conference evaluation
• How to deal with over-positive feedback
• Evaluation findings can be used more for
accountability and marketing rather than for
learning and improvements
• Challenges in getting adequate resources for
conference evaluation
• Challenges in moving beyond the basic satisfaction
measures
• Factoring in the importance of participants’ views

12. Q&A
Thanks to Laetitia Lienart, former evaluation manager, IAS
Join the Conference Evaluation Google Group:
http://groups.google.com/group/conference_evaluation
Download this presentation on my blog
www.intelligentmeasurement.wordpress.com
oneil@owlre.com
@glenn_oneil
glennoneil
www.owlre.com 12