Learning Objectives• At the end of this session you should be able to:• Articulate the nature and importance of evaluation to the events management function• Describe what event evaluation should focus on• Explain the purpose of event evaluation, including the needs and expectations of stakeholders• Describe the cyclical, holistic nature of event evaluation using the three critical Ss of events• Assess the different forms of evaluation models and methods• Formulate and prepare an event evaluation report
Event Evaluation Is concerned with assessment, which usually involves measuring a set of key variables, as well as monitoring those variables todetermine positive and negative outcomes. It is a subjective determination that can utilize objective quantitative measures (Getz, 1997)
Event Evaluation Data Descriptive Numerical Numerical and Descriptive
Event Types & Evaluation Tools Image Maker • Content Analysis Tourism Product • Visitor Exit SurveyEconomic Development • Economic Impact Assessment Catalyst Transformer of Socio- • Social Impact Scale Analysis cultural LandscapeSustainable Development • Environmental Impact Analysis Model
Conducting Event Evaluation (1) • Determine the purpose of the event evaluation exercise, including whether the evaluation isStep 1 necessary. • Identify what should be the focus of evaluation.Step 2 • Select the most suitable approach for conducting theStep 3 evaluation exercise. • Select model(s) for evaluation as a framework for data collection based on suitability, practicality andStep 4 relevance.
Conducting Event Evaluation (2) • Develop appropriate instruments and data collection strategies based on the methodsStep 5 selected. • Collect and analyse data and findings.Step 6Step 7 • Prepare and disseminate event evaluation report. • Formulate and implement decisions for improvement based on findings of event evaluationStep 8 report.
REASONS FOR ALL EVENT EVALUATION Allows for more informed decisions towards greater efficiency and more positive results to be made To measure success or failure INTERNALLY-DRIVEN REASONS EXTERNALLY-DRIVEN REASONS To determine whether goals and To determine level of awarenessobjectives have been met of sponsors’ products/services To engender accountability To satisfy accountability requirements To identify and address problemsand challenges To determine level of media interest and coverage To determine whether eventmanagement functions have To determine event’s level ofachieved expected outcomes impact on tourist arrivals To understand who attends the To determine event’s level ofevent to determine who else can impact on business and otherbe targeted related industries To determine worth of the event To determine whether eventto its workers and volunteers satisfies the expectations of community stakeholders To determine whether and howthe event can remain viable and To determine worth of the eventbecome sustainable to its patrons
Evaluation Considerations• Does the event organization/event host have, or have access to the requisite resources to conduct an event evaluation exercise?• Have event evaluations been conducted on the event in the past? And if so, how often and what kind of evaluation?• What use was made of the evaluation report?• What is the nature of the environment in which the event operates?• Can it facilitate the execution of an event evaluation?• Does the event have access to a wide range of information it can use to conduct the evaluation exercise?
What to Evaluate?1. Event Elements – number of patrons; size of group; demographics of patrons; attendees address; source of information on the event; number of times attending; patron satisfaction; quality and impact of event programme; quality of merchandise; food and beverage, etc.2. Event Organization Elements – nature and quantum of resources; human resources; volunteers perceptions working on the event; income and expenditure; cash flow; level of investment; sponsorship dollar, etc.3. Event Context Elements – nature and amount of local suppliers used; impacts on event; impacts of event; community perceptions; level of media coverage; media value; interest and presence
Event Evaluation Perspectives• Event evaluations are focused on measuring and monitoring the implementation of an event (Allen et al., 2008; Bowdin et al., 2006)• Every aspect of the event must be evaluated (Tum et al., 2006), including factors such as human resource management and volunteerism, facilities and access and hospitality (Wendroff, 2004)• Event evaluation is mostly conducted after the event (Tum et al., 2006)
Problems with Event Evaluation• Missed opportunities to correct detected problems• Minimal focus on evaluation at the planning phases, which is vital to understanding many decisions taken at the implementation stage• Key elements of the evaluations may not be deployed effectively, if at all
Event Evaluation Approaches (1)• Quantitative – primarily concerned with counting complex data sets such as attendance levels, information sources and event activities.• Challenges and limitations of using quantitative approaches exclusively: Low response rates to surveys Inconsistencies arising out of varied interpretations of terminology used in surveys leading to over- calculation and possible misrepresentation of data
Event Evaluation Approaches (2)• Qualitative – utilizes open-ended to allow for in depth responses and focuses on capturing opinion and attitudes on matters such as reasons for attending the event; benefits sought; level of satisfaction; stakeholder attitudes, etc.• Can complement quantitative approaches because useful details can emerge that may not be derived or represented numerically• Challenges and limitations of using qualitative approaches exclusively: Cannot reach the numbers of quantitative approaches
Event Evaluation Approaches (3)• Financial approaches – are concerned with collecting, assessing and monitoring numerical data on the fiscal elements of the event such as cash flow, gate receipts, patron expenditure, profit, loss and debt• Economic approaches – examine numerical data related to employment, tax, visitor expenditure and triggered economic activity in other businesses and related industries• Challenge with both approaches can be address through the use of non-financial and non-economic approaches
Event Evaluation Approaches (4)• Non-financial and non-economic approaches – utilize qualitative approaches to monitor and assess fiscal and economic matters that cannot be easily quantified but which are relevant such as: Assessment of intangible costs and benefits Assessment of net value Assessment of stakeholder perceptions Assessment of economic and market factors that can influence the event
Event Evaluation Approaches (4)• Critical three Ss Event Significance – assessing, measuring and monitoring the nature, purpose and intent of the event and feasibility to realize the expected outcome Event Success – determining whether the event execution and contributing elements have achieved the expected outcome Event Sustainability – measuring, assessing and monitoring the events impact on the environment and community and; the potential for the event to be continued in future
Models Used in Event Evaluation• Impact Assessment – focuses on measuring economic, social, environmental and cultural impact of events, particularly in instances where significant justification for the event may be required• Cost-benefit Analysis – identifies and measures the costs and benefits of an event and takes into account externalities or spill-over benefits for the host community (Burgen and Mules, 2000)• Triple-bottom Line Evaluation – a systematic framework for measuring and reporting the events performance against economic, social and environmental parameters, to determine negative or positive impacts on the host community (Fredline et al., 2005)• Ethnographic Profile - Typically used in the evaluation of cultural festivals/special events, ethnography provides an assessment of these experience-related events through a process of analysis which encompasses participant observation, interviewing and documentary resources
Summary• Evaluating events provides a measure for performance and delivery for event organizers, stakeholders and event consumers• An event evaluation should focus on, among several other factors, the three Critical Ss – Event Significance, Event Success, and Event Sustainability• Impact Assessment , Cost-benefit Analysis, Triple-bottom Line Evaluation and Ethnographic Profile are the four models that can be utilized to evaluate events• Evaluation methods depend on the nature of the event and the requirements of stakeholders