Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Eventroi
Similar to Eventroi (20)
More from CommGres Conference Agency
More from CommGres Conference Agency (17)
Recently uploaded
Recently uploaded (20)
Eventroi
- 1. Measuring ROI for Meetings and Events Dr. Elling Hamso Managing Partner European Event ROI Institute
- 2. To LEARN something which enables me to DO something which provides VALUE to my company Why are you here?
- 3. To INFLUENCE* participants to DO something which adds VALUE to stakeholders at the lowest possible COST * or educate, empower, inspire, persuade The Purpose of Corporate Events
- 4. Satisfaction & Planned Actions Learning Application Impact ROI The Event ROI Pyramid Level 1 Level 2 Level 3 Level 4 Level 5 (information, skills, attitudes, relationships)
- 5. How will you use what you learned? No Probably Not Maybe Probably Yes Definitely Average Score Discuss with my colleagues 0% 0% 0% 37.5% 62.5% 4.6 Review risk management procedures in my organisation 0% 0% 0% 50% 50% 4.5 Conduct formal risk assessments before major events 0% 0% 50% 37.5% 12.5% 3.6 Review how we check on suppliers’ safety procedures 0% 12.5% 12.5% 62.5 12.5 3.8 Meet with our insurance brokers to review current policies 12.5% 12.5% 62.5% 0% 12.5% 2.9 Other (specify below) 0% 0% 0% 0% 0% 0.0
- 6. Do you know the basic principles of how to assess risk? 87% 6 2 2 4 0 0 0 0 0 0 0 AFTER the seminar 40% 0 0 0 0 4 2 2 4 0 2 0 BEFORE the seminar Avg 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Do you feel comfortable with leading a risk assessment exercise for a major event? 70% 2 2 4 0 0 4 2 0 0 0 0 AFTER the seminar 31% 0 0 0 0 2 4 0 2 2 2 2 BEFORE the seminar Avg 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Could you draw a matrix for classification for different types of risk? 70% 4 0 2 2 0 4 2 0 0 0 0 AFTER the seminar 14% 0 0 0 0 0 0 2 2 2 2 6 BEFORE the seminar Avg 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Could you list the four different risk management alternatives? 71% 4 0 0 2 6 2 0 0 0 0 0 AFTER the seminar 13% 0 0 0 0 0 2 0 2 0 2 8 BEFORE the seminar Avg 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
- 7. Assigning objectives to levels of measurement 1. Satisfaction, reaction, planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: 90% of attendees will indicate an intention to implement new sales strategies within two months after the meeting Answer: Level 1 - Satisfaction & planned actions ( intentions are always level 1) Evaluation Planning
- 8. Assigning objectives to levels of measurement 1. Satisfaction & planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Participants rate the relevance of the meeting to success in their jobs as 4.5 out of 5 Answer: Level 1 - Satisfaction & planned actions Evaluation Planning
- 9. Assigning objectives to levels of measurement Objective: Participants s core 75 out of 100 or better on the new strategy quiz Answer: Level 2 - Learning Evaluation Planning 1. Satisfaction & planned action 2. Learning 3. Application 4. Impact 5. ROI
- 10. Assigning objectives to levels of measurement 1. Satisfaction & planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Participants will be using the new customer interaction skills in 90% of situations where they are needed. Answer: Level 3 - Application Evaluation Planning
- 11. Assigning objectives to levels of measurement 1. Satisfaction & planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Decrease the amount of time required to complete projects by 5% within 6 months of the meeting Answer: Level 4 - Impact ( time may be converted to money ) Evaluation Planning
- 12. Assigning objectives to levels of measurement Objective: Participants r ate the meeting as a good investment for the company with an average of 4.3 out of 5. Answer: Level 1 - Satisfaction & planned actions Evaluation Planning 1. Satisfaction & planned action 2. Learning 3. Application 4. Impact 5. ROI
- 13. Assigning objectives to levels of measurement Objective: Increase sales from existing customers by 5% within 9 months of the meeting Answer: Level 4 - Impact Evaluation Planning 1. Satisfaction & planned action 2. Learning 3. Application 4. Impact 5. ROI
- 14. Assigning objectives to levels of measurement Objective: Achieve a 25% return on investment within the 12 months of the meeting Answer: Level 5 - ROI Evaluation Planning 1. Satisfaction & planned action 2. Learning 3. Application 4. Impact 5. ROI
- 15. Assigning objectives to levels of measurement Objective: Successfully demonstrate new selling techniques during classroom role play Answer: Level 2 - Learning Evaluation Planning 1. Satisfaction &, planned action 2. Learning 3. Application 4. Impact 5. ROI
- 18. Participants’ estimates of factors influencing sales increase following training seminar Data Analysis ROI estimated to be120% (manufacturer) 320% (UK import agent) Elements influencing sales increase: Sales training seminar Advertizing campaign Incentive programme Product improvements Other factors Total Confidence of estimate Adjusted influence factor 65% (i.e. 6 – 12%) 6% Influence factor 9% 16% 7% 49% 19% 100%
- 22. Some useful websites: www.eventroi.org European Event ROI Institute www.roiinstitute.net ROI Institute, Inc. Books: “ Proving the Value of Meetings & Events” Jack J. Phillips, Monica Myhill and James B. McDonough, available from www.mpiweb.org ” Measuring the Return on Investment in Meetings and Events” Jack. J. Phillips and Terri Brining, available from major booksellers and www.amazon.com “ Return on Investment in Training and Performance Improvement Programs” Jack J. Phillips, available from major booksellers and www.amazon.com ” Show me the Money” Jack J. Phillips, available from major booksellers and www.amazon.com Books and Resources
- 25. Contact Details Dr. Elling Hamso Managing Partner E-mail: elling.hamso@eventroi.org Telephone: +47 90 12 24 18
Editor's Notes
- Key Concept: Clear, detailed definitions of each level of data collection. This is an animated slide; bring in each level separately as you discuss. Ask participants to identify each level as a review activity before you show it. We will collect these six types of data as part of the data collection phase. Let’s not forget the intangibles. The intangibles are those measures that we purposely choose not to convert to monetary value. If we can credibly convert data to a monetary value, we’ll do it. But if we cannot, we leave it as an intangible because it’s still very important. Many of your executives buy into those intangibles. They appreciate those intangibles. And for the most part, executives have funded our meetings in the past on those intangibles. Notice the last line. If you’re taking your evaluation to the ROI level, you MUST include some technique to isolate the effects of your meeting from other influences. Let’s look at the Evaluation Levels. What’s our first level of measurement? Reaction and satisfaction with the meetings. We ask attendees for their reaction. We ask them if and what they plan to do differently as a result of the meeting. This is planned action. “Smile sheets” or on-site evaluation surveys are an example of this level of measurement. What’s our next level of measurement? 2. Learning. What did you learn? What are your takeaways? Did you change your perception or attitude about something? Tests or exams are an example of this level of measurement. So is an electronic audience-response system. What’s our next level of measurement? Application and implementation …what did we do differently as a result of the meeting? What new skills, new tasks, new procedures, new policies have I implemented? What have I done differently? Performance records, interviews with attendees several weeks after an event are ways to measure this. What’s our next level of measurement? What are the consequences of doing something different? That consequence is your business measure . That could be productivity, quality improvement, sample time improvement, employee retention, accidents, attitudes, customer service, sales, customer satisfaction. What’s our next level of measurement? ROI , where we compare the monetary value of the consequence. The monetary benefits of the meeting measured against the total costs of the meeting. Here’s where we need to think of issues such as isolating the effects of our meeting and converting the data to monetary value. BA: Explain Sample based on current presentation Level 1 (Reaction, Satisfaction and Planned Action) is the usual smilesheet -> What did you LIKE / What do you PLAN to do? Level 2 (Learning) is e.g. if you will remember this 5 levels at the end of the presentation Level 3 (Application) is if in your next survey, you apply these 5 levels and calculate the ROI according to the methodology Level 4 (Impact) is if you get new clients, increased budget or a new carreer opportunity based on this knowledge Level 5 (ROI) will be the benefits of above impact calculated against the cost you had to come here…
- Key Concept: Chain of impact means that each level builds on the next level – you can’t have a higher level of evaluation without the levels below it. Instructor Notes: We want to think through this chain of impact . We want participants to react favorably to our meeting. We do that now. We want participants to learn something. What new skills? What new knowledge? We want them to apply the skills or the knowledge. We want them to do something differently. Even if it’s conducting a meeting with their staff or implementing a new procedure or task or process. We want them to do something. And there’s a consequence of that for our business. And that consequence, when converted to monetary value compared to the cost, results in ROI, whether it’s negative or positive.  Chain of impact. To do a higher level, you MUST do the lower levels. React, learn, apply, have an impact, have an ROI. It starts with the objectives around each of these.  Let’s review the five levels. Activity: Ask participants to pair off with a partner to complete the “Matching Evaluation Levels with Objectives” activity. 7 minutes. Walk around to see what help they need. Review the answers together.
- Key Concept: Chain of impact means that each level builds on the next level – you can’t have a higher level of evaluation without the levels below it. Instructor Notes: We want to think through this chain of impact . We want participants to react favorably to our meeting. We do that now. We want participants to learn something. What new skills? What new knowledge? We want them to apply the skills or the knowledge. We want them to do something differently. Even if it’s conducting a meeting with their staff or implementing a new procedure or task or process. We want them to do something. And there’s a consequence of that for our business. And that consequence, when converted to monetary value compared to the cost, results in ROI, whether it’s negative or positive.  Chain of impact. To do a higher level, you MUST do the lower levels. React, learn, apply, have an impact, have an ROI. It starts with the objectives around each of these.  Let’s review the five levels. Activity: Ask participants to pair off with a partner to complete the “Matching Evaluation Levels with Objectives” activity. 7 minutes. Walk around to see what help they need. Review the answers together.
- Key Concept: Chain of impact means that each level builds on the next level – you can’t have a higher level of evaluation without the levels below it. Instructor Notes: We want to think through this chain of impact . We want participants to react favorably to our meeting. We do that now. We want participants to learn something. What new skills? What new knowledge? We want them to apply the skills or the knowledge. We want them to do something differently. Even if it’s conducting a meeting with their staff or implementing a new procedure or task or process. We want them to do something. And there’s a consequence of that for our business. And that consequence, when converted to monetary value compared to the cost, results in ROI, whether it’s negative or positive.  Chain of impact. To do a higher level, you MUST do the lower levels. React, learn, apply, have an impact, have an ROI. It starts with the objectives around each of these.  Let’s review the five levels. Activity: Ask participants to pair off with a partner to complete the “Matching Evaluation Levels with Objectives” activity. 7 minutes. Walk around to see what help they need. Review the answers together.
- Key Concept: Chain of impact means that each level builds on the next level – you can’t have a higher level of evaluation without the levels below it. Instructor Notes: We want to think through this chain of impact . We want participants to react favorably to our meeting. We do that now. We want participants to learn something. What new skills? What new knowledge? We want them to apply the skills or the knowledge. We want them to do something differently. Even if it’s conducting a meeting with their staff or implementing a new procedure or task or process. We want them to do something. And there’s a consequence of that for our business. And that consequence, when converted to monetary value compared to the cost, results in ROI, whether it’s negative or positive.  Chain of impact. To do a higher level, you MUST do the lower levels. React, learn, apply, have an impact, have an ROI. It starts with the objectives around each of these.  Let’s review the five levels. Activity: Ask participants to pair off with a partner to complete the “Matching Evaluation Levels with Objectives” activity. 7 minutes. Walk around to see what help they need. Review the answers together.
- Key Concept: Chain of impact means that each level builds on the next level – you can’t have a higher level of evaluation without the levels below it. Instructor Notes: We want to think through this chain of impact . We want participants to react favorably to our meeting. We do that now. We want participants to learn something. What new skills? What new knowledge? We want them to apply the skills or the knowledge. We want them to do something differently. Even if it’s conducting a meeting with their staff or implementing a new procedure or task or process. We want them to do something. And there’s a consequence of that for our business. And that consequence, when converted to monetary value compared to the cost, results in ROI, whether it’s negative or positive.  Chain of impact. To do a higher level, you MUST do the lower levels. React, learn, apply, have an impact, have an ROI. It starts with the objectives around each of these.  Let’s review the five levels. Activity: Ask participants to pair off with a partner to complete the “Matching Evaluation Levels with Objectives” activity. 7 minutes. Walk around to see what help they need. Review the answers together.
- Key Concept: Chain of impact means that each level builds on the next level – you can’t have a higher level of evaluation without the levels below it. Instructor Notes: We want to think through this chain of impact . We want participants to react favorably to our meeting. We do that now. We want participants to learn something. What new skills? What new knowledge? We want them to apply the skills or the knowledge. We want them to do something differently. Even if it’s conducting a meeting with their staff or implementing a new procedure or task or process. We want them to do something. And there’s a consequence of that for our business. And that consequence, when converted to monetary value compared to the cost, results in ROI, whether it’s negative or positive.  Chain of impact. To do a higher level, you MUST do the lower levels. React, learn, apply, have an impact, have an ROI. It starts with the objectives around each of these.  Let’s review the five levels. Activity: Ask participants to pair off with a partner to complete the “Matching Evaluation Levels with Objectives” activity. 7 minutes. Walk around to see what help they need. Review the answers together.
- Key Concept: Chain of impact means that each level builds on the next level – you can’t have a higher level of evaluation without the levels below it. Instructor Notes: We want to think through this chain of impact . We want participants to react favorably to our meeting. We do that now. We want participants to learn something. What new skills? What new knowledge? We want them to apply the skills or the knowledge. We want them to do something differently. Even if it’s conducting a meeting with their staff or implementing a new procedure or task or process. We want them to do something. And there’s a consequence of that for our business. And that consequence, when converted to monetary value compared to the cost, results in ROI, whether it’s negative or positive.  Chain of impact. To do a higher level, you MUST do the lower levels. React, learn, apply, have an impact, have an ROI. It starts with the objectives around each of these.  Let’s review the five levels. Activity: Ask participants to pair off with a partner to complete the “Matching Evaluation Levels with Objectives” activity. 7 minutes. Walk around to see what help they need. Review the answers together.
- Key Concept: Chain of impact means that each level builds on the next level – you can’t have a higher level of evaluation without the levels below it. Instructor Notes: We want to think through this chain of impact . We want participants to react favorably to our meeting. We do that now. We want participants to learn something. What new skills? What new knowledge? We want them to apply the skills or the knowledge. We want them to do something differently. Even if it’s conducting a meeting with their staff or implementing a new procedure or task or process. We want them to do something. And there’s a consequence of that for our business. And that consequence, when converted to monetary value compared to the cost, results in ROI, whether it’s negative or positive.  Chain of impact. To do a higher level, you MUST do the lower levels. React, learn, apply, have an impact, have an ROI. It starts with the objectives around each of these.  Let’s review the five levels. Activity: Ask participants to pair off with a partner to complete the “Matching Evaluation Levels with Objectives” activity. 7 minutes. Walk around to see what help they need. Review the answers together.
- Key Concept: Chain of impact means that each level builds on the next level – you can’t have a higher level of evaluation without the levels below it. Instructor Notes: We want to think through this chain of impact . We want participants to react favorably to our meeting. We do that now. We want participants to learn something. What new skills? What new knowledge? We want them to apply the skills or the knowledge. We want them to do something differently. Even if it’s conducting a meeting with their staff or implementing a new procedure or task or process. We want them to do something. And there’s a consequence of that for our business. And that consequence, when converted to monetary value compared to the cost, results in ROI, whether it’s negative or positive.  Chain of impact. To do a higher level, you MUST do the lower levels. React, learn, apply, have an impact, have an ROI. It starts with the objectives around each of these.  Let’s review the five levels. Activity: Ask participants to pair off with a partner to complete the “Matching Evaluation Levels with Objectives” activity. 7 minutes. Walk around to see what help they need. Review the answers together.
- Key Concept: It is possible to isolate the effects of a meeting. Instructor Notes: If you recall your learner outcomes, performing an isolation isn’t one of them – this is a fairly involved process. However, it is important to recognize that it can be done. Control Groups: You could have one group that attends a meeting and one that doesn’t, and measure the changed behavior of each against the other. Trend Analysis: You could measure your meeting attendees’ performance against industry trends. Forecasting Methods: You could use past data to forecast the change you would expect to see from a meeting. End User/Performance Estimates: You can ask attendees to tell you the level of change in their behavior they attribute to the meeting, and their confidence in this estimate. Supervisor Estimate: You can ask supervisors to tell you the level of change in attendee behavior they attribute to the meeting, and their confidence in this estimate. Outside Experts: Can provide you with an estimate of the change caused soley by the meeting. Let’s use an example of sales training. Participants have gone through a sales training program that increases their prospecting skills. How would each of the following be impacted and measured? They are listed in the order of their credibility. Converting output to contribution: What percentage of post-training output over a set amount of time (eg: 3 months) is directly attributable to the prospecting skills learned in the training program? Converting employee’s time: How much sales time has been “saved” because of these new skills which can then be converted to a monetary amount based on the employees wages and benefits? Using historical costs: What is the average cost to produce a sale? This information is developed from cost statements within the organization. The next 4 are based on values assigned by “experts”. This, of course, is based on the credibility of the “experts”. Using internal and external experts:. That said, the internal Directors of Sales and/or Marketing can estimate the value of the newly developed prospecting skills. The trainer who delivered the training can be used as an external expert. Using participants’ estimates: What monetary does the participant place on their newly developed skills in prospecting? Using supervisors’ and managers’ estimates: Based on the supervisors’ and managers’ expertise, what value do they place on these skills? Using staff estimates: Survey the staff to determine a value on the skills learned in the training.
- Key Concept: It is possible to isolate the effects of a meeting. Instructor Notes: If you recall your learner outcomes, performing an isolation isn’t one of them – this is a fairly involved process. However, it is important to recognize that it can be done. Control Groups: You could have one group that attends a meeting and one that doesn’t, and measure the changed behavior of each against the other. Trend Analysis: You could measure your meeting attendees’ performance against industry trends. Forecasting Methods: You could use past data to forecast the change you would expect to see from a meeting. End User/Performance Estimates: You can ask attendees to tell you the level of change in their behavior they attribute to the meeting, and their confidence in this estimate. Supervisor Estimate: You can ask supervisors to tell you the level of change in attendee behavior they attribute to the meeting, and their confidence in this estimate. Outside Experts: Can provide you with an estimate of the change caused soley by the meeting. Let’s use an example of sales training. Participants have gone through a sales training program that increases their prospecting skills. How would each of the following be impacted and measured? They are listed in the order of their credibility. Converting output to contribution: What percentage of post-training output over a set amount of time (eg: 3 months) is directly attributable to the prospecting skills learned in the training program? Converting employee’s time: How much sales time has been “saved” because of these new skills which can then be converted to a monetary amount based on the employees wages and benefits? Using historical costs: What is the average cost to produce a sale? This information is developed from cost statements within the organization. The next 4 are based on values assigned by “experts”. This, of course, is based on the credibility of the “experts”. Using internal and external experts:. That said, the internal Directors of Sales and/or Marketing can estimate the value of the newly developed prospecting skills. The trainer who delivered the training can be used as an external expert. Using participants’ estimates: What monetary does the participant place on their newly developed skills in prospecting? Using supervisors’ and managers’ estimates: Based on the supervisors’ and managers’ expertise, what value do they place on these skills? Using staff estimates: Survey the staff to determine a value on the skills learned in the training.
- Key Concept: It is possible to isolate the effects of a meeting. Instructor Notes: If you recall your learner outcomes, performing an isolation isn’t one of them – this is a fairly involved process. However, it is important to recognize that it can be done. Control Groups: You could have one group that attends a meeting and one that doesn’t, and measure the changed behavior of each against the other. Trend Analysis: You could measure your meeting attendees’ performance against industry trends. Forecasting Methods: You could use past data to forecast the change you would expect to see from a meeting. End User/Performance Estimates: You can ask attendees to tell you the level of change in their behavior they attribute to the meeting, and their confidence in this estimate. Supervisor Estimate: You can ask supervisors to tell you the level of change in attendee behavior they attribute to the meeting, and their confidence in this estimate. Outside Experts: Can provide you with an estimate of the change caused soley by the meeting. Let’s use an example of sales training. Participants have gone through a sales training program that increases their prospecting skills. How would each of the following be impacted and measured? They are listed in the order of their credibility. Converting output to contribution: What percentage of post-training output over a set amount of time (eg: 3 months) is directly attributable to the prospecting skills learned in the training program? Converting employee’s time: How much sales time has been “saved” because of these new skills which can then be converted to a monetary amount based on the employees wages and benefits? Using historical costs: What is the average cost to produce a sale? This information is developed from cost statements within the organization. The next 4 are based on values assigned by “experts”. This, of course, is based on the credibility of the “experts”. Using internal and external experts:. That said, the internal Directors of Sales and/or Marketing can estimate the value of the newly developed prospecting skills. The trainer who delivered the training can be used as an external expert. Using participants’ estimates: What monetary does the participant place on their newly developed skills in prospecting? Using supervisors’ and managers’ estimates: Based on the supervisors’ and managers’ expertise, what value do they place on these skills? Using staff estimates: Survey the staff to determine a value on the skills learned in the training.