Your SlideShare is downloading. ×
0
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs

867

Published on

Presented by: Ryan Lanyon, BA …

Presented by: Ryan Lanyon, BA
Presented at: ACT Canada 2008 TDM Summit, Halifax, October 2008

Published in: Automotive
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
867
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  1. Smart Commute Evaluation Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
  2. Outline <ul><li>Background </li></ul><ul><li>Approach </li></ul><ul><li>Monitoring Tools </li></ul><ul><li>Evaluation </li></ul><ul><li>Results </li></ul><ul><li>Challenges </li></ul><ul><li>Lessons </li></ul><ul><li>Next Steps </li></ul>Ryan Lanyon Team Lead, Smart Commute Metrolinx (GTTA) October 22, 2008
  3. At-a-Glance <ul><li>Build evaluation into your initial planning </li></ul><ul><li>Monitor activities, benefits and customer service </li></ul><ul><li>Scrutinize and understand indicators </li></ul><ul><li>Learn from results and mistakes </li></ul><ul><li>Resources available at www.smartcommute.ca </li></ul><ul><li>Just do it! </li></ul>Smart Commute works with various stakeholders to relieve traffic congestion, improve air quality and health, and reduce emissions that cause climate change.
  4. Background <ul><li>Smart Commute </li></ul><ul><ul><li>Focus on commuters and workplaces </li></ul></ul><ul><ul><li>Partnership with municipalities </li></ul></ul><ul><ul><li>Staged development </li></ul></ul><ul><ul><ul><li>2001: Pilot TMA - BCRTMA </li></ul></ul></ul><ul><ul><ul><li>2004: Pilot regional implementation - Municipalities </li></ul></ul></ul><ul><ul><ul><li>2008: On-going implementation – Metrolinx + Municipalities </li></ul></ul></ul>
  5. Background <ul><li>Metrolinx (Greater Toronto Transportation Authority) </li></ul><ul><ul><li>Formed in 2007 </li></ul></ul><ul><ul><li>Coordinates transportation across GTHA </li></ul></ul><ul><ul><li>Regional Transportation Plan </li></ul></ul><ul><ul><li>6.13 million residents, 3 million commuters </li></ul></ul>
  6. Background <ul><li>Two-tiered program delivery </li></ul><ul><ul><li>Metrolinx </li></ul></ul><ul><ul><ul><li>Central coordinating body </li></ul></ul></ul><ul><ul><ul><li>Centralized service operation e.g. Carpool Zone </li></ul></ul></ul><ul><ul><ul><li>TMA Toolkit </li></ul></ul></ul><ul><ul><ul><li>Funding agency – 50% </li></ul></ul></ul><ul><ul><li>Local Smart Commute / TMA </li></ul></ul><ul><ul><ul><li>Delivery agent </li></ul></ul></ul><ul><ul><ul><li>Municipality, NGO, Chamber of Commerce/Board of Trade </li></ul></ul></ul><ul><ul><ul><li>Find other funding – municipalities, grants, service fees </li></ul></ul></ul>
  7. Background <ul><li>2001: BCRTMA / NTV </li></ul><ul><li>2004: 404-7 </li></ul><ul><li>2005: </li></ul><ul><ul><li>Mississauga </li></ul></ul><ul><ul><li>NE Toronto </li></ul></ul><ul><li>2006: </li></ul><ul><ul><li>Brampton-Caledon </li></ul></ul><ul><ul><li>Halton </li></ul></ul><ul><li>2007: </li></ul><ul><ul><li>Central York </li></ul></ul><ul><ul><li>Durham </li></ul></ul><ul><li>2008: Toronto-Central </li></ul><ul><li>2009: Airport? </li></ul>
  8. Background <ul><li>Employer program </li></ul><ul><ul><li>Improve and encourage commuting options </li></ul></ul><ul><ul><li>Assessment of current conditions - Baseline </li></ul></ul><ul><ul><li>Development of a plan </li></ul></ul><ul><ul><li>Implementation – ‘Commuter benefits program’ </li></ul></ul><ul><ul><ul><li>Carpool Zone and ERH </li></ul></ul></ul><ul><ul><ul><li>Walking, cycling, transit, carpool, telework programs </li></ul></ul></ul><ul><ul><ul><li>Campaigns, contests and special events </li></ul></ul></ul><ul><ul><li>Follow-up assessment – Measured against baseline </li></ul></ul>
  9. Approach - Ideal Goals Objectives Regional Transportation Plan Municipal Plans Implementation Strategic Plan Monitoring Evaluation
  10. Approach - Actual Goals Objectives RTP Municipal Plans Implementation Strategic Plan Monitoring Evaluation Federal Funding Requirements Implementation Implementation
  11. Approach <ul><li>2004-2007 - Contribution Agreement </li></ul><ul><ul><li>Negotiated between funder and municipalities </li></ul></ul><ul><li>2008 - Monitoring Framework </li></ul><ul><ul><li>Established by Smart Commute stakeholders </li></ul></ul><ul><ul><ul><li>TMAs and Smart Commute Association (implementers) </li></ul></ul></ul><ul><ul><ul><li>Municipalities and Metrolinx (funders) </li></ul></ul></ul><ul><ul><li>Multipurpose </li></ul></ul><ul><ul><ul><li>Measure activity levels for funders </li></ul></ul></ul><ul><ul><ul><li>Track impacts to justify funding </li></ul></ul></ul><ul><ul><ul><li>Collect statistics for marketing purposes </li></ul></ul></ul>
  12. Approach <ul><li>Inputs / Activities </li></ul><ul><ul><li>Number of businesses </li></ul></ul><ul><ul><ul><li>Active </li></ul></ul></ul><ul><ul><ul><li>Engaged </li></ul></ul></ul><ul><ul><ul><li>Supporting </li></ul></ul></ul><ul><ul><li>Number of pamphlets distributed </li></ul></ul><ul><ul><li>Number of cycling programs </li></ul></ul><ul><ul><li>Number of media releases </li></ul></ul><ul><ul><li>Number of commuters reached </li></ul></ul>
  13. Approach <ul><li>Outputs / Benefits / Results </li></ul><ul><ul><li>Brand awareness </li></ul></ul><ul><ul><ul><li>% of employees aware; change from baseline </li></ul></ul></ul><ul><ul><li>Commuter satisfaction </li></ul></ul><ul><ul><ul><li>Ratings of Smart Commute program </li></ul></ul></ul><ul><ul><li>Reductions </li></ul></ul><ul><ul><ul><li>GHG emissions </li></ul></ul></ul><ul><ul><ul><li>CAC emissions </li></ul></ul></ul><ul><ul><ul><li>Vkt and trips </li></ul></ul></ul><ul><ul><li>Commuter cost savings </li></ul></ul>
  14. Monitoring <ul><li>Levels of monitoring </li></ul><ul><ul><li>Regional </li></ul></ul><ul><ul><li>TMA </li></ul></ul><ul><ul><li>Employer </li></ul></ul><ul><ul><li>Commuter / customer </li></ul></ul><ul><li>Types of monitoring </li></ul><ul><ul><li>Behaviour change </li></ul></ul><ul><ul><li>Attitudinal change </li></ul></ul><ul><ul><li>Customer service </li></ul></ul>
  15. Monitoring <ul><li>Regional </li></ul><ul><ul><li>Existing sources </li></ul></ul><ul><ul><ul><li>Statistics Canada </li></ul></ul></ul><ul><ul><ul><li>Transportation Tomorrow Survey (TTS) </li></ul></ul></ul><ul><ul><ul><ul><li>Conducted by University of Toronto </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Detailed behaviour survey </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Conducted every five years </li></ul></ul></ul></ul><ul><ul><ul><li>Cordon Count Program </li></ul></ul></ul><ul><ul><ul><ul><li>Conducted by municipalities, compiled by University of Toronto </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Observed data </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Conducted every two to three years </li></ul></ul></ul></ul>
  16. Monitoring <ul><li>Regional </li></ul><ul><ul><li>New sources </li></ul></ul><ul><ul><ul><li>Commuter Attitudes Survey </li></ul></ul></ul><ul><ul><ul><ul><li>Conducted by Metrolinx </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Detailed attitudinal questions </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Conducted every two years </li></ul></ul></ul></ul><ul><ul><ul><li>Carpool Zone </li></ul></ul></ul><ul><ul><ul><ul><li>Conducted by Metrolinx </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Monthly activity survey </li></ul></ul></ul></ul><ul><ul><ul><li>TMAs and Employers </li></ul></ul></ul><ul><ul><ul><ul><li>Compiled data from localized sources </li></ul></ul></ul></ul>
  17. Monitoring <ul><li>TMA </li></ul><ul><ul><li>Quarterly reports – tied to Metrolinx funding </li></ul></ul><ul><ul><li>Summary of activities per employers </li></ul></ul><ul><ul><li>Communications and outreach activity </li></ul></ul><ul><ul><li>Qualitative results </li></ul></ul><ul><ul><li>Comparison of activity levels between TMAs </li></ul></ul>
  18. Monitoring <ul><li>Employer – Baseline Module </li></ul><ul><ul><li>Baseline employee survey </li></ul></ul><ul><ul><ul><li>Standardized across all TMAs and employers </li></ul></ul></ul><ul><ul><ul><li>Census survey </li></ul></ul></ul><ul><ul><li>Site assessment </li></ul></ul><ul><ul><ul><li>Current conditions </li></ul></ul></ul><ul><ul><ul><li>Inventory of advantages and deficiencies </li></ul></ul></ul><ul><ul><li>Vehicle and occupancy count </li></ul></ul><ul><ul><ul><li>Discreet sites </li></ul></ul></ul><ul><ul><ul><li>1,000+ employees </li></ul></ul></ul>
  19. Monitoring <ul><li>Customer – Commuter </li></ul><ul><ul><li>Service-related </li></ul></ul><ul><ul><ul><li>Monthly Carpool Zone microsurvey </li></ul></ul></ul><ul><ul><ul><li>Emergency Ride Home post-ride survey </li></ul></ul></ul><ul><ul><li>Employee surveys </li></ul></ul><ul><ul><li>Campaign-related data collection e.g. Car Free Day </li></ul></ul>
  20. Monitoring 10% 28% 19.7% 19.93% Transit 75.91% 48% 64.7% 65.90% Drive Alone 0.7% TTS (2006) 0.89% 1% 0.97% Cycling Surveys (2001-08) CAS (2008) Statistics Canada (2006) Mode
  21. Evaluation <ul><li>Benchmarks and Trends </li></ul><ul><ul><li>75 employee baseline surveys </li></ul></ul><ul><ul><li>Statistics Canada </li></ul></ul><ul><ul><li>Transportation Tomorrow Survey </li></ul></ul><ul><ul><li>Cordon Counts </li></ul></ul><ul><ul><li>Commuter Attitudes Survey </li></ul></ul>
  22. Evaluation <ul><li>External </li></ul><ul><ul><li>Employer surveys </li></ul></ul><ul><ul><ul><li>Biannual survey – program satisfaction rating </li></ul></ul></ul><ul><ul><ul><li>Satisfaction with Smart Commute services </li></ul></ul></ul><ul><ul><ul><li>Conducted by third party through Metrolinx </li></ul></ul></ul><ul><ul><li>Commuter surveys </li></ul></ul><ul><ul><ul><li>Employee follow-up surveys – program satisfaction rating </li></ul></ul></ul><ul><ul><ul><li>Annual survey – Carpool Zone satisfaction </li></ul></ul></ul><ul><li>Internal </li></ul><ul><ul><li>Smart Commute Technical Committee </li></ul></ul>
  23. Evaluation <ul><li>External Review </li></ul><ul><ul><li>Consultant evaluation of operational models </li></ul></ul><ul><li>Research Partnerships </li></ul><ul><ul><li>University of Toronto (Mississauga) </li></ul></ul><ul><ul><ul><li>Factors for successful carpool formation </li></ul></ul></ul><ul><ul><ul><li>Further research </li></ul></ul></ul><ul><ul><li>Ryerson University </li></ul></ul><ul><ul><ul><li>TMA Toolkit assessment </li></ul></ul></ul>
  24. Results <ul><li>Almost 100 members and partners </li></ul><ul><ul><li>March 2007 – 50 employers </li></ul></ul><ul><ul><li>March 2008 – 75 employers </li></ul></ul><ul><ul><li>June 2008 – 89 employers </li></ul></ul>
  25. Results <ul><li>Transportation indicators </li></ul><ul><ul><li>76.4 million vkt </li></ul></ul><ul><ul><li>1.3 million trips </li></ul></ul><ul><ul><li>Equivalent of 10,000 cars </li></ul></ul><ul><ul><li>Increase in carpooling </li></ul></ul><ul><ul><ul><li>7% to 12% to 13% </li></ul></ul></ul>Cars Off the Road
  26. Results <ul><li>Primary Mode Shift </li></ul>2006 2005 2008 Bike Walk/Jog Car/vanpool Public Transit Drive alone
  27. Results <ul><li>Environmental indicators </li></ul><ul><ul><li>17,500 tonnes of greenhouse gas emissions </li></ul></ul><ul><ul><ul><li>Enough to fill Rogers Centre almost six times </li></ul></ul></ul><ul><ul><li>100,000 kg of criteria air contaminants </li></ul></ul><ul><ul><ul><li>Equivalent amount of NOx to smoking 3 billion cigarettes </li></ul></ul></ul>
  28. Results <ul><li>Communications and Outreach </li></ul><ul><ul><li>400+ stories </li></ul></ul><ul><ul><li>75+ million media impressions </li></ul></ul><ul><ul><li>Brand recognition </li></ul></ul><ul><ul><ul><li>Smart Commute ↑ </li></ul></ul></ul><ul><ul><ul><li>Carpool Zone ↓ </li></ul></ul></ul>Awareness (2008) Clean Air Commute Commuter Challenge Carpool Zone Website Carpool Week Smart Commute
  29. Challenges <ul><li>Pressures to implement right away </li></ul><ul><ul><li>Show action </li></ul></ul><ul><ul><li>Pent-up demand </li></ul></ul><ul><ul><li>More interesting than planning or evaluating </li></ul></ul><ul><li>All the pieces not always in place </li></ul><ul><li>Monitoring not appreciated until results urgently needed </li></ul><ul><li>Objectivity </li></ul><ul><ul><li>Of course my ‘baby’ was successful! </li></ul></ul>
  30. Challenges <ul><li>Measurement and Evaluation </li></ul><ul><ul><li>Employer buy-in is difficult </li></ul></ul><ul><ul><ul><li>Some see value in surveys, monitoring </li></ul></ul></ul><ul><ul><ul><li>Others feel employees are oversurveyed </li></ul></ul></ul><ul><ul><ul><li>Often considered an expensive waste of employee time </li></ul></ul></ul><ul><ul><li>Standardization required, but unenforceable </li></ul></ul><ul><ul><ul><li>Moving to centralized service provision </li></ul></ul></ul><ul><ul><ul><li>Standardization vs. continuous improvement </li></ul></ul></ul><ul><ul><li>Data comparison can be unequal </li></ul></ul><ul><ul><ul><li>Apples to apples and apples to oranges </li></ul></ul></ul>
  31. Challenges <ul><li>Defining success </li></ul><ul><ul><li>Setting targets without enough information </li></ul></ul><ul><ul><ul><li>Are examples applicable? </li></ul></ul></ul><ul><ul><li>Are targets achievable with the resources alloted? </li></ul></ul><ul><ul><li>Can these targets be effectively monitored? </li></ul></ul><ul><ul><li>What happens if we don’t reach these targets? </li></ul></ul>
  32. Lessons <ul><li>Measurement and Evaluation </li></ul><ul><ul><li>Some items simple, but time-consuming </li></ul></ul><ul><ul><ul><li>Do you really need the data? </li></ul></ul></ul><ul><ul><li>Project benefits less easy to isolate </li></ul></ul><ul><ul><ul><li>Double-counting also possible through employee surveys and Carpool Zone surveys </li></ul></ul></ul><ul><ul><ul><li>External factors – environmental concerns, gas prices </li></ul></ul></ul><ul><ul><li>One clear conclusion: an impact was made. </li></ul></ul><ul><ul><li>Time and consistency open up opportunities </li></ul></ul><ul><ul><ul><li>Benchmarking </li></ul></ul></ul>
  33. Lessons <ul><li>Measure and Evaluation </li></ul><ul><li>Access resources at your disposal for help </li></ul><ul><ul><li>Universities </li></ul></ul><ul><ul><li>Municipalities </li></ul></ul><ul><ul><li>Provincial and Federal Governments </li></ul></ul><ul><li>Evaluate what you do and how you do it </li></ul><ul><li>Give an adjustment period to monitoring </li></ul><ul><ul><li>Test! Test! Test! </li></ul></ul><ul><ul><li>Allow partners time to understand and see value </li></ul></ul>
  34. Lessons <ul><li>Provide incentives for monitoring </li></ul><ul><ul><li>Tie completion to access </li></ul></ul><ul><ul><li>Award and reward success </li></ul></ul><ul><li>Measurement and Evaluation </li></ul><ul><ul><li>Learn from mistakes – you will make them! </li></ul></ul><ul><ul><li>Learn from poor results </li></ul></ul><ul><ul><ul><li>Was the program effective? </li></ul></ul></ul><ul><ul><ul><li>Was the monitoring effective? </li></ul></ul></ul>
  35. Next Steps <ul><li>Continue and improve data collection </li></ul><ul><ul><li>Address data management and manipulation </li></ul></ul><ul><li>Benchmarks and comparisons </li></ul><ul><li>Strategic planning </li></ul><ul><ul><li>Service evaluation </li></ul></ul><ul><ul><li>Targets </li></ul></ul><ul><li>Baseline module refinement </li></ul>
  36. Questions? Ryan Lanyon Team Lead, Smart Commute Metrolinx [email_address] 416-874-5933

×