• Like
  • Save
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs

  • 832 views
Published

Presented by: Ryan Lanyon, BA …

Presented by: Ryan Lanyon, BA
Presented at: ACT Canada 2008 TDM Summit, Halifax, October 2008

Published in Automotive
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
832
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
2

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Smart Commute Evaluation Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
  • 2. Outline
    • Background
    • Approach
    • Monitoring Tools
    • Evaluation
    • Results
    • Challenges
    • Lessons
    • Next Steps
    Ryan Lanyon Team Lead, Smart Commute Metrolinx (GTTA) October 22, 2008
  • 3. At-a-Glance
    • Build evaluation into your initial planning
    • Monitor activities, benefits and customer service
    • Scrutinize and understand indicators
    • Learn from results and mistakes
    • Resources available at www.smartcommute.ca
    • Just do it!
    Smart Commute works with various stakeholders to relieve traffic congestion, improve air quality and health, and reduce emissions that cause climate change.
  • 4. Background
    • Smart Commute
      • Focus on commuters and workplaces
      • Partnership with municipalities
      • Staged development
        • 2001: Pilot TMA - BCRTMA
        • 2004: Pilot regional implementation - Municipalities
        • 2008: On-going implementation – Metrolinx + Municipalities
  • 5. Background
    • Metrolinx (Greater Toronto Transportation Authority)
      • Formed in 2007
      • Coordinates transportation across GTHA
      • Regional Transportation Plan
      • 6.13 million residents, 3 million commuters
  • 6. Background
    • Two-tiered program delivery
      • Metrolinx
        • Central coordinating body
        • Centralized service operation e.g. Carpool Zone
        • TMA Toolkit
        • Funding agency – 50%
      • Local Smart Commute / TMA
        • Delivery agent
        • Municipality, NGO, Chamber of Commerce/Board of Trade
        • Find other funding – municipalities, grants, service fees
  • 7. Background
    • 2001: BCRTMA / NTV
    • 2004: 404-7
    • 2005:
      • Mississauga
      • NE Toronto
    • 2006:
      • Brampton-Caledon
      • Halton
    • 2007:
      • Central York
      • Durham
    • 2008: Toronto-Central
    • 2009: Airport?
  • 8. Background
    • Employer program
      • Improve and encourage commuting options
      • Assessment of current conditions - Baseline
      • Development of a plan
      • Implementation – ‘Commuter benefits program’
        • Carpool Zone and ERH
        • Walking, cycling, transit, carpool, telework programs
        • Campaigns, contests and special events
      • Follow-up assessment – Measured against baseline
  • 9. Approach - Ideal Goals Objectives Regional Transportation Plan Municipal Plans Implementation Strategic Plan Monitoring Evaluation
  • 10. Approach - Actual Goals Objectives RTP Municipal Plans Implementation Strategic Plan Monitoring Evaluation Federal Funding Requirements Implementation Implementation
  • 11. Approach
    • 2004-2007 - Contribution Agreement
      • Negotiated between funder and municipalities
    • 2008 - Monitoring Framework
      • Established by Smart Commute stakeholders
        • TMAs and Smart Commute Association (implementers)
        • Municipalities and Metrolinx (funders)
      • Multipurpose
        • Measure activity levels for funders
        • Track impacts to justify funding
        • Collect statistics for marketing purposes
  • 12. Approach
    • Inputs / Activities
      • Number of businesses
        • Active
        • Engaged
        • Supporting
      • Number of pamphlets distributed
      • Number of cycling programs
      • Number of media releases
      • Number of commuters reached
  • 13. Approach
    • Outputs / Benefits / Results
      • Brand awareness
        • % of employees aware; change from baseline
      • Commuter satisfaction
        • Ratings of Smart Commute program
      • Reductions
        • GHG emissions
        • CAC emissions
        • Vkt and trips
      • Commuter cost savings
  • 14. Monitoring
    • Levels of monitoring
      • Regional
      • TMA
      • Employer
      • Commuter / customer
    • Types of monitoring
      • Behaviour change
      • Attitudinal change
      • Customer service
  • 15. Monitoring
    • Regional
      • Existing sources
        • Statistics Canada
        • Transportation Tomorrow Survey (TTS)
          • Conducted by University of Toronto
          • Detailed behaviour survey
          • Conducted every five years
        • Cordon Count Program
          • Conducted by municipalities, compiled by University of Toronto
          • Observed data
          • Conducted every two to three years
  • 16. Monitoring
    • Regional
      • New sources
        • Commuter Attitudes Survey
          • Conducted by Metrolinx
          • Detailed attitudinal questions
          • Conducted every two years
        • Carpool Zone
          • Conducted by Metrolinx
          • Monthly activity survey
        • TMAs and Employers
          • Compiled data from localized sources
  • 17. Monitoring
    • TMA
      • Quarterly reports – tied to Metrolinx funding
      • Summary of activities per employers
      • Communications and outreach activity
      • Qualitative results
      • Comparison of activity levels between TMAs
  • 18. Monitoring
    • Employer – Baseline Module
      • Baseline employee survey
        • Standardized across all TMAs and employers
        • Census survey
      • Site assessment
        • Current conditions
        • Inventory of advantages and deficiencies
      • Vehicle and occupancy count
        • Discreet sites
        • 1,000+ employees
  • 19. Monitoring
    • Customer – Commuter
      • Service-related
        • Monthly Carpool Zone microsurvey
        • Emergency Ride Home post-ride survey
      • Employee surveys
      • Campaign-related data collection e.g. Car Free Day
  • 20. Monitoring 10% 28% 19.7% 19.93% Transit 75.91% 48% 64.7% 65.90% Drive Alone 0.7% TTS (2006) 0.89% 1% 0.97% Cycling Surveys (2001-08) CAS (2008) Statistics Canada (2006) Mode
  • 21. Evaluation
    • Benchmarks and Trends
      • 75 employee baseline surveys
      • Statistics Canada
      • Transportation Tomorrow Survey
      • Cordon Counts
      • Commuter Attitudes Survey
  • 22. Evaluation
    • External
      • Employer surveys
        • Biannual survey – program satisfaction rating
        • Satisfaction with Smart Commute services
        • Conducted by third party through Metrolinx
      • Commuter surveys
        • Employee follow-up surveys – program satisfaction rating
        • Annual survey – Carpool Zone satisfaction
    • Internal
      • Smart Commute Technical Committee
  • 23. Evaluation
    • External Review
      • Consultant evaluation of operational models
    • Research Partnerships
      • University of Toronto (Mississauga)
        • Factors for successful carpool formation
        • Further research
      • Ryerson University
        • TMA Toolkit assessment
  • 24. Results
    • Almost 100 members and partners
      • March 2007 – 50 employers
      • March 2008 – 75 employers
      • June 2008 – 89 employers
  • 25. Results
    • Transportation indicators
      • 76.4 million vkt
      • 1.3 million trips
      • Equivalent of 10,000 cars
      • Increase in carpooling
        • 7% to 12% to 13%
    Cars Off the Road
  • 26. Results
    • Primary Mode Shift
    2006 2005 2008 Bike Walk/Jog Car/vanpool Public Transit Drive alone
  • 27. Results
    • Environmental indicators
      • 17,500 tonnes of greenhouse gas emissions
        • Enough to fill Rogers Centre almost six times
      • 100,000 kg of criteria air contaminants
        • Equivalent amount of NOx to smoking 3 billion cigarettes
  • 28. Results
    • Communications and Outreach
      • 400+ stories
      • 75+ million media impressions
      • Brand recognition
        • Smart Commute ↑
        • Carpool Zone ↓
    Awareness (2008) Clean Air Commute Commuter Challenge Carpool Zone Website Carpool Week Smart Commute
  • 29. Challenges
    • Pressures to implement right away
      • Show action
      • Pent-up demand
      • More interesting than planning or evaluating
    • All the pieces not always in place
    • Monitoring not appreciated until results urgently needed
    • Objectivity
      • Of course my ‘baby’ was successful!
  • 30. Challenges
    • Measurement and Evaluation
      • Employer buy-in is difficult
        • Some see value in surveys, monitoring
        • Others feel employees are oversurveyed
        • Often considered an expensive waste of employee time
      • Standardization required, but unenforceable
        • Moving to centralized service provision
        • Standardization vs. continuous improvement
      • Data comparison can be unequal
        • Apples to apples and apples to oranges
  • 31. Challenges
    • Defining success
      • Setting targets without enough information
        • Are examples applicable?
      • Are targets achievable with the resources alloted?
      • Can these targets be effectively monitored?
      • What happens if we don’t reach these targets?
  • 32. Lessons
    • Measurement and Evaluation
      • Some items simple, but time-consuming
        • Do you really need the data?
      • Project benefits less easy to isolate
        • Double-counting also possible through employee surveys and Carpool Zone surveys
        • External factors – environmental concerns, gas prices
      • One clear conclusion: an impact was made.
      • Time and consistency open up opportunities
        • Benchmarking
  • 33. Lessons
    • Measure and Evaluation
    • Access resources at your disposal for help
      • Universities
      • Municipalities
      • Provincial and Federal Governments
    • Evaluate what you do and how you do it
    • Give an adjustment period to monitoring
      • Test! Test! Test!
      • Allow partners time to understand and see value
  • 34. Lessons
    • Provide incentives for monitoring
      • Tie completion to access
      • Award and reward success
    • Measurement and Evaluation
      • Learn from mistakes – you will make them!
      • Learn from poor results
        • Was the program effective?
        • Was the monitoring effective?
  • 35. Next Steps
    • Continue and improve data collection
      • Address data management and manipulation
    • Benchmarks and comparisons
    • Strategic planning
      • Service evaluation
      • Targets
    • Baseline module refinement
  • 36. Questions? Ryan Lanyon Team Lead, Smart Commute Metrolinx [email_address] 416-874-5933