• Save
Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
Upcoming SlideShare
Loading in...5
×
 

Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs

on

  • 1,201 views

Presented by: Ryan Lanyon, BA

Presented by: Ryan Lanyon, BA
Presented at: ACT Canada 2008 TDM Summit, Halifax, October 2008

Statistics

Views

Total Views
1,201
Views on SlideShare
1,118
Embed Views
83

Actions

Likes
2
Downloads
0
Comments
0

4 Embeds 83

http://www.smartcommute.ca 50
http://www.plantogether.org 25
http://plantogether.org 5
http://www.slideshare.net 3

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs Smart Commute Evaluation: Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs Presentation Transcript

    • Smart Commute Evaluation Tools, Techniques and Lessons Learned in Monitoring and Evaluating Workplace-based TDM Programs
    • Outline
      • Background
      • Approach
      • Monitoring Tools
      • Evaluation
      • Results
      • Challenges
      • Lessons
      • Next Steps
      Ryan Lanyon Team Lead, Smart Commute Metrolinx (GTTA) October 22, 2008
    • At-a-Glance
      • Build evaluation into your initial planning
      • Monitor activities, benefits and customer service
      • Scrutinize and understand indicators
      • Learn from results and mistakes
      • Resources available at www.smartcommute.ca
      • Just do it!
      Smart Commute works with various stakeholders to relieve traffic congestion, improve air quality and health, and reduce emissions that cause climate change.
    • Background
      • Smart Commute
        • Focus on commuters and workplaces
        • Partnership with municipalities
        • Staged development
          • 2001: Pilot TMA - BCRTMA
          • 2004: Pilot regional implementation - Municipalities
          • 2008: On-going implementation – Metrolinx + Municipalities
    • Background
      • Metrolinx (Greater Toronto Transportation Authority)
        • Formed in 2007
        • Coordinates transportation across GTHA
        • Regional Transportation Plan
        • 6.13 million residents, 3 million commuters
    • Background
      • Two-tiered program delivery
        • Metrolinx
          • Central coordinating body
          • Centralized service operation e.g. Carpool Zone
          • TMA Toolkit
          • Funding agency – 50%
        • Local Smart Commute / TMA
          • Delivery agent
          • Municipality, NGO, Chamber of Commerce/Board of Trade
          • Find other funding – municipalities, grants, service fees
    • Background
      • 2001: BCRTMA / NTV
      • 2004: 404-7
      • 2005:
        • Mississauga
        • NE Toronto
      • 2006:
        • Brampton-Caledon
        • Halton
      • 2007:
        • Central York
        • Durham
      • 2008: Toronto-Central
      • 2009: Airport?
    • Background
      • Employer program
        • Improve and encourage commuting options
        • Assessment of current conditions - Baseline
        • Development of a plan
        • Implementation – ‘Commuter benefits program’
          • Carpool Zone and ERH
          • Walking, cycling, transit, carpool, telework programs
          • Campaigns, contests and special events
        • Follow-up assessment – Measured against baseline
    • Approach - Ideal Goals Objectives Regional Transportation Plan Municipal Plans Implementation Strategic Plan Monitoring Evaluation
    • Approach - Actual Goals Objectives RTP Municipal Plans Implementation Strategic Plan Monitoring Evaluation Federal Funding Requirements Implementation Implementation
    • Approach
      • 2004-2007 - Contribution Agreement
        • Negotiated between funder and municipalities
      • 2008 - Monitoring Framework
        • Established by Smart Commute stakeholders
          • TMAs and Smart Commute Association (implementers)
          • Municipalities and Metrolinx (funders)
        • Multipurpose
          • Measure activity levels for funders
          • Track impacts to justify funding
          • Collect statistics for marketing purposes
    • Approach
      • Inputs / Activities
        • Number of businesses
          • Active
          • Engaged
          • Supporting
        • Number of pamphlets distributed
        • Number of cycling programs
        • Number of media releases
        • Number of commuters reached
    • Approach
      • Outputs / Benefits / Results
        • Brand awareness
          • % of employees aware; change from baseline
        • Commuter satisfaction
          • Ratings of Smart Commute program
        • Reductions
          • GHG emissions
          • CAC emissions
          • Vkt and trips
        • Commuter cost savings
    • Monitoring
      • Levels of monitoring
        • Regional
        • TMA
        • Employer
        • Commuter / customer
      • Types of monitoring
        • Behaviour change
        • Attitudinal change
        • Customer service
    • Monitoring
      • Regional
        • Existing sources
          • Statistics Canada
          • Transportation Tomorrow Survey (TTS)
            • Conducted by University of Toronto
            • Detailed behaviour survey
            • Conducted every five years
          • Cordon Count Program
            • Conducted by municipalities, compiled by University of Toronto
            • Observed data
            • Conducted every two to three years
    • Monitoring
      • Regional
        • New sources
          • Commuter Attitudes Survey
            • Conducted by Metrolinx
            • Detailed attitudinal questions
            • Conducted every two years
          • Carpool Zone
            • Conducted by Metrolinx
            • Monthly activity survey
          • TMAs and Employers
            • Compiled data from localized sources
    • Monitoring
      • TMA
        • Quarterly reports – tied to Metrolinx funding
        • Summary of activities per employers
        • Communications and outreach activity
        • Qualitative results
        • Comparison of activity levels between TMAs
    • Monitoring
      • Employer – Baseline Module
        • Baseline employee survey
          • Standardized across all TMAs and employers
          • Census survey
        • Site assessment
          • Current conditions
          • Inventory of advantages and deficiencies
        • Vehicle and occupancy count
          • Discreet sites
          • 1,000+ employees
    • Monitoring
      • Customer – Commuter
        • Service-related
          • Monthly Carpool Zone microsurvey
          • Emergency Ride Home post-ride survey
        • Employee surveys
        • Campaign-related data collection e.g. Car Free Day
    • Monitoring 10% 28% 19.7% 19.93% Transit 75.91% 48% 64.7% 65.90% Drive Alone 0.7% TTS (2006) 0.89% 1% 0.97% Cycling Surveys (2001-08) CAS (2008) Statistics Canada (2006) Mode
    • Evaluation
      • Benchmarks and Trends
        • 75 employee baseline surveys
        • Statistics Canada
        • Transportation Tomorrow Survey
        • Cordon Counts
        • Commuter Attitudes Survey
    • Evaluation
      • External
        • Employer surveys
          • Biannual survey – program satisfaction rating
          • Satisfaction with Smart Commute services
          • Conducted by third party through Metrolinx
        • Commuter surveys
          • Employee follow-up surveys – program satisfaction rating
          • Annual survey – Carpool Zone satisfaction
      • Internal
        • Smart Commute Technical Committee
    • Evaluation
      • External Review
        • Consultant evaluation of operational models
      • Research Partnerships
        • University of Toronto (Mississauga)
          • Factors for successful carpool formation
          • Further research
        • Ryerson University
          • TMA Toolkit assessment
    • Results
      • Almost 100 members and partners
        • March 2007 – 50 employers
        • March 2008 – 75 employers
        • June 2008 – 89 employers
    • Results
      • Transportation indicators
        • 76.4 million vkt
        • 1.3 million trips
        • Equivalent of 10,000 cars
        • Increase in carpooling
          • 7% to 12% to 13%
      Cars Off the Road
    • Results
      • Primary Mode Shift
      2006 2005 2008 Bike Walk/Jog Car/vanpool Public Transit Drive alone
    • Results
      • Environmental indicators
        • 17,500 tonnes of greenhouse gas emissions
          • Enough to fill Rogers Centre almost six times
        • 100,000 kg of criteria air contaminants
          • Equivalent amount of NOx to smoking 3 billion cigarettes
    • Results
      • Communications and Outreach
        • 400+ stories
        • 75+ million media impressions
        • Brand recognition
          • Smart Commute ↑
          • Carpool Zone ↓
      Awareness (2008) Clean Air Commute Commuter Challenge Carpool Zone Website Carpool Week Smart Commute
    • Challenges
      • Pressures to implement right away
        • Show action
        • Pent-up demand
        • More interesting than planning or evaluating
      • All the pieces not always in place
      • Monitoring not appreciated until results urgently needed
      • Objectivity
        • Of course my ‘baby’ was successful!
    • Challenges
      • Measurement and Evaluation
        • Employer buy-in is difficult
          • Some see value in surveys, monitoring
          • Others feel employees are oversurveyed
          • Often considered an expensive waste of employee time
        • Standardization required, but unenforceable
          • Moving to centralized service provision
          • Standardization vs. continuous improvement
        • Data comparison can be unequal
          • Apples to apples and apples to oranges
    • Challenges
      • Defining success
        • Setting targets without enough information
          • Are examples applicable?
        • Are targets achievable with the resources alloted?
        • Can these targets be effectively monitored?
        • What happens if we don’t reach these targets?
    • Lessons
      • Measurement and Evaluation
        • Some items simple, but time-consuming
          • Do you really need the data?
        • Project benefits less easy to isolate
          • Double-counting also possible through employee surveys and Carpool Zone surveys
          • External factors – environmental concerns, gas prices
        • One clear conclusion: an impact was made.
        • Time and consistency open up opportunities
          • Benchmarking
    • Lessons
      • Measure and Evaluation
      • Access resources at your disposal for help
        • Universities
        • Municipalities
        • Provincial and Federal Governments
      • Evaluate what you do and how you do it
      • Give an adjustment period to monitoring
        • Test! Test! Test!
        • Allow partners time to understand and see value
    • Lessons
      • Provide incentives for monitoring
        • Tie completion to access
        • Award and reward success
      • Measurement and Evaluation
        • Learn from mistakes – you will make them!
        • Learn from poor results
          • Was the program effective?
          • Was the monitoring effective?
    • Next Steps
      • Continue and improve data collection
        • Address data management and manipulation
      • Benchmarks and comparisons
      • Strategic planning
        • Service evaluation
        • Targets
      • Baseline module refinement
    • Questions? Ryan Lanyon Team Lead, Smart Commute Metrolinx [email_address] 416-874-5933