Published on

Presentation by the Synergy Group on the Vancouver BikeShare program for the Case Competition Final Round

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this


  2. 2. The Synergy GroupMary-Jean CostelloSenior AssociateKelly SkinnerSenior AssociateRenata ValaitisSenior AssociateAnum Irfan KhanJunior AssociateSarah SousaJunior Associate
  3. 3. Program ContextSocial andculturalenvironmentRegulatory/policyenvironmentThe builtenvironmentIndividualfactors• Improved healththrough increasedphysical activity• Reduced pollution• Reduced congestionon roads and publictransport• Strengthening of thetourism, businessclimate andsustainability inalternatetransportation modesActive Transport
  4. 4. Vancouver Public Bikeshare Program• Increase access to and use of bicycles by placing dockingstations through out the city• Enables individuals to make one way trips for a minimalfee• Program aims:1. Support active living2. Extend the reach of transit andwalking trips3. Replace vehicle and transit trips4. Trigger participation in cyclingat the broad level
  5. 5. Vancouver Public Bikeshare Program• Yearly operation – 1000 bicycles at sites across city core• UNIQUE! All users are required to wear helmets• Program will include helmet vending machines
  6. 6. Stakeholder DiagramDirector ofEvaluationCity ofVancouverEvaluationAdvisoryGroupResidents ofVancouver(Users andnon-users)Ministry ofTransportationVancouverPublicBikeshareProgramOtherBikesharePrograms
  7. 7. Evaluation Objectives• Limited evaluation of the population-level impacts ofbikeshare programs• To conduct a process and outcome evaluation to:• Determine the uptake of the Vancouver Bikeshare Program• Identify barriers and facilitators to the program and the use ofhelmet vending machines• Assess equity of the program in terms of equal access to theprogram and helmets and equal use of helmets and bikes• Compare outcomes of the program to other similar programs inCanada
  8. 8. EvaluationMethodology
  9. 9. Program Logic ModelInputsResources Activities-Equipment(bikes/helmet)-Infrastructure(bike/helmetstations)-BikeScoremaps (bikelanes, hills,destinations)-Staff tomanagestations-Funding-Implementationof bike/helmetstations-Communityawareness-EvaluationOutcomes/ImpactShort Term Long Term- ↑ availability+ affordabilityof bicycles- ↑ 5 As forhelmets- ↑ cycling- ↑ commutingto work- ↑ health- ↓ barriers tocycling andhelmet use- ↑ activetransportationand triggerparticipationin cyclingmore broadlyOutputs- # of stations- # of bikes/helmetsavailable- # of users +memberships- # of bike trips- Days and minutesof cyclingAssumptions- Implementation of program is just beginning –basic data from memberships (e.g. demographic) –survey adds data- Focus on adults 18+- Proximity of bike and helmet stations to each other
  10. 10. Evaluation MatrixQ1. What is the uptake of the VancouverPublic Bikeshare Program?Indicators Data Sources Methods # bike trips/day per station average # trips per bicycle # bike users # helmets rented average distance per trip # monthly memberships # annual memberships # single day passes # multi-day passesDemographicinformation frommembership intakeformGeographic/spatialinformation fromcensus dataCycling In Cities BikeScore IndexData review combinedwith GIS mapping
  11. 11. Evaluation MatrixQ2. What are the health, environmental andeconomic impacts of the program?Indicators Data Sources Methods Projected financial savings from usingbike transport # minutes of bike use (proxy to physicalactivity) # days and minutes of total andrecreational cycling/week Changes to environmental infrastructure Air pollution Traffic congestion/changes in trafficvolumeHealth – IPAQquestionnaire (usedin Montreal, Fuller2013)Air qualityassessment/environmental/meteorological dataTrends in trafficreportingOnline surveycontaining IPAQquestionData review
  12. 12. Evaluation MatrixQ3. What are the barriers and facilitators ofthe Vancouver Public Bikeshare Program?Indicators Data Sources Methods Sample Biking connectivity (e.g., #roads that have cyclinglanes) Bike route characteristics Perceived risks of bikesafety Perceived barriers toparticipation in the program Benefits identified byparticipants Awareness of programservicesCensus data,geographicaldatabasesUser & non-user feedbackGIS mappingOnline surveyUsers (Monthly-Yearly)Users (Single-use, nomembership)Non-Users
  13. 13. Evaluation MatrixQ4. What is the use and accessibility of thehelmet vending machines?Indicators Data Sources Methods• # helmets rented/day per station• # of helmets available per day/perstation• Proximity of helmet station to bikestation• # of helmet sizes available in vendingmachine• User helmet cost• # of bike users who are also helmetrenters• # helmet not returned• Acceptability/AccommodationinformationInventory RecordsHelmet MachineUsers/Non-usersInventory andHelmet MachineTransaction recordreviewOnline andtelephone surveys
  14. 14. Evaluation MatrixQ5. How equitable is the VancouverBikeshare Program?Indicators Data Sources MethodsDemographic data(i.e., age, household composition,families with children, race andethnic group, income class, peoplewith disabilities)Bike scoreCensus data GIS mapping
  15. 15. Evaluation MatrixQ6. How do outcomes compare to othersimilar programs in Canada?Indicators Data Sources MethodsComparisons will be made based onoutcomes available in publishedliterature for relevant indicatorslisted in the above matrix (e.g., #bike trips per day)Publishedoutcomeinformation fromother programsin CanadiancitiesLiterature Review
  16. 16. METHODSRECORD REVIEWTYPE OF RECORDS - Membership data collected (including type ofmembership, social demographic information, bikescore)- Inventory of helmet machine transactionsRELATED EVALUATIONQUESTIONSWhat is the uptake of the program?What is the use/accessibility of helmet vendingmachinesBENEFITS- Will provide valuabledemographic data of users- affordable & low burdenRISKS- Incomplete data- Time consuming to completemembership survey
  17. 17. Methods ContinuedLITERATURE REVIEWTYPE OF RECORDS - Review published literature describing outcomedata from other bike sharing programs acrossCanadaRELATED EVALUATIONQUESTIONSHow do outcomes compare to other similar programsin Canada?BENEFITS- Will provide valuablecomparison data from acrossCanada- affordable & low burdenRISKS- Limited to available publishedliterature on Bike Sharing- May be limited to certainindicators
  18. 18. SURVEY DATAONLINE SURVEYMETHOD(for users)- Online survey will address demographic information, program use,facilitators & barriers to program use- Closed & open-ended questions used- Existing validated survey questions will be used, ex. Internationalphysical activity questionnaire- For users with membership: survey will be emailed to appropriateaddress- For daily/single users: link to survey will be printed on ticketreceipt)- Incentive will be offered (voucher for a free ride/discount for nextyears’ membership)RELATEDEVALUATIONQUESTIONSWhat are health, environmental, and economic impacts of theprogram?What are the facilitators & barriers of the program?What is the use & accessibility of helmet vending machines?(addressing 5 dimensions –availability/accessibility/affordability/acceptability/accommodation)BENEFITS- Inexpensive & easy to administer- Can use existing validated questions- No interviewer biasRISKS- Poor response rate- Limited for populations of low literacy
  19. 19. TELEPHONE INTERVIEWSTELEPHONEINTERVIEWS(for non-users)- Telephone interviews conducted with non-users of theprogram- Random-digit-dialing- Short interview will be conducted relating to use andperceived facilitators and barriers to bike sharing (ex. Why areindividuals NOT using bike sharing)?- Interviews will be tape recorded & transcribed- Free voucher/membership reduction will be offered as anincentiveRELATEDEVALUATIONQUESTIONSWhat are the facilitators & barriers of the program?What is the use & accessibility of helmet vending machines?(addressing 5 dimensions – availability/accessibility/affordability/acceptability/accommodation)BENEFITS- Ability to obtain rich, valuableinformation- Can probe for additionalinformationRISKS- Poor response rate- Client burden/time consuming
  20. 20. KEY INFORMANT FOCUS GROUPDescription: - One or two informant focus groups (10 stakeholdersmaximum) will be conducted with key stakeholders (advisoryteam, Bike Sharing operators, etc.)- Used to assess their perceptions regarding facilitators/barriersand implementation of the program- One hour, tape recorded focus group- Conducted at a convenient time & location for all stakeholdersRELATEDEVALUATIONQUESTIONSWhat are the facilitators & barriers of the program?BENEFITS- Ability to obtain rich, valuableinformation from an important group- Can probe for additional information- Flexible in the course of collectingdataRISKS- Social desirability effects- Observer bias
  21. 21. GIS MAPPINGDescription: - Will combine geographic information with demographic data, andprogram outcomes to create multilayered visual maps- Will explore how the program is situated within and interacts with itsenvironment- The layers used:- Demographic information- Neighbourhood land use- Transit use- SES- Street connectivity- Bikeability scoresRELATEDEVALUATIONQUESTIONSWhat is the uptake of the program?What are the health, environmental and economic impacts of theprogram?What are the barriers and facilitators of the program?BENEFITS- Well suited for multi-site programs- Disentangle program outcomesacross sitesRISKS- Expensive- Data is not always available
  22. 22. GIS Mapping ExampleBike Scores: Hills:
  23. 23. DATA ANALYSIS PLANQuantitative Data - Quantitative analysis using SPSS- frequency counts, cross tabulations will be conducted whereappropriateQualitative Data - Qualitative analysis using Nvivo10- Identification of common codes & themes- Thematic analysisGIS Mapping - Analysis will include quantity and density mapping- GIS maps can be combined/compared with other relevantdata- Patterns will be assessed
  24. 24. Challenges &Solutions
  25. 25. Key Considerations:Internal FactorsInternal Factors1. Access limited tocredit card users Bikes can only be accessed via a credit card limitingaccess to other forms of payment and individuals withouta credit card Restricts access to individuals under legal limits2. Helmet and biketransactions Lack of integration between helmet and bike rentaltransactions
  26. 26. Key Considerations:External FactorsExternal Factors1. Public perceptionsaround bike safety- Previous data reveals that women and older adults tendnot to use bikeshare programs due to safety concerns2. Environmentalcharacteristics- Bike Score may impact program utilization andoutcomes (i.e., Bike route density, connectivity with otherbike-friendly streets, topography)3. Helmet uselegislation in BC- BC legislation upholds mandated helmet use- Limited data on success of helmet vending machines- Proximity to helmet rental to bicycle rental (5dimensions of access)
  27. 27. EquityAnalyses: Internal and External FactorsInternal andExternal FactorProposed Equity Assessment1. Limited paymentmethods/optionsUtilization data along with online and telephone surveyswill be used to determine what extent this inhibits bothexisting and potential users2. Environmentalcharacteristics Bikescores will be overlaid with program utilization andoutcome data Examine user and non-user feedback pertaining togeographic and environmental factors affecting programuse (environmental justice perspectives)3. Station locationand access Comparing socio-demographic and utilization dataassess potential gaps in access and utilization4. Helmet accessand use Location and utilization of helmet vending machines willbe assessed in comparison to the feedback received fromsurveys of user and non-user groups
  28. 28. Possible Challenge and Potential Solutions• Cost-effective data collection tools• Triangulation of data sourcesBudgetary and timeconstraints• Establishing baseline measures• Drawing upon evaluations conducted inother jurisdictions• Leverage pre-existing dataLimited pre-existing orbaseline data
  29. 29. Possible Challenge and Potential Solutions• Data collected via group interviews &surveys will not be linked to anyidentifying information to protectconfidentiality and privacy rightsConfidentiality andethical Concerns• Data collection tools designed to reacha wide range of users, non-users andprogram staff• Incentives will be utilized to promoteresponses and engage both users andnon-users in sharing their perspectivesCoverage, geographyand number of sitesinvolved
  30. 30. Possible Challenge and Potential Solutions• Identification and integration oflinguistic profiles/needs into datacollection toolsLanguage and culturalappropriation of datacollection tools
  31. 31. Next Steps• Working in conjunction with the evaluation advisory committee torefine the design of the proposed evaluation• Integrate stakeholder feedback into evaluation design• Finalize evaluation design and establish timeline and reportingexpectations
  32. 32. THANK YOUFind ussynergygroup@evaluate.comSynergyGEvaluate The Synergy Group
  33. 33. Acknowledgements• Canadian Evaluation Society• 2013 CES Student Case Competition Working Group• CES Travel and accommodation funding supporters• Competition Judges• University of Waterloo• Dr. Anita Myers & Dr. Jennifer Yessis• School of Public Health and Health Systems• Family, friends and the beautiful city of Toronto