• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Intro to SMS
 

Intro to SMS

on

  • 1,013 views

Introduction to the CSA 2010 Safety Measurement System

Introduction to the CSA 2010 Safety Measurement System

Statistics

Views

Total Views
1,013
Views on SlideShare
999
Embed Views
14

Actions

Likes
0
Downloads
4
Comments
0

2 Embeds 14

http://www.complystar.com 10
http://www.slideshare.net 4

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Talking Points :3 components but will be focusing this presentation on the new measurement system (click the mouse for the SMS bullet to turn red)
  • Talking Points :The measurement system is essentially FMCSA’s radar system. It is a better measurement system, identifying more carriers with safety compliance problems, and more clearly identifying those problems so they can be address. While the proposed SFD is part of the new approach it will not be rolled out with SMS and new interventions in 2010. It is dependent on future rulemaking. An explanation of how SFD would work will be detailed and presented for public comment in a Notice of Proposed Rulemaking later this year.
  • Talking Points :The new SMS takes all the on road safety performance data in FMCSA’s data system – MCMIS – and distributes it into the “right buckets” and assigns a severity risk. After getting the data, the SMS weighs the data and gives the carrier a percentile based on its peers. For example, how does ABC trucking stand in with other carriers that have a fatigue BASIC?BASIC measures is “how much does this bucket weighs now?”The percentile tells us how the “bucket” compares to the carrier’s peers.
  • Talking Points:Here we describe Safety Events.For carriers– we look at a 2 year window.We also look at intrastate carriers
  • Talking Points :This slide lists the seven Behavior Analysis Safety Improvement CategoriesThe methodology is designed to weight on-road safety data based on its relationship to crash risk and focuses on behaviors that lead to crash risk.The data is also time-weighted over a 24 month time period so that it is reflective of current on-road safety performance. If a carrier’s performance improves over the time, the safety performance score improves.HM regulation violations (171, 172, 173, 177, 178, 180) may also be found/included in other BASICs such as driver fitness, but the most concentrated BASIC for these is Cargo Related so they are listed there.
  • Talking PointsAfter the data is in the appropriate bucket – it is weighted. Several things are considered:Time weighting; Severity weighting; Normalizing – violations vs. power units; Single inspection cap – limits weight of a single violation - especially important to small carriers; Violation cap – helps establish uniformity and consistencyThis step allows the Agency to see where a particular carrier stands regarding safety by BASICSome considerations to be aware of:Time weighting – puts focus on more recent eventsSeverity – this is a “risk” weightNormalizing – this is a denominator for “exposure” – usually this is the number of inspections.Two different caps:Single Inspection and Violation
  • Talking Points :Now this presentation will go through how each BASIC is measured.This is a similar construct for other BASICSThe considerations are time and severity weight – and the number of power units.
  • Talking Points:Something different here – time and severity weight is based on crash risk.
  • Talking Points:Here we look at CDL violations. About 2/3 to 3/4s of these violations are due to medical certificate issues.
  • Talking Points :This BASIC measure is normalized by number of power units/carrier size.
  • Talking pointsThis slide shows how a BASIC measure is calculated for Vehicle Maintenance.
  • Talking Points :This slide shows a BASIC measure is calculated for Cargo-Related.
  • Talking Points :This measure shows how a BASIC measure is calculated for the Crash Measure.
  • Talking Points :This slide introduces the percentile concept.After looking at all the safety events – they are measured with the formulas just covered. The next step is assigning a percentile to the carrier.Here SMS does “peer grouping” to compare similar carriers and also make sure there is enough data to get a measure.The system also looks at more current information – with more focus on what has happened in the past 12 months even though it uses 24 months of data.
  • Talking Points :The percentile is based on a carrier’s peer grouping and this slide shows the peer groups (1-5).This is done to compare entities that have similar exposure.The left column is the peer groups.
  • Talking Points :Percentiles are not assigned until sufficient data is obtained.Agency wants to make sure there are a minimum number of inspections.With that data, patterns of a carrier’s behavior can be seen.
  • Talking Points:Thresholds are different depending on the BASIC and type of carrier. Here are intervention thresholds in the test.In the far-right column the threshold is for “all other” carriers. The reason why Unsafe, Fatigue, and Crash thresholds are lower is because research shows these have a greater relation to future crash risk.A methodology was applied to determine these thresholds. These thresholds are not arbitrary. THE NEW INTERVENTIONS:Warning LetterOffsite InvestigationOnsite Focused InvestigationOnsite Comprehensive Investigation
  • Talking Points :A common question from carriers is how do I “get better?”The best way to improve is to get “good” inspections (i.e. no violations) since this is a performance-based system.For Fatigued Driving, Driver Fitness, Vehicle Maintenance and Cargo Related BASICsNo percentile assigned if:No inspections with a violation in that BASIC within the past year; andMost recent relevant inspection does not have a violation of that
  • Talking Points Percentile results are the next step after the Data Preview, which will allow carriers to assess and address safety concerns.
  • Talking Points The Agency will not give individual driver ratings as part of CSA 2010 despite rumors. In order for the Agency to rate drivers new authorities would be required through reauthorization AND rulemaking. The next slide will address driver data that will be available to carriers and drivers in the near future.
  • Talking Points
  • Talking Points Clear understanding that data collected at the roadside is a critical component of all traffic safety initiatives. For example, CSA 2010, TACT etc…So for both the carriers and the enforcement agencies that use this data its importance cannot be underestimated. During the summer of 2008 FMCSA and its state partners working on the Op Model test (FSWG) identified a need to enhance the uniformity and quality of roadside violation data. During discussions the consensus was that the data in its current form is fundamentally sound, valid and useableHowever, opportunities exist to enhance the quality of data and in turn, improve the integrity of all traffic safety programs.
  • The effort to enhance the quality and uniformity of the roadside violation data started at the Fall 2008 CVSA meeting with the creation of an Ad-hoc committee to look at this issue. Currently this committee is managing an Alliance-wide effort to enhance the overall quality and uniformity of the data. The efforts has four core components listed on the screen.Consistent documentation of roadside inspection and violation data- Through a FMCSA funded high priority grant, CVSA began work on guidance that will promote and/or mandate the consistent documentation of roadside inspection and violation data. Standardized processes for challenging data- This initiative will provide procedural guidance on the management of the roadside data challenge process through our Data Q’s management systemIncreased awareness of the high level goals of the inspection program- This component of the roadside data uniformity initiative will focus on the increased importance of the roadside violation data and an understanding of how the data will be used. The goals is to broaden the understanding that every inspection counts and that there is a direct relation between the collection of the data and the end use of the data. Uniform inspection selections system-CVSAis currently facilitating discussions regarding the policies that govern when and how vehicles should be selected for an inspection. The goal is to focus discussions on implementing a valid and consistent vehicle and driver inspection selection process operated within the scope of a jurisdiction’s rules and policies that will promote roadside data uniformity.
  • Talking Points Successful implementation of CSA 2010 is NOT dependent on SFD going through.CSA 2010 incorporates the existing rating process and will continue to do so until SFD goes into effect.Ratings are issued based on investigation findings:On-site comprehensive investigations can result in Sat., Conditional or Unsat. ratingsOn-site focused investigations can result in Conditional or Unsat. RatingsOff-site investigations do not result in a ratingCarriers can apply for a request for upgrade (§385.17)
  • This slide highlights the differences between SafeStat and SMS SMS Applies risk-based weightings to violations in order to identify high crash-risk carriers demonstrating PATTERNS of unsafe behavior roadsideSMS Matches poor safety performance with appropriate level of interventionSMS Uses all SAFETY BASED roadside data, allowing more carriers with unsafe behavior to be identified for interventionSMS Assesses carriers and drivers – the driver SMS is a tool for investigators to identify drivers with safety problems during carrier investigations

Intro to SMS Intro to SMS Presentation Transcript

  • CSA 2010 Operational Model TestIntroduction to the Safety Measurement SystemApril 2010
  • Comprehensive Safety Analysis 2010
    What is CSA 2010?
    CSA 2010 is a pro-active initiative to improve the efficiency and effectiveness of FMCSA’s enforcement and compliance program to achieve the Agency’s mission to reduce commercial motor vehicle (CMV) crashes, fatalities, and injuries.
    2
  • Three Core Components
    New Safety Measurement System (SMS)
    Improved ability to identify demonstrated safety problems
    New interventions process
    Employs an array of interventions instead of the single option, labor-intensive compliance review
    New approach to Safety Fitness Determination (SFD)
    SFD would be tied to current safety performance; not limited to results of acute/critical violations from a Compliance Review
    3
  • New Safety Measurement System
    Assesses safety of carriers and drivers based on unsafe behaviors that lead to crashes
    Calculates safety performance based on 7 Behavior Analysis and Safety Improvement Categories (BASICs)
    Weights time and severity of violations based on relationship to crash risk
    Uses crash records and all safety-based violations found roadside.
    Identifies carriers for interventions
    Determines what problems need to be addressed by intervention process
    Monitors carriers on road performance for improvements throughout the process
    In the future, measurement scores would support future Safety Fitness Determinations
    New Safety Fitness Determination methodology is currently in rulemaking; initial rollout of CSA 2010 is not dependent on rule
    4
  • Methodology Overview
    Obtain on-road safety data (e.g. inspections, crashes) and attribute to carrier to create a safety event history
    Place each carrier’s violations/crashes into a BASIC
    Convert BASIC data to quantifiable measure/rate (in future, Safety Fitness Determination will likely be based on absolute performance)
    Based on each carrier’s BASIC measure, develop rank and percentile for each entity’s BASIC performance
    5
  • Safety Data
    Safety Data Attributed to Carrier
    Carrier Safety Measurement System (CSMS)
    Includes 24 months of carrier on-road safety performance
    6.6 Million inspections
    290 Thousand crashes
    690 Thousand carriers
    6
  • BASIC Data
    Safety Data Sorted by BASIC
    Unsafe Driving (Parts 392 & 397)
    Fatigued Driving (Hours-of-Service) (Parts 392 & 395)
    Driver Fitness (Parts 383 & 391)
    Controlled Substances/Alcohol (Parts 382 & 392)
    Vehicle Maintenance (Parts 393 & 396)
    Cargo-Related (Parts 392, 393, 397 & HM)
    Crash Indicator
    7
  • BASIC Measures
    Convert BASIC Data into Quantifiable Measure
    Factors used in calculating a BASIC measure:
    Time Weighting/Time Frame - More recent events more relevant
    Severity Weightings – Increase weighting of violations that have been shown to create a greater risk of crash involvement
    Normalizing – Based on exposure; use of number of inspections and power units
    Violation Cap –Cited section number only counts once per inspection
    BASIC Severity cap –Limits the severity weight applied to a BASIC measure from a single poor inspection.
    8
  • Unsafe Driving Measure
    Operation of Commercial Motor Vehicles (CMVs) in a dangerous or careless manner
    Examples – speeding, reckless driving, improper lane change
    Factors used in calculating the measure:
    Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)
    Violation Severity Weight
    Based on crash risk – Range from 1-10, where 10 is the most severe
    Normalized by Average Power Units
    9
  • Fatigued Driving (HOS) Measure
    Operation of CMVs by drivers ill, fatigued or in non-compliance with the hours-of-service (HOS) regulations
    Examples: HOS, logbook, and operating CMV while ill or fatigued
    Factors used in calculating the measure:
    Time Weight - 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)
    Violation Severity Weight
    Based on crash risk – Range from 1-10, where 10 is the most severe
    OOS (+2)
    Normalized by Relevant Inspections – Levels 1,2,3,6 and any other inspections resulting in related violations
    10
  • Driver Fitness Measure
    Operation of CMVs by drivers who are unfit to operate a CMV due to lack of training, experience or medical qualifications
    Failure to have a valid and appropriate commercial driver’s license, being medically unqualified to operate a CMV.
    Factors used in calculating the measure :
    Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)
    Violation Severity Weight
    Based on crash risk – Range from 1-10, where 10 is the most severe
    OOS (+2)
    Normalized by Relevant Inspections – Levels 1,2,3,6 and any other inspections resulting in related violations
    11
  • Controlled Substances/Alcohol Measure
    Operation of CMVs by drivers who are impaired due to alcohol, illegal drugs, and misuse of prescription or over-the-counter medications
    Examples: Use or possession of controlled substances or alcohol
    Factors used in calculating the measure:
    Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)
    Violation Severity Weight
    Based on crash risk – Range from 1-10, where 10 is the most severe
    Normalized by Average Power Units
    12
  • Vehicle Maintenance Measure
    Failure to properly maintain a CMV
    Examples: brakes, lights, and other mechanical defects, and failure to make required changes
    Factors used in calculating the measure:
    Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)
    Violation Severity Weight
    Based on crash risk – Range from 1-10, where 10 is the most severe
    OOS (+2)
    Normalized by Relevant Inspections – Levels 1,2,5,6 and any other inspections resulting in related violations
    13
  • Cargo-Related Measure
    Failure to properly prevent shifting loads, spilled or dropped cargo, and unsafe handling of hazardous materials on a CMV
    Examples: improper load securement, cargo retention, and hazardous material handling
    Factors used in calculating the measure:
    Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)
    Violation Severity Weight
    Based on crash risk – Range from 1-10, where 10 is the most severe
    OOS (+2)
    Normalized by Relevant Inspections – Levels 1,2,5,6 and any other inspections resulting in related violation
    14
  • Crash Measure
    Histories or patterns of high crash involvement, including frequency and severity
    Based on state-reported crash records
    Factors used in calculating the measure:
    Time Weight – 0-6 Months (3), 6-12 Months (2), 12-24 Months (1)
    Crash Severity Weight
    Range from 1-3 – Crashes involving injury/fatality or HM release have more weight
    Normalized by Average Power Units
    15
  • Percentile
    Based on each BASIC measure, develop percentile indicating carrier’s BASIC performance
    Provides a relative assessment of performance
    Allows for prioritizing intervention resources by behavior
    Considerations:
    Peer Grouping – compare measures of entities with similar levels of exposure
    Data Sufficiency standards – define events/exposure necessary to generate a robust measure
    SFD/Intervention standards – define “critical mass” of poor performance necessary for inclusion of entity in intervention process or detrimental SFD
    Current Inspection and Crash Data – assignment of percentile dependent on age and result of most recent inspection (12 months)
    16
  • Peer Grouping
    BASICs
    Fatigued Driving (HOS)
    Peer
    Unsafe Driving
    Driver Fitness
    Group
    Controlled Substances/Alcohol
    Vehicle Maintenance
    Crash
    Cargo-Related
    1
    0 < PU<= 5
    5 – 10 Inspections; (3-10 Fatigued)
    2
    5 < PU <= 15
    11 – 20 Inspections
    3
    15 < PU <= 50
    21 – 100 Inspections
    4
    50 < PU <= 500
    101 – 500 Inspections
    5
    500 < PU
    501+ Inspections
    Create percentile based on measure for carrier with similar exposure (same peer group)
    *PU = Power Unit
    17
  • Data Sufficiency
    Number of
    BASIC
    Inspections
    Unsafe Driving
    3
    Fatigued Driving (HOS)
    3
    Driver Fitness
    5
    Controlled Substances / Alcohol
    1
    Vehicle Maintenance
    5
    Cargo-Related
    5
    Crash
    2 Crashes
    Minimum number of inspections with applicable violations required for percentile to be assigned
    Assists in identification of patterns of carrier behavior – note safety problems across multiple inspections
    18
  • BASIC percentiles trigger Interventions
    Carriers that meet data sufficiency are assigned a percentile
    Ex: Driver Fitness BASIC percentile of 85% means the carrier is worse than 85% of its peers
    Carriers that exceed the BASIC threshold are identified for interventions
    19
  • Example of SafeStat vs. SMS
    Carrier under the Radar with Existing
    SafeStat System
    20
  • Carrier Measurement: SafeStat Results
    21
  • Carrier Measurement: SMS Results
    22
  • Violation Details Provided in SMS
    23
  • Further Drilldown in SMS
    24
  • How does a carrier improve and get out of the intervention process?
    “Good” Inspections
    “Get Well” Rules
    Unsafe Driving and Controlled Substances/Alcohol BASICs
    No percentile assigned if no inspections with a violation in these BASICs in the last year
    Crash Indicator
    No percentile assigned if no crashes in last year
    Fatigued Driving (HOS), Driver Fitness, Vehicle Maintenance and Cargo-Related BASICs
    No percentile assigned if:
    No inspections with a violation in that BASIC within the past year; and
    Most recent relevant inspection does not have a violation of that BASIC
    25
  • Carrier Access to Percentiles
    When will carriers’ SMS results be made available?
    Currently, only test state carriers have access to SMS results by using the Comprehensive Safety Information (CSI) system
    FMCSA will provide all carriers an early review of their own safety data by BASIC starting April 12, 2010
    Non-test carriers’ SMS results will be available to carriers in August
    Public will have access to carrier SMS results in the winter of 2010
    26
  • New Agency Plans for Drivers
    The new measurement system provides an internal tool to address CMV drivers:
    Provides enhanced information on individual drivers to investigators to identify drivers with safety problems
    Allows for prioritizing driver sampling during carrier investigation
    Supports investigator follow up on serious violations
    Under CSA 2010, individual drivers will not be assigned safety ratings or safety fitness determinations
    27
  • New Agency Plans for Drivers (cont’d)
    Other Agency initiatives are underway, including the Pre-employment Screening Program (PSP)
    PSP was mandated by Congress and is not a part of CSA 2010
    “Driver Profiles” from FMCSA’s Driver Information Resource (DIR) will be available to carriers through PSP
    Driver Profiles will only be released with driver authorization
    PSP is under development, more information can be found at www.psp.fmcsa.dot.gov
    28
  • Roadside Data Uniformity
    Data collected at the roadside is the foundation of all data driven traffic safety initiatives
    CSA 2010 relies on roadside data in its SMS Methodology
    The CSA 2010 SFD methodology would use roadside data as a component of safety fitness determinations
    29
  • Roadside Uniformity-Background
    Effort organized into four core initiatives:
    Consistent documentation of roadside inspection and violation data
    Standardized processes for challenging data
    Increased awareness of high level goals of the inspection program
    Good inspections can support systematic enforcement program
    Screening vs. Inspection
    Uniform inspection selection processes
    30
  • SMS and Safety Fitness Determination
    SFD would:
    Incorporate on-road safety performance via new SMS which updates on a monthly basis
    Continue to include major safety violations found as part of CSA 2010 investigations
    Produce a Safety Fitness Determination (SFD) of
    Unfit or
    Marginal or
    Continue Operation
    31
    Draft rulemaking is currently in review within DOT; NPRM expected to be published late 2010.
  • Safety Measurement System vs. SafeStat
    32
  • What Can Carriers Do To Prepare Now?
    Educate Yourselves and Your Employees:
    Understand the SMS Methodology and the BASICs
    Check the website for information and updates (http://csa2010.fmcsa.dot.gov)
    Raise awareness that every inspection counts and every violation counts
    Check and update records:
    Motor Carrier Census Form (MCS -150)
    Routinely monitor and review inspection and crash data
    Question potentially incorrect data (DataQs: https://dataqs.fmcsa.dot.gov)
    Ensure compliance:
    Review inspections and violation history over the past 2 years
    Address safety problems now
    Educate drivers about how their performance impacts their own driving record and the safety assessment of the carrier
    33
  • For more information, visit csa2010.fmcsa.dot.gov