How to Evaluate  Driver Education Programs
Published August 2006 Prepared by: Northport Associates Co-funded by: BMW of North America Available online at: www.aaafou...
The Problem <ul><li>Traffic crashes remain the #1 killer of teens </li></ul><ul><li>Most research indicates that driver ed...
Purpose of This Research <ul><li>To develop comprehensive guidelines on how to evaluate the effectiveness of novice driver...
<ul><li>Does novice driver training and education make an impact on skills and safety? </li></ul>
History of Evaluation in Driver Education <ul><li>Long history – Jim Nichols NTSB paper </li></ul><ul><li>Recent reviews –...
History of Evaluation in Driver Education <ul><li>Prominent Findings: </li></ul><ul><li>DeKalb </li></ul><ul><li>Reviews <...
…  the  systematic assessment  of the  operation  and/or  outcomes  of a  program or policy , compared to a set of explici...
Why Evaluate Driver Education Programs? <ul><li>1. We have big questions left unanswered:  </li></ul><ul><ul><li>Do progra...
Why Is  Systematic  Evaluation Important? <ul><li>2. Credible answers allow us to:  </li></ul><ul><ul><li>Identify program...
What Is Important to Know About Evaluation? <ul><li>Key Concepts:  </li></ul><ul><li>1.  Formative Evaluation </li></ul><u...
What Is Important to Know About Evaluation? <ul><li>Key Concepts:  </li></ul><ul><li>2.  Summative Evaluation </li></ul><u...
<ul><li>Evaluating Driver Education Evaluation </li></ul><ul><li>Has evaluation  improved  &  proved  driver education? </...
Five Steps to Evaluating Step 1 FOCUS the EVALUATION Step 5 INTERPRET FINDINGS & TAKE ACTION Step 4 GATHER & ANALYZE DATA ...
Overcoming Barriers to Evaluating? <ul><li>What if researchers find your program does not work?  </li></ul><ul><ul><li>Sys...
Retooling Driver Education for Progress  <ul><li>World has changed around driver education – reform is required </li></ul>...
The Future: Bridging the Great Divide  <ul><li>Practitioners & managers “believe in” driver education </li></ul><ul><li>Re...
Evaluating Driver Education Programs : Comprehensive Guidelines  Management Overview How-To Manual  What is next? Large-Sc...
The AAA Foundation for Traffic Safety is a 501(c)(3) public charity located in Washington, DC that is dedicated to saving ...
Upcoming SlideShare
Loading in...5
×

Evaluation And Futureof Driver Ed

790
-1

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
790
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Evaluation And Futureof Driver Ed

    1. 1. How to Evaluate Driver Education Programs
    2. 2. Published August 2006 Prepared by: Northport Associates Co-funded by: BMW of North America Available online at: www.aaafoundation.org
    3. 3. The Problem <ul><li>Traffic crashes remain the #1 killer of teens </li></ul><ul><li>Most research indicates that driver education courses are not effective tools to reduce these deaths and injuries </li></ul>
    4. 4. Purpose of This Research <ul><li>To develop comprehensive guidelines on how to evaluate the effectiveness of novice driver training and education programs </li></ul>
    5. 5. <ul><li>Does novice driver training and education make an impact on skills and safety? </li></ul>
    6. 6. History of Evaluation in Driver Education <ul><li>Long history – Jim Nichols NTSB paper </li></ul><ul><li>Recent reviews – little critical analysis </li></ul><ul><li>One recent meta-analysis </li></ul><ul><li>Mainly summative evaluation </li></ul><ul><ul><li>Experimental RCTs – records, surveys </li></ul></ul><ul><ul><li>Quasi-experiments – +/- statistical control </li></ul></ul><ul><ul><li>Ecological studies – population effects </li></ul></ul>
    7. 7. History of Evaluation in Driver Education <ul><li>Prominent Findings: </li></ul><ul><li>DeKalb </li></ul><ul><li>Reviews </li></ul><ul><ul><li>Mayhew & Simpson </li></ul></ul><ul><ul><li>Australian reviews </li></ul></ul><ul><ul><li>“ Systematic” reviews </li></ul></ul><ul><ul><li>Elvik & Vaa meta-analysis </li></ul></ul>
    8. 8. … the systematic assessment of the operation and/or outcomes of a program or policy , compared to a set of explicit or implicit standards as a means of contributing to the improvement of the program or policy. Weiss, 1998 Management Overview Program Evaluation Definition
    9. 9. Why Evaluate Driver Education Programs? <ul><li>1. We have big questions left unanswered: </li></ul><ul><ul><li>Do programs have logical learning objectives? </li></ul></ul><ul><ul><li>Do they achieve their learning objectives? </li></ul></ul><ul><ul><li>Do they have the desired safety impacts? </li></ul></ul><ul><ul><li>Do some programs lead to better results than others? </li></ul></ul><ul><ul><li>Which components of programs contribute to success? </li></ul></ul><ul><ul><li>How can programs be improved to maximize impact </li></ul></ul>
    10. 10. Why Is Systematic Evaluation Important? <ul><li>2. Credible answers allow us to: </li></ul><ul><ul><li>Identify program strengths & weaknesses </li></ul></ul><ul><ul><li>Measure progress objectively </li></ul></ul><ul><ul><li>Identify ways to improve programs </li></ul></ul><ul><ul><li>Share objective information about what works </li></ul></ul><ul><ul><li>Influence policy & funding </li></ul></ul><ul><ul><li>Build an effective base of knowledge </li></ul></ul>
    11. 11. What Is Important to Know About Evaluation? <ul><li>Key Concepts: </li></ul><ul><li>1. Formative Evaluation </li></ul><ul><ul><li>Intended to improve program logic, content, delivery </li></ul></ul><ul><ul><li>Asks: </li></ul></ul><ul><ul><ul><li>Does the program operate according to plan? </li></ul></ul></ul><ul><ul><ul><li>Is it meeting its teaching objectives? </li></ul></ul></ul><ul><ul><ul><li>How can it be improved? </li></ul></ul></ul>
    12. 12. What Is Important to Know About Evaluation? <ul><li>Key Concepts: </li></ul><ul><li>2. Summative Evaluation </li></ul><ul><ul><li>Intended to prove or “sum-up” results & benefits </li></ul></ul><ul><ul><li>Asks: </li></ul></ul><ul><ul><ul><li>Does the program make students better drivers? </li></ul></ul></ul><ul><ul><ul><li>Does it reduce crashes? </li></ul></ul></ul>
    13. 13. <ul><li>Evaluating Driver Education Evaluation </li></ul><ul><li>Has evaluation improved & proved driver education? </li></ul><ul><ul><li>Negative influence on beliefs and policy </li></ul></ul><ul><ul><li>Weak in formative/process areas </li></ul></ul><ul><ul><li>Statistical weaknesses in summative evaluations </li></ul></ul><ul><ul><li>Unsystematic – “hit & run” research </li></ul></ul><ul><ul><li>Lots of room for improvement: </li></ul></ul><ul><ul><ul><ul><ul><li>Positive, cost-effective program impacts </li></ul></ul></ul></ul></ul><ul><ul><ul><li>Continuous improvement </li></ul></ul></ul><ul><ul><ul><li>Knowledge & know-how </li></ul></ul></ul><ul><ul><ul><ul><ul><li>Progressive policy </li></ul></ul></ul></ul></ul>
    14. 14. Five Steps to Evaluating Step 1 FOCUS the EVALUATION Step 5 INTERPRET FINDINGS & TAKE ACTION Step 4 GATHER & ANALYZE DATA Step 3 IDENTIFY DATA COLLECTION PLAN & TOOLS Step 2 SELECT EVALUATION METHODS PROGRAM
    15. 15. Overcoming Barriers to Evaluating? <ul><li>What if researchers find your program does not work? </li></ul><ul><ul><li>Systematic program evaluation is the key </li></ul></ul><ul><ul><li>Ongoing, stepwise “system” of evaluation </li></ul></ul><ul><ul><li>Small steps & continuous improvement </li></ul></ul><ul><ul><li>There should be little risk of nasty surprises </li></ul></ul><ul><ul><li>Still, plan for unexpected findings </li></ul></ul>
    16. 16. Retooling Driver Education for Progress <ul><li>World has changed around driver education – reform is required </li></ul><ul><li>Most jurisdictions lack resources & political will </li></ul><ul><li>Improve Uniformity – there cannot be 60 “best” ways </li></ul><ul><li>Behavior change is tough & programs need to reflect this </li></ul><ul><li>Evidence-based development is badly needed </li></ul><ul><li>R&D & comprehensive evaluation need high priority </li></ul>
    17. 17. The Future: Bridging the Great Divide <ul><li>Practitioners & managers “believe in” driver education </li></ul><ul><li>Researchers typically believe “driver education doesn’t work” </li></ul>“ For more effective driver education in the future, key people in both camps must work together toward improving evaluation and using evaluation tools to improve driver education.” Management Overview p.15
    18. 18. Evaluating Driver Education Programs : Comprehensive Guidelines Management Overview How-To Manual What is next? Large-Scale Evaluations of Driver Education Programs Implementation of Guidelines Guidelines
    19. 19. The AAA Foundation for Traffic Safety is a 501(c)(3) public charity located in Washington, DC that is dedicated to saving lives and reducing injuries. It is supported by donations from AAA/CAA Clubs, AAA/CAA members, and other organizations associated with AAA/CAA.
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×