RESEARCH IN ACTION ISSUE 2 Effectiveness of Mentoring Program Practices
Research In Action:  Overview of Series Last year, MENTOR released the  National Agenda for Action: How to Close America’s...
Research In Action  Issues: Issue 1:  Mentoring: A Key Resource for Promoting PYD Issue 2:  Effectiveness of Mentoring Pro...
Using the  Research In Action  Series <ul><li>Research:  a peer-reviewed article, written by a leading researcher, summari...
MPM Webinar Series <ul><li>Review  RESEARCH  presented in peer-reviewed article </li></ul><ul><li>ACTION  dialogue – how c...
Issue 2 – Effectiveness of Mentoring Program Practices  <ul><li>David L. DuBois, Ph.D., University of Illinois at Chicago ...
Issue 2 - Overview <ul><li>Introduction </li></ul><ul><li>Conceptual and Methodological Issues </li></ul><ul><li>Framework...
Mentoring Objective <ul><li>The central objective of any youth mentoring program is to establish and support beneficial re...
High-Quality Programs Establish and Support Relationships that are: <ul><li>Effective </li></ul><ul><li>Safe </li></ul><ul...
Effectiveness vs. Quality <ul><li>Program Effectiveness </li></ul><ul><li>Impacts on mentoring relationship quality </li><...
Program Effectiveness Problem DuBois, Holloway, Valentine, & Cooper, 2002
Evidence-Based Practice <ul><li>Using research to identify practices that reliably serve to enhance program effectiveness ...
Mentoring Best Practices:  Circles of Evidence Best Practices *  Includes evidence obtained from sources other than formal...
Conceptual and Methodological Issues <ul><li>Effects of any practice on program effectiveness will be dependent on a range...
Summarizing & Evaluating Mentoring Research <ul><li>Evidence Framework </li></ul><ul><li>Levels of Evidence </li></ul><ul>...
Levels of evidence: Level 1 – isolate mentoring program practice of interest (POI) Level 2 –less precise comparisons in wh...
Strength of Evidence <ul><li>Study Design </li></ul><ul><li>Methodological Rigor </li></ul><ul><ul><li>Sample size and rep...
Sources of Evidence <ul><li>BOTH  formal research  literature and  internal analyses  of information by mentoring programs...
Applying the Evidence Framework <ul><li>Identify research relevant to the POI </li></ul><ul><li>Categorize each study acco...
Research on Program Practices <ul><li>Descriptive Studies </li></ul><ul><li>Highlight important trends that merit further ...
Researching Program Practices <ul><li>Typical program evaluation is designed to estimate cumulative effects that are produ...
Meta-Analysis  (DuBois et al., 2002) <ul><li>Program practices predictive of stronger positive effects on youth outcomes <...
Pre-Match Training <ul><li>Available evidence regarding the effectiveness of pre-match training for mentors is mixed </li>...
Conclusions <ul><li>This framework is first step toward a systematic approach for integrating and evaluating evidence rega...
ACTION Dialogue <ul><li>How can you incorporate these research  </li></ul><ul><li>findings into your program?  </li></ul><...
Share RESOURCES <ul><li>MPM Training  www.mentoringworks.org/traininginstitute </li></ul><ul><li>Web sites & PDFs  www.del...
MENTOR/National Mentoring Partnership <ul><li>The leader in expanding the power of mentoring to millions of young American...
Search Institute <ul><li>Provides leadership, information, and resources to promote positive healthy development of youth....
Northwest Regional Educational Laboratory - National Mentoring Center <ul><li>National organization that provides training...
THANK YOU! <ul><li>Next webinar is Wednesday, April 1st;  12-1pm CST </li></ul><ul><li>Issue 3:  Program Staff in Youth Me...
Upcoming SlideShare
Loading in...5
×

Research In Action #2

471

Published on

Research In Action Webinar - March 4, 2009 - with Dr. David DuBois

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
471
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
40
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Research In Action #2

    1. 1. RESEARCH IN ACTION ISSUE 2 Effectiveness of Mentoring Program Practices
    2. 2. Research In Action: Overview of Series Last year, MENTOR released the National Agenda for Action: How to Close America’s Mentoring Gap . Representing the collective wisdom of the mentoring field, the Agenda articulates five key strategies and action items necessary to move the field forward and truly close the mentoring gap. In an effort to address one of these critical strategies—elevating the role of research—MENTOR created the Research and Policy Council , an advisory group composed of the nation’s leading mentoring researchers, policymakers, and practitioners. In September 2006, MENTOR convened the first meeting of the Research and Policy Council with the goal of increasing the connection and exchange of ideas among practitioners, policymakers, and researchers to strengthen the practice of youth mentoring. The Research in Action series is the first product to evolve from the work of the Council—taking current mentoring research and translating it into useful, user-friendly materials for mentoring practitioners.
    3. 3. Research In Action Issues: Issue 1: Mentoring: A Key Resource for Promoting PYD Issue 2: Effectiveness of Mentoring Program Practices Issue 3: Program Staff in Youth Mentoring Programs Issue 4: Fostering Close and Effective Relationships Issue 5: Why Youth Mentoring Relationships End Issue 6: School-Based Mentoring Issue 7: Cross-Age Peer Mentoring Issue 8: Mentoring Across Generations: Engaging Age 50+ Adults Issue 9: Youth Mentoring: Do Race and Ethnicity Really Matter? Issue 10: Mentoring: A Promising Intervention for Children of Prisoners
    4. 4. Using the Research In Action Series <ul><li>Research: a peer-reviewed article, written by a leading researcher, summarizing the latest research available on the topic and its implications for the field; </li></ul><ul><li>Action: a tool, activity, template, or resource, created by MENTOR, with concrete suggestions on how practitioners can incorporate the research findings into mentoring programs; and </li></ul><ul><li>Resources: a list of additional resources on the topic for further research. </li></ul>
    5. 5. MPM Webinar Series <ul><li>Review RESEARCH presented in peer-reviewed article </li></ul><ul><li>ACTION dialogue – how can you incorporate the research findings into your program? </li></ul><ul><li>Share RESOURCES with others in the field </li></ul>
    6. 6. Issue 2 – Effectiveness of Mentoring Program Practices <ul><li>David L. DuBois, Ph.D., University of Illinois at Chicago </li></ul>
    7. 7. Issue 2 - Overview <ul><li>Introduction </li></ul><ul><li>Conceptual and Methodological Issues </li></ul><ul><li>Framework for Summarizing and Evaluating Research on Mentoring Program Practice Effectiveness </li></ul><ul><li>Research on Program Practices </li></ul><ul><li>Pre-Match Training </li></ul>
    8. 8. Mentoring Objective <ul><li>The central objective of any youth mentoring program is to establish and support beneficial relationships between youth and their identified mentors. </li></ul>
    9. 9. High-Quality Programs Establish and Support Relationships that are: <ul><li>Effective </li></ul><ul><li>Safe </li></ul><ul><li>Efficient </li></ul><ul><li>Sustainable/growth-oriented </li></ul>
    10. 10. Effectiveness vs. Quality <ul><li>Program Effectiveness </li></ul><ul><li>Impacts on mentoring relationship quality </li></ul><ul><li>Youth outcomes </li></ul><ul><li>Program Quality </li></ul><ul><li>Safety </li></ul><ul><li>Efficiency </li></ul><ul><li>Sustainability </li></ul>
    11. 11. Program Effectiveness Problem DuBois, Holloway, Valentine, & Cooper, 2002
    12. 12. Evidence-Based Practice <ul><li>Using research to identify practices that reliably serve to enhance program effectiveness </li></ul>
    13. 13. Mentoring Best Practices: Circles of Evidence Best Practices * Includes evidence obtained from sources other than formal research, such as client satisfaction surveys, program participant outcomes, and community demographic trends. Research * Professional Expertise & Experience *Local Resources & Needs *Client/Stakeholder Preferences & Beliefs
    14. 14. Conceptual and Methodological Issues <ul><li>Effects of any practice on program effectiveness will be dependent on a range of other factors </li></ul><ul><li>“Local adaptation-fidelity debate” </li></ul><ul><li>Clear guidelines/benchmarks needed to assess presence or absence of evidence-based practices </li></ul>
    15. 15. Summarizing & Evaluating Mentoring Research <ul><li>Evidence Framework </li></ul><ul><li>Levels of Evidence </li></ul><ul><li>Strength of Evidence </li></ul><ul><li>Sources of Evidence </li></ul>
    16. 16. Levels of evidence: Level 1 – isolate mentoring program practice of interest (POI) Level 2 –less precise comparisons in which the POI is not isolated Level 3 – qualitative research
    17. 17. Strength of Evidence <ul><li>Study Design </li></ul><ul><li>Methodological Rigor </li></ul><ul><ul><li>Sample size and representativeness </li></ul></ul><ul><ul><li>Reliability and validity of measures </li></ul></ul><ul><ul><li>Appropriateness of data analytic procedures and statistical inferences </li></ul></ul>
    18. 18. Sources of Evidence <ul><li>BOTH formal research literature and internal analyses of information by mentoring programs are potentially relevant to evaluating the effectiveness of a practice. </li></ul>
    19. 19. Applying the Evidence Framework <ul><li>Identify research relevant to the POI </li></ul><ul><li>Categorize each study according to level of evidence </li></ul><ul><li>Summarize positive, null and negative findings </li></ul><ul><li>Synthesize available evidence and make a judgment regarding the practice’s effectiveness </li></ul>
    20. 20. Research on Program Practices <ul><li>Descriptive Studies </li></ul><ul><li>Highlight important trends that merit further investigation </li></ul><ul><li>Provide information concerning the extent to which evidence-based practices are being adopted and adapted </li></ul>MENTORING IN AMERICA 2005
    21. 21. Researching Program Practices <ul><li>Typical program evaluation is designed to estimate cumulative effects that are produced by the multiple practices that comprise a given program rather than the unique contribution of any one practice </li></ul>
    22. 22. Meta-Analysis (DuBois et al., 2002) <ul><li>Program practices predictive of stronger positive effects on youth outcomes </li></ul>Structured activities for mentors and youth Ongoing (post-match) training for mentors Clearly established expectations for frequency of mentor-youth contact Using mentors with backgrounds in helping roles or professions Use of community settings for mentoring Procedures for systematic monitoring of program implementation Support for parent involvement
    23. 23. Pre-Match Training <ul><li>Available evidence regarding the effectiveness of pre-match training for mentors is mixed </li></ul><ul><li>Application of the evidence framework could yield collective informed judgments regarding the likely effectiveness of pre-match training, recommended benchmarks, and strength of available evidence </li></ul>
    24. 24. Conclusions <ul><li>This framework is first step toward a systematic approach for integrating and evaluating evidence regarding the effectiveness of different mentoring program practices </li></ul>These considerations underscore the importance of coupling application and ongoing refinement of the proposed framework with intensified efforts to strengthen the available evidence base.
    25. 25. ACTION Dialogue <ul><li>How can you incorporate these research </li></ul><ul><li>findings into your program? </li></ul><ul><li>How can we strengthen the available evidence base? </li></ul><ul><ul><ul><li>PROGRAM EVALUATION </li></ul></ul></ul><ul><ul><ul><li>POLICY BRIEF </li></ul></ul></ul>
    26. 26. Share RESOURCES <ul><li>MPM Training www.mentoringworks.org/traininginstitute </li></ul><ul><li>Web sites & PDFs www.delicious.com/traininginstitute </li></ul><ul><li>This presentation & others www.slideshare.net/traininginstitute </li></ul>
    27. 27. MENTOR/National Mentoring Partnership <ul><li>The leader in expanding the power of mentoring to millions of young Americans who want and need adult mentors. </li></ul><ul><ul><li>The Elements of Effective Practice™ </li></ul></ul><ul><ul><li>How to Build a Successful Mentoring Program Using the Elements of Effective Practice™ </li></ul></ul><ul><ul><li>How to Select a Survey to Assess Your Adult-Youth Mentoring Program </li></ul></ul>
    28. 28. Search Institute <ul><li>Provides leadership, information, and resources to promote positive healthy development of youth. </li></ul><ul><li>What's Working? Tools for Evaluating Your Mentoring Program </li></ul><ul><ul><ul><li>Now available as a free download </li></ul></ul></ul>
    29. 29. Northwest Regional Educational Laboratory - National Mentoring Center <ul><li>National organization that provides training and technical assistance to youth mentoring programs. </li></ul><ul><ul><li>Frequently Asked Questions About Research and Evaluation </li></ul></ul><ul><ul><li>Measuring the Quality of Mentor Youth Relationships: A Tool for Mentoring Programs </li></ul></ul>
    30. 30. THANK YOU! <ul><li>Next webinar is Wednesday, April 1st; 12-1pm CST </li></ul><ul><li>Issue 3: Program Staff in Youth Mentoring Programs: Qualifications, Training, and Retention   featuring Thomas E. Keller, Ph.D. ! </li></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×