Preparation for  Outcome Monitoring
Presenters:  1. Jeff Wagers:  Tx DSHS Program/ Training Specialist 2. Ken Ripperger-Suhler, Ph D: University of Texas at A...
Introductions <ul><li>Presenters </li></ul><ul><li>Audience </li></ul><ul><li>Agenda </li></ul><ul><li>Objectives </li></ul>
What is an Outcome? <ul><li>The resulting changes in the client during or after a client attends an intervention.  </li></ul>
Brainstorm-Outcomes! <ul><li>What outcomes would you like to see in your intervention? </li></ul><ul><li>Each intervention...
Agenda <ul><li>Review Basics of Interventions and OM </li></ul><ul><li>Walk through a sample intervention from logic model...
Objectives <ul><li>Review basics of interventions and OM </li></ul><ul><li>Discuss parameters and preparation for OM in Su...
Evidence-Based Interventions (EBI) <ul><li>Clearly defined audience, goals & objectives </li></ul><ul><li>Based in behavio...
Evidence-Based Interventions (EBI) <ul><li>Individual-Level Interventions: CRCS </li></ul><ul><li>Group Level Intervention...
Key Terms <ul><li>Internal Logic </li></ul><ul><li>Core elements </li></ul><ul><li>Fidelity </li></ul><ul><li>Behavioral D...
Behavioral Determinants; Factors Influencing Behavior (FIBs) <ul><li>The “why” </li></ul><ul><li>Reasons a  client or popu...
Program Evaluation <ul><li>What is evaluation? </li></ul><ul><li>Why evaluate? </li></ul>
Program Evaluation <ul><li>Systematic, consistent collection of data </li></ul><ul><li>Shows program effectiveness </li></...
What is Process?  <ul><li>“ How” you provide your intervention  </li></ul><ul><li>Fidelity to the intervention </li></ul><...
Process Questions <ul><li>Is the location accessible? </li></ul><ul><li>Is the program implemented as designed? </li></ul>...
What is an Outcome? <ul><li>The resulting changes in the client during or after a client attends an intervention.  </li></ul>
Outcome Questions <ul><li>Did the client  </li></ul><ul><ul><li>Learn anything? </li></ul></ul><ul><ul><li>Improve their s...
Outcomes measure: <ul><li>S  - skills </li></ul><ul><li>K  - knowledge </li></ul><ul><li>A  - attitude </li></ul><ul><li>B...
Logic Models
Problem Statement Risk Behaviors Influencing Factors Intervention/ Activities Outcomes Immediate  Intermediate <ul><li>Out...
Factors Influencing Behavior STD rates are  high among young Hispanic MSM Problem Statement Young Hispanic  MSM don’t use ...
Intervention Activities STD rates are  high among young Hispanic  MSM Problem Statement Young Hispanic  MSM don’t use cond...
Immediate and Intermediate Outcomes Young Hispanic MSM lack skills to properly use condoms Young Hispanic MSM don’t  know ...
Problem Statement Risk Behaviors Influencing Factors Intervention/ Activities Outcomes Immediate  Intermediate Impacts   I...
Outcome Monitoring Data Collection And  Analysis
Steps to Outcome Monitoring <ul><li>Develop *SMART outcome objectives </li></ul><ul><ul><li>Specific, Measurable, Appropri...
Example of Process Objective <ul><li>Process objective : “Clients will complete SISTA” </li></ul><ul><li>Revised :  “By De...
Example of Outcome Objective <ul><li>Outcome objective :  “Clients who complete SISTA will show an increase self efficacy”...
Ways to Collect Outcomes
Collect Outcomes <ul><li>Skills </li></ul><ul><ul><li>Correct condom use, needle-bleaching </li></ul></ul><ul><ul><li>Nego...
Measuring Skills <ul><li>Demonstrations and Practice </li></ul><ul><li>Role Plays </li></ul>
Demonstrations <ul><li>Examples: </li></ul><ul><ul><li>Condom  Use Skills  </li></ul></ul><ul><ul><li>Needle Bleaching Ski...
Role Plays   <ul><li>Examples: </li></ul><ul><ul><li>Condom or bleaching negotiation skills </li></ul></ul><ul><ul><li>Sel...
Types of Questions <ul><li>Open Ended </li></ul><ul><li>Closed Ended </li></ul><ul><ul><li>True/False or Yes/No </li></ul>...
Question Answer Options MC Multiple Choice OE Open Ended FI Frequency Indicators S Scale  FB Fill in the Blank Y/N Yes/No ...
Sample Questions <ul><li>Perceived susceptibility </li></ul><ul><ul><li>“Given my current behavior, I am worried that I mi...
Sample Questions <ul><li>Expected outcome related to condom use </li></ul><ul><ul><li>“I will get HIV from my main partner...
Steps to Outcome Monitoring   <ul><li>Gathering survey data </li></ul><ul><ul><li>Individual interviews </li></ul></ul><ul...
Putting It All Together
Example-VOICES/VOCES <ul><li>VOICES/ VOCES is a one-session group level intervention for HRH men and women in which groups...
Logic Model Increased Condom  Use Increased Condom Use Knowledge Increased Condom Use Self-Efficacy Increased Condom Use I...
<ul><li>At least 40% of clients will demonstrate greater knowledge of condom usage by identifying on a written test at the...
Survey Items (Knowledge of  Condom Use) 3 2 1 2. You should unroll a condom before putting it on a man’s penis. 3 2 1 1. V...
Spreadsheet for storing data yes  yes  -  GAY  M  27  H  A03  yes  yes  -  BI  M  25  AA  A02    yes yes -  STR  M  30    ...
Frequencies P r e & Post 0 (0%) 20 (100%) 0 (0%) (POST) 0 (0%) 0 (0%) 20 (100%) (PRE) Don’t Know (%) No (%) Yes (%) Item  ...
Vaseline and lotion are good lubricants for condoms/rubbers. 20 (0%) 0 20 0 Total 0 (0%) 0 0 0 Don’t Know 0 (0%) 0 0 0 No ...
Frequencies Pre & Post 0 (0%) 0 (0%) 20 (100%) (POST) 0 (0%) 0 (0%) 20 (100%) (PRE) Don’t Know (%) No (%) Yes (%) Item  Yo...
You should unroll a condom before putting it on a man’s penis. 20 (100%) 0 (0%) 0 (0%) 20 (100%) Total 0 (0%) 0 (0%) 0 (0%...
Table 1. Change in Knowledge Scores from Pre to Post 0 0 0 0 2. You should unroll a condom before putting it on a man’s pe...
Outcome Monitoring:  VOICES/VOCES Report <ul><li>Condom Use Knowledge </li></ul><ul><li>The results of cross tab analysis ...
Planning for Outcome   Monitoring
Planning for Outcome Monitoring   <ul><ul><li>Readiness for Outcome Monitoring </li></ul></ul><ul><li>Done on an individua...
DSHS Parameters for Outcome Monitoring <ul><li>Skills-building, knowledge, behavioral determinants ( AKA FIBs) </li></ul><...
Some Differences in  Data Collection…. <ul><li>CRCS </li></ul><ul><ul><li>Intermediate and Immediate Outcomes </li></ul></...
Some Differences in  Data Collection…. <ul><li>Group Level </li></ul><ul><ul><li>Immediate Outcomes </li></ul></ul><ul><ul...
Some Differences in  Data Collection…. <ul><li>Community Level:   2 components </li></ul><ul><ul><li>Group Component- meas...
Some Differences in  Data Collection…. <ul><li>Community Level:  </li></ul><ul><ul><li>Community Component-measured second...
Outcome Monitoring Surveys <ul><li>Ask 3-4 questions per outcome  </li></ul><ul><li>10-20 questions per survey </li></ul><...
Facilitated Group Activity 1 <ul><li>CLI: Discuss group and community level components of intervention </li></ul><ul><li>G...
Guided Group Discussion <ul><li>Ways to Collect Outcomes </li></ul><ul><li>Giving Instructions </li></ul><ul><li>Oral Inte...
Activity 2: Data Input and Analysis <ul><li>Input sample data into a spreadsheet </li></ul><ul><ul><li>Sample data is loca...
Next Steps Per intervention, brainstorm challenges and solutions to Outcome Monitoring
<ul><li>Final Questions? </li></ul><ul><li>Please complete your course evaluations   </li></ul><ul><li>Thank you for your ...
Contact information <ul><li>Jeff Wagers, TxDSHS </li></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>512-533-3...
Upcoming SlideShare
Loading in...5
×

Preparation For Outcome Monitoring 2010

1,482

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,482
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
32
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Evaluation is: A systematic collection of data and information Why Evaluate? Assists in determining program effectiveness Must be done consistently Informs you for future program improvement 4. Helps make you accountable 5. Social justice
  • Participants will be completing a logic model for their intervention. They have already begun this process throughout the last two days. Refer participants to the newsprint and activity “Linking FIBs to Interventions” that they completed the previous day. Participants will pick one risk behavior to address in their expanded logic model. Participants are encouraged to complete the six-column logic model sheet before transposing it to newsprint. Instructions for doing this activity are on the following two pages. Participants get back to the groups they began the training with. Refer to the participant handout #9 “Logic Model” (with six columns) and pass out newsprint. 45 minutes-1 hour to complete the logic model.
  • Answer any final questions! Address final Parking Lot issues if not yet addressed. Mention that any tools we have used in this training can be located on the HIV/STD program website. Let them know that you will be available for technical assistance if they have any further questions about outcome monitoring. Jeff Wagers is also available for technical assistance (his name and phone number are listed in the manual). DSHS field operations and regional staff are also available for TA. Remind participants to complete the evaluation forms and leave them on the table before they leave. Thank participants for their attention and participation! Manual pg. 60 Trainers write their names and phone numbers/ email addresses on newsprint for Evaluations and TA, or can hand out business cards.
  • Preparation For Outcome Monitoring 2010

    1. 1. Preparation for Outcome Monitoring
    2. 2. Presenters: 1. Jeff Wagers: Tx DSHS Program/ Training Specialist 2. Ken Ripperger-Suhler, Ph D: University of Texas at Austin, Researcher
    3. 3. Introductions <ul><li>Presenters </li></ul><ul><li>Audience </li></ul><ul><li>Agenda </li></ul><ul><li>Objectives </li></ul>
    4. 4. What is an Outcome? <ul><li>The resulting changes in the client during or after a client attends an intervention. </li></ul>
    5. 5. Brainstorm-Outcomes! <ul><li>What outcomes would you like to see in your intervention? </li></ul><ul><li>Each intervention will report </li></ul>
    6. 6. Agenda <ul><li>Review Basics of Interventions and OM </li></ul><ul><li>Walk through a sample intervention from logic model to analysis </li></ul><ul><li>Readiness to do Outcome Monitoring </li></ul><ul><li>Small Group Breakout by Intervention </li></ul><ul><li>Administering surveys in a trusting atmosphere </li></ul><ul><li>Practice inputting survey data </li></ul><ul><li>Data analysis and Implications </li></ul><ul><li>Final questions and wrap-up </li></ul>
    7. 7. Objectives <ul><li>Review basics of interventions and OM </li></ul><ul><li>Discuss parameters and preparation for OM in Summer 2010 </li></ul><ul><li>Examine and assess OM pre/post surveys by intervention </li></ul><ul><li>Learn how to input and analyze data </li></ul><ul><li>Brainstorm challenges and solutions to OM </li></ul><ul><li>Reassess readiness to do OM </li></ul>
    8. 8. Evidence-Based Interventions (EBI) <ul><li>Clearly defined audience, goals & objectives </li></ul><ul><li>Based in behavioral theory </li></ul><ul><li>Focused on specific risk behavior </li></ul><ul><li>Could be an individual, group, or community level intervention </li></ul>
    9. 9. Evidence-Based Interventions (EBI) <ul><li>Individual-Level Interventions: CRCS </li></ul><ul><li>Group Level Interventions </li></ul><ul><li>Community Level Interventions </li></ul>
    10. 10. Key Terms <ul><li>Internal Logic </li></ul><ul><li>Core elements </li></ul><ul><li>Fidelity </li></ul><ul><li>Behavioral Determinants/ Factors that Influence Behavior/ Risk Factors </li></ul><ul><li>Tailoring </li></ul><ul><li>Adapting </li></ul><ul><li>Reinventing </li></ul>
    11. 11. Behavioral Determinants; Factors Influencing Behavior (FIBs) <ul><li>The “why” </li></ul><ul><li>Reasons a client or population is engaged in the risk behavior </li></ul><ul><li>Come from the client through: </li></ul><ul><ul><li>CRCS: asking clients questions </li></ul></ul><ul><ul><li>GLI, CLI: community assessment </li></ul></ul><ul><li>Interventions focus on the BD/FIB as well as the behavior </li></ul>
    12. 12. Program Evaluation <ul><li>What is evaluation? </li></ul><ul><li>Why evaluate? </li></ul>
    13. 13. Program Evaluation <ul><li>Systematic, consistent collection of data </li></ul><ul><li>Shows program effectiveness </li></ul><ul><li>Accountability </li></ul><ul><li>Program Improvement </li></ul><ul><li>Increases Capacity </li></ul><ul><li>Defines parameters </li></ul><ul><li>Identifies unmet needs </li></ul><ul><li>Process vs. outcomes </li></ul>
    14. 14. What is Process? <ul><li>“ How” you provide your intervention </li></ul><ul><li>Fidelity to the intervention </li></ul><ul><li>Quality Assurance issues </li></ul><ul><li>Skill level of agency staff </li></ul><ul><li>Numbers (who attended, how many sessions, # of target population, etc) </li></ul>
    15. 15. Process Questions <ul><li>Is the location accessible? </li></ul><ul><li>Is the program implemented as designed? </li></ul><ul><li>Was the curriculum followed? </li></ul><ul><li>How many of the target population attended the sessions? </li></ul><ul><li>How well is the RRS able to keep the session focused on risk behavior? </li></ul><ul><li>Was staff culturally competent? </li></ul><ul><li>Others? </li></ul>
    16. 16. What is an Outcome? <ul><li>The resulting changes in the client during or after a client attends an intervention. </li></ul>
    17. 17. Outcome Questions <ul><li>Did the client </li></ul><ul><ul><li>Learn anything? </li></ul></ul><ul><ul><li>Improve their skills? </li></ul></ul><ul><ul><li>Change their behavior? </li></ul></ul><ul><ul><li>Realize they are at risk of getting HIV? </li></ul></ul>
    18. 18. Outcomes measure: <ul><li>S - skills </li></ul><ul><li>K - knowledge </li></ul><ul><li>A - attitude </li></ul><ul><li>B - belief </li></ul><ul><li>B - behavior </li></ul>
    19. 19. Logic Models
    20. 20. Problem Statement Risk Behaviors Influencing Factors Intervention/ Activities Outcomes Immediate Intermediate <ul><li>Outcomes </li></ul><ul><li>Immediate Outcomes: Immediate </li></ul><ul><li>results of the intervention, such as </li></ul><ul><li>changes in knowledge, attitudes, </li></ul><ul><li>beliefs, and skills. </li></ul><ul><li>Increased perception of HIV risk </li></ul><ul><li>- Increased condom use skills </li></ul><ul><li>- Increased self efficacy to use condoms </li></ul><ul><li>Intermediate Outcomes: Intervention </li></ul><ul><li>Results that occur some time after the </li></ul><ul><li>intervention is completed, such as </li></ul><ul><li>changes in behaviors, policies, and </li></ul><ul><li>environmental barriers. </li></ul><ul><li>Increased condom use </li></ul><ul><li>Decreased number of partners </li></ul>
    21. 21. Factors Influencing Behavior STD rates are high among young Hispanic MSM Problem Statement Young Hispanic MSM don’t use condoms Young Hispanic MSM have multiple partners Risk Behaviors Young Hispanic MSM lack skills to properly use condoms Young Hispanic MSM don’t know how to negotiate condom use with their partners Influencing Factors It’s the norm for young Hispanic MSM to have a lot of sex partners
    22. 22. Intervention Activities STD rates are high among young Hispanic MSM Problem Statement Young Hispanic MSM don’t use condoms Risk Behaviors Young Hispanic MSM lack skills to properly use condoms Young Hispanic MSM don’t know how to negotiate condom use with their partners Safe-sex negotiation scenarios & role plays Self-assertion skills training Intervention Activities Influencing Factors Young Hispanic MSM have multiple partners Condom skills demonstration and practice
    23. 23. Immediate and Intermediate Outcomes Young Hispanic MSM lack skills to properly use condoms Young Hispanic MSM don’t know how to negotiate condom use with their partners Influencing Factors Safe-sex negotiation scenarios & role plays Self-assertion skills training Intervention Activities Immediate Outcomes Young Hispanic MSM know how to negotiate condom use with their partners Young Hispanic MSM use condoms Intermediate Outcomes Condom demo and practice Young Hispanic MSM know how to use condoms
    24. 24. Problem Statement Risk Behaviors Influencing Factors Intervention/ Activities Outcomes Immediate Intermediate Impacts Impacts Impacts: Long-term results of one or more interventions over time, such as changes in HIV infection, morbidity, and mortality - Decreased HIV rates
    25. 25. Outcome Monitoring Data Collection And Analysis
    26. 26. Steps to Outcome Monitoring <ul><li>Develop *SMART outcome objectives </li></ul><ul><ul><li>Specific, Measurable, Appropriate, Realistic, Time-Based </li></ul></ul><ul><li>Develop/ review/ modify surveys or other methods that collect outcomes </li></ul><ul><li>Gathering survey/ method data </li></ul><ul><li>Input data into spreadsheets </li></ul><ul><li>Analyze and report data (Outcome Monitoring) </li></ul><ul><li>Make program adjustments if needed </li></ul>
    27. 27. Example of Process Objective <ul><li>Process objective : “Clients will complete SISTA” </li></ul><ul><li>Revised : “By December 31, 2010, 50 African-American women will complete SISTA as documented by client data system.” </li></ul>
    28. 28. Example of Outcome Objective <ul><li>Outcome objective : “Clients who complete SISTA will show an increase self efficacy” </li></ul><ul><li>Revised : “75% of participants who complete SISTA will increase self-efficacy to correctly put on a condom as measured by pre/post survey.” </li></ul>
    29. 29. Ways to Collect Outcomes
    30. 30. Collect Outcomes <ul><li>Skills </li></ul><ul><ul><li>Correct condom use, needle-bleaching </li></ul></ul><ul><ul><li>Negotiation, Communication, Assertiveness </li></ul></ul><ul><li>Questions </li></ul><ul><ul><li>Knowledge </li></ul></ul><ul><ul><li>Attitudes </li></ul></ul><ul><ul><li>Beliefs </li></ul></ul><ul><ul><li>Behaviors </li></ul></ul>
    31. 31. Measuring Skills <ul><li>Demonstrations and Practice </li></ul><ul><li>Role Plays </li></ul>
    32. 32. Demonstrations <ul><li>Examples: </li></ul><ul><ul><li>Condom Use Skills </li></ul></ul><ul><ul><li>Needle Bleaching Skills </li></ul></ul><ul><li>Checklists </li></ul><ul><ul><li>Consistent </li></ul></ul><ul><ul><li>Provide “clean” pre-test </li></ul></ul><ul><ul><li>Demonstrate correct condom use </li></ul></ul><ul><ul><li>Practice </li></ul></ul><ul><ul><li>Next session, do a post-test and compare results </li></ul></ul>
    33. 33. Role Plays <ul><li>Examples: </li></ul><ul><ul><li>Condom or bleaching negotiation skills </li></ul></ul><ul><ul><li>Self Assertiveness skills </li></ul></ul><ul><li>Observations: </li></ul><ul><ul><li>Consistent </li></ul></ul><ul><ul><li>Provide “clean” pre-role play (or reverse role play) </li></ul></ul><ul><ul><li>Demonstrate correct method </li></ul></ul><ul><ul><li>Practice </li></ul></ul><ul><ul><li>Next session, do a post-role play and compare results </li></ul></ul>
    34. 34. Types of Questions <ul><li>Open Ended </li></ul><ul><li>Closed Ended </li></ul><ul><ul><li>True/False or Yes/No </li></ul></ul><ul><ul><li>Fill in the Blank </li></ul></ul><ul><ul><li>Likert Scale </li></ul></ul><ul><ul><li>Frequency Indicator </li></ul></ul><ul><ul><li>Multiple Choice </li></ul></ul>
    35. 35. Question Answer Options MC Multiple Choice OE Open Ended FI Frequency Indicators S Scale FB Fill in the Blank Y/N Yes/No T/F True/False Answer Code Answer Option
    36. 36. Sample Questions <ul><li>Perceived susceptibility </li></ul><ul><ul><li>“Given my current behavior, I am worried that I might get HIV” (T/F) (Y/N) </li></ul></ul><ul><ul><li>“Given my current behavior, how worried am I that I might get HIV? (Scale) </li></ul></ul><ul><ul><li>1 2 3 4 5 </li></ul></ul><ul><ul><li>Not worried Very worried </li></ul></ul>
    37. 37. Sample Questions <ul><li>Expected outcome related to condom use </li></ul><ul><ul><li>“I will get HIV from my main partner without using a condom” (T/F) (Y/N) </li></ul></ul><ul><ul><li>“How likely is it that I will get HIV from my main partner without using a condom?” </li></ul></ul><ul><ul><li>1 2 3 4 5 </li></ul></ul><ul><ul><li>Not likely at all Very likely </li></ul></ul>
    38. 38. Steps to Outcome Monitoring <ul><li>Gathering survey data </li></ul><ul><ul><li>Individual interviews </li></ul></ul><ul><ul><li>Written questionnaires </li></ul></ul><ul><ul><li>Focus groups </li></ul></ul><ul><li>Input data into spreadsheets </li></ul><ul><li>Analyze and report data (Outcome Monitoring) </li></ul><ul><li>Make program adjustments </li></ul>
    39. 39. Putting It All Together
    40. 40. Example-VOICES/VOCES <ul><li>VOICES/ VOCES is a one-session group level intervention for HRH men and women in which groups watch culturally appropriate videos of condom negotiation, condom use and assertive communication, and practice. </li></ul><ul><li>Participants role-play condom use scenarios and practice putting on condoms </li></ul>
    41. 41. Logic Model Increased Condom Use Increased Condom Use Knowledge Increased Condom Use Self-Efficacy Increased Condom Use Intention INTERMEDIATE OUTCOMES (BEHAVIOR) IMMEDIATE OUTCOMES ACTIVITIES INFLUENCING FACTORS Practice Condom Use; Distribute condoms View Videos; Condom Negotiation Role Plays RISK BEHAVIORS Unprotected Sex Condom Use Knowledge Condom Use Self-Efficacy Condom Use Intention
    42. 42. <ul><li>At least 40% of clients will demonstrate greater knowledge of condom usage by identifying on a written test at the end of the session the correct answer (yes or no) with regard to the following knowledge questions : </li></ul><ul><li>a. Are specific lubricants appropriate for use with condoms; </li></ul><ul><li>b. Whether or not to unroll a condom before putting it on a man’s penis. </li></ul>Objective (Knowledge of Condom Use)
    43. 43. Survey Items (Knowledge of Condom Use) 3 2 1 2. You should unroll a condom before putting it on a man’s penis. 3 2 1 1. Vaseline and lotion are good lubricants for condoms/rubbers. Don’t Know No Yes
    44. 44. Spreadsheet for storing data yes  yes  -  GAY  M  27  H  A03  yes  yes  -  BI  M  25  AA  A02    yes yes -  STR  M  30    AA A01 Pre ConKnow 2 Pre ConKnow 1 HIV Status Sexual Orient Gender Age at intake Race/ Ethnic Client ID
    45. 45. Frequencies P r e & Post 0 (0%) 20 (100%) 0 (0%) (POST) 0 (0%) 0 (0%) 20 (100%) (PRE) Don’t Know (%) No (%) Yes (%) Item Vaseline and lotion are good lubricants for condoms/rubbers.
    46. 46. Vaseline and lotion are good lubricants for condoms/rubbers. 20 (0%) 0 20 0 Total 0 (0%) 0 0 0 Don’t Know 0 (0%) 0 0 0 No Responses 20 (100%) 0 20 0 Yes Pre Total Don’t Know No Yes Post Responses
    47. 47. Frequencies Pre & Post 0 (0%) 0 (0%) 20 (100%) (POST) 0 (0%) 0 (0%) 20 (100%) (PRE) Don’t Know (%) No (%) Yes (%) Item You should unroll a condom before putting it on a man’s penis.
    48. 48. You should unroll a condom before putting it on a man’s penis. 20 (100%) 0 (0%) 0 (0%) 20 (100%) Total 0 (0%) 0 (0%) 0 (0%) 0 (0%) Don’t Know 0 (0%) 0 (0%) 0 (0%) 0 (0%) No Responses 20 (100%) 0 (0%) 0 (0%) 20 (100%) Yes Pre Total Don’t Know No Yes Post Responses
    49. 49. Table 1. Change in Knowledge Scores from Pre to Post 0 0 0 0 2. You should unroll a condom before putting it on a man’s penis. 100 0 100 100 1. Vaseline and lotion are good lubricants for condoms/rubbers. % change to correct answers % changed to incorrect % changed to correct % correct post Item
    50. 50. Outcome Monitoring: VOICES/VOCES Report <ul><li>Condom Use Knowledge </li></ul><ul><li>The results of cross tab analysis find that </li></ul><ul><li>100% of clients showed increased knowledge of appropriate lubricants for condom use. </li></ul><ul><li>0% of clients showed increased knowledge regarding when to unroll a condom to put on a man’s penis. </li></ul>
    51. 51. Planning for Outcome Monitoring
    52. 52. Planning for Outcome Monitoring <ul><ul><li>Readiness for Outcome Monitoring </li></ul></ul><ul><li>Done on an individual agency basis </li></ul><ul><li>Stable implementation </li></ul><ul><li>Data collection instruments developed by Ken with agency input </li></ul><ul><li>Outcomes may change over time </li></ul>
    53. 53. DSHS Parameters for Outcome Monitoring <ul><li>Skills-building, knowledge, behavioral determinants ( AKA FIBs) </li></ul><ul><li>BD/ FIBs identified in your intervention based on research </li></ul><ul><li>Pre/post survey (unless an agency wants to do observations) </li></ul><ul><li>Based on logic model of intervention </li></ul><ul><li>3-4 outcomes for GLI </li></ul>
    54. 54. Some Differences in Data Collection…. <ul><li>CRCS </li></ul><ul><ul><li>Intermediate and Immediate Outcomes </li></ul></ul><ul><ul><li>Use questions from behavioral assessment </li></ul></ul><ul><ul><li>No standardized survey </li></ul></ul><ul><ul><li>One initial assessment, up to two reassessments </li></ul></ul><ul><ul><li>Driven by prevention plan and goals </li></ul></ul><ul><ul><li>Designated outcomes </li></ul></ul><ul><ul><li>Pre/post done on same individual </li></ul></ul><ul><ul><li>Percentages for outcomes 15-50% </li></ul></ul><ul><ul><li>Not all outcomes will apply to all individuals </li></ul></ul>
    55. 55. Some Differences in Data Collection…. <ul><li>Group Level </li></ul><ul><ul><li>Immediate Outcomes </li></ul></ul><ul><ul><li>Pre/post tests </li></ul></ul><ul><ul><li>Standardized survey </li></ul></ul><ul><ul><li>3-4 outcomes individualized per intervention </li></ul></ul><ul><ul><li>Pre/post done on same individual </li></ul></ul><ul><ul><li>Percentages for outcomes 40-75% </li></ul></ul><ul><ul><li>Can measure any SKABB </li></ul></ul>
    56. 56. Some Differences in Data Collection…. <ul><li>Community Level: 2 components </li></ul><ul><ul><li>Group Component- measured first </li></ul></ul><ul><ul><ul><li>2-4 outcomes </li></ul></ul></ul><ul><ul><ul><li>Will measure M-Groups or training of peer volunteers </li></ul></ul></ul><ul><ul><ul><li>Standardized survey </li></ul></ul></ul><ul><ul><ul><li>Pre/post; percentages 50-80% </li></ul></ul></ul><ul><ul><ul><li>Knowledge, skill-level, able to communicate very important </li></ul></ul></ul><ul><ul><ul><li>Will be collected first stage </li></ul></ul></ul>
    57. 57. Some Differences in Data Collection…. <ul><li>Community Level: </li></ul><ul><ul><li>Community Component-measured second </li></ul></ul><ul><ul><ul><li>2-4 Outcomes </li></ul></ul></ul><ul><ul><ul><li>Done during community assessment </li></ul></ul></ul><ul><ul><ul><li>Pre/Post tests; Venue-based </li></ul></ul></ul><ul><ul><ul><li>Mainly measures knowledge, attitudes, beliefs, community norms </li></ul></ul></ul><ul><ul><ul><li>Percentages up to 15% </li></ul></ul></ul><ul><ul><ul><li>Gathered yearly </li></ul></ul></ul><ul><ul><ul><li>Will be collected in second stage </li></ul></ul></ul>
    58. 58. Outcome Monitoring Surveys <ul><li>Ask 3-4 questions per outcome </li></ul><ul><li>10-20 questions per survey </li></ul><ul><li>Can measure skills through survey </li></ul><ul><li>Knowledge can be a component </li></ul><ul><li>Include key outcomes of intervention, in line with behavioral determinants </li></ul><ul><li>Consistent data collection </li></ul>
    59. 59. Facilitated Group Activity 1 <ul><li>CLI: Discuss group and community level components of intervention </li></ul><ul><li>GLI: Discuss and review OM surveys by intervention </li></ul><ul><li>CRCS: Brainstorm problems and possible solutions to gathering OM data </li></ul>
    60. 60. Guided Group Discussion <ul><li>Ways to Collect Outcomes </li></ul><ul><li>Giving Instructions </li></ul><ul><li>Oral Interviews </li></ul><ul><li>Written Questions </li></ul><ul><li>Focus Groups </li></ul>
    61. 61. Activity 2: Data Input and Analysis <ul><li>Input sample data into a spreadsheet </li></ul><ul><ul><li>Sample data is located in packet </li></ul></ul><ul><ul><li>Input into your laptop using the sample spreadsheet (partner with someone if needed) </li></ul></ul><ul><li>Analyze the data </li></ul><ul><ul><li>Spreadsheet calculates the data </li></ul></ul><ul><ul><li>What can you determine from analysis? </li></ul></ul>
    62. 62. Next Steps Per intervention, brainstorm challenges and solutions to Outcome Monitoring
    63. 63. <ul><li>Final Questions? </li></ul><ul><li>Please complete your course evaluations </li></ul><ul><li>Thank you for your participation! </li></ul>Thank You!
    64. 64. Contact information <ul><li>Jeff Wagers, TxDSHS </li></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>512-533-3022 </li></ul></ul><ul><li>Ken Ripperger-Suhler, University of Texas </li></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>254-624-1028 </li></ul></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×