Metrics for Monitoring and Evaluating the Enhanced Comprehensive Prevention Plan
1. Tiffany West, DC Department of
Health, Strategic Information
Bureau
2. Monitor Burden of Disease
Develop and Implement scalable
Interventions
Ensure Quality Service Delivery
Assess Impact of Response
Assess Effectivness and Cost
Effectiveness
3. Routine Opt Out HIV Screening Health Communication and
HIV Screening in Non-Clinical Settings Social Marketing
PEP
Policy Review Integrated Hep, STD, TB,
Incorporating HIV Prevention HIV testing, partner
into HIV Care services, vaccination and
Scale up of STD Screening Guidelines treatment
Prevention of Perinatal Transmission Targeted use of HIV/STD
Integrated HIV/STD Partner Services
Behavioral Risk Reduction for HIV +
Surveillance Data
Retention and Re-engagement Focus on High Risk
Condom Distribution Negatives
Community Mobilization
EVALUATION
PCSI
4. Program Collaboration and Service Integration (PCSI) Process
Reduce the HIV, STD, TB, and hepatitis-
related morbidity and mortality
Increasing Access to Care and Reducing HIV-Related Achieving a More Coordinated
Reduce New HIV
Improving Health Outcomes for Disparities and Health National/Local Response to the
Infections
People Living with HIV Inequities HIV Epidemic
Immediately link people to
Reduce # of people Reduce HIV-related
continuous and coordinated Ensure there data driven
unaware of their HIV mortality in communities at
quality care when they are policies and programs
Status high risk for HIV infection
diagnosed with HIV
Adopt community-level Increase the coordination of
Reduce Impact of Risk Strengthen retention and recapture
approaches to reduce HIV HIV, STD, Hepatitis and
Behaviors among Target of HIV positive individuals into
infection in high-risk TB Prevention, care and
Populations HIV care
communities treatment programs (PCSI)
Intensify HIV prevention Increase the number and diversity
Reduce stigma and Improve fiscal and
efforts in the communities of available providers of clinical
discrimination against operational efficiencies and
where HIV is most care and related services for
people living with HIV accountability
heavily concentrated. people living with HIV.
Establish models to Support people living with HIV Assess effective, cost
Routinize HIV prevention,
more efficiently link
HIV-infected clients to with co-occurring health care and treatment as effectiveness and impact of
care conditions standard of care programs
Improve results and health Expand innovative
outcomes partnerships
Data to Drive Policies and Programs: Strategic Information
5. Care Information
Systems
Medicaid ADAP
HIV, STD TB,
Hepatitis
PEMS I: CTR
Laboratory Data, Surveillance
PEMS 2: HERR
Vital Statistics,
Disease Registries
Electronic Medical BRFSS, YRBS, Epidemiologic
Records, Hospital Other Population Data: NHBS, Local
Discharge Data
Studies Study Data, HPTN
065
6. Defining Indicators by Type (1)
Input Indicators: Measure the quantity, quality, and timeliness of
resources — human, financial and material, technological and
information — provided for an activity/project/program.
• # of FTEs
• Cost of Intervention
• Materials or In prints (social marketing)
• Change in legislation or rules governing HIV testing
Process Indicators: Measure the progress of activities in a
programme/project and the way these are carried out
• # of providers fully implementing routine opt out HIV
screening
• # of stakeholder meeting to discuss testing policy
7. Defining Indicators by Type (2)
Output Indicator: Measure the quantity, quality, and timeliness of
the intervention that are the result of an
activity/service/project/program
• # of HIV publically funded HIV tests
• # of Positives
• Proportion of positives linked to care within 3 months of
diagnosis
Outcome Indicators: Measure the quality and quantity of long-
term results generated by programme outputs (morbidity and
mortality)
• Decrease in AIDS Cases
• Reduction in incidence
• Change in behavior
8. # of providers conducting routine opt out
screening by site type
# of hospitals ED conducting routine screening
# of providers approached to expand testing
efforts
# of non clinical targeted testing sites
Description of Provider Population
# of providers doing social network testing
Number of social marketing Advertisements on
testing
9. Impact Indicators: HIV Testing
HIV Routine Opt-Out Testing Expansion
Publicly-Supported Rapid Tests, 2007-2010
120,000
Start of routine testing expansion
19.0% increase in
100,000 number of tests done 18.0%
27.3% increase in
number of tests done 14.7%
80,000
Non-Clinical
12.7% Clinical
60,000 68.4% increase in 82.0%
number of tests done
40,000 85.3%
40,000 18.2%
87.3%
20,000
20,000 81.8%
0
2004 2005 2006 2007 2008 2009
0
FY 2007 FY 2008 FY 2009 FY 2010
N=43,271 N=72,864 N=92,748 N=110,358
10. # of HIV Tests conducted by site type
(publically funded settings)
Positivity Rate by site type (publically
funded settings)
# of publically funded HIV tests by high
risk population
Positivity Rate by high risk population
Provider Population
Positivity rate among social network
providers and within social networks
11. 800
Start of routine testing expansion
700
600
Number of AIDS Cases
500
400 Men
Women
Total
300
200
100
0
2005 2006 2007 2008 2009
Year of Diagnosis
Confidential Source: DC Department of Health HIV/AIDS Epidemiology Update, 2009
12. Proportion of AIDS cases who are late testers
(eHARS)
Proportion of DC residents tested within the
last 12 months (BRFSS)
Number of new HIV cases diagnosed by
funded and non funded clinical providers
(eHARS)
HIV Prevalence among MSM, IDU, HET (NHBS)
Proportion of MSM, IDU, HET Unaware of HIV
Status (NHBS)
Proportion of MSM, IDU, HET tested in last 12
months (NHBS)
13. # and Type of Provider distributing condoms
(public and private sector)
Location of providers distribution condoms
Assess Client Population of Providers
Distribution Condoms (Population Served)
# and imprints of Social Marketing Messages
Related to Condoms
14.
15.
16. Condom use at last sex (overall)-BRFSS
Condom use at last vaginal sex (IDU, HET) –
NHBS
Condom use at last anal sex (MSM, IDU, HET) –
NHBS
Proportion of youth a condom at last sexual
intercourse among currently sexually active
youth -YRBS
17. Process Evaluation
• Challenge: Different Funding Streams, (CDC-Linkage, HRSA-
EIS)
• Align to create comprehensive linkage to care strategy
• One set of tools (PEMS Form 2 Plus)
• Preliminary Positive=Named Case/ False Positives <2%
# of providers funded for linkage programs (by provider and by
type of program)
# of preliminary positives by program by type
• Red Carpet Entry, Latino, Adolescent, Unaffiliated Linkage,
Department of Correction
# and Proportion of clients linked to care within 3 months by
linkage or navigation program type
# of confirmed positives by provider and linkage type
18. 1,114 1,222 1,226 999 726
100%
*2008 Clinical
80%
Site Partners,
77% Linkage
60%
Percent
40%
20%
0%
2005 2006 2007 2008 2009
Year of HIV Diagnosis
<3 Months 3-6 Months 6-12 Months >1 Year
18
19. Proportion of Ryan White clients with at least 2 outpatient
ambulatory visits in the last 12 months (Impact)
Proportion of new diagnosis remaining in care after 12 months (
Impact)
Proportion of people living with HIV receiving care primary and
support service( Impact )
# of providers funded for retention programs ( Process)
# of clients served through retention programs, by type (Process)
# of visits per program (Impact )
Proportion of clients served by retention programs that had at least
2 medical visits in the last 12 months (Impact )
20. Re-engagement
# of clients loss to follow up among funded
providers ( Impact)
# of clients contacted through loss to follow up
programs ( Impact )
# of appointments made through loss to follow
up programs (Impact)
Number of appointments kept through loss to
follow up programs ( Impact)
Average number of contacts required for
recapture by program type ( Impact )
21.
22. Purpose: To re-engage people living with HIV in care who
are ‘loss to care’
Define: Loss to care: Not in care for more than 6 months
Methods: Primary Medical Care Providers send list of
clients not seen in their clinics for greater than 6 months.
HAHSTA “matched” these lists to e-HARS, labs surveillance
and ADAP databases.
• Providers are given “yes” or “no” for those found to be in
care in another location
• 90 day “blitz”; Providers prioritized for recapture those
whose last activity was > 6 months
23.
24. “ Treatment of Demand”: Proportion of Newly Diagnosed People with
HIV/AIDS Meeting HHS Treatment Guidelines for ARV Initiation, 2005-
2009
N = 1,144 N = 1,281 N = 1,315 N = 1,087 N = 766
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
2005 2006 2007 2008 2009
24 Confidential Eligible)
<350 (Treatment 350-499 500 and above No CD4
25. Mean Community Viral Load among White
and Black MSM with HIV/AIDS in DC, 2008
45,000
Mean Community Viral Load (copies/mL)
40,000
35,000
31,404
30,000
25,000
20,000 19,732
15,000
10,000
5,000
0
White MSM Black MSM
N=645 N=901
26. Mean Community Viral Load among Whites
and Blacks Living with HIV/AIDS in DC, 2008
50,000
45,000
Mean Community Viral Load (copies/mL)
40,000 39,173
35,000
30,000
25,000
20,000
18,283
15,000
10,000
5,000
0
White Black
N=762 N=3,395
27. 100
95
90
85 Site 10
80
75
70
65 Site 9
60
55 Site 2
Average is 51.5% 50 Site 1
45 Site 4 and Site 13
40 Site 11
35 Site 5
30
25
20
15
10
5
0
28. Defining Input Indicators: DC PCSI
Project
Identify Efficiencies in Current System for Collaboration
and Integration
• Increase Impact, Increase Cost Effectiveness,
• Decrease Cost
Hepatitis/STD/HIV/TB
• Integrated Strategic Information to measure health
outcomes
• Moving towards medical system as a point of delivery
for HIV Prevention
• Multiple Morbidities Testing, Screening and Treatment
29. Data Driven Social Marketing
80.0% “Safer” Sex Behaviors among Study Participants
72.7%
70.0% 67.4%
66.4%
64.1%
60.9%
60.0% 57.4%
49.7%
50.0%
40.0% Heterosexuals
MSM*
29.9% 30.0%
30.0%
IDU
20.0%
10.0%
0.0%
**Other mode of transmission includes hemophilia, blood transfusion, Condoms Use at Knew own Status Knew Partner's
occupational exposure (healthcare workers), and perinatal.
Last Sex Status
31. Act on Results
• Reallocation of Resources (Internal and
External)
Silos in Reporting, Communication, Data
Collection Local and Federal Level
Multiple Data Reporting Streams contrary to
systemic planning and coordination
• RW Reporting for All Parts
• Locally and Directly Funded Prevention
Programs
Capacity vs. Ideology
32. 12 Cities Initiative is an opportunity to
develop new, innovative programs,
develop and strengthen relationships and
change existing policies
• SAMSHA/Mental Health/Substance Abuse FOA
• Surveillance Supplemental Application
Redefine Response to Epidemic
• Affordable Care Act
• Provider Response
33. Monitoring and Evaluation is a critical
component to scale up of NHAS
Existing Models and Best Practices: Peer
to Peer TA-Collectively Learning
Opportunity for Jurisdictions
Success Depends on Continued
Coordination, Collaboration,
Infrastructure Investments