The development of a shortened
verbal autopsy instrument for
routine surveillance
June 18, 2013
Peter Serina
Post-Bachelor...
Outline
• The context
• Construction of a shortened instrument
• Measuring performance
• Next steps
2
Cause of death data
3
• Cause of death is important
• Vital registration systems are limited
• An information gap exists
One solution: verbal autopsy
• Trained interviewer
• Structured questionnaire
o Signs
o Symptoms
o Demographic characteris...
Verbal autopsy: an evolution
5
• Survey structure
• Analytic techniques
• Data capture
Wikimedia commons
Outline
• The Context
• Construction of a shortened instrument
• Measuring performance
• Next steps
• Conclusion
6
Construction of shortened instrument
Two approaches:
1. Expert-driven
2. Data-driven
7
Cost
(Time and
money)
Quality
(Surv...
Data-driven approach
The data:
• Population Health Metrics
Research Consortium (PHMRC)
• Gold standard verbal autopsy
vali...
Tariff method
9
AIDS Malaria Fall
Suffered a fall 0 0 32
Decedent had
TB 5 0 0
Fever 1 1.5 -1
Decedent had
dementia 0 -1 0...
Performance metrics
• Mean chance-corrected concordance for cause j
o Performance at the individual level
• Cause-specific...
Individual performance vs. question quantity for adults
11
Chance-
corrected
concordance
Number of questions
Outline
12
• The context
• Construction of a shortened instrument
• Measuring performance
• Next steps
• Conclusion
Questionnaire size
13
Neonate Child Adult
Full instrument 149 127 183
Full analyzed 117 84 170
Shortened analyzed 50 60 90...
Population-level performance
14
15
Individual-level performance
Outline
16
• The context
• Construction of a shortened instrument
• Measuring performance
• Next steps
• Conclusion
Next steps
• Link survey size to time to
implement
• Pilot surveys in:
o Philippines
o Bangladesh
o Sri Lanka
17
2.2: Did ...
Outline
18
• The Context
• Construction of a shortened instrument
• Measuring performance
• Next steps
• Conclusion
Conclusion
Our findings:
• > 40% fewer questions chosen in systematic, data-driven
process
• Negligible effects on perform...
Thank you
Peter Serina
ptserina@uw.edu
Upcoming SlideShare
Loading in …5
×

The development of a shortened verbal autopsy instrument for routine surveillance

907 views

Published on

GHME 2013 Conference
Session: Verbal autopsy
Date: June 18 2013
Presenter: Peter Serina
Institution:
Institute for Health Metrics and Evaluation (IHME),
University of Washington

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
907
On SlideShare
0
From Embeds
0
Number of Embeds
40
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Purpose:Reasoning behind and scientific basis for shortened verbal autopsy survey instrument.
  • COD importantTrack changes in burden of disease across space and timeSet priorities for interventionsMonitor public health programsInform allocation and distribution of resourcesVital RegistrationVR being viewed as the best, though most expensive to set up and maintain. Vital registration also limitedFor adults, concordance of 67% when corrected for chanceWork at IHME: 38% of deaths in 2010 were covered by vital registration systems.How do we fill this critical information gap
  • VA used to determine COD, using survey-interview methods to ask about symptoms a deceased experienced prior to death from caregivers, family, and friends.
  • FamousRembrant Painting of an autopsyNot a new idea17th century London – death searchers (communicable disease)informal, taken on paper1950s through 1970s Survey structure change to be more systemmaticQuestionnaires were certified by physiciansPast 10 yearsNew advancements in analyzing techniques using statistical models and computer algorithmsData capture paper-based electronicHow can we use these advancements to create a shortened instrument both in terms of time and data entry that is able help fill that information gapEven with all of these advancements, surveys take 45 min, logging entry
  • Instrument too longDon’t mention ByassWhat you hope to achieve with a reduced instrumentCostCoverage QualityExpert-based: drop items from a long instrument based on expert opinion Been done by other research teamsCould take less time and money to createEmpirical: use a validation dataset and test the effect of dropping items or sets of items on chance-corrected concordance and CSMF accuracy.
  • Explain table
  • Don’t call it a toy, It’s a simplified examplehigh tariff= more related0 tariff= no relationNegative tariff= counter-indicativePresent as a story/anecdote imagine if family member of recently deceased…
  • NUMBER of QUESTIONSLooking at performance as we systematically reduce the number of questions based on the tariff standard deviationSet 90 questions as the starting point that we don’t want to go below (we can add questions back in if needed, for example health record information)Question=SymptomExplain axes
  • IHME study I cited earlier had 78% CSMF Accuracy for AdultsExplain axes, colors
  • CCC- concordance, % correct, corrected for chanceExplain axes, colors
  • A reduced VA tool has important implications for the costs, upkeep, and strengthening of routine surveillance systems, especially for accurately capturing COD dataAs we’ve learned, question reduction can really benefit from data-driven testing and approaches as the quantity and quality of questions – and their pairings – make a difference in the performance of the toolEnd on what you hope to achieve with the pilot study and how it will feed into the bigger COD/information systems/data collection issues and challenges Give it a go J
  • The development of a shortened verbal autopsy instrument for routine surveillance

    1. 1. The development of a shortened verbal autopsy instrument for routine surveillance June 18, 2013 Peter Serina Post-Bachelor Fellow
    2. 2. Outline • The context • Construction of a shortened instrument • Measuring performance • Next steps 2
    3. 3. Cause of death data 3 • Cause of death is important • Vital registration systems are limited • An information gap exists
    4. 4. One solution: verbal autopsy • Trained interviewer • Structured questionnaire o Signs o Symptoms o Demographic characteristics • Lay respondent familiar with the deceased • Population Health Metrics Research Consortium (PHMRC) instrument: o 153 questions o 45 minutes 4 PHMRC GC13 Project
    5. 5. Verbal autopsy: an evolution 5 • Survey structure • Analytic techniques • Data capture Wikimedia commons
    6. 6. Outline • The Context • Construction of a shortened instrument • Measuring performance • Next steps • Conclusion 6
    7. 7. Construction of shortened instrument Two approaches: 1. Expert-driven 2. Data-driven 7 Cost (Time and money) Quality (Survey performance) Coverage (Population, caus es, and areas)
    8. 8. Data-driven approach The data: • Population Health Metrics Research Consortium (PHMRC) • Gold standard verbal autopsy validity study The method: • Tariff • Systematic reduction 8 Country Site GS deaths Mexico Mexico City 2,031 Tanzania Pemba Island 822 Dar es Salaam 3,239 India Uttar Pradesh 2,170 Andhra Pradesh 2,382 Philippines Bohol 1,898 Total 12,542
    9. 9. Tariff method 9 AIDS Malaria Fall Suffered a fall 0 0 32 Decedent had TB 5 0 0 Fever 1 1.5 -1 Decedent had dementia 0 -1 0 Tariff standard deviation 18.5 2.9 1.3 0.6 Symptoms Causes
    10. 10. Performance metrics • Mean chance-corrected concordance for cause j o Performance at the individual level • Cause-specific mortality fraction (CSMF) accuracy o Performance at the population level 10
    11. 11. Individual performance vs. question quantity for adults 11 Chance- corrected concordance Number of questions
    12. 12. Outline 12 • The context • Construction of a shortened instrument • Measuring performance • Next steps • Conclusion
    13. 13. Questionnaire size 13 Neonate Child Adult Full instrument 149 127 183 Full analyzed 117 84 170 Shortened analyzed 50 60 90 Shortened instrument 76 72 105 % reduction 49% 43% 43%
    14. 14. Population-level performance 14
    15. 15. 15 Individual-level performance
    16. 16. Outline 16 • The context • Construction of a shortened instrument • Measuring performance • Next steps • Conclusion
    17. 17. Next steps • Link survey size to time to implement • Pilot surveys in: o Philippines o Bangladesh o Sri Lanka 17 2.2: Did Juan Garcia have a fever?
    18. 18. Outline 18 • The Context • Construction of a shortened instrument • Measuring performance • Next steps • Conclusion
    19. 19. Conclusion Our findings: • > 40% fewer questions chosen in systematic, data-driven process • Negligible effects on performance with same population size Impact: • Greater number of verbal autopsies can be collected • Strengthen routine surveillance • Address the information gap 19
    20. 20. Thank you Peter Serina ptserina@uw.edu

    ×