Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Heather Richards


Published on

  • Be the first to comment

  • Be the first to like this

Heather Richards

  1. 1. Ensuring Qualityof Health Care Data:A Canadian PerspectiveData Quality Asia Pacific Congress 2011Heather RichardsConsultantCanadian Institute for Health Information (CIHI)Tel:+1 250 220 2206Email: 1
  2. 2. Agenda> The Canadian Institute for Health Information> Data Quality Challenges in Canada> Strategies to Ensure Data Quality: – CIHI’s Data Quality Framework – Data Quality Reporting Tools and Studies – Techniques for Communicating Data Quality 2
  3. 3. The Canadian Institute forHealth Information 3
  4. 4. Canadian Institute for Health Information> National, independent, not-for-profit agency, established in 1994> One of Canada’s leading sources of high-quality, reliable and timely health information> 27 health databases and registries> 7 offices 4
  5. 5. CIHI’s Mandate> Coordinate, develop, maintain and disseminate health information on Canada’s health system and the health of Canadians 5
  6. 6. CIHIs Mandate Cont> Provide accurate and timely information required for: – Sound health policy – Effective management of the health system – Public awareness about factors affecting good health 6
  7. 7. Data Quality Challengesin Canada 7
  8. 8. Data Challenge: Variety of Partners> Accommodating different coding standards at provincial/territorial level versus national level;> Recognizing different uses of the data and different focus on data quality;> Adjusting for differing data collection methods 8
  9. 9. CIHI Partners Health Regional facilities health authorities Statistics Health Canada Canada Ministries CIHI Professional of health associations Non-governmental Private sector organizations organizations Researchers 9
  10. 10. Data Challenge: Secondary Data Collector> CIHI does not collect data directly> Our data comes from: – provincial governments; – hospitals; and – professional associations … this means that we cannot affect first hand how that data is captured and collected. 10
  11. 11. Data Challenge: Secondary Data Collector> CIHI relies on data providers (some are voluntary data providers) to report accurate information> Poor quality data often result from difficulties in collection standards, coding standards and chart documentation – and lack of training 11
  12. 12. Data Challenges: Other> Variety of databases and usability> Data flow and timeliness> Coding and comparability> Hospital practices and data completeness 12
  13. 13. End-stage renal failure 13
  14. 14. Question: Are Risk Factors CompletelyCaptured at all Facilities? Prevalence of Pulmonary Edema70%60%50%40% Inter-Quartile Range: 13-27%30%20%10%0% 14
  15. 15. Questionnaire Reveals a Correlation ofData Completeness to Hospital Practices Prevalence of Pulmonary Edema70%60%50%40% IQR: 8-21% IQR: 16-29%30%20%10%0% Reviews select Reviews all documentation documentation 15
  16. 16. Chart Review Confirms Under-Reporting Prevalence (%) Data Captured Data Captured by CIHI coder by Dialysis during Chart Clinic Staff ReviewPulmonary edema 22 27 Sensitivity=62% Specificity=93% Epidemiologists and clinical researchers prefer seeing PPV=77% these statistics… NPV=87% 16
  17. 17. Strategies to Ensure Data Quality • CIHI’s Data Quality Framework • Data quality reports and studies • Techniques for communicating DQ 17
  18. 18. Strategies to Ensure Data Quality CIHI’s Data Quality Framework 18
  19. 19. CIHI’s Data Quality Framework> Objective approach to assessing data quality and producing standard documentation> Three parts 1. Work Cycle 2. Assessment Tool 3. Documentation 19
  20. 20. 1. Data Quality Work Cycle Plan Assess Implement 20
  21. 21. 2. Data Quality Assessment Tool> Provides a consistent approach for defining data quality 5> Five dimensions Dimensions – Accuracy – Comparability 19 Characteristics – Timeliness – Usability 61 – Relevance Criteria 21
  22. 22. 2. Data Quality Assessment ToolAccuracyComparability CoverageTimeliness Capture and collectionUsability Unit non-response Item (partial) non-responseRelevance Measurement error Edit and imputation Processing and estimation Population of reference explicitly stated Coverage issues are documented Frame validated Under or over-coverage rate 22
  23. 23. Assessment Tool: Educational Component 23
  24. 24. 3. Data Quality Documentation> Details the data quality documentation required for each data holding 24
  25. 25. Metadata DocumentationRetainknowledgeabout themanagementof adatabasewith thedatabase. 25
  26. 26. Strategies to Ensure Data Quality Data Quality Reporting Tools and Studies 26
  27. 27. Deputy Minister Data Quality Reports> Bird’s eye view> Broad DQ scope: assess accuracy, timeliness, comparability and usability> Specific audience: Deputy Ministers of Health 27
  28. 28. Features of the Deputy MinisterData Quality Reports> Each indicator is important to the success of a database and has a defined action to improve performance – Snapshot of results across all jurisdictions – Trending over time> 11 databases – 8 from CIHI – 3 from Statistics Canada 28
  29. 29. Components of the Deputy MinisterData Quality Reports P/T indicator tables Trending Database- results specific reports TechnicalFlags table documents Each DM package 29
  30. 30. Trending: Discharge Abstract Database Indicator 1: Total Outstanding Hard Error Rate, per 1,000 Abstracts2.52.0 2003-041.5 2007-08 2008-091.0 2009-100.5 2004-05 2005-06 2006-070.0 Optimal Value = 0 30
  31. 31. Response to ReportsPositive:> Highlights to DM the value of a database; increases coverage of data holdings> Reveals systemic problems causing DQ issues; helps Deputy Ministers prioritize and reallocate resources> Congratulates on past DQ improvements; facilitates creation of DQ improvement action plans 31
  32. 32. Reabstraction Studies> Detailed review> Narrow DQ scope: assess coding consistency, correctness, completeness> Wide audience 32
  33. 33. Study Methods> A chart review to recapture the data and compare Reabstractor assigns Application reasons for compares differences data Application reveals original data Reabstractor recodes from chart 33
  34. 34. Overview Determine Share study results method Study Objectives Process Develop and data analyze collection data tool Train coders, collect data 34
  35. 35. Reabstraction Study ExampleDAD: Discharge Abstract Database> Data on acute-care hospital activity> Data supports: – funding and system planning decisions at government level – management decisions at the facility level – clinical research at the academic level 35
  36. 36. Strategies to Ensure Data Quality Techniques for Communicating Data Quality 36
  37. 37. Communicating Data Quality UsingDifferent LensesStatistics for OECDinternational Isolating determinantscomparisons of good healthHealth Assessing quality of careindicators ClinicalCategorizing researchhospitalizations purposesfor hospital such asmanagement purposes survival analysis 37
  38. 38. Health > Assess population health and health system performanceIndicators > Will look at one indicator: ACSC hospitalizations 38
  39. 39. Health Indicator: ACSC Hospitalizations Age-Standardized Rate of ACSC Hospitalizations per 100,000 Population600500 459400 2001-02 2002-03 2003-04 326 2004-05 2005-06300 2006-07 2007-08200100 0 39
  40. 40. 2007-08 DAD Study: ACSC Hospitalizations> Question: Is the decrease in ACSC hospitalizations real or is it due to changes in coding quality?> Answer: The observed decrease is real – National rates are indeed decreasing – Reabstraction studies found that certain patient populations had lower quality data 40
  41. 41. 2007-08 DAD Study: ACSC Hospitalizations SensitivityGrand mal status, epileptic convulsions 81%Chronic obstructive pulmonary diseases 91%Asthma 90%Diabetes 95%Heart failure and pulmonary edema 84%Hypertension 100%Angina 94%Any ACSC hospitalization 90% 41
  42. 42. Data Quality Challenges that Lie Ahead> The health sector is a changing landscape – Electronic health record – Health care funding – New technologies – New modes of delivering health care> New data will bring new quality challenges 42
  43. 43. “Taking health information further” 43