Ensuring Qualityof Health Care Data:A Canadian PerspectiveData Quality Asia Pacific Congress 2011Heather RichardsConsultan...
Agenda> The Canadian Institute for Health Information> Data Quality Challenges in Canada> Strategies to Ensure Data Qualit...
The Canadian Institute forHealth Information                             3
Canadian Institute for Health Information> National, independent, not-for-profit agency,  established in 1994> One of Cana...
CIHI’s Mandate> Coordinate, develop, maintain and disseminate  health information on Canada’s health system  and the healt...
CIHIs Mandate Cont> Provide accurate and  timely information  required for:  – Sound health policy  – Effective management...
Data Quality Challengesin Canada                          7
Data Challenge: Variety of Partners> Accommodating different coding standards at  provincial/territorial level versus nati...
CIHI Partners                    Health              Regional                    facilities          health authorities   ...
Data Challenge: Secondary Data Collector> CIHI does not collect data directly> Our data comes from:  – provincial governme...
Data Challenge: Secondary Data Collector> CIHI relies on data providers (some are voluntary  data providers) to report acc...
Data Challenges: Other> Variety of databases and usability> Data flow and timeliness> Coding and comparability> Hospital p...
End-stage renal failure                          www.vancouversun.com   13
Question: Are Risk Factors CompletelyCaptured at all Facilities?         Prevalence of Pulmonary Edema70%60%50%40%        ...
Questionnaire Reveals a Correlation ofData Completeness to Hospital Practices               Prevalence of Pulmonary Edema7...
Chart Review Confirms Under-Reporting                           Prevalence (%)                                   Data Capt...
Strategies to Ensure Data Quality  •   CIHI’s Data Quality Framework  •   Data quality reports and studies  •   Techniques...
Strategies to Ensure Data Quality     CIHI’s Data Quality Framework                                     18
CIHI’s Data Quality Framework> Objective approach to  assessing data quality  and producing standard  documentation> Three...
1. Data Quality Work Cycle                    Plan        Assess               Implement                                  ...
2. Data Quality Assessment Tool> Provides a consistent  approach for defining  data quality                               ...
2. Data Quality Assessment ToolAccuracyComparability                CoverageTimeliness      Capture and collectionUsabilit...
Assessment Tool: Educational Component                                         23
3. Data Quality Documentation> Details the data quality  documentation required  for each data holding                    ...
Metadata DocumentationRetainknowledgeabout themanagementof adatabasewith thedatabase.                         25
Strategies to Ensure Data Quality     Data Quality Reporting Tools     and Studies                                    26
Deputy Minister Data Quality Reports> Bird’s eye view> Broad DQ scope:  assess accuracy,  timeliness, comparability  and u...
Features of the Deputy MinisterData Quality Reports> Each indicator is important to the success  of a database and has a d...
Components of the Deputy MinisterData Quality Reports                   P/T indicator                      tables  Trendin...
Trending: Discharge Abstract Database        Indicator 1: Total Outstanding Hard Error                Rate, per 1,000 Abst...
Response to ReportsPositive:> Highlights to DM the value of a  database; increases coverage of  data holdings> Reveals sys...
Reabstraction Studies> Detailed review> Narrow DQ scope: assess  coding consistency,  correctness, completeness> Wide audi...
Study Methods> A chart review to  recapture the data and  compare                                                     Reab...
Overview                              Determine            Share               study            results            method ...
Reabstraction Study ExampleDAD: Discharge Abstract Database> Data on acute-care hospital activity> Data supports:   – fund...
Strategies to Ensure Data Quality     Techniques for Communicating     Data Quality                                    36
Communicating Data Quality UsingDifferent LensesStatistics for OECDinternational             Isolating determinantscompari...
Health       > Assess population health and               health system performanceIndicators             > Will look at o...
Health Indicator: ACSC Hospitalizations            Age-Standardized Rate of ACSC         Hospitalizations per 100,000 Popu...
2007-08 DAD Study: ACSC Hospitalizations> Question: Is the decrease in ACSC  hospitalizations real or is it due to changes...
2007-08 DAD Study: ACSC Hospitalizations                                          SensitivityGrand mal status, epileptic c...
Data Quality Challenges that Lie Ahead> The health sector is a changing landscape   – Electronic health record   – Health ...
“Taking health information further”                                      43
Upcoming SlideShare
Loading in...5
×

Heather Richards

323

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
323
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Heather Richards

  1. 1. Ensuring Qualityof Health Care Data:A Canadian PerspectiveData Quality Asia Pacific Congress 2011Heather RichardsConsultantCanadian Institute for Health Information (CIHI)Tel:+1 250 220 2206Email: hrichards@cihi.ca 1
  2. 2. Agenda> The Canadian Institute for Health Information> Data Quality Challenges in Canada> Strategies to Ensure Data Quality: – CIHI’s Data Quality Framework – Data Quality Reporting Tools and Studies – Techniques for Communicating Data Quality 2
  3. 3. The Canadian Institute forHealth Information 3
  4. 4. Canadian Institute for Health Information> National, independent, not-for-profit agency, established in 1994> One of Canada’s leading sources of high-quality, reliable and timely health information> 27 health databases and registries> 7 offices 4
  5. 5. CIHI’s Mandate> Coordinate, develop, maintain and disseminate health information on Canada’s health system and the health of Canadians 5
  6. 6. CIHIs Mandate Cont> Provide accurate and timely information required for: – Sound health policy – Effective management of the health system – Public awareness about factors affecting good health 6
  7. 7. Data Quality Challengesin Canada 7
  8. 8. Data Challenge: Variety of Partners> Accommodating different coding standards at provincial/territorial level versus national level;> Recognizing different uses of the data and different focus on data quality;> Adjusting for differing data collection methods 8
  9. 9. CIHI Partners Health Regional facilities health authorities Statistics Health Canada Canada Ministries CIHI Professional of health associations Non-governmental Private sector organizations organizations Researchers 9
  10. 10. Data Challenge: Secondary Data Collector> CIHI does not collect data directly> Our data comes from: – provincial governments; – hospitals; and – professional associations … this means that we cannot affect first hand how that data is captured and collected. 10
  11. 11. Data Challenge: Secondary Data Collector> CIHI relies on data providers (some are voluntary data providers) to report accurate information> Poor quality data often result from difficulties in collection standards, coding standards and chart documentation – and lack of training 11
  12. 12. Data Challenges: Other> Variety of databases and usability> Data flow and timeliness> Coding and comparability> Hospital practices and data completeness 12
  13. 13. End-stage renal failure www.vancouversun.com 13
  14. 14. Question: Are Risk Factors CompletelyCaptured at all Facilities? Prevalence of Pulmonary Edema70%60%50%40% Inter-Quartile Range: 13-27%30%20%10%0% 14
  15. 15. Questionnaire Reveals a Correlation ofData Completeness to Hospital Practices Prevalence of Pulmonary Edema70%60%50%40% IQR: 8-21% IQR: 16-29%30%20%10%0% Reviews select Reviews all documentation documentation 15
  16. 16. Chart Review Confirms Under-Reporting Prevalence (%) Data Captured Data Captured by CIHI coder by Dialysis during Chart Clinic Staff ReviewPulmonary edema 22 27 Sensitivity=62% Specificity=93% Epidemiologists and clinical researchers prefer seeing PPV=77% these statistics… NPV=87% 16
  17. 17. Strategies to Ensure Data Quality • CIHI’s Data Quality Framework • Data quality reports and studies • Techniques for communicating DQ 17
  18. 18. Strategies to Ensure Data Quality CIHI’s Data Quality Framework 18
  19. 19. CIHI’s Data Quality Framework> Objective approach to assessing data quality and producing standard documentation> Three parts 1. Work Cycle 2. Assessment Tool 3. Documentation 19
  20. 20. 1. Data Quality Work Cycle Plan Assess Implement 20
  21. 21. 2. Data Quality Assessment Tool> Provides a consistent approach for defining data quality 5> Five dimensions Dimensions – Accuracy – Comparability 19 Characteristics – Timeliness – Usability 61 – Relevance Criteria 21
  22. 22. 2. Data Quality Assessment ToolAccuracyComparability CoverageTimeliness Capture and collectionUsability Unit non-response Item (partial) non-responseRelevance Measurement error Edit and imputation Processing and estimation Population of reference explicitly stated Coverage issues are documented Frame validated Under or over-coverage rate 22
  23. 23. Assessment Tool: Educational Component 23
  24. 24. 3. Data Quality Documentation> Details the data quality documentation required for each data holding 24
  25. 25. Metadata DocumentationRetainknowledgeabout themanagementof adatabasewith thedatabase. 25
  26. 26. Strategies to Ensure Data Quality Data Quality Reporting Tools and Studies 26
  27. 27. Deputy Minister Data Quality Reports> Bird’s eye view> Broad DQ scope: assess accuracy, timeliness, comparability and usability> Specific audience: Deputy Ministers of Health 27
  28. 28. Features of the Deputy MinisterData Quality Reports> Each indicator is important to the success of a database and has a defined action to improve performance – Snapshot of results across all jurisdictions – Trending over time> 11 databases – 8 from CIHI – 3 from Statistics Canada 28
  29. 29. Components of the Deputy MinisterData Quality Reports P/T indicator tables Trending Database- results specific reports TechnicalFlags table documents Each DM package 29
  30. 30. Trending: Discharge Abstract Database Indicator 1: Total Outstanding Hard Error Rate, per 1,000 Abstracts2.52.0 2003-041.5 2007-08 2008-091.0 2009-100.5 2004-05 2005-06 2006-070.0 Optimal Value = 0 30
  31. 31. Response to ReportsPositive:> Highlights to DM the value of a database; increases coverage of data holdings> Reveals systemic problems causing DQ issues; helps Deputy Ministers prioritize and reallocate resources> Congratulates on past DQ improvements; facilitates creation of DQ improvement action plans 31
  32. 32. Reabstraction Studies> Detailed review> Narrow DQ scope: assess coding consistency, correctness, completeness> Wide audience 32
  33. 33. Study Methods> A chart review to recapture the data and compare Reabstractor assigns Application reasons for compares differences data Application reveals original data Reabstractor recodes from chart 33
  34. 34. Overview Determine Share study results method Study Objectives Process Develop and data analyze collection data tool Train coders, collect data 34
  35. 35. Reabstraction Study ExampleDAD: Discharge Abstract Database> Data on acute-care hospital activity> Data supports: – funding and system planning decisions at government level – management decisions at the facility level – clinical research at the academic level 35
  36. 36. Strategies to Ensure Data Quality Techniques for Communicating Data Quality 36
  37. 37. Communicating Data Quality UsingDifferent LensesStatistics for OECDinternational Isolating determinantscomparisons of good healthHealth Assessing quality of careindicators ClinicalCategorizing researchhospitalizations purposesfor hospital such asmanagement purposes survival analysis 37
  38. 38. Health > Assess population health and health system performanceIndicators > Will look at one indicator: ACSC hospitalizations 38
  39. 39. Health Indicator: ACSC Hospitalizations Age-Standardized Rate of ACSC Hospitalizations per 100,000 Population600500 459400 2001-02 2002-03 2003-04 326 2004-05 2005-06300 2006-07 2007-08200100 0 39
  40. 40. 2007-08 DAD Study: ACSC Hospitalizations> Question: Is the decrease in ACSC hospitalizations real or is it due to changes in coding quality?> Answer: The observed decrease is real – National rates are indeed decreasing – Reabstraction studies found that certain patient populations had lower quality data 40
  41. 41. 2007-08 DAD Study: ACSC Hospitalizations SensitivityGrand mal status, epileptic convulsions 81%Chronic obstructive pulmonary diseases 91%Asthma 90%Diabetes 95%Heart failure and pulmonary edema 84%Hypertension 100%Angina 94%Any ACSC hospitalization 90% 41
  42. 42. Data Quality Challenges that Lie Ahead> The health sector is a changing landscape – Electronic health record – Health care funding – New technologies – New modes of delivering health care> New data will bring new quality challenges 42
  43. 43. “Taking health information further” 43
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×