The document discusses improving the clinical quality measure testing and certification process. It provides an agenda for a meeting that will discuss feedback on CQM testing and certification, and improvements to the certification process. It then summarizes feedback received on various aspects of the testing and certification tools and process, and the responses and improvements being made based on that feedback.

1. Improving the Clinical Quality Measure
Testing and Certification Process
HIMSS13
March 4, 2013

2. Agenda
• Cypress v2 Overview
• Feedback on CQM Testing and Certification
• Improvements to Certification Process
– Jacob Reider, Office of the National Coordinator
for Health Information Technology (ONC)
– Saul Kravitz, MITRE
– Steve Emrick, National Library of Medicine
– Bob Dolin, Lantana Consulting Group
• Open Discussion
2

3. Cypress Overview
• Meaningful Use (MU) Stage 2 Clinical Quality Measure
(CQM) Testing and Certification Tool
– ONC championed activity
– MITRE designed and stewarded free open source software
• Enables repeatable testing that an EHRs’ ability to calculate
Eligible Professional (EP) CQMs, Eligible Hospital (EH) CQMs,
and Quality Reporting Document Architecture (QRDA) XML
is accurate and aligned with MU Stage 2
1. Capture and Export
2. Import and Calculate
3. Electronic Submission
3

4. Status of Cypress and MU Stage 2 CQMs
• Cypress released to Authorized Testing Labs (ATLs)
– V2.0.0 released 12/21/2012
– V2.1.0 released 2/28/2013
• ONC’s 2014 HIT Certification Program operational
January 2, 2013
• ATLs have successfully certified vendors using the new
CQM testing procedure and Cypress
• We are receiving feedback from vendors and ATLs
– 125+ discussions on the public open source email list
– 200+ issues on ONC’s Jira tool
• We are improving the certification tools and process
based on your feedback
4

5. Types of Feedback Collected
• eCQM Logic
• Value Sets
• QRDA-I Test Data (Capture/Export)
• QRDA-III Test Data (Compute)
5

6. eCQM Logic Feedback
• Many questions regarding proper interpretation
of MU2 eCQMs
• Questions about specific eCQMs, e.g.:
– Measure X doesn’t reflect the clinical guidelines
• General Questions, e.g.:
– How to interpret specific occurrence idioms?
– How are episode of care eCQMs computed?
6

7. Response to eCQM Feedback
• Short-Term Actions
– V1.1 of eCQM Implementation Guide on CMS Website
– eCQMs updated as part of 2013 annual update
• Longer-Term Actions
– Publication of test data for use by vendors to
demonstrate intended behavior for each measure
– Improvements to logic syntax
– Incorporation of automated testing in MU3 eCQM
development process
7

8. Value Set Authority Center (VSAC)
• Provides Value Sets that support MU Stage 2 CQMs
– Requires UMLS Metathesaurus License for access to GUI and
download files
• Received lots of useful feedback and input from
users, e.g.:
– How can I obtain pre-packaged value sets for all eCQMs?
– Can’t you sort these in various ways – CMS ID, QDM category?
– How can I filter my search results by code system (SNOMED CT,
RxNorm, etc), in the user interface?
– How do I find out about updates to value sets or the VSAC?
8

9. Value Set Authority Center (VSAC)
Phase 2 Phase 1
Authorship Curation Delivery Consumption
Integration with Web page
Authoring Tool(s) to Controlled population for
support value set Value Set human
authorship Repository consumption
API &
Validation Web download /
Free-standing, web- Engine API web services for
based value set machine
authoring tool(s) NLM consumption
ONC
Value Set / Measure NLM Vendors
Consensus
Authors Providers
Org(s)
Consensus Public Health
Organization(s) Orgs
9

10. Value Sets – Downloadable Resources Table
Packages available for EP, EH, and EP+EH measures
CMS ID Value Set Name QDM Category
10

11. Value Sets – Code System Filter
Statin prescribed at discharge
Show only value sets containing RxNorm codes
11

12. VSAC: Ongoing Work
• Development of VSAC authoring environment
– Improve quality of value sets
– Improve authoring productivity
• Value set harmonization
– Gather ‘purpose’ of all value sets as structured metadata
– Will provide ‘semantic boundaries’ of value sets, which
could be helpful on determining overlap of value sets, new
code appropriateness, etc.
• Gathering of non-MU value sets
– e.g., Route of Administration
– Could eventually be used in some MU context
12

13. QRDA-I Certification Feedback
• “Excessive” data entry burden
• Test data incompatible with clinical
applications
• QRDA-I specification issues
• Many spurious validation errors and warnings
13

14. Response to
QRDA-I Certification Feedback
• Near Term (Following 2/28 release)
– Additional QRDA-I implementation guidance
– Revise test procedure to reuse Logic Test Deck for
QRDA-I Testing
• Initial focus on syntactic correctness only
• Elimination of MANY warnings
• Longer Term (With 4/26 release)
– Software changes to check that QRDA-I files are within
upper and lower bounds of information content
– Minor revision to test procedure
14

15. QRDA-III Certification Feedback
• Comments on the challenge of working with
HQMF files
• Questions about correctness of eCQM
interpretation
• Spurious validation errors on QRDA-III output
15

16. Response to
QRDA-III Certification Feedback
• Feedback to HL7 HQMF revision process
• Software fixes
• Feedback to eCQM update process
16

17. Changes to QRDA-III Test Deck
• For EH and EP Certification:
– Moved dates off of January 1st
– All ‘active’ states have a duration
– Tried to use widely supported codes
– Avoid unprescribable medications*
– Moved all events inside of business hours
– Moved all events inside encounters
• For EH Certification:
– Reduced length of stay
17

18. Cypress Timeline
• Version 2.0 – 12/21/2012
• Version 2.1.0 – 2/28/2013
– Addresses all resolved QRDA-I validation issues
– Supports shift to use of Logic Test Decks for QRDA-I Certification
– Updates to EH and EP test decks based on feedback
– Support ATL testing of multiple eCQM releases with a single
Cypress installation
• Version 2.2.0 – 4/26/2013
– Support for ‘dynamic’ logic testing for all measures
– Improved QRDA-I testing based on Logic Test Decks
– Support for 2013 EH Update, versioned value sets
• Updated Test Decks
– EH – Early May 2013
– EP – Early July 2013
18

19. Communications Channels
• Cypress Email Lists
– talk@projectcypress.org - public e-mail list
– announcement@projectcypress.org - broadcast list
– Details on http://projectcypress.org/faq.html
– ONC Testing and Certification Team ONC.Certification@hhs.gov
• ONC Jira for Issue Tracking
– General CQM Tracker https://cqm-issue-
tracker.atlassian.net/browse/CQM
– Cypress-specific Trackerhttps://cqm-issue-
tracker.atlassian.net/browse/CYPRESS
• VSAC updates listserv
– Go to https://list.nih.gov/cgi-bin/wa.exe?A0=nlm_vsac_updates
– Click on ‘Subscribe or Unsubscribe’
– Enter your name, e-mail, and click ‘Subscribe (NLM_VSAC_UPDATES)’
19

20. QUESTIONS AND FEEDBACK AND
FEEDBACK

21. BACKUP
21

22. Capture and Export
Current Concept of Operations
22

23. Import, Calculate and Electronic Submission
Concept of Operations
23

24. Release Notes for Cypress V2.1.0
Test Deck Changes
• Measurement period updated from 2010 to 2012, ending Dec 31, 2012 at 23:59
• Code provided for each data element trigger logic in a consistent manner across all
measures
• Added code description mouse overs to the HTML patient records (RxNorm
descriptions and all other code sets)
• Length of stay for encounters minimized for EP
• Patient data with the status of "active" given a duration in the test deck
• EP and EH events occur during business hours
• EP and EH Test deck events occur during encounters
• EP GP_Peds, E immunization end time fixed
• EH Newborn, A and Newborn, B birthdate and delivery times brought into sync
• Value sets added to the Cypress bundle to allow coordinating value set & measure
releases
• EH Newborn, A/B and OB A/B updated admit and discharge times on encounter to
match start/end times
24

25. Release Notes for Cypress V2.1.0
Cypress Software Changes
• Quality Measure Engine:
– Dynamic Support for CV measures in EH and EP
– Bundle Import code Moved to Health Data Standards library
• Measures:
– Reloaded measure definitions from CMS to fix issue with measure population criteria ids
• Cypress:
– Added support for multiple bundles to be loaded simultaniously
– Fixed double quoate bug in
resources/schematron/qrda/qrda_cat_1/measure_specific/d78ce034-8288-4012-a31e-
7f485a74f2a9.sch
– Removed xsi:type='CE' checks from generic Cat I schematron rules
– Updated schematron rule for CONF:7618
– to remove check for unit='1'
– Added ability to import bundle from within the application
– Updated QRDA Cat I test patient generator to eliminate duplicate entries
25