Follow the Quality Assurance journey taken by CoverMyMeds as it transitioned from a start-up to a growth company. Founded in 2008 with a mission to help patients get the medication they need to be healthy, CoverMyMeds has doubled its staff and revenue every year since inception. As the company expands, quality processes have become paramount in protecting Patient Health Information (PHI) and mitigating risk. Implementing Juran’s concept of Big Q, CoverMyMeds has begun its journey in creating its own quality center of excellence.

1. CoverMyQuality: Implementing a
Quality Program
February 7, 2017
Rick Neighbarger & Susan Bradley

2. Agenda
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Who is CMM?
• From Startup to Growth Company
• Special Challenges
• Implementing Big Q Strategy
• Initial Implementation
• PhaseTwo Implementation
• CMM Journey
• Questions?

3. Founded in 2008, CoverMyMeds automates Prior
Authorization (PA) for all plans and any medication.
We automate the medication prior authorization
(PA) process, saving health care professionals
valuable time and ensuring patients receive the
medication they need to be well and don’t abandon
their prescription.
We currently process more than 2.65 million PA
requests each month, with more than 50 million
requests processed to date.
Who is CMM?
© 2016 CoverMyMeds LLC. All Rights Reserved.
About CMM

4. What We Do
Patient Pharma
Prior authorization requests are a permission slip from health plans for their patient to begin taking a medication, ensuring the right therapy for
the condition and cost-effectiveness. Before CoverMyMeds, the PA process was paper-based and accomplished through multiple faxes and
phone calls between providers, pharmacists and health plans.
Today, CoverMyMeds works with electronic health record systems (EHRs), health plans, pharmacy systems and providers to initiate, transmit
and track the status of PA requests within the clinical workflow.

5. From Start-up to Growth Company
© 2016 CoverMyMeds LLC. All Rights Reserved.
2008 2011 2014 2016
Employees: 5 14 73 466
Yearly PAs: 187,500 1.56 million 12.5 million 50 million
Ave. Monthly PAs: n/a 61,861 729,486 1,612,710
Lives Helped: n/a 1 million 12 million 46 million

6. Challenges:
CMM moved from startup to quickly growing company—new set of risks
• From “Just Do It”, get a foothold to sustaining growth and rapidly increasing complexity
Data privacy for rapidly growing database of Protected Health Information
Organizational growth requires more process & communication than before
• Many new managers & individual contributors
• Not familiar with processes, systems & the organization
Complexity of systems, customer hierarchy & regulations requires strategic planning
• Can’t rely on series of tactical steps & “you’re smart/you’ll figure it out” like startup phase
Initial applications not meant to handle current rate of growth (scaling/technical debt vs. new business)
Quality was primarily associated with testing activities

7. Special Challenges
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Company culture: Start-up—resistance to central control &
standards; freedom to choose whatever tools, processes, etc. by
team or even individuals
• Constantly hiring: bringing on new staff, promoting new managers
• Organization structured in business verticals: need to maintain
their ecosystems while working as one business

8. Solution: Implement Big Q strategy
© 2016 CoverMyMeds LLC. All Rights Reserved.
Joseph Juran established the field of quality
management and wrote the QualityControl
Handbook which taught manufacturers
manufacturers worldwide how to be more
efficient and productive
Little “q” quality generally just equates to
testing, which is good, but not enough!
Big “Q” Quality is a system for achieving
high quality results that supports:
•Bullet-proof privacy and security
•Rock-solid reliability
•Best-in-class performance

9. Big Q vs. Little q strategy
© 2016 CoverMyMeds LLC. All Rights Reserved.

10. Getting Buy-In for Big Q Strategy
• Talking to Leadership
• Outlined Benefits (What’s In It forYou?)
• Targeted their goals: Promoted expense/waste reduction as an enabler for revenue
enhancement
• Included them so they felt vested in the outcome
• Sold the “Why?” to leadership, mid-level managers, and individual contributors
• Reduction in risk
• Improve customer relationships
• Avoid the “Company Killer”
• Best Practice

11. CMM Initial Implementation: 3-Legged Stool
© 2016 CoverMyMeds LLC. All Rights Reserved.
Requirements Management
Risk Management
Root Cause Analysis
Measurements &
Metrics
Remember, our biggest risks
involve the privacy and security of
our patients’ data!
1
2
3
M&M

12. Big Q, Phase 1: The 3 R’s – Requirements, Risk &
Root Cause
© 2016 CoverMyMeds LLC. All Rights Reserved.
1) Requirements Management 2) Risk Management 3) Root Cause Analysis
• Owned By: Product Owner (from
(from the Business!)
• Answers: What should we build,
build, NOT build, in what order,
and why?
• Activities: Project vetting,
grooming and prioritizing;
Requirements definition; Project
Management;
• Other Players: Users, Account
Mgrs., PMs, Engineering Mgrs.,
Testing Engineers, Devs,
InfoSecurity, DBAs, Analysts
• Owned By: Engineer
Manager/Testing Engineer
• Answers: What are the worst
things that could go wrong, and
how can we mitigate them?
(Especially PHI!!!)
• Activities: Risk Analysis; Risk-
based test planning; Test
execution; Monitoring; Security
assessments;
• Other Players: POs, Engineering
Engineering Mgrs., Devs,
InfoSecurity, ProdOps, DBAs,
Analysts
• Owned By: Testing
Engineer/Quality Analyst
• Answers: What went wrong, and
and what should we do about it?
• Activities: Incident mitigation &
reporting; Retrospectives; Root
Cause Analysis (RCA); Change
Management
• Other Players: POs, Engineering

13. 1: Requirements Management
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Define Product Owner role
• Prioritize Backlog
• Serve as liaison between customer and tech groups
• Hired an AgileCoach
• Create checklist of considerations when compiling requirements for small requests
• Create Epics, Feature, User Stories to create hierarchy
• Implement quality measures so quality is built in
• House enterprise-wide tool so it’s accessible by all areas

14. 2: Risk Management
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Implemented cross-org Risk Management Team that reviews all production deployment requests;
Establishes more consistency in risk management activities throughout development teams
• All deployments checked by 3rd
party
• Moved risk assessment upstream from pre-deployment to code review
• Enterprise Risk Management Program develops and promotes company culture of enterprise risk
management:
• Encourages open discussion regarding risk
• Integrates risk management into the Company’s goals
• Creates corporate culture where people at all levels manage risks

15. 3: Root Cause
© 2016 CoverMyMeds LLC. All Rights Reserved.
• Implemented Incident/Problem Management process
• Root Cause Analysis approach:Timeline, Mindmap, Action Items
• Cross-org team of RCA facilitators
• Treat “disease”, not symptoms
• Ensure common issues remediated across verticals
• Create comprehensive and prioritized remediation action plans
• Create metrics and reporting
• Moved to new standard tool available across all areas

16. Big Q, Phase 2
© 2016 CoverMyMeds LLC. All Rights Reserved.
Expand Quality and Risk Management focus to the left in the SDLC lifecycle
Plan Requirements Design Code Test Implement Verify
Quality

17. Big Q, Phase 2
© 2016 CoverMyMeds LLC. All Rights Reserved.
In the Cost of Poor Quality continuum, Shift focus to proactive Costs of Prevention & Assessment to exponentially
reduce Reactive Costs of Failure
”Cost of poor quality (COPQ) or poor quality costs (PQC), are costs that would disappear if systems, processes, and
products were perfect.”

18. Big Q, Phase 2
© 2016 CoverMyMeds LLC. All Rights Reserved.

19. Big Q, Phase 2: Quality & Risk at CMM
© 2016 CoverMyMeds LLC. All Rights Reserved.
Keep
evangelizing
the merits of
risk prevention
Risk & Quality
Blog &
Newsletter
Onboarding
training for
new hires
Route Root
Cause
feedback to
upstream Risk
Management
Teams
Formal course
through
Learning
Center

20. Big Q- Phase 2
© 2016 CoverMyMeds LLC. All Rights Reserved.
Risk Management Root Cause Analysis
(RCA)
Requirements Metrics Testing Role &
Processes
Performance Testing Add Enterprise-wide
toolsets
Expand Use of FMEA
(Threat Modeling):
• Software Design
• Testing
• Requirements
• Business/Product
t Mgt.
Integrate action items
into team intake
processes across the
company
Implement
consistent Product
Owner role and
processes
Improve visibility of key
metrics for leadership
and individual
contributors
Implement appropriate
roles and processes to
ensure coverage and
continuity between:
• Unit & Integration
Integration
• System
• Regression
• User Acceptance
Acceptance
Expand to begin earlier
in development
lifecycle
Incident Management
NewApp Risk
Assessment
Improve trending & data
analysis
Implement
consistent Business
Analyst role and
processes
Assess Evidence Based
Metrics for feasibility
AddCapacity Planning
& Management
function
Problem Management
Created Enterprise Risk
Management Program
Integrate with data
analytics from other
sources
Implement
consistent
requirements best
practices
Defect Management
Team Intake/Backlog
Management?

21. CMM Journey: Initial, Current, Future State
© 2016 CoverMyMeds LLC. All Rights Reserved.
Initial Current Future
Requirements
1) Wide process range from no
requirements to overly designed
feature request
2) Engineering Mgs deciding priority of
work
1) Addition of BA/QA and PM roles
2) Epics & Stories entered in Jira
3) Hired AgileCoach
1) Build quality into the requirement
phase
2) Formally assess risk for customer-
facing applications
3) Product Owner deciding priority of
work
4) Implement a set of best practices
for requirements elicitation and
management for PRs and small
enhancements
Risk
1) Pre-deployment checklist
2) No formal risk assessment or
mitigation process
1) Formal risk review before
deployment
2) Code Reviews
1) New product assessment
2) Enterprise Risk Group
Root Cause
1) No consistent formal assessments
2) No documented methodology
1) Formal process with facilitators
2) Documented outcomes
1) Group of facilitators
2) Convert trends into actions
Measurement
No formal metrics around incidents
No trending
1) Meaningful metrics around
incidents,
2) Monthly metric report
1) Expand metric report to include risk
components
2)Target quality metrics to business
goals of the organization
Tooling
MultipleTools or noTools used Centralizing a core tool for work intake,
help tickets, and incidents
Expand use of core tool, add new data
analytic tools

22. Q & A
Rick: https://www.linkedin.com/in/rick-neighbarger/
Susan: https://www.linkedin.com/in/susan-bradley-cste
www.covermymeds.com