Modern software testing for Healthcare Organizations. Learn about best practices for software testing in the healthcare industry featuring Mike Cooper, Chief Quality Officer of Healthcare IT Leaders and Kevin Dunne, VP of Business Development at QASymphony

1. Evolve or Die:
Healthcare ITTesting
August 30th, 2016
Michael Cooper – Healthcare IT Leaders
Kevin Dunne - QASymphony
Join the conversation – use the hashtag #HITTesting on Twitter

2. Guest Speaker: Michael Cooper,
Chief Quality Officer
• Mike Cooper is an acknowledged global thought leader in QA andTesting, Michael
does QA andTesting Consulting andAdvisory work at leading corporations
worldwide. Michael was previously the software QA andTesting evangelist for HP
Software. Prior to that, he was Sr. Director for IT Quality Assurance atT-Mobile
USA, where he led a group of 500+ QAT professionals. He holds an MBA from
Georgia State University and a BA from the University of Florida.
OUR PRESENTERS
https://www.linkedin.com/in/qacooper
@QACooper
Join the conversation – use the hashtag #HITTesting on Twitter
Guest Speaker: Kevin Dunne,
VP Strategy and Business Development
• Kevin Dunne is theVP of Strategy and Business Development at QASymphony,
ensuring their continued commitment to innovation and delivering tools to create
better software.With a deep interest in the emerging trends in software development
and testing, Kevin is dedicated to collaborating with thought leaders in this space.
Kevin comes to QASymphony from Deloitte, where he managed testing on large
government and Fortune 500 engagements.

3. • The Big Picture
• Modernization of Technology across Healthcare IT Value Chain
• Evolving role of Healthcare Testing
– Modern Healthcare Testing Techniques
• Security, Compliance and Government Regulatory Mandates
• Real world applications and examples
• Importance of interoperability
• Q&A
AGENDA
Join the conversation – use the hashtag #HITTesting on Twitter

4. THE BIG PICTURE
Join the conversation – use the hashtag #HITTesting on Twitter

5. HEALTHCARE ITVALUE CHAIN
Join the conversation – use the hashtag #HITTesting on Twitter

6. KEY DRIVERS FOR CHANGE
• Upgrades & new
releases
• Maintenance
• Integrations
• Mobility
• Cloud and SaaS
Technology
• Revenue / Cost
initiatives
• Quality initiatives
• Changes in
business process
• M & A activity
• External business
relationships
Business
• State and Federal
• International
• Industry standards
and practices
• Compliance and
audits
• Penalties and
remediation
Regulatory
Join the conversation – use the hashtag #HITTesting on Twitter

7. EVOLVING HEALTHCARETECHNOLOGYTOTEST
Join the conversation – use the hashtag #HITTesting on Twitter

8. Good Quality (the cost you see)
QA & Test Best Practices
Internal & External Test Resources
Poor Quality (hidden costs)
Data / Security Breaches
Under Billing or Improper Billing
Government Fines & Penalties
Patient Dissatisfaction
Customer Churn or Loss
Suboptimal Business Outcomes
Litigation
Provider Dissatisfaction
Harm to Brand
Software Fix/Issue Remediation
…and more!
Total Cost of Quality
Poor Quality
80% of true cost

9. QA and Testing Impact
• Failure to achieve compliance can result in
millions of dollars of impact
• Requires testing of several systems
resulting in a spike in testing needs
• Must test any system which stores,
processes, sends, receives, or reports
diagnosis code information.
Guidance
• Good opportunity to document key end-to-
end business process test cases
• Test automation
COMPLIANCETESTING
Join the conversation – use the hashtag #HITTesting on Twitter

10. QA and Testing Impact
• Agile teams often have limited
documentation
• The role of the tester is often unclear on
Agile projects
• Agile sprints have short time lines 2-4
weeks
Guidance
• Testers should be considered a part of the
Scrum/Sprint Team
• Testing tasks should be reflected in the
backlog
• Agile/Scrum training is recommended
• Automation and performance testing are
essential in Agile projects
AGILE DEVELOPMENTANDTESTING
46% of organizations
are using Agile
development
methodologies
Source: Forrester Research
Join the conversation – use the hashtag #HITTesting on Twitter

11. • Moving towards Agile Testing (TDD/ATDD/BDD) also demand a
more complete testing strategy focused on more than just UI
testing:
BUILDINGA COMPLETETESTING STRATEGY
UI Tests
Integration
Tests
Unit Tests
UI Tests
Integration
Tests
Unit Tests
Join the conversation – use the hashtag #HITTesting on Twitter

12. EXPLORATORY TESTING – PROS & CONS
-Reduce time spent
writing tests
-Find new bugs by testing
new paths
-Increase speed of
execution
-Increase tester morale &
engagement
-Increased level of skill
required
-Risk of failing to test a
critical path
-Potential increase in test
plan scope
-Risk of backfilling if last
tester leaves
Join the conversation – use the hashtag #HITTesting on Twitter

13. SESSION BASED TEST MANAGEMENT
Introducing exploratory testing within a framework will greatly increase the odds of success,
and will reduce fear and uncertainty among the practitioners as well as executives.
Session Based Test Management is a popular framework for this, as it tracks all the important
data on testing:
More info on SBTM: http://www.satisfice.com/articles/sbtm.pdf
• Session charter (includes a mission
statement, and areas to be tested)
• Testers involved Date and time executed
• Task breakdown
• Data files
• Test notes
• Issues
• Bugs
Join the conversation – use the hashtag #HITTesting on Twitter

14. A HYBRID APPROACH
Exploratory
Manual
Scripted
Automated
Charge Testing
Enrollment
Data Setup
Regression
Smoke Testing
Critical Path Scenarios
Security
Usability
New Feature Testing
User Acceptance Testing
Role Based Testing
Integration
API Testing
Join the conversation – use the hashtag #HITTesting on Twitter

15. QA and Testing Impact
• QA teams have new apps to test
• Data migration, Big Data testing projects
require specialized skills
Guidance
• Consider parallel system testing during
migration
• Test role-based work-flows in production
• Performance testing from the cloud
• Consider service virtualization for SaaS
testing of third party services
CLOUD, SAAS AND BIG DATA
By 2018, 80% of
healthcare data will
pass through the cloud
at some point in its
lifetime, as providers
seek to leverage cloud
…for data collection,
aggregation, analytics
and decision-making
Source: IDC
Join the conversation – use the hashtag #HITTesting on Twitter

16. LEADING EHR’S SUPPORT THE CLOUD
Company System Cloud Hosting?
1 Cerner Corporation Cerner PowerChart Ambulatory EHR Yes
2 Epic Systems Corporation EpicCare Ambulatory - Core EMR Yes
3 Allscripts Allscripts Professional™ EHR" Yes
4
NextGen Healthcare Information
Systems Inc*
NextGen Ambulatory EHR Yes
5 athenahealth athenaClinicals Yes
*
As of 2013 http://medicaleconomics.modernmedicine.com/medical-
economics/content/tags/top100ehrs/top-100-ehr-companies-part-1-4
Join the conversation – use the hashtag #HITTesting on Twitter

17. OTHER TOOLS AVAILABLE IN THE CLOUD
Tool Category Value Representative Vendors
Test Case Management
Centralize and store test cases,
report on test progress and
coverage
qTest
Collaboration
Centralize requirements and
planning documents
Confluence
Communication
Chat and share relevant issues
with teams in real time
HipChat, Slack
Test Environment
Leverage cloud resources to
build test environments
AWS, SauceLabs
Test Data Management
Define, Subset, and Mask data
from production for testing
Informatica, Delphix
Defect Management
Submit, monitor, and track
defects and issues found in test
& production systems
JIRA, ServiceNow
Join the conversation – use the hashtag #HITTesting on Twitter

18. QA and Testing Impact
• Application level security is an aspect
of quality
• Cross-site scripting and sql-injection are
common hacker techniques
Guidance
• Explore test automation tools available
to identify security vulnerabilities
• Be aware of false positives
• Focus on the OWASP Top 10
SECURITYTESTING
By 2015, 50% of
healthcare
organizations will have
experienced 1 to 5
cyberattacks in the last
12 months with 1 out of
3 attacks deemed
successful
Source: IDC
Join the conversation – use the hashtag #HITTesting on Twitter

19. THE GREAT PII DEBATE
Source: https://www.healthit.gov/policy-researchers-implementers/faqs/what-information-will-be-sent-providers-test-ehr
Join the conversation – use the hashtag #HITTesting on Twitter

20. QA and Testing Impact
• Testers with specialist skills are can be difficult
to find
• Specialist may come at premium rates
Guidance
• Provider your organizations hiring managers
with specific screening criteria
• Benchmark specialist salaries and, talent-
wise, be wary of the low end
• Some specialties will require training and/or
certification (e.g. Epic)
TEST SPECIALIZATION
In 2015, new
development projects
will exceed maintenance
projects. Testing for new
development (including
SMAC) requires
specialist skills.
Source: 2014-15 World Quality Report -
Capgemini, HP, Sogeti
Join the conversation – use the hashtag #HITTesting on Twitter

21. • Mix of Certified Epic Professionals and experience HITTesters
• 1460+ EpicTest Cases
– 8-10 test steps on average across multiple modules
• Epic Modules: ADT, ADT Prelude,Ambulatory, Anesthesia,Aria, ASAP (ED), Cadence,
Clarity Reports, ClinDoc, Cupid, Dietary, Downstream Interfaces, HIM, Home Health
Clinical, Home Health Hospice, Home Health Billing, Hospital BillingClaims, Interfaces,
iSite, Lab, My Chart, Optime,Orders, PhysicalOutpatientThera, PowerScribe,
Professional Billing, Professional BillingClaims, Radiant, Stork,Willow Inpatient,Willow
Ambulatory
• 120+ other applications to be tested including such: as Apache Outcomes, Aria,
Cloverleaf, HealthTouch, Midas, eScription,NurseCall, Patient Call Manager, Perinatal,
ResponderNet, SMS,Teletracking, RALS, Softlab, Pyxis, Perinatal….
EXAMPLE: EHRTESTING ENGAGEMENT
Join the conversation – use the hashtag #HITTesting on Twitter

22. INTEROPERABILITY STILL BELOW 50%
Source: http://dashboard.healthit.gov/evaluations/faca/HITJC_%20Data_Update_on_Interoperability_2016-07-27.pdf
Join the conversation – use the hashtag #HITTesting on Twitter

23. MANY EXTERNAL BARRIERS TO ADOPTION
Source: http://avalere-health-production.s3.amazonaws.com/uploads/1460384367_201604_Pew.jpg
Join the conversation – use the hashtag #HITTesting on Twitter

24. INTEROPERABILITY WILL RULE…SOON
Source: https://pbs.twimg.com/media/CZ0fozGWQAA_rcT.png
Join the conversation – use the hashtag #HITTesting on Twitter

25. HEAT MAP – SAMPLE PAYER CLIENT
ASSESSMENT
Join the conversation – use the hashtag #HITTesting on Twitter

26. QA and Testing Impact
• May need to test Multiple Device/OS
combinations (IOS, Android, Windows)
• Performance may vary depending on
network conditions (3G, 4G, LTE)
Guidance
• Testing with emulators is insufficient
consider using real devices on a private
or public mobile device cloud
• Don’t forget about performance testing
• Use of network virtualization and
service virtualization can improve
accuracy of testing
MOBILETESTING
By 2015, mobile app
projects will outnumber
PC projects 4 to 1
Source: Gartner
65% of consumer
transactions with
healthcare
organizations will be
mobile by 2018
Source: IDC
Join the conversation – use the hashtag #HITTesting on Twitter

27. • Leverage Healthcare domain expertise inTesting
• Meet critical regulatory requirements
• Improve application quality and usability
• Reduce security risks
• Capitalize on macro trends around healthcare technology (i.e. big data,
mobile, interoperability, agile)
• Troubleshoot and remediate business, functional and performance
defects
SUMMARY BENEFITS
Join the conversation – use the hashtag #HITTesting on Twitter

28. Join the conversation – use the hashtag #HITTesting on Twitter

29. Robust test management platform purpose-built to help agile teams
centralize, organize and accelerate software testing
ABOUT QASYMPHONY

30. • Clients include large health systems, hospitals, payers and
life sciences companies, including Scripps Health, Catholic
Health Initiatives, Philips Health, Excellus BCBS and Shire
• Offer staffing and advisory QA andTesting solutions.
• Deep database of certified QA Analysts, software testers,
test automation specialists, performance and load testers
with EMR and healthcare-specific test experience
• Fastest-growing IT staffing company inU.S.*
• Learn more at www.healthcareitleaders.com
ABOUT HEALTHCARE IT LEADERS
* 2016 Staffing Industry Analysts, Fastest Growing Staffing Companies
Join the conversation – use the hashtag #HITTesting on Twitter

31. Thank you
Join the conversation – use the hashtag #HITTesting on Twitter