This document discusses the need for automated testing in digital health applications. It notes that 35% of prescribing errors and an average of 35 safety incidents per month in the UK's National Health Service are software-related. Manual testing is still the primary method used, but automated testing could help avoid errors by providing better testing coverage and eliminating human mistakes. The document provides an example where automating 270 test cases for a health management application reduced regression testing time from 300 hours to 15 hours. Automated testing can help lower project costs, accelerate solution delivery, and increase quality and safety.

1. Mojdeh Toyserkani, B.Eng, PMP, CPHIMS-CA
Aversan Inc.
30 Eglington Avenue West, Suite 500
Mississauga, ON L5R 3E7
Why Digital Health Needs Automated Testing?
INTRODUCTION
Aversan empowers leading health care organizations to build,
deploy and maintain highly available and adaptable applications
that are designed to improve the quality of patient care, reduce
per capita cost of care and seamlessly integrate with various
clinical and enterprise systems.
THE PROBLEM
A recent study of e-prescribing systems in hospitals revealed
35% of prescribing errors are software related 1
The National Health Service has identified and documented
an average of 35 software safety incidents every month. 2
All these errors could be avoided if better testing was
performed.
.
COST TO FIX
This diagram shows the cost to fix software defects escalates
exponentially from the design phase through the
maintenance phase.3
1
10
100
1000
RelativeCostRatio(Log)
OUR RESEARCH DATA
Marketing research conducted to learn about the type and
method of testing applied in many health care organizations
including hospitals in Ontario and British Columbia reveals that
primary method of testing in these organizations is still manual
testing. Among the concerns we hear are:
 We were not aware of automated method of testing and its
associated benefits
 Software vendors do not support automated testing
 Automated testing will add cost
TEST AUTOMATION FOR A
HEALTH SYSTEM MANAGEMENT APPLICATION
AIM
The biggest inherent risks and challenges in this ecosystem are with the following types of testing:
• System Integration & Interoperability
• User interface testing
This e-poster is aimed to provide an illustration of how Test Automation can reduce the risks
inherent with the above challenges and is a far better choice in many cases over Manual Testing.
WHAT IS TEST AUTOMATION?
Project Background:
• Client with ~1000s of Manual Tests: In need to maintain a
legacy application
• Application upgrades and enhancements : Never enough time
or $$ to complete a full regression, only a subset run regularly
What we did:
• Automated a subset of 270 selected test cases
• Automation effort: 4 weeks, with 2 FTEs
• Automated: 15 hours to run regression (overnight)
• Manual: 300 hours to run regression (business hours)
Lower Project costs by
Reducing test cycles - Months to Hours
Reusing tests for future upgrades
Accelerate Solution Delivery with:
 24/7 automation
 Reusable test components
Increase Quality and Safety by:
 Improved test coverage
 Eliminating human error
 Detecting un-intended changes
RESULTS:
We believe there needs to be a business case for test
automation, in other words a return on investment
(ROI). One needs to make the case for the initial
investment in the test automation methodology, tools
and setup, which then generates a payback based on
use.
When calculating the ROI, one needs to look at two
types of ROI:
 Patient Safety ROI
 Financial ROI
We can all agree that avoiding human harm or death
due to software errors in a digital health system is
priceless and hence Test Automation is a sure
approach to eliminate the risk on these errors.
CONCLUSIONS:
TEST AUTOMATION AREAS OF APPLICATION:
TEST AUTOMATION DEMO ON OSCAR EMR 15.0
ACKNOWLEDGMENT
Thanks Dennis Giokas, our former Business and Strategy advisor for his thoughtful advise and
leadership in preparing the original abstract for this presentation. Thanks Sana Rashid for creating
the video and Sucheta Mehra for designing the poster.
REFERENCES
FACTS are from 2013 eSafety Guidelines – Available on Coach Store :
1- J. Westbrook et al. “Effects of Two Commercial Electronic Prescribing Systems on Prescribing
Error Rates in Hospital In-Patients: A Before and After Study.” PLoS Med 9(1): e1001164.doi:
10.1371/journal.pmed.101164.
2- J. Car et al. The Impact of eHealth on the Quality and Safety of Healthcare: A Systemic Overview
and Synthesis of the Literature. Report for the NHS Connecting for Health Evaluation Programme.
The University of Edinburgh and The Imperial College of London, March 2008.
3. Ref: Error Cost Escalation Through the Project Life Cycle - NASA Johnson Space Center
www.aversan.com
In software testing, test automation is the use of
special software to control the execution of tests and
the comparison of actual outcomes with predicted
outcomes.