1 Extreme Automation Enables Satellite TV Provider to ”Shift Left” API Testing
”Shifting Left” Testing with Extreme Automation
Recently, the Enterprise Integration group, which is responsible for ensuring the integrity of the
REST and SOAP APIs associated with the company’s business enablement systems, grew concerned
that the accelerating pace of development was cutting into the time available for testing. They
began exploring ways to optimize their testing efforts so that shrinking test windows would not
require them to compromise their exhaustive testing and stringent quality processes.
After exploring several API testing solutions, the group determined that Parasoft’s solution was the
best fit for their needs due to its mature enterprise-grade functionality, extensive set of built-in
validation tools, and ability to create robust, flexible, and highly-reusable test assets.
By applying extreme automation to their API testing, the Enterprise Integration group was able
to significantly reduce the time required to construct, execute, evaluate, and manage their test
scenarios. Shortly after adopting Parasoft’s API Testing solution, the team quickly developed into
“power users,” taking full advantage of the solution’s flexibility to automate their complex test and
validation processes. As a result, they can not only test significantly faster than they did previously,
but also test more extensively.
The team’s investment in automation has helped them achieve nearly 100% regression test
coverage. “Previously, it took us 3 days to run a regression test suite that consisted of about 500
test cases which primarily covered the highest-traffic transactions,” the Enterprise Integration
group’s QA Lead explained. “Now, in the same 3 days, we’re able to cover approximately 3500 test
cases—and we’re adding from 400-1000 new test cases to the suite with each release.”
This expanded test coverage has made it much easier for the team to uncover many types of edge
scenario issues that are extremely time-consuming to find via manual testing. “Software issues
escaped to production have a very direct impact on customer satisfaction, so we’ve always made it
top priority to maintain an industry-low rate of escaping defects,” the group’s QA Lead noted. “With
this automation, we’ve been able to further reduce the rate of escaping defects by about 60-70%.”
Committed to leading the market in customer satisfaction, DirecTV is continuously enhancing
and upgrading the business enablement systems that drive communications across the
disparate systems critical to their business. Ensuring positive, seamless customer experiences
with everything from programming, to billing, to customer support depends on data being
properly translated and moved between core systems. Using Parasoft API Testing solution to
automate a complex manual testing process, the company’s Enterprise Integration group has
dramatically increased the speed and scope of their testing—enabling them to bring
top-quality software innovations to market in record time. This helps DirecTV foster high
customer satisfaction and bolster their competitive edge.
Extreme Automation Enables
DirecTV to ”Shift Left” API Testing
2 Extreme Automation Enables Satellite TV Provider to ”Shift Left” API Testing
Advancing from Manual Testing to Extreme Automation
Some specific ways that the team squeezed more testing into shorter test cycles include:
Previously With Parasoft API Testing Solution
The team had to wait for the preparation and
delivery of data that met their very specific
data requirements in each iteration—then
manually incorporate the data into their test
scenarios.
Tests automatically retrieve and use the
most recent data from the appropriate data
sources at the time of test execution. No
waiting or test updating is required.
Manual inspection of the response message
was required to determine whether test
transactions were successful and, if not, to
identify exactly where the problem occurred.
Given that a single manual validation could
take up to an hour, validation of the complete
message (with thousands of rows of data)
simply was not feasible. Tests focused
exclusively on data that was expected to
change.
5000+ fully-automated validations all run
completely unattended and report results
through the HP ALM (the team’s primary test
management system). Each end-to-end test
includes myriad intra-process validations to
ensure that even the most subtle problems
are exposed and to help the team rapidly
pinpoint the source of test failures. These
tests cover corner cases and expose any
regressions impacting existing functionality.
Tests had to be manually repeated in the
various test environments (development,
staging, triage, etc.).
Flexible tests are built once, and reused
across all of the required environments with
zero test updating required. This allows them
to begin testing in each environment the
moment that the application under test is
deployed there.
Manual testing required a very specific skill
set. Each time a domain expert left the team,
it was difficult to find someone with the
expertise required to continue the same level
of testing.
Automated tests capture domain experts’
knowledge. Tests can be executed by any
team member, and new team members
can get up to speed quickly by studying the
existing set of test artifacts.
Mitigating the Risks of Middleware Migration
Having a robust set of reusable tests is also helping the team ensure the integrity of core functionality
during their multi-year middleware migration process. The test assets they built over time essentially
place “bracers” around the legacy functionality. Whenever a part of the infrastructure is updated,
the team’s existing test suites automatically determine if the modifications impacted the existing
functionality.
This automated regression testing allows the company to upgrade with confidence—enabling
them to continuously optimize the performance and reliability of their systems without risking any
negative impact to customer or partner experiences.
3
© Parasoft Corporation All rights reserved. Parasoft and all Parasoft products and services listed within are trademarks or registered trademarks of Parasoft Corporation.
All other products, services, and companies are trademarks, registered trademarks, or servicemarks of their respective holders in the US and/or other countries.
USA PARASOFT HEADQUARTERS
101 E. Huntington Drive, Monrovia, CA 91016
Phone: (888) 305-0041, Email: info@parasoft.com
Extreme Automation Enables Satellite TV Provider to ”Shift Left” API Testing
Next Steps: ”Shifting Left” Further into Development
The teams’ success at “shifting left” their regression testing, which is now completed in parallel
with their functional testing, has already had a major impact on the speed and length of delivery
cycles. The next step in their optimization efforts will be to roll out the solution to the development
team so they can create and execute basic tests for each service before passing it on to QA.
In addition to enabling test execution, reporting, and analysis directly within the HP ALM environment
where testers spend the majority of their time, Parasoft API Testing solution also integrates into
Eclipse, which is the company’s standard development environment. After completing a service,
developers can use the solution to build basic tests that verify whether that service works properly.
With these service-level tests augmenting their current unit testing efforts, developers will have
additional feedback on whether a service is truly “done” before move they move on to the next
development task. These tests will also extend the development regression test suite, reducing the
risk that an inadvertent change to existing functionality could progress to the next phase of the SDLC.
In terms of QA, the Enterprise Integration group’s QA Lead anticipates that this approach will yield
several benefits:
	 Since defects will be exposed earlier, QA will gain a more stable testing environment.
A defect that escapes development can impact multiple systems, preventing the QA team
frombeingabletoexecutefunctionalorregressiontests.Duetothehighly-interconnected
nature of their systems, just one defect can commonly limit or block testing for up to
4 days.
	 The developers’ tests will serve as “building blocks” that QA can integrate and extend as
they design their end-to-end test scenarios.
	 Collaborating with development, QA can start designing functional tests even earlier
in each cycle.

Extreme Automation Enables DirecTV to ”Shift Left” API Testing

  • 1.
    1 Extreme AutomationEnables Satellite TV Provider to ”Shift Left” API Testing ”Shifting Left” Testing with Extreme Automation Recently, the Enterprise Integration group, which is responsible for ensuring the integrity of the REST and SOAP APIs associated with the company’s business enablement systems, grew concerned that the accelerating pace of development was cutting into the time available for testing. They began exploring ways to optimize their testing efforts so that shrinking test windows would not require them to compromise their exhaustive testing and stringent quality processes. After exploring several API testing solutions, the group determined that Parasoft’s solution was the best fit for their needs due to its mature enterprise-grade functionality, extensive set of built-in validation tools, and ability to create robust, flexible, and highly-reusable test assets. By applying extreme automation to their API testing, the Enterprise Integration group was able to significantly reduce the time required to construct, execute, evaluate, and manage their test scenarios. Shortly after adopting Parasoft’s API Testing solution, the team quickly developed into “power users,” taking full advantage of the solution’s flexibility to automate their complex test and validation processes. As a result, they can not only test significantly faster than they did previously, but also test more extensively. The team’s investment in automation has helped them achieve nearly 100% regression test coverage. “Previously, it took us 3 days to run a regression test suite that consisted of about 500 test cases which primarily covered the highest-traffic transactions,” the Enterprise Integration group’s QA Lead explained. “Now, in the same 3 days, we’re able to cover approximately 3500 test cases—and we’re adding from 400-1000 new test cases to the suite with each release.” This expanded test coverage has made it much easier for the team to uncover many types of edge scenario issues that are extremely time-consuming to find via manual testing. “Software issues escaped to production have a very direct impact on customer satisfaction, so we’ve always made it top priority to maintain an industry-low rate of escaping defects,” the group’s QA Lead noted. “With this automation, we’ve been able to further reduce the rate of escaping defects by about 60-70%.” Committed to leading the market in customer satisfaction, DirecTV is continuously enhancing and upgrading the business enablement systems that drive communications across the disparate systems critical to their business. Ensuring positive, seamless customer experiences with everything from programming, to billing, to customer support depends on data being properly translated and moved between core systems. Using Parasoft API Testing solution to automate a complex manual testing process, the company’s Enterprise Integration group has dramatically increased the speed and scope of their testing—enabling them to bring top-quality software innovations to market in record time. This helps DirecTV foster high customer satisfaction and bolster their competitive edge. Extreme Automation Enables DirecTV to ”Shift Left” API Testing
  • 2.
    2 Extreme AutomationEnables Satellite TV Provider to ”Shift Left” API Testing Advancing from Manual Testing to Extreme Automation Some specific ways that the team squeezed more testing into shorter test cycles include: Previously With Parasoft API Testing Solution The team had to wait for the preparation and delivery of data that met their very specific data requirements in each iteration—then manually incorporate the data into their test scenarios. Tests automatically retrieve and use the most recent data from the appropriate data sources at the time of test execution. No waiting or test updating is required. Manual inspection of the response message was required to determine whether test transactions were successful and, if not, to identify exactly where the problem occurred. Given that a single manual validation could take up to an hour, validation of the complete message (with thousands of rows of data) simply was not feasible. Tests focused exclusively on data that was expected to change. 5000+ fully-automated validations all run completely unattended and report results through the HP ALM (the team’s primary test management system). Each end-to-end test includes myriad intra-process validations to ensure that even the most subtle problems are exposed and to help the team rapidly pinpoint the source of test failures. These tests cover corner cases and expose any regressions impacting existing functionality. Tests had to be manually repeated in the various test environments (development, staging, triage, etc.). Flexible tests are built once, and reused across all of the required environments with zero test updating required. This allows them to begin testing in each environment the moment that the application under test is deployed there. Manual testing required a very specific skill set. Each time a domain expert left the team, it was difficult to find someone with the expertise required to continue the same level of testing. Automated tests capture domain experts’ knowledge. Tests can be executed by any team member, and new team members can get up to speed quickly by studying the existing set of test artifacts. Mitigating the Risks of Middleware Migration Having a robust set of reusable tests is also helping the team ensure the integrity of core functionality during their multi-year middleware migration process. The test assets they built over time essentially place “bracers” around the legacy functionality. Whenever a part of the infrastructure is updated, the team’s existing test suites automatically determine if the modifications impacted the existing functionality. This automated regression testing allows the company to upgrade with confidence—enabling them to continuously optimize the performance and reliability of their systems without risking any negative impact to customer or partner experiences.
  • 3.
    3 © Parasoft CorporationAll rights reserved. Parasoft and all Parasoft products and services listed within are trademarks or registered trademarks of Parasoft Corporation. All other products, services, and companies are trademarks, registered trademarks, or servicemarks of their respective holders in the US and/or other countries. USA PARASOFT HEADQUARTERS 101 E. Huntington Drive, Monrovia, CA 91016 Phone: (888) 305-0041, Email: info@parasoft.com Extreme Automation Enables Satellite TV Provider to ”Shift Left” API Testing Next Steps: ”Shifting Left” Further into Development The teams’ success at “shifting left” their regression testing, which is now completed in parallel with their functional testing, has already had a major impact on the speed and length of delivery cycles. The next step in their optimization efforts will be to roll out the solution to the development team so they can create and execute basic tests for each service before passing it on to QA. In addition to enabling test execution, reporting, and analysis directly within the HP ALM environment where testers spend the majority of their time, Parasoft API Testing solution also integrates into Eclipse, which is the company’s standard development environment. After completing a service, developers can use the solution to build basic tests that verify whether that service works properly. With these service-level tests augmenting their current unit testing efforts, developers will have additional feedback on whether a service is truly “done” before move they move on to the next development task. These tests will also extend the development regression test suite, reducing the risk that an inadvertent change to existing functionality could progress to the next phase of the SDLC. In terms of QA, the Enterprise Integration group’s QA Lead anticipates that this approach will yield several benefits: Since defects will be exposed earlier, QA will gain a more stable testing environment. A defect that escapes development can impact multiple systems, preventing the QA team frombeingabletoexecutefunctionalorregressiontests.Duetothehighly-interconnected nature of their systems, just one defect can commonly limit or block testing for up to 4 days. The developers’ tests will serve as “building blocks” that QA can integrate and extend as they design their end-to-end test scenarios. Collaborating with development, QA can start designing functional tests even earlier in each cycle.