Testing bottlenecks and the late detection of defects make Continuous Delivery impossible. This talk will examine how this can be resolved, making the exhaustive testing of any basic UI or API possible within 30 minutes. For more information, please visit http://cainc.to/Nv2VOe

1. Exhaustively Test Any Basic UI or API
in 30 Minutes
The Ideal for Any Organization Serious About
Continuous Delivery
Benjamin Johnson-Ward
DevOps: Continuous Delivery
CA Technologies
Presales Consultant
DO4X85E
@GridTools
#CAWorld

2. 2 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
© 2015 CA. All rights reserved. All trademarks referenced herein belong to their respective companies.
The content provided in this CA World 2015 presentation is intended for informational purposes only and does not form any type of
warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA.
For Informational Purposes Only
Terms of this Presentation

3. 3 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Abstract
Testing bottlenecks and the late detection of
defects make Continuous Delivery impossible.
This talk will examine how this can be resolved,
making the exhaustive testing of any basic UI or
API possible within 30 minutes.
Benjamin
Johnson-
Ward
CA Technologies
Presales Consultant

4. 4 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Agenda
COMMON TESTING CHALLENGES
BREAKING DOWN THE TESTING INTO COMPONENTS
DEMO & SUMMARY
DEMO: FORMERLY MODELLING EXISTING TEST ASSETS
DEMO: AUTO-GENERATE TEST CASES & SCRIPTS
STORING TEST COMPONENTS AS RE-USABLE ASSETS
1
2
3
4
5
6

5. 5 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Why Bother?
 Over 60% of IT projects fail1
 And 31.1% will be cancelled before completion
2
 Only 16.2% are completed on-time and on-budget
 While testing takes half the time in the SDLC
 Testing can eat up to half of the budget
3
 $312 billion a year is spent on software bugs
4
1 & 2 – Standish Chaos Manifesto 2014
3 – Bloor, 2014
4 – Cambridge University Judge Business School, 2013

6. 6 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Design Dev. Test. Automation Deploy
A Typical SDLC: Too Manual, Too Linear, and Too Slow

7. 7 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Requirements are Practically Static, are Ambiguous,
and are Incomplete.

8. 8 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
The Results are Often Poor.
Over 50% of defects are introduced in the design phase

9. 9 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Manual Test Case Design
 Currently manual – a time consuming, error-prone process
 Is unsystematic, ad hoc, and has no real notion of “coverage”
 Over-testing and under-testing – 10-20% coverage with 4 times over-testing
 No linkage to test data – process is manual, painstaking and very time-consuming
 No flexibility for change requests: a critical weakness in an agile or Continuous
Delivery environment. Changes take longer than the original requirement!

10. 10 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Why Automation is Not a “Silver Bullet” for Testing
 Automated testing frameworks are heavily scripted
 Script generation is usually done manually
 As is script maintenance
 Alternative solutions, use:
 Record Playback
 Use script-less automation frameworks (keyword)
 But you’re back to Manual test case design

11. 11 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
 Test cases and scripts are created automatically from
“Active” requirements
 They are executed automatically
 Testing is Model Based for maximum coverage
 Use a Component Library of common and optimized
tests
 Test Data and Virtual End Points are created or found as
part of the Automation
 Automating the automation
Active Automation
Active Automation

12. 12 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Active Automation with CA Test Case Optimizer
 Increase the level of automation
 Re-use and standardize common test components
 Adapt quickly to change
 Maintain high levels of coverage
 Support Continuous Development frameworks

13. 13 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Model Based Testing – Step by Step

14. 14 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Linked Processes – Apply the Same Technique

15. 15 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Model Based Testing – Step by Step
CUSTOMER NUMBER:
CUSTOMER NAME:
CUSTOMER COUNTRY:
DATA UPDATED
123456
HUW PRICE
UK

16. 16 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Validation Coverage ALL POSSIBLE ALL PAIRS ALL EDGES

17. 17 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Model Based Testing – Process Coverage
ALL POSSIBLEALL PAIRS
ALL EDGES & PATH HINTS
THE DECISION MODEL

18. 18 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Model Based Testing – Use Libraries of Test Cases

19. 19 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Model Based Testing – Use Libraries of Test Cases

20. 20 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Model Based Testing – Use Libraries of Test Cases

21. 21 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Active Automation with CA Test Case Optimizer
 Increase the level of automation
 Reuse and standardize common test components
 Adapt quickly to change
 Maintain high levels of coverage
 Support Continuous Development frameworks

22. 22 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
 Test cases and scripts are created automatically from
“Active” requirements
 They are executed automatically
 Testing is Model Based for maximum coverage
 Use a Component Library of common and optimized
tests
 Test Data and Virtual End Points are created or found as
part of the Automation
 Automating the automation
Active Automation
Active Automation

23. 23 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Automated Tests: Manual Generation
 Automated testing frameworks are heavily scripted
 Script generation is usually done manually
- As well as the maintenance
 Alternative solution:
Use scriptless automation frameworks (keyword)
– But you’re back to Manual test case design
 Optimal solution:
From automated test case generation ->
Through automated script generation->
To automated execution
- Automating the automation

24. 24 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Q & A

25. 25 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Recommended Sessions
SESSION # TITLE DATE/TIME
DO4T15S
Case Study: Nordstrom Takes Retail Relief for Test Data
Management
11/18/2015 at 11:30 am
DO4T05S
Vision & Roadmap: The CA Roadmap for Test Data
Management
11/18/2015 at 1:00 pm
DO4T26T
Tech Talk: Fines in the Millions Levied Every Year Coming
Soon—The Business Case for Synthetic Data Generation
11/18/2015 at 2:00 pm

26. 26 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Must-See Demos
Test Data
Manager
DevOps
Theater 4
DevOps Sim
DevOps
Theater 3
Test Case
Optimizer
DevOps
Theater 4
TDM
Integrations
DevOps
Theater 4

27. 27 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
Follow On Conversations At…
Smart Bar
DevOps
Theater 4
Tech Talks
DevOps
Theater 4

28. 28 © 2015 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD
For More Information
To learn more, please visit:
http://cainc.to/Nv2VOe
CA World ’15