Scenarios or Why Some Automation Projects Fail - Webinar Presentation

®
WEBINAR: Automation Demystified
Part 6: Scenarios, or Why Some Automation
Projects Fail
• July 24th, 2019 – Alexey Grinevich, alexey@inflectra.com
We will be starting the webinar shortly, please stand by…
All phones will be automatically on mute until the Q&A.

®
2 | 7/24/2019 © Copyright 2006-2019 Inflectra Corporation
Webinar Series: Useful Information
• Housekeeping Matters
• Post Webinar Follow-Up:
• Blog
• Webinar Recording
• Presentation On Slideshare
• Post-Webinar Quiz
• Webinar Series: Test Automation,
Demystified
• Webinar Series Certificate Rules &
FAQs

®
Hey, I am Alexey
• Working on Rapise test
automation tool @Inflectra
• 20 years in software
development
• 10 years in test automation
• Twitter: @AlexeyGrinevich
• Email: alexey@Inflectra.com

®
Topics
 Why Automation may Fail
 Test Case Features
 Test Set Features
 Test Set Design

®
Test Automation Efficiency #1

®

®
Cost
Time

®
Single Manual Test Case
Cost
Iterations

®
Ideal Automated Test Case
Cost
Iterations

®
Good Automated Test Case
Cost
Iterations

®
Troublemaker
Cost
Iterations

®
Goals
 Maximize Ideal
 Keep Good
 Avoid Toublemakers

®
Maintainability

®
Crashes
 404, 403, 500
 Problem in application?
 Problem for test?
 Packaged application?
 Need to clear in advance.

®
Base State
Account Settings
Billing
HomeRequest Help
Search
Purchase Check Out

®
Test Case Length
 1min vs 10min
 20 runs to create
 20 runs to fix

®
Data Driven Test Cases
 May take more time
 It is not a problem
 Each row should be executable
 Report should be clear about data row
 Limited number of objects involved

®
Test Case: Short, Simple, Stupid
 Regular expressions: antipattern

®
Click Everywhere

®
Click Everywhere
 About 20 Controls
 20! = 2.5 * 10E18 combinations
 Billion 1,000,000,000 (9 zeros)
Trillion 1,000,000,000,000 (12 zeros)
Quadrillion 1,000,000,000,000,000 (15 zeros)
Quintillion 1,000,000,000,000,000,000 (18 zeros)
 Over 2 Quintillion combinations

®
Redundant Testing
 Per-step checks
 Invariants
 Should have a reason
 System stable

®
External Dependencies
 APIs
 More stable than UI
 The older the better
 Example: Exchange Web Services vs Office 365 REST API

®
Test Set Level: System Deployment
 Easy vs Hard
 Staging or Prod
 VMs and VM server

®
Test Set Level: Sanity Checks
 100 UI test cases
 30 sec per test case
 DB failure
 => 50 min of time loss
 Sanity checks:
 Common functionality
 DB
 System Status (ready/deployment/maintenance)
 Connection
 Disk Size(!!!)

®
System Maturity
 Focus on stable areas
 More mature module – more comprehensive test
 Unstable areas – manual or unit

®
Test Set: Data Feed
 100k generated => 1 issue
 50 manual => 3 issues

®
Typical Scenarios
 Security Testing
 https://github.com/Inflectra/rapise-dynamics-
samples/tree/master/DynamicsAXSecurityTestingFramework
 Load Testing
 Rapise NeoLoad

®
Test Set and CI
 CI is unattended
 Not all test cases for CI

®
Test Framework
Leads Contacts Accounts Quotes Orders Products Admin Dashboard ...
Login
Security
Search
Create
Edit
Promote
Clone
View
Delete

®
Manual Test (promote Lead)
Login
Security
Search
Create
Edit
Promote
Clone
View
Delete
v
v
v v
v
v v
v
v v
v

®
Auto Test
Login
Security
Search
Create
Edit
Promote
Clone
View
Delete
v
VVV

®
Security Testing Pattern
Login
Security
Search
Create
Edit
Promote
Clone
View
...
v
VVV VVV VVV VVV VVV VVV VVV VVV

®
Grey Areas
Login
Security
Search
Create
Edit
Promote
Clone
...

®
Summary
 Test Case: Simple, Short, Stupid (old T)
 Test Set: Smart (T)
 Beware: Grey Areas, Maturity, Crashes, Requirements

®
Next Webinar
 AI in Test Automation
 We hear about AI everywhere. Unfortunately, there is no public
plan or roadmap on when AI is going to replace humans
as software testers.
 Presenter: Alexey Grinevich– automation expert at Inflectra
 https://www.inflectra.com/Company/Event/live-webinar-ai-
in-test-automation-858.aspx

®
Questions are Welcome
Contact me
• Twitter: @AlexeyGrinevich
• Email: alexey@inflectra.com
• Web: https://www.inflectra.com/Rapise/

®
Q&A
Thank You for Watching
Subscribe to Our Channel for More Videos on Software Testing

Scenarios or Why Some Automation Projects Fail - Webinar Presentation

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Scenarios or Why Some Automation Projects Fail - Webinar Presentation

Similar to Scenarios or Why Some Automation Projects Fail - Webinar Presentation (20)

More from Inflectra

More from Inflectra (20)

Recently uploaded

Recently uploaded (20)

Scenarios or Why Some Automation Projects Fail - Webinar Presentation

Editor's Notes