5. Myth #1 - Test Automation is simple, that every tester can do itMyth #1 - Test Automation is simple, that every tester can do it
• This myth is promoted by the tool sales people.
They are trying to promote the following test
automation process:
o Record the script
o Enhance the script by adding functions and data
driving
o Run the scripts
o Report results
• Under the influence of this myth QA manager
can proudly report: All our testers are
developing test automation.
6. Reality - Test automation is a software development taskReality - Test automation is a software development task
• Automation should be designed, developed and
tested
• You need to have some kind of a programming
background to implement test automation. Test
Automation is not as complex as C++/C#/Java
development.
• Test automation standards should be developed
• Automated test components are assets that should
be treated like application source code
7. Myth #2 – Commercial test tools are expensiveMyth #2 – Commercial test tools are expensive
• Under the influence of this myth some companies,
especially the small ones:
o Try to develop their own test automation tools
o Use scripting languages like Perl and Ruby
o Use shareware test tools
o Do not consider test automation at all
8. Reality – Commercial tools are cheapReality – Commercial tools are cheap
• Per seat license for most expensive automation tool
is $8K
• This tool will be used for 5 years.
• Maintenance/Support fees are 20% of tool cost or
$1,800 per year
• The cost of this tool is $8K/5+$1,800 = $3,100 per year
• The automation developer cost with overhead is
$100K per year
• The cost of this tool is just 3% of the person who uses
it, but productivity gain can be very significant
9. Commercial Tool Benefits:Commercial Tool Benefits:
• Customer support. Many of the open source tools come and go
with little to no support
• Most commercial tools are constantly being updated as
technologies change
• Most commercial tools usually have more functionality (QTP can
test various GUI applications: Web, .Net, Java, VB, C/C++,
PowerBuilder, etc. vs. WATIR – Web only)
• Commercial tools usually have a large community of users, which
translates into better availability of qualified resources
• Commercial tools require less advanced programming
• More test automation frameworks are available for commercial
tools
• Commercial tools are integrated with Test Management tools
which makes reporting and execution much simpler.
10. Dice search results across US (30 days)Dice search results across US (30 days)
Tool Search string Matches
QTP Qtp OR quicktest OR "quick test“ 613
Functional Tester "rational robot" OR "functional tester” 118
SilkTest silktest OR "silk test” 86
TestComplete testcomplete OR "test complete” 26
TestPartner testpartner OR "test partner“ 19
VSTE VSTE AND test 45
WATIR watir 26
Selenium Selenium 108
11. Demand per tool
HP QTP
60%
Open Source
WATIR
2%
Microsoft
VSTE
4%
AutomatedQA
TestComplete
2%
Compuware
TestPartner
2%
Borland
SilkTest
8%
IBM
Functional
Tester
12%
Open Source
Selenium
10%
QTP
Functional Tester
SilkTest
TestComplete
TestPartner
VSTE
WATIR
Selenium
12. Excerpt from IDC Report:
“Worldwide Automated Software Quality
2007-2011 Forecast and 2006 Vendor
Shares…”
15. Classic ROI CalculationClassic ROI Calculation
ROI = BENEFIT/COST
Automation Cost = Price Of HW + Price of SW + Development Cost +
Maintenance Cost + Execution Cost
Manual Testing Cost = Development Cost + Maintenance Cost +
Execution Cost
ROI = (Manual Testing Cost - Automation Cost)/Automation Cost
Looks right, isn’t it?
16. Problems with Classic ROI CalculationProblems with Classic ROI Calculation
You can’t compare Automated Testing
and Manual Testing. There are not the
same and they provide different
information about the AUT.
You can’t compare cost of multiple
execution of automated tests vs. manual
tests. You would never dream of
executing that many test cases
manually.
17. Automation Real ROIAutomation Real ROI
ROI value IS NOT the value of Automation vs. Cost
of executing these tests manually
Automation ROI value IS the benefit of this type of
testing, and it can be:
Reducing Time to Market
Increased Test Efficiency (Productivity)
Increased Test Effectiveness
18. Reduced Time to MarketReduced Time to Market
• Can get a greater market share
• Makes people available to work on
other projects
• Higher margins, if no competitive
products are currently available
19. Productivity and EffectivenessProductivity and Effectiveness
• More testing gets done faster, increasing the odds of
finding defects
• Defects found early have better chances to be fixed
• Manual Testers can concentrate on clever ways to
finding defects, instead of typing test inputs and verify
output.
• About 7% of bug fixes create new bugs, sometimes in
already tested parts of the system. With automation you
can rerun tests for those modules. This almost never
happened when testing done manually.
20. ROI summaryROI summary
• Each project requires different types of automation
- there is often no easy formula available to
calculate ROI
• Performing ROI calculation can help to determine
upfront what type of automation, what level of skills,
what tools will be required.
22. Test Automation FrameworkTest Automation Framework
• A Test Automation Framework is a set of
assumptions, concepts and tools that provide
support for Automated Software Testing.
• Correctly implemented Test Automation Framework
can further improve ROI by reducing the
development and maintenance costs.
23. Types of Test AutomationTypes of Test Automation
FrameworksFrameworks
• Modular
• Data-Driven
• Keyword–Driven
• Model-Based
24. Modular frameworkModular framework
• The Modular framework is the natural progression
from Record-and-Playback
• The modular framework seeks to minimize this
repetition of code by grouping similar actions into
“modules” (e.g.: login)
• Test Data is in a script
This is what tool sales people are promoting.
25. Data-Driven/Keyword DrivenData-Driven/Keyword Driven
• These frameworks are similar in that the data is
separated from the test script
• The script is just a "driver" or delivery mechanism for
the data.
• The difference:
o In keyword-driven testing, the navigation data and test data are
contained in the data source
o In data-driven testing, only test data is contained in the data source.
26. Model-Based testingModel-Based testing
• Model-based testing is software testing in which test
cases are derived in whole or in part from a model
that describes some (usually functional) aspects of
the system.
• Model-based testing for complex software systems
is still an evolving field.
27. We’ll be concentrating on Keyword-Driven
testing since it is a most beneficial framework
for large-scale test automation.
28. Advantages of Keyword-Driven Test automationAdvantages of Keyword-Driven Test automation
• This Framework addresses the most common problem
with test automation:
Automation Engineers do not have domain knowledge
and the End Users (Subject Matter Experts/Test
Engineers) usually do not have automation expertise.
• When properly implemented and maintained, it presents
a superior ROI because each business event is designed,
automated and maintained as a discrete entity.
• Keywords can then be used to design test cases, but the
design and automation overhead for the keyword has
already been paid.
29. Advantages of Keyword-Driven Test automationAdvantages of Keyword-Driven Test automation
• Reduced the cost and time spent maintaining
and updating tests
• The modular structure of keyword-driven testing
means that new tests can easily be created
from pre-existing modules
• The test team is capable of entirely automating
tests, even without programming knowledge
• Can be easily modified to use with different test
tool
• Reusability across different projects
31. Recent Success with Keyword-Driven approachRecent Success with Keyword-Driven approach
• Estimate using Modular Framework: 1 year
• Estimate with Keyword-Driven Framework: 6 months
Project was completed in 4 months by 1 person, 2 month
ahead of schedule.
This was the first project for the automation developer
utilizing the Keyword-Driven Framework.
170 components were developed (1 component per Web
page).
109 Regression Test cases were automated.
32. Example of Test Data for Keyword-Driven test automation developed byExample of Test Data for Keyword-Driven test automation developed by
Connected Testing Inc. for TestComplete and QTPConnected Testing Inc. for TestComplete and QTP
33. ThankThank You !!!You !!!
For More Information click below link:
Follow Us on:
http://vibranttechnologies.co.in/qtp-classes-in-mumbai.html