SlideShare a Scribd company logo
1 of 33
TEST AUTOMATION.
А ЧТО НА ВЫХОДЕ?
QA FEST
PAVLO BASIUK, ITERA
What is test automation outcome?
System Under
Test
Tests inputs Tests results
What is test automation outcome?
System Under
Test
Tests inputs Tests results
Precondition data
Environmental status
Program start state
Environmental changes
Program end state
Post condition data
Program end state
Post condition data
Environmental changes
Point of view
Automation engineers
QA team
Developers
Customer, QA Manager
Test Report should be:
Requirements for report
• Structured
– Test-suites, test-cases, test-steps
– Filters for results (to see only pass/fail/other)
• Summarized
– High-level overview of results, so it should take less time to understand
how good results are
– Summary per feature/tag/component
• Rich
– Screenshots, additional info (e.g. attachments, system configuration)
• Detailed
– With all input data used
– With all actions done and all expected/actual results
– Execution time (for test-suites, test-cases)
• Easy to create
– We don’t want to spend extra time to prepare any reports
Analyzing Results
• test environment
• application changes
• test errors
False failure
• test defect
• missed error
False success
• same error reported
• dependent tests failed
Duplicate failures
Examples
Real-life reports
Hand-made Html Report Overview
Hand-made Html Report Details
ReportNG
ReportNG sample overview
ReportNG sample details
ReportNG real-case overview
ReportNG real-case details
Thucydides
Thucydides overview
04.11.2014 / 18
Thucydides details
04.11.2014 / 20
Ranorex
04.11.2014
Allure
Allure overview
Failures are groupped by assertion
Test suites
Test case details
Even more details on test steps
Nice charts
Possible tweaks
Test statuses
Input data
Special modes
Resource utilization
Known bugs during test run
Semi-automatic defect report
Possible improvements
Review (again) who is target audience
What are main goals for report
Were do you spend most time
References
1. AT.info http://automated-testing.info
2. ReportNG http://reportng.uncommons.org
3. QA Automation Notes http://qa-automation-notes.blogspot.com
4. Thucydides http://thucydides.info
5. Ranorex http://ranorex.com
6. Allure http://allure.qatools.ru
Thank you!
Any questions?
QA Fest 2014. Павел Басюк. Test automation: а что на выходе?

More Related Content

What's hot

Severity & priority in software testing
Severity & priority in software testingSeverity & priority in software testing
Severity & priority in software testingmedsherb
 
ENASE 2013 - SEM - (Francia) From Functional Test Scripts to Performance Test...
ENASE 2013 - SEM - (Francia) From Functional Test Scripts to Performance Test...ENASE 2013 - SEM - (Francia) From Functional Test Scripts to Performance Test...
ENASE 2013 - SEM - (Francia) From Functional Test Scripts to Performance Test...Federico Toledo
 
Banking Case study: UAT Support for CBS Migration
Banking Case study: UAT Support for CBS MigrationBanking Case study: UAT Support for CBS Migration
Banking Case study: UAT Support for CBS MigrationOak Systems
 
Pa chapter08-testing integrating-the_programs-cs_390
Pa chapter08-testing integrating-the_programs-cs_390Pa chapter08-testing integrating-the_programs-cs_390
Pa chapter08-testing integrating-the_programs-cs_390Rana Tallal
 
Performance testing Web Application - A complete Guide
Performance testing Web Application - A complete GuidePerformance testing Web Application - A complete Guide
Performance testing Web Application - A complete GuideTestingXperts
 
Monitoring your physical, virtual and cloud infrastructure with Applications ...
Monitoring your physical, virtual and cloud infrastructure with Applications ...Monitoring your physical, virtual and cloud infrastructure with Applications ...
Monitoring your physical, virtual and cloud infrastructure with Applications ...ManageEngine, Zoho Corporation
 
Motor vehicle emission checker danu-lap
Motor vehicle emission checker danu-lapMotor vehicle emission checker danu-lap
Motor vehicle emission checker danu-lapaidsdatahub
 
VCS_QAPerformanceSlides
VCS_QAPerformanceSlidesVCS_QAPerformanceSlides
VCS_QAPerformanceSlidesMichael Cowan
 
Corp implementation presentation 2
Corp implementation presentation 2Corp implementation presentation 2
Corp implementation presentation 2Slide.Works
 
The right way to manage your load testing project
The right way to manage your load testing projectThe right way to manage your load testing project
The right way to manage your load testing projectPerfecto Mobile
 
What is Automation Testing?
What is Automation Testing?What is Automation Testing?
What is Automation Testing?QA InfoTech
 

What's hot (16)

Qulaity strategy template
Qulaity strategy templateQulaity strategy template
Qulaity strategy template
 
Severity & priority in software testing
Severity & priority in software testingSeverity & priority in software testing
Severity & priority in software testing
 
[Mush Honda] Metrics & Reports from Test Teams (QA)
[Mush Honda] Metrics & Reports from Test Teams (QA)[Mush Honda] Metrics & Reports from Test Teams (QA)
[Mush Honda] Metrics & Reports from Test Teams (QA)
 
ENASE 2013 - SEM - (Francia) From Functional Test Scripts to Performance Test...
ENASE 2013 - SEM - (Francia) From Functional Test Scripts to Performance Test...ENASE 2013 - SEM - (Francia) From Functional Test Scripts to Performance Test...
ENASE 2013 - SEM - (Francia) From Functional Test Scripts to Performance Test...
 
Banking Case study: UAT Support for CBS Migration
Banking Case study: UAT Support for CBS MigrationBanking Case study: UAT Support for CBS Migration
Banking Case study: UAT Support for CBS Migration
 
Pa chapter08-testing integrating-the_programs-cs_390
Pa chapter08-testing integrating-the_programs-cs_390Pa chapter08-testing integrating-the_programs-cs_390
Pa chapter08-testing integrating-the_programs-cs_390
 
RaviTeja_Resume
RaviTeja_ResumeRaviTeja_Resume
RaviTeja_Resume
 
Performance testing Web Application - A complete Guide
Performance testing Web Application - A complete GuidePerformance testing Web Application - A complete Guide
Performance testing Web Application - A complete Guide
 
Monitoring your physical, virtual and cloud infrastructure with Applications ...
Monitoring your physical, virtual and cloud infrastructure with Applications ...Monitoring your physical, virtual and cloud infrastructure with Applications ...
Monitoring your physical, virtual and cloud infrastructure with Applications ...
 
Motor vehicle emission checker danu-lap
Motor vehicle emission checker danu-lapMotor vehicle emission checker danu-lap
Motor vehicle emission checker danu-lap
 
Improving User Experience with Applications Manager
Improving User Experience with Applications ManagerImproving User Experience with Applications Manager
Improving User Experience with Applications Manager
 
VCS_QAPerformanceSlides
VCS_QAPerformanceSlidesVCS_QAPerformanceSlides
VCS_QAPerformanceSlides
 
Corp implementation presentation 2
Corp implementation presentation 2Corp implementation presentation 2
Corp implementation presentation 2
 
The right way to manage your load testing project
The right way to manage your load testing projectThe right way to manage your load testing project
The right way to manage your load testing project
 
An Insight Into Software Testing
An Insight Into Software TestingAn Insight Into Software Testing
An Insight Into Software Testing
 
What is Automation Testing?
What is Automation Testing?What is Automation Testing?
What is Automation Testing?
 

Similar to QA Fest 2014. Павел Басюк. Test automation: а что на выходе?

ISTQB / ISEB Foundation Exam Practice - 2
ISTQB / ISEB Foundation Exam Practice - 2ISTQB / ISEB Foundation Exam Practice - 2
ISTQB / ISEB Foundation Exam Practice - 2Yogindernath Gupta
 
Fundamentals of software 2 | Test Case | Test Suite | Test Plan | Test Scenario
Fundamentals of software 2 | Test Case | Test Suite | Test Plan | Test ScenarioFundamentals of software 2 | Test Case | Test Suite | Test Plan | Test Scenario
Fundamentals of software 2 | Test Case | Test Suite | Test Plan | Test ScenarioSiddharth Sharma
 
ISTQBCH2.ppt
ISTQBCH2.pptISTQBCH2.ppt
ISTQBCH2.pptghkadous
 
C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010
C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010
C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010TEST Huddle
 
Branching Strategy 2.0 - NASCAR Digital Media
Branching Strategy 2.0 - NASCAR Digital MediaBranching Strategy 2.0 - NASCAR Digital Media
Branching Strategy 2.0 - NASCAR Digital MediaJoshua Miller
 
Software testing performance testing
Software testing  performance testingSoftware testing  performance testing
Software testing performance testingGaneshKumarKanthiah
 
ISTQB, ISEB Lecture Notes- 2
ISTQB, ISEB Lecture Notes- 2ISTQB, ISEB Lecture Notes- 2
ISTQB, ISEB Lecture Notes- 2onsoftwaretest
 
ISTQB Foundation - Chapter 2
ISTQB Foundation - Chapter 2ISTQB Foundation - Chapter 2
ISTQB Foundation - Chapter 2Chandukar
 
Test Automation
Test AutomationTest Automation
Test Automationrockoder
 
Bab 1 Fundamentals Of Testing
Bab 1 Fundamentals Of TestingBab 1 Fundamentals Of Testing
Bab 1 Fundamentals Of Testinglolayoriva
 
Testing process
Testing processTesting process
Testing processTerry Cho
 
Software Testing Process
Software Testing ProcessSoftware Testing Process
Software Testing Processguest1f2740
 
Performance Assurance for Packaged Applications
Performance Assurance for Packaged ApplicationsPerformance Assurance for Packaged Applications
Performance Assurance for Packaged ApplicationsAlexander Podelko
 
Software engineering quality assurance and testing
Software engineering quality assurance and testingSoftware engineering quality assurance and testing
Software engineering quality assurance and testingBipul Roy Bpl
 

Similar to QA Fest 2014. Павел Басюк. Test automation: а что на выходе? (20)

ISTQB / ISEB Foundation Exam Practice - 2
ISTQB / ISEB Foundation Exam Practice - 2ISTQB / ISEB Foundation Exam Practice - 2
ISTQB / ISEB Foundation Exam Practice - 2
 
Fundamentals of software 2 | Test Case | Test Suite | Test Plan | Test Scenario
Fundamentals of software 2 | Test Case | Test Suite | Test Plan | Test ScenarioFundamentals of software 2 | Test Case | Test Suite | Test Plan | Test Scenario
Fundamentals of software 2 | Test Case | Test Suite | Test Plan | Test Scenario
 
Stlc ppt
Stlc pptStlc ppt
Stlc ppt
 
ISTQBCH2.ppt
ISTQBCH2.pptISTQBCH2.ppt
ISTQBCH2.ppt
 
ISTQBCH2.ppt
ISTQBCH2.pptISTQBCH2.ppt
ISTQBCH2.ppt
 
C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010
C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010
C.V, Narayanan - Open Source Tools for Test Management - EuroSTAR 2010
 
Branching Strategy 2.0 - NASCAR Digital Media
Branching Strategy 2.0 - NASCAR Digital MediaBranching Strategy 2.0 - NASCAR Digital Media
Branching Strategy 2.0 - NASCAR Digital Media
 
Software testing performance testing
Software testing  performance testingSoftware testing  performance testing
Software testing performance testing
 
ISTQB, ISEB Lecture Notes- 2
ISTQB, ISEB Lecture Notes- 2ISTQB, ISEB Lecture Notes- 2
ISTQB, ISEB Lecture Notes- 2
 
ISTQB Foundation - Chapter 2
ISTQB Foundation - Chapter 2ISTQB Foundation - Chapter 2
ISTQB Foundation - Chapter 2
 
Performance testing
Performance testingPerformance testing
Performance testing
 
Test Automation
Test AutomationTest Automation
Test Automation
 
Bab 1 Fundamentals Of Testing
Bab 1 Fundamentals Of TestingBab 1 Fundamentals Of Testing
Bab 1 Fundamentals Of Testing
 
Test Automation for QTP
Test Automation for QTPTest Automation for QTP
Test Automation for QTP
 
Test Automation
Test AutomationTest Automation
Test Automation
 
Testing process
Testing processTesting process
Testing process
 
Software Testing Process
Software Testing ProcessSoftware Testing Process
Software Testing Process
 
Testing
TestingTesting
Testing
 
Performance Assurance for Packaged Applications
Performance Assurance for Packaged ApplicationsPerformance Assurance for Packaged Applications
Performance Assurance for Packaged Applications
 
Software engineering quality assurance and testing
Software engineering quality assurance and testingSoftware engineering quality assurance and testing
Software engineering quality assurance and testing
 

More from QAFest

QA Fest 2019. Сергій Короленко. Топ веб вразливостей за 40 хвилин
QA Fest 2019. Сергій Короленко. Топ веб вразливостей за 40 хвилинQA Fest 2019. Сергій Короленко. Топ веб вразливостей за 40 хвилин
QA Fest 2019. Сергій Короленко. Топ веб вразливостей за 40 хвилинQAFest
 
QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The Future
QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The FutureQA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The Future
QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The FutureQAFest
 
QA Fest 2019. Doug Sillars. It's just too Slow: Testing Mobile application pe...
QA Fest 2019. Doug Sillars. It's just too Slow: Testing Mobile application pe...QA Fest 2019. Doug Sillars. It's just too Slow: Testing Mobile application pe...
QA Fest 2019. Doug Sillars. It's just too Slow: Testing Mobile application pe...QAFest
 
QA Fest 2019. Катерина Спринсян. Параллельное покрытие автотестами и другие и...
QA Fest 2019. Катерина Спринсян. Параллельное покрытие автотестами и другие и...QA Fest 2019. Катерина Спринсян. Параллельное покрытие автотестами и другие и...
QA Fest 2019. Катерина Спринсян. Параллельное покрытие автотестами и другие и...QAFest
 
QA Fest 2019. Никита Галкин. Как зарабатывать больше
QA Fest 2019. Никита Галкин. Как зарабатывать большеQA Fest 2019. Никита Галкин. Как зарабатывать больше
QA Fest 2019. Никита Галкин. Как зарабатывать большеQAFest
 
QA Fest 2019. Сергей Пирогов. Why everything is spoiled
QA Fest 2019. Сергей Пирогов. Why everything is spoiledQA Fest 2019. Сергей Пирогов. Why everything is spoiled
QA Fest 2019. Сергей Пирогов. Why everything is spoiledQAFest
 
QA Fest 2019. Сергей Новик. Между мотивацией и выгоранием
QA Fest 2019. Сергей Новик. Между мотивацией и выгораниемQA Fest 2019. Сергей Новик. Между мотивацией и выгоранием
QA Fest 2019. Сергей Новик. Между мотивацией и выгораниемQAFest
 
QA Fest 2019. Владимир Никонов. Код Шредингера или зачем и как мы тестируем н...
QA Fest 2019. Владимир Никонов. Код Шредингера или зачем и как мы тестируем н...QA Fest 2019. Владимир Никонов. Код Шредингера или зачем и как мы тестируем н...
QA Fest 2019. Владимир Никонов. Код Шредингера или зачем и как мы тестируем н...QAFest
 
QA Fest 2019. Владимир Трандафилов. GUI automation of WEB application with SV...
QA Fest 2019. Владимир Трандафилов. GUI automation of WEB application with SV...QA Fest 2019. Владимир Трандафилов. GUI automation of WEB application with SV...
QA Fest 2019. Владимир Трандафилов. GUI automation of WEB application with SV...QAFest
 
QA Fest 2019. Иван Крутов. Bulletproof Selenium Cluster
QA Fest 2019. Иван Крутов. Bulletproof Selenium ClusterQA Fest 2019. Иван Крутов. Bulletproof Selenium Cluster
QA Fest 2019. Иван Крутов. Bulletproof Selenium ClusterQAFest
 
QA Fest 2019. Николай Мижигурский. Миссия /*не*/выполнима: гуманитарий собесе...
QA Fest 2019. Николай Мижигурский. Миссия /*не*/выполнима: гуманитарий собесе...QA Fest 2019. Николай Мижигурский. Миссия /*не*/выполнима: гуманитарий собесе...
QA Fest 2019. Николай Мижигурский. Миссия /*не*/выполнима: гуманитарий собесе...QAFest
 
QA Fest 2019. Володимир Стиран. Чим раніше – тим вигідніше, але ніколи не піз...
QA Fest 2019. Володимир Стиран. Чим раніше – тим вигідніше, але ніколи не піз...QA Fest 2019. Володимир Стиран. Чим раніше – тим вигідніше, але ніколи не піз...
QA Fest 2019. Володимир Стиран. Чим раніше – тим вигідніше, але ніколи не піз...QAFest
 
QA Fest 2019. Дмитрий Прокопук. Mocks and network tricks in UI automation
QA Fest 2019. Дмитрий Прокопук. Mocks and network tricks in UI automationQA Fest 2019. Дмитрий Прокопук. Mocks and network tricks in UI automation
QA Fest 2019. Дмитрий Прокопук. Mocks and network tricks in UI automationQAFest
 
QA Fest 2019. Екатерина Дядечко. Тестирование медицинского софта — вызовы и в...
QA Fest 2019. Екатерина Дядечко. Тестирование медицинского софта — вызовы и в...QA Fest 2019. Екатерина Дядечко. Тестирование медицинского софта — вызовы и в...
QA Fest 2019. Екатерина Дядечко. Тестирование медицинского софта — вызовы и в...QAFest
 
QA Fest 2019. Катерина Черникова. Tune your P’s: the pop-art of keeping testa...
QA Fest 2019. Катерина Черникова. Tune your P’s: the pop-art of keeping testa...QA Fest 2019. Катерина Черникова. Tune your P’s: the pop-art of keeping testa...
QA Fest 2019. Катерина Черникова. Tune your P’s: the pop-art of keeping testa...QAFest
 
QA Fest 2019. Алиса Бойко. Какнезапутаться в коммуникативных сетях IT
QA Fest 2019. Алиса Бойко. Какнезапутаться в коммуникативных сетях ITQA Fest 2019. Алиса Бойко. Какнезапутаться в коммуникативных сетях IT
QA Fest 2019. Алиса Бойко. Какнезапутаться в коммуникативных сетях ITQAFest
 
QA Fest 2019. Святослав Логин. Как найти уязвимости в мобильном приложении
QA Fest 2019. Святослав Логин. Как найти уязвимости в мобильном приложенииQA Fest 2019. Святослав Логин. Как найти уязвимости в мобильном приложении
QA Fest 2019. Святослав Логин. Как найти уязвимости в мобильном приложенииQAFest
 
QA Fest 2019. Катерина Шепелєва та Інна Оснач. Що українцям потрібно знати пр...
QA Fest 2019. Катерина Шепелєва та Інна Оснач. Що українцям потрібно знати пр...QA Fest 2019. Катерина Шепелєва та Інна Оснач. Що українцям потрібно знати пр...
QA Fest 2019. Катерина Шепелєва та Інна Оснач. Що українцям потрібно знати пр...QAFest
 
QA Fest 2019. Антон Серпутько. Нагрузочное тестирование распределенных асинхр...
QA Fest 2019. Антон Серпутько. Нагрузочное тестирование распределенных асинхр...QA Fest 2019. Антон Серпутько. Нагрузочное тестирование распределенных асинхр...
QA Fest 2019. Антон Серпутько. Нагрузочное тестирование распределенных асинхр...QAFest
 
QA Fest 2019. Петр Тарасенко. QA Hackathon - The Cookbook 22
QA Fest 2019. Петр Тарасенко. QA Hackathon - The Cookbook 22QA Fest 2019. Петр Тарасенко. QA Hackathon - The Cookbook 22
QA Fest 2019. Петр Тарасенко. QA Hackathon - The Cookbook 22QAFest
 

More from QAFest (20)

QA Fest 2019. Сергій Короленко. Топ веб вразливостей за 40 хвилин
QA Fest 2019. Сергій Короленко. Топ веб вразливостей за 40 хвилинQA Fest 2019. Сергій Короленко. Топ веб вразливостей за 40 хвилин
QA Fest 2019. Сергій Короленко. Топ веб вразливостей за 40 хвилин
 
QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The Future
QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The FutureQA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The Future
QA Fest 2019. Анна Чернышова. Self-healing test automation 2.0. The Future
 
QA Fest 2019. Doug Sillars. It's just too Slow: Testing Mobile application pe...
QA Fest 2019. Doug Sillars. It's just too Slow: Testing Mobile application pe...QA Fest 2019. Doug Sillars. It's just too Slow: Testing Mobile application pe...
QA Fest 2019. Doug Sillars. It's just too Slow: Testing Mobile application pe...
 
QA Fest 2019. Катерина Спринсян. Параллельное покрытие автотестами и другие и...
QA Fest 2019. Катерина Спринсян. Параллельное покрытие автотестами и другие и...QA Fest 2019. Катерина Спринсян. Параллельное покрытие автотестами и другие и...
QA Fest 2019. Катерина Спринсян. Параллельное покрытие автотестами и другие и...
 
QA Fest 2019. Никита Галкин. Как зарабатывать больше
QA Fest 2019. Никита Галкин. Как зарабатывать большеQA Fest 2019. Никита Галкин. Как зарабатывать больше
QA Fest 2019. Никита Галкин. Как зарабатывать больше
 
QA Fest 2019. Сергей Пирогов. Why everything is spoiled
QA Fest 2019. Сергей Пирогов. Why everything is spoiledQA Fest 2019. Сергей Пирогов. Why everything is spoiled
QA Fest 2019. Сергей Пирогов. Why everything is spoiled
 
QA Fest 2019. Сергей Новик. Между мотивацией и выгоранием
QA Fest 2019. Сергей Новик. Между мотивацией и выгораниемQA Fest 2019. Сергей Новик. Между мотивацией и выгоранием
QA Fest 2019. Сергей Новик. Между мотивацией и выгоранием
 
QA Fest 2019. Владимир Никонов. Код Шредингера или зачем и как мы тестируем н...
QA Fest 2019. Владимир Никонов. Код Шредингера или зачем и как мы тестируем н...QA Fest 2019. Владимир Никонов. Код Шредингера или зачем и как мы тестируем н...
QA Fest 2019. Владимир Никонов. Код Шредингера или зачем и как мы тестируем н...
 
QA Fest 2019. Владимир Трандафилов. GUI automation of WEB application with SV...
QA Fest 2019. Владимир Трандафилов. GUI automation of WEB application with SV...QA Fest 2019. Владимир Трандафилов. GUI automation of WEB application with SV...
QA Fest 2019. Владимир Трандафилов. GUI automation of WEB application with SV...
 
QA Fest 2019. Иван Крутов. Bulletproof Selenium Cluster
QA Fest 2019. Иван Крутов. Bulletproof Selenium ClusterQA Fest 2019. Иван Крутов. Bulletproof Selenium Cluster
QA Fest 2019. Иван Крутов. Bulletproof Selenium Cluster
 
QA Fest 2019. Николай Мижигурский. Миссия /*не*/выполнима: гуманитарий собесе...
QA Fest 2019. Николай Мижигурский. Миссия /*не*/выполнима: гуманитарий собесе...QA Fest 2019. Николай Мижигурский. Миссия /*не*/выполнима: гуманитарий собесе...
QA Fest 2019. Николай Мижигурский. Миссия /*не*/выполнима: гуманитарий собесе...
 
QA Fest 2019. Володимир Стиран. Чим раніше – тим вигідніше, але ніколи не піз...
QA Fest 2019. Володимир Стиран. Чим раніше – тим вигідніше, але ніколи не піз...QA Fest 2019. Володимир Стиран. Чим раніше – тим вигідніше, але ніколи не піз...
QA Fest 2019. Володимир Стиран. Чим раніше – тим вигідніше, але ніколи не піз...
 
QA Fest 2019. Дмитрий Прокопук. Mocks and network tricks in UI automation
QA Fest 2019. Дмитрий Прокопук. Mocks and network tricks in UI automationQA Fest 2019. Дмитрий Прокопук. Mocks and network tricks in UI automation
QA Fest 2019. Дмитрий Прокопук. Mocks and network tricks in UI automation
 
QA Fest 2019. Екатерина Дядечко. Тестирование медицинского софта — вызовы и в...
QA Fest 2019. Екатерина Дядечко. Тестирование медицинского софта — вызовы и в...QA Fest 2019. Екатерина Дядечко. Тестирование медицинского софта — вызовы и в...
QA Fest 2019. Екатерина Дядечко. Тестирование медицинского софта — вызовы и в...
 
QA Fest 2019. Катерина Черникова. Tune your P’s: the pop-art of keeping testa...
QA Fest 2019. Катерина Черникова. Tune your P’s: the pop-art of keeping testa...QA Fest 2019. Катерина Черникова. Tune your P’s: the pop-art of keeping testa...
QA Fest 2019. Катерина Черникова. Tune your P’s: the pop-art of keeping testa...
 
QA Fest 2019. Алиса Бойко. Какнезапутаться в коммуникативных сетях IT
QA Fest 2019. Алиса Бойко. Какнезапутаться в коммуникативных сетях ITQA Fest 2019. Алиса Бойко. Какнезапутаться в коммуникативных сетях IT
QA Fest 2019. Алиса Бойко. Какнезапутаться в коммуникативных сетях IT
 
QA Fest 2019. Святослав Логин. Как найти уязвимости в мобильном приложении
QA Fest 2019. Святослав Логин. Как найти уязвимости в мобильном приложенииQA Fest 2019. Святослав Логин. Как найти уязвимости в мобильном приложении
QA Fest 2019. Святослав Логин. Как найти уязвимости в мобильном приложении
 
QA Fest 2019. Катерина Шепелєва та Інна Оснач. Що українцям потрібно знати пр...
QA Fest 2019. Катерина Шепелєва та Інна Оснач. Що українцям потрібно знати пр...QA Fest 2019. Катерина Шепелєва та Інна Оснач. Що українцям потрібно знати пр...
QA Fest 2019. Катерина Шепелєва та Інна Оснач. Що українцям потрібно знати пр...
 
QA Fest 2019. Антон Серпутько. Нагрузочное тестирование распределенных асинхр...
QA Fest 2019. Антон Серпутько. Нагрузочное тестирование распределенных асинхр...QA Fest 2019. Антон Серпутько. Нагрузочное тестирование распределенных асинхр...
QA Fest 2019. Антон Серпутько. Нагрузочное тестирование распределенных асинхр...
 
QA Fest 2019. Петр Тарасенко. QA Hackathon - The Cookbook 22
QA Fest 2019. Петр Тарасенко. QA Hackathon - The Cookbook 22QA Fest 2019. Петр Тарасенко. QA Hackathon - The Cookbook 22
QA Fest 2019. Петр Тарасенко. QA Hackathon - The Cookbook 22
 

Recently uploaded

Software Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerSoftware Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerAnchore
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdfHCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdfROWELL MARQUINA
 
Deliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceDeliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceOpsTree solutions
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationBuild Intuit
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...amber724300
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024BookNet Canada
 

Recently uploaded (20)

Software Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerSoftware Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey Hightower
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdfHCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
HCI Lesson 1 - Introduction to Human-Computer Interaction.pdf
 
Deliver Latency Free Customer Experience
Deliver Latency Free Customer ExperienceDeliver Latency Free Customer Experience
Deliver Latency Free Customer Experience
 
Dynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientationDynamical Context introduction word sensibility orientation
Dynamical Context introduction word sensibility orientation
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
JET Technology Labs White Paper for Virtualized Security and Encryption Techn...
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
 

QA Fest 2014. Павел Басюк. Test automation: а что на выходе?

Editor's Notes

  1. In this example, test automation outcome has output data (as result of test process, and test report as result from comparison process.
  2. Automation engineers Created all auto tests, so only need stacktrace in case of failure And in case if investigation required, then can read debug log QA team Need to reproduce failure, so should be able to accurately, step-by-step repeat automated test Can use automation test output for bug reports (screenshots, application logs, steps) Developers Need to know exact build/version/environment details Application logs, screenshots Access details if any (e.g. login/password to reproduce issue on qa env) Application snapshots (if possible) Customer, QA Manager Summary information, progress, application readiness for release
  3. ReportNG is a simple HTML reporting plug-in for the TestNG unit-testing framework. It is intended as a replacement for the default TestNG HTML report. The default report is comprehensive but is not so easy to understand at-a-glance. ReportNG provides a simple, colour-coded view of the test results. ReportNG generates 100% valid XHTML 1.0 files. The output can be customised by over-riding the default stylesheet with your own CSS file.
  4. На морде слева (см. первый скрин) список testng сьютов, обозванных по имени / версии браузера (берется из testng xml). 3) Каждый сьют кликабелен и перемещает нас к детальному отчету (см. второй скрин). 4) Помимо всего прочего, морда содержит даты запуска / завершения, длительность, результаты и pass rate. 5) Сверху можно еще найти информацию об экзекьюторе (юзер / доменное имя тачки, с которой были запущены тесты) + конфиг ее системы. 6) Из центральной части морды также можно перебраться на детализированный отчет по сьюту.
  5. 7) Внутренности содержат в шапке инфу о ноде (ip:port / domain name), на котором был заэкзекьючен сьют + конфигурация (ОС / браузер / версия). 8) Центральная часть содержит список всех тестов, разбитых на passed / failed + группировка по классам. Ну и базовая инфа: имя / длительность / степы с тестовыми данными пришедшими в качестве аргументов. 9) Опциональный линк на HarStorage с инфой по реквестам / респонсам теста. 10) Опциональный линк на видеозапись теста. 11) Список верификейшенов (выдранных из софт ассертов). При этом, каждый верификейшен содержит линк на релевантную стори в Jira. 12) Скрин последнего дыхания теста. 13) В случае фейла под скрином еще будет раскрывающийся линк со стек стрейсом + ID зафейлившихся верификейшенов.
  6. Введите 2 режима запуска тестов: Один для разработки и отладки скриптов А второй для полновесных запусков (которые могут занимать по несколько часов) Пересмотрите еще раз список статусов (результатов): А есть ли у вас статус Pending, для тестов которые уже закомичены, но еще не полностью готовы к запуску? А статус Broken, для тех тестов которые попадали не с AssertionException а с какой-то другой ошибкой? А где и как в отчете можно увидеть входные данные? А еще в отчете можно сохранять использование ресурсов (например, сколько оперативной памяти занимает тестируемое приложение). На этом можно еще и дополнительную проверку сделать, и фейлить тест если приложение начало использовать много памяти. And do you have any special handling for cases when you know for sure that bug appears? (e.g. Unhandled Exception just appears, and this is definitely a bug).