SFO15-409: Introduction to QA Reports Speaker: Milosz Wasilewski Date: September 24, 2015 ★ Session Description ★ Reporting and visualization of test results are the most critical piece of a continuous integration system. QA Reports is a prototype web front end to help developers wade through the massive amounts of test results in a sane manner. This session will provide a brief overview of the application, and then it will be an open discussion on how to improve this solution. ★ Resources ★ Video: https://www.youtube.com/watch?v=UV6rsgae3yc Presentation: Etherpad: pad.linaro.org/p/sfo15-409 Pathable: https://sfo15.pathable.com/meetings/303081 Video on server (if unable to view on YouTube): https://s3.amazonaws.com/connect.linaro.org/sfo15/Videos/09-24-Thursday/SFO15-409 Introduction to QA Reports.mp4 Presentation on server (if unable to view on Slideshare): ★ Event Details ★ Linaro Connect San Francisco 2015 - #SFO15 September 21-25, 2015 Hyatt Regency Hotel http://www.linaro.org http://connect.linaro.org

1. Presented by
Date
Event
SFO15-409: Introduction to
QA reports
Milosz WasilewskiMilosz Wasilewski
Thursday 24 September 2015
SFO15

2. Overview
Presentation of testing results
Sources of test results
Manual testing
Reporting
Known issues recognition

3. Presentation of testing results
● Overall status for all builds (dashboard)
● 3 states
○ green
○ yellow
○ red
● Result grouping
● Data backend

4. Presentation of testing results cont.
● Overall status for all builds (dashboard)

5. Presentation of testing results cont.
● 3 state status (red/yellow/green)

6. Presentation of testing results cont.
● 3 state status
(red/yellow/green)

7. Presentation of testing results cont.
● result grouping

8. Sources of test results
● Automated tests:
○ LAVA
○ Pyboot
○ Jenkins
○ …
● Manual tests

9. Manual testing
● Work in progress
○ Viewing test cases
○ Preparing test plans
○ Test execution

10. Reporting
● Proposals/ideas:
○ Release reporting
○ Trend graphing

11. Known issues/defects recognition
● Similar results comparison
○ UI based
○ email notification
● Manual monitoring
● Automated monitoring