Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

3

Share

Download to read offline

Test cases and bug report v3.2

Download to read offline

DataArt IT school in Odessa (Autumn/Winter 2014)

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Test cases and bug report v3.2

  1. 1. Test case and BUG report Andrey Oleynik 2014
  2. 2. Agenda 1. Test Case. 2. Examples. 3. BDD. 4. Examples. 5. Defects 6. BUG Report 7. Examples 8. Good BUG 9. Additional links 2
  3. 3. About myself Andrey Oleynik • About 7 years of work in Quality Assurance area • QA Automation Engineer • PhD, Theoretical physics 3
  4. 4. What is a Test Case? • A set of input values, execution preconditions, expected results and execution postconditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement. [IEEE] • Documentation specifying inputs, predicted results, and a set of execution conditions for a test item. [ANSI] • A test case is a bunch of steps that lead to consider one "item" as successfully working. 4
  5. 5. Why do we write test cases? • The basic objective of writing test cases is to validate the testing coverage of the application • Usually QA engineers follow the test cases standards • So writing test cases brings some sort of standardization and minimizes the ad-hoc approach in testing 5
  6. 6. What information would the test manager want out of test case document/s? • Which features have been tested/ will be tested eventually? • How many user scenarios/ use cases have been executed? • How many features are stable? • Which features need more work? • Are sufficient input combinations exercised? • Is software ready to ship? Is testing enough? • What is the quality of the application? 6
  7. 7. Test Case design • Specification based testing; • Input Domain testing; • Risk based testing; • Scenario testing. 7
  8. 8. Test Case Fields (the most important) • Test Case id • Title • Priority • Description • Steps • Expected result • Feature/Area 8
  9. 9. Test Case fields • Test case status (Idea/Design/Ready/Update/etc.) • Product version • Responsible person • Preparation notes • Notes to clean the system • Attachments • Examples • Linked feature/requirement/bug • ... 9
  10. 10. HP ALM example 10
  11. 11. ALM example (Test Case steps) 11
  12. 12. Execution example (ALM) 12
  13. 13. Example Title: Open file. Description: a file can be opened in the Application. 1. Launch the Application. 2. Select "File" menu. File menu pulls down. 3. Choose "Open“. "Open" dialog box appears. 4. Select a file to open. 5. Click OK. Expected result: File should be opened. 13
  14. 14. Important: 1. Template 2. Descriptive and specific 3. Reusable 4. Atomic 5. Positive and negative 6. Refactor 7. Test data 8. Setup and tear down 9. Test Case review 10. Screenshots 14
  15. 15. Test Suite • A test suite is a set of related tests, usually pertaining to a group of features or software component and usually defined over the same database. Suites are combined into groups. [Beizer] 15
  16. 16. Test management tools • Gemini • HP ALM (QC) • IBM Rational Quality Manager • Meliora Testlab • Seapine TestTrack • TestLink • Google Docs ! • … 16
  17. 17. Q & A 17
  18. 18. Testing pyramid 18
  19. 19. TDD Test-driven development is a software development methodology which essentially states that for each unit of software, a software developer must: • define a test set for the unit first; • then implement the unit; • finally verify that the implementation of the unit makes the tests succeed. 19
  20. 20. TDD • Java (Junit, TestNG); • C (Check, CUnit); • C++ (Boost, CppUnit, CppUnitLite, Google Test); • C# (xUnit, NUnit, MSTest); • Ruby (Test::Unit, Shoulda, RSpec); • Python (pyUnit, py.test, doctest); • … 20
  21. 21. BDD • Behavior-driven development is a specialized version of test-driven development which focuses on behavioral specification of software units. • Behavior-driven development specifies that tests of any unit of software should be specified in terms of the desired behavior of the unit. 21
  22. 22. BDD 22 Title The story should have a clear, explicit title. Narrative A short, introductory section that specifies •who •which effect the stakeholder wants the story to have •what business value the stakeholder will derive from this effect Acceptance criteria or scenarios a description of each specific case of the narrative. •It starts by specifying the initial condition •It then states which event triggers the start of the scenario. •Finally, it states the expected outcome, in one or more clauses.
  23. 23. BDD example 23 Feature: New account In order to have new ready to use account As a user I want to get an account with balance $0 Scenario: Create new account Given I have no account When I create an account Then I should have an account with balance $0
  24. 24. BDD • Jbehave (Java) • Thucidydes • Rspec (Ruby) • Cucumber • SpecFlow (C#) • … 24
  25. 25. Q & A 25
  26. 26. What is a BUG (defect)? • A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g. an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system. 26
  27. 27. BUG (Harvard Mark II, 1947) 27
  28. 28. Another BUG definition • A software bug is an error, flaw, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways.[Wikipedia] 28
  29. 29. BUG names • Mistake • Anomaly • Fault • Failure • Error • Exception • Crash • Bug • Defect • Incident • Side Effect • Change request 29
  30. 30. Types of BUGs • UI • Functional issues • Programming logic • Installation • Memory allocation • System resources • Documentation • Requirements • Localization (L10N) • Internalization (I18N) • Performance • Security • … 30
  31. 31. How to find a bug? • Understand the whole application in depth • Prepare good test cases and test data before testing • Execute tests with different test environment • Compare actual and expected results 31
  32. 32. What to do next? • Do monkey testing • Use test data pattern from previous versions/projects • Check standard scenarios from other products • Try to break the application! 32
  33. 33. BUG report Defect/BUG report: A document reporting on any flaw in a component or system that can cause the component or system to fail to perform its required function. [IEEE] 33 “The point of writing problem report(bug report) is to get bugs fixed” [Cem Kaner].
  34. 34. Defect Fields (the most important) • Bug ID • Summary • Description/Steps to Reproduce • Expected/Actual result • Status • Severity • Priority • Detected by • Assigned to • Module/Area 34
  35. 35. Defect Fields • Environment • Reproducible • Fixed in Version • Detected Date • Modified/Last Updated • Detected in Version/Build Version • Comments • Attachments • Linked testcases/requirements/defects 35
  36. 36. Defect Life Cycle 36
  37. 37. Defect Resolution 37
  38. 38. Defect Resolution • Fixed • Won't Fix • Not Reproducible • Duplicate • By Design • External / 3d party 38
  39. 39. Defect Severity 1 - Urgent • (Critical) • Data loss or corruption. Product is down. Customer operations are impacted very seriously. There is no any workaround. • For documentation: copyright or offensive content issues, documentation steps lead to data loss or corruption. 2- High • (Serious) • System / feature is still usable to a high degree but has a serious issue. Defect has serious customer business impact. • Some data is at risk. No obvious workaround. • For documentation: core functionality is not described. 39
  40. 40. Defect Severity 3 - Medium • Product operational • Application customer data is not at risk. • Obvious/easy workaround available. • Non informative error reporting. • For documentation: Conceptual information is incorrect or incomplete. 4 - Low • It is more of a cosmetic issue. No serious impedance to system functionality is noted. Annoyances. • For documentation: typography, formatting. 40
  41. 41. Priority 0. Show-stopper - this needs to be fixed right now, everything else can wait, the build cannot be released with this defect 1. Urgent - needs to be fixed before any other high, medium or low defect should be fixed 2. High - should be fixed as early as possible 3. Medium - should take precedence over low priority defects 4. Low - fixing can be deferred until all other priority defects are fixed 41
  42. 42. JIRA example 42
  43. 43. ALM/QC example 43
  44. 44. BUG example Title/Summary: Application crash on clicking the Browse button while file open. Area: File operations Build Number: x.x.x Severity: HIGH Priority: HIGH Status: New/Open/Active Environment: Windows 2008 Steps To Reproduce: 1) Launch the application 2) Open menu File -> Open. 3) Click on ‘Browse’ button Actual result: Application crash. Expected result: On clicking Browse button file browse dialogue should be opened. Notes: the screenshot and the logs are attached. 44
  45. 45. Good BUG 1. Short and precise title 2. Concise but complete description: Steps to reproduce, consequences of the bug, and suggested behavior 3. Good attachments 4. Complete definition of the categorizing fields 5. Correct Severity & Priority 6. Follow-up and comment 45
  46. 46. Good BUG 7. Include the Results you Expected 8. Include the Results you Actually Observed 9. Explain the Impact on the Customer 10. Avoid speculation 11. Be careful of the tone of your report 12. Avoid duplication - search first (!) 46
  47. 47. Collaboration • QA engineers and developers are not in fight but are partners • The main thing is product and its quality 47
  48. 48. BUG Tracking Systems • Gemini • HP ALM (QC) • IBM Rational Quality Manager • JIRA • Seapine TestTrack • Bugzilla • Google Docs ! • … 48
  49. 49. More details 49 • Software testing Help: http://www.softwaretestinghelp.com/ • Про Тестинг: http://www.protesting.ru/ • http://software-testing.ru • What is severity and priority in software testing? http://istqbexamcertification.com/what-is-severity- and-priority-in-software-testing/ • IEEE Std 1044-1993 IEEE Standard Classification for Software Anomalies http://standards.ieee.org/findstds/standard/1044- 1993.html • ISTQB® Glossary http://www.istqb.org/downloads/viewcategory/20 .html
  50. 50. More details 50 • Сэм Канер, Джек Фолк Тестирование программного обеспечения • Роман Савин Тестирование Дот Ком, или Пособие по жестокому обращению с багами в интернет-стартапах • Луиза Тамре Введение в тестирование программного обеспечения
  51. 51. More details 51 • Introducing BDD by Dan North http://dannorth.net/introducing-bdd/ • Портал об автоматизации тестирования ПО: http://automated-testing.info/ • Конференции sqadays: http://www.sqadays.com • Software testing Help: http://www.softwaretestinghelp.com/ • Про Тестинг: http://www.protesting.ru/ • http://software-testing.ru
  52. 52. More details 52 • Э.Дастин, Дж. Рэшка, Дж.Пол Автоматизированное тестирование программного обеспечения. Внедрение, управление и эксплуатация • Matt Wynne and Aslak Hellesoy The Cucumber Book: Behaviour-Driven Development for Testers and Developers • Криспин, Грегори Гибкое тестирование
  53. 53. Q & A 53
  54. 54. The End • Thank you! 54
  • TatianaOlefirova

    Jul. 4, 2019
  • DaedAlJohary

    Dec. 17, 2016
  • ravim7

    Feb. 2, 2016

DataArt IT school in Odessa (Autumn/Winter 2014)

Views

Total views

4,607

On Slideshare

0

From embeds

0

Number of embeds

22

Actions

Downloads

72

Shares

0

Comments

0

Likes

3

×