Make good use of explortary testing


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • En ANALOGI for å forklare ET!! Kjenner deltakerne til XP?? en “revolusjon” innen systemutvikling sammenliknet med tradisjonell “krav og spesifikasjons”-drevet (vannfalls)utvikling med fokus på små grupper, intern kommunikasjon, mindre dokumentasjon Analogi – likhet: XP vs. WaterFall ET vs. Scripted Testing
  • En ANALOGI for å forklare ET!! ET er IKKE tilfeldig Planned and Disciplined
  • The test-methods used proved very efficient, and can be conducted with a relatively small amount of resources. One should, however, consider the fact that more planning and “research” makes a better test, and that not all systems are suitable for this kind of testing.
  • Make good use of explortary testing

    1. 1. Liang
    2. 2. E.T. 没有测试用例? Bug 越测越多? 测试不可复现? 如何积累?
    3. 3. Agenda Definition of E.T. and it is differencebetween “Scripted Testing) How to do a good E.T. E.T testing management
    4. 4. Definition of E.T. Exploratory testing is simultaneous learning, test design, and testexecution"Exploratory testing involves simultaneously learning, planning,running tests, and reporting / troubleshooting results."Dr. Cem Kaner (2001)"Exploratory testing is an interactive process of concurrentproduct exploration, test design and test execution.”” To the extent that the next test we do is influenced by theresult of the last test we did, we are doing exploratorytesting.”James Bach, Satisfice (2001)
    5. 5. No E.T? It is Everywhere 场景:首轮测试之前对特性不熟悉,首轮测试用于用例熟悉,没有 Bug次论涌现出许多测试想法,开始发散 / 冒险 / 即兴 ,测出 Bug按照此方法,越测越有感觉, Bug 越多,总不收敛软件发布日期推迟 场景:规定测试人员每周必须发现的 Bug 的数量和质量测试人员:“照着测试用例测肯定找不到 Bug” ,开始发散 / 冒险 / 即兴
    6. 6. Different Between E.T andScriptedProduct ProductE.T.BugReviewReleaseTest PlanTest StrategyTest PlanReviewManual ExecutionBugReleaseAdapting
    7. 7. Difference between E.T. andScripted E.T.’s goal is to find Bug Scripted testing is to measure coverage,and be comprehensiveFullyScriptedTestingAd-hocTestingAutomatedTestsBugHuntingExploratory Testing
    8. 8. Agile software development is not conventional software developmentdone more quickly or done on tippie-toe. Agile software development issoftware done differently.Ron Jeffries, (e-mail on agile-testing list, April 24, 2002)•proven (no single technique is new)•application oriented•planned and disciplined•controllable and reliable•risk minimizing•Two sides of extreme programming:• for the developer: freedom, flexibility, fun• for the manager: controllability, reliability, highqualityExtreme Programming
    9. 9. Agile software testing is not conventional (scripted)software testing done more quickly or done on tippie-toe.Exploratory Testing:•proven (no single technique is new)•application oriented•planned and disciplined•controllable and reliable•risk minimizing•Two sides of Exploratory Testing:• for the tester: freedom, flexibility, fun• for the manager: controllability, reliability, highqualityExploratory Testing
    10. 10. Agenda Definition of E.T. and it is differencebetween “Scripted Testing) How to do a good E.T. E.T testing management
    11. 11. Let’s watch some Video first Play exploratory testing demo video
    12. 12. Make sure the right attitude Experiential and intuitive approaches:Develop a sense of smell“We have good hunches about where the bugsare lurking.”“Let’s jump in an explore the system’s behaviorhands-on, so we can decide how to test it.”“Let’s find the important bugs fast, and worryabout the test paperwork later.”
    13. 13. Doing Exploratory Testing Keep your mission clearly in mind. Keep notes that help you report what youdid, why you did it, and support yourassessment of product quality. Keep track of questions and issues raisedin your exploration. To supercharge your testing, pair upwith another tester and test the samething on the same computer at thesame time.
    14. 14. Taking Notes Test Coverage Outline/Matrix Evaluation Notes Risk/Strategy List Test Execution Log Issues, Questions & AnomaliesIt would be easier to test if youchanged/added…How does … work?Is this important to test? How should I test it?I saw something strange…
    15. 15. Approaches / Styles of ET At the heart of all ET styles:Questions and Questioning Skills Characterize the styles with respect to eachother:Do they focus on:○ Method of questioning?○ Method of describing or analysing the product?○ The details of the product?○ The patterns of use of the product?○ The environment in which the product is run?To what extent would this style benefit fromgroup interaction?
    16. 16. Heuristic Risk Based Testing The purpose of finding errorsMake a prioritised list of risks.Perform testing that exploreseach riskAs risks evaporate and newones emerge, adjust your testeffort to stay focused on thecurrent crop.
    17. 17. Agenda Definition of E.T. and it is differencebetween “Scripted Testing) How to do a good E.T. E.T testing management
    18. 18. E.T. is good for Provide rapid feedback on a new product orfeature Learn the product quickly You already tested using scripts, and seekto diversify the testing Want to find the single most important bugduring the shortest time Want to investigate the status of a particularrisk Improving on scripted tests
    19. 19. E.T. in Session 1 pair (2 engineers) of each team Call it the “Extreme testing”, or “BugSprinting” to energize our tester Rank the outcome if you want Make sure the top performer gets rewardedShort: 60 minutes (+-15)Normal: 90 minutes (+-15)Long: 120 minutes (+-15)
    20. 20. E.T. in Session – Tools you canuse Session Tester
    21. 21. Reviewable Result: The session sheet Charter#AREAS Start Time Tester Name(s) Breakdown#DURATION#TEST DESIGN AND EXECUTION#BUG INVESTIGATION AND REPORTING#SESSION SETUP#CHARTER/OPPORTUNITY Data Files
    22. 22. Controlled E.T. Make testing fun Improve test cases and test plans
    23. 23. Controlled E.T.代码库 开发 测试反馈意见手工第一轮手工第二轮手工第三轮手工第四轮开发新特性允许新特性代码 Checkin发布日期只允许 Bug 修复 Check in只允许重大 Bug 修复 Check in, 中等以下 Bug 不修复,会放在 Releasenote 中
    24. 24. Controlled E.T.ScriptedTestingScriptedTestingScriptedTestingE.T.E.T.E.T.Test Plan Update Test Plan Update Test Plan UpdateRound 1 Round 2 Round 3
    25. 25. Controlled E.T. Test team’s resource consumption perweek:25% of the group’s time developing new tests50% executing old tests (including bugregression)25% on exploratory testing
    26. 26. Lessons learned...(1) Pair testing Speeded up reporting errors Excellent to promote creativity Encouraged discussion Demanded a “strong voice” to keep everyone focused! Enables effective testing of short-lived, intense projects... but has also proved very useful on many test projects,also larger (i.e. at Microsoft, ref. J.Bach)
    27. 27. Lessons Learned...(2) Exploratory testing Requires skilled testers Requires some pre-set restraints, and thorough planning Requires a system for briefing and reporting/ documenting (ref.J.Bach - “Session based test management”) Requires guidance from test manager ..but when the requirements are fulfilled, it proved to be verysuccessful. Easy to motivate MANY deviations found …and it gave testers a greater feeling of participation