Your SlideShare is downloading. ×
Agile testing (n)
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Agile testing (n)


Published on

Published in: Technology, Education

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Agile Testing
  • 2. Agile Manifesto 1. people and communication over processes and tools 2. working software over documentation 3. customer collaboration over contract negotiation 4. responding to change over following a plan
  • 3. Roles and activities on agile team "Testers have a foot in each world, understanding the customer viewpoint as well as the complexities of the technical implementation." Customer Team ● business experts ● product owners ● domain experts ● product managers ● business analysts ● subject matter experts ● testers Developer Team ● programmers ● system administrators ● architects ● security specialists ● database administrators ● technical writers ● testers
  • 4. "Testing" ● ● ● ● ● ● ● ● ● ● ● Unit and component testing Functional testing System testing Load testing Performance testing Security testing Stress testing Usability testing Exploratory testing End-to-end testing User acceptance testing
  • 5. Ten principles for Agile Testers 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Provide continuous feedback Deliver value to the customer Enable face-to-face communication Have courage Keep it simple Practice continuous improvement Respond to change Self-organize Focus on people Enjoy
  • 6. What's the difference...? Traditional testing: ● long iterations, "mini-waterfall"-trap ● testers are not involved until release planning ● testers have requirement documentation, they write long test plans, wait for ● the developers and then execute tests -> "fast and furious testing" ● if anything goes wrong it causes rushed testing phase or/and release is ● usually postponed ● traditionally testers are considered as gatekeepers of quality -> but they ● have no control over code quality or developer testing ● separated QA team
  • 7. What's the difference...? Agile testing: ● short iterations, frequent, small releases ● testers are involved as soon as project planning begins ● testers have access to all knowledge and help from the team they need, ● they can focus on big picture and small tasks simultaneously ● if anything goes wrong it can be discovered and fixed in time ● testable design and code ● team understands that quality is more important than quantity or speed ● independent, but not separated QA team (everyone is a tester)
  • 8. So, What's the difference...? "One of the biggest differences in agile development versus traditional development is the agile 'whole-team' approach. With agile, it's not only the testers or a quality assurance team who feel responsible for quality." ● no more quality police mentality - quality is not only the testers' responsibility ● whole-team approach; in agile teams everyone gets testinfected -> code is designed for testability ● tester team is integrated, there's no "QA-time" (no time to cut when deadline is close, no rushed QA-phase) ● every team member has equal value to the team ● self-organizing team, continuous improvement
  • 9. Agile testing Quadrants Quadrant 1: this represents test-driven development, which is a core agile development practice Quadrant 2: business-facing tests, these define external quality and the features that the customer wants Quadrant 3: business-facing tests to critique the product (manual) Quadrant 4: technology-facing tests
  • 10. Quadrant 1 Test-driven development TDD ● Unit tests ● Component tests ● internal quality isn't negotiated with the customer, it's defined by the programmers ● programmer tests make lots of testing activities easier to accomplish ● TDD means programmers are always consciously making testable code and it provides immediate feedback ● each unit test is independent and tests one dimension at a time -> when a unit test fails, it's easy to identify where and what to fix ● TDD is a design activity, not only testing
  • 11. Quadrant 2 - External quality ● ● ● ● ● Functional tests Examples Story tests Prototypes Simulations ● business-facing tests address business requirements based on examples ● quadrant 2 tests define and verify external quality ● these are written for each story before coding starts, they help the team ● understand what code to write ● generally written in executable format, automated, so that team members ● can run the tests as often as they like ● these become later automated regression tests as well
  • 12. Quadrant 3 tests that critique the product ● ● ● ● ● ● ● ● ● ● ● ● ● Exploratory testing Scenarios Usability testing UAT (User Acceptance Testing) Alpha/Beta testing critiquing or evaluating the product difficult to automate because these are based on human intellect, experience and instinct this manual testing is not enough to produce high-quality software (without automation) trying to recreate actual experiences of the end users "soap opera testing" - "what's the worst thing that can happen, and how did it happen?"
  • 13. Quadrant 4 - Performance, Security and “-ility” testing ● ● ● ● ● ● ● ● ● Security Maintainability Interoperability Compatibility Reliability Installability technology-facing tests that critique the product nonfunctional requirements, configuratiion issues many of these tasks require specialized knowledge
  • 14. Why are all of these important? ● help to get emerge technical debt (shortcuts, quick fixes and skipped automated tests making the code harder to maintain) ● responsibility is shared between the quadrants ● quality is not only the testers responsibility ● "When developers don't have an automated suite of tests acting as a safety net, they may start viewing the testers themselves as a safety net."
  • 15. Why manual testing isn't enough ● ● ● ● ● ● ● it gets boring very quickly, it's repetitive, so it's easy to overlook simple bugs manually testing a number of different scenarios can take a lot of time especially if tester is keying inputs into a UI -> only a limited number of scenarios may be tested, and important defects can be missed automated tests can help with consistency across the application automated regression tests are the safety net, not the QA person; without them manual regression testing will grow in scope and eventually may simply be ignored automated test frees the QA team for more important work, such as exploratory testing "when post-development testing time is occupied with finding and fixing bugs that could have been detected by programmer tests, there's no time to find the serious issues"
  • 16. How to be an agile tester while release planning ● avoid big design up front, but understand it correctly ● "create" high-level test plan ● try to identify risk, impact ● think about 'ripple effects' as well ● help correctly sizing stories with correct questions
  • 17. "Intuition is something that a machine cannot learn." ● ● ● ● ● the tester needs to keep the big picture in mind to create test cases so they can ask uncomfortable questions ○ how much effort does it require to create a test environment? ○ what's the worst scenario that can happen? ○ what if the end user doesn't understand the application? ○ what's the difference between test and production environment? how will it affect the performance? ○ what kind of test data should we use and where do we get it? ○ etc. they always have more different viewpoints (developer-customer team) they can identify 'thin slice' and 'critical path' to help prioritize stories they give quick feedback to developers, helping them to create quality product UX/UI testing is the most important after quality code -> "if a user has a choice of applications or websites, and has a bad experience, they likely won't user your application again"
  • 18. How to be an agile tester while story sizing: ● everyone's goal is to deliver real value in each iteration > help identify one path through the functionality to code and test first, then help adding more features after the first critical path works ● identify 'small testable chunks' -> testable doesn't necessarily means GUI ● always think about the big picture, know how each story affects the whole system (if there's a bug in the inventory code, what's the worse thing that can happen?) -> help identify impact
  • 19. How to be an agile tester while test planning: ● the biggest benefit of test planning is planning itself ● wait for iteration planning to create detailed tests ● consider automation and what is needed for test environment ● test planning is a risk mitigation strategy ● consider test infrastructure ● collect test data, real examples and real use-cases ● choose the right testing method, considering: ○ what's in scope ○ what are the impacts, is there any unusual risk ○ do you need test reports (for the customer team? for your team?)
  • 20. Test plan alternatives Lightweight plans ● plan should not cover every eventuality or every story ● not meant to address traceability ● it's a tool to help thinking about testing risks Test Matrix ● it's a list of functionality down the side and test conditions across the top ● high-level test matrix -> can be used to show to customer team what has been tested already and what's left ● more detailed matrix -> can be used to show to developer team what's planned for testing and track the progress
  • 21. Test plan alternatives Test spreadsheet ● it's more like a functionality list and impact analysis in a spreadsheet format, similar to test matrix but on more tabs Whiteboard ● sometimes it's enough to list the risks and assumptions on a whiteboard or index cards or a wiki page Automated test list ● provided by tools (at EU Edge: TestLink) ● it's more like a detailed test plan ● it's only available after test cases are written ● no added value
  • 22. How to be an agile tester while iteration planning: ● ● ● ● ● ● ● ● ● ● think about how you can test the stories help the programmers to design the application to be effectively testable "when testability is an issue, make it the team's problem to solve" work closely with the customers think about 'big picture' and detailed tests as well high-level tests should cover the main functionality behind the story think both about the desired and undesired behaviour 'What's important as you begin the iteration is that you quickly learn the basic requirement for each story and express them in context in a way that works for the whole team." review the test cases with the customers and developers as well well-written tests form the core of the application's documentation
  • 23. How to be an agile tester while development: ● start simple -> create a happy path to show the core functionality ● add complexity -> as soon as happy path works start adding more test cases ● remember the purpose of the tests: they should provide examples that tell the programmers what code to write ● use risk analysis to help prioritize testing ● identify variations ● use the 'power of three' (disagreement -> programmer + QA person + Product Owner) ● focus on one story, try to get it 'done'
  • 24. How to be an agile tester while development: ● as soon as testable chunks are ready take time to explore functionality ● if you find 'nice-to-have' stories consult with the customer, if there's time to add it in the iteration and it has business value, go ahead (these are cheaper now), but don't jeopardize other stories by adding 'blind' that doesn't have big ROI (ROI = Return on investment) ● technology-facing tests to critique the product ("-ility testing") are often best done during coding ● if the team doesn't have automated tests, everyone should plan time to address manual regression tests and manually testing new features
  • 25. How to be an agile tester while dealing with bugs: ● bug or feature? -> in the end, does it really matter if it is a bug or a feature if it needs to be fixed? ● bugs -> technical debt: the longer a bug stays in the system and goes undetected, the greater is the impact ● sometimes a bug is really a missed requirement and needs to be handled as a story; estimated and prioritized for a future iteration ● set rules like "the number of bugs should never get higher than ten at any one time"
  • 26. How to be an agile tester while delivery: ● ● ● ● ● ● ● ● the goal is to deliver value to the business in a timely manner plan as much time for the end game as you need (it varies with the maturity of the team and the size of the application) use this end-game time to do a final exploratory testing step back and look at the whole system; do some end-to-end scenarios find a way to guarantee that all changes made in the test environments will be done in the staging and production environments during release 'customers have a habit of using the application in weird and wonderful ways, and the data is not always as clean as we would like' - everyting needs to be tested again with real data in a real environment try to avoid high risk -> release in smaller stages or use a system property to turn on new features and use the old as long as the new one is safe enough communicate, have reminders
  • 27. Summary Success factor 1: Whole-team approach ● ● ● ● you have a large variety of skill sets and experience levels when testing is a team priority, the team designs testable code remember that quality, not speed, is the goal of agile development use the "power of three" Success factor 2: Adopt an agile testing mindset ● ● ● ● ● ● "Ten principles for agile testers", no more "Quality police mentality" be courageous focus on delivering value be flexible in responding to change drive to continually find better ways to work experiment with new practices, tools, and techniques
  • 28. Summary Success factor 3: Automate regression testing ● ● ● if you spend all your time doing manual regression testing, you'll never have time for the important exploratory testing remember to start simply Success factor 4: Provide and obtain feedback ● ● ● feedback is a core agile value the short iterations are designed to provide constant feedback one of the most valuable skills to learn is how to ask for feedback ○ ask the programmers if they get enough information ○ ask the customers if they feel their quality criteria are being met
  • 29. Summary Success factor 5: Build a foundation of core practices ● ● ● ● ● "you can't test quality into the product" every agile team needs source code management and continuous integration -> you can't test effectively if you don't know exactly what you're testing you can't test effectively without a test environment that you can control manage technical debt; don't let time pressure get you into hacking, or "quickfixing" - good test coverage, automated regression tests can help use small-scaled stories -> develop and test step-by-step to keep quality "Agile practices were designed to complement each other. Take time to understand the purpose of each one, consider what is needed to take full advantage of each practice, and make thoughtful decisions about what works for your team.”
  • 30. Summary Success factor 6: Collaborate with customers ● ● ● use the "Power of three" encourage direct communication between customers and developers help customers clarify requirements, desired behaviour with concrete examples Success factor 7: Look at the big picture ● ● ● ● testers tend to look at the big picture usually from a customer point of view developers have to focus on the story they're working at use agile testing quadrants as a guide to help you plan testing that will cover all angles make your test environments as similar as possible to production (test data as well)