Key Principles for a Generic Test Organization
Purpose•   This presentation outlines a number of key principles that could be    applicable for any test organization•   ...
Testing as a                                       Service               Right                                            ...
Key Principle: Testing as a Service•   A tester is not a quality police – clients make decisions based on a    service the...
Key Principle: Smart Automation•   Smart test automation is not automating a set of manual tests [2]•   Smart automation i...
Key Principle: Useful Metrics•   Not everything that counts can be measured. Not everything that can    be measured counts...
Key Principle: Risk-based Testing•   All testing should be based on a proper risk analysis [4]•   Clients decide what they...
Key Principle: Customer Focused Reporting•   Don’t ask the client what information he wants – ask what decisions    the cl...
Key Principle: Changing Requirements•   Don’t expect complete requirements up front•   When parameters change, the tester ...
Key Principle: Context Dependent Test                Methods & Approaches•   The most effective and efficient testing is c...
Key Principle: Empowering Tools•   A tester should spend a majority of the time testing – streamline    everything else wi...
Key Principle: Right Competence•   There are many different tester roles that each require a diverse set    of skills – on...
Summary•   These key principles should be applicable for most test organizations,    even if there will always be some exc...
Reference[1] Ten Things about Testing that should die: Egocentricityhttp://scott-barber.blogspot.com/2011/11/10-things-abo...
Upcoming SlideShare
Loading in …5
×

Key principles for a generic external test organization

871 views
770 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
871
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
12
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Key principles for a generic external test organization

  1. 1. Key Principles for a Generic Test Organization
  2. 2. Purpose• This presentation outlines a number of key principles that could be applicable for any test organization• By applying these key principles to a test organization it becomes more efficient and effective• Not everything is applicable in every context, but the key principles should be generic in nature and usable under most circumstances
  3. 3. Testing as a Service Right Smart Competence AutomationEmpowering Useful Metrics Tools Key Principles Context Risk-based Dependent Testing Test Methods Customer Changing Focused Requirements Reporting
  4. 4. Key Principle: Testing as a Service• A tester is not a quality police – clients make decisions based on a service the tester provides [1]• A test organization should provide an understandable and simple service catalogue of how they can help their clients get the information they need to make the correct decisions• A test organization should be able to support their clients in selecting what services are most suitable for their specific needs• A test organization should always be able to show and explain their value
  5. 5. Key Principle: Smart Automation• Smart test automation is not automating a set of manual tests [2]• Smart automation is designing a good automated test scope from the beginning• Manual tests are not designed to be automated – create automated tests that are suitable for automation, don’t force automation on something that was designed to be manual• Always have a return-on-investment analysis, and consider maintenance, portability and re-usability costs• Drive testability in the code so that automation becomes a natural part of the development process
  6. 6. Key Principle: Useful Metrics• Not everything that counts can be measured. Not everything that can be measured counts. [3]• Metrics should be used to pinpoint where an analysis or audit must be conducted, not as hard evidence• Always consider what kind of behavior a certain metric will promote before communicating it• Never be afraid to measure, but understand that metrics never give a complete picture
  7. 7. Key Principle: Risk-based Testing• All testing should be based on a proper risk analysis [4]• Clients decide what they want, but a tester must be able to explain why a certain scope is more or less effective for the clients needs and acceptable risk levels• Use as much empirical data as possible for these risk analysis, don’t just go on a gut feeling – quantify that feeling when possible• Inform the client of the costs of different levels of uncertainty and risk; if the client wants everything tested and no risk it will cost an indefinite amount, if the clients accepts any risk, then nothing is tested, and the cost is zero
  8. 8. Key Principle: Customer Focused Reporting• Don’t ask the client what information he wants – ask what decisions the clients intend to take based on the information• Tailor the test and defects reports based on the clients needs; qualitative and quantitative information can be equally powerful – but always back up statements with facts• A tester must communicate what confidence the client can have in the results – just because no issues are found does not mean that the software is bug free• But always remember; “We don’t manage the project managers. We don’t decide what information they have to give to the people they report to.” [5]
  9. 9. Key Principle: Changing Requirements• Don’t expect complete requirements up front• When parameters change, the tester must adapt to meet new stakeholder needs; be adaptable to changing requirements during a project• If requirements are needed and none are available, write down expected behaviour and request stakeholder acknowledgement – communication with stakeholders is key when requirements are unclear• Try to minimize unnecessary test artefacts – combining requirements and test cases into one entity can be one way forward
  10. 10. Key Principle: Context Dependent Test Methods & Approaches• The most effective and efficient testing is context dependant [6]• It is the tester’s job to find the most effective and efficient method for each context – Scripted testing - Exploratory testing – Black box - Gray box - White box – Manual - Automated – Tool supported or not – Boundary analysis -Equivalence partitioning -Fuzz testing - State transition testing – Etc.
  11. 11. Key Principle: Empowering Tools• A tester should spend a majority of the time testing – streamline everything else with the help of tools [7]• Risk-analysis tools help the tester perform more efficient and effective risk analysis• Integrated reporting tools help the tester file defects without changing systems [7]• Tools to support time efficient test plans [8]• Record and playback framework for repeating tests [9]
  12. 12. Key Principle: Right Competence• There are many different tester roles that each require a diverse set of skills – one size does not fit all• Understand what roles are needed in a specific organization and secure that the testers have the skills needed to fill those roles• Make sure that every tester has an idea of what their career might look like in five years, based on their specific skillsets [10]• Understand that it is not a numbers game – do not hire more testers to solve a problem – secure that you have the right testers [7]• More does not equal better – it can sometimes lead to more entropy and chaos
  13. 13. Summary• These key principles should be applicable for most test organizations, even if there will always be some exceptions• While they do not cover everything, they should cover the most critical areas that most test organizations need to think about• There is no best practice – but this is a good start to help a test organization finding a best practice for their context
  14. 14. Reference[1] Ten Things about Testing that should die: Egocentricityhttp://scott-barber.blogspot.com/2011/11/10-things-about-testing-that-should-die.html[2] Exploring Test Automationhttp://angryweasel.com/blog/?p=412[3] Albert Einstein qoutehttp://www.famous-quotes.net/Quote.aspx?Not_everything_that_counts_can_be_measured[4] Risk-based Testinghttp://en.wikipedia.org/wiki/Risk-based_testing[5] Contexts Differhttp://context-driven-testing.com/?p=38[6] Context Driven Testinghttp://context-driven-testing.com/[7] Turning Quality on its Headhttp://www.youtube.com/watch?v=cqwXUTjcabs[8] The 10-minute Test Planhttp://googletesting.blogspot.com/2011/09/10-minute-test-plan.html[9] RPFhttp://googletesting.blogspot.com/2011/11/rpf-googles-record-playback-framework.html[10] Career Paths using Tester Personashttp://www.stickyminds.com/s.asp?F=S16548_COL_2

×