Slide 1 Slide 1 Trends in the Marketplace


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Assurance with Intelligence
  • Slide 1 Slide 1 Trends in the Marketplace

    1. 1. Trends in the Marketplace Testers will have to change – but how? Paul Gerrard Gerrard Consulting 1 Old Forge Close Maidenhead Berkshire SL6 2RD UK e: w: t: 01628 639173 Slide Assurance with Intelligence
    2. 2. <ul><li>Paul is the founder and Principal of Gerrard Consulting, a services company focused on increasing the success rate of IT-based projects for clients. He has conducted assignments in all aspects of Software Testing and Quality Assurance. Previously, he has worked as a developer, designer, project manager and consultant for small and large developments using all major technologies and is the webmaster of and several other websites. </li></ul><ul><li>Paul has degrees from the Universities of Oxford and London, is Web Secretary for the BCS SIG in Software Testing (SIGIST), Founding Chair of the ISEB Tester Qualification Board and the host/organiser of the UK Test Management Forum conferences. He is a regular speaker at seminars and conferences in the UK, continental Europe and the USA and was recently awarded the “Best Presentation of the Year” prize by the BCS SIGIST. </li></ul><ul><li>Paul has written many papers and articles, most of which are on the Gerrard website. With Neil Thompson, Paul wrote “Risk-Based E-Business Testing” – the standard text for risk-based testing. </li></ul>Paul Gerrard Assurance with Intelligence Slide
    3. 3. <ul><li>It’s the Benefits, Stupid! </li></ul><ul><li>Automation frameworks </li></ul><ul><li>Test Process Improvement is a Waste of Time </li></ul><ul><li>Software Success Improvement </li></ul><ul><li>What Makes a Good Tester? </li></ul><ul><li>Recommendations </li></ul>Agenda Slide Assurance with Intelligence
    4. 4. It’s the Benefits, Stupid Slide Assurance with Intelligence
    5. 5. <ul><li>Won’t bore you with yet another survey of IT projects that fail - take it for granted, most do </li></ul><ul><li>But why? </li></ul><ul><li>We can trace most failures to: </li></ul><ul><ul><li>Bad decisions </li></ul></ul><ul><ul><li>Decisions that were made too late </li></ul></ul><ul><ul><li>Decisions that were not made at all </li></ul></ul><ul><li>We suggest this happens because stakeholders and project managers lack the right information at the right time </li></ul><ul><li>We’ll call this information Project Intelligence </li></ul>Why projects fail The frame of reference for making those decisions is often wrong too! Slide Assurance with Intelligence
    6. 6. Four-eyed plans Increasingly Inaccurate IT- focused Initial Plans The plan is a model of the project. The real project consists of people, organisation, goals and risks. Slide Assurance with Intelligence
    7. 7. <ul><li>The people like to see the broad range of issues covered by the politicians </li></ul><ul><li>Makes politicians feel important </li></ul><ul><li>But it’s all froth </li></ul>Team Poster It’s the economy, stupid! benefits Ultimately, our customers are only interested in the benefits of IT Slide Assurance with Intelligence
    8. 8. <ul><li>Number of people in IT will drop by 15% by 2010 </li></ul><ul><li>60% percent of technology professionals will move to more business-focused roles, concentrating on the use of IT and processes rather than IT delivery </li></ul><ul><li>In 2010, the typical IT department in a large company will be at least one third smaller than it was in 2000 </li></ul><ul><li>Departments within business will take on the traditional roles of IT. </li></ul>Gartner predictions Slide Assurance with Intelligence
    9. 9. <ul><li>Increasingly, businesses will focus on benefits and will take control of projects or IT completely </li></ul><ul><li>Projects involving IT will no longer be dominated or managed by IT </li></ul><ul><li>The disciplines of Benefits Realisation, Goal-Directed Project Management and Project Intelligence will become mainstream </li></ul><ul><li>Most testers will work for (or come from?) business </li></ul><ul><li>(Some) Test Managers become PI Managers. </li></ul>Here’s my interpretation… Slide Assurance with Intelligence
    10. 10. Automation Frameworks I am very grateful to Susan Windsor ( for the use of some of her material Slide Assurance with Intelligence
    11. 11. <ul><li>Automation frameworks and more complex business requirements </li></ul><ul><li>Agile development methods mean developers undertake more unit and component testing </li></ul><ul><li>Growth of outsourced testing to different geographies => greater competition for the roles </li></ul>Testing is in demand, and solutions reduce the number of functional testers required Is the role of the functional tester (who is neither technical nor business specialist) dead? Slide Assurance with Intelligence
    12. 12. <ul><li>Focus on technology rather than business needs </li></ul><ul><li>80% of functional testing still manual </li></ul><ul><li>60% to 70% of automation tools used for non-functional testing </li></ul><ul><li>Typically, traditional functional automation stops at 100 scripts, regardless of test coverage requirement </li></ul><ul><li>Critical factors </li></ul><ul><ul><li>Cost of implementation and maintenance prohibitive </li></ul></ul><ul><ul><li>Insufficient and expensive skills required </li></ul></ul><ul><ul><li>Inability to asset share over different technologies </li></ul></ul>Functional test automation is broken! Source: Paul Herzlich, OVUM UK Slide Assurance with Intelligence
    13. 13. <ul><li>Home grown frameworks built within organisations to meet business demands </li></ul><ul><li>Niche suppliers provide frameworks - try Google: </li></ul><ul><ul><li>34,400 exact matches for “test automation framework” </li></ul></ul><ul><ul><li>A few are now mature </li></ul></ul><ul><ul><li>Latest review by Paul Herzlich (OVUM analyst) </li></ul></ul><ul><li>Market Leaders such as Mercury developing Business Process Tester (BPT) </li></ul>Business analysts already using them, and use will grow This seems to be the tools industry direction now Slide Assurance with Intelligence
    14. 14. Where frameworks fit Assurance with Intelligence Manual Test Execution & Documentation UI Automated Test Execution Tools Test Results Non UI Component Test Execution Harness Requirements Test Management Governance Project Management Analysis & Design Test Case Definition – Automation Framework Test Case Documentation Test Requirements Sign Off
    15. 15. Test Process Improvement is a Waste of Time Slide Assurance with Intelligence
    16. 16. <ul><li>I want to improve my ( insert any activity here ) </li></ul><ul><li>_______ people improvement </li></ul><ul><li>_______ organisation improvement </li></ul><ul><li>_______ process improvement </li></ul>How to improve…  Changing people (like me) and organisation (like my company) is so hard – let’s not even think about it Slide Assurance with Intelligence
    17. 17. <ul><li>There are no “practice” Olympics to determine the best </li></ul><ul><li>There is no consensus about which practices are best, unless consensus means “people I respect also say they like it” </li></ul><ul><li>There are practices that are more likely to be considered good and useful than others, within a certain community and assuming a certain context </li></ul><ul><li>Good practice is not a matter of popularity. It’s a matter of skill and context. </li></ul>The delusion of ‘best practice’ Derived from “ No Best Practices”, James Bach, Slide Assurance with Intelligence
    18. 18. <ul><li>Google search </li></ul><ul><ul><li>“ CMM” – 12,100,000 </li></ul></ul><ul><ul><li>“ CMM Training” – 12,200 </li></ul></ul><ul><ul><li>“ CMM improves quality” – 4 </li></ul></ul><ul><li>A recent client… </li></ul><ul><ul><li>CMM level 3 and proud of it (chaotic, hero culture) </li></ul></ul><ul><ul><li>Hired us to assess their overall s/w process and make recommendations (quality, time to deliver is slipping) </li></ul></ul><ul><ul><li>40+ recommendations, only 7 adopted – they couldn’t change </li></ul></ul><ul><ul><li>How on earth did they get through the CMM 3 audit? </li></ul></ul>The delusion of process models (e.g. CMM) Slide Assurance with Intelligence
    19. 19. Slide Assurance with Intelligence
    20. 20. <ul><li>People like simple models: </li></ul><ul><ul><li>levels of maturity, stepping stones, checklists, roadmaps and outside support for credibility </li></ul></ul><ul><li>But life is much more complicated, unfortunately </li></ul><ul><li>“ Things should be made as simple as possible, </li></ul><ul><li>but no simpler” </li></ul><ul><li>Albert Einstein </li></ul>But process models make improvement simple don’t they? Slide Assurance with Intelligence
    21. 21. <ul><li>A big problem with process is it becomes all encompassing </li></ul><ul><li>Process folk sell process and cast all things in terms of it </li></ul><ul><ul><li>They ignore that people who are smart </li></ul></ul><ul><ul><li>Smart people succeed regardless of process, not because of it </li></ul></ul><ul><li>It could be argued, that less smart people need process </li></ul><ul><ul><li>(By less smart, we're talking about people who need so much structure and enforced discipline they can only operate in the military, or in prison probably) </li></ul></ul><ul><li>Is our industry really staffed by such people? </li></ul><ul><li>Do we really want production-line workers? </li></ul><ul><li>Do YOU really want to be a production-line worker? </li></ul>People need process? Slide Assurance with Intelligence
    22. 22. <ul><li>“ I believe that a scientist looking at nonscientific problems is just as dumb as the next guy” </li></ul><ul><li>“ It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong” </li></ul><ul><li>Richard P. Feynman </li></ul>Physics quotes… Slide Assurance with Intelligence
    23. 23. <ul><li>“ I believe that a process consultant looking at non-process problems is just as dumb as the next guy” </li></ul><ul><li>“ It doesn't matter how beautiful your process model is, it doesn't matter how smart you are. If it doesn't agree with reality, it's wrong” </li></ul><ul><li>“ Test process improvement is tinkering” </li></ul><ul><li>Paul Gerrard ;-) </li></ul>You can quote me if you want… Slide Assurance with Intelligence
    24. 24. Software Success Improvement Slide Assurance with Intelligence
    25. 25. From this… Current Maturity Future Maturity Current Capability Future Capability Acts of Faith Better Capability = better, faster, cheaper Perceived Results Chain Slide Assurance with Intelligence Process Change
    26. 26. To this… Current Constraints/ Problems Current Capability Future Capability Acts of Change Better Capability = better, faster, cheaper Actual Results Chain Slide Assurance with Intelligence Changed Aspirations
    27. 27. <ul><li>Constraints are fixed headcount, budget, timescales, quality of requirements, contracts etc. </li></ul><ul><li>Problems are “testing takes too long; too expensive; can’t hire testers; bugs get through” etc. etc.… </li></ul><ul><li>Aspirations: </li></ul><ul><ul><li>Personal: personal development, fulfilment, motivation </li></ul></ul><ul><ul><li>Organisational: hero culture to team culture, outsourced, higher consistency, predictability </li></ul></ul><ul><li>Acts of change are… </li></ul>Constraints, problems and aspirations Slide Assurance with Intelligence
    28. 28. <ul><li>Changes in behaviour to address specific problems (time, cost, quality etc.) </li></ul><ul><li>Targeted personal and team development </li></ul><ul><li>Infrastructure change (process, techniques, tools, environments) to support the changes </li></ul><ul><li>Managed Transition… </li></ul>Acts of change – focused on constraints, problems and aspirations Slide Assurance with Intelligence
    29. 29. <ul><li>Current behaviour assessed in the context of current problems, goals and constraints </li></ul><ul><li>Aspirations drive the definition of goals </li></ul><ul><li>People in the job define and consent to the required changes in behaviour </li></ul><ul><li>People supported by </li></ul><ul><ul><li>Personal/team development plans </li></ul></ul><ul><ul><li>Infrastructure investment (process, technology, tools, environment) specific to the change </li></ul></ul><ul><li>Transitions are managed, not assumed. </li></ul>Principles of change Slide Assurance with Intelligence
    30. 30. <ul><li>Establish a sense of urgency </li></ul><ul><li>Create a guiding coalition </li></ul><ul><li>Develop a vision and strategy </li></ul><ul><li>Communicate the change vision </li></ul><ul><li>Empower broad-based action </li></ul><ul><li>Generate short term wins </li></ul><ul><li>Consolidate gains, produce more change </li></ul><ul><li>Anchor new approaches in the culture. </li></ul>Eight stage change process (Kotter) Derived from “Leading Change ”, John Kotter, Slide Assurance with Intelligence
    31. 31. <ul><li>Mission </li></ul><ul><li>Coalition </li></ul><ul><li>Vision </li></ul><ul><li>Communication </li></ul><ul><li>Action </li></ul><ul><li>Wins </li></ul><ul><li>Consolidation </li></ul><ul><li>Anchoring </li></ul>Eight stage change process (after Kotter) Changes identified here This is where your ‘process model’ comes into play Slide Assurance with Intelligence
    32. 32. What Makes a Good Tester? Slide Assurance with Intelligence
    33. 33. <ul><li>Pedantic </li></ul><ul><li>Sceptic </li></ul><ul><li>Nitpicker </li></ul>A perfect tester? Slide Assurance with Intelligence
    34. 34. Attributes of a good tester? intelligent persistent systematic pedantic thorough sceptical logical curious articulate conscientious imaginative accurate observant thick-skinned assertive redoubtable stubborn Slide Assurance with Intelligence
    35. 35. Attributes of a good tester? intelligent persistent systematic pedantic thorough sceptical logical curious articulate conscientious imaginative accurate observant thick-skinned assertive redoubtable stubborn Personal intelligence and skill Personality to cope with your environment Slide Assurance with Intelligence
    36. 36. <ul><li>Intelligence - are they smart enough? </li></ul><ul><li>Thinking skills </li></ul><ul><ul><li>Approach to problem solving; ability to reason </li></ul></ul><ul><li>Interpersonal skills </li></ul><ul><ul><li>Communication, assertiveness, conflict handling, survival, team skills etc. etc. </li></ul></ul><ul><li>Testing skills </li></ul><ul><ul><li>Theory (ISEB/ISTQB Etc.) </li></ul></ul><ul><ul><li>Practical (hands on testing) </li></ul></ul><ul><ul><li>Technical (technology, business domain etc.) </li></ul></ul><ul><li>Plus… Personality </li></ul><ul><ul><li>Many well-known attributes are usually required. </li></ul></ul>Tester selection criteria Slide Assurance with Intelligence
    37. 37. Tester skills (and training budget?) <ul><li>Testing theory (ISEB/ISTQB etc.) </li></ul><ul><ul><li>see </li></ul></ul><ul><ul><li> </li></ul></ul><ul><li>Thinking skills </li></ul><ul><ul><li>verbal reasoning, numerical/abstract reasoning, fault diagnosis, accuracy </li></ul></ul><ul><li>Testing Practice (Testing Case Studies) </li></ul><ul><ul><li>hands-on practical test activity </li></ul></ul><ul><li>Interpersonal skills </li></ul><ul><ul><li>communication, assertiveness, conflict handling, survival, team skills </li></ul></ul>20% 40% 20% 20% Slide Assurance with Intelligence
    38. 38. Summary and Discussion Slide Assurance with Intelligence
    39. 39. <ul><li>Take a closer look at benefits realization, results-based management, performance management, project intelligence </li></ul><ul><li>Become more business-oriented </li></ul><ul><ul><li>Be ready to talk their language – benefits and risk </li></ul></ul><ul><ul><li>Align your test activities with the need to provide information on benefits and risk to them </li></ul></ul><ul><ul><li>Be prepared to be managed by your customer! </li></ul></ul>It’s the benefits, Stupid! Slide Assurance with Intelligence
    40. 40. <ul><li>Take a close look at frameworks </li></ul><ul><li>What is your organisation doing about it? </li></ul><ul><li>Understand how Frameworks support testing so you have a view, because you will be asked </li></ul><ul><li>If you are functional tester – evolve or… </li></ul><ul><li>Look to enhance your skills… </li></ul><ul><ul><li>Become an automation/framework specialist </li></ul></ul><ul><ul><li>Move towards management </li></ul></ul><ul><ul><li>Take on non-functional skills </li></ul></ul><ul><ul><li>Get closer to your business/customers </li></ul></ul>Automation frameworks Slide Assurance with Intelligence
    41. 41. <ul><li>Don’t take on Test Process Improvement projects </li></ul><ul><ul><li>Too many problems in test are caused elsewhere </li></ul></ul><ul><ul><li>Testers shouldn’t compensate for other people’s failures </li></ul></ul><ul><ul><li>Whole-process is in scope for change - or nothing! </li></ul></ul><ul><li>Ask why things are the way they are </li></ul><ul><li>Focus on people, culture and organisation </li></ul><ul><li>“ The art of the possible” – get your suggestions from your practitioners, not a book </li></ul><ul><li>Follow the 8 step change process (or another transition management method – there are many) </li></ul><ul><li>Be realistic about benefits, and be prepared to measure them </li></ul>Software Success Improvement Slide Assurance with Intelligence
    42. 42. <ul><li>Functional testing skills are a commodity – so move on </li></ul><ul><li>Have a vision of where you want to be… manager, specialist, consultant, business expert… </li></ul><ul><li>Following technology changes can be lucrative, but is a never-ending journey </li></ul><ul><li>Broaden your horizon – thinking, interpersonal, business, management, project management skills </li></ul><ul><li>Create a development plan – training yes, but look out for practical hands-on, not theory </li></ul><ul><li>Most learning takes place on the job so choose jobs wisely </li></ul><ul><li>Find a good coach. </li></ul>Developing, as a tester Slide Assurance with Intelligence
    43. 43. Good luck in your career! Trends in the Marketplace Testers will have to change – but how? Slide Assurance with Intelligence