A paradigm shift for testing - how to increase productivity 10x!
•
2 likes•3,506 views
European IT industry need to deal with a huge salary gap with developing countries. How can we increase our productivity and quality to compensate for the salary differences? This is a systems-thinking / Lean based approach to that problem
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (19)
Similar to A paradigm shift for testing - how to increase productivity 10x!
Similar to A paradigm shift for testing - how to increase productivity 10x! (20)
More from Vasco Duarte
More from Vasco Duarte (16)
Recently uploaded
Recently uploaded (20)
A paradigm shift for testing - how to increase productivity 10x!
- 1. A paradigm shift for testing Vasco Duarte Testaus forum, August 2010
- 2. Vasco Duarte @duarte_vasco http://bit.ly/tf10_blog
- 3. Vasco
- 7. Disclaimer I’m experienced, but that means: I understand your question NOT: I know the answer!
- 8. Act I – The problem
- 10. ~7 200 $ / Year Average Software Engineer salary in China Source: http://bit.ly/tf10_chinapay
- 11. ~40 100€ / Year Average salary for a specialist in Finland, 2009 data Source: http://bit.ly/tf10_fipay
- 12. 9.4 times higher! Including ~30% associated social costs
- 13. Cost Productivity € How ? €
- 14. Problems cannot be solved by the same level of thinking that created them Paraphrasing a quote from Einstein
- 15. We need a paradigm shift! How can we achieve it?
- 17. Act II – Why do we get poor results?
- 18. 1. Customer need 2. Product managers 3. Business analyst …
- 19. 4a. Designers / Architects … 5. Designers / Programmers 6. Testers 4b. Test designer … …
- 21. Analytical design - prototype Silo 2 Silo 3 Silo 1 Execution Knowledge Actual execution Business Knowledge
- 22. Silo design leads to ineffective processes. A story…
- 23. Trying to reduce costs with analytical processes Driving costs up! =
- 24. Act III – The solution: a paradigm shift!
- 26. 8. Drive out fear Management by fear is counter-productive because it prevents workers from acting the best interest of the organisation 1. Create constancy of purpose towards improvement 9. Break down barriers between departments 2. Management must adopt the new philosophy Replace short-term reaction with long-term planning Each department servers not the management but the needs from other departments that use its outputs. The implication is that management should actually adopt his philosophy, rather than merely expect the workforce to do so 10. Eliminate slogans 3. Cease dependence on inspection It is not the people who make mistakes (in 95% of the cases). It is the process. Slogans do not help improve processes and harass the people doing the actual work! The implication is that we must relentlessly remove the root causes for defects rather than inspecting them out of the final product 11. Eliminate management by objectives 4. Move towards a single supplier for any one item Production targets subvert the system. Workers start working for the targets instead of working for the purpose of the organization. Metrics: yes, targets: no! Multiple suppliers mean variation in the quality of work as well as lost knowledge in hand-over 12. Remove barriers to pride of workmanship 5. Improve constantly and forever Many of the other problems outlined above reduce worker satisfaction and therefore reduce focus on quality Constantly strive to improve how you work, focusing on the purpose rather than short term perspectives only. 13. Institute education and self-improvement 6. Institute training on the job A result of pride of craftsmanship is the desire to learn and improve. This, in turn leads to better quality. If people are not trained properly they will not all work in a consistent way. This leads to defects, mis-communication, etc. 14. The transformation is everyone’s job 7. Institute Leadership It is manager’s job to lead, but it is everyone’s job to contribute to the needed transformation of our business. Deming makes a distinction between Leadership and mere Supervision. “Banish targets, substitute leadership” Deming used to say
- 27. 9. Break down barriers between departments 3. Cease dependence on inspection The implication is that we must relentlessly remove the root causes for defects rather than inspecting them out of the final product Each department servers not the management but the needs from other departments that use its outputs.
- 28. 2. Product managers 1. Customer need Co-operation Colaboration … 2. Business analyst and…
- 29. 3. Testers 3. Test designer and… 4. Testers Co-operation Colaboration Co-operation Colaboration … … 3. Business analyst 3. Designers / Programmers 4. Designers / Programmers
- 30. Your product rockz! Happy(!) Customer
- 31. Act IV – The action plan
- 32. Cost Productivity € Paradigm Shift! €
- 33. Organize your process around your customer need, not your silo-based organization
- 34. You can today start by applying the following principles
- 35. 9. Break down barriers between departments 3. Cease dependence on inspection The implication is that we must relentlessly remove the root causes for defects rather than inspecting them out of the final product Each department servers not the management but the needs from other departments that use its outputs.
- 36. Later you can apply the other 12…
- 37. Recommended reading + interesting links Reading Deming, Out of the Crisis, http://bit.ly/tf10_demingbook Reinertsen, Flow in product development, http://bit.ly/tf10_flow Video on a different paradigm for process design: Systems thinking intro, http://bit.ly/tf10_video
- 38. Currently an Operational Development specialist at Nokia, Vasco Duarte is an experienced product and project manager, having worked in the software industry since 1997. Vasco has also been an Agile practitioner since 2004, he is one of the leaders and a catalyst in the adoption of Agile methods and an Agile culture at Nokia and previously at F-Secure. Vasco's contributions to the improvement of the software development profession can be read in his blog: http://softwaredevelopmenttoday.blogspot.com. You can follow Vasco on twitter: @duarte_vasco About the speaker:
Editor's Notes
- How many of you develop software (write code)?How many of you test software?How many of you write tests but do not execute them? – This talk is for you!
- My goal today is to make you think. If you get one single idea that you would like to test back at work, then I’ve done my job.It is impossible for me to give you any solution without understanding your environment, but I hope to make you think critically about how you organize your work.How you execute your work, and ultimately give you some idea (and motivation) to try something new!
- As of today there are companies moving their R&D to China, India and other countries. Why are they doing that? For business reasons, of course!They do this because it makes business sense.
- Here's a simple business calculation: the average software engineer salary in China is ~7.200 USD/year.That is about 5 497 Eur in today’s money!
- Compare that with the average for a similar paid job in Finland and you can see why
- In Finland, according to Tilastokeskus (2009 data) it is 40 087,5 Eur/year. (52 130 Eur if you add social costs at 30% rate)Who would not consider that differential? No one. Every responsible business person has to consider that differential! If you don't do it others will and others will make the jump. More and more start-ups are hiring abroad (Finnish business, chinese R&D).
- How do we revert the difference? How do we make our industry 10 times more productive in Finland to ensure that our IT industry will stay competitive?
- Before going further let me make this point….
- That is the subject of my talk today. My message to you is this: "we have to fundamentally change the paradigm of software development, and therefore testing in our industry. The current state of affairs is unsustainable!"
- We are not the first to face this problem.In 1937,TaiichiOhno who was at the time a young worker heard that the American Industry was 9 times more productive than their Japanese equivalent.He was in the same situation as we are today. And today we know that Toyota is a manufacturing power-house in the world. In 2005 they were more valuable in the stock market than Volkswagen, GM combined. In 2009 they became the largest car manufacturer in the world, surpassing GM. But how was TaiichiOhno and Toyota able to turn around their story? TaiichiOhno, just like we will do today, started in the same way to look at the problem. By analysing what was forcing the japanese worker to produce 10x less than their american counterparts. We start from a point of view of cost, but we really want to look at productivity because increasing productivity compensates for higher costs.
- So, how do we make our industry 10 times more productive? First let's establish what is the current situation.
- In most companies the software development process is traditionally set-up this way.1. Marketing discovers a need or the sales person comes in with a requirement. 2. The product managers discuss the need or requirement and come up with their own understanding of that need/requirement based on their knowledge of the product, the market and the technology.3. A business analyst will then specify the solution based on their understanding of the need as specified by product management, the product and the technology. They may or may not involve technical experts in this process.
- 4. Once the business spec is ready it is then given to two groups: the designers/architects that will spec the solution in technological terms and the test designers who will spec the tests for the solution. They may or may not share knowledge between them5. Then the designers or programmers will implement the solution based on their understanding of the technical specification and very rarely with the understanding of the need (this is a problem!)6. Finally the testers will run the test spec against the implementation and give feedback to the programmers, the designers/architects, the business analysts and the product managers as to the spec adherence of the implementation This all leads to…
- And all this will, no doubt lead to a happy (?) customer. Or will it? Normally there would be many step 5-6 iterations, but I'll spare you the pain and horror of looking at that (you have enough of that at work anyway).This example here is simplified for the purpose of making a point. The example above is by no means an extreme, I've seen worse chains through a development process in my life.
- And the point is this: We created analytical management, and with that analytical process designs came into existence. These analytical approaches to process design are extremely ineffective because they separate the knowledge of what needs to be done from the people who do the work. Those that have the knowledge (marketing, sales, etc.) are not the ones creating the solution, or (God forbid) testing it! This type of design is typically called the "silo" design. In real life, in our companies the silo design leads to a lack of information in different organizational units!How many of you ever had one Project review explain that we have a “communication problem”? Yeah, all of us I guess. I know I have seen my fair share of “communication problems” in the past.
- Silo-driven design leads to ineffective, slow and expensive processes. YOu don't believe me, then let me tell you this story (true story BTW!).A certain company (which shall remain unnamed) opened an office overseas for the reason of reducing costs (the reason was good, but the solution was disastrous as I shall explain). This particular overseas office was a testing office. The thinking goes like this:1- we have quality problems (correct)2- we need more testers to find more bugs (huge #fail, right there!)3- let's get them where it is cheap4- let's use our best testers to write boring specs that tell the cheap testers what to test, and some of the other good testers to manage the office overseas5- testing is then moved to the office overseas and many testers are hired (say a ratio of 10 to 1?)6- testing in overseas office is conducted in the following manner: each tester is given a quota of bugs to create (you must create x bugs so that we get paid bonuses). Then each tester proceeds to find the quickest bug they can find for two reasons: a) they need that to get bonuses, b) there's only 3 test machines capable of running the software under testing.7- Each tester queues for the test machines for 3-4 hours a day. As soon as they get the machine, they need to set it up (some 30 min or so), then they start running the tests one-by-one and then, when some test result does not match expectedresult they get up and file a bug immediately because of their quota and because there are some other testers waiting for access to the machine. Rinse and repeat!What does this lead to? I think the answer should be obvious: A lot of easy to find bugs and a lot of unspecified behaviour goes untested. The quality of the testing is bad (irony, I know).
- This story is not an example of how testing is bad overseas! This is a story about how testing is bad in our own back-yard! The same phenomenon is happening right now at your office, perhaps in a different scale, but it is. This story is an example of how our analytical, silo-based process design is failing us!In this particular example we have another critical illustration of how, when we try to reduce costs, we end up driving costs up! The same phenomenon can be seen on other industries as well (I'll link to some videos about that in the notes).
- So, what is our solution? What paradigm should we use to design our process?What I'm about to tell you is not new, but it is revolutionary and a complete shift in paradigm.Back in the 1940-1950's Japan was in trouble. The second world war had just ended, the japanese industry needed reconstruction. A person flew in from the USA at the request of the US and Japanese government to help reconstruct the Japanese industry.
- This person was Deming, a person who had tried to help the American industry change but had been ignored. Deming proceeded to change drastically the Japanese industry and make it one of the most renowned industries in the world when it comes to quality. He did this very simply by telling the Japanese that they needed a paradigm shift. A way to look at industry that was, and still is, different from industries all over the world.To this day, the Japanese admiration for Deming is illustrated by the highest quality-prize given in Japan today: The Deming Prize.
- Here are the 14 points that Deming delivered, known as the Demings 14 points:1."Create constancy of purpose towards improvement". Replace short-term reaction with long-term planning.2."Adopt the new philosophy". The implication is that management should actually adopt his philosophy, rather than merely expect the workforce to do so.3."Cease dependence on inspection". The idea here is not to stop inspection, but rather to design a process that builds quality into the products we develop. 4."Move towards a single supplier for any one item." Multiple suppliers have different ways of working, different standards and therefore increase the communication and coordination related problems.5."Improve constantly and forever". Improvement leads to better results, which in turn allow you to invest further in improvements. This is the ultimate positive spiral.6."Institute training on the job". If people are inadequately trained, they will not all work the same way, and this will introduce problems. Train your people on the job.7."Institute leadership". Deming makes a distinction between leadership and mere supervision. "Banish targets, substitute leadership!" Deming used to say. Leadership is about helping people perform, not threatening if they don’t. That’s an error of omission on the part of managers8."Drive out fear". Deming sees management by fear as counter- productive in the long term, because it prevents workers from acting in the organisation's best interests.9."Break down barriers between departments". Another idea central to Deming’s philosophy is the concept of the 'internal customer', that each department serves not the management, but the other departments that use its outputs. Focusing on collaboration is the way to improve the work.10."Eliminate slogans". Another central Deming idea is that it's not people who make most mistakes - it's the process they are working within. Harassing the workforce without improving the processes they use is counter-productive.11."Eliminate management by objectives". Deming saw production targets as encouraging the delivery of poor-quality goods.12."Remove barriers to pride of workmanship". Many of the other problems outlined reduce worker satisfaction.13."Institute education and self-improvement".14."The transformation is everyone's job".
- But, I'll focus on two particular points that are key to the model I propose:3. Cease dependency on inspection9. Break down barriers between departmentsSo, taking these 2 principles, how do we design our processes so that we cease our dependency on inspection and remove barriers between departments?We need to recognize what "inspection" means in our context. In software inspection is the act of verifying whether a certain implementation follows the initial design by means of executing a test and comparing the output with the specified output.Inspection is still needed, and it is largely understood, so I won't explain further how that must be executed, but rather focus on the alternative. The alternative is a key shift in paradigm, a revolutionary change to the status quo. My proposal is that testers should be the ones working with Marketing and sales -- not business analysts alone!The basic principle at work here is that we must build quality in not slap it on at the end. This "build quality in" principle was also cherished by Deming and others that learned from him. How would it work in practice?
- How would it work in practice?We still start from the same starting point. The customer:Involve the business analysts and the product managers with the customers or the customer facing people. They need to “feel” the pain of the customer! The purpose is to create a deep understanding of the customer environment and need. We break the silo by involving the next process step (business analyst) in the clarification of the customer need. Here, Product Managers and Sales’s responsibility is that Business analysts understand the customer!
- And then we take that principle and apply it again.3. The tester together with the business analyst will design a test spec and a business analysis document for the software to be produced. The key here is “together”3. the business analysis document will also be given to testers and the test spec will also be given to designers and programmersWe need both these views into the software because the mechanical aspect of software development (does it do the right things?) is not the same as the business aspect (does it solve the business problem?).Testers then become customers and designers become business experts. This in turn helps us create an environment where understanding the customer environment and problem is not an accident but a direct result of what we are trying to achieve.Notice how in here we bring the previous role into the next step: this is based on the principle that it is people who communicate information, not documents. Documents serve as storage for information, maybe reminders. But without dialogue there is no information transfer. You need people to communicate information.If you design the process around the "understanding the customer and the problem" instead of the silo-ed step-by-step process of creating software you will be creating an organization that is designed to help the customer, not just design software! And this will lead to…
- Yep, happy customers!And all this just by applying only 2 of Deming's 14 points. Imagine what you can achieve by applying the other 12!
- So, what are we going to do next?
- In our competitive world of today we need to make our software industry 10x more productive to be able to compete internationally.
- For this we need to shed our old paradigms or analytical management and embrace the new paradigm of Organic process design around the customer's needs. We saw what the analytical process design can lead to and acknowledge that in most cases, analytical process designs lead to increased costs and reduced productivity. Applying a different design, by focusing on customer at every step in our process we can derive a much more effective process.In testing specifically we require a revolutionary mind-shift. We need to have the testers at the beginning of the process (not at the end!) to design our process around the customer need, not the mechanical steps of writing software. Key point: by understanding the customer needs our testers can provide much more value and their work can be much more interesting: they represent the customer, not the spec!
- You can start today alreadyby applying two simple principles:
- a) cease dependency on inspectionsb) break down barriers between departmentsAnd with these alone you can create a process that will be 10 times more effective!