Your SlideShare is downloading. ×
0
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Stephen K. Allott - People Challenges for Test Teams - SoftTest Ireland

691

Published on

Visit SoftTest Ireland www.softtest.ie and sign up for access to free Irish Software Testing events.

Visit SoftTest Ireland www.softtest.ie and sign up for access to free Irish Software Testing events.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
691
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. People Challenges for Test Teams Stephen K. AllottSoftTest Ireland Copyright 2008 ElectroMind Ltd 1
  • 2. SoftTest Ireland Copyright 2008 ElectroMind Ltd 2
  • 3. SoftTest Ireland Copyright 2008 ElectroMind Ltd 3
  • 4. SoftTest Ireland Copyright 2008 ElectroMind Ltd 4
  • 5. Introductions My background Aims of this workshop Explain some common testing challenges based on recent experience Ask for your experiences Agree a list of challenges we all face as testers Try to come up with some quick wins & longer term approaches to solutions SoftTest Ireland Copyright 2008 ElectroMind Ltd 5
  • 6. My recent clients Investment bank Airline web site Travel company web site Mobile phone operator Retail web site Travel company back office Recruitment web site Includes assignments done in association with TestHouse (London) and Neueda Technologies (Antrim & Dublin) SoftTest Ireland Copyright 2008 ElectroMind Ltd 6
  • 7. Their technologies Microsoft Dot (.) NET 1/2/3 Microsoft VSTS / TFS Oracle MS SQL Server Windows XP Citrix Metaframe Quality Centre / QTP SoftTest Ireland Copyright 2008 ElectroMind Ltd 7
  • 8. Number of testers Typically 12 to 14 in the test team A couple of clients had only 1, some have none! SoftTest Ireland Copyright 2008 ElectroMind Ltd 8
  • 9. Scale 0 1 2 3 4 5 6 7 8 9 10 11 12 13Key Area Controlled Efficient OptimisingTest strategy A B C DLife-cycle model A BMoment of involvement A B C DEstimating and planning A BTest specification techniques A BStatic test techniques A BMetrics A B C DTest automation A B CTest environment A B COffice environment ACommitment and motivation A B CTest functions and training A B CScope of methodology A B CCommunication A B CReporting A B C DDefect management A B CTestware management A B C DTest process management A B CEvaluation A BLow-level testing A B C Composite TPI Test Maturity Matrix from several clients SoftTest Ireland Copyright 2008 ElectroMind Ltd 9
  • 10. Summary of TPI results No documented test process No formal test policy, strategy or plan Many tests, no formal design No metrics No proper test environment Some tested in “live” No documented test basis Not involved early enough SoftTest Ireland Copyright 2008 ElectroMind Ltd 10
  • 11. Some common observations Constantly changing business priorities Increasing (frantic) demand for new functionality Weekly, sometimes daily release cycles Attempt to use automation to solve all the problems No numbers to explain testing’s added value Small important teams yet with little influence SoftTest Ireland Copyright 2008 ElectroMind Ltd 11
  • 12. People challenges – behaviour Motivation A few are resigned to how it is, no hope? Some expect spoon feeding Some adopt a negative attitude Many are willing to make improvements, if there was time Turnover – very high offshore Culture Some managers get too involved in detail Co-dependency – testers must push back It hasn’t broken yet so . . . SoftTest Ireland Copyright 2008 ElectroMind Ltd 12
  • 13. People challenges – skillsTesting Skills IT Skills Processes Programming Standards Databases Techniques Infrastructure Tools WebDomain Knowledge Soft Skills Industry specific Communication Application knowledge Negotiation E-Business Managing technical teams SoftTest Ireland Copyright 2008 ElectroMind Ltd 13
  • 14. People challenges – perception In general testers are not valued Testers are often blamed Project delays Poor quality Managing expectations with RBT Lack of awareness of what the test team does SoftTest Ireland Copyright 2008 ElectroMind Ltd 14
  • 15. People challenges – managers Demand for certified staff Demand for skills & knowledge Pressure to cut costs and use offshore resources Developing good test leads Role of testers in an agile project Selling the value of testing to the organisation SoftTest Ireland Copyright 2008 ElectroMind Ltd 15
  • 16. ThemesFacilitated discussion bearing in mind the following themes:1. Relentless business pressure for new functionality2. Constant and last minute changes to requirements3. Introduction of new development / testing methodologies e.g. agile4. Very frequent releases, often weekly, sometimes daily!5. Increasing application complexity over time6. Knowledge gap (e.g. what lies beyond the browser?)7. Steep learning curve (business as well as technical)8. Use of contract and offshore resources (loss of in-house expertise)9. Little or no documentation against which to test10. Lack of attention to all testing requirements (e.g. middleware, back end database, non-functional quality characteristics) SoftTest Ireland Copyright 2008 ElectroMind Ltd 16
  • 17. Discussion – how do we respond?Business Pressures Technology Offshore New functionality Complex Cheaper Frequent change Rapid change Plentiful Maturity Interfaces ExpertiseBehaviour Skills Managers Motivation Analysis Costs Attitude IT / Technical Time to market Culture Business Agile Testing Soft skills SoftTest Ireland Copyright 2008 ElectroMind Ltd 17
  • 18. Conclusions In my opinion testers of the future must . . . Measure / appreciate application complexity Be able to accept and manage last minute changes Be flexible but push back with estimates Increase their technical knowledge Learn more about testing, especially techniques Apply the tried and tested techniques Capture metrics to help prove the business case Get involved in the projects much earlier Recruit / train / certify / develop / coach = Retain Share your views with the group SoftTest Ireland Copyright 2008 ElectroMind Ltd 18
  • 19. Final Thoughts / Questions? SoftTest Ireland Copyright 2008 ElectroMind Ltd 19
  • 20. People Challenges for Test Teams stephen.allott@electromind.com +44(0) 773 476 1363 www.electromind.comSoftTest Ireland Copyright 2008 ElectroMind Ltd 20
  • 21. People Challenges for Test Teams Appendix – Bonus SlidesSoftTest Ireland Copyright 2008 ElectroMind Ltd 21
  • 22. A typical project in trouble CR Churn by Month - AveCR=47 AveChurn=150 AveChurnCR=3 300 279 300 263 250 254 250 229 213 200 200 Changes to CRs CR count 150 150 116 124 100 100 65 58 55 50 40 50 27 14 0 0 0 0 June July August September October Months CRs Churn Bugs SoftTest Ireland Copyright 2008 ElectroMind Ltd 22
  • 23. Bug snapshot by priority Sev 1 Sev 2 Sev 3 Sev 4 Total Pri 1 57 81 14 152 Pri 2 19 52 12 1 84 Pri 3 6 14 10 30 Pri 4 2 1 3 None 68 288 114 10 480 Total 146 428 154 21 749 SoftTest Ireland Copyright 2008 ElectroMind Ltd 23
  • 24. Bug snapshot by resolution Sev 1 Sev 2 Sev 3 Sev 4 TotalBy design 10 37 29 4 80Duplicate 6 33 7 1 47External 2 16 4 0 22Fixed 86 254 95 12 447Not repro 29 56 4 0 89Postponed 12 17 6 4 39Will not fix 1 15 9 0 25Total 146 428 154 21 749 SoftTest Ireland Copyright 2008 ElectroMind Ltd 24
  • 25. Impact of the improvements CR Churn by Month - AveCR=47 AveChurn=150 AveChurnCR=3 300 279 300 263 250 254 250 229 213 Changes to CRs 200 200 CR count 150 150 116 124 100 86 100 65 66 90 58 55 61 50 40 50 27 30 14 17 18 15 15 10 10 5 5 13 12 9 0 0 0 5 1 0 r ry ne ry r ly ch ril r er t be be be s Ju Ap ua a gu ob Ju ar m nu m em br M Au ct ve ce Ja Fe O pt No De Se Months CRs Churn Bugs SoftTest Ireland Copyright 2008 ElectroMind Ltd 25
  • 26. Corporate Sales and Sales and Marketing Systems 1. Consulting Practice Mgt.Marketing Systems 2. Vendor Account Mgt. Events Systems 3. Event Mgt. 1. Vendor 4. Leads and Referral 2. User Reg. Databases 3. User Shop 1. Profile 2. Common 3. Event OLTPSystem Interfaces 4. Transaction OLTP 1. Event Orders 5. Contact History OLTP 2. Billing & Financials 6. Domain Reference 3. Credit Card (real-time) 4. Event Communication / FAX 5. Direct email service World-wide Marketing DB 6. Direct mail Postal interface 1. Profile OLAP 7. Leads & Referrals 2. Event OLAP 8. OLTP Staging Interface 3. Analytical Reporting 9. OLAP Staging Interface 4. Contract History OLAP 10. SAP 5. Domain 11. Licensing 12. Call Center Source: Reality Test Inc.
  • 27. Basic complexity calculation Characteristic Value Factor Total Windows 37 8 296 Objects 23 1 23 Tables 47 1 47 Apps Impacted 3 10 30 Factors are weighted from a testing perspective SoftTest Ireland Copyright 2008 ElectroMind Ltd 27
  • 28. Comparison across applicationsApplication UI Back Env Test Middle Total End Code TierSales Mgr. 1360 2250 152 0 0 3762Call Center 960 824 90 129 0 2003Web Sales 389 183 109 0 2 681Middleware 0 146 23 27 232 428component SoftTest Ireland Copyright 2008 ElectroMind Ltd 28
  • 29. Complexity comparison 4000 Complexity Value 3000 2000 1000 0 er er r e ss TR ta es te as ag ag el la en al C B D G an an S C SS e g eb M M dg l in al P W nt es ok le C ou w al Lo no S cc K A Application Name SoftTest Ireland Copyright 2008 ElectroMind Ltd 29
  • 30. Actual staffing comparison 20 Number of Testers 15 Estimated Actual 10 5 0 er er r e ss TR ta es te as ag ag el la en al C B D G an an S C SS e eb g M M dg l in al P W nt es ok C le ou w al Lo no S cc K A Applications SoftTest Ireland Copyright 2008 ElectroMind Ltd 30
  • 31. Organisation evaluation Organisation within the company Process definition and execution Teamwork Ability to do the job SoftTest Ireland Copyright 2008 ElectroMind Ltd 31
  • 32. Organisation within the company Section Item being evaluated Max Score 1.1 Reporting structure 50 1.2 Professionalism 170 1.3 Organisational churn 135 1.4a Staffing – permanence 30 1.4b Staffing – stability 35 1.4c Staffing – morale 50 1.4d Staffing – number of testers 40 1.4e Staffing – organisation type 40 1.5 Budget control 30 SoftTest Ireland Copyright 2008 ElectroMind Ltd 32
  • 33. Process definition and execution Section Item being evaluated Max Score 2.1 Software development life-cycle 60 2.2 Design reviews and documentation 70 2.3 Early participation in development 50 2.4 Ability to influence schedule deadlines 60 2.5 Development standards 60 2.6 Defined testing approach & procedures 190 2.7 Metrics reporting 60 2.8 Churn 90 2.9 Patch releases 40 2.10 Post-project reviews 60 SoftTest Ireland Copyright 2008 ElectroMind Ltd 33
  • 34. Teamwork Section Item being evaluated Max Score 3.1 Support from production support 60 3.2 Support from management 60 3.3 Correct level of quality 90 3.4 Resource allocation 80 SoftTest Ireland Copyright 2008 ElectroMind Ltd 34
  • 35. Ability to do the job Section Item being evaluated Max Score 4.1 Adequate equipment / lab 200 4.2 Automated testing 70 4.3 Programming ability 70 4.4 Non-testing activities 50 SoftTest Ireland Copyright 2008 ElectroMind Ltd 35
  • 36. Total evaluation score 1300 Total Evaluation Score 1200Evaluation 1100 1000 900 Q1/00 Q2/00 Q3/00 Q4/00 Q1/01 Q2/01 Q3/01 Date SoftTest Ireland Copyright 2008 ElectroMind Ltd 36
  • 37. Organisational Evaluation 500 Organization 400 Process TeamingEvaluation Ability 300 200 100 Q1/00 Q2/00 Q3/00 Q4/00 Q1/01 Q2/01 Q3/01 Date SoftTest Ireland Copyright 2008 ElectroMind Ltd 37
  • 38. Components of a centre of excellence l TesProject 1 p oo t l ab e o urc s Re Tool ExpertsProject 2 Test Managers SIGProject 3 e Knowledge Bas C onProject 4 re su ent lta DC nt R& s SoftTest Ireland Copyright 2008 ElectroMind Ltd 38
  • 39. Benefits of a centre of excellence Independent testing Variety of work Breadth & depth Identity, pride Fewer product recalls Morale, professionalism Fewer production failures Extra revenue stream Central reference point Consistent approach Knowledgeable staff Multi-methodologies Expertise across org. Earlier involvement Structured career path SoftTest Ireland Copyright 2008 ElectroMind Ltd 39

×