Lean testing

7,019 views

Published on

This PowerPoint presentation explains the best practices for the use of Lean Management in the domain of IT Testing.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,019
On SlideShare
0
From Embeds
0
Number of Embeds
243
Actions
Shares
0
Downloads
104
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Lean testing

  1. 1. Lean Management for IT Testing Jean Louis LEQUEUX http://weltram.eu AFISI CONFERENCE - NDP 2010 V 3.0 – December 6 th , 2010
  2. 2. Grand Summary <ul><li>What is Lean Management? </li></ul><ul><li>Pledging for a Test repository </li></ul><ul><li>How to apply Lean Management to Testing </li></ul>
  3. 3. Lean Management for IT <ul><li>Reduce Waste </li></ul><ul><li>Optimize Processes </li></ul><ul><li>Increase Performances </li></ul><ul><li>Get better Quality, better Customer Satisfaction </li></ul>
  4. 4. Where does it apply? <ul><li>In all Service Areas where production is involved: </li></ul><ul><ul><li>Maintenance </li></ul></ul><ul><ul><li>Operation </li></ul></ul><ul><ul><li>Customer Support </li></ul></ul><ul><li>But also in any time & energy consuming subset of the Development Cycle </li></ul>
  5. 5. The Development Cycle <ul><li>Specification </li></ul><ul><li>Prototyping </li></ul><ul><li>Conception </li></ul><ul><li>Coding </li></ul><ul><li>Integration </li></ul><ul><li>End-to-end tests </li></ul><ul><li>Pre-production </li></ul><ul><li>Production </li></ul><ul><li>Evolution </li></ul><ul><li>Specification </li></ul><ul><li>Prototyping tests </li></ul><ul><li>Conception </li></ul><ul><li>Coding tests </li></ul><ul><li>Integration tests </li></ul><ul><li>End-to-end tests </li></ul><ul><li>Pre-production tests </li></ul><ul><li>Maintenance tests </li></ul><ul><li>Evolution tests </li></ul>
  6. 6. Test issues for Lean Management <ul><li>Issue 1: different teams </li></ul><ul><li>Issue 2: different periods </li></ul><ul><li>Issue 3: an everlasting “ do it again!” </li></ul>
  7. 7. Solution! <ul><li>A Test Repository is necessary </li></ul><ul><li>Lean Management on the different issues of the Test Repository </li></ul>
  8. 8. A T est R epository for the IS Jean Louis LEQUEUX [email_address] TRIS Un référentiel de tests pour le SI
  9. 9. Summary of Repository <ul><li>What does test means? </li></ul><ul><li>The IS Tests methodology </li></ul><ul><ul><li>8 steps </li></ul></ul><ul><ul><li>A Tests Repository for the Information System </li></ul></ul><ul><ul><li>ISO Approach </li></ul></ul><ul><li>Tests are the basement of a good IT System </li></ul><ul><li>Que signifie « tests »? </li></ul><ul><li>Méthodologie de tests </li></ul><ul><ul><li>8 étapes </li></ul></ul><ul><ul><li>Un référentiels de tests pour le système d’information </li></ul></ul><ul><ul><li>L’approche ISO </li></ul></ul><ul><li>Les tests sont fondamentaux ! </li></ul>
  10. 10. Testing <ul><li>Functional tests </li></ul><ul><ul><li>Unit testing </li></ul></ul><ul><ul><li>Assembly testing </li></ul></ul><ul><ul><li>Non regression tests </li></ul></ul><ul><li>Performance tests </li></ul><ul><ul><li>Load tests </li></ul></ul><ul><ul><li>Technical tests </li></ul></ul><ul><li>Security tests </li></ul><ul><li>Tests fonctionnels </li></ul><ul><ul><li>Unitaires </li></ul></ul><ul><ul><li>D’intégration </li></ul></ul><ul><ul><li>De non-régression </li></ul></ul><ul><li>Tests de performance </li></ul><ul><ul><li>Montée en charge </li></ul></ul><ul><ul><li>Tests techniques </li></ul></ul><ul><li>Tests de sécurité </li></ul>
  11. 11. Objectivity, Conformity… <ul><li>Manage a trustful tests program </li></ul><ul><li>Rely on repeatable results </li></ul><ul><li>Tests must be </li></ul><ul><ul><li>Conform to reality </li></ul></ul><ul><ul><li>Usable for predictive behavior of SI </li></ul></ul><ul><li>Gérer un programme de tests fiables </li></ul><ul><li>Se baser sur des résultats répétables </li></ul><ul><li>Les tests doivent être </li></ul><ul><ul><li>Conformes à la réalité </li></ul></ul><ul><ul><li>Utilisables pour le comportement prédictif du SI </li></ul></ul>
  12. 12. Today’s Situation <ul><li>For each application, different scenarios, different tools, even different sets of data are used… </li></ul><ul><li>No leading concepts for testing! </li></ul><ul><li>Pour chaque application, de différents scénarios, différentes données, différents outils… </li></ul><ul><li>Pas d’idées directrices pour les tests! </li></ul>
  13. 13. How Did We Get Here? <ul><li>Tests are often “forgotten” until the last stages of integration </li></ul><ul><li>Only functional tests draw the attention: </li></ul><ul><ul><li>Because the end-users are particularly interested </li></ul></ul><ul><li>Les tests sont souvent « oubliés » jusqu’aux dernières étapes de l’intégration… </li></ul><ul><li>Seuls les tests fonctionnels attire l’attention: </li></ul><ul><ul><li>Parce que les utilisateurs finaux s’y intéressent plus particulièrement </li></ul></ul>
  14. 14. Types of tests <ul><li>Tests apply to </li></ul><ul><li>Management Information Systems </li></ul><ul><li>Technical Information Systems </li></ul><ul><li>Les tests s’appliques aux: </li></ul><ul><li>Systèmes de gestion </li></ul><ul><li>Systèmes d’information techniques </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  15. 15. Tests Procedures <ul><li>Define tests procedures at early stages of project </li></ul><ul><li>Requested Performances must be listed </li></ul><ul><li>Define clearly current operational conditions </li></ul><ul><li>Propose tests records and tests validation chains </li></ul><ul><li>Définir les procédures de tests dès les phases initiales du projet </li></ul><ul><li>Liste obligatoires des performances requises </li></ul><ul><li>Définir clairement les conditions d’exploitation </li></ul><ul><li>Définir les rapports de tests et la chaîne de validation des tests </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  16. 16. Typical Transactions <ul><li>Define and describe typical transactions, with chaining of elementary transactions </li></ul><ul><li>Define end-to-end transactions </li></ul><ul><li>Définir et décrire les transactions types avec l’enchaînement des transactions élémentaires </li></ul><ul><li>Définir les chaînes de transactions de bout en bout </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  17. 17. Typical Data <ul><li>Build test data (the tests data set must be reusable) </li></ul><ul><li>Tests data must be realistic </li></ul><ul><li>Ideally, tests data structure derived from the data models </li></ul><ul><li>Construire des données de tests permanentes </li></ul><ul><li>Les données de tests doivent refléter la réalité </li></ul><ul><li>Idéalement, la structure des données de tests dérive des modèles de données. </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  18. 18. Re-usable Tools <ul><li>Choose pertinent tools, according to environment, tests objectives, etc. </li></ul><ul><li>Develop necessary complementary tools, such as injectors </li></ul><ul><li>Choisir les outils de tests en fonctions de l’environnement, des objectifs, etc. </li></ul><ul><li>Développer les outils complémentaires nécessaires, tels que les injecteurs </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  19. 19. Re-usable scenarios <ul><li>Write test scenarios as close as possible to real applications </li></ul><ul><li>Scenarios should be reusable </li></ul><ul><li>Scripts must be adaptable according to the evolutions of the IT System </li></ul><ul><li>Écrire des scénarios de tests aussi proches que possible de la réalité </li></ul><ul><li>Les scénarios doivent être réutilisables </li></ul><ul><li>Les scripts doivent pouvoir évoluer avec le SI </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  20. 20. Tests Platforms <ul><li>Tests platforms must be as much conformant as possible with the operational systems </li></ul><ul><li>Use simulators to build a model of target systems </li></ul><ul><li>Emulate physical and technical environment (bandwidth, traffic, etc.) </li></ul><ul><li>Les plateformes de tests doivent être aussi conformes que possibles aux systèmes </li></ul><ul><li>Utiliser des simulateurs pour modéliser le système cible </li></ul><ul><li>Émuler l’environnement technique et physique (bande passante, etc.) </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  21. 21. Tests Results <ul><li>Create templates for tests reports </li></ul><ul><li>Manage Tests Results </li></ul><ul><li>Use Document Management System </li></ul><ul><li>Results of past tests must be accessible </li></ul><ul><li>Créer des modèles de rapports de tests </li></ul><ul><li>Gérer les résultats des tests </li></ul><ul><li>Utiliser un Système de Gestion Documentaire </li></ul><ul><li>Résultats accessibles! </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  22. 22. Recommendation <ul><li>Urbanize First! </li></ul><ul><li>Define tests on an Urbanized Model </li></ul><ul><li>Define Domain for tests on Urbanized System </li></ul><ul><li>Urbaniser d’abord le SI! </li></ul><ul><li>Les tests sont défini sur le modèle urbanisé </li></ul><ul><li>Définir les domaines de tests! </li></ul>
  23. 23. A Tests Repository <ul><li>Build-up the Test Repository with : </li></ul><ul><ul><li>Best Practices recommendations </li></ul></ul><ul><ul><ul><li>Typical tests requested </li></ul></ul></ul><ul><ul><ul><li>Tests Procedures </li></ul></ul></ul><ul><ul><li>Tests objects </li></ul></ul><ul><ul><ul><li>Re-usable Tools </li></ul></ul></ul><ul><ul><ul><li>Test Data extracted from real Data Models </li></ul></ul></ul><ul><ul><ul><li>Typical transactions for each urbanized domain </li></ul></ul></ul><ul><ul><ul><li>Re-usable scenarios </li></ul></ul></ul><ul><ul><ul><li>Adaptable Scripts </li></ul></ul></ul><ul><ul><li>Tests Results </li></ul></ul><ul><li>Construire le référentiel de test comprenant: </li></ul><ul><ul><li>Les Meilleures Pratiques: </li></ul></ul><ul><ul><ul><li>Procédures </li></ul></ul></ul><ul><ul><ul><li>Tests requis </li></ul></ul></ul><ul><ul><li>Items de tests …/ </li></ul></ul><ul><ul><ul><li>Outils réutilisables </li></ul></ul></ul><ul><ul><ul><li>Données de tests extraites de modèles réels </li></ul></ul></ul><ul><ul><ul><li>Description des transactions typiques dans chaque domaine urbanisé </li></ul></ul></ul><ul><ul><ul><li>Scénarios réutilisables </li></ul></ul></ul><ul><ul><ul><li>Scripts adaptables </li></ul></ul></ul><ul><ul><li>Résultats des tests </li></ul></ul>
  24. 24. Pledging for an ISO Approach <ul><li>Beyond ISO 9001, a normative approach is necessary </li></ul><ul><li>IT tests are definitely “measures” </li></ul><ul><li>Is ISO/IEC 17025 adaptable to IT Tests Labs? </li></ul><ul><li>Au-delà de l’ISO 9001, une approche normative est nécessaire </li></ul><ul><li>Les tests informatiques font partie du domaine de la mesure </li></ul><ul><li>L’ISO/IEC/17025 est-elle adaptable à nos professions? </li></ul>
  25. 25. Lean Management Applied to IT Testing Jean Louis Lequeux http://weltram.eu
  26. 26. Lean Testing Summary <ul><li>Starting point </li></ul><ul><li>Where to optimize the processes </li></ul><ul><li>How to manage coordination </li></ul>
  27. 27. Lean Management of the Test Repository <ul><li>Internal processes </li></ul><ul><li>Inter processes </li></ul><ul><li>Specificity of each layer </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  28. 28. Internal processes <ul><li>Lean Management for each team: </li></ul><ul><ul><li>Optimize Resources </li></ul></ul><ul><ul><li>Optimize tools </li></ul></ul><ul><ul><li>Optimize exchanges </li></ul></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  29. 29. Inter processes <ul><li>Make sure each team pass all necessary information </li></ul><ul><li>Allow fluidity between teams </li></ul><ul><li>Cases and discussions: focus on solutions! </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  30. 30. Types of tests <ul><li>Apply Agile Method for Unit Testing </li></ul><ul><li>Use tools for detecting exhaustively cases </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  31. 31. Tests procedures <ul><li>Capitalize procedures </li></ul><ul><li>Make them reusable </li></ul><ul><li>Discuss with leaders of other testers’ teams </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  32. 32. Typical transactions <ul><li>Customers First! </li></ul><ul><li>Only real customers’ transactions should be considered </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  33. 33. Typical data <ul><li>Avoid costy tools </li></ul><ul><li>Extracts of real data are better, however </li></ul><ul><ul><li>They must be anonymized </li></ul></ul><ul><ul><li>… tools could be necessary </li></ul></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  34. 34. Reusable tools <ul><li>Tools are an important issue in terms of costs </li></ul><ul><li>Optimize tool set </li></ul><ul><li>Optimize training of teams </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  35. 35. Reusable Scenarios <ul><li>Users’ experience is absolutely necessary </li></ul><ul><li>Listen customers </li></ul><ul><li>Reusability submitted to identical environment </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  36. 36. Tests platforms <ul><li>Evaluate costs </li></ul><ul><li>TDaaS would cut down budget! </li></ul><ul><li>However TDaaS means also Management! </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  37. 37. Tests results <ul><li>The test results must be capitalized and centralized on a site </li></ul><ul><li>Use Wiki in order to exchange and enrich testing knowledge! </li></ul>Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  38. 38. There is no dichotomy! <ul><li>While building up the Test Repository, think in terms of Lean Management </li></ul><ul><li>The Repository is the first step of Lean Management for testing </li></ul><ul><li>Lean Management applied to testing is a good start for </li></ul>
  39. 39. Apply Lean Management to testing <ul><li>Lean Management for testing is applicable on the whole cycle (horizontal bar….)! </li></ul>Prototyping Coding Integration End-to-end Pre-prod Maintenance Evolution Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  40. 40. Why Lean? <ul><li>Avoid over-testing </li></ul><ul><li>Insure complete transfer of testing information: </li></ul><ul><ul><li>Thru the complete life cycle </li></ul></ul><ul><ul><li>Thru the different teams of the project </li></ul></ul>Prototyping Coding Integration End-to-end Pre-prod Maintenance Evolution Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  41. 41. 5S?  OK <ul><li>Seiri: Sort out </li></ul><ul><li>Seiton: Systemize </li></ul><ul><li>Seso: Cleaning/Shining  Tools! Docs! </li></ul><ul><li>Seiketsu: Standardize </li></ul><ul><li>Shitsuke - Sustain/Discipline </li></ul>Prototyping Coding Integration End-to-end Pre-prod Maintenance Evolution Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  42. 42. DMAIC Testing <ul><li>Define </li></ul><ul><li>Measure </li></ul><ul><li>Analyze </li></ul><ul><li>Improve </li></ul><ul><li>Control </li></ul>Prototyping Coding Integration End-to-end Pre-prod Maintenance Evolution Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  43. 43. Six Sigma: Metrics ? <ul><li>In terms of </li></ul><ul><li>Test Cases </li></ul><ul><li>Type of errors </li></ul><ul><li>Type of solution </li></ul>Prototyping Coding Integration End-to-end Pre-prod Maintenance Evolution Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  44. 44. Key indicators <ul><li>Test program </li></ul><ul><li>Daily follow-up </li></ul><ul><li>Tests achievement </li></ul>Prototyping Coding Integration End-to-end Pre-prod Maintenance Evolution Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  45. 45. Beyond technical tools: Communication tools forever! <ul><li>Testing should be perennial! </li></ul><ul><li>Capitalize </li></ul><ul><li>Use forums, Wiki, etc. </li></ul><ul><li>Update documents on Intranet </li></ul><ul><li>Create or join Communities </li></ul>Prototyping Coding Integration End-to-end Pre-prod Maintenance Evolution Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  46. 46. Don’t Forget CMMI! <ul><li>If CMMI is requested: </li></ul><ul><li>CMMI applies to the complete cycle </li></ul><ul><li>Lean Management helps in maintaining CMMI level and classification </li></ul>Prototyping Coding Integration End-to-end Pre-prod Maintenance Evolution Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  47. 47. Don’t forget ITIL! <ul><li>Important points: </li></ul><ul><li>Production! </li></ul><ul><li>ITIL Approach is necessary </li></ul>Prototyping Coding Integration End-to-end Pre-prod Maintenance Evolution Types of tests Typical transactions Typical data Re-usable tools Re-usable scenarios Tests platforms Tests results Tests procedures
  48. 48. More on http://weltram.eu/leanit.aspx

×