Download to read offline
Uploaded byAsh Winter
Architectural Testability Workshop for Test Academy Barcelona
The document discusses architectural testability, emphasizing its crucial role in quality assurance and system functionality. It introduces the CODS model, which outlines controllability, observability, decomposability, and simplicity as key factors that enhance testability. The challenges and 'smells' of poor architectural testability are also examined, along with strategies for improvement through better communication and collaboration within teams.
More Related Content
![[DevDay2019] Power of Test Automation and DevOps combination - One click savi...](https://cdn.slidesharecdn.com/ss_thumbnails/devdaydanang2019-powerofdevopsintestautomationtoanleedit-190410073339-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Architectural Testability Workshop for Test Academy Barcelona
More from Ash Winter
Architectural Testability Workshop for Test Academy Barcelona
- 1. Testing the Bigger Picture- Architectural Testability @northern_tester Ash Winter – tester, speaker, author, testability advocate
- 2. • Distracted bylocal optima… • Where is the biggest gain to be made in the system? • Your architecture silly! Testers. Always chasing the feature squirrel… @northern_tester
- 3. @northern_tester If the architecturaltestability of your system is poor, it means… …it won’t get tested …the tester will test it …it won’t work …it will be hard to operate …it will be frustrating to use …it will make your team sad …it will create bottlenecks …it will make less money
- 4. What we willcover… @northern_tester • Team Testability Test • Your testing smells • CODS model • Improving your world
- 5. Testability is just avague concept @northern_tester Testability is only a system property
- 6. It’s a social problemto solve @northern_tester It’s a technical problem to solve
- 7.
- 8. Can’t we just geton with it? @northern_tester Confused with automatability
- 9. Software testability ishow easy a product or system is to test. @northern_tester
- 10. Software testability isthe degree to which a software artefact supports testing in a given test context. If the testability of the software artefact is high, then finding faults in the system by means of testing is easier. @northern_tester
- 11. Practical Testability by JamesBach @northern_tester
- 12. Useful but hard to discusswith stakeholders @northern_tester
- 13.
- 14. Better! A familiar modelto get people on board @northern_tester
- 15. @northern_tester People, teams, organisations, relationships, information flow Technologies, languages,libraries, tools, architecture, ops
- 16. @northern_tester Barely a mentionin many testing models
- 17. @northern_tester Ability to test •Impactful • Performing testing increases testability • Diverse techniques • Bottleneck
- 18. Testing your teamfor testability… @northern_tester Without them suspecting a thing…
- 19. @northern_tester Who is this… •It’s Joel Spolsky • Writer of blogs at joelonsoftware.com • FogBugz • Trello • Co-founder of Stack Overflow • Most importantly though, The Joel Test!
- 20. @northern_tester WTF… • The JoelTest • 12 Steps to Better Code • Not all patterns but teams, working conditions etc. • Insight without the maturity model…
- 21. @northern_tester Test and Testability •Your testing tells you about your testability • You can cover a lot of ground • While not really talking about testability • Just talking about the testing you do already
- 22. Exercise: The TeamTest for Testability @northern_tester
- 23. @northern_tester The Team Test •Yes or No • Fast • Social aspects • Technical aspects • Retrospective • Team similarities & differences
- 24. Learning @northern_tester • Testability challengesmanifest in our testing but testing may not be the root • Talking about testability too early can cause engagement problems • Use existing ceremonies to gather testability information • Might expose ‘easy wins’ such as access problems
- 25. Architectural testability allowsyour team to make changes with very low cost and risk. @northern_tester
- 26. Architectural testability allowsyour team to create tests that provide valuable feedback within minutes, if not seconds. Each team member can run a set of tests every time they make a change. This feedback provides a safety net that encourages your team members to make changes frequently without fear. Confidence in testing encourages refactoring and continuous improvement of the code. @northern_tester
- 27. Draw your architecture @northern_tester •My architecture • Take 5 minutes to draw yours out, don’t worry about too much detail
- 28. What does goodlook like? @northern_tester • Embraces change • Decoupled but cohesive • Minimizes waste • Sustainable • Relationships
- 29. Good memories… @northern_tester • Rightsize microservices, responsible for just enough • Each service has a mock for early testing by consumers • Code instrumented with structured, contextual events • Consistent API docs expressed in Swagger
- 30. What does badlook like? @northern_tester • Slows as complexity grows • Technical debt pooling • Untested areas • Side effects • Persistent team siloing
- 31. Painful memories… @northern_tester • Threetypes of consumer, SOAP, batch, web • Mixed technologies - LAMP but with Microsoft SQL Server • Two minutes for new application test environment, two weeks for the database • Different logging and monitoring tooling by layer administered by different teams.
- 32. @northern_tester Retrofitting • Is really,really hard • Architectures act as anchors • Have you ever tried to change an architecture for testing? • Tiny changes to move the needle
- 33. Introducing the CODSmodel for architectural testability @northern_tester
- 34. Controllability… @northern_tester • You canset the system into its important states • Identify and control important variables Configuration • Control the environment the system resides in Test data
- 35. Observability… @northern_tester • Infer internalstate from external outputs • Logs - a message • Metrics – a measure • Events – a collection for a unit of work • Instrument for the information we want
- 36. Decomposability… @northern_tester • Independent testable components •Test alone • Test unique combinations • Learn and debug
- 37. Simplicity… @northern_tester • How easythe system is to understand • Consumers, inputs, outputs, interactions, technologies, protocols, configurations • Transparency • Cognitive load again
- 38.
- 39. When can weuse this… @northern_tester • Story kick off • Post-mortem • Architecture design and review
- 40. We need to beable to detect problems @northern_tester Use the CODS model to help us make the right changes
- 41. @northern_tester Smells of PoorArchitectural Testability
- 42. @northern_tester Not always obvious •Symptoms show in daily work • Queues • Bottlenecks • Cadence • Frustrated team • Frustrated stakeholders
- 43. @northern_tester My mobile testingstory… Pull a feature branch, build a development environment, configure relevant external feeds, find device, join a specific Wi-Fi network, change DNS, configure a local proxy to intercept and change certain headers and URI's.
- 44. @northern_tester Problems • Desensitized • Hoursfrom build to test • Shallow testing • Automation? • Ready to test work • Frustrated everyone
- 45. What smells doyou recognize? @northern_tester Release Management Theatre Mono strategies Fear of change Teams looking for more testers Too many user interface tests Valuable scenarios not tested Lack of resilience testing “Sunny” days only Cluttered logging with no insights Excessive test repetition Issues hard to isolate and debug Tests that don’t die Lengthy build times Too many persistent environments Environments no one cares about Inanimate documentation Customer hand holding Poor relations with Ops Long lists of deferred bugs Hard to test dependencies Wrangling over scale 95th Percentile? Tester turnover
- 46. Exercise: Detecting TestingSmells @northern_tester
- 47. Experiencing @northern_tester • High maintenanceautomation due to dependencies • Hard to debug and isolate problems • Mock two key external services • Unique tracing id throughout their logging
- 48. Doing @northern_tester • Rate eachsmell for your current context based on the handout • If you have friends here from your company, get together.
- 49. Learning @northern_tester • Causes arenot obvious • The whole architecture is the target area, not just the app. • Looking for smells can yield testability improvements • Common smells and unique smells…
- 50.
- 51. @northern_tester Tactics for testability •Controllability • Feature flags • Canary release • Automated deployment • Blue/green • Ephemeral environments
- 52. @northern_tester Tactics for testability •Observability • Events over logs • Tracing • Visualization • Traffic replay • Diagnostic & application information
- 53. @northern_tester Tactics for testability •Decomposability • Microservices • Circuit breakers • Message brokers • Mocks and spies
- 54. @northern_tester Tactics for testability •Simplicity • Automated documentation • Team on call • Technical debt • Pairing & mobbing • Architecture Decision Records
- 55. @northern_tester When testing isn’tenough • Scared testers • Updating core library • Major versions • Small increments • Load testing • Deployed to limited live services
- 56. Exercise: Applying the CODSModel @northern_tester
- 57.
- 58.
- 59. @northern_tester Operability Operability Operability Operability Logs to events Retired unused API’s New services forhigh throughput API’s Circuit breakers Throttling Tracing ID’s Actionable alerts Architectural Decision Records Feature Toggle Dashboard
- 60. Doing @northern_tester • Look whereyour testing stinks • Determine where to focus CODS • Add to your architecture • Annotate with recommendations
- 61. Thinking @northern_tester Think about: • Sizeof change? • Where to start?
- 62. Learning @northern_tester • Smells giveus clues on where to focus • CODS provides a framework for change • Visualizing is great for advocacy • Choose changes carefully…
- 63.
- 64.
- 65.
- 66.
Editor's Notes
- #19 Controllability determines the dept and breath of our testing efforts - how deep you can go while still knowing what breadth you have covered. Without this you can go down the rabbit hole and miss the bigger picture. Without control testing is pushed later and later, Controllability determines what scenarios we can exercise - whether it be setting test data to the right state or ensuring a dependency returns a specific response.
- #39 Controllability determines the dept and breath of our testing efforts - how deep you can go while still knowing what breadth you have covered. Without this you can go down the rabbit hole and miss the bigger picture. Without control testing is pushed later and later, Controllability determines what scenarios we can exercise - whether it be setting test data to the right state or ensuring a dependency returns a specific response.
- #46 2000 bugs in 2 years Communicated through tickets Longs test cycles against builds on long lived environments Mastered weirdly named tooling “Quality Centre” Left with a weird feeling - we did tons of testing, but we never got any faster, no one got what they wanted…
- #58 Controllability determines the dept and breath of our testing efforts - how deep you can go while still knowing what breadth you have covered. Without this you can go down the rabbit hole and miss the bigger picture. Without control testing is pushed later and later, Controllability determines what scenarios we can exercise - whether it be setting test data to the right state or ensuring a dependency returns a specific response.
- #59 Controllability determines the dept and breath of our testing efforts - how deep you can go while still knowing what breadth you have covered. Without this you can go down the rabbit hole and miss the bigger picture. Without control testing is pushed later and later, Controllability determines what scenarios we can exercise - whether it be setting test data to the right state or ensuring a dependency returns a specific response.
- #60 Controllability determines the dept and breath of our testing efforts - how deep you can go while still knowing what breadth you have covered. Without this you can go down the rabbit hole and miss the bigger picture. Without control testing is pushed later and later, Controllability determines what scenarios we can exercise - whether it be setting test data to the right state or ensuring a dependency returns a specific response.