Architectural Testability Workshop for Test Academy Barcelona
•Download as PPTX, PDF•
2 likes•247 views
Workshop delivered at Test Academy Barcelona on 30th January 2020. Including the Team Test for Testability, Testability Tactics, Testing Smells and the CODS Model.
More Related Content
What's hot
What's hot (20)
Similar to Architectural Testability Workshop for Test Academy Barcelona
Similar to Architectural Testability Workshop for Test Academy Barcelona (20)
More from Ash Winter
More from Ash Winter (20)
Recently uploaded
Recently uploaded (20)
Architectural Testability Workshop for Test Academy Barcelona
- 1. Testing the Bigger Picture - Architectural Testability @northern_tester Ash Winter – tester, speaker, author, testability advocate
- 2. • Distracted by local optima… • Where is the biggest gain to be made in the system? • Your architecture silly! Testers. Always chasing the feature squirrel… @northern_tester
- 3. @northern_tester If the architectural testability of your system is poor, it means… …it won’t get tested …the tester will test it …it won’t work …it will be hard to operate …it will be frustrating to use …it will make your team sad …it will create bottlenecks …it will make less money
- 4. What we will cover… @northern_tester • Team Testability Test • Your testing smells • CODS model • Improving your world
- 5. Testability is just a vague concept @northern_tester Testability is only a system property
- 6. It’s a social problem to solve @northern_tester It’s a technical problem to solve
- 8. Can’t we just get on with it? @northern_tester Confused with automatability
- 9. Software testability is how easy a product or system is to test. @northern_tester
- 10. Software testability is the degree to which a software artefact supports testing in a given test context. If the testability of the software artefact is high, then finding faults in the system by means of testing is easier. @northern_tester
- 11. Practical Testability by James Bach @northern_tester
- 12. Useful but hard to discuss with stakeholders @northern_tester
- 14. Better! A familiar model to get people on board @northern_tester
- 15. @northern_tester People, teams, organisations, relationships, information flow Technologies, languages, libraries, tools, architecture, ops
- 16. @northern_tester Barely a mention in many testing models
- 17. @northern_tester Ability to test • Impactful • Performing testing increases testability • Diverse techniques • Bottleneck
- 18. Testing your team for testability… @northern_tester Without them suspecting a thing…
- 19. @northern_tester Who is this… • It’s Joel Spolsky • Writer of blogs at joelonsoftware.com • FogBugz • Trello • Co-founder of Stack Overflow • Most importantly though, The Joel Test!
- 20. @northern_tester WTF… • The Joel Test • 12 Steps to Better Code • Not all patterns but teams, working conditions etc. • Insight without the maturity model…
- 21. @northern_tester Test and Testability • Your testing tells you about your testability • You can cover a lot of ground • While not really talking about testability • Just talking about the testing you do already
- 22. Exercise: The Team Test for Testability @northern_tester
- 23. @northern_tester The Team Test • Yes or No • Fast • Social aspects • Technical aspects • Retrospective • Team similarities & differences
- 24. Learning @northern_tester • Testability challenges manifest in our testing but testing may not be the root • Talking about testability too early can cause engagement problems • Use existing ceremonies to gather testability information • Might expose ‘easy wins’ such as access problems
- 25. Architectural testability allows your team to make changes with very low cost and risk. @northern_tester
- 26. Architectural testability allows your team to create tests that provide valuable feedback within minutes, if not seconds. Each team member can run a set of tests every time they make a change. This feedback provides a safety net that encourages your team members to make changes frequently without fear. Confidence in testing encourages refactoring and continuous improvement of the code. @northern_tester
- 27. Draw your architecture @northern_tester • My architecture • Take 5 minutes to draw yours out, don’t worry about too much detail
- 28. What does good look like? @northern_tester • Embraces change • Decoupled but cohesive • Minimizes waste • Sustainable • Relationships
- 29. Good memories… @northern_tester • Right size microservices, responsible for just enough • Each service has a mock for early testing by consumers • Code instrumented with structured, contextual events • Consistent API docs expressed in Swagger
- 30. What does bad look like? @northern_tester • Slows as complexity grows • Technical debt pooling • Untested areas • Side effects • Persistent team siloing
- 31. Painful memories… @northern_tester • Three types of consumer, SOAP, batch, web • Mixed technologies - LAMP but with Microsoft SQL Server • Two minutes for new application test environment, two weeks for the database • Different logging and monitoring tooling by layer administered by different teams.
- 32. @northern_tester Retrofitting • Is really, really hard • Architectures act as anchors • Have you ever tried to change an architecture for testing? • Tiny changes to move the needle
- 33. Introducing the CODS model for architectural testability @northern_tester
- 34. Controllability… @northern_tester • You can set the system into its important states • Identify and control important variables Configuration • Control the environment the system resides in Test data
- 35. Observability… @northern_tester • Infer internal state from external outputs • Logs - a message • Metrics – a measure • Events – a collection for a unit of work • Instrument for the information we want
- 36. Decomposability… @northern_tester • Independent testable components • Test alone • Test unique combinations • Learn and debug
- 37. Simplicity… @northern_tester • How easy the system is to understand • Consumers, inputs, outputs, interactions, technologies, protocols, configurations • Transparency • Cognitive load again
- 39. When can we use this… @northern_tester • Story kick off • Post-mortem • Architecture design and review
- 40. We need to be able to detect problems @northern_tester Use the CODS model to help us make the right changes
- 41. @northern_tester Smells of Poor Architectural Testability
- 42. @northern_tester Not always obvious • Symptoms show in daily work • Queues • Bottlenecks • Cadence • Frustrated team • Frustrated stakeholders
- 43. @northern_tester My mobile testing story… Pull a feature branch, build a development environment, configure relevant external feeds, find device, join a specific Wi-Fi network, change DNS, configure a local proxy to intercept and change certain headers and URI's.
- 44. @northern_tester Problems • Desensitized • Hours from build to test • Shallow testing • Automation? • Ready to test work • Frustrated everyone
- 45. What smells do you recognize? @northern_tester Release Management Theatre Mono strategies Fear of change Teams looking for more testers Too many user interface tests Valuable scenarios not tested Lack of resilience testing “Sunny” days only Cluttered logging with no insights Excessive test repetition Issues hard to isolate and debug Tests that don’t die Lengthy build times Too many persistent environments Environments no one cares about Inanimate documentation Customer hand holding Poor relations with Ops Long lists of deferred bugs Hard to test dependencies Wrangling over scale 95th Percentile? Tester turnover
- 46. Exercise: Detecting Testing Smells @northern_tester
- 47. Experiencing @northern_tester • High maintenance automation due to dependencies • Hard to debug and isolate problems • Mock two key external services • Unique tracing id throughout their logging
- 48. Doing @northern_tester • Rate each smell for your current context based on the handout • If you have friends here from your company, get together.
- 49. Learning @northern_tester • Causes are not obvious • The whole architecture is the target area, not just the app. • Looking for smells can yield testability improvements • Common smells and unique smells…
- 51. @northern_tester Tactics for testability • Controllability • Feature flags • Canary release • Automated deployment • Blue/green • Ephemeral environments
- 52. @northern_tester Tactics for testability • Observability • Events over logs • Tracing • Visualization • Traffic replay • Diagnostic & application information
- 53. @northern_tester Tactics for testability • Decomposability • Microservices • Circuit breakers • Message brokers • Mocks and spies
- 54. @northern_tester Tactics for testability • Simplicity • Automated documentation • Team on call • Technical debt • Pairing & mobbing • Architecture Decision Records
- 55. @northern_tester When testing isn’t enough • Scared testers • Updating core library • Major versions • Small increments • Load testing • Deployed to limited live services
- 56. Exercise: Applying the CODS Model @northern_tester
- 58. @northern_tester
- 59. @northern_tester Operability Operability Operability Operability Logs to events Retired unused API’s New services for high throughput API’s Circuit breakers Throttling Tracing ID’s Actionable alerts Architectural Decision Records Feature Toggle Dashboard
- 60. Doing @northern_tester • Look where your testing stinks • Determine where to focus CODS • Add to your architecture • Annotate with recommendations
- 61. Thinking @northern_tester Think about: • Size of change? • Where to start?
- 62. Learning @northern_tester • Smells give us clues on where to focus • CODS provides a framework for change • Visualizing is great for advocacy • Choose changes carefully…
- 63. Handouts
Editor's Notes
- Controllability determines the dept and breath of our testing efforts - how deep you can go while still knowing what breadth you have covered. Without this you can go down the rabbit hole and miss the bigger picture. Without control testing is pushed later and later, Controllability determines what scenarios we can exercise - whether it be setting test data to the right state or ensuring a dependency returns a specific response.
- Controllability determines the dept and breath of our testing efforts - how deep you can go while still knowing what breadth you have covered. Without this you can go down the rabbit hole and miss the bigger picture. Without control testing is pushed later and later, Controllability determines what scenarios we can exercise - whether it be setting test data to the right state or ensuring a dependency returns a specific response.
- 2000 bugs in 2 years Communicated through tickets Longs test cycles against builds on long lived environments Mastered weirdly named tooling “Quality Centre” Left with a weird feeling - we did tons of testing, but we never got any faster, no one got what they wanted…
- Controllability determines the dept and breath of our testing efforts - how deep you can go while still knowing what breadth you have covered. Without this you can go down the rabbit hole and miss the bigger picture. Without control testing is pushed later and later, Controllability determines what scenarios we can exercise - whether it be setting test data to the right state or ensuring a dependency returns a specific response.
- Controllability determines the dept and breath of our testing efforts - how deep you can go while still knowing what breadth you have covered. Without this you can go down the rabbit hole and miss the bigger picture. Without control testing is pushed later and later, Controllability determines what scenarios we can exercise - whether it be setting test data to the right state or ensuring a dependency returns a specific response.
- Controllability determines the dept and breath of our testing efforts - how deep you can go while still knowing what breadth you have covered. Without this you can go down the rabbit hole and miss the bigger picture. Without control testing is pushed later and later, Controllability determines what scenarios we can exercise - whether it be setting test data to the right state or ensuring a dependency returns a specific response.