Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lets test2014 keepinglondononthemove


Published on

  • Be the first to comment

  • Be the first to like this

Lets test2014 keepinglondononthemove

  1. 1. Transport For London Keeping London on the move ! Anand Ramdeo / @testinggeek
  2. 2. Inspiration “Tester who helped TFL in revamping their web presence and presented his work at one of the best testing conference - Let’s Test 2014”
  3. 3. Good to know ! • 150+ years old • 24 million Journeys every day • 1.17 billion passenger journeys in a year • 75% Londoners use TFL website regularly • 250 million visits a year ! “Lifeline of London”
  4. 4. Responsive design with better! usability and consistent! platform for developers. Revamp
  5. 5. Journey! Planner Disruptions Lines Station Info TimeTable And lots of other Interesting and useful services.. External developers who build interesting apps based on the data available from TFL. Tight coupling, inconsistent experience
  6. 6. Journey! Planner Disruptions Station Info And lots of other Interesting and useful services.. Consistent Restful Interface External developers who build interesting apps based on the data available from TFL. Loosely coupled, RESTful! interface Timetable Lines
  7. 7. Bad agile, good waterfall Bad Agile! • No testers in the team • Not enough unit tests • No Continuous Integration • Sprint and Scrum Good Waterfall! • Excellent user research • Almost finalised design
  8. 8. Assignment “You have got 21 days to do automated regression testing for Journey Planner. Look at the stories we have already finished.”
  9. 9. Challenge #1 Journey Planner New, responsive web design which consumes RESTful interface
  10. 10. Context Revealing Questions • Who are the stakeholders? • What’s important for them? • What is already tested by the team? • When is it supposed to be released? • What would be my oracle? • What could possibly go wrong? • Do I have access to data, environment, people who can help me test?
  11. 11. Clear Priorities Key stake holder! • Technical Architect Key concerns! • Quality of results, reliability. • User experience, on web, mobile & devices. • Validate architecture & deployment on cloud. Scope! • API isn’t important for beta release
  12. 12. Quality of Results How do I know if route is right or wrong?
  13. 13. Simple Oracle “Results should be identical to the live website.”
  14. 14. Process 1. Plan a journey on the existing website (Source, destination, time, date, mode, preference etc.) 2. Note results (Journey time, departure, arrival, options, walking, escalators, earlier, later etc.) 3. Plan same journey with same options on the new website 4. Make a note of results 5. Compare
  15. 15. Key Problems! ! • Different UI • Too Many things to check • Time consuming • Error prone • Tiring • Difficult to scale Automation! for the ! Rescue
  16. 16. Journey j = new Journey(Victoria, Waterloo)! ! routesFromExistingSite = JourneyPlanner.planJourneyOnOldSite(j)! ! routesFromNewSite = JourneyPlanner.planJourneyOnNewSite(j)! ! compareRoutes(routesFromExistingSite, routesFromNewSite) Domain! Objects Abstraction Encapsulation Clean code Robust, Reliable, Maintainable & Useful automation!
  17. 17. Progress so far.. • Who are my stakeholders? • What’s important for them? • How am I going to test?
  18. 18. Which routes do we need to test? • Popular journeys • Which modes are missing? • Expand coverage • Bug bash
  19. 19. Challenge #2! Status and Disruptions • Realtime status of all the tube lines. • Information about disruptions across all the modes. • Not testable - could not inject disruptions. • Oracle - compare with the live What’s the biggest challenge?
  20. 20. Solution Continuous monitoring using Jenkins, PhantomJS and GhostDriver.
  21. 21. Challenge #3! Lines Data Migration - Every bus, train, tram, river etc.
  22. 22. WaterlooNorthern Bakerloo Jubilee WestministerJubilee Circle District StanmoreJubilee Stratford Jubilee Central DLR London OG Epping Central North Acton Central West RulslipCentral StockwellNorthern Victoria Brixton Victoria Green park Victoria Jubilee Piccadilly VictoriaVictoria Circle District
  23. 23. TFL Spider • Start from one station • Get all lines from this station • Get first line and check - is data present and correct? • If yes, get list of all stations and mark this line as processed.! • Add unique stations in the list of stations to process. • If no, record missing data for this line and mark this line as processed. Follow this for all lines on current station and move to the next station
  24. 24. Challenge #4! Public API, Internal customer • Assumed domain knowledge • Team’s access to API developers • Lack of consistency • No versioning and regard for contract
  26. 26. Functionality Consumability Maintainability Paging Domain specific names Diagnostic Authentication Examples Versioning Parameters / Payloads Easy to learn Logging Authorization Documentation Accessibility Security Hard to misuse Purpose Behave Ease to use Consumer Error handling Lead to readable code Principle of least surprise State Filter Guessability Order Consistency API Testing Mnemonic!
  27. 27. Lesson Learned Pro tips?
  28. 28. Pro Tip #1 Automation does not have to be binary. Use tools in the best possible way to help you test.
  29. 29. Pro Tip #2 Break problem into manageable chunks. Define layers, focus on one thing at a time.
  30. 30. Pro Tip #3 Be context driven Question and do the right things
  31. 31. Pro Tip #4 know your stakeholders. What’s important for them and why?
  32. 32. Pro Tip #5 Use automation as a safety net, but be aware of its limitations.
  33. 33. Pro Tip #6 Be vocal ! Blocked? Making progress? Done something well? ! Don’t assume people would know.
  34. 34. Pro Tip #7 Find ways to make progress
  35. 35. Pro Tip #8 Embrace throwaway automation and utility scripts.
  36. 36. Pro Tip #9 Make testing visible !
  37. 37. Pro Tip #10 Keep asking Are we doing the right thing? ! Remember ! Projects unfold over time in ways that are often not predictable.
  38. 38. Thank You Anand Ramdeo ! @testinggeek! ! ! !