Transport For London is a huge organisation. I was part of their revamping efforts. There were many challenges with respect to agile implementation and complexity of the solution. This presentation will highlight some of those challenges and solutions we had in place to deal with them.

1. Transport For London
Keeping London on the move
!
Anand Ramdeo / @testinggeek

2. Inspiration
“Tester who helped TFL in
revamping their web
presence and presented his
work at one of the best
testing conference - Let’s
Test 2014”

3. Good to know
!
• 150+ years old
• 24 million Journeys every day
• 1.17 billion passenger
journeys in a year
• 75% Londoners use TFL
website regularly
• 250 million visits a year
!
“Lifeline of London”

4. Responsive design with better!
usability and consistent!
platform for developers.
Revamp

5. Journey!
Planner
Disruptions Lines
Station
Info
TimeTable
And lots of other Interesting and useful
services..
External developers who build
interesting apps based on the
data available from TFL.
Tight coupling, inconsistent
experience

6. Journey!
Planner
Disruptions
Station
Info
And lots of other Interesting and useful
services..
Consistent Restful Interface
External developers who
build interesting apps based
on the data available from
TFL.
Loosely coupled, RESTful!
interface
Timetable Lines

7. Bad agile, good waterfall
Bad Agile!
• No testers in the team
• Not enough unit tests
• No Continuous Integration
• Sprint and Scrum
Good Waterfall!
• Excellent user research
• Almost finalised design

8. Assignment
“You have got 21 days
to do automated
regression testing for
Journey Planner. Look
at the stories we have
already finished.”

9. Challenge #1
Journey
Planner
New, responsive web design
which consumes RESTful
interface

10. Context Revealing Questions
• Who are the stakeholders?
• What’s important for them?
• What is already tested by the team?
• When is it supposed to be
released?
• What would be my oracle?
• What could possibly go wrong?
• Do I have access to data,
environment, people who can help
me test?
http://www.developsense.com/blog/2010/11/context-free-questions-for-testing/

11. Clear Priorities
Key stake holder!
• Technical Architect
Key concerns!
• Quality of results, reliability.
• User experience, on web, mobile
& devices.
• Validate architecture &
deployment on cloud.
Scope!
• API isn’t important for beta release

12. Quality of
Results
How do I know if route is
right or wrong?

13. Simple Oracle
“Results should be
identical to the live
website.”

14. Process
1. Plan a journey on the existing
website (Source, destination,
time, date, mode, preference
etc.)
2. Note results (Journey time,
departure, arrival, options,
walking, escalators, earlier, later
etc.)
3. Plan same journey with same
options on the new website
4. Make a note of results
5. Compare

15. Key Problems!
!
• Different UI
• Too Many things to check
• Time consuming
• Error prone
• Tiring
• Difficult to scale
Automation!
for the !
Rescue

16. Journey j = new Journey(Victoria, Waterloo)!
!
routesFromExistingSite = JourneyPlanner.planJourneyOnOldSite(j)!
!
routesFromNewSite = JourneyPlanner.planJourneyOnNewSite(j)!
!
compareRoutes(routesFromExistingSite, routesFromNewSite)
Domain!
Objects
Abstraction Encapsulation Clean code
Robust, Reliable, Maintainable &
Useful automation!
http://www.testinggeek.com/a-software-tester-s-journey-from-manual-to-political

17. Progress so far..
• Who are my stakeholders?
• What’s important for them?
• How am I going to test?

18. Which routes do we need to
test?
• Popular journeys
• Which modes are missing?
• Expand coverage
• Bug bash

19. Challenge #2!
Status and Disruptions
• Realtime status of all the tube
lines.
• Information about disruptions
across all the modes.
• Not testable - could not inject
disruptions.
• Oracle - compare with the live
What’s the biggest
challenge?

20. Solution
Continuous monitoring using
Jenkins, PhantomJS and
GhostDriver.

21. Challenge #3!
Lines
Data Migration - Every bus, train, tram, river etc.

22. WaterlooNorthern
Bakerloo
Jubilee
WestministerJubilee
Circle
District
StanmoreJubilee
Stratford Jubilee
Central
DLR
London OG
Epping
Central
North
Acton Central
West
RulslipCentral
StockwellNorthern
Victoria
Brixton
Victoria
Green
park Victoria
Jubilee
Piccadilly
VictoriaVictoria
Circle
District

23. TFL Spider
• Start from one station
• Get all lines from this station
• Get first line and check - is data
present and correct?
• If yes, get list
of all stations
and mark this
line as
processed.!
• Add unique
stations in the
list of stations
to process.
• If no, record
missing data
for this line
and mark this
line as
processed.
Follow this for all lines on current
station and move to the next
station

24. Challenge #4!
Public API, Internal customer
• Assumed domain knowledge
• Team’s access to API
developers
• Lack of consistency
• No versioning and regard for
contract

25. PAPAS BE SFO!
DEED HELP GC!
DVLA PC

26. Functionality Consumability Maintainability
Paging Domain specific names Diagnostic
Authentication Examples Versioning
Parameters / Payloads Easy to learn Logging
Authorization Documentation Accessibility
Security
Hard to misuse Purpose
Behave Ease to use Consumer
Error handling Lead to readable code
Principle of least surprise
State
Filter Guessability
Order Consistency
http://www.testinggeek.com/testing-restful-webservices-or-api-testing-remember-papas-be-sfo-deed-help-gc-and-dvla-pc
API Testing Mnemonic!

27. Lesson Learned
Pro tips?

28. Pro Tip #1
Automation does not have to be binary.
Use tools in the best possible way to help you test.

29. Pro Tip #2
Break problem into manageable chunks.
Define layers, focus on one thing at a time.

30. Pro Tip #3
Be context driven
Question and do the right
things

31. Pro Tip #4
know your stakeholders.
What’s important for them
and why?

32. Pro Tip #5
Use automation as a safety net,
but be aware of its limitations.

33. Pro Tip #6
Be vocal
!
Blocked? Making progress?
Done something well?
!
Don’t assume people would
know.

34. Pro Tip #7
Find ways to make progress

35. Pro Tip #8
Embrace throwaway
automation and utility
scripts.

36. Pro Tip #9
Make testing visible
!
http://www.satisfice.com/presentations/dashboard.pdf

37. Pro Tip #10
Keep asking
Are we doing the right thing?
!
Remember
!
Projects unfold over time in
ways that are often not
predictable.

38. Thank You
Anand Ramdeo !
@testinggeek!
!
anand@testinggeek.com
!
TestingGeek.com
TestSpicer.com
PlannedDeparture.com
!