Trainline is an online train tickets retailer for UK and Europe with 11 milion monthly visitors and more than 100,000 tickets sold every day. In this talk we will share how we are using Amazon Web Services for deploying data products using serverless architecture and providing analytics in real time. As a case study, we will present BusyBot, a special feature that helps you to locate train carriages most likely to have empty seats and more space, so that you can travel more comfortably.

1. ‡
Data Science at Trainline
for Smarter Journeys
London, 22/11/2016
@DataScienceFest
@TrainlineTalent

2. ‡
Outline
• A bit about Trainline.
• Cloud-based serverless architecture for Big Data.
• Case Study: BusyBot
• Other Case Studies
2
John Telford, Head of Data Architecture.
Leading the adoption of Big Data technology at Trainline. Manages a team of Data Engineers and
Database Administrators. Previously worked on Data Warehousing and Big Data at Channel 4.
Computer Science degree from Brunel University.
Twitter: @jtelford1
Marco Rossetti, Senior Data Scientist.
Leading personalisation initiatives, like providing context-aware personalised services, journey
recommendations, and tailored travel options. Previously worked on recommender systems for
researchers at Mendeley. He has a PhD in Computer Science from University of Milan-Bicocca.
Twitter: @ross85

3. ‡
Trainline - Smarter Journeys
Help our customers save,
• Time (no more queuing for tickets at station)
• Money (book early, find cheap tickets)
• Energy (remove complexity)
Headlines...
• We process more than £2.3 billion in ticket sales annually.
• 100,000 smarter journeys every single day.
• 44 train companies, across 24 European countries.
• ~400 employees (London, Edinburgh, Paris).
• More than 30m visits per month
• 1 ticket sold every three seconds
3
Trainline takeover of Kings X, Oct 2016.

4. ‡ 4

5. ‡ 5

6. ‡ 6

7. ‡
Bob's cloud laws
It’s cloud if…
1. It offers self provisioning.
2. It offers pay-as-you-go pricing.
3. It is, for all intents and purposes, infinitely scalable.
Thus, no need for support from the provider for set-up, no upfront payments for
licences or minimum term agreements, and no constraints on what I can do!
• Hosting is not cloud.
• BYO licensing is not cloud.
7

8. ‡
From servers... to serverless
8
Servers = Pets
Virtual Machines= Cattle
Containers & Serverless
= Herds
Trainline policy:
Use PaaS wherever possible,
Use Serverless wherever possible,
... so long as they are good enough.

9. ‡
Data Gateway
9

10. ‡
Data Platform
10

11. ‡
Lessons: Lambda
• Effortless scaling; we often have >
100 λs running at once.
• Warm-up time.
• Choose language / framework
carefully.
• Consequences of 'freeze'.
• Monitoring– single thread.
Google "Trainline Engineering Lambda"
11
Service Time
Distribution
Execution (ms)

12. ‡
Lessons: Kinesis Streams
• TCO is generally low.
• But... understand costs, related to capacity of stream (number & size of
messages), time-to-live, etc.
• Monitoring / alerting... CloudWatch is (probably) not enough.
• Compress & encrypt?
Google "AWS Overview of Security Processes"
12

13. ‡ 13
BusyBot

14. ‡
0% 10% 20% 30% 40% 50% 60% 70%
Delays
Overcrowding
Value for money
Toilet Facilities
Luggage Space
Availability of staff
Car Parking
Unhappy customers
Source : National Rail Passenger Survey (NRPS) 2015
14

15. ‡ 15

16. ‡ 16
Google "BusyBot overcrowding"

17. ‡
Busy Bot Discovery
Data from March to May - approx. 100k feedback from our Android
users.
17

18. ‡
Infrastructure – Data Gateway
Feedback
collection
Daily
Enrichment
{
"train_destination": "RDG",
"retail_train_number": "GW2980",
"train_origin": "NRC",
"train_date": "2016-08-08T07:38:00.000Z",
"customer_longitude": 0,
"train_hashid": "NRC:RDG:08/08/2016 08:38:00:GW2980",
"customer_location_on_train": "Back",
"customer_hashid": ”…",
"customer_got_seat": 1,
"customer_feedback": "Yes",
"feedback_type": 1,
"customer_latitude": 0,
"feedbackid": ”…",
"device_id": ”…",
"timestamp": "2016-08-08T07:41:39.390Z",
"customer_id": ”…”
} 18

19. ‡ 19

20. ‡ 20

21. ‡
ALL
feebacks
≧100
feedbacks
≧1000
feedbacks
City Thameslink: 50%
0% with a seat
100% with a seat
21

22. ‡
Infrastructure – Data Platform
Model Building
And Validation
Service
route-
origin
route-
destination stop
customer-location-
train
percentage-who-got-
seat
feedback-
count
EUS MAN EUS middle 0.738059701 4020
EUS BHM EUS middle 0.63788222 3532
KGX LDS KGX middle 0.704984154 3471
BHM EUS BHM middle 0.679082241 3356
KGX EDB KGX middle 0.5589236 3233
EUS GLC EUS middle 0.676663543 3201
MAN EUS MAN middle 0.769495772 3193
PAD SWA PAD middle 0.608086078 3067
EUS BHM EUS front 0.672365666 2866
EUS MAN EUS front 0.790479625 2773
{
"retailTrainIdentifier": "VT7280",
"isBusy": false,
"callingPoints": [
{
"stationCode": "EUS",
"coaches": [
{"position": "Back", "recommend": true},
{"position": "Front", "recommend": false},
{"position": "Middle", "recommend": false}
]
},
{
"stationCode": "MKC",
"coaches": [
{"position": "Back", "recommend": false},
…
22

23. ‡
• At least N feedbacks
• At least feedbacks for
D days
• CI on the percentage
who got a seat <= p
Data Validation
23

24. ‡
Journey Results
Live Tracker
BusyBot V1
Sep 2016
24

25. ‡ 25
Coming soon…

26. ‡
Hotels
26

27. ‡
Journey
Recommendations
27

28. ‡
Search
Prediction
28

29. ‡
Summary
BusyBot Hotels
Journey Recommendations
Search
Prediction
Delays
Prices
Real Time Information
Personalisation
….
29

30. ‡
Any Questions?
(we are hiring!)
Data Scientist positions: mali.mehmood@thetrainline.com
Data Engineer positions: david.smith@thetrainline.com
30