Personal view on applicability of Graph database technologies to the Transport industry, especially public railways in the UK

1. Rail Ticketing Assistance from the Graph Way
Andy Smale

2. 2
Agenda
KCOM and Me - Introduction
The challenges in transport
Solutions and why neo4j?

3. 3
Provides communications services and IT solutions to
organisations and consumers
£350m turnover, 1700 employees
Head Office: Hull, UK City of Culture 2017
Development Base: Ipswich, East Anglia

4. 4
Contact & Collaboration
Cloud
Development
Integration
AWS
Azure I&AM
API
Java Microsoft
UI/UX
Project
Management
Security
Business
Analysis
Testing
Digital
Databases
Data
Voice
Video
Architecture

5. 5
“A bit of a Geek”
Farmer in Devon
Got a Sinclair ZX80 kit. BASIC!
Developed in C, Pascal and Prolog
Student of Artificial Intelligence
Solution Architect
Work in Mobile and Transport Sectors
neo4j Certified Architect

6. 6
Transport ticketing - Functional Overview

7. 7
Transport Service Challenges
Multiple businesses
Thousands of locations
Flexible retailing of products
Fulfilled in different formats (paper, barcode, ITSO)
>30,000 Services per day, 1.7billion journeys/year
Layers of complexity
It even looks like a graph!

8. 8
API challenges
Flexible bookings
- File-based
- Existing APIs are fine-grained
- Aligned with entity/table
- Clients relate data

9. 9
Integration and APIs
Fast imports of file-based data feeds
Design new, richer APIs
• Unrestricted by underlying data models
• More “graph-friendly”
Adapt legacy APIs in service layer
Migrate clients to richer APIs

10. 10
Demand Management Best Product at the right Price
at the right Time
Capacity
DemandPrice
products
Yield
Management
products

11. 11
Demand Management Solutions
Nests, groups, hierarchies, rules… all configurable
• “Sweet spot” for graph technology
Decision Support System integration
Dynamic updates
• Ripple through the graph
• “What if…?” modelling
Secure, role-based access protects strategies

12. 12
Reporting
Real-time Operational Reports
Data exports
Scheduled reports
Ad hoc Reports
Data Analysis
Streaming Updates

13. 13
Reporting Solutions
Delivered in real time through published interfaces
Ability to monitor and update based on changes
Dedicated read replicas serving operational reports
Analytic tool integration
Visualisation tooling
Integration with AWS Kinesis for streaming of data

14. 14
Technical Architecture Challenges
100% Availability Targets
Fast Recovery from Failure
Resilient to Network Issues
Redundancy built in to Components
Automated build and deploy
Fast Performance (<50ms writes)
Consistently scalable for reads (<30ms, 10k TPS)
Thousands of Client systems

15. 15
Software Engineering Solution
Highly Relational Complex Data Model
Thousands of queries per second
Consistent Writes (hundreds/sec)
Security of Commercial Strategies (Yield)
Break the problem into bounded domains
Autonomous Software Components
Continuous Integration and Load Testing
Proactive Monitoring and Alerting

16. 16
Why do we choose Neo4J?
Data Model
Transactional
Scalable
Reliable
Highly Available
Consistent Fast Query Response
Enterprise Grade Support

17. 17
How did we validate Neo4J?
Building cluster (on v2.6)
Early Load Test
Simple data model
Created using Stored Procedure
Load injectors
10,000 TPS @ <3ms reads

18. 18
And What’s Next?
Full-Scale Automated Load Testing
• Adding demand management
Causal Clustering
• Write performance impact
Edge services
• Replication lag monitoring
Blue-Green Application Upgrades ->3.2
• Aim for zero downtime upgrades

19. Thank you
Contact
Andy Smale
E andrew.smale@kcom.com
T 01473 421421