Transportation is undergoing as big a revolution as when the automobile was invented. Thanks to the convergence of constantly connected devices, advanced analytics, and artificial intelligence, this change incorporates some of the latest technologies and data science methods. This area is a huge opportunity with more than 600 companies and $17B in funding. In this Meetup we’ll take a look into the analytics technologies and data science techniques used in a couple of connected transportation applications.

1. The Data Science behind Predictive
Maintenance in Connected Vehicles
Esther Vasiete
Srivatsan Ramanujam
Pivotal Data Science
Data Engineers Guild - Meetup
June-21, 2016

2. Picture credit (from L to R):
http://www.techlicious.com/blog/ericsson-mobility-report-internet-connected-devices/
http://www.mdpi.com/1424-8220/14/10/19260/htm
http://www.thehindubusinessline.com/info-tech/other-gadgets/care-for-a-connected-car/article5777444.ece
Devices are Increasingly Connected

4. How can these connected devices in our home be smart enough to make daily
life easier?

5. How does this…
…become this?
By recognizing this
And by processing this
Sensors + Other Unstructured Data

6. How can we know a tree has
fallen on a power line before
the residents complain?

7. How can we use data
to help prevent
accidents like the Macondo Disaster ?

8. Gene Sequencing
Smart Grids
COST TO SEQUENCE
ONE GENOME
HAS FALLEN FROM
$100M IN 2001
TO $10K IN 2011
TO $1K IN 2014
READING SMART METERS
EVERY 15 MINUTES IS
3000X MORE
DATA INTENSIVE
Stock Market
Social Media
FACEBOOK UPLOADS
250 MILLION
PHOTOS EACH DAY
In all industries billions of data points represent
opportunities for the Internet of Things
Oil Exploration
Video Surveillance
OIL RIGS GENERATE
25000
DATA POINTS
PER SECOND
Medical Imaging
Mobile Sensors

9. To realize this opportunity requires the right tools and
techniques
Problem
Formulation
Modeling Step
Data StepApps Step

11. Data Lake
Ingest
Business Levers
Dashboard/App
PL/X
Modeling• Data cleaning
• Data Exploration
• Feature
Engineering
Model Validation
Feedback loop for
continuous
model improvement
Driver and
Vehicle Meta
Data
Data Ingestion
Platform
✔
✔ ✔ ✔ ✔✔ ✔ ✔
Data to Apps

12. Data Science
Use-cases for connected cars
1

13. Data Science Use-Cases
1
● Predictive Car Maintenance
‒ More accurately predict part failure
‒ Optimize part repair and replacement schedule
● Leveraging Driving Behaviour
‒ Useful to differentiate insurance pricing based on driving
style
‒ Optimize car design
● Improving GPS Systems
‒ Establish baseline for traffic congestion
‒ Create more meaningful metrics for routing
‒ Infer public transportation effects on traffic
‒ Predict how long incidents would take to clear
● Predictive Power for Assistance
Systems
‒ Optimize fuel efficiency
‒ Predict the future state of a car in the next 2
minutes (starts, stops, emergency braking)
● Traffic Light Assistance
‒ Signal timing of traffic lights
‒ Crowd sourcing of traffic signals
‒ Optimize traffic light patterns to reduce congestion

14. Preventive Maintenance for
Connected Vehicles
1

15. On-Board Diagnostics
Diagnostic Trouble Codes (DTC)
Unscheduled repairs
AB1029 – Power steering pump replacement
CT3408 – Wheel alignment

16. Solving the preventive maintenance problem
Automakers
Customer
Satisfaction
Auto Repairs

17. Data Sources for Predictive Maintenance
VIN
Timestamp
DTC Code
Odometer
Speed
Acceleration
Engine Temperature
Engine Torque GPS
Coordinates
etc.
VIN
Date vehicle in
Date vehicle out
Repair code
Parts replaced
Warranty claims
Repair Comments
Vehicle Sensor Data Vehicle Repairs Data

18. Predicting Job Type from Diagnostic Trouble Codes
(DTCs)
Time
Job Type:
Transmission
Job Type:
Transmission
Engine
Job Type:
Regular check
DTC: B DTC:
B,
P, C
DTC: U
DTC: B DTC: B
DTC:
B, P, C, U
DTC:
P, B, U
DTC: P DTC: B DTC:
B,P
DTC:
B,P
Can the DTCs
observed here predict
this Job Type?
Can the DTCs observed
here predict this Job
Type?
Can the DTCs observed
here predict this Job
Type?

19. Predicting Job Type: a multi-class classification
problem
DF
12
10
DF
12
15
DF
29
80
AB
10
29
AB
16
22
AB
16
25
AB
86
22
CT
34
02
CT
34
08
CT
35
60
CT
24
09
Vehicle
Features

20. Two-stage Hierarchical Classification Framework
Vehicle
Features
DF
12
10
DF
12
15
DF
29
80
AB
10
29
AB
16
22
AB
16
25
AB
86
22
CT
34
02
CT
34
08
CT
35
60
CT
24
09

21. Model Parallelism
One or more job on the same day
Multi-labeling problem
One-vs-rest classifiers
built in parallel
1
0
0
1
0 1
0
Class 1
Class 2
Class 3
One-vs-Rest Classification
Red vs.
Non Red
On Segment 1
Green vs.
Non Green
On Segment 2
Blue vs.
Non Blue
On Segment N

22. • Predictive maintenance problems are challenging because
DTC signals are not always symptomatic of an ensuing
repair.
• Given the hierarchical nature of repair codes, we built a two
stage hierarchical classification framework comprising a top-
down cascade of classifiers.
• Major system jobs can be predicted earlier to the repair
date.
Key Takeaways

23. Reference Architecture
%%publish
model info.
/
Microservices
(Spring Boot)
/load_model
/score_model
Spring Cloud Data Flow
vehicle data (streaming)
connector
exploratory data
analysis & model
training
Rabbit/Kafka
source
training (offline)
scoring (online)
/
web or mobile app dashboard