Nauto devices are installed in fleets to help improve driving behavior as well as fleet managers to know who is driving well and who is not. One of the fundamental components around which everything revolves is the notion of “trips” -- a trip being the time when the vehicle started to when it came a full stop parked. Vehicle states such as moving and stopped are inferred from accelerometer data and send to the cloud servers over LTE connections sampled at very short intervals. These states are then combined in an online algorithm that builds trip segments, extending or combining them as and when we get more state updates. Further each trip segment records attributes such as the route and speed. In order to be able to do this at scale, we create, merge, and delete these trips as they grow in a time-series store on Scylla. The web servers directly serve these routes out of Scylla.

1. An online method for merging
time ranges on top of Scylla
Rohit Saboo
ML Engineering Lead, Nauto Inc.

2. Presenter bio
▪ Leading an ML engineering team at Nauto -- working on
various things such as finding trips, driver identification,
lossless sensor data compression.
▪ Previously a founding engineer at a search startup, and in
Google search and robotics.
▪ MS., Ph.D in Computer Science from UNC Chapel Hill, and
B.Tech, Computer Science, IIT Madras.
▪ I enjoy hiking, photography, swimming, and biking in my
spare time.

3. Nauto

4. ▪ Identifies risky driving using
machine learning
▪ Enables fleet managers to
correlate internal driver
behaviour with external vehicle
movements
▪ Coaching drivers for safer roads
Nauto

5. Trips and Identifying the Driver
▪ Record trip taken.
▪ Identify the driver taking the
trip.
Device takes
● snapshot & crops image
● GPS, speed ...
● vehicle state
N2
Upload

6. Merging Time Ranges
(Building Trips)

7. GPS
Moving/Stopped
Speed
Data Flow
N2
Kafka
worker 1
Sharded by
Device
worker i
worker n
Get
neighboring
ranges
Save merged
range
(Delete old)

8. 8:00 8:05 8:10 8:15 8:20 8:25 8:30 8:35 8:40 8:45 8:50 8:55 9:00 9:05
8:00am 8:12am 8:18am 8:35am 8:45am 8:55am
Time Ranges (Trips)

9. Time ranges (trips) table
CREATE TABLE device_trips (
version int,
id text,
bucket timestamp,
end_ms timestamp,
start_ms timestamp,
details blob,
PRIMARY KEY ((version, id, bucket), end, start)
) WITH CLUSTERING ORDER BY (end DESC, start DESC)

10. Time ranges (trips) table
CREATE TABLE device_trips (
version int,
id text,
bucket timestamp,
end_ms timestamp,
start_ms timestamp,
details blob,
PRIMARY KEY ((version, id, bucket), end, start)
) WITH CLUSTERING ORDER BY (end DESC, start DESC)
The id of the time series. In our case, the vehicle/device id.

11. Time ranges (trips) table
CREATE TABLE device_trips (
version int,
id text,
bucket timestamp,
end_ms timestamp,
start_ms timestamp,
details blob,
PRIMARY KEY ((version, id, bucket), end, start)
) WITH CLUSTERING ORDER BY (end DESC, start DESC)
For bucketing the time ranges into manageable partitions. Truncated from end_ms.

12. Time ranges (trips) table
CREATE TABLE device_trips (
version int,
id text,
bucket timestamp,
end_ms timestamp,
start_ms timestamp,
details blob,
PRIMARY KEY ((version, id, bucket), end, start)
) WITH CLUSTERING ORDER BY (end DESC, start DESC)
The start and end of the time range.

13. Time ranges (trips) table
CREATE TABLE device_trips (
version int,
id text,
bucket timestamp,
end_ms timestamp,
start_ms timestamp,
details blob,
PRIMARY KEY ((version, id, bucket), end, start)
) WITH CLUSTERING ORDER BY (end DESC, start DESC)
To simplify running multiple versions of the algorithms or experiments.

14. Time ranges (trips) table
CREATE TABLE device_trips (
version int,
id text,
bucket timestamp,
end_ms timestamp,
start_ms timestamp,
details blob,
PRIMARY KEY ((version, id, bucket), end, start)
) WITH CLUSTERING ORDER BY (end DESC, start DESC)
Application-specific.
For us, a marshaled protobuf containing gps, speed, etc. along the trip
(with json type now available, it may be preferable to use json.)

15. Time ranges (trips) table
CREATE TABLE device_trips (
version int,
id text,
bucket timestamp,
end_ms timestamp,
start_ms timestamp,
details blob,
PRIMARY KEY ((version, id, bucket), end, start)
) WITH CLUSTERING ORDER BY (end DESC, start DESC)
Partitioned by time series id and bucketed to manage partition size.
Clustering order chosen to make the common case of data arriving in-order optimal.

16. By utilizing a sharded worker pool for merging,
neither are there races nor locks are necessary.
As a result, there will (almost) always be at most
2 neighboring time ranges:
one before and one after

17. 8:00 8:05 8:10 8:15 8:20 8:25 8:30 8:35 8:40 8:45 8:50 8:55 9:00 9:05
8:00am 8:12am
Merging

18. 8:00 8:05 8:10 8:15 8:20 8:25 8:30 8:35 8:40 8:45 8:50 8:55 9:00 9:05
8:00am 8:12am
8:40am 8:55am
Merging
end_ms ≥ 8:35am start_ms ≤ 9:00am

19. 8:00 8:05 8:10 8:15 8:20 8:25 8:30 8:35 8:40 8:45 8:50 8:55 9:00 9:05
8:00am 8:12am
8:40am 8:55am
Merging
end_ms ≥ 8:35am start_ms ≤ 9:00am

20. 8:00 8:05 8:10 8:15 8:20 8:25 8:30 8:35 8:40 8:45 8:50 8:55 9:00 9:05
8:00am 8:12am 8:40am 8:55am
Merging

21. 8:00 8:05 8:10 8:15 8:20 8:25 8:30 8:35 8:40 8:45 8:50 8:55 9:00 9:05
8:00am 8:12am
8:15am 8:35am
8:40am 8:55am
Merging
end_ms ≥ 8:10am

22. 8:00 8:05 8:10 8:15 8:20 8:25 8:30 8:35 8:40 8:45 8:50 8:55 9:00 9:05
8:00am 8:12am 8:40am 8:55am
Merging
8:00am 8:55am

23. 8:00 8:05 8:10 8:15 8:20 8:25 8:30 8:35 8:40 8:45 8:50 8:55 9:00 9:05
Merging
8:00am 8:55am

24. 8:00am 8:55am
Merging
8:00 8:05 8:10 8:15 8:20 8:25 8:30 8:35 8:40 8:45 8:50 8:55 9:00 9:05

25. The Team
Rohit Saboo, Adam Sowinski, Christian Merkwirth
Yingyi Hu, Karol Kokoszka, Mykola Terelia
and many others

26. Thank You
Any Questions ?
Please stay in touch
rohit@nauto.com
@NAUTODriver