This document describes Chris Whong's project to visualize a day in the life of a NYC taxi using open taxi trip and fare data from the NYC Taxi and Limousine Commission (TLC). It outlines how Whong obtained the raw data, processed it to extract individual taxi trips and convert them to lines on a map, stored the results in a SQLite database, and built a web application using Leaflet, D3, and other tools to visualize a single taxi's movements over 24 hours along with earnings and passenger information. The project aims to better understand taxi operations and showcase the potential of open civic data.

1. NYC Taxis: A Day in the Life
A Data Visualization/Civic Hack by Chris Whong
FutureStack
October 8, 2014 @chris_whong

2. A bit about me…
Data Solutions Architect
Socrata, Inc.
Democratizing Public Data
Urban Planner
Coder

3. Urbanist,
Mapmaker,
Data Junkie

4. OPEN CIVIC
My
Favorit
e
Data
GEOSPATIAL

5. It all started with a tweet…

10. “…you will need to send or bring an
external hard drive with a minimum
capacity of 200 GB to the TLC
offices. The address is listed below.
The hard drive must be brand new,
still in the box and unopened. “

12. Data!

13. Fare Data – 12 CSVs (~2GB
each)
• Payment Type
• Fare
• Tax
• Tip (Credit Card Only)
• Tolls
Trip Data – 12 CSVs (~2GB each)
• Medallion
• Driver
• Pickup Time and Location
• Dropoff Time and Location
173 Million Trips in 2013!

16. • Shared via Torrent and Direct Download
• Urban Data Nerds Rejoice!
• Lots of Interesting Projects and Analysis

19. Driver and Vehicle De - Anonymization
hash = md5(medallionNumber)
Medallion: 6B111958A39B24140C973B262EA9FEA5
Hack License: D3B035A03C8A34DA17488129DA581EE7
There are only 19 million possible values!

20. Passenger Identification
“Riding with the Stars” by Anthony Tockar

21. But what will I do with the data?
I love:
• Data Visualization
• Mapping
• Transportation
• Civic Hacking
• Node.js
• D3.js
• Learning how cities work
SpatioTemporal Data is PERFECT for map animations!

22. Burning Questions:
Where do cabs go?
How much $ to they make?
How much time do they spend empty/full?

23. The Vision:
Pick a single Taxi from the data,
follow it for 24 hours on a map.
Repeat.

24. 1) Get Raw Data
(Isolate a collection of full “cab/days”)
Figure it out myself : HARD!
Ask the internets for help: EASY!
+

26. 2) Turn each trip into a line :
Directions API
Start and End Points IN
Directions with encoded
polylines OUT
Raw Data (Trips & Fares) Node.js Script Trips + Polylines

27. 3) Store the Results in SQLite

28. Getting a Day’s Trips into the
Browser
Backend (heroku):
SQLite Database Absurdly simple node server
/trip
1 day’s trips as geoJSON API
• HTML/CSS/JQuery
• D3.js
• Leaflet.js
• Mapbox
• Moment.js
• Bootstrap
Frontend:

29. GeoJSON in Leaflet via D3

30. GOT DATA.
Now how to visualize it?

33. Follow the Bouncing Ball:

34. Recursive SetTimeout( ) Counts Minutes:
Correct time at Start of Each Trip:

35. Empty/Full Chart w/ Cumulative Earnings

36. Todo:
• Allow pause & inspect
• Show driver shifts & totals
• Show miles traveled
• Add a non-cumulative column chart for
fares

40. Thanks for sponsoring!
&

41. A Few More Examples…

42. Tech
Chops
Design
Eye
DataViz
!
Vision &
StoryTelling

43. Thanks!
@chris_whong
https://github.com/chriswhong
http://chriswhong.com
Support Open Data!