While high-resolution satellite imagery is the foundation of digital mapping, the demands of urban mobility require highly accurate, frequently updated data from a different vantage point – the ground. Recent advances in street-level imagery collection and data extraction are lifting up new trends in location-based services, smart cities, and autonomous vehicles. As map technologists push the boundaries of machine intelligence for extracting data from images, human collaboration will drive the creation of maps for mobility for all.

1. This is a street in Hong Kong, seen through the eyes of a machine, precisely distilling the image into segments that make sense: car, road, pedestrian, traffic light, crosswalk. This is called
semantic segmentation, and it’s today’s computer vision technology.
The Future of Maps for Mobility
November 2017 / Geography 2050
Janine Yoong
VP Business Development

2. Back in 1956, we imagined cars driving themselves along embedded electronic tracks on roads. We’ve done one better - we’ve instead started giving cars eyes - a broad array of
sensors - to see and translate what's around them into maps so they can drive themselves.
1956: 2017:
Image Credit: Boy’s Life 1956, The Economist.

3. But in future, how do we create maps from 700 trillion images captured per day? Furthermore, how do we create maps that are accurate, complete, and fresh, so that they’re effective at the very
moment your car is turning into a bike lane that wasn’t there yesterday?
3.8 billion cameras in vehicles by 2050 means
700 trillion images captured per day!
Source: LDV Capital: “Five Year Visual Technology Market Analysis”, August 2017.
Image Credit: Bladerunner 2049.

4. Computer Vision > Speed
New advances in computer vision drives automation in the processing workflow,
bringing maps closer to realtime
What’s brought us here is what will get us there: great strides in computer vision, combined with rapid improvements in cameras and GPS hardware. We all know the smartphone as a means to
consume maps, but it’s also a tool to create maps.

5. Imagine walking along holding your phone up, taking pictures continually - much like a sensor on a vehicle. The rectangular frames represents the position of the camera based on GPS data,
from matched points we can create a 3D reconstruction of the scene. This means that from imagery alone, we can tell you the location of that tree in the park.
Structure from Motion reconstructs locations in 3D

6. But how does the car know it’s a tree? How does it know it’s a park? From where we started in Hong Kong, we now go to Moscow, where we see again semantic segmentation in action.
Deep learning enables object recognition

7. The combination of these two processes - 3D reconstruction and semantic segmentation - gives us what we need to automatically generate map data, fast. These examples sidewalk density
and traffic signs in San Francisco, both generated within minutes of the images hitting the cloud.
An automated workflow for generating map data

8. To see a larger scale example of this, we go to Amsterdam. Last June, the City of Amsterdam released 800,000 hi-res 360 images of the whole city. We stitched these into dense a 3D point
cloud, with segmentations colored in.
Amsterdam: 2.2 billion 3D points

9. So now we have a wealth of information extracted from the imagery: from bikes and bridges to trash bins and traffic signs. This was all processed within days - not quite realtime, but that’s
where we are going with computer vision.
Amsterdam: Map data processed within days

10. Collaboration > Scale
In recent partnerships, the historically competitive mapping industry
acknowledges that covering the world is beyond any single player’s reach
This brings us to the second driver of new maps: collaboration. OpenStreetMap is the heart of collaborative mapping and we’re seeing its spirit spread beyond itself, as commercial maps and
automotive OEMs realize that the world is to big take on alone. But how do we get different sources and formats of imagery to play nice?

11. We believe that a sensor-agnostic approach is a difficult but worthy computer vision challenge. If we support any kind of sensor on any vehicle, or if we empower anyone, anywhere, with any
device, we’re essentially scaling up the effort to build the most complete map of the world.
A sensor-agnostic approach drives data sharing

12. Our proof that this can work is in our numbers: over 200 million images amassed on Mapillary from different sources today. To visually demonstrate our reach, let’s have a look at my favorite
one: a view from the Rothera Research Center in Antarctica.
216 million images
3.7 million km mapped
190 countries

13. But let’s bring this back to urban mobility. Here’s a look at the tool HERE Maps uses to review Mapillary imagery for map editing; on top of the data they
collect using their fleet of street view cars. Why?
HERE: Community supplements in-house data

14. …to get more data. When HERE maps uses Mapillary imagery, they get access to data contributed by others: cities using Esri’s GIS suite of products, OpenStreetMappers, and other commercial
mapping players.
HERE: Access to data contributed by others
OpenStreetMap

15. The relationship is two-way. Contributions from HERE’s community are shared back: take a look at this swell of data since we started
working with HERE in Astana, Kazakhstan. This data comes back to OpenStreetMap and other platforms to improve maps, everywhere.
HERE: Community contributions are shared back

16. Crowds > Sidewalks
Communities and individuals keep us focused on the need for maps for broader
urban mobility challenges
Maps everywhere should also mean maps for everyone. While all eyes are trained on news coming out of the automotive industry, the last point I’ll make is about how communities and
individuals drive us to address mobility challenges beyond cars.

17. At Mapillary, we’ve learned that individuals are motivated by specific passions and interests as they map their world: local pride, bike safety, infrastructure improvements.
Self-driving communities

18. The epitome of this passion are events like this pedestrian walk in Mexico City last year, where able bodies came together with people with limited mobility to traverse the streets in whe
Mexico City: Pedestrian Walk

19. Mexico City: Curb cuts, sidewalks, and crosswalks

20. Thank you!
@mapillary
www.mapillary.com
While I’m lucky to work in the company of those who push the boundaries of machine intelligence everyday, I’m constantly reminded that the future of mobility is in the hands and hearts of
humans creating effective and equitable maps together.