PoliMappers - Mapillary: Introduction guide

Mapillary: Introduction guide
Federica Gaspari
Responsible for Communication and Social Media - PoliMappers

Summary
Introduction
Computer Vision concepts
Mapillary website and application
Contribution
Installation and usage
Mapillary community and events
Final task
1/54

2/54
Introduction
 Mapillary is a collaborative street-level imagery platform
for extracting map data at scale using computer vision.
 The project was born in Malmö (Sweden) in September
2013.

3/54
Mapillary Manifesto
Introduction - The idea
At Mapillary we want to create an image
representation of the world, a map with
images of every place on Earth.
The general idea of the creators is to represent the entire world with a
street-level imagery using crowdsourcing combined with an
independent and systematic approach to cover interesting areas.
The data collected can be use to validate or update existing maps but
also can be integrated in navigation, delivery apps… and more!

4/54
Introduction - The context
Mapillary use
• Collaborative capturing
• Street-level perspective for map editing
• Urban mobility improvement
• Humanitarian mapping
• Disaster recovering mapping
• Urban landscape evolution tracking
• …

5/54
Introduction - The process
Capture
Get map data
Explore Upload

6/54
Introduction – The evolution
 On April 4th, 2019, the 500 millionth image has
been uploaded on the database.
 On December 17th, 2019, Mapillary reached the
number of one billion high-resolution images
hosted on its platform.

7/54
Similarities
They are both platform in which you can see street-level imagery
and navigate through it in a map.
Introduction – Comparison with Google Street View
Differences with Mapillary
• Crowd-sourced approach: users upload photos and the data is
freely available under CC-BY-SA;
• Data is available in an OSM environment;
• Mapillary scientists use data to improve existing data and to
generate new info with computer vision tools.
VS

8/54
On Mapillary computer vision is used to detect and
recognize map features on the images uploaded by the
users.
Computer Vision
Structure from Motion Image recognition

9/54
Computer Vision – Structure from Motion/1
From a sequence of photos, similar groups of pixel are clustered
in pixel clouds in order to create a 3D model of the environment;
The model is obtained only from photo sequences, without
using any other geodetic tool (LiDAR etc.);
This functionality is not available in the website at the moment
but it is for use of the Mapillary research staff only.

10/54
Courtesy of Christopher Beddow

11/54

12/54
Computer Vision – Image recognition/1
Face and car license plates recognition in order to blur them for
privacy. This tool has been introduced in 2015 with a gamified
approach involving the users in the training.
Traffic sign are automatically mapped from photos uploaded
(more than 900 signs in 60 countries) and text are recognized.
Semantic segmentation: groups of pixel are clustered in
different semantic classes as «human», «car», «street», «sky»,
«plant»… with many different subclasses and labels.

13/54
Semantic segmentation
Mapillary processes photos on its platform to understand the
scene and a variety of objects within it at pixel level.
This has been achieved using deep learning which teaches the
computer to identify scenes and objects, by comparing them to
existing scenes and objects the computer has been shown.

14/54
Semantic segmentation

15/54
Courtesy of Christopher Beddow

16/54

17/54

18/54
Website/1
https://www.mapillary.com/
From the home page of the Mapillary is possible to access the web app showing
all the data collected up to now.

19/54
Website/2
On the global map view, all the images uploaded by users on Mapillary
database are highlighted with green dots. With the search symbol is possible to
zoom on a specified location.

20/54
Website/3
Zooming on a specific place, it is already possible to roughly evaluate the level
of completeness by looking at the density of green dots and presence of lines
connecting them. Every photo sequences uploaded by users is identified by
green dots and lines.

21/54
Website/3
On the right side, expanding the profile section, every user can highlight his/her
personal coverage on the area visible on the map.

22/54
Website/4
If you are looking for the contribution pattern of a specific user on a defined
time range, it is possible to set a filter to the main view.

23/54
Website/5
If you are looking for specific map data, you can query the AI specifying the
object you’re interested in. On the map all the correspondent features
automatically detected by the AI will be highlighted.

24/54
Website/6
Clicking on a specific photo where the AI detected the requested object, it is
possible to see how the image recognition process work.

25/54
Website/7
Another option that can be tested from the AI menu is the traffic signs
detection. It highlights the position of all the sign identified and their
symbology.

Contribution
26/54
3 possible configurations
Smartphone
+
Mapillary application
1 device tracking GPS
while collecting photos
Camera
+
GPS track recorder
The photo sequence file
can be downloaded once
the survey is finished and
then uploaded directly on
the Mapillary website
Most precise but complex
option!

Contribution – Mobile application
27/54
The Mapillary application is the easiest way to contribute.
With the app, available for both iOS and Android devices, it is
possible to capture images while walking, cycling or driving.
For the last two cases, the Mapillary organization sends free
car/bike mount to people interested in the project.
Mount request info

28/54
 Install Mapillary
Application from
App Store on your
iOS mobile phone or from
Google Play on your
Android device.

29/54
Installation and usage – Log in
iOS Android
When you first open the app, you are required to sign into your Mapillary through
your email, Facebook, Google or OpenStreetMap account.

30/54
Main menu – Explore view
Moving between photos
of an existing sequence
User who
uploaded the
picture

31/54
Main menu – Profile view
In this section it is possible to see all the all the info and stats of your profile and the
contribution history.

32/54
Main menu – Other sections
The Mapillary
Marketplace is
where
organizations list
mapping requests
that any
contributor can
help fulfill.
In the Upload page
will be listed all the
photos captured
during the survey
activity.

33/54
Capturing mode/iOS
Clicking the Camera icon on the main menu
you enter the photo collecting mode.
By default, the view is switched to landscape
orientation.
You can start a collecting sequence clicking
the start button.

34/54
Capturing mode/Android
Clicking the Camera icon on the main menu
you enter the photo collecting mode.
By default, the view is switched to landscape
orientation.
You can start a collecting sequence clicking
the start button.

35/54
Automatic capture mode/iOS
The default settings use the automatic
mode that constantly collects a new
picture automatically after you’ve
moved a certain distance (based on the
phone’s GPS). This type of capture mode
is suggested for walking, cycling and
driving activities.
If you tap on the map symbol, you can
see your actual position on the map and
the number of photos already taken.
To stop the photo sequence, click again
on the red round button.

36/54
Automatic capture mode distance or time based/iOS
By default, the automatic mode is
distance-based, taking autonomously
pictures every 5 meters.
Clicking on the gear icon, it is possible to
change the default option defining a
different distance between captures or
switching to time-based automatic
mode that helps capturing images by a
defined time range.
Once defined, save the settings by
clicking Done.

37/54
Automatic capture mode/Android
The default settings use the automatic
mode that constantly collects a new
picture automatically after you’ve
moved a certain distance (based on the
phone’s GPS). This type of capture mode
is suggested for walking, cycling and
driving activities.
If you tap on the map symbol, you can
see your actual position on the map and
the number of photos already taken.
To stop the photo sequence, click again
on the round button.

38/54
Automatic capture mode distance or time based/Android
By default, the automatic mode is
distance-based, taking autonomously
pictures every 5 meters.
Clicking on the gear icon, it is possible to
change the default option defining a
different distance between captures or
switching to time-based automatic
mode that helps capturing images by a
defined time range.

39/54
Advanced mode/iOS
In the advanced settings info and stats
about the capture session are
highlighted.
From this section it is possible to set the
manual capture mode and also define a
fixed camera direction if the phone’s
compass option is not required.

40/54
Manual capture mode/iOS
With the manual mode you have to click
the round button every time you want
to collect a new image.
In this view there is an additional icon
that has to be clicked every time you
want to collect a new collecting
sequence.
This mode is suggested to capture
panoramas, specific objects, facades,
corners and intersections.

41/54
Advanced and manual mode/Android
With the Android application it is
possible to switch to the manual mode
directly from the main capturing view
changing the settings on the upper right
corner.

42/54
Practical suggestion for setup and capturing
Before starting, be sure to have enough memory space on your device.
Also, pay attention to the phone battery level.
Choose carefully which direction you want to map (forward, backwards,
sideways, …).
Once you’ve started collecting pictures, be sure that your device is
properly aligned (on the app you can always make a comparison with the
line of the horizon);
In the capture mode, you may also see an indicator of GPS status when
the quality is not good enough. It will appear red when the signal does not
assure a sufficient quality level.

43/54
Practical suggestions before uploading
Remove bad quality images (shaken, blurred, not horizontal at all…).
Remove duplicates (do not need to upload 30 photos of the same
semaphore light).
Considering that the process implies an important amount of high
resolution images, think about using a Wi-Fi connection to upload
data after the survey.

44/54
Installation and usage – Upload/1
After the collecting activity, on the uploads section you can see listed all your photo
sequences. Tap any of them to see the images. In the single image view you can
change the orientation of the image or delete or save it directly in your phone.
iOS

45/54
After the collecting activity, on the uploads section you can see listed all your photo
sequences. Tap any of them to see thumbnails of the images. If you want to
remove a picture, you have to click on the trash bin icon.
Android

46/54
Alternatively, you can preview the sequence on the map. Tap on the Select option to
be able to select multiple images that you want to save on your camera roll, delete, or
approve for upload. Once you’ve reviewed the sequence, you can upload the images
clicking the button with the same name.
iOS

47/54
After reviewing all the collected pictures, you can finally upload them. To do that
click on the green round button on lower-right corner and then confirm the upload.
Android

48/54
Practical suggestions after uploading
Double check if positions and angles are correct in all the photos from
your sequences.
Double check if sensible information (faces and license plates) has
been automatically detected and blurred.
The images will be processed after completing the uploading. Every
contribution is combined together with other images from the area, faces
and license plates blurred. The application sends a notification as soon as
the uploads are on the map.
After checking the presence of your picture sequences on the map:

49/54
Mapillary community
The success of Mapillary is based on a constantly growing audience of
passionate contributors from all over the world. Every local
community is connected to the global one through events and
challenges periodically organised.
Mapillary also supports a range of OSM initiatives including State of
the Map conferences, Maptime meetups and humanitarian projects.
Through a partnership with Humanitarian OSM, Mapillary provides
equipment and technical support for projects that are actively
contributing OSM.

50/54
Mapillary events
Photo walk ChallengesMapathon
PoliMappersintheMountains
(MonteBoletto,Brunate)
#CompleteTheMap
(Milano)

51/54
Final task
Piacenza on Mapillary… now

52/54
Final task – Useful tips for the field activity
 Use the manual capture mode to capture details visible on the building
facades (for ex. shop name or accessibility).
This mode is strongly recommended to map specific characteristics of
shops  Photos of the facades of all the shops are mandatory for the task!
 Use the default automatic capture mode for street frontal view in order to
improve the Mapillary mapping of Piacenza. This capture modality could be
helpful to define some tags values in case of ambiguities derived from the
manual collected images  Street imagery of all the streets is highly
recommended!
 Keep in mind the area assigned to your group, plane carefully your survey,
pick your equipment and go out exploring and capturing pictures!
N.B.: The final evaluation of your work will depend on the level of
completeness and quality of your collection!

Bibliography
• Mapillary website: https://www.mapillary.com/ (Last check online: May 16th 2020)
• Stucchi L., Montani M.(2019). “Mapillary training - PoliMappers” PoliMappers Meeting,
Politecnico di Milano – Città Studi campus
53/54
Useful links
• Mapillary 4 Edu
• Mapillary Research

Federica Gaspari
Responsible for Communication and Social Media
PoliMappers
federica.gaspari@mail.polimi.it
Contacts
Politecnico di Milano | Department of Civil and
Environmental Engineering

PoliMappers - Mapillary: Introduction guide

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to PoliMappers - Mapillary: Introduction guide

Similar to PoliMappers - Mapillary: Introduction guide (20)

Recently uploaded

Recently uploaded (20)

PoliMappers - Mapillary: Introduction guide

Editor's Notes