SlideShare a Scribd company logo
© Copyright 2016 Jawg 1
Tiles under heavy
Artillery
Loïc Ortola
http://jawg.io
© Copyright 2016 Jawg 2
Maps before
Maps have changed
© Copyright 2016 Jawg 3
Maps today
© Copyright 2016 Jawg 4
Is OpenStreetMap
business-grade?
© Copyright 2016 Jawg 5
What does business-grade mean?
Efficient Scalable Available
Elastic Support
© Copyright 2016 Jawg 6
OpenStreetMap
• Community-driven • Experienced• Independent
© Copyright 2016 Jawg 7
https://github.com/openstreetmap/
osm2pgsql/blob/master/README
.md
http://wiki.openstreetmap.org/wiki/Osmosis/Replication
OpenStreetMap… map servers?
© Copyright 2016 Jawg 8
Entry cost to OpenStreetMap
© Copyright 2016 Jawg 9
OpenStreetMap for business
1. Introduction
2. The market of maps
3. Map services
1. Definition
2. Geocoding
3. Routing
4. Tiles
5. GIS
4. Constraints / architecture
5. Case study : mapping platforms
1. Primitives
2. KPIs
3. Bottlenecks
4. Market Solutions
6. Analysis
1. Scenario
2. Documentation
3. Support services
4. Scalability
5. Service-discovery compatibility
6. Cloud integration
7. Assuming people are paying attention
6. I am pretty sure No one will ever read this line
7. If you did though, I guess I owe you a beer
8. A real beer. You know, since we’re in Brussels…
8. It’s been a long day don’t you think?
6. About that beer… My twitter is @loicortola
9. Results
10. I - need - pizza
11. Raw data
12. Conclusion
© Copyright 2016 Jawg 10
OpenStreetMap for business
1. Introduction
2. The market of maps
3. Map services
1. Definition
2. Geocoding
3. Routing
4. Tiles
5. GIS
4. Constraints / architecture
5. Case study : mapping platforms
1. Primitives
2. KPIs
3. Bottlenecks
4. Market Solutions
6. Analysis
1. Scenario
2. Documentation
3. Support services
4. Scalability
5. Service-discovery compatibility
6. Cloud integration
7. Assuming people are paying attention
6. I am pretty sure No one will ever read this line
7. If you did though, I guess I owe you a beer
8. A real beer. You know, since we’re in Brussels…
8. It’s been a long day don’t you think?
6. About that beer… My twitter is @loicortola
9. Results
10. I - need - pizza
11. Raw data
12. Conclusion
© Copyright 2016 Jawg 11
Can OSM services sustain a
worldwide zombie infection?
© Copyright 2016 Jawg 12
The Zombie Pandemic
Virus = ZOMBIES Most services down
© Copyright 2016 Jawg 13
The Zombie Pandemic
• OSM app launched: WAZE
World
Against the
Zombie
Epidemic
• Localizes Safe Zones and resources
© Copyright 2016 Jawg 14
The Zombie Pandemic Scenario
200,000
potential users
@ 8pm local
release
1.12 billion
16% of
population
uninfected
1.12 million
0.1% of
remaining pop.
w/ access
© Copyright 2016 Jawg 15
The Zombie Pandemic Scenario
Main.mp4
© Copyright 2016 Jawg 16
Load testing tool
• Cluster mode
• Proper metrics
• Live reporting
© Copyright 2016 Jawg 17
Architecture
Injectors
Caches
Load-balancer
Renderers
CPU: 32RAM: 120GB
BW: 2Gb/s
Software: mod_tile, renderd, postgresql
CPU: 8RAM: 30 GB
BW: 2 Gb/s
Software: squid3
CPU: 8RAM: 30 GB
BW: 2 Gb/s
Software: HAProxy
CPU: 16RAM: 60 GB
BW: 2 Gb/s
Software: Gatling
Internet
© Copyright 2016 Jawg 18
© Copyright 2016 Jawg 19
YES
Objective: 1 billion maps
Reached: 56 billion maps
WAZE successfully saved the planet
Can OSM services sustain a
worldwide zombie infection?
© Copyright 2016 Jawg 20
Load testsNew bottlenecks
In reality
Tuning storm
• > 50 Gbps BW
• Saturation of Cloud’s
internal switch
infrastructures
• Kernel
• Configuration
• Load-balancing
+50 load tests to find
the most adequate
simulation
© Copyright 2016 Jawg 21
The Bottleneck Podium
© Copyright 2016 Jawg 22
The State Of The Map (Server)
Efficient Scalable Available
Elastic Support
© Copyright 2016 Jawg 23
White Papers
1. Map services: from theory to implementation
• Available now @ jawg.io
2. Map services: Benchmarks & high-scale profiles
• To be released in November 2016
© Copyright 2016 Jawg 24
Thank You
Loïc Ortola
@LoicOrtola
@jawgio

More Related Content

Similar to Tiles Under Heavy Artillery - Serving One Billion Maps Per Month

Philippine Geospatial Forum Presentation 20130311
Philippine Geospatial Forum Presentation 20130311Philippine Geospatial Forum Presentation 20130311
Philippine Geospatial Forum Presentation 20130311
esambale
 
OpenPOWER Overview - August 2016
OpenPOWER Overview - August 2016OpenPOWER Overview - August 2016
OpenPOWER Overview - August 2016
Mandie Quartly
 
"The OpenCV Open Source Computer Vision Library: What’s New and What’s Coming...
"The OpenCV Open Source Computer Vision Library: What’s New and What’s Coming..."The OpenCV Open Source Computer Vision Library: What’s New and What’s Coming...
"The OpenCV Open Source Computer Vision Library: What’s New and What’s Coming...
Edge AI and Vision Alliance
 
Lesson1 esa summer_school_brovelli
Lesson1 esa summer_school_brovelliLesson1 esa summer_school_brovelli
Lesson1 esa summer_school_brovelli
Maria Antonia Brovelli
 
Introduction to OpenStreetMap and Humanitarian OSM Team for Plan Internationa...
Introduction to OpenStreetMap and Humanitarian OSM Team for Plan Internationa...Introduction to OpenStreetMap and Humanitarian OSM Team for Plan Internationa...
Introduction to OpenStreetMap and Humanitarian OSM Team for Plan Internationa...
chippy
 
FreeGIS.net, INSPIRE, Open Source Software and OGC standards
FreeGIS.net, INSPIRE, Open Source Software and OGC standardsFreeGIS.net, INSPIRE, Open Source Software and OGC standards
FreeGIS.net, INSPIRE, Open Source Software and OGC standards
Arnulf Christl
 
Open streetmapによる鳥取ガイドの試み3
Open streetmapによる鳥取ガイドの試み3Open streetmapによる鳥取ガイドの試み3
Open streetmapによる鳥取ガイドの試み3
Hiroyuki Nakaji
 
Status of WorldMap, 2016
Status of WorldMap, 2016Status of WorldMap, 2016
Status of WorldMap, 2016
Paolo Corti
 
Sahana Disaster Management System
Sahana Disaster Management SystemSahana Disaster Management System
Sahana Disaster Management System
Biplov Bhandari
 
Interoperability and Standards for Disaster Risk Management
Interoperability and Standards for Disaster Risk ManagementInteroperability and Standards for Disaster Risk Management
Interoperability and Standards for Disaster Risk Management
Luis Bermudez
 
OSGeo LARS march 2016 Chile
OSGeo LARS march 2016 ChileOSGeo LARS march 2016 Chile
OSGeo LARS march 2016 Chile
Dirk Frigne
 
20130606 ISTS mapconcierge "Space/Geo x Innovation"
20130606 ISTS mapconcierge "Space/Geo x Innovation"20130606 ISTS mapconcierge "Space/Geo x Innovation"
20130606 ISTS mapconcierge "Space/Geo x Innovation"
Taichi Furuhashi
 
OSGi IoT Demo @ CeBIT 2016
OSGi IoT Demo @ CeBIT 2016OSGi IoT Demo @ CeBIT 2016
OSGi IoT Demo @ CeBIT 2016
mfrancis
 
DSD-INT 2016 A crowd-sourced spatial database can change the way we work - Va...
DSD-INT 2016 A crowd-sourced spatial database can change the way we work - Va...DSD-INT 2016 A crowd-sourced spatial database can change the way we work - Va...
DSD-INT 2016 A crowd-sourced spatial database can change the way we work - Va...
Deltares
 
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
Mesosphere Inc.
 
Activities Of The Foss4 G20080122
Activities Of The Foss4 G20080122Activities Of The Foss4 G20080122
Activities Of The Foss4 G20080122
Toru Mori
 
NASA Web World Wind: welcome to the new era of virtual globes
NASA Web World Wind: welcome to the new era of virtual globes NASA Web World Wind: welcome to the new era of virtual globes
NASA Web World Wind: welcome to the new era of virtual globes
Maria Antonia Brovelli
 
No one puts java in the container
No one puts java in the containerNo one puts java in the container
No one puts java in the container
kensipe
 
Progressing JavaScript and Apps the Web way…
 Progressing JavaScript and Apps the Web way…  Progressing JavaScript and Apps the Web way…
Progressing JavaScript and Apps the Web way…
Christian Heilmann
 
Fun and education with the PolarSys Rover and PolarSys Solutions
Fun and education with the PolarSys Rover and PolarSys SolutionsFun and education with the PolarSys Rover and PolarSys Solutions
Fun and education with the PolarSys Rover and PolarSys Solutions
Gaël Blondelle
 

Similar to Tiles Under Heavy Artillery - Serving One Billion Maps Per Month (20)

Philippine Geospatial Forum Presentation 20130311
Philippine Geospatial Forum Presentation 20130311Philippine Geospatial Forum Presentation 20130311
Philippine Geospatial Forum Presentation 20130311
 
OpenPOWER Overview - August 2016
OpenPOWER Overview - August 2016OpenPOWER Overview - August 2016
OpenPOWER Overview - August 2016
 
"The OpenCV Open Source Computer Vision Library: What’s New and What’s Coming...
"The OpenCV Open Source Computer Vision Library: What’s New and What’s Coming..."The OpenCV Open Source Computer Vision Library: What’s New and What’s Coming...
"The OpenCV Open Source Computer Vision Library: What’s New and What’s Coming...
 
Lesson1 esa summer_school_brovelli
Lesson1 esa summer_school_brovelliLesson1 esa summer_school_brovelli
Lesson1 esa summer_school_brovelli
 
Introduction to OpenStreetMap and Humanitarian OSM Team for Plan Internationa...
Introduction to OpenStreetMap and Humanitarian OSM Team for Plan Internationa...Introduction to OpenStreetMap and Humanitarian OSM Team for Plan Internationa...
Introduction to OpenStreetMap and Humanitarian OSM Team for Plan Internationa...
 
FreeGIS.net, INSPIRE, Open Source Software and OGC standards
FreeGIS.net, INSPIRE, Open Source Software and OGC standardsFreeGIS.net, INSPIRE, Open Source Software and OGC standards
FreeGIS.net, INSPIRE, Open Source Software and OGC standards
 
Open streetmapによる鳥取ガイドの試み3
Open streetmapによる鳥取ガイドの試み3Open streetmapによる鳥取ガイドの試み3
Open streetmapによる鳥取ガイドの試み3
 
Status of WorldMap, 2016
Status of WorldMap, 2016Status of WorldMap, 2016
Status of WorldMap, 2016
 
Sahana Disaster Management System
Sahana Disaster Management SystemSahana Disaster Management System
Sahana Disaster Management System
 
Interoperability and Standards for Disaster Risk Management
Interoperability and Standards for Disaster Risk ManagementInteroperability and Standards for Disaster Risk Management
Interoperability and Standards for Disaster Risk Management
 
OSGeo LARS march 2016 Chile
OSGeo LARS march 2016 ChileOSGeo LARS march 2016 Chile
OSGeo LARS march 2016 Chile
 
20130606 ISTS mapconcierge "Space/Geo x Innovation"
20130606 ISTS mapconcierge "Space/Geo x Innovation"20130606 ISTS mapconcierge "Space/Geo x Innovation"
20130606 ISTS mapconcierge "Space/Geo x Innovation"
 
OSGi IoT Demo @ CeBIT 2016
OSGi IoT Demo @ CeBIT 2016OSGi IoT Demo @ CeBIT 2016
OSGi IoT Demo @ CeBIT 2016
 
DSD-INT 2016 A crowd-sourced spatial database can change the way we work - Va...
DSD-INT 2016 A crowd-sourced spatial database can change the way we work - Va...DSD-INT 2016 A crowd-sourced spatial database can change the way we work - Va...
DSD-INT 2016 A crowd-sourced spatial database can change the way we work - Va...
 
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
Mesosphere & Magnetic: Take the pain out of running complex and critical serv...
 
Activities Of The Foss4 G20080122
Activities Of The Foss4 G20080122Activities Of The Foss4 G20080122
Activities Of The Foss4 G20080122
 
NASA Web World Wind: welcome to the new era of virtual globes
NASA Web World Wind: welcome to the new era of virtual globes NASA Web World Wind: welcome to the new era of virtual globes
NASA Web World Wind: welcome to the new era of virtual globes
 
No one puts java in the container
No one puts java in the containerNo one puts java in the container
No one puts java in the container
 
Progressing JavaScript and Apps the Web way…
 Progressing JavaScript and Apps the Web way…  Progressing JavaScript and Apps the Web way…
Progressing JavaScript and Apps the Web way…
 
Fun and education with the PolarSys Rover and PolarSys Solutions
Fun and education with the PolarSys Rover and PolarSys SolutionsFun and education with the PolarSys Rover and PolarSys Solutions
Fun and education with the PolarSys Rover and PolarSys Solutions
 

Recently uploaded

Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
saastr
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
c5vrf27qcz
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Neo4j
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
Edge AI and Vision Alliance
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 

Recently uploaded (20)

Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Y-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PPY-Combinator seed pitch deck template PP
Y-Combinator seed pitch deck template PP
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 

Tiles Under Heavy Artillery - Serving One Billion Maps Per Month

  • 1. © Copyright 2016 Jawg 1 Tiles under heavy Artillery Loïc Ortola http://jawg.io
  • 2. © Copyright 2016 Jawg 2 Maps before Maps have changed
  • 3. © Copyright 2016 Jawg 3 Maps today
  • 4. © Copyright 2016 Jawg 4 Is OpenStreetMap business-grade?
  • 5. © Copyright 2016 Jawg 5 What does business-grade mean? Efficient Scalable Available Elastic Support
  • 6. © Copyright 2016 Jawg 6 OpenStreetMap • Community-driven • Experienced• Independent
  • 7. © Copyright 2016 Jawg 7 https://github.com/openstreetmap/ osm2pgsql/blob/master/README .md http://wiki.openstreetmap.org/wiki/Osmosis/Replication OpenStreetMap… map servers?
  • 8. © Copyright 2016 Jawg 8 Entry cost to OpenStreetMap
  • 9. © Copyright 2016 Jawg 9 OpenStreetMap for business 1. Introduction 2. The market of maps 3. Map services 1. Definition 2. Geocoding 3. Routing 4. Tiles 5. GIS 4. Constraints / architecture 5. Case study : mapping platforms 1. Primitives 2. KPIs 3. Bottlenecks 4. Market Solutions 6. Analysis 1. Scenario 2. Documentation 3. Support services 4. Scalability 5. Service-discovery compatibility 6. Cloud integration 7. Assuming people are paying attention 6. I am pretty sure No one will ever read this line 7. If you did though, I guess I owe you a beer 8. A real beer. You know, since we’re in Brussels… 8. It’s been a long day don’t you think? 6. About that beer… My twitter is @loicortola 9. Results 10. I - need - pizza 11. Raw data 12. Conclusion
  • 10. © Copyright 2016 Jawg 10 OpenStreetMap for business 1. Introduction 2. The market of maps 3. Map services 1. Definition 2. Geocoding 3. Routing 4. Tiles 5. GIS 4. Constraints / architecture 5. Case study : mapping platforms 1. Primitives 2. KPIs 3. Bottlenecks 4. Market Solutions 6. Analysis 1. Scenario 2. Documentation 3. Support services 4. Scalability 5. Service-discovery compatibility 6. Cloud integration 7. Assuming people are paying attention 6. I am pretty sure No one will ever read this line 7. If you did though, I guess I owe you a beer 8. A real beer. You know, since we’re in Brussels… 8. It’s been a long day don’t you think? 6. About that beer… My twitter is @loicortola 9. Results 10. I - need - pizza 11. Raw data 12. Conclusion
  • 11. © Copyright 2016 Jawg 11 Can OSM services sustain a worldwide zombie infection?
  • 12. © Copyright 2016 Jawg 12 The Zombie Pandemic Virus = ZOMBIES Most services down
  • 13. © Copyright 2016 Jawg 13 The Zombie Pandemic • OSM app launched: WAZE World Against the Zombie Epidemic • Localizes Safe Zones and resources
  • 14. © Copyright 2016 Jawg 14 The Zombie Pandemic Scenario 200,000 potential users @ 8pm local release 1.12 billion 16% of population uninfected 1.12 million 0.1% of remaining pop. w/ access
  • 15. © Copyright 2016 Jawg 15 The Zombie Pandemic Scenario Main.mp4
  • 16. © Copyright 2016 Jawg 16 Load testing tool • Cluster mode • Proper metrics • Live reporting
  • 17. © Copyright 2016 Jawg 17 Architecture Injectors Caches Load-balancer Renderers CPU: 32RAM: 120GB BW: 2Gb/s Software: mod_tile, renderd, postgresql CPU: 8RAM: 30 GB BW: 2 Gb/s Software: squid3 CPU: 8RAM: 30 GB BW: 2 Gb/s Software: HAProxy CPU: 16RAM: 60 GB BW: 2 Gb/s Software: Gatling Internet
  • 18. © Copyright 2016 Jawg 18
  • 19. © Copyright 2016 Jawg 19 YES Objective: 1 billion maps Reached: 56 billion maps WAZE successfully saved the planet Can OSM services sustain a worldwide zombie infection?
  • 20. © Copyright 2016 Jawg 20 Load testsNew bottlenecks In reality Tuning storm • > 50 Gbps BW • Saturation of Cloud’s internal switch infrastructures • Kernel • Configuration • Load-balancing +50 load tests to find the most adequate simulation
  • 21. © Copyright 2016 Jawg 21 The Bottleneck Podium
  • 22. © Copyright 2016 Jawg 22 The State Of The Map (Server) Efficient Scalable Available Elastic Support
  • 23. © Copyright 2016 Jawg 23 White Papers 1. Map services: from theory to implementation • Available now @ jawg.io 2. Map services: Benchmarks & high-scale profiles • To be released in November 2016
  • 24. © Copyright 2016 Jawg 24 Thank You Loïc Ortola @LoicOrtola @jawgio

Editor's Notes

  1. Hello, My name is Loïc Ortola, and I am part of the Jawg maps family
  2. 20 Years ago, when most people were hearing the word “maps”, this is what they’d imagine. That little piece of paper you’d see in your glove compartment, lying under an old melted chocolate bar from last year Back then, making a trip was something you planned in advance, at home.
  3. Today, maps have completely reshaped our vision of transit In fact, maps are now everywhere. In your mall, in your building, integrated within your car. Now, maps are in your pocket 24 hours a day, You don’t only use them before but also in real-time. When you exit the metro, to catch the nearest Pokemon, to be driven home at night More importantly, every single one of you are here in this room because of a map. There isn’t a single person here which has not seen or used a map in the last 24 hours. (Alright, maybe the odds are a little bigger than usual in this room, but whatever) When digital maps started to appear, a whole new world of services started popping out. But someone had to provide those maps. Great mapping leaders started to emerge. With their awesome features, and their limitations.
  4. When maps become critical assets for a service … new matters come into play. Independence Data Governance / Intellectual Property Custom needs OpenStreetMap = most open option Is it business grade?
  5. What does business-grade even mean? Among other things, a business grade service usually means that it is: @Antoine Efficient, Scalable, Available, (and that you might get some support out of it.>> Ne pas oublier d’en parler à la fin)
  6. OSM is a great solution: Its licensing model is awesome, community-driven, etc… 12 years of experience -> it’s a sure thing
  7. It gets trickier the second you open the pandora’s box to setup your first map server. A few years ago, when we started working with OSM, someone told me “don’t worry, OSM is like a stairway, you’d rather take the elevator, but once you reach the first step, you just forget about the rest and move up” It took me years to understand what he meant. But I finally got it. Ladies and gentleman, I am proud to share with you
  8. I know you gave me 30 minutes, and such a question would require a decent amount of explanation. If we had all day, this is what we would be covering You know what? That’s boring”
  9. If we get back to our original question “Is OpenStreetMap business-grade?” We could probably find some better way to assert that than going over the big picture. We thought : wouldn’t it be better to try to show a real-world example proving that an OSM application is business-grade? So tonight, instead, we wanted to answer a really important question. A question that keeps everyone up at night. A question you may not even have dared to ask yourself:
  10. TODO: rajouter un zombie, changer la carte
  11. Zombies have infected earth
  12. A community of OpenStreetMap contributors
  13. 84% contaminated Thanks
  14. Description scénario gatling
  15. 3 things: In terms of architecture, we wanted to stick to the classic OpenStreetMap stack: ImpOSM, mod_tile, renderd, squid. What changed is that we tried to do an elastic scenario: when the demand gets higher, new servers get spawned until it reaches the pool limit.
  16. 99th percentile: 2.1s
  17. 99th percentile: 2.1s
  18. We are running out of time so I wont get into much details For instance, two nights ago, we went so high on activity that we triggered OVH’s internal Anti-DDoS system Which pretty much quarantined all our servers in a blink of an eye.
  19. 3 – CPU & I/O Cache / proxy servers (Squid, nginx, …) with disk cache need some memory caching strategy to avoid making too much disk reads Rendering & PostGIS DB heavy on CPU & I/O  Consider mounting your whole DB on SSD or in-memory Consider buffy rendering machines 2 – Tile Overdraw Different rendering servers can work on rendering the same meta-tiles Lack of renderer orchestration Lack of meta-tile aware load balancers 1 – Network Network bandwidth Most cloud providers provide between 100mbps – 1Gbps of bandwidth. 1 uncompressed tile  15kB  10k tiles/s  1 Gbps Network connections On buffy caches / load balancers with high throughput (~40k tiles/s) network IRQ and socket use can impact performance