SlideShare a Scribd company logo
1 of 95
Download to read offline
State of the Art
  Web Mapping
with Open Source
OSCON 2012 | Dane Springmeyer
@springmeyer
 github / twitter
see also:
           Justin Miller
Building a mobile, offline mapping stack
        using open tools & data

        5pm Wednesday, F150
Background

    Engineer @ MapBox
 Building TileMill and Mapnik
Web performance / rendering
github / twitter
http://mapbox.com
We provide services & open source tools
open source tools
    to cover
CartoDB
TileMill
•
maps are simple
   (a primer)
geodata as
  just another
data field / type
cartography as just a
 sexy form of data
    visualization
location: lat/lon, x/y

    attributes:
 name, type, value
styles separate from
data akin to css/html
CartoCSS
@motorway: #90BFE0;

.highway[TYPE='motorway'] {

  .line[zoom>=0]    {

      line-color: spin(darken(@motorway,36),-10);

      line-cap:round;

      line-join:round;

  }

  .fill[zoom>=10] {

      line-color:@motorway;

      line-cap:round;

      line-join:round;}}
spatial types
point:	

	

 	

 	

 •
line:	

	

 	

 	

 	

 	

 -----------
polygon:
                             _____
                            |_____|
Multi* types
  many to one
geometries   feature
tabular geo-csv
 latitude    longitude   name



45.5        -122.6       PDX
tabular geo-csv
        (multipoint)
        WKT                     Name
 MULTIPOINT ((10 40),
(40 30), (20 20), (30 10))
                             Group of Cities
geojson
 { "type": "FeatureCollection",
   "features": [
     { "type": "Feature",
         "properties": { "name": "PDX" },
         "geometry" : { "type": "Point",
            "coordinates": [ -122.6, 45.5 ]
          }
     }]}

Works everywhere: e.g QGIS, TileMill, Web clients
postgis
postgis=# select 'POINT(-122.6
45.5)'::geometry as geom,'PDX'::text as name;

                    geom                    | name
--------------------------------------------+------
 01010000006666666666A65EC00000000000C04640 | PDX
(1 row)



                                     WKB
                              (Well Known Binary)
postgis
postgis=# select ST_Distance('POINT(-122.6
45.5)'::geography,'POINT(-122.3
47.6)'::geography)/1609.344 as
dist_in_miles_from_pdx_to_sea;

 dist_in_miles_from_pdx_to_sea
-------------------------------
              145.755555956692
(1 row)
minimal code for
simple maps, both
 server and client
Mapnik

var mapnik = require('mapnik');
var map = new mapnik.Map(256, 256);
map.loadSync('map.xml');
map.zoomAll();
map.renderFileSync('map.png');
Leaflet
<html>
<head>
  <link rel="stylesheet" href="http://cdn.leafletjs.com/leaflet-0.3.1/leaflet.css" />
  <script src="http://cdn.leafletjs.com/leaflet-0.3.1/leaflet.js"></script>
</head>
<body>
! <div id="map" style="width: 100%; height: 100%"></div>
  <script>
   var map = new L.Map('map');
   var osm = new L.TileLayer('http://tile.osm.org/{z}/{x}/{y}.png');
   map.setView(new L.LatLng(45.5, -122.65), 12).addLayer(osm);
   var pdx = { "type": "FeatureCollection", "features": [ { "type": "Feature",
     "properties": { "name": "PDX" },
     "geometry" : { "type": "Point","coordinates": [ -122.65, 45.5 ]}}]
   };
   map.addLayer(new L.GeoJSON(pdx));
  </script>
</body>
</html>
•
but maps are
    hard
geodata can be
messy and multi-
  resolution
geodata can be
    huge
geodata can be
   dynamic
data    story
takes too long
maps as the single
lock-in point (google)
  or point of failure
 (Slow WMS, IE support, clashing
           design)
•
how modern
web maps work
or, how to tell stories
  with maps quickly,
 ensure they are fast
    under load and
      work in IE
open data
         osm.org
   naturalearthdata.com
us census (geo/www/tiger)
local governments portals
serverside
pre-processing
gradually work
   clientside
tile renderers
mapnik / mapserver
fast app servers
Node.js/Python/C++
pre-processed
pre-rendered
  cacheable
beautiful graphics
 Antigrain Geometry
   Cairo Graphics
standard formats
  geojson, wkt, csv,
  shapefile, postgis,
       geotiff
tiled data
bake big data   bitmaps
pre-render where
possible, but beware the
      world is big
coming...
optimized tiled formats
like msgpack, protobuf
    (not just bitmaps)
coming ...
optimized tiled storage
     and indexing
coming ...
  more robust and
configurable clientside
     renderers
UI: Browser JS client / Mobile Native client

   Render:                            Display:
Hardware Vector                       <img> tiles
   Renderer



 Data Tiles:                        Image Tiles:
Optimized vectors                   Software Vector
                                       Renderer


  Storage: Database / Flat file (Spatially Indexed)
     Data Processing: GIS Apps, Scripting
UI: Leaflet / ModestMaps / RouteMe

    Render:                         Display:
 Kothic.js / Vecnik                all browers
(Canvas/WebGL in                     support
 some browsers)


  Data Tiles:                    Image Tiles:
  TileStache /                  Mapnik via TileMill
   Kothic.js /                    or TileStache
   TileMill ?                 + UTFGrid interactivity

   Storage: Postgres-PostGIS / CSV / GeoJSON
   Data Processing: Qgis, R stats, Python, SQL
•
TileMill
http://mapbox.com/tilemill/
Installing TileMill
TileMill: Ubuntu
sudo apt-add-repository
ppa:developmentseed/mapbox
sudo apt-get -y update
sudo apt-get -y install tilemill
sudo start tilemill
TileMill: Mac
TileMill: Basics
 Cross platform - Linux, Win, OS X
  Same code both desktop & web
Outputs PNG, MBTiles, Mapnik XML
Written in Javascript (Node.js) and C
             ++ (Mapnik)
Art of the possible
http://project.wnyc.org/stop-frisk-guns/
http://gop.sites.devseed.com/
foursquare.com
http://www.npr.org/censusmap/
http://streeteasy.com/
Millions of points without sacrificing speed
TileMill: Live

       http://bit.ly/MFjLnG
       http://bit.ly/SFeBfJ
      Ec2 machines only available on July 17,2012
set one up yourself like: http://mapbox.com/tilemill/docs/
                guides/ubuntu-service/
Demos...

• TileMill: layer ordering, fonts, labeling,
  plugins, mbtiles export, mapnik xml export,
  svg/marker-transforms
• TileMill: reinforce basics through demos:
  arc.js geojson, cartodb csv, etherpad csv
• OSM-bright setup
Thanks!

 @springmeyer
  github / twitter
dane@mapbox.com
•
Do not miss Stamen
   and Vizzuality
    (Cartodb)
TileMill Extra Details
IVE                                                                         doi:10.1038/nature10836




pen computer programs
m-Cumming3



 idence that cannot be entirely included in publications, but the rise of
 layer of inaccessibility. Although it is now accepted that data should be made
 ons regarding the availability of software are inconsistent. We argue that, with
 elease of source programs is intolerable for results that depend on computation.
nd natural language will always ensure that exact reproducibility remains
es the chances that efforts to reproduce results will fail.


 d to unprecedented       rerunning of the code on, say, a different combination of hardware and
 e powerful computers     systems software, to detect the sort of numerical computation11,12 and
were thought almost       interpretation13 problems found in programming languages, which we
gies allow data collec-   discuss later. Without code, direct reproducibility is impossible. Indirect
data are collected, and   reproducibility refers to independent efforts to validate something other
vailable with which to    than the entire code package, for example a subset of equations or a par-
                          ticular code module. Here, before time-consuming reprogramming of an
 y, by which we mean      entire model, researchers may simply want to check that incorrect coding of
ing, rather than exact    previously published equations has not invalidated a paper’s result, to
 n to several decimal     extract and check detailed assumptions, or to run their own code against
 (for an early attempt    the original to check for statistical validity and explain any discrepancies.
y available computer         Any debate over the difficulties of reproducibility (which, as we will
 d the point that, with   show, are non-trivial) must of course be tempered by recognizing the
ual source code is an     undeniable benefits afforded by the explosion of internet facilities and the
 depend on computa-       rapid increase in raw computational speed and data-handling capability
dless, and needlessly     that has occurred as a result of major advances in computer technology14.
                          Such advances have presented science with a great opportunity to address
  computer programs       problems that would have been intractable in even the recent past. It is
cies still ranging from   our view, however, that the debate over code release should be resolved as
 ral language descrip-    soon as possible to benefit fully from our novel technical capabilities. On
program algorithms.       their own, finer computational grids, longer and more complex compu-
 n computer program       tations and larger data sets—although highly attractive to scientific
 e in the list of items   researchers—do not resolve underlying computational uncertainties of
State of the Art Web Mapping with Open Source
State of the Art Web Mapping with Open Source

More Related Content

Viewers also liked

Declarative web data visualization using ClojureScript
Declarative web data visualization using ClojureScriptDeclarative web data visualization using ClojureScript
Declarative web data visualization using ClojureScriptOSCON Byrum
 
GEOPROCESSING IN QGIS
GEOPROCESSING IN QGISGEOPROCESSING IN QGIS
GEOPROCESSING IN QGISSwetha A
 
Bringing Cities to Life Using Big Data & WebGL
Bringing Cities to Life Using Big Data & WebGLBringing Cities to Life Using Big Data & WebGL
Bringing Cities to Life Using Big Data & WebGLRobin Hawkes
 
Digital image processing using matlab
Digital image processing using matlab Digital image processing using matlab
Digital image processing using matlab Amr Rashed
 
Basics of Image Processing using MATLAB
Basics of Image Processing using MATLABBasics of Image Processing using MATLAB
Basics of Image Processing using MATLABvkn13
 
Projects types
Projects types Projects types
Projects types Ahmed Sakr
 
Project Types and their importance
Project Types and their importanceProject Types and their importance
Project Types and their importanceUjjwal Joshi
 
Introduction to remote sensing and gis
Introduction to remote sensing and gisIntroduction to remote sensing and gis
Introduction to remote sensing and gisMohsin Siddique
 
types of planning
types of planningtypes of planning
types of planningphenixtech
 
Basic of Remote Sensing
Basic of Remote SensingBasic of Remote Sensing
Basic of Remote Sensinggueste5cfed
 
Introduction to project management
Introduction to project managementIntroduction to project management
Introduction to project managementBarun_agnihotri
 
Fundamentals of Project Management
Fundamentals of Project ManagementFundamentals of Project Management
Fundamentals of Project ManagementRodolfo Siles
 

Viewers also liked (20)

Declarative web data visualization using ClojureScript
Declarative web data visualization using ClojureScriptDeclarative web data visualization using ClojureScript
Declarative web data visualization using ClojureScript
 
MapBox
MapBoxMapBox
MapBox
 
planning nature
planning natureplanning nature
planning nature
 
GEOPROCESSING IN QGIS
GEOPROCESSING IN QGISGEOPROCESSING IN QGIS
GEOPROCESSING IN QGIS
 
Bringing Cities to Life Using Big Data & WebGL
Bringing Cities to Life Using Big Data & WebGLBringing Cities to Life Using Big Data & WebGL
Bringing Cities to Life Using Big Data & WebGL
 
Digital image processing using matlab
Digital image processing using matlab Digital image processing using matlab
Digital image processing using matlab
 
Basics of Image Processing using MATLAB
Basics of Image Processing using MATLABBasics of Image Processing using MATLAB
Basics of Image Processing using MATLAB
 
Projects types
Projects types Projects types
Projects types
 
Project Types and their importance
Project Types and their importanceProject Types and their importance
Project Types and their importance
 
Project Planning
Project PlanningProject Planning
Project Planning
 
Project planning
Project planning Project planning
Project planning
 
Introduction to remote sensing and gis
Introduction to remote sensing and gisIntroduction to remote sensing and gis
Introduction to remote sensing and gis
 
types of planning
types of planningtypes of planning
types of planning
 
Basic of Remote Sensing
Basic of Remote SensingBasic of Remote Sensing
Basic of Remote Sensing
 
Project Planning
Project PlanningProject Planning
Project Planning
 
What Is GIS?
What Is GIS?What Is GIS?
What Is GIS?
 
Chapter 3-THE RESEARCH PROBLEM
Chapter 3-THE RESEARCH PROBLEMChapter 3-THE RESEARCH PROBLEM
Chapter 3-THE RESEARCH PROBLEM
 
Introduction to project management
Introduction to project managementIntroduction to project management
Introduction to project management
 
Fundamentals of Project Management
Fundamentals of Project ManagementFundamentals of Project Management
Fundamentals of Project Management
 
Planning
PlanningPlanning
Planning
 

Similar to State of the Art Web Mapping with Open Source

Tilemill gwu-wboykinm
Tilemill gwu-wboykinmTilemill gwu-wboykinm
Tilemill gwu-wboykinmBill Morris
 
INAC Online Hazards Database App
INAC Online Hazards Database AppINAC Online Hazards Database App
INAC Online Hazards Database AppGerry James
 
Graph computation
Graph computationGraph computation
Graph computationSigmoid
 
maXbox Starter 39 GEO Maps Tutorial
maXbox Starter 39 GEO Maps TutorialmaXbox Starter 39 GEO Maps Tutorial
maXbox Starter 39 GEO Maps TutorialMax Kleiner
 
D3 Mapping Visualization
D3 Mapping VisualizationD3 Mapping Visualization
D3 Mapping VisualizationSudhir Chowbina
 
Concepts and Methods of Embedding Statistical Data into Maps
Concepts and Methods of Embedding Statistical Data into MapsConcepts and Methods of Embedding Statistical Data into Maps
Concepts and Methods of Embedding Statistical Data into MapsMohammad Liton Hossain
 
mloc.js 2014 - JavaScript and the browser as a platform for game development
mloc.js 2014 - JavaScript and the browser as a platform for game developmentmloc.js 2014 - JavaScript and the browser as a platform for game development
mloc.js 2014 - JavaScript and the browser as a platform for game developmentDavid Galeano
 
Scalding big ADta
Scalding big ADtaScalding big ADta
Scalding big ADtab0ris_1
 
Event Visualization with OpenStreetMap Data, Interdisciplinary Project
Event Visualization with OpenStreetMap Data, Interdisciplinary ProjectEvent Visualization with OpenStreetMap Data, Interdisciplinary Project
Event Visualization with OpenStreetMap Data, Interdisciplinary ProjectBibek Shrestha
 
The 'right' choices in GIS - Grontmij
The 'right' choices in GIS - GrontmijThe 'right' choices in GIS - Grontmij
The 'right' choices in GIS - GrontmijXander Bakker
 
Programmable Exascale Supercomputer
Programmable Exascale SupercomputerProgrammable Exascale Supercomputer
Programmable Exascale SupercomputerSagar Dolas
 
Developing Spatial Applications with Google Maps and CARTO
Developing Spatial Applications with Google Maps and CARTODeveloping Spatial Applications with Google Maps and CARTO
Developing Spatial Applications with Google Maps and CARTOCARTO
 
060128 Galeon Rept
060128 Galeon Rept060128 Galeon Rept
060128 Galeon ReptRudolf Husar
 
Introduction to mago3D, an Open Source Based Digital Twin Platform
Introduction to mago3D, an Open Source Based Digital Twin PlatformIntroduction to mago3D, an Open Source Based Digital Twin Platform
Introduction to mago3D, an Open Source Based Digital Twin PlatformSANGHEE SHIN
 
Eclipse Con Europe 2014 How to use DAWN Science Project
Eclipse Con Europe 2014 How to use DAWN Science ProjectEclipse Con Europe 2014 How to use DAWN Science Project
Eclipse Con Europe 2014 How to use DAWN Science ProjectMatthew Gerring
 
Android Development w/ ArcGIS Server - Esri Dev Meetup - Charlotte, NC
Android Development w/ ArcGIS Server - Esri Dev Meetup - Charlotte, NCAndroid Development w/ ArcGIS Server - Esri Dev Meetup - Charlotte, NC
Android Development w/ ArcGIS Server - Esri Dev Meetup - Charlotte, NCJim Tochterman
 

Similar to State of the Art Web Mapping with Open Source (20)

Tilemill gwu-wboykinm
Tilemill gwu-wboykinmTilemill gwu-wboykinm
Tilemill gwu-wboykinm
 
INAC Online Hazards Database App
INAC Online Hazards Database AppINAC Online Hazards Database App
INAC Online Hazards Database App
 
Graph computation
Graph computationGraph computation
Graph computation
 
HTML5 and CartoDB
HTML5 and CartoDBHTML5 and CartoDB
HTML5 and CartoDB
 
Portfolio
PortfolioPortfolio
Portfolio
 
Sudheer
SudheerSudheer
Sudheer
 
maXbox Starter 39 GEO Maps Tutorial
maXbox Starter 39 GEO Maps TutorialmaXbox Starter 39 GEO Maps Tutorial
maXbox Starter 39 GEO Maps Tutorial
 
D3 Mapping Visualization
D3 Mapping VisualizationD3 Mapping Visualization
D3 Mapping Visualization
 
Concepts and Methods of Embedding Statistical Data into Maps
Concepts and Methods of Embedding Statistical Data into MapsConcepts and Methods of Embedding Statistical Data into Maps
Concepts and Methods of Embedding Statistical Data into Maps
 
mloc.js 2014 - JavaScript and the browser as a platform for game development
mloc.js 2014 - JavaScript and the browser as a platform for game developmentmloc.js 2014 - JavaScript and the browser as a platform for game development
mloc.js 2014 - JavaScript and the browser as a platform for game development
 
Scalding big ADta
Scalding big ADtaScalding big ADta
Scalding big ADta
 
Event Visualization with OpenStreetMap Data, Interdisciplinary Project
Event Visualization with OpenStreetMap Data, Interdisciplinary ProjectEvent Visualization with OpenStreetMap Data, Interdisciplinary Project
Event Visualization with OpenStreetMap Data, Interdisciplinary Project
 
The 'right' choices in GIS - Grontmij
The 'right' choices in GIS - GrontmijThe 'right' choices in GIS - Grontmij
The 'right' choices in GIS - Grontmij
 
Programmable Exascale Supercomputer
Programmable Exascale SupercomputerProgrammable Exascale Supercomputer
Programmable Exascale Supercomputer
 
Developing Spatial Applications with Google Maps and CARTO
Developing Spatial Applications with Google Maps and CARTODeveloping Spatial Applications with Google Maps and CARTO
Developing Spatial Applications with Google Maps and CARTO
 
The 21st Century Harvard Map
The 21st Century Harvard MapThe 21st Century Harvard Map
The 21st Century Harvard Map
 
060128 Galeon Rept
060128 Galeon Rept060128 Galeon Rept
060128 Galeon Rept
 
Introduction to mago3D, an Open Source Based Digital Twin Platform
Introduction to mago3D, an Open Source Based Digital Twin PlatformIntroduction to mago3D, an Open Source Based Digital Twin Platform
Introduction to mago3D, an Open Source Based Digital Twin Platform
 
Eclipse Con Europe 2014 How to use DAWN Science Project
Eclipse Con Europe 2014 How to use DAWN Science ProjectEclipse Con Europe 2014 How to use DAWN Science Project
Eclipse Con Europe 2014 How to use DAWN Science Project
 
Android Development w/ ArcGIS Server - Esri Dev Meetup - Charlotte, NC
Android Development w/ ArcGIS Server - Esri Dev Meetup - Charlotte, NCAndroid Development w/ ArcGIS Server - Esri Dev Meetup - Charlotte, NC
Android Development w/ ArcGIS Server - Esri Dev Meetup - Charlotte, NC
 

More from OSCON Byrum

OSCON 2013 - Planning an OpenStack Cloud - Tom Fifield
OSCON 2013 - Planning an OpenStack Cloud - Tom FifieldOSCON 2013 - Planning an OpenStack Cloud - Tom Fifield
OSCON 2013 - Planning an OpenStack Cloud - Tom FifieldOSCON Byrum
 
Protecting Open Innovation with the Defensive Patent License
Protecting Open Innovation with the Defensive Patent LicenseProtecting Open Innovation with the Defensive Patent License
Protecting Open Innovation with the Defensive Patent LicenseOSCON Byrum
 
Using Cascalog to build an app with City of Palo Alto Open Data
Using Cascalog to build an app with City of Palo Alto Open DataUsing Cascalog to build an app with City of Palo Alto Open Data
Using Cascalog to build an app with City of Palo Alto Open DataOSCON Byrum
 
Finite State Machines - Why the fear?
Finite State Machines - Why the fear?Finite State Machines - Why the fear?
Finite State Machines - Why the fear?OSCON Byrum
 
Open Source Automotive Development
Open Source Automotive DevelopmentOpen Source Automotive Development
Open Source Automotive DevelopmentOSCON Byrum
 
How we built our community using Github - Uri Cohen
How we built our community using Github - Uri CohenHow we built our community using Github - Uri Cohen
How we built our community using Github - Uri CohenOSCON Byrum
 
The Vanishing Pattern: from iterators to generators in Python
The Vanishing Pattern: from iterators to generators in PythonThe Vanishing Pattern: from iterators to generators in Python
The Vanishing Pattern: from iterators to generators in PythonOSCON Byrum
 
Distributed Coordination with Python
Distributed Coordination with PythonDistributed Coordination with Python
Distributed Coordination with PythonOSCON Byrum
 
An overview of open source in East Asia (China, Japan, Korea)
An overview of open source in East Asia (China, Japan, Korea)An overview of open source in East Asia (China, Japan, Korea)
An overview of open source in East Asia (China, Japan, Korea)OSCON Byrum
 
Oscon 2013 Jesse Anderson
Oscon 2013 Jesse AndersonOscon 2013 Jesse Anderson
Oscon 2013 Jesse AndersonOSCON Byrum
 
US Patriot Act OSCON2012 David Mertz
US Patriot Act OSCON2012 David MertzUS Patriot Act OSCON2012 David Mertz
US Patriot Act OSCON2012 David MertzOSCON Byrum
 
OSCON 2012 US Patriot Act Implications for Cloud Computing - Diane Mueller, A...
OSCON 2012 US Patriot Act Implications for Cloud Computing - Diane Mueller, A...OSCON 2012 US Patriot Act Implications for Cloud Computing - Diane Mueller, A...
OSCON 2012 US Patriot Act Implications for Cloud Computing - Diane Mueller, A...OSCON Byrum
 
Big Data for each one of us
Big Data for each one of usBig Data for each one of us
Big Data for each one of usOSCON Byrum
 
BodyTrack: Open Source Tools for Health Empowerment through Self-Tracking
BodyTrack: Open Source Tools for Health Empowerment through Self-Tracking BodyTrack: Open Source Tools for Health Empowerment through Self-Tracking
BodyTrack: Open Source Tools for Health Empowerment through Self-Tracking OSCON Byrum
 
Using and Building Open Source in Google Corporate Engineering - Justin McWil...
Using and Building Open Source in Google Corporate Engineering - Justin McWil...Using and Building Open Source in Google Corporate Engineering - Justin McWil...
Using and Building Open Source in Google Corporate Engineering - Justin McWil...OSCON Byrum
 
A Look at the Network: Searching for Truth in Distributed Applications
A Look at the Network: Searching for Truth in Distributed ApplicationsA Look at the Network: Searching for Truth in Distributed Applications
A Look at the Network: Searching for Truth in Distributed ApplicationsOSCON Byrum
 
Life After Sharding: Monitoring and Management of a Complex Data Cloud
Life After Sharding: Monitoring and Management of a Complex Data CloudLife After Sharding: Monitoring and Management of a Complex Data Cloud
Life After Sharding: Monitoring and Management of a Complex Data CloudOSCON Byrum
 
Faster! Faster! Accelerate your business with blazing prototypes
Faster! Faster! Accelerate your business with blazing prototypesFaster! Faster! Accelerate your business with blazing prototypes
Faster! Faster! Accelerate your business with blazing prototypesOSCON Byrum
 
Comparing open source private cloud platforms
Comparing open source private cloud platformsComparing open source private cloud platforms
Comparing open source private cloud platformsOSCON Byrum
 
Building an Ecosystem of FLOSS to Educate Students with Disabilities
Building an Ecosystem of FLOSS to Educate Students with DisabilitiesBuilding an Ecosystem of FLOSS to Educate Students with Disabilities
Building an Ecosystem of FLOSS to Educate Students with DisabilitiesOSCON Byrum
 

More from OSCON Byrum (20)

OSCON 2013 - Planning an OpenStack Cloud - Tom Fifield
OSCON 2013 - Planning an OpenStack Cloud - Tom FifieldOSCON 2013 - Planning an OpenStack Cloud - Tom Fifield
OSCON 2013 - Planning an OpenStack Cloud - Tom Fifield
 
Protecting Open Innovation with the Defensive Patent License
Protecting Open Innovation with the Defensive Patent LicenseProtecting Open Innovation with the Defensive Patent License
Protecting Open Innovation with the Defensive Patent License
 
Using Cascalog to build an app with City of Palo Alto Open Data
Using Cascalog to build an app with City of Palo Alto Open DataUsing Cascalog to build an app with City of Palo Alto Open Data
Using Cascalog to build an app with City of Palo Alto Open Data
 
Finite State Machines - Why the fear?
Finite State Machines - Why the fear?Finite State Machines - Why the fear?
Finite State Machines - Why the fear?
 
Open Source Automotive Development
Open Source Automotive DevelopmentOpen Source Automotive Development
Open Source Automotive Development
 
How we built our community using Github - Uri Cohen
How we built our community using Github - Uri CohenHow we built our community using Github - Uri Cohen
How we built our community using Github - Uri Cohen
 
The Vanishing Pattern: from iterators to generators in Python
The Vanishing Pattern: from iterators to generators in PythonThe Vanishing Pattern: from iterators to generators in Python
The Vanishing Pattern: from iterators to generators in Python
 
Distributed Coordination with Python
Distributed Coordination with PythonDistributed Coordination with Python
Distributed Coordination with Python
 
An overview of open source in East Asia (China, Japan, Korea)
An overview of open source in East Asia (China, Japan, Korea)An overview of open source in East Asia (China, Japan, Korea)
An overview of open source in East Asia (China, Japan, Korea)
 
Oscon 2013 Jesse Anderson
Oscon 2013 Jesse AndersonOscon 2013 Jesse Anderson
Oscon 2013 Jesse Anderson
 
US Patriot Act OSCON2012 David Mertz
US Patriot Act OSCON2012 David MertzUS Patriot Act OSCON2012 David Mertz
US Patriot Act OSCON2012 David Mertz
 
OSCON 2012 US Patriot Act Implications for Cloud Computing - Diane Mueller, A...
OSCON 2012 US Patriot Act Implications for Cloud Computing - Diane Mueller, A...OSCON 2012 US Patriot Act Implications for Cloud Computing - Diane Mueller, A...
OSCON 2012 US Patriot Act Implications for Cloud Computing - Diane Mueller, A...
 
Big Data for each one of us
Big Data for each one of usBig Data for each one of us
Big Data for each one of us
 
BodyTrack: Open Source Tools for Health Empowerment through Self-Tracking
BodyTrack: Open Source Tools for Health Empowerment through Self-Tracking BodyTrack: Open Source Tools for Health Empowerment through Self-Tracking
BodyTrack: Open Source Tools for Health Empowerment through Self-Tracking
 
Using and Building Open Source in Google Corporate Engineering - Justin McWil...
Using and Building Open Source in Google Corporate Engineering - Justin McWil...Using and Building Open Source in Google Corporate Engineering - Justin McWil...
Using and Building Open Source in Google Corporate Engineering - Justin McWil...
 
A Look at the Network: Searching for Truth in Distributed Applications
A Look at the Network: Searching for Truth in Distributed ApplicationsA Look at the Network: Searching for Truth in Distributed Applications
A Look at the Network: Searching for Truth in Distributed Applications
 
Life After Sharding: Monitoring and Management of a Complex Data Cloud
Life After Sharding: Monitoring and Management of a Complex Data CloudLife After Sharding: Monitoring and Management of a Complex Data Cloud
Life After Sharding: Monitoring and Management of a Complex Data Cloud
 
Faster! Faster! Accelerate your business with blazing prototypes
Faster! Faster! Accelerate your business with blazing prototypesFaster! Faster! Accelerate your business with blazing prototypes
Faster! Faster! Accelerate your business with blazing prototypes
 
Comparing open source private cloud platforms
Comparing open source private cloud platformsComparing open source private cloud platforms
Comparing open source private cloud platforms
 
Building an Ecosystem of FLOSS to Educate Students with Disabilities
Building an Ecosystem of FLOSS to Educate Students with DisabilitiesBuilding an Ecosystem of FLOSS to Educate Students with Disabilities
Building an Ecosystem of FLOSS to Educate Students with Disabilities
 

Recently uploaded

Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Karmanjay Verma
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...itnewsafrica
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 

Recently uploaded (20)

Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 

State of the Art Web Mapping with Open Source

  • 1. State of the Art Web Mapping with Open Source OSCON 2012 | Dane Springmeyer
  • 3. see also: Justin Miller Building a mobile, offline mapping stack using open tools & data 5pm Wednesday, F150
  • 4. Background Engineer @ MapBox Building TileMill and Mapnik Web performance / rendering
  • 6.
  • 7. We provide services & open source tools
  • 8. open source tools to cover
  • 9.
  • 10.
  • 11.
  • 12.
  • 14.
  • 15.
  • 17.
  • 18. maps are simple (a primer)
  • 19. geodata as just another data field / type
  • 20. cartography as just a sexy form of data visualization
  • 21. location: lat/lon, x/y attributes: name, type, value
  • 22. styles separate from data akin to css/html
  • 23. CartoCSS @motorway: #90BFE0; .highway[TYPE='motorway'] { .line[zoom>=0] { line-color: spin(darken(@motorway,36),-10); line-cap:round; line-join:round; } .fill[zoom>=10] { line-color:@motorway; line-cap:round; line-join:round;}}
  • 24. spatial types point: • line: ----------- polygon: _____ |_____|
  • 25. Multi* types many to one geometries feature
  • 26. tabular geo-csv latitude longitude name 45.5 -122.6 PDX
  • 27. tabular geo-csv (multipoint) WKT Name MULTIPOINT ((10 40), (40 30), (20 20), (30 10)) Group of Cities
  • 28. geojson { "type": "FeatureCollection", "features": [ { "type": "Feature", "properties": { "name": "PDX" }, "geometry" : { "type": "Point", "coordinates": [ -122.6, 45.5 ] } }]} Works everywhere: e.g QGIS, TileMill, Web clients
  • 29. postgis postgis=# select 'POINT(-122.6 45.5)'::geometry as geom,'PDX'::text as name; geom | name --------------------------------------------+------ 01010000006666666666A65EC00000000000C04640 | PDX (1 row) WKB (Well Known Binary)
  • 30. postgis postgis=# select ST_Distance('POINT(-122.6 45.5)'::geography,'POINT(-122.3 47.6)'::geography)/1609.344 as dist_in_miles_from_pdx_to_sea; dist_in_miles_from_pdx_to_sea ------------------------------- 145.755555956692 (1 row)
  • 31. minimal code for simple maps, both server and client
  • 32. Mapnik var mapnik = require('mapnik'); var map = new mapnik.Map(256, 256); map.loadSync('map.xml'); map.zoomAll(); map.renderFileSync('map.png');
  • 33. Leaflet <html> <head> <link rel="stylesheet" href="http://cdn.leafletjs.com/leaflet-0.3.1/leaflet.css" /> <script src="http://cdn.leafletjs.com/leaflet-0.3.1/leaflet.js"></script> </head> <body> ! <div id="map" style="width: 100%; height: 100%"></div> <script> var map = new L.Map('map'); var osm = new L.TileLayer('http://tile.osm.org/{z}/{x}/{y}.png'); map.setView(new L.LatLng(45.5, -122.65), 12).addLayer(osm); var pdx = { "type": "FeatureCollection", "features": [ { "type": "Feature", "properties": { "name": "PDX" }, "geometry" : { "type": "Point","coordinates": [ -122.65, 45.5 ]}}] }; map.addLayer(new L.GeoJSON(pdx)); </script> </body> </html>
  • 34.
  • 35. but maps are hard
  • 36. geodata can be messy and multi- resolution
  • 38. geodata can be dynamic
  • 39. data story takes too long
  • 40. maps as the single lock-in point (google) or point of failure (Slow WMS, IE support, clashing design)
  • 41.
  • 43. or, how to tell stories with maps quickly, ensure they are fast under load and work in IE
  • 44. open data osm.org naturalearthdata.com us census (geo/www/tiger) local governments portals
  • 46. gradually work clientside
  • 50. beautiful graphics Antigrain Geometry Cairo Graphics
  • 51. standard formats geojson, wkt, csv, shapefile, postgis, geotiff
  • 53. bake big data bitmaps
  • 54. pre-render where possible, but beware the world is big
  • 55.
  • 56.
  • 57.
  • 58.
  • 59.
  • 60. coming... optimized tiled formats like msgpack, protobuf (not just bitmaps)
  • 61. coming ... optimized tiled storage and indexing
  • 62. coming ... more robust and configurable clientside renderers
  • 63. UI: Browser JS client / Mobile Native client Render: Display: Hardware Vector <img> tiles Renderer Data Tiles: Image Tiles: Optimized vectors Software Vector Renderer Storage: Database / Flat file (Spatially Indexed) Data Processing: GIS Apps, Scripting
  • 64. UI: Leaflet / ModestMaps / RouteMe Render: Display: Kothic.js / Vecnik all browers (Canvas/WebGL in support some browsers) Data Tiles: Image Tiles: TileStache / Mapnik via TileMill Kothic.js / or TileStache TileMill ? + UTFGrid interactivity Storage: Postgres-PostGIS / CSV / GeoJSON Data Processing: Qgis, R stats, Python, SQL
  • 65.
  • 68. TileMill: Ubuntu sudo apt-add-repository ppa:developmentseed/mapbox sudo apt-get -y update sudo apt-get -y install tilemill sudo start tilemill
  • 70. TileMill: Basics Cross platform - Linux, Win, OS X Same code both desktop & web Outputs PNG, MBTiles, Mapnik XML Written in Javascript (Node.js) and C ++ (Mapnik)
  • 71. Art of the possible
  • 77. Millions of points without sacrificing speed
  • 78. TileMill: Live http://bit.ly/MFjLnG http://bit.ly/SFeBfJ Ec2 machines only available on July 17,2012 set one up yourself like: http://mapbox.com/tilemill/docs/ guides/ubuntu-service/
  • 79. Demos... • TileMill: layer ordering, fonts, labeling, plugins, mbtiles export, mapnik xml export, svg/marker-transforms • TileMill: reinforce basics through demos: arc.js geojson, cartodb csv, etherpad csv • OSM-bright setup
  • 80. Thanks! @springmeyer github / twitter dane@mapbox.com
  • 81.
  • 82. Do not miss Stamen and Vizzuality (Cartodb)
  • 83.
  • 84.
  • 86.
  • 87.
  • 88.
  • 89.
  • 90.
  • 91.
  • 92.
  • 93. IVE doi:10.1038/nature10836 pen computer programs m-Cumming3 idence that cannot be entirely included in publications, but the rise of layer of inaccessibility. Although it is now accepted that data should be made ons regarding the availability of software are inconsistent. We argue that, with elease of source programs is intolerable for results that depend on computation. nd natural language will always ensure that exact reproducibility remains es the chances that efforts to reproduce results will fail. d to unprecedented rerunning of the code on, say, a different combination of hardware and e powerful computers systems software, to detect the sort of numerical computation11,12 and were thought almost interpretation13 problems found in programming languages, which we gies allow data collec- discuss later. Without code, direct reproducibility is impossible. Indirect data are collected, and reproducibility refers to independent efforts to validate something other vailable with which to than the entire code package, for example a subset of equations or a par- ticular code module. Here, before time-consuming reprogramming of an y, by which we mean entire model, researchers may simply want to check that incorrect coding of ing, rather than exact previously published equations has not invalidated a paper’s result, to n to several decimal extract and check detailed assumptions, or to run their own code against (for an early attempt the original to check for statistical validity and explain any discrepancies. y available computer Any debate over the difficulties of reproducibility (which, as we will d the point that, with show, are non-trivial) must of course be tempered by recognizing the ual source code is an undeniable benefits afforded by the explosion of internet facilities and the depend on computa- rapid increase in raw computational speed and data-handling capability dless, and needlessly that has occurred as a result of major advances in computer technology14. Such advances have presented science with a great opportunity to address computer programs problems that would have been intractable in even the recent past. It is cies still ranging from our view, however, that the debate over code release should be resolved as ral language descrip- soon as possible to benefit fully from our novel technical capabilities. On program algorithms. their own, finer computational grids, longer and more complex compu- n computer program tations and larger data sets—although highly attractive to scientific e in the list of items researchers—do not resolve underlying computational uncertainties of