Mapnik 2
                                Performance Optimizations


                             Dane Springmeyer | Development Seed




Sunday, September 11, 2011
Breakdown

                   •         Core (rendering pipeline)

                   •         Vector Output

                   •         Datasources

                   •         Image encoding

                   •         Styling




Sunday, September 11, 2011
Geometry Storage
                         faster, more memory efficient allocator
                         japan quake / massive landuse polygons
                         why tile.osm.org is already running mapnik
                         trunk
                         Artem/MapQuest contribution

     type: core
Sunday, September 11, 2011
Spirit2 Parsing
                         Grammer based string parsing
                         (expressions, filters, colors, wkt, dbf, svg
                         transforms)
                         Artem / Weait Contribution



     type: core
Sunday, September 11, 2011
Proj4 reprojection
                         Leveraging lock-free transforms if >= 4.8
                         4.8 not yet released so you need proj trunk
                         Tom Hughes - Mapnik London 2010 sprint



     type: core
Sunday, September 11, 2011
Raster
                             reprojection
                         New and fast: mesh based interpolation
                         Alberto Valverde - MeteoGrid/Madrid




     type: core
Sunday, September 11, 2011
Faster AA lines
                         New option to use 2-3X faster anti-aliased
                         line rasterization:

                             <LineSymbolizer rasterizer=”fast” />

                         Drops short segments, less precise line
                         caps/joins

     type: core
Sunday, September 11, 2011
Label Placement
                         Faster math
                         New options for fast placements or return
                         early (minimum-path-length)
                         Konstatin Kaefer / Herman Kraus



     type: core
Sunday, September 11, 2011
Clang support
                         2x speed of compiles than gcc
                         Supports link-time optimization (gold linker
                         and libLTO)



     type: core
Sunday, September 11, 2011
MetaWriters
                         Metadata collected during image rendering
                         (in same pass).
                         Can be output as GeoJSON or streamed
                         using mapnik::memory_datasource
                         Hermann / GSOC project

     type: vector output
Sunday, September 11, 2011
grid_renderer
                         Highly optimized hit grids encoded as utf8
                         feature id’s in json
                         Invisible, tiled interactivity
                         http://developmentseed.org/blog/2011/sep/
                         07/see-how-utfgrid-works-visible-map/

     type: vector output
Sunday, September 11, 2011
MemoryDatasource

                         Dynamic, direct in-memory creation to
                         render from http/json (think couchdb)




     type: datasources
Sunday, September 11, 2011
Other new Plugins
                         rasterlite (wavelets in sqlite)
                         mongodb (experimental: github/
                         springmeyer)




     type: datasources
Sunday, September 11, 2011
Postgres pool
                         Fixed to allow fuller utilization and benefit
                         of pg connections between many threads




     type: datasources
Sunday, September 11, 2011
Shapefile mmap
                         Ultra fast shared memory-mapped region
                         cache - > 5x faster i/o
                         Artem / FOSS4G “WMS Shootout”



     type: datasources
Sunday, September 11, 2011
SQLite
                         * rtree indexes
                         * shared cache
                         * fast, lock free access between threads
                         (Almost as fast as shapefiles now in mapnik)


     type: datasources
Sunday, September 11, 2011
PNG Size and Speed
                         * faster, more flexible png quantiziation
                         * fixed palettes - ultra fast color reduction
                         (save on encoding time without quant cost)
                         * direct access to zlib fast compression


     type: image encoding
Sunday, September 11, 2011
Filter-mode=first
                         Option on <Style /> to break after first rule
                         match
                         Critical for CSS support and avoiding
                         combinatorial explosion



     type: styling
Sunday, September 11, 2011
Feature Caching
                         Option on Layer:
                             <Layer feature-caching=”true”/>
                         Allows re-use of features within layer with
                         multiple styles (road-casing)



     type: styling
Sunday, September 11, 2011
Expressions
                         framework for runtime evaluation of styling
                         properties
                         <PolygonSymbolizer color=”[field]” />



     type: styling
Sunday, September 11, 2011
Raster Colorizer
                         framework for runtime evaluation of styling
                         properties
                       <RasterSymbolizer>
                        <RasterColorizer default-mode="discrete" default-color="white" >
                          <stop color="#222222" value = "0" mode = "linear" />
                          <stop color="#EEEEEE" value = "25" />
                        </RasterColorizer>
                       </RasterSymbolizer>


     type: styling
Sunday, September 11, 2011
Carto
                         New CSS language/parser (js, C++)
                         Inherits features of less.js (variables,nested
                         styles,mixins)



     type: styling
Sunday, September 11, 2011
SVG Support
                         SVG rendering of icons
                         Gradients, transforms (rotate/scale/shear)
                         http://developmentseed.org/



     type: styling
Sunday, September 11, 2011
Mapnik 2 release

               • Next week, more than a year in process
               • Try now:
                       sudo add-apt-repository ppa:mapnik/nightly-trunk
                       sudo apt-get update
                       sudo apt-get install libmapnik2 libmapnik2-dev mapnik2-utils

               • Learn more: http://trac.mapnik.org/
                       milestone/Mapnik%202.0



Sunday, September 11, 2011
Thank you!
                             Dane Springmeyer
                              twitter/springmeyer
                               github/springmeyer



Sunday, September 11, 2011

Mapnik2 Performance, September 2011

  • 1.
    Mapnik 2 Performance Optimizations Dane Springmeyer | Development Seed Sunday, September 11, 2011
  • 2.
    Breakdown • Core (rendering pipeline) • Vector Output • Datasources • Image encoding • Styling Sunday, September 11, 2011
  • 3.
    Geometry Storage faster, more memory efficient allocator japan quake / massive landuse polygons why tile.osm.org is already running mapnik trunk Artem/MapQuest contribution type: core Sunday, September 11, 2011
  • 4.
    Spirit2 Parsing Grammer based string parsing (expressions, filters, colors, wkt, dbf, svg transforms) Artem / Weait Contribution type: core Sunday, September 11, 2011
  • 5.
    Proj4 reprojection Leveraging lock-free transforms if >= 4.8 4.8 not yet released so you need proj trunk Tom Hughes - Mapnik London 2010 sprint type: core Sunday, September 11, 2011
  • 6.
    Raster reprojection New and fast: mesh based interpolation Alberto Valverde - MeteoGrid/Madrid type: core Sunday, September 11, 2011
  • 7.
    Faster AA lines New option to use 2-3X faster anti-aliased line rasterization: <LineSymbolizer rasterizer=”fast” /> Drops short segments, less precise line caps/joins type: core Sunday, September 11, 2011
  • 8.
    Label Placement Faster math New options for fast placements or return early (minimum-path-length) Konstatin Kaefer / Herman Kraus type: core Sunday, September 11, 2011
  • 9.
    Clang support 2x speed of compiles than gcc Supports link-time optimization (gold linker and libLTO) type: core Sunday, September 11, 2011
  • 10.
    MetaWriters Metadata collected during image rendering (in same pass). Can be output as GeoJSON or streamed using mapnik::memory_datasource Hermann / GSOC project type: vector output Sunday, September 11, 2011
  • 11.
    grid_renderer Highly optimized hit grids encoded as utf8 feature id’s in json Invisible, tiled interactivity http://developmentseed.org/blog/2011/sep/ 07/see-how-utfgrid-works-visible-map/ type: vector output Sunday, September 11, 2011
  • 12.
    MemoryDatasource Dynamic, direct in-memory creation to render from http/json (think couchdb) type: datasources Sunday, September 11, 2011
  • 13.
    Other new Plugins rasterlite (wavelets in sqlite) mongodb (experimental: github/ springmeyer) type: datasources Sunday, September 11, 2011
  • 14.
    Postgres pool Fixed to allow fuller utilization and benefit of pg connections between many threads type: datasources Sunday, September 11, 2011
  • 15.
    Shapefile mmap Ultra fast shared memory-mapped region cache - > 5x faster i/o Artem / FOSS4G “WMS Shootout” type: datasources Sunday, September 11, 2011
  • 16.
    SQLite * rtree indexes * shared cache * fast, lock free access between threads (Almost as fast as shapefiles now in mapnik) type: datasources Sunday, September 11, 2011
  • 17.
    PNG Size andSpeed * faster, more flexible png quantiziation * fixed palettes - ultra fast color reduction (save on encoding time without quant cost) * direct access to zlib fast compression type: image encoding Sunday, September 11, 2011
  • 18.
    Filter-mode=first Option on <Style /> to break after first rule match Critical for CSS support and avoiding combinatorial explosion type: styling Sunday, September 11, 2011
  • 19.
    Feature Caching Option on Layer: <Layer feature-caching=”true”/> Allows re-use of features within layer with multiple styles (road-casing) type: styling Sunday, September 11, 2011
  • 20.
    Expressions framework for runtime evaluation of styling properties <PolygonSymbolizer color=”[field]” /> type: styling Sunday, September 11, 2011
  • 21.
    Raster Colorizer framework for runtime evaluation of styling properties <RasterSymbolizer> <RasterColorizer default-mode="discrete" default-color="white" > <stop color="#222222" value = "0" mode = "linear" /> <stop color="#EEEEEE" value = "25" /> </RasterColorizer> </RasterSymbolizer> type: styling Sunday, September 11, 2011
  • 22.
    Carto New CSS language/parser (js, C++) Inherits features of less.js (variables,nested styles,mixins) type: styling Sunday, September 11, 2011
  • 23.
    SVG Support SVG rendering of icons Gradients, transforms (rotate/scale/shear) http://developmentseed.org/ type: styling Sunday, September 11, 2011
  • 24.
    Mapnik 2 release • Next week, more than a year in process • Try now: sudo add-apt-repository ppa:mapnik/nightly-trunk sudo apt-get update sudo apt-get install libmapnik2 libmapnik2-dev mapnik2-utils • Learn more: http://trac.mapnik.org/ milestone/Mapnik%202.0 Sunday, September 11, 2011
  • 25.
    Thank you! Dane Springmeyer twitter/springmeyer github/springmeyer Sunday, September 11, 2011