Clip and Ship: MapServer for  Data Distribution   Frank Warmerdam, Google, Inc. Michael Smith, US Army Corps of Engineers
Clip and Ship <ul><li>Interactively: </li></ul><ul><ul><li>visualization </li></ul></ul><ul><ul><li>layer selection </li><...
OGC Web Protocols <ul><li>All are HTTP web service protocols </li></ul><ul><li>  </li></ul><ul><li>WMS (Web Map Service)  ...
MapServer and GDAL/OGR <ul><li>GDAL/OGR  </li></ul><ul><ul><li>multi format raster/vector data access  </li></ul></ul><ul>...
CorpsMap <ul><ul><li>MapServer  </li></ul></ul><ul><ul><li>OpenLayers  </li></ul></ul><ul><ul><li>GeoExt </li></ul></ul><u...
MapServer Implementation <ul><li>Map File:  </li></ul><ul><ul><li>OUTPUTFORMAT declarations for output formats </li></ul><...
Metadata to allow Export  and Support Projections <ul><li>WEB </li></ul><ul><li>    ...... </li></ul><ul><li>    METADATA ...
Including Attributes <ul><ul><li>Controlled via  gml_include_items  and  gml_exclude_items </li></ul></ul><ul><ul><li>Sepa...
Attribute Level Controls <ul><ul><li>gml_ itemname _alias </li></ul></ul><ul><ul><li>gml_ itemname _width </li></ul></ul><...
Geometry Type <ul><li>gml/ows/wfs_geomtype (Optional) </li></ul><ul><ul><li>Geometry, Point, LineString, Polygon, MultiPoi...
OutputFormat Declaration <ul><li>Describes a OGR Format including </li></ul><ul><ul><li>layer creation options (LCO:*) </l...
Example Output Formats <ul><li>OUTPUTFORMAT </li></ul><ul><li>   NAME &quot;Shapefile&quot; </li></ul><ul><li>   DRIVER &q...
More Examples  <ul><li>OUTPUTFORMAT </li></ul><ul><li>   NAME &quot;geojson&quot; </li></ul><ul><li>   DRIVER &quot;OGR/GE...
Even More Examples  <ul><li>OUTPUTFORMAT </li></ul><ul><li>   NAME &quot;MIDMIF&quot; </li></ul><ul><li>   DRIVER &quot;OG...
Make Request via WFS <ul><li>http://server/cgi-bin/mapserv6?map=mymapfile.map </li></ul><ul><li>&service=wfs&version=1.1.0...
Make a request for projected data   (requires WFS 1.1.0 or later) <ul><li>http://server/cgi-bin/mapserv6?map=mymapfile.map...
Clip and Ship <ul><li>http://server/cgi-bin/mapserv6?map=mymapfile.map </li></ul><ul><li>&service=wfs&version=1.1.0&reques...
Validation for Filename Substitution <ul><li>WEB </li></ul><ul><li>    .... </li></ul><ul><li>      METADATA </li></ul><ul...
Request with Filename <ul><li>http://server/cgi-bin/mapserv6?map=mymapfile.map </li></ul><ul><li>&service=wfs&version=1.1....
Other Clip and Ship Options <ul><ul><li>GeoServer has roughly similar WFS output extensions </li></ul></ul><ul><ul><li>Sug...
Lessons Learned <ul><ul><li>No temporary files (for some formats)  </li></ul></ul><ul><ul><li>Only method to limit downloa...
Conclusion <ul><li>Docs are available at  </li></ul><ul><ul><li>OGR Output - http://mapserver.org/output/ogr_output.html <...
CorpsMap Viewer  OpenLayers / GeoExt / MapServer 6 / Oracle based Web Viewer
Exporting a data layer
Shapefile Export
Zipped Shapefile Generated http://maps.crrel.usace.army.mil:7778/cgi-bin/mapserv6?map=/htdocs/cm2_dev/cm2_test.map &servic...
Upcoming SlideShare
Loading in …5
×

Clip and Ship: MapServer for Data Distribution

5,266 views

Published on

Clip and Ship: MapServer for Data Distribution by Frank Warmerdam of Google and Michael Smith of US Army Corps of Engineers

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
5,266
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
40
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Clip and Ship: MapServer for Data Distribution

  1. 1. Clip and Ship: MapServer for  Data Distribution   Frank Warmerdam, Google, Inc. Michael Smith, US Army Corps of Engineers
  2. 2. Clip and Ship <ul><li>Interactively: </li></ul><ul><ul><li>visualization </li></ul></ul><ul><ul><li>layer selection </li></ul></ul><ul><ul><li>spatial drill down </li></ul></ul><ul><li>  </li></ul><ul><li>Deliver Data: </li></ul><ul><ul><li>file format </li></ul></ul><ul><ul><li>coordinate system </li></ul></ul><ul><ul><li>raw data  </li></ul></ul><ul><ul><li>bundled up (zip, etc) </li></ul></ul><ul><ul><li>immediate delivery </li></ul></ul><ul><li>  </li></ul>
  3. 3. OGC Web Protocols <ul><li>All are HTTP web service protocols </li></ul><ul><li>  </li></ul><ul><li>WMS (Web Map Service) </li></ul><ul><ul><li>portrayal/visualization oriented  </li></ul></ul><ul><ul><li>Returns PNG/JPEG/GIF </li></ul></ul><ul><li>WFS (Web Feature Service) </li></ul><ul><ul><li>Raw GIS vector feature access. </li></ul></ul><ul><ul><li>Defaults to returning GML. </li></ul></ul><ul><li>  </li></ul><ul><li>WCS (Web Coverage Service) </li></ul><ul><ul><li>Raw Raster data access. </li></ul></ul><ul><ul><li>Returns GeoTIFF, etc. </li></ul></ul><ul><li>  </li></ul>
  4. 4. MapServer and GDAL/OGR <ul><li>GDAL/OGR </li></ul><ul><ul><li>multi format raster/vector data access </li></ul></ul><ul><ul><li>multi format raster/vector data writing </li></ul></ul><ul><ul><li>http://www.gdal.org/ </li></ul></ul><ul><li>MapServer </li></ul><ul><ul><li>Web map server </li></ul></ul><ul><ul><li>Supports WMS, WFS and WCS protocols </li></ul></ul><ul><ul><li>Uses GDAL/OGR for some data access and encoding </li></ul></ul><ul><ul><li>http://www.mapserver.org/ </li></ul></ul><ul><li>  </li></ul>
  5. 5. CorpsMap <ul><ul><li>MapServer  </li></ul></ul><ul><ul><li>OpenLayers  </li></ul></ul><ul><ul><li>GeoExt </li></ul></ul><ul><li>  </li></ul><ul><li>  </li></ul>
  6. 6. MapServer Implementation <ul><li>Map File: </li></ul><ul><ul><li>OUTPUTFORMAT declarations for output formats </li></ul></ul><ul><ul><li>enable WMS, WFS and WCS  </li></ul></ul><ul><ul><li>DUMP TRUE for underlying data access </li></ul></ul><ul><ul><li>attribute access and type details </li></ul></ul><ul><ul><li>declare supported coordinate systems </li></ul></ul><ul><li>  </li></ul>
  7. 7. Metadata to allow Export  and Support Projections <ul><li>WEB </li></ul><ul><li>    ...... </li></ul><ul><li>    METADATA </li></ul><ul><li>        &quot;ows_title&quot; &quot;mywfs&quot; </li></ul><ul><li>        &quot;ows_enable_request&quot; &quot;*&quot; </li></ul><ul><li>        &quot;ows_srs&quot; &quot;EPSG:4326 EPSG:26915 &quot; </li></ul><ul><li>        &quot;wfs_getfeature_formatlist&quot; &quot;Shapefile,FileGDB,geojson&quot; </li></ul><ul><li>    END </li></ul><ul><li>END </li></ul><ul><li>  </li></ul>
  8. 8. Including Attributes <ul><ul><li>Controlled via gml_include_items and gml_exclude_items </li></ul></ul><ul><ul><li>Separate multiple columns with commas </li></ul></ul><ul><ul><li>Use the special keyword all with the gml_include_items. </li></ul></ul><ul><li>Example: METADATA   &quot;gml_include_items&quot; &quot;all&quot;    &quot;gml_exclude_items&quot; &quot;my_sensitive_column,other&quot;    END </li></ul><ul><ul><li>Default behavior is to expose no attributes at all </li></ul></ul>
  9. 9. Attribute Level Controls <ul><ul><li>gml_ itemname _alias </li></ul></ul><ul><ul><li>gml_ itemname _width </li></ul></ul><ul><ul><li>gml_ itemname _precision </li></ul></ul><ul><ul><li>gml_ itemname _type </li></ul></ul><ul><ul><ul><li>Integer, Real, Character,Date, Boolean </li></ul></ul></ul><ul><li>  </li></ul><ul><li>or </li></ul><ul><ul><li>&quot;gml_types&quot; &quot;auto&quot; </li></ul></ul><ul><ul><ul><li>defines all field types from source </li></ul></ul></ul><ul><ul><ul><li>OGR, Oracle, PostGIS and Shapefile </li></ul></ul></ul><ul><ul><li>Default is everything is a string! </li></ul></ul>
  10. 10. Geometry Type <ul><li>gml/ows/wfs_geomtype (Optional) </li></ul><ul><ul><li>Geometry, Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon, GeometryCollection or None </li></ul></ul><ul><ul><li>Also 2.5D version - Point25D, LineString25D, ... </li></ul></ul><ul><ul><li>Defaults to 2D based on layer type </li></ul></ul><ul><ul><li>Used for GML and OGR output </li></ul></ul>
  11. 11. OutputFormat Declaration <ul><li>Describes a OGR Format including </li></ul><ul><ul><li>layer creation options (LCO:*) </li></ul></ul><ul><ul><li>dataset creation options (DSCO:*) </li></ul></ul><ul><ul><li>form </li></ul></ul><ul><ul><ul><li>Simple / Multipart / Zip </li></ul></ul></ul><ul><ul><li>  storage </li></ul></ul><ul><ul><ul><li>  Memory / Filesystem / Stream </li></ul></ul></ul><ul><ul><li>filename (default is result.dat) </li></ul></ul>
  12. 12. Example Output Formats <ul><li>OUTPUTFORMAT </li></ul><ul><li>  NAME &quot;Shapefile&quot; </li></ul><ul><li>  DRIVER &quot;OGR/ESRI Shapefile&quot; </li></ul><ul><li>  FORMATOPTION &quot;STORAGE=memory&quot; </li></ul><ul><li>  FORMATOPTION &quot;FORM=zip&quot; </li></ul><ul><li>  FORMATOPTION &quot;FILENAME=%outfile%.zip&quot; </li></ul><ul><li>  END </li></ul><ul><li>  </li></ul><ul><li>  OUTPUTFORMAT </li></ul><ul><li>  NAME &quot;FileGDB&quot; </li></ul><ul><li>  DRIVER &quot;OGR/FileGDB&quot; </li></ul><ul><li>  FORMATOPTION &quot;STORAGE=filesystem&quot; </li></ul><ul><li>  FORMATOPTION &quot;FORM=zip&quot; </li></ul><ul><li>  FORMATOPTION &quot;FILENAME=%outfile%.gdb.zip&quot; </li></ul><ul><li>  END </li></ul><ul><li>  </li></ul>
  13. 13. More Examples  <ul><li>OUTPUTFORMAT </li></ul><ul><li>  NAME &quot;geojson&quot; </li></ul><ul><li>  DRIVER &quot;OGR/GEOJSON&quot; </li></ul><ul><li>  MIMETYPE &quot;application/json; subtype=geojson&quot; </li></ul><ul><li>  FORMATOPTION &quot;STORAGE=stream&quot; </li></ul><ul><li>  FORMATOPTION &quot;FORM=SIMPLE&quot; </li></ul><ul><li>END </li></ul><ul><li>OUTPUTFORMAT </li></ul><ul><li>  NAME &quot;OGRGML&quot; </li></ul><ul><li>  DRIVER &quot;OGR/GML&quot; </li></ul><ul><li>  FORMATOPTION &quot;STORAGE=memory&quot; </li></ul><ul><li>  FORMATOPTION &quot;FORM=zip&quot; </li></ul><ul><li>  FORMATOPTION &quot;FILENAME=%outfile%.gml.zip&quot; </li></ul><ul><li>END </li></ul>
  14. 14. Even More Examples  <ul><li>OUTPUTFORMAT </li></ul><ul><li>  NAME &quot;MIDMIF&quot; </li></ul><ul><li>  DRIVER &quot;OGR/MapInfo File&quot; </li></ul><ul><li>  FORMATOPTION &quot;STORAGE=filesystem&quot; </li></ul><ul><li>  FORMATOPTION &quot;FORM=multipart&quot; </li></ul><ul><li>  FORMATOPTION &quot;DSCO:FORMAT=MIF&quot; </li></ul><ul><li>  FORMATOPTION &quot;FILENAME=result.mif&quot; </li></ul><ul><li>END </li></ul><ul><li>OUTPUTFORMAT </li></ul><ul><li>  NAME &quot;CSV&quot; </li></ul><ul><li>  DRIVER &quot;OGR/CSV&quot; </li></ul><ul><li>  MIMETYPE &quot;text/csv&quot; </li></ul><ul><li>  FORMATOPTION &quot;LCO:GEOMETRY=AS_WKT&quot; </li></ul><ul><li>  FORMATOPTION &quot;STORAGE=filesystem&quot; </li></ul><ul><li>  FORMATOPTION &quot;FORM=simple&quot; </li></ul><ul><li>  FORMATOPTION &quot;FILENAME=result.csv&quot; </li></ul><ul><li>END </li></ul>
  15. 15. Make Request via WFS <ul><li>http://server/cgi-bin/mapserv6?map=mymapfile.map </li></ul><ul><li>&service=wfs&version=1.1.0&request=getFeature </li></ul><ul><li>&typename=myLayer&outputformat= Shapefile </li></ul>
  16. 16. Make a request for projected data   (requires WFS 1.1.0 or later) <ul><li>http://server/cgi-bin/mapserv6?map=mymapfile.map </li></ul><ul><li>&service=wfs&version=1.1.0&request=getFeature </li></ul><ul><li>&typename=myLayer &outputformat= FileGDB </li></ul><ul><li>&srsname=EPSG:26915 </li></ul>
  17. 17. Clip and Ship <ul><li>http://server/cgi-bin/mapserv6?map=mymapfile.map </li></ul><ul><li>&service=wfs&version=1.1.0&request=getFeature </li></ul><ul><li>&typename=myLayer &outputformat=Shapefile </li></ul><ul><li>&Filter= </li></ul><ul><li><Filter> </li></ul><ul><li>  <BBOX> </li></ul><ul><li>    <PropertyName> </li></ul><ul><li>      <Name>NAME</Name> </li></ul><ul><li>    </PropertyName> </li></ul><ul><li>    <Box srsName='EPSG:4326'> </li></ul><ul><li>      <Coordinates>-95.844,40.253 -95.384,40.483</coordinates> </li></ul><ul><li>    </Box> </li></ul><ul><li>  </BBOX> </li></ul><ul><li></Filter> </li></ul>
  18. 18. Validation for Filename Substitution <ul><li>WEB </li></ul><ul><li>    .... </li></ul><ul><li>      METADATA </li></ul><ul><li>      ... </li></ul><ul><li>      END </li></ul><ul><li>      </li></ul><ul><li>      VALIDATION </li></ul><ul><li>          outfile &quot;.&quot; </li></ul><ul><li>          default_outfile &quot;result&quot; </li></ul><ul><li>      END </li></ul><ul><li>END </li></ul>
  19. 19. Request with Filename <ul><li>http://server/cgi-bin/mapserv6?map=mymapfile.map </li></ul><ul><li>&service=wfs&version=1.1.0&request=getFeature </li></ul><ul><li>&typename=myLayer &outputformat= Shapefile </li></ul><ul><li>&outfile=myshapefile </li></ul>
  20. 20. Other Clip and Ship Options <ul><ul><li>GeoServer has roughly similar WFS output extensions </li></ul></ul><ul><ul><li>Suggestions from audience? </li></ul></ul>
  21. 21. Lessons Learned <ul><ul><li>No temporary files (for some formats) </li></ul></ul><ul><ul><li>Only method to limit download size is maxfeatures </li></ul></ul><ul><ul><li>No method to limit extent of data download </li></ul></ul><ul><ul><li>Every SRS supported for download needs to be enumerated </li></ul></ul>
  22. 22. Conclusion <ul><li>Docs are available at  </li></ul><ul><ul><li>OGR Output - http://mapserver.org/output/ogr_output.html </li></ul></ul><ul><ul><li>WFS Server - http://mapserver.org/ogc/wfs_server.html </li></ul></ul><ul><li>Supported by MapGears                www.mapgears.com </li></ul><ul><li>US Army Corps of Engineers  </li></ul>
  23. 23. CorpsMap Viewer  OpenLayers / GeoExt / MapServer 6 / Oracle based Web Viewer
  24. 24. Exporting a data layer
  25. 25. Shapefile Export
  26. 26. Zipped Shapefile Generated http://maps.crrel.usace.army.mil:7778/cgi-bin/mapserv6?map=/htdocs/cm2_dev/cm2_test.map &service=wfs&version=1.1.0&request=getFeature&outputformat=Shapefile&outfile=NOAA_Streamgages &Filter=<Filter><BBOX><PropertyName><Name>NAME</Name></PropertyName> <Box srsName='EPSG:4326'><coordinates>-111.626587,36.998444 98.673706,40.992601</coordinates> </Box></BBOX></Filter>&SRSNAME=EPSG:4326&typename=ahpsobsl

×