Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

GDAL & WebGL: European Commission, Joint Research Center, Ispra

Research & Development with WebGL, Closure, GDAL & MapRank.

  • Login to see the comments

GDAL & WebGL: European Commission, Joint Research Center, Ispra

  1. 1. Joint Research Centre - European Commision, November 22, Ispra, Italy. Research & Development withWebGL, Closure, GDAL & MapRank Petr Pridal <petr.pridal@klokantech.com>
  2. 2. WebGL Earth Closure Tools 3D digital globe. JavaScript Toolset.MapTiler Cluster GDAL2Tiles on steroids. GeoSearch with ranking.
  3. 3. ! beta WebGL EarthFree and open source 3D digital globe for web and mobile devices
  4. 4. HTML5 & WebGL• HTML5 <canvas> WebGL extension• Based on OpenGL ES KLOKAN TECHNOLOGIES - Museo 300 • Supported by NVidia, ATI, Intel on desktop 500 KLOKAN TECHNOLOGIES - Museo KLOKAN TECHNOLOGIES - Museo 700 • HW acceleration on mobile devices • Firefox 4+, Chrome 9+, Opera, Safari • Internet Explorer: with Chrome Frame
  5. 5. WebGL: Shaders in GLSL KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700
  6. 6. KLOKAN TECHNOLOGIES - Museo 300KLOKAN TECHNOLOGIES - Museo 500KLOKAN TECHNOLOGIES - Museo 700
  7. 7. KLOKAN TECHNOLOGIES - Museo 300KLOKAN TECHNOLOGIES - Museo 500KLOKAN TECHNOLOGIES - Museo 700
  8. 8. KLOKAN TECHNOLOGIES - Museo 300KLOKAN TECHNOLOGIES - Museo 500KLOKAN TECHNOLOGIES - Museo 700
  9. 9. KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700Inspired by OpenLayers.Layer.GoogleNG (done by Andreas Hocevar) Compliant to Google TOS?
  10. 10. Try the demo application: KLOKAN TECHNOLOGIES - Museo 300 www.webglearth.com KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700
  11. 11. Mission of the project• User friendly (dragging, zooming, ...)• Keep it simple & maximally web oriented KLOKAN TECHNOLOGIES - Museo 300• Raster data from prerendered tiles, WMS, - Museo 500 KLOKAN TECHNOLOGIES KLOKAN TECHNOLOGIES - Museo 700 existing tile APIs and TileCache, GeoServer...• “NoServer” = o ine usage & cloud• Calculations from CPU to GPU. Preprocess!• Open-Source (GPLv3), Community powered
  12. 12. Open Source project• Simple JavaScript API • Embed <script src=“...../api.js”> • Mashup with your own tiles, custom WMS, 500 KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo overlays, markers and popups TECHNOLOGIES - Museo 700 KLOKAN• The full codebase + demo app sources on GitHub, object oriented, documented code, manual describing internals available• Contributors: KlokanTech, CampToCamp, AI2 Spain, BIMServer.org, NOAA
  13. 13. JavaScript API: Demo KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700
  14. 14. JavaScript API• Embedding of the 3D globe• Custom base maps and overlays with adjustable transparency (tilesKLOKAN TECHNOLOGIES - Museo 300 + WMS) KLOKAN TECHNOLOGIES - Museo 500• Camera API, flyTo, bbox KLOKAN TECHNOLOGIES - Museo 700• User interaction and events• Markers & popups• <canvas> is part of HTML DOM
  15. 15. Codebase: we, weapp• Demo application (www.webglearth.com) completely open-source, API exports too• Compiled with Closure Compiler with types: 300 KLOKAN TECHNOLOGIES - Museo KLOKAN TECHNOLOGIES - Museo 500 http://blog.klokantech.com/2010/12/closure-compiler-for-openlayers-3x.html KLOKAN TECHNOLOGIES - Museo 700• Documented (google style guideline)• Technically, anything is possible :-)• Manual describing the implementation details: http://data.webglearth.com/doc/
  16. 16. Object hierarchy KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700
  17. 17. “Keep it simple” design• Sphere with diameter 1.0• WGS84 coordinates used without ellipsoid transformation (a la Google Maps, OSM, etc.) KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700• Textures in Spherical Mercator (EPSG:3857, EPSG:900913)• Clip stack for megatextures with lookup• Segmented plane as the model
  18. 18. Segmented plane KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700
  19. 19. Terrain KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700
  20. 20. KLOKAN TECHNOLOGIES - Museo 300KLOKAN TECHNOLOGIES - Museo 500KLOKAN TECHNOLOGIES - Museo 700
  21. 21. KLOKAN TECHNOLOGIES - Museo 300KLOKAN TECHNOLOGIES - Museo 500KLOKAN TECHNOLOGIES - Museo 700
  22. 22. Vector data (direct rendering)• Now: rasterized only (WMS, tiles with Mapnik, ...). Raster tiles must be there for aerial data. KLOKAN TECHNOLOGIES - Museo 300• Future: direct rendering (from GeoJSON, KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700 KML, WFS, ...)• Sharing code for loading data formats with OpenLayers 3, Leaflet, Polymaps?• Vector Tiles - Google MapsGL
  23. 23. Future: Google MapsGL KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700
  24. 24. Future: Mobile devices KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700
  25. 25. Problems: drivers + CORS• Long list of blacklisted graphic cards• June 2011: Introduced restrictions on cross- domain textures for WebGL :-( KLOKAN TECHNOLOGIES - Museo 300 • CORS HTTP Header: Access-Control-Allow-Origin: * - Museo 500 KLOKAN TECHNOLOGIES KLOKAN TECHNOLOGIES - Museo 700 • http://enable-cors.org/ or chrome --disable-web-security • Support only in Chrome 15+ (beta), Firefox 8 (11/2011)• Workaround for now: HTTP proxy• Tiles hosted on your own server runs always!
  26. 26. Try it: 10 minut cookbook• JavaScript API: check examples• Prepare data (one of): KLOKAN TECHNOLOGIES - Museo 300 • WMS / TMS / WMTS in EPSG:900913 - Museo 500 KLOKAN TECHNOLOGIES KLOKAN TECHNOLOGIES - Museo 700 • Convert any GeoTIFF with GDAL2Tiles / MapTiler• Copy everything on your web server• Display in any modern web browser
  27. 27. Try it at: http://www.webglearth.com/Project website, mailing-list, documentation: http://www.webglearth.org/ KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700 Plans for integration with OpenLayers 3.0 Support, open-source contracted development: Klokan Technologies: http://www.klokantech.com/
  28. 28. WebGL Earth Closure Tools 3D digital globe. JavaScript Toolset.MapTiler Cluster GDAL2Tiles on steroids. GeoSearch with ranking.
  29. 29. "Closure ToolsOpenSource JavaScript Toolset developed by Google Inc.
  30. 30. " Compiler LibraryTemplates Linter Stylesheets JS Test (Coding Style Guidelines) http://code.google.com/closure/
  31. 31. It is all about compiler...1. Compiles readable JavaScript into compressed machine-readable JavaScript: dead code removal, function inlining, etc. KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 5002. JSDoc Tags: typos, wrong use of a @constructor, KLOKAN TECHNOLOGIES - Museo 700 wrong type of a variable, misuse of a field annotated with @private and @protected, etc.3. A reusable JavaScript library, such as OpenLayers, exports public API - and the compiler optimizes the internal code. http://goo.gl/H6twe
  32. 32. It is all about compiler...4. Unused parts of the library stripped. Application dependencies solved during compilation.5. Constants to remove unwanted functionality: - Museo 300 KLOKAN TECHNOLOGIES Mobile WebKit only, no Quirks mode support, - Museo 500 KLOKAN TECHNOLOGIES KLOKAN TECHNOLOGIES - Museo 700 compilation without support of IE66. Debugging with FireBug possible even for the compiled version7. Dynamically loadable modules, additional functionality loaded only when required. http://goo.gl/H6twe
  33. 33. Hello worldgoog.require(goog.dom);function sayHi() { KLOKAN TECHNOLOGIES - Museo 300  var newHeader = goog.dom.createDom(h1, TECHNOLOGIES - Museo 500 KLOKAN {style: KLOKAN TECHNOLOGIES - Museo 700background-color:#EEE},    Hello world!);  goog.dom.appendChild(document.body, newHeader);}goog.exportSymbol(sayHi, sayHi);
  34. 34. How to start?• Easiest is to get plovr - where all the tools are packed into one .jar application.• Run java -jar KLOKAN TECHNOLOGIES - Museo 300 plovr.jar serve config.json and use KLOKAN TECHNOLOGIES - Museo 500 it with HTML as <script src="http://localhost: - Museo 700 KLOKAN TECHNOLOGIES 9810/compile?id=hello"></script>• Plovr: Hello world + skeletons at GitHub: https://github.com/klokantech/closure-library-plovr-hello-world-skeletons/• Closure Library websites: http://code.google.com/closure/library/
  35. 35. Where it excels?• Complex online applications - any serious work done in JS• More developers working on KLOKANsame code 300 the TECHNOLOGIES - Museo KLOKAN TECHNOLOGIES - Museo 500• KLOKAN TECHNOLOGIES - Museo 700 High peformance app + mobile apps• Libraries and APIs• :-( no fast copy & paste hacking• :-( steep learning curve, smaller community
  36. 36. Read more at: http://code.google.com/closure/ Plovr.jar - the “packed” toolset: KLOKAN TECHNOLOGIES - Museo 300 http://plovr.com/KLOKAN TECHNOLOGIES - Museo 500 KLOKAN TECHNOLOGIES - Museo 700The book: Closure Library, The Definitive Guide Blog post: http://goo.gl/H6twe
  37. 37. WebGL Earth Closure Tools 3D digital globe. JavaScript Toolset.MapTiler Cluster GDAL2Tiles on steroids. GeoSearch with ranking.
  38. 38. #MapTiler Cluster GDAL2Tiles on steroids.
  39. 39. MAPTILER - WHAT IS IT?• MapTiler is an application for rapid online map publishing• Input: raster geodata (GeoTIFF, JPEG2000, MrSID, ECW, Erdas HFA, BSB, ...). Output: jpeg/png tiles• Support for existing georeference (World Files, WKT, EPSG,...). Transformation (warping) of projections possible.• Standards: OSGEO Tile Map Service Specification - TMS, comming OGC WMTS: • Web viewers (OpenLayers, Google Maps API) + applications • KML files (Google Earth) • iPhone + iPad + Android native applications • ArcGIS - with extensions (ArcBruTile). 10.1+ native WMTS support
  40. 40. GOOGLE MAPS OVERLAY DEMO
  41. 41. OPENLAYERS DEMO
  42. 42. GOOGLE EARTH DEMO
  43. 43. MAP TILER GENERATES TILES:
  44. 44. NECESSARY TRANSFORMATIONS AND WARPINGGEODETIC COORDINATES SPHERICAL MERCATOR PYRAMID COORDINATES TILE INDEX LATITUDE LONGITUDE METERS XYZ PIXELS / ZOOM XYZ TILE / ZOOM WGS 84 (EPSG:4326) EPSG:3785 / EPSG:900913 WEB VIEWERS TILE MAP SERVICE
  45. 45. SPHERICAL MERCATOR: TILE PROJECTION AND COORDINATES NOT ONLY IN GOOGLE MAPS!• Popular on the web: Google Maps, Microsoft Bing Maps, Yahoo Maps, OpenStreetMap, ...• Lat/Long WGS84 (GPS devices) datum coordinates always displayed to users - KML• Map rasters are projected by a global Mercator projection: “Spherical Mercator”. EPSG:900913 / EPSG:3857 (meters)• Conformal, cylindrical. Cropped to square (maximal latitude ~ 85.05 degrees), WGS84 ellipsoid is sphere!
  46. 46. OPEN-SOURCE ALTERNATIVES: PREGENERATING X CACHING TILES• Generating from raster input: • Caching tiles from WMS servers (MapServer, GeoServer): • MapTiler & GDAL2Tiles • TileCache.org (Python) • MSR MapCruncher • GeoWebCache.org (Java Servlet)• Generating from features (vectors) • Mapnik • GMapCreator, • Commercial alternatives • ... • License per server (ArcGIS Server)
  47. 47. EXTREMELY FAST ONLINE MAPS FOR THOUSANDS OF PEOPLE• Easy hosting: only static files / or raster blobs in a database• Extremely fast delivery, highest-quality output possible (you have more time for rendering), SLA for up to 99.99% availability
  48. 48. Our software is used ...• ... in: web applications, mashups, native mobile applications (iPhone/iPad/Android), hardware appliances and 3D visualizations• ... for: standard base maps, aerial imagery, KLOKAN TECHNOLOGIES - Museo 300 KLOKAN TECHNOLOGIES - Museo 500 coverage maps, thematic maps, marine KLOKAN TECHNOLOGIES - Museo 700 navigation, by pilots in airplains, on the ground during disaster events, ...• ... by: Google, NOAA, NASA, CERN, Vodafone, T-Mobile, Orange, Three, United Nations, and thousands of developers
  49. 49. PARALLELIZED RENDERING: MAPTILER CLUSTER• Parallelized rendering on Multi-Core processors or even on computer clusters (Amazon EC2 supported) • Speed-up the rendering on modern computers • Rendering on a computer cluster (Amazon EC2) - price: 0.1 USD / hour / core • Publishing of tiles on a world-wide CDNs (Amazon S3/CloudFront, Google App Engine, Azure, SimpleCDN) - fast multi-threaded upload for tiles possible• Merging of huge raster datasets into a seamless map a la Google Maps on a computer cluster: • Fast parallelized rendering and merging of large data collections s nt • Cluster parallelization based on MPI lie rc • Test of MapReduce approach (via Nokia labs Disco project) fo
  50. 50. MapTiler Cluster vs GDAL2Tiles• Fast rendering: multi-core CPU & MPI cluster• Direct merging of several input files• Excellent visual quality• Heavily optimized PNGs/JPEGs tiles to speed up the delivery and save disk space• Precise "data over-zooming"• Various coordinate systems supported s nt lie rc fo
  51. 51. WWW . MAPTILER . ORGRuns under Windows, Linux and Mac OS X + GDAL2Tiles You can try it on the FOSS4G LiveDVD! Community: MapTiler User Group maptiler@googlegrups.com MapTiler Cluster, commercial support, integration, customization http://www.maptiler.com/
  52. 52. WebGL Earth Closure Tools 3D digital globe. JavaScript Toolset.MapTiler Cluster GDAL2Tiles on steroids. GeoSearch with ranking.
  53. 53. $Geosearch with ranking.
  54. 54. GEOGRAPHICAL SEARCHING • Research, scientific articles, work on Ph.D. thesis (Petr Přidal), Computer Science + Geodesy and Cartography • Alternative projects tested in OldMapsOnline.org (R&D project in MZK): • Lucene with GeoTemporal Ext. (LGTE) from DigMap.eu • Alexandria Digital Library • TimeMap.net (ECAI.org) • Google GeoSearch API • Knowledge of traditional GIS search methods (PostGIS, ...)Result: Development of a new geographical search engine for map collections.
  55. 55. CORE FEATURES• Map-driven interface: Simple. Intuitive. Web 2.0. Users love it!• Combined queries: Where, When, Who & What• Results sorted geographically with an advanced spatial ranking algorithm.• Instant search. Immediate delivery of the results by moving the map or typing.• It is fast. Even for large databases (million+ records).
  56. 56. METADATA & CUSTOMIZATION• Indexing metadata in MARC, DublinCore, ISO 19139 formats, ...• Batch import or harvesting: OAI-PMH or OGC CWS• Numerical geographical boundaries necessary! (MARC 034, DublinCore DCMI) - enrichment with Geoparser or Georeferencer tools• Possible customization of the user interface, integration into products or websites• For developers: REST API (JSON format), queries following OpenSearch.org
  57. 57. Try it at: http://demo.mapranksearch.com/
  58. 58. Try it at: http://rumsey.mapranksearch.com/
  59. 59. Try it at: http://mapy.mzk.cz/
  60. 60. TOUCH INTERFACE
  61. 61. MAPRANK RANKING & SCALABILITY• MapRank ranking: an advanced algorithm calculating similarity of geographical area of query and area covered by every document and combining it with map scale, time range, predefined categories and traditional full text search.• Data indexing implemented specifically to provide ranked real-time results even for large databases. Traditional R- Tree indexing or Lucene are failing.• Server component: FastCGI, C++• Default user interface developed with Closure JavaScript Library + Closure Compiler.
  62. 62. ENRICHMENT OF METADATAGeoparserText parsing and semi-automatic assignment ofbounding boxes to map records with detectablegeonames (gazetteer), known physical size and mapscale.GeoreferencerCrowdsourced georeferencing. Higher accuracy, moreoutputs beside geometadata such as visualization, 3Dview, comparing of maps, OGC WMS for use in GIS.
  63. 63. WWW.MAPRANKSEARCH . COM
  64. 64. Thank you! Questions, ideas? Cooperation on OpenSource and R&D?Coding week for OpenLayers3 & WebGL Earth?Interest in applications of MapTiler / MapRank? Petr Pridal <petr.pridal@klokantech.com>

×