Building A Map Caching Tile Server

  • 23,445 views
Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
23,445
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
305
Comments
0
Likes
12

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Building a Map Caching Tile Server
    • Using your own map in the Map API
    • GM/VE/YM tile scheme and projections
    • Cut tiles using ArcIMS or ArcMap
    • Optimization for fast cutting
    • Cache tiles with SQL server
    • Cache tiles using Web Server caching
    • Querying data behind tiles
  • 2. Replacing Online Map with your own map
    • Your GIS System has more accurate and up-to-date spatial data.
    • Most likely you have a mapping engine to produce maps, for example, ArcIMS or MapServer.
    • The most efficient way to serve your own map is replacing their map with yours.
    • Means generating maps according to the JavaScript API’s tile schema.
  • 3. Map API’s support of tile layers
    • Google Map (as of 2.7) has the most comprehensive support. You can:
      • Create customized tile layer and add to existing map types, such as “map”, “satellite”, “hybrid”.
      • Add additional map types.
      • Completely replace the default map types.
      • Add a single map image on the view port.
    • Virtual Earth (as of 5.0) has limited support. You can:
      • Create customized tile layer and add on top of default maps.
      • Can NOT replace the default maps.
    • Yahoo AJAX Map (as of 3.7) does not support custom tile layer.
      • There is a undocumented feature that can be used to substitute default tile server.
  • 4. GMap, GMapType and GTileLayer
    • GMap is the Google map instance.
    • A GMap has a collection of GMapTypes, e.g. NORMAL, SATELLITE, HYBRID
    • A GMapType has one or more GTileLayer, e.g. HYBRID maptype has two Tile Layers: one street name and one satellite image.
    • A GTileLayer is a collection of small tiled images and stitch together to form a seamless View port.
    • The API use getTileUrl function to retrieve images.
  • 5. Using Custom Tile Layers In Google Map
    • The key function is getTileUrl(tile, zoom).
      • The API will pass in a tile context, which include tile coordinates x, y and zoom level.
      • The API will retrieve the image from the return the URL and position the image on view port.
      • All Default Maps (normal, satellite, hybrid) use same mechanism to retrieve images from Google’s tile server.
  • 6. Use Custom Tiles in Google Map
    • 1. Replace the default MapTypes’s default layers’ Image URL
      • G_NORMAL_MAP and G_SATELLITE _MAP has 1 tile layer, and G_HYBRID_MAP has 2 layers;
      • Simple swap the image URL with
        • G_NORMAL_MAP.getTileLayers()[0].getTileUrl=function (t , z){…}
      • All other features of the default map type, including copyright information are unchanged, which is not desirable.
      • Have minimal impact on other coding dealing with Markers etc.
      • Sample: GMap/1_replace_def_url.htm.
  • 7.
    • 2. Add custom Tile Layer on top of default Map Type
      • Setup up Copyright information
        • var myCopy =new GCopyrightCollection('Data ©2007');
      • Create a instance of GTileLayer
        • var myMapTileLayer = new GTileLayer( myCopy , 10, 18);
      • Override is getTileUrl(tile, zoom)
        • myMapTileLayer.getTileUrl=function(tile,zoom){..}
      • Add the tile layer into the default map type.
        • G_NORMAL_MAP.getTileLayers().push( myLayer );
      • Will still load images from Google, which may not be necessary because it may be covered by your layer.
      • Code Sample: GMap/2_add_on_top.htm
    Use Custom Tiles in Google Map
  • 8.
    • 3. Replace Tile Layers in default Map Type
      • Create a custom tile layer.
        • var myCopy =new GCopyrightCollection('Data ©2007'); var myMapTileLayer = new GTileLayer( myCopy , 10, 18); myMapTileLayer.getTileUrl=function(tile,zoom){..}
      • Replace the tile layer in default map type.
        • G_NORMAL_MAP.getTileLayers() [0]= myMapTileLayer ;
      • Will NOT load images from Google. Can be placed in a external file or server script and have minimal impact on API coding.
      • Ideal for assisting non-GIS developers doing Google Map development.
      • Sample: GMap3_replace_def_layer.htm
    Use Custom Tiles in Google Map
  • 9.
    • 4. Create custom Map Type
      • Create a custom tile layer.
        • var myCopy =new GCopyrightCollection('Data ©2007'); var myMapTileLayer = new GTileLayer( myCopy , 10, 18); myMapTileLayer.getTileUrl=function(tile,zoom){..}
      • Create a new Map Type
        • var myMapType = new GMapType([myMapTileLayer], new GMercatorProjection(19), "MyMap");
      • If needed, remove default Map Types before add the new type
        • map.getMapTypes().length = 0;
      • Add the new Map Type to Map
        • m ap.addMapType(myMapType );
      • Sample: GMap4_custom_maptype.htm
    Use Custom Tiles in Google Map
  • 10.
    • 5. Using Tile Overlay
      • As overlay in all map types, should be transparent layer
      • Create a custom tile layer.
        • var myMapTileLayer = new GTileLayer( myCopy , 10, 18); myMapTileLayer.getTileUrl=function(tile,zoom){..}
      • Create a Tile Overlay.
        • var myTileOverlay=new GTileLayerOverlay(myMapTileLayer);
      • Add Overlay to Map, AFTER setCenter().
        • map.addOverlay(myTileOverlay);
      • Idea for non-basemap, special subject layers, such as bus routes..
      • Sample: GMap5_tileoverlay.htm
    Use Custom Tiles in Google Map
  • 11.
    • Virtual Earth API only allows custom tile on top of default layers
      • Default Tiles are loaded from Microsoft even they are covered.
      • Functions like GTileLayerOverlay in Google Map API.
      • Opacity can be defined to allow see-through default Maps, if necessary.
      • Use the same tiling schema as Google Map V2.x.
      • Have very good documentation on Tile Schema for V4.
      • The PNG8 transparency may not rendered correctly.
    Use Custom Tiles in Virtual Earth
  • 12.
    • Add Custom Tile Layer in Virtual Earth
    • Create a VETileSourceSpecification instance.
      • var tileSourceSpec = new VETileSourceSpecification("myMap", "GetTilePath");
    • Define GetTilePath (tileContext) function.
      • tileSourceSpec.GetTilePath=function (tile){ … }
    • Framework will pass tileContext with XPos, YPos, and ZoomLevel.
    • Add Tile Layer to map
      • map.AddTileLayer(tileSourceSpec, true);
    • Sample: VEMap/1_add_on_top.htm, 2_png8_problem.htm
    Use Custom Tiles in Virtual Earth
  • 13.
    • Yahoo’s Primary Map API is Flash based.
    • The AJAX API has less features compare to those of Google Map and Virtual Earth.
    • Does not officially support custom Tile Layer.
    • Useful if you plan to use YUI (Yahoo User Interface) as your front end development because it shares some common JavaScript library from Yahoo.
    Use Custom Tiles in Yahoo AJAX map
  • 14.
    • Use a slightly different tile schema than Google Map and Virtual Earth, use equator as 0 instead of latitude ~85, longitude 0.
    • The zoom level is reverse of GM2 and VE, 1 is most detail. 18 is global.
    • There is an undocumented method can be used to replace the default map tiles:
      • YMapConfig.setRegTile( … +"&tilename=basemap&");
      • YMapConfig.setSatTile( … +"&tilename=aerial&");
      • YMapConfig.setHybTile(tileServerBaseURL+"&tilename=hybrid&");
    • The URL is in the format of baseurl&x=&y=&z=
    • Sample: YMap1_replace_tile.htm
    Use Custom Tiles in Yahoo AJAX
  • 15. Understanding Tile System
    • Compare to traditional GIS web applications, the tile maps are pre-rendered according to a specific tile system, and the API retrieves the pre-rendered tile images and display in a view port.
    • Google Map, Virtual Earth and Yahoo Map uses a similar tile system with small variation.
    • The Same tile set can be used in all three APIs with simple conversion.
    • The basic idea is to prepare a list of fixed zoom levels, divide earth into small square tiles and create small images for each tile.
  • 16. Understanding Tile System
    • Microsoft has an excellent article about the tile system used in VE, available at
      • http://msdn2.microsoft.com/en-us/library/bb259689.aspx
    • Mercator Projection is used by all three vendorsm because it’s simple, fast, and preserve shape and direction. It is also relatively simple to implement in JavaScript.
    • The actual Coordinate system is Sphere_Mercator (53004) instead of World_Mercator (54004). Some small gaps between tiles maybe visible at smaller scale if the projection is not set correctly. See below.
  • 17. Understanding Tile System World Mercator (54004) Sphere Mercator (53004)
  • 18. Map Scales
    • The tile system started at zoom level 1, and the earth is rendered as 4 tiles, 256x256 each.
    • Each successive zoom level, the map width and height grew by factor of 2
    • At zoom level n, the map size is 256 * 2 ^ level pixels
    • Assuming DPI of 96, at latitude 0, the actual zoom scale can be calculated, and a set of commonly used map scale can be used as reference to set up the map engine for cutting tiles.
    • For most local governments, the relevant zoom level are from 10-19.
  • 19. Map Scales Source: http://msdn2.microsoft.com/en-us/library/bb259689.aspx , modified True Scale Zoom Level 1 : 1,128.50 1 : 2,257.00 1 : 4,514.00 1 : 9,028.00 1 : 18,055.99 1 : 36,111.98 1 : 72,223.96 1 : 144,447.93 1 : 288,895.85 1 : 577,791.71 19 18 17 16 15 14 13 12 11 10 Common Scale 1:2,000 1:4,000 1:5000 or 7500 1:10,000 1:20,000 1:50,000 1:100,000 1:200,000 1:500,000 1:1,000,000
  • 20. Pixel and Tile Coordinates
    • At each zoom level, the earth is rendered as a giant virtual image of (256*2^level)x(256*2^level). Each pixel will have a coordinate from 0 to the max size of the virtual image.
    • When converting Lat/Lon to pixel, the actual radius of the earth actually does not matter. It will be cancelled in the calculation in Mercator projection.
    • The virtual image is then split into 2^level number of tiles in each dimension.
    • The tile coordinates is pixel coordinates divided by tile size then take the floor part.
  • 21. Quadkeys
    • Quadkey is used by Virtual Earth to optimize indexing and storage of map tiles.
    • The length of the key is same as zoom level.
    • Each digit (0-3) at each level is decided by the tile’s position in it’s parent tile.
    Image Source: http://msdn2.microsoft.com/en-us/library/bb259689.aspx
  • 22. Cutting your Tiles
    • What you need:
    • 1) A Mapping Engine. It must be able to render a bitmap for a given specific geographic area, output the image in Sphere Mercator projection.
      • Examples include ArcIMS, UMN Map Server, Any WMS Server, and, ArcMap!
  • 23. Cutting your Tiles
    • 2) A Data Storage Repository. It must be able to store large amount of binary data and can be retrieved quickly.
      • Examples: File System, SQL Server (Express), Oracle, or any RDBMs that can handle binary format.
  • 24. Cutting your Tiles
    • 3) A Web Server and possibly some server code written in programming language of your choice, unless you choose file system as storage repository and do not want additional flexibility.
    • 4) Some code to run across the tile system and making request to Map engine.
  • 25. Cut Tiles: Define Interfaces and core classes
    • Common Interface, different implementations.
    • Two Interface is needed for a tile server:
      • ITileMap: to make a image based on a geographic area. The actual implementation can be ArcIMS or something else.
      • ITileCache: to store and retrieve images. The actual implementation can be file system or RDBMS.
    • Helper classes: Tiles, Images, Coordinate transformation classes.
  • 26. Cutting Tiles: Define Interfaces and core classes
  • 27. Cutting Tiles: Implementing Interfaces
    • Organize tools by technology type as “Resources”.
    • Each resource can implement one or more interfaces. For example, an ArcIMS resource can produce Map and perform query, and a SQL resource can store, retrieve image and possibly query. While a File resource can only store and retrieve image.
    • A particular tile set can mix and match its functionalities. For example, the base map tiles can use SQL server to store tiles, use ArcIMS to make tiles on the fly, and use MS Spatial for data query.
  • 28. Cutting Tiles: Implementing Interfaces
  • 29. A Simple Scenario
    • Use ArcIMS to make tile images.
    • Store Tile Images in a file system
    • Retrieve tile from the file system
  • 30. Use ArcIMS to make images
    • The Input coordinate system should be WGS Geographic Coordinate System, unless converted in the cutter.
    • The output image should be in Sphere Mercator Coordinate System (53004).
    • Reaspect must be set to false to avoid ArcIMS automatically adjust the envelopment if the filtercoordinate system is GCS 4326.
    • For vector layers, it maybe necessary to set a transparent background for PNG output types.
    • For Vector layers, PNG8 is recommended for smaller size. For raster, JPG is best.
    • Sample code: TileSystemResourceArcIMSArcIMSTileResource.cs
  • 31. Use ArcIMS to make images
    • public TileImage MakeTile(LatLngBounds bnds, int imageSize, string imageType, bool transparent) { … string axlReq = String.Format(&quot;<?xml version='1.0' encoding='UTF-8'?><ARCXML version='1.1'><REQUEST><GET_IMAGE><PROPERTIES><FEATURECOORDSYS id='53004' datumtransformid='1188' /><FILTERCOORDSYS id='4326' datumtransformid='1188'/><IMAGESIZE width='{0}' height='{0}'/> <ENVELOPE reaspect='false' minx='{1}' miny='{2}' maxx='{3}' maxy='{4}' /><LEGEND display='false' /><BACKGROUND color='254,254,254' {5} /><OUTPUT type='{6}'/></PROPERTIES></GET_IMAGE></REQUEST></ARCXML>&quot;, imageSize, bnds.SouthWest.Lng, bnds.SouthWest.Lat, bnds.NorthEast.Lng, bnds.NorthEast.Lat, transparent ? &quot;transcolor = '254,254,254'&quot; : &quot;&quot;, imageType); }
  • 32. Use File System to store images
    • Simple and can be checked directly.
    • The best storage file structure is TileSetNamez**x**y**.*
  • 33. Store and Retrieve Images from File System
    • Conversion between file directory and tile coordinate system
      • <name>z<>x<>y<>.type
    • Setup configuration about some metadata about the tile set, for example, type (png8, jpg etc)
    • Create directory tree as needed in cut process.
    • Using File IO API to read/write images from disk.
    • Sample code: TileSystemResourceFileSysFileTileResource.cs
  • 34. Build The Cutter
    • The Cutter will travel the tile system, make calls to map engine (implementing ITileMap) to create map, then call storage engine (implementing ITileCache) to store images.
    • First query the map engine for initial map extent, in Lat/Lng format.
    • Use Coordinate transformation tool to find the southwest and northeast corner's pixel coordinates, then convert to tile coordinates.
    • With the known boundary of tile coordinates, travel horizontally then vertically for all tiles in the range.
    • Repeat for each Zoom Level.
  • 35. Build The Cutter
  • 36. Config and Run Cutter
    • Config image type, tile size, transparency for tile set.
    • Config storage mechanism (cacheSource)
    • Config map maker (mapSource)
    • Set up additional parameters (zoom level, force refresh etc)
    • If the tile cache is file system image can be directly referenced in getTileUrl.
    • Code: TileCutter.cs
    • Check results: GMap6_filesystem_direct.htm
  • 37. Serving Image with Server side components
    • Serving images from a server component has several advantages over direct url reference to image file:
      • Meta data configuration to allow more flexibility on image type (png/gif/jpg etc)
      • Allow storage other than file system.
      • Possible more flexibility in server caching.
      • Allow more tile system(GM/VE/YM) conversion.
    • In ASP.NET, use a generic HTTP Handler.
    • In Java, use an HTTP Servlet.
  • 38. Serving Image with Server side components (ASP.NET)
    • Use similar tile set configuration in Tile cutter.
    • Perform tile system conversion if necessary.
    • Cache Tile set metadata.
    • Find the Cache source for the requested tile set.
    • Retrieve the actual image from the cache source and write to browser.
    • Code: TileServerMapTile.cs, GMap7_filesystem_handler.htm
  • 39. Store Tile Image In Database
    • SQL Express is a good option for moderate datasets (4GB limit).
    • Faster to move between systems( detach, attach)
    • More meta data options such as timeout rules etc.
    • Can use x, y, z index or Quadkey.
    • Tile Image store as Image type (or varbinary)
    • MDF file can be put into TileServer application’s App_Data folder, or register with SQLExpress using Management Studio.
    • The Cutter write bytes into a table, and tile server handler read from database and send to browser.
  • 40. Improving Tile Cutter Quality
    • Tuning Cartography.
    • Because the map will be pre-rendered, it is acceptable to use advanced cartographic features such as antialiasing.
    • Use scale dependent renderers for different zoom level, using the map scale table for reference.
    • Use multiple composite renderers to achieve outline effects for polyline feature class
    • Use multiple layers to make sure correct symbol order. e.g Freeway on top of local streets.
  • 41. Improve cutting performance
    • Only Google Map API supports tile size other than 256x256. Larger tile means less server requests but larger download size.
    • Having map engine render 256 tiles is not very efficient considering data IO, rendering etc.
    • For most map engine, rendering a 512 x 512, even 1024 x 1024 map takes only slightly longer time than rendering a 256x256 map.
    • Cut larger tiles then slice into smaller pieces will dramatically decrease the number of map images the map engine has to make. Cut 512 instead of 256 will need only about ¼ of map requests (the boundary may cause small variations)
  • 42. Improve cutting performance
    • Use Graphics API such as GDI+ or Java2D can slice images into smaller pieces.
    • PNG8 can cause smaller problem because the GDI+ will convert it to PNG24 and the saved file will lose transparency setting.
    • To fix the PNG8 problem, use the Image Color Quantization code from http://msdn2.microsoft.com/en-us/library/aa479306.aspx
    • The cutter uses larger steps when traveling in tile system, requesting larger images from map engine, then slice them, store in tile repository in normal size.
  • 43. Caching Tiles
    • Most web server or application framework has some caching mechanism.
    • Tiles can be cached in memory of web server and served directly if someone requested same tile within a timeout window.
    • ASP.NET has built-in HttpContext.Cache system, along with configurable CacheDependency, the tile server will have a better performance.
  • 44. Tile On-Demand
    • For less popular area, the tile can be rendered on demand and store in to the tile repository.
    • There are multiple level of caching:
      • If the browser can not find the tile or expired.
      • Server in-memory
      • Tile repository
      • Generate on demand then store the tile.
  • 45. Spatial Query/Identify
    • Push-pin or Markers are generally clickable in GM/VE/YM API.
    • The map itself can not be clicked, or “identified”
    • However, click events can be captured, along with information about the clicked point, including lat/lon, zoom level, map view point extent etc.
    • Those information can be passed via AJAX request to server and query the underline spatial data.
  • 46. Spatial Query/Identify
    • Normally a query engine can work off the same map service that produces the actual map image, in that case, the scale control on layer can be used to check which layer should be queried.
    • Identify function can be de-coupled from the map service, so additional tools and features can be used.
    • A Common interface ITileData can be defined and resources can implement the interface.
  • 47. Spatial Query/Identify
    • The query can be implemented by ArcIMS query server, or MS spatial, or ArcSDE API, or other spatially enabled RDBMS, such as PostGIS etc.
    • Attach click event to Google Map instance
    • Upon click, form an http request and send to server via XmlHttp
    • Perform spatial query based on clicked location, adjust to correct spatial reference system.
    • Display results in callback.
  • 48. Putting it together
    • An example that:
      • Replace the default Map type’s tile layer
      • Add a new map type for a new tile set.
      • Tile cut from an ArcIMS map service.
      • Store/retrieve map tiles in SQL express
      • Render on demand when the requested tile is not pre-rendered.
      • Identify map features from ArcIMS map service.