• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Advanced Map Caching Topics

Advanced Map Caching Topics



Sterling Quinn´s great presentation about map caching on Esri European User Conference 2011.

Sterling Quinn´s great presentation about map caching on Esri European User Conference 2011.



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Advanced Map Caching Topics Advanced Map Caching Topics Presentation Transcript

    • Advanced Map Caching TopicsTecnicas avanzadas de almacenamiento en cache de mapas Esri European User Conference 2011
    • Prerequisite knowledge • What is map caching? • How to author a map for caching • How to build map cache tiles
    • Topics in this session • What’s new in caching in ArcGIS 10 • Approaching a large caching job - Building a test caches - Choosing the tiles to cache - Monitoring the tile creation - Case study: State of Indiana imagery basemap • Cache update strategies
    • What’s new in caching in ArcGIS 10
    • Mixed mode image format• Creates JPEG tiles unless transparent pixels detected• If transparent pixels detected, creates PNG32• Enormous space savings for large caches Mixed format helps with this JPEG on JPEG cache overlay problem
    • How the mixed format creates tiles
    • Compact cache storage format • Stores tiles in compact, continuous file streams (“bundles”) - Maximum ~16,000 tiles per bundle • Faster copying • Smaller size on disk • Generally faster to create • Less fragmentation
    • Move spatial subsets of tiles between caches • New export and import tools allow collaborative and “best available” caching Export Import
    • Cache export tool • Export Map Server Cache - Based on extent or polygon features - Convert storage format - Use for cache import or as a disconnected cache Export Exported using Nevada and Utah state boundary features.
    • Cache import tool • Import Map Server Cache - Based on extent or polygon features - Must have same storage format Import Import from a previously exported map cache.
    • Cache import for imagery • Merge high resolution imagery into a base cache • Seamless integration import feature boundary
    • Collaborative caching • Use export and import tools - Import the “best available” cached tiles - Esri Community Base Maps program Pasadena City College Contribution
    • Caches treated as a raster dataset in ArcGIS • Add Data and browse to cache directory. - Looks like any other raster • Export caches for disconnected field work - Export to compact format recommended
    • Approaching a large caching job
    • What should you cache? • Base maps • Operational layers that satisfy one of the following: - High volumes of traffic - Don’t change often - Cover small scales only
    • Choosing scales Sample 10 level cache• Build just the scales you need• Determine closest scale (Raster resolution)• Divide scale by 2 for each subsequent scale• Adjust smallest scale to full extent Final level is ~75% of the total• Consider Web Mercator scales
    • Choosing an image format • Image format affects - Tile storage space requirements - Web application performance (speed and supported browsers) - Tile image quality - Tile transparency
    • Basemap: Image format guidelines • Fewer colors (~256) - PNG 8 • Many colors - JPEG - Small file size - Quality settings range from 55 (orthos) - 90 (vector maps)
    • Choosing JPEG quality • Which one looks better? JPEG 90 – 21KB JPEG 55 – 15KB JPEG 55 – 10KB JPEG 90 – 33KB
    • Operational layers: image format guidelines • Vector overlays (roads, boundaries) - PNG 32 because of good antialiasing • Other overlays (weather, etc) - MIXED or PNG (bit depth depends on number of colors)
    • Example: Tiles are too large Aerial photo and vector blend using PNG 32
    • Antialiasing: Often worth the cost
    • Apply antialiasing in the MSD for best performance 1. 2.Gives you this… Box already checked shows that antialiasing is happening through MSD ☺
    • Reducing duplicate labels• Caches drawn using supertiles• Duplicates can occur across supertile Supertile No antialiasing boundaries 4096x4096 Supertile• Solution is annotation antialiasing 2048x2048 512x512 tile size 256x256 tile size ArcGIS Online / Bing / Google
    • B u i l d i n g a t e s t c a c h e
    • How to build a test cache 1. Select test area with varying geography 2. Create a simple feature class covering the test area 3. Create tiles at all scale levels based on this feature class
    • Note the following from the test cache • Appearance of tiles • Performance of tiles in client • Cache creation time • Cache size on disk
    • Choosing the tiles to cache
    • Not all tiles are created equal http://hotmap.msresearch.us
    • Use GIS modeling to predict frequently visited areas This feature class covers 25% of California’s area but includes 97% of its population
    • Tiles you don’t want to create and store • Cache on demand OR • Insert a “Data not available” tileMake your own heatmap of tile usage: ArcGIS Server descriptive tile analysis
    • Caching based on a feature class • Features should be: - Big - Generalized
    • Why do the features need to be big?• How do I find out the bundle size? - Map Server Cache Tiling Scheme to Polygons tool - See Esri Dev Summit 2011 presentation online for full demo
    • Monitoring the tile creation
    • Setting the number of instances • Keep your CPU below 100% • Start with N+1 instances where N = the number of cores on server • Adjust number of instances based on test cache Task Manager Manage Map Server Cache Tiles 2 CPU’s 3 Instances
    • Generate using multiple LAN hosts • Configure instances for CPU’s on all SOC hosts • Enable “Use local cache directory…” Temp cache Shared SOC Temp Cache cache hosts SOM Temp cache
    • Use status tracking with your featuresHelpful script:Create gridded feature class for tracking ArcGIS Server map caching jobs
    • Recovering from errors • Generate rebuild features from error log - Convert map cache-build log failures to footprint polygons tool • Cache Validation tool - Looks for missing tiles - Looks for tiles below a certain size (white or gray tiles) - False positives with building roofs and water - Useful for validating source imagery
    • Case study: State of Indiana imagery basemap
    • Requirements • Single Indiana state basemap with most recent data - Maintain 6” resolution where available • ArcGIS Online/Bing/Google tiling scheme • Integrate neighboring data for small scales • Record metadata for imagery in basemap
    • Approach6" scale 576.00 20 Individual 6" counties 1,128.50 19 2,256.99 18 Use gridded tile 4,513.99 17 Web Mercator Tiling Scheme 9,027.98 16 18,055.96 15 36,111.91 14 Clip to state boundary 72,223.82 13 144,447.64 12 288,895.29 11 577,790.58 10 1,155,581.15 9 IN Full extent 2,311,162.31 8 4,622,324.61 7 9,244,649.23 6 18,489,298.45 5 36,978,596.91 4 73,957,193.82 3 Not Built 147,914,387.60 2 295,828,775.30 1 591,657,550.50 0
    • Stats on the final cache • Image format: JPEG 55 • 62,245,522 tiles • Total cache size 452.72 GB • Average tile size 7.63 KB • 3 weeks to build
    • C a c h e u p d a te s tr a te g ie s
    • Can you update the cache? • How current does the data need to be? • How fast can the update take place? • Can the update be scripted? • Esri Spain elections example
    • Strategic cache updates 1. Detect where changes have taken place 2. Merge these features 3. Use them to define the update area Helpful scripts •Show edits since reconcile •Compare two feature classes in a file geodatabase
    • Updating a cache using a staging server Staging ArcGIS Production ArcGIS Server Instance Server InstanceMap service Layers for TOC All layers for and Query cartography of map service Map serviceCache folder Copy tiles Cache folder (On-demand caching needs the full map to build the cache)
    • Updating a cache and data using a staging server Staging ArcGIS Production ArcGIS Server Instance Server Instance All layers for cartography of map service Layers for TOC Copy tiles and QueryMap service Map service Cache folder Cache folder Geodatabase ReplicationMulti-user versioned Geodatabase Geodatabase (File or Multi-user)
    • Q u e s t i o n s