The document discusses optimizing web-based geospatial services for serving large LiDAR datasets. It describes how the Oregon Department of Geology and Mineral Industries (DOGAMI) has collected over 16 million acres of LiDAR data totaling over 40 TB. Due to bandwidth limitations, DOGAMI explored techniques like mosaicking datasets by county and projecting data to reduce file sizes by 50%. Caching, definition queries, generalization, and tuning ArcGIS Server settings were also used to optimize performance. The results were improved web presence and public access to DOGAMI's data through their website and ArcGIS Online.

1. Optimizing Web-based
Geospatial Services
Tuesday, October 2nd , 2012

2. Serving up 33TB of Lidar Data
Through a Small Pipe
Paul Ferro
GIS Analyst | Web Services
Oregon Department of Geology & Mineral Industries
DOGAMI

3. • The Oregon Department of Geology and Mineral
Industries (DOGAMI) is the administrator of the
OLC and works with local, state, and federal
agencies to pool financial resources to fund lidar
acquisition in the state of Oregon
• Since 2008 we have acquired just under 16
million acres
• Distribute via DVD that includes BE, HH, Intensity
images
Agency Background:

4. Bare Earth
Highest Hit

5. Intensity
Dark surfaces (asphalt roads) absorb more light
than brighter surfaces (center line of road).

6. Funding Partners
The City of
Philomath
The City
of Turner

7. Update on Lidar Projects

8. Background on Web Services
• DOGAMI migrated to ArcGIS Server 10
• Still had some old Web Sites built on the old 9.2 adfs.
(application developer framework)
• ESRI will no longer support old .adfs moving forward

10. •Customizable, Widgets,
Stability
•API most supported by
ESRI
•Custom Widget Gallery
Flex
•Simplified UX
•High Performance
•Compact JS library
JavaScript/HTML5
•Cross browser, cross-
platform
•Highly Interactive apps
•Visually rich
Silverlite
•MapServer
•GeoServer
•Open Layers
•Sencha
•MapBox/TileMill
Open Source
•For Organizations
•Mobile Enabled
•Web Templates
•GeoIQ part of
ArcGIS.com team
ArcGIS.com

11. • Using versions of Flexviewer 2.0 – 2.5
• The ArcGIS Viewer for Flex is architected to help develop and deploy focused
web mapping applications that can fully leverage the power of server side spatial
services – ArcGIS Server, ArcGIS online, WMS, REST Services, Bing MapsFlex
The Flex Viewer application follows the basic
flow of any Flash application.
1. The wrapper file (usually index.html or
default.htm) is loaded by the browser.
2. It checks for appropriate version of Flash
Player.
3. Loads the SWF file (onto client browser) in
Flash Player which starts the Flexviewer
4. API makes the call to REST Service urls

12. The ArcGIS Server REST API, short
for Representational State Transfer,
provides a simple, open Web
interface to services hosted by
ArcGIS Server.
Map Services offer access to
basemaps and operational layers
2 Types:
 Cached
 Dynamic
Several Ways
to consume
Web
Services:

13. Breaking down the Flexviewer Config.xml
<basemaps>
<UI elements>
<operationallayers>
<sublayers>

14. Config.xml
<widgetcontainer>

15. Difficulties serving up massive Lidar datasets
• DOGAMI has collected ~ 16 million acres of
Lidar data at 8pts/m2
 ~ 514 trillion points
 > 40 TB of data
 Expected to increase by 15
TB by 2013

16. • DOGAMI has collected ~ 16 million acres of Lidar data at
8pts/m2
 ~ 514 trillion points
 > 40 TB of data
 Expected to increase by 15 TB by 2013
• Bandwidth issues
 State operating on 1/9 of T9 line
10 Meg Max.
 Small pipe for a tremendous amount of data

17. • DOGAMI has collected ~ 16 million acres of Lidar data at 8pts/m2
 ~ 514 trillion points
 > 40 TB of data
 Expected to increase by 15 TB by 2013
• Bandwidth issues
 State operating on 1/9 of T9 line
10 Meg Max.
 Small pipe for a tremendous amount of data
• DOGAMI’s Lidar service is not open to the public
 This service alone accounts for 86% of web traffic
 Exploring other hosting options

19. • Decided to mosaic DEM’s by County
 Resampled 3ft to 9ft grids -reduced our files size by a 1/3 rd
 Each stored it’s own FileGDB.
 Published .msds (map service definition)
 ESRI’s optimized map drawing engine
 Created Map Services
• Web requires WGS_1984_Web_Mercator_Auxiliary_Sphere
coordinate system, if you want to match ESRI Basemap
tiling schemes:
 Projecting adds a 1/3rd
 Net file size savings ~ 50% or more

20. Lidar Processing Techniques

21. Use Definition Queries
156, 186 Total records
Hide Fields
Total records served 81, 854

22. Generalize/Simplify your data

23. Tuning ArcGIS Server

24. Instances of a pooled service can be
shared between multiple application
sessions. Not pooled dedicated to one.
-i.e. Instance is used for the time it takes
to process a address locator request
When an application asks the server object
manager (SOM) for an instance of that
service, it gets a reference to one of the
instances.
Editing services 1:1
Support more users
4 core = 16 server instances
The amount of time between when a client
gets a reference to a service and when it
releases it is the usage time.
Idle instance consumes server memory
Check your sever logs and statistics and fine
tune your settings appropriately.

25. Cache performance considerations
• Minimize file size of the cache by choosing
appropriate output image format
 Lossy compression can result in fuzzy text
 Lossless can result in large size for tiles

26. Will cache
tiles around
the area
that client is
viewing
Uncheck if
adding tiles,
or updating
service
frequently

27. Caching
• 192 GB of cached data
• Use a feature class to define tiles to update
– Update with feature class option

28. • Greater web presence for DOGAMI- Deployed
over half dozen web sites
• Great resource for the public to view DOGAMI
Lidar and Hazard Data
• www.oregongeology.org
• http://www.oregongeology.org/sub/pub&data/i
nteractivemaps.htm
Results

29. DOGAMI Content on ArcGIS.com
• Use in ArcMap, your APIs, make your own web map

30. DOGAMI Content on ArcGIS.com

31. OpenTopography.org – Resource to
download Lidar data

32. http://www.csc.noaa.gov/digitalcoast/
data/coastallidar

33. www.oregongeology.org

34. Search for DOGAMI on
ArcGIS.com

35. DOGAMI Lidar Viewer

36. Flexviewer V 2.5.1
• Built in Transparency, Descriptions for Layer, Pop-ups

37. Landslide Inventory Maps
USGS map, 1994: 7 landslides
Landslide Inventory Map from lidar: 93 landslides

38. Web Solution for Rural Communities
with little GIS Support
FEMA Project

39. Flood Hazard Mapping

40. Custom Header
• Done in index.html
• Write some CSS in between the
<style>
</style> tags

42. Email yourself the link