OSGeo Conferences Report Jeff McKenna, OSGeo Board of Director
Lausanne, Switzerland 2006
Victoria BC, Canada 2007
Victoria BC, Canada 2007
Cape Town, South Africa 2008
Sydney, Australia – Oct 2009
Sydney, Australia – Oct 2009
FOSS4G 2010 Barcelona, Spain – 6 to 9 September
FOSS4G 2011
OSGeo Conference Future? <ul><li>Rely more on OSGeo Chapters to host regional events </li></ul>
OSGeo Japan Event - 2008
But What Happened at FOSS4G 2009?
WMS Benchmarking Exercise
Benchmarking Summary <ul><li>Compare the performance of WMS servers </li><ul><li>GeoServer
MapServer </li></ul><li>In a number of different workloads:  </li><ul><li>Vector: plain polygon, point with graphical icon...
Raster: global imagery </li></ul><li>Against different data backends:  </li><ul><li>Vector: shapefiles, PostGIS, Oracle sp...
Raster: ECW and GeoTiff mosaics </li></ul></ul>
Benchmarking History <ul><li>3 rd  FOSS4G benchmarking exercise. Past exercises included: </li><ul><li>FOSS4G 2007: Refrac...
FOSS4G 2008: OpenGeo run and published the second comparison with some review from the MapServer developers. Focus on simp...
Past MapServer improvements <ul><li>improvements in large shapefile indexing
raster read optimization (single pass for multiple bands)
enhancing polygon labelling placement
EPSG codes caching
PostGIS improvements
Oracle improvements </li></ul>
Past GeoServer improvements <ul><li>Overall rendering pipeline improvements
GML writing speed-ups
Raster data access improvements
PostGIS data store optimizations </li></ul>
Rules of engagement <ul><li>Each server is tested in its latest version
Each server performs exactly the same workload </li><ul><li>Same set of WMS requests
Same data backends
Same image output format </li></ul><li>All modifications made to improve performance are to be included in a future releas...
Data used cannot be modified for display, other than indexing
All testing to be done on the same benchmarking machines </li></ul>
Hardware Configuration Bench WMS Database JMeter GeoServer MapServer Shapefiles ECW GeoTIFF Oracle PostGIS SDE
Software Specs <ul><li>MapServer: </li><ul><li>5.6.0-beta3, GDAL SVN
Apache 2.2
FastCGI - 8 static processes </li></ul><li>GeoServer: </li><ul><li>2.0-RC2, Java 6 update18, GDAL 1.4.5
Tomcat 6.0.20 - 8 threads </li></ul><li>Database: </li><ul><li>Oracle 11.1.0.7
SDE 9.3.1
PostGIS 1.4.0 </li></ul></ul>
Hardware specs <ul><li>Bench (2003): </li><ul><li>Dell PowerEdge 1750
Upcoming SlideShare
Loading in …5
×

OSGeo Conferences Report

2,114 views

Published on

Presented at GeoLivre 2009 conference in Brasilia, Brazil. http://www.geolivre.com/

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,114
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
41
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

OSGeo Conferences Report

  1. 1. OSGeo Conferences Report Jeff McKenna, OSGeo Board of Director
  2. 2. Lausanne, Switzerland 2006
  3. 3. Victoria BC, Canada 2007
  4. 4. Victoria BC, Canada 2007
  5. 5. Cape Town, South Africa 2008
  6. 6. Sydney, Australia – Oct 2009
  7. 7. Sydney, Australia – Oct 2009
  8. 8. FOSS4G 2010 Barcelona, Spain – 6 to 9 September
  9. 9. FOSS4G 2011
  10. 10. OSGeo Conference Future? <ul><li>Rely more on OSGeo Chapters to host regional events </li></ul>
  11. 11. OSGeo Japan Event - 2008
  12. 12. But What Happened at FOSS4G 2009?
  13. 13. WMS Benchmarking Exercise
  14. 14. Benchmarking Summary <ul><li>Compare the performance of WMS servers </li><ul><li>GeoServer
  15. 15. MapServer </li></ul><li>In a number of different workloads: </li><ul><li>Vector: plain polygon, point with graphical icons, road network with cased lines, road network with labels
  16. 16. Raster: global imagery </li></ul><li>Against different data backends: </li><ul><li>Vector: shapefiles, PostGIS, Oracle spatial, SDE over Oracle spatial
  17. 17. Raster: ECW and GeoTiff mosaics </li></ul></ul>
  18. 18. Benchmarking History <ul><li>3 rd FOSS4G benchmarking exercise. Past exercises included: </li><ul><li>FOSS4G 2007: Refractions Research run and published the first comparison with the help of GeoServer and MapServer developers. Focus on big shapefiles, postgis, minimal styling
  19. 19. FOSS4G 2008: OpenGeo run and published the second comparison with some review from the MapServer developers. Focus on simple thematic mapping, raster data access, WFS and tile caching </li></ul><li>Friendly competition: goal is to improve all software </li></ul>
  20. 20. Past MapServer improvements <ul><li>improvements in large shapefile indexing
  21. 21. raster read optimization (single pass for multiple bands)
  22. 22. enhancing polygon labelling placement
  23. 23. EPSG codes caching
  24. 24. PostGIS improvements
  25. 25. Oracle improvements </li></ul>
  26. 26. Past GeoServer improvements <ul><li>Overall rendering pipeline improvements
  27. 27. GML writing speed-ups
  28. 28. Raster data access improvements
  29. 29. PostGIS data store optimizations </li></ul>
  30. 30. Rules of engagement <ul><li>Each server is tested in its latest version
  31. 31. Each server performs exactly the same workload </li><ul><li>Same set of WMS requests
  32. 32. Same data backends
  33. 33. Same image output format </li></ul><li>All modifications made to improve performance are to be included in a future release of the software
  34. 34. Data used cannot be modified for display, other than indexing
  35. 35. All testing to be done on the same benchmarking machines </li></ul>
  36. 36. Hardware Configuration Bench WMS Database JMeter GeoServer MapServer Shapefiles ECW GeoTIFF Oracle PostGIS SDE
  37. 37. Software Specs <ul><li>MapServer: </li><ul><li>5.6.0-beta3, GDAL SVN
  38. 38. Apache 2.2
  39. 39. FastCGI - 8 static processes </li></ul><li>GeoServer: </li><ul><li>2.0-RC2, Java 6 update18, GDAL 1.4.5
  40. 40. Tomcat 6.0.20 - 8 threads </li></ul><li>Database: </li><ul><li>Oracle 11.1.0.7
  41. 41. SDE 9.3.1
  42. 42. PostGIS 1.4.0 </li></ul></ul>
  43. 43. Hardware specs <ul><li>Bench (2003): </li><ul><li>Dell PowerEdge 1750
  44. 44. 1 Xeon Nocona 3Ghz, 1 GB RAM, OS: RedHat Enterprise 5.3 </li></ul><li>WMS (2004): </li><ul><li>Dell PowerwEdge 2850
  45. 45. 4 Xeon Nocona 3.4Ghz, 8 GB RAM
  46. 46. 6 73 GB, 15K RPM hard drives, OS: RedHat Enterprise 5.3 </li></ul><li>Database (2007): </li><ul><li>Dell Optiplex 755 tower
  47. 47. 1 x Intel Core2 Duo CPU E6750 @ 2.66GHz / Dual Core, 4Gb RAM
  48. 48. 100Gb SATA 3.0Gb/s 7200 rpm HD, OS: RedHat Enterprise 5.3 </li></ul></ul>
  49. 49. Methodology <ul><li>Each test run performs requests with 1, 10, 20 and 40 parallel clients (for a total of 1500 requests)
  50. 50. Each test uses a random set of requests stored in a CSV file: no two requests in the same run are equal, but all servers perform the same workload
  51. 51. For each request the random factors are: </li><ul><li>The image size (between 640x480 and 1024x768)
  52. 52. The geographic envelope (extent) </li></ul><li>Each test is run three times in a row, the results of the third run are used for the comparison: this benchmark assumes full file system caches (“hot” benchmark)
  53. 53. The other GIS server is shut down while the tests are run </li></ul>
  54. 54. Datasets Used <ul><li>Polygon layer: areawater_merge : the TIGER 2008 set of polygons describing water surfaces for the state of Texas. 380000 polygons, 195MB shapefile, EPSG:4269
  55. 55. Point layer: gnis_names09 : all location and points of interest names for the state of Texas in the GNIS database. 103000 points, 2.8 MB shapefile, EPSG:4269
  56. 56. Line layer, edges_merge : all line elements (rivers and roads) from the TIGER 2008 dataset for the state of Texas. over 5M lines, 1.2GB shapefile, 1.4GB dbf, EPSG:4269
  57. 57. Raster layer : Bluemarble TNG, 86400 x 43200 pixels, 7 overview layers (TIF, ECW, tiled 512&8, GeoRaster) </li></ul>
  58. 58. T1: plain polygon rendering <ul><li>TIGER 2008 “areawater” layer, merged over Texas
  59. 59. Simply fill the polygon in blue, no outline
  60. 60. Image scales roughly between 1:10.000 and 1:100.000 </li></ul>
  61. 61. T1: Shapefiles
  62. 62. T1: PostGIS
  63. 63. T1: Oracle
  64. 64. T1: SDE
  65. 65. T1: observations <ul><li>GeoServer is between 25% and 50% faster reading shapefiles out of the box, MapServer with FastCGI is slightly faster
  66. 66. PostGIS performs roughly 5% better than Oracle on higher loads
  67. 67. SDE seems to introduce no overhead compared to direct Oracle access for GeoServer, and a slight overhead for MapServer
  68. 68. GeoServer and MapServer have exactly the same performance when the backend is PostGIS and Oracle </li></ul>
  69. 69. T3: roads map <ul><li>TIGER 2008 edges, just roads (rivers filtered out), three different line styles based on the road class </li></ul>
  70. 70. T3: shapefiles
  71. 71. T3: PostGIS
  72. 72. T3: Oracle
  73. 73. T3: SDE
  74. 74. T3: notes <ul><li>Oracle and SDE data access is significantly slower in this benchmark (15-20% slower) compared to PostGIS, possibly pointing to a more efficient indexing in PostGIS
  75. 75. The slowdown observed in most tests is probably due to more expensive calls being performed in the 20 clients section of the benchmark (requests are random, thus are not guaranteed to be uniformly expensive) </li></ul>
  76. 76. T5: raster <ul><li>Bluemarble TNG, as single ECW file, 512 mosaic TIFF, and BigTIFF (MapServer) vs 8 tiles mosaic TIFF (GeoServer)
  77. 77. ECW file is 260MB
  78. 78. TIFFs are uncompressed, tiled, with overviews. Around 16GB for each mosaic </li></ul>
  79. 79. T5: ECW
  80. 80. T5: 512 tiles mosaic
  81. 81. T5: BigTiff vs 8 tiles tiff GeoServer cannot do BigTiff, so a 8 tiles mosaic was used (almost 2G B per tile)
  82. 82. FOSS4G 2009 improvements <ul><li>Several code enhancements were made during this year's exercise: </li><ul><li>MapServer </li><ul><li>Fixed problem with labels following features
  83. 83. Fixed problem with 5.6.0-beta for SDE layers
  84. 84. Found problem with 5.6.0-beta for Oracle fastcgi and memory leak </li></ul><li>GeoServer </li><ul><li>Fixed problem with Oracle connection pooling and index scanning
  85. 85. Fixed problem with ECW crashes under high load </li></ul></ul></ul>
  86. 86. Hopeful Changes for Next Shootout <ul><li>Involvement of more WMS servers
  87. 87. Put the servers under a faster internet connection for quicker data transfers and tests setups
  88. 88. Modify the number of parallel client requests performed so that the 2-4-8 parallel clients case gets exposed
  89. 89. Phases/cycles for benchmarking – to allow for teams to interpret results and make software enhancements
  90. 90. Reprojection in both vector and raster data
  91. 91. Audience suggestions? </li></ul>
  92. 92. More Information <ul><li>Scripts, configuration files, results stored in OSGeo SVN: http://svn.osgeo.org/osgeo/foss4g/benchmarking/
  93. 93. Wiki home: http://wiki.osgeo.org/wiki/Benchmarking_2009
  94. 94. Mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking </li></ul>
  95. 95. Thanks! [email_address] www.gatewaygeomatics.com Brazil visa, 2010?

×