Speed up your GIS server       GIS software on solid-state drives (SSD)FOSS4G2011 - #1   GIS software on solid-state drive...
Speed up your GIS server        GIS software on solid-state drives (SSD)     Seongbong Kim                                ...
Motivation●    Involved in pgRouting project●    Lots of questions about    performance●    pgRoutings bottleneck is    lo...
… what about running  PostgreSQL/pgRouting on SSD?FOSS4G2011 - #4   GIS software on solid-state drives (SSD)
Who is using SSD with a laptop?FOSS4G2011 - #5   GIS software on solid-state drives (SSD)
Who is using SSD on a server?FOSS4G2011 - #6   GIS software on solid-state drives (SSD)
What is a solid-state drive?A solid-state drive (SSD) is a data storage device thatuses solid-state memory to store persis...
SSD vs. HDD (1)  Characteristic            Solid-state drive                    Hard disk drive   Spin-up time            ...
SSD vs. HDD (2)  Characteristic             Solid-state drive                    Hard disk drive Parallel operation       ...
MLC vs. SLCMulti-Level-Cell                               Single-Level-Cell●    shorter lifespan                          ...
Test Server SpecificationModel Name              Intel(R) Xeon(R) CPU - E5620 @ 2.40GHzProcessors              6CPU cores ...
BenchmarkFOSS4G2011 - #12   GIS software on solid-state drives (SSD)
Data & Tools●    Geodata    ●        Raster: Landsat imagery North America    ●        Vector: OpenStreetMap Planet●    Pr...
Ambitious Plans●    File system benchmarks●    WMS map rendering from raster data●    WMS map rendering from vector data (...
Reality●    File system benchmarks●    WMS map rendering from raster data●    WMS map rendering from vector data (DB)●    ...
Benchmarking Tools●    Apache JMeter for WMS●    pgBench for PostgreSQL●    FIO for file system                       => W...
File I/O Benchmark4000350030002500                                                                                  16k   ...
FOSS4G2011 - #18   GIS software on solid-state drives (SSD)
FOSS4G2011 - #19   GIS software on solid-state drives (SSD)
FOSS4G2011 - #20   GIS software on solid-state drives (SSD)
FOSS4G2011 - #21   GIS software on solid-state drives (SSD)
Use Case - VOD or FTPRequirements●    Mainly reading access●    Fast read rate●    Serving dataExample●    Tile server●   ...
Use Case - DB / FS onlyRequirements●    Combined writing    and reading access●    Fast read rateExamples●    Geo-database...
Lessons Learned●    Different field of expertize●    Relatively small amount of community    resources●    Standards & bes...
FOSS4G2011 - #25   GIS software on solid-state drives (SSD)
Want to benchmark your     Open Source software?FOSS4G2011 - #26   GIS software on solid-state drives (SSD)
Thank you!                      LSD Tech Korea          KimSeongbong@next-group.jp                   daniel@georepublic.de...
Upcoming SlideShare
Loading in...5
×

Speed up your GIS server - run GIS software on solid-state drives (SSD)

2,541

Published on

Solid-State Drives (SSDs) are recently becoming popular storage mediums in laptops, and are praised for their speed and shock resistance. However, there has been some debate over whether or not SSDs are ready for the data center.

GIS software is known for its high demands on server hardware. Especially hard-drives became a bottleneck when serving map tiles, rendering map images or running spatial computations. SSDs seem to be one possibility to increase IO performance, but optimized server hardware is rare and still very expensive.

We contacted a Korean maker specialising in high-performance SSD servers, mainly used to provide video-on-demand services, and we received two of their enterprise servers for testing and benchmarking with FOSS4G software.

In this presentation we present our benchmarking results and explain the strong and weak points by comparing traditional hard-drives and solid-state disks.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,541
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
29
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Speed up your GIS server - run GIS software on solid-state drives (SSD)

  1. 1. Speed up your GIS server GIS software on solid-state drives (SSD)FOSS4G2011 - #1 GIS software on solid-state drives (SSD)
  2. 2. Speed up your GIS server GIS software on solid-state drives (SSD) Seongbong Kim Daniel KastlKimSeongbong@next-group.jp daniel@georepublic.de FOSS4G2011 - #2 GIS software on solid-state drives (SSD)
  3. 3. Motivation● Involved in pgRouting project● Lots of questions about performance● pgRoutings bottleneck is loading data from hard disks into memory● How to be as fast as Google? There is no simple answer ... FOSS4G2011 - #3 GIS software on solid-state drives (SSD)
  4. 4. … what about running PostgreSQL/pgRouting on SSD?FOSS4G2011 - #4 GIS software on solid-state drives (SSD)
  5. 5. Who is using SSD with a laptop?FOSS4G2011 - #5 GIS software on solid-state drives (SSD)
  6. 6. Who is using SSD on a server?FOSS4G2011 - #6 GIS software on solid-state drives (SSD)
  7. 7. What is a solid-state drive?A solid-state drive (SSD) is a data storage device thatuses solid-state memory to store persistent data with theintention of providing access in the same manner of atraditional block i/o hard disk drive.SSDs are distinguished from traditional hard disk drives(HDDs), which are electromechanical devices containingspinning disks and movable read/write heads.In contrast, SSDs use microchips which retain data in non-volatile memory chips and contain no moving parts. http://en.wikipedia.org/wiki/Solid-state_drive FOSS4G2011 - #7 GIS software on solid-state drives (SSD)
  8. 8. SSD vs. HDD (1) Characteristic Solid-state drive Hard disk drive Spin-up time short longRandom access time ~ 0.1 ms 5-10 ms Read latency time short longRead performance consistent varying Moving parts no yes Weight/size smaller larger Write longevity type dependent unlimited FOSS4G2011 - #8 GIS software on solid-state drives (SSD)
  9. 9. SSD vs. HDD (2) Characteristic Solid-state drive Hard disk drive Parallel operation possible no Shock, altitude, resistant not resistant vibration Magnetic no possible susceptibilitySoftware encryption limited yes Costs (energy) lower high Costs (capacity) high “low” FOSS4G2011 - #9 GIS software on solid-state drives (SSD)
  10. 10. MLC vs. SLCMulti-Level-Cell Single-Level-Cell● shorter lifespan ● longer lifespan (10.000 write cycles) (100.000 write cycles)● higher data density ● lower data density● slower throughput ● higher throughput● less efficient writing ● more efficient writing● less expensive ● more expensive FOSS4G2011 - #10 GIS software on solid-state drives (SSD)
  11. 11. Test Server SpecificationModel Name Intel(R) Xeon(R) CPU - E5620 @ 2.40GHzProcessors 6CPU cores 4Cache Size 12.288 KBTotal Memory 24.675.980 KB (24 GB)RAID 0Total SSD 750 GB (SLC)OS CentOS 5.6 FOSS4G2011 - #11 GIS software on solid-state drives (SSD)
  12. 12. BenchmarkFOSS4G2011 - #12 GIS software on solid-state drives (SSD)
  13. 13. Data & Tools● Geodata ● Raster: Landsat imagery North America ● Vector: OpenStreetMap Planet● Processing ● PostgreSQL import with Imposm ● Image processing with GDAL ● Routing topology with osm2po ● Image rendering with Mapserver (Fast CGI) FOSS4G2011 - #13 GIS software on solid-state drives (SSD)
  14. 14. Ambitious Plans● File system benchmarks● WMS map rendering from raster data● WMS map rendering from vector data (DB)● Tile seeding from raster data● Tile seeding from vector data (DB)● Serving from tile cache FOSS4G2011 - #14 GIS software on solid-state drives (SSD)
  15. 15. Reality● File system benchmarks● WMS map rendering from raster data● WMS map rendering from vector data (DB)● Tile seeding from raster data● Tile seeding from vector data (DB)● Serving from tile cache FOSS4G2011 - #15 GIS software on solid-state drives (SSD)
  16. 16. Benchmarking Tools● Apache JMeter for WMS● pgBench for PostgreSQL● FIO for file system => WMS Shootout FOSS4G2011 - #16 GIS software on solid-state drives (SSD)
  17. 17. File I/O Benchmark4000350030002500 16k 32k 64k2000 128k 256k 512k 1024k15001000500 0 Sequential Read Sequential Write Random Read Random Write FOSS4G2011 - #17 GIS software on solid-state drives (SSD)
  18. 18. FOSS4G2011 - #18 GIS software on solid-state drives (SSD)
  19. 19. FOSS4G2011 - #19 GIS software on solid-state drives (SSD)
  20. 20. FOSS4G2011 - #20 GIS software on solid-state drives (SSD)
  21. 21. FOSS4G2011 - #21 GIS software on solid-state drives (SSD)
  22. 22. Use Case - VOD or FTPRequirements● Mainly reading access● Fast read rate● Serving dataExample● Tile server● Data server http://www.maptiler.org/ FOSS4G2011 - #22 GIS software on solid-state drives (SSD)
  23. 23. Use Case - DB / FS onlyRequirements● Combined writing and reading access● Fast read rateExamples● Geo-database● Map rendering● Data processing FOSS4G2011 - #23 GIS software on solid-state drives (SSD)
  24. 24. Lessons Learned● Different field of expertize● Relatively small amount of community resources● Standards & best practices● Comparability● PostgreSQL/PostGIS – how to tune it right? … well, everyone asks for benchmarks FOSS4G2011 - #24 GIS software on solid-state drives (SSD)
  25. 25. FOSS4G2011 - #25 GIS software on solid-state drives (SSD)
  26. 26. Want to benchmark your Open Source software?FOSS4G2011 - #26 GIS software on solid-state drives (SSD)
  27. 27. Thank you! LSD Tech Korea KimSeongbong@next-group.jp daniel@georepublic.deFOSS4G2011 - #27 GIS software on solid-state drives (SSD)
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×