OSS Presentation Metro Cluster by Andy Bennett & Roel De Frene
How to Design your Metro Cluster with NexentStor Roel De Frene, CEO S3S Andy Bennett, Nexenta
Stretched Metro Cluster – Key Features• Provides High-Availability Clustering across Geographically disparate sites• Non Disruptive Failover for Block and File Storage Services• Manual or Automatic Failover between sites• ZFS Filesystem Block Level Mirroring delivers optimal performance• End to End checksum on every transaction• Volume Service Locking prevents traditional exposure to Site Split Brain• World leading Triple Site Mirroring• Local Read weighting and SSD cache accelerates performance
About MeRoel De FreneCTO of Server Storage Solutions Belgium12-years in Server & Storage businessMy company offers Built to Order Server & Storage solutions, on customer demand and weoffer the best-fit solution available. We started as Nexenta partner in 2009, and after 2 yearswe can profit off all our hard work to get “certified” Nexenta dealership.At OpenStorage Summit 2011, we will explain how to built a Stretched Metro cluster withNexenta. This case study is in production since last Summit in Amsterdam. In Amsterdam wealso showed a live failover between 15km distance, running VMware hypervisors...
About the end-user BENESOLBenesol is one of the key players in managed services, cloud computing, and server colocation inBelgium. They own one of the greenest datacenters in Benelux and have multiple datacenterrooms connected with a dark fibre ring across Brussel - Antwerp & Oostkamp with a total rangeof 120km max distance between sites.
Technical requests of BENESOL- High Available iSCSI services between the headnodes- Distribute data synchronous between 2 physical sites (+15km and more)- Failover time to data center 2 must be less then a minute, and no crashes to VM’s- Vmware storage motion and Vmotion between two sites- High IOPS > 10.000 IOPS (4K random)- High Performance > 200MB/s- Datacenter shutdown proof!- Start with 11TB netto capacity and able to grow to 500TB- budget: <100.000 $
The ingredients to answer the requests- Nexenta HA cluster running on off the shelve hardware (Supermicro) +- Coraid SRX2800 AoE storage enclosures +- 10Gbit Fibre-optic switches & cabling with a latency less then 1ms between the sites +- Customer agreed for a first time POC on Nexenta Alpha solution +- Lot’s of technical support from Nexenta & Coraid working together on this !- Once fully tested customer goes to Beta with small production Trial
Stretched Metro Cluster – PerformanceVolume test1 is the only available volume, starting benchmark...test1: running optimal mode benchmarktest1: generating 49134MB files, using 32768 blockstest1: started 2 benchmark processestest1: this may take a few minutes, please wait or press Ctrl-C to interrupt ............test1: benchmark in progress, elapsed 60 sec............test1: benchmark in progress, elapsed 120 sec............test1: benchmark in progress, elapsed 180 sec............test1: benchmark in progress, elapsed 240 sec............test1: benchmark in progress, elapsed 300 sec............test1: benchmark in progress, elapsed 360 sec............test1: benchmark in progress, elapsed 420 sec.........WRITE CPU RE-WRITE CPU READ CPU RND-SEEKS356MB/s 26% 220MB/s 22% 535MB/s 24% 678/sec356MB/s 27% 221MB/s 22% 538MB/s 24% 675/sec--------- ---- --------- ---- --------- ---- ---------713MB/s 26% 441MB/s 22% 1073MB/s 24% 677/sec
Actual Performance Benchmarks - EMEA Demo• Local Site performance Brussels • 25,625 IOPS at 4K • 32K Benchmarked at over 400 MB/sec• Failover to Antwerp Site • Failover event takes less than 25 Seconds • I/O paused on Client machines for approx 10 secs • I/O Continues without Client disconnect• Performance picks backup to steady state • 15,000 IOPS at 4K • 32K Benchmarked at over 300 MB/sec• End to End checksum on every transaction
Stretched Metro Cluster Storage Options SRX 3200 AoE 24 Disk High Performance Ethernet SAN Array 10 GbE Connectivity up to 72TB per shelf NXTSMC-3600F 36 Disk High Performance Array Fibre channel Connectivity up to 108 TB per shelf NXTSMC-3600FE 36 Disk High Performance Ethernet SAN Array FCoE Connectivity up to 108 TB per shelf Other Storage Options In Certification
Triple Site Redundancy – Extreme Metro Cluster