Big Data Infrastructure    for Vertical Markets  Isilon Scale out NAS         1          Cloud Meets Big Data             ...
!!!                                 !!!“Big Data Is Less About Size, And More About Freedom”                              ...
Isilon Drives Down the Cost of Storing    File-based, Unstructured Data…”Big Data”                                        ...
Big Data Is Changing Enterprise Storage                   90                   80                   70                   6...
Digital Storage ExplosionMobile, Personal and VirtualOver 1 Billion transistors in a single ICOver 2 Billion MP3s distribu...
6          Cloud Meets Big Data    17-18 November 2011. Grand Hyatt - Mumbai
It‟s Day and Night Different                Scale-Out                           Scale-Up     One Large Storage pool 10+PB ...
Enterprises Had Deployed a Scale-Up Strategy                             Scalability                             Performan...
Clicker QuestionDo you face challenges in your big data infrastructure.• Our biggest challenge was manageability between f...
Enterprises are Transitioning to Scale-out                                Scalability                                Perfo...
Isilon Scale-Out Storage Innovation     •     •                                   11      Cloud Meets Big Data11          ...
Core Innovation… Isilon‟s OneFS Scale-Out Operating System                                       12      Cloud Meets Big D...
Core Innovation… Isilon‟s OneFS Scale-Out Operating System     •     •     •     •     •     •                            ...
Isilon‟s Scale-Out NAS Platforms         S-SERIES                      X-SERIES                                        NL-...
Elements of Scale-out        (Distributed File Systems, Clustered File Systems, etc) •                               • •  ...
16      Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
Isilon IQ Network Architecture  Servers  Servers  Servers                                                                 ...
18      Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
Scalability & Performance              Scale-out                                            Scale-up                      ...
Largest and Most Scalable File System 500X More Scalable than Traditional Storage SystemsOneFS™ can scale from 18TB to ove...
VIDEO        21      Cloud Meets Big Data         17-18 November 2011. Grand Hyatt - Mumbai
SimplicityManagement & Economics                                                                                          ...
Clicker QuestionHow do you plan archival and DR for your big data stack.• We don‟t have a DR solution with our big data st...
One File System for Multiple Tiers of Storage               MB/s                                       SmartPools™ Innovat...
Suite of Enterprise Software Applications• Combine multiple storage tiers into a single file system• Simple, scalable and ...
Case Study - Project Background        Background • Automobile Company has 3000 clients - document to be stored in electro...
System Configuration                                                 Client Users : 3000 ~ 30,000                         ...
The Scale Out Differentiation• Client Load expected to grow from 3000 to 3000.   - Need scalable Architecture for performa...
The Scale Out Differentiation• Tiering within Same Architecture     -SmartPool Solution offered a effective tiering within...
Isilon Scale-Out NAS -- Benefits                               30         Cloud Meets Big Data                            ...
Why Isilon?Because Big Data Demands New ThinkingProduct Families Purpose-built to Optimizefor IOps, Throughput and/or $/TB...
Thank You .SIMPLE IS SMART                  32      Cloud Meets Big Data                   17-18 November 2011. Grand Hyat...
Upcoming SlideShare
Loading in …5
×

Track 3, session 3,big data infrastructure by sunil brid

1,016 views

Published on

Published in: Technology, Travel
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,016
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Track 3, session 3,big data infrastructure by sunil brid

  1. 1. Big Data Infrastructure for Vertical Markets Isilon Scale out NAS 1 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  2. 2. !!! !!!“Big Data Is Less About Size, And More About Freedom” !!! ―Techcrunch THE ERA OF !!! “Findings: „Big BIG DATA !!! Data‟ Is More Extreme Than Volume” “Big Data! It‟s Real, It‟s Real-time, and IS HERE… ― Gartner It‟s Already Changing Your “Total data: „bigger‟ World” than big data” ―IDB !!! !!! !!! ― 451 Group 2 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  3. 3. Isilon Drives Down the Cost of Storing File-based, Unstructured Data…”Big Data” 3 Cloud Meets Big Data3 17-18 November 2011. Grand Hyatt - Mumbai
  4. 4. Big Data Is Changing Enterprise Storage 90 80 70 60 Big 50 Data 40 Sources 30 20 10 0 2009 2010 2011 2012 2013 2014 File Based: 60.7% CAGR Block Based: 21.8% CAGR By 2012, 80% of all storage capacity sold will be for file-based data Source: IDC 4 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  5. 5. Digital Storage ExplosionMobile, Personal and VirtualOver 1 Billion transistors in a single ICOver 2 Billion MP3s distributed Are technologies and ITOver 2 Billion e-mail accounts organizations ready toOver 5 Billion JPEGs shared and distributedOver 250 Million video streams/day hosted manage the resultingOver 30 TB in a single genome mapping complexity and seizeOver 20 TB in a seismic model the opportunity??By 2010 total data will exceed 1,000 Exabytes 5 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  6. 6. 6 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  7. 7. It‟s Day and Night Different Scale-Out Scale-Up One Large Storage pool 10+PB Shared Storage Automated Manual Linear Scalability ( Perf & Cap) Performance Bottlenecks Operational Efficiency Increasing Complexity (Multiprotocol Support ,DR,Archival) 7 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  8. 8. Enterprises Had Deployed a Scale-Up Strategy Scalability Performance Management Availability Cost 8 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  9. 9. Clicker QuestionDo you face challenges in your big data infrastructure.• Our biggest challenge was manageability between file systems , server , storage and network• A built up solution with isolated pieces was very hard to tune for performance• We prefer a scale out approach• Access from different clients was an issue 9 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  10. 10. Enterprises are Transitioning to Scale-out Scalability Performance Management Availability Cost 10 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  11. 11. Isilon Scale-Out Storage Innovation • • 11 Cloud Meets Big Data11 17-18 November 2011. Grand Hyatt - Mumbai
  12. 12. Core Innovation… Isilon‟s OneFS Scale-Out Operating System 12 Cloud Meets Big Data12 17-18 November 2011. Grand Hyatt - Mumbai
  13. 13. Core Innovation… Isilon‟s OneFS Scale-Out Operating System • • • • • • 13 Cloud Meets Big Data13 17-18 November 2011. Grand Hyatt - Mumbai
  14. 14. Isilon‟s Scale-Out NAS Platforms S-SERIES X-SERIES NL-SERIES 14 Cloud Meets Big Data14 17-18 November 2011. Grand Hyatt - Mumbai
  15. 15. Elements of Scale-out (Distributed File Systems, Clustered File Systems, etc) • • • • • • 15 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  16. 16. 16 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  17. 17. Isilon IQ Network Architecture Servers Servers Servers Intracluster Standard Gigabit Ethernet Isilon IQ Storage Client/Application Layer Communication Infiniband Layer Layer or GigE Layer 17 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  18. 18. 18 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  19. 19. Scalability & Performance Scale-out Scale-up Isilon Traditional • Scale-out • Scale-upScalability • Performance, Capacity, Both • Capacity onlyPerformance • True linear predictability • Degradation at scale “With NTAP my best day is my first day…” 19 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  20. 20. Largest and Most Scalable File System 500X More Scalable than Traditional Storage SystemsOneFS™ can scale from 18TB to over 10,000 TB in asingle file system••• 20 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  21. 21. VIDEO 21 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  22. 22. SimplicityManagement & Economics File Systems Volumes Aggregates (~16TB) RAID Isilon Traditional Storage Isilon Traditional Storage • 1 volume, 1 file system • Multiple RAID, volumes, file systemsManagement • Single volume up to 10PB • Single volume up 100TB (16TB typical) • 80%+ utilization, <1FTE • 57% utilization, Multiple FTE‟sEconomics • Business agility • Silos, limitations • Investment protection • Controller swaps and relicensing 22 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  23. 23. Clicker QuestionHow do you plan archival and DR for your big data stack.• We don‟t have a DR solution with our big data stack• We would prefer an integrated deep archival strategy with tiering 23 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  24. 24. One File System for Multiple Tiers of Storage MB/s SmartPools™ Innovation Single Point of Management Single File System Single Volume I/Ops Tiers of Performance I/Ops Nearline Automatic Data Movement Throughput I/Ops Nearline Policy-Based Movement Throughput I/Ops Nearline Transparent Reallocation Throughput Nearline NO application-changes Throughput Throughput Nearline Investment Protection Throughput Nearline Eliminate Data Migration Scale Any Application Completely Transparent 24 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  25. 25. Suite of Enterprise Software Applications• Combine multiple storage tiers into a single file system• Simple, scalable and flexible data protection• Policy-based client load balancing with NFS failover• Quota management and thin provisioning• Fast and flexible file-base asynchronous replication• Analytics platform to maximize performance and resource utilization• High performance wide area file and content delivery 25 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  26. 26. Case Study - Project Background Background • Automobile Company has 3000 clients - document to be stored in electronic format • First projects for 3,000 clients and then will expand to 30,000 clients in coming years Customer Environment • ECM introduction for all unstructured business contents to create, storrage, management • DR in the event of a disaster with no loss of data • ECM Solution : Oracle UCM(Universal Content Management) Introduction of the storage infrastructure requiremnets • Main Storage: Usable 110TB / Main Storage DR : Usable 110TB • Archive Storage : Usable 140TB / Archive Storage DR : Usable 140TB • Main Introduction Storage Spec/ Main Storage DR : S200 3node + IQ72NL 3node Storage : S200 3node + IQ72NL3node • Archive Storage : IQ5000S 6node + IQ72NL 3node / Archive Storage DR : IQ5000S 6node + IQ72NL 3node 26 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  27. 27. System Configuration Client Users : 3000 ~ 30,000 …… NAS Clients ECM Server User Network 1/10G Network 10GbE 10GbE Service Network One Filesystem One Filesystem One Filesystem One Filesystem DATASET DATASET DATASETSnapshotIQ SnapshotIQ SmartPool 아카이브 운영. SmartPool 운영.DR Local Point 아카이브.Local Point DR. (110TB) (110TB) Backup (140TB) Auto Data Bakcup Auto Data (140TB) 원본 Management 원본 Management 원본 DATASET DATASET DATASET 원본 Main Storage NAS Main Storage DR Archive Stgorage NAS Archive Storage DR SyncIQ – Replication SyncIQ – Replication 27 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  28. 28. The Scale Out Differentiation• Client Load expected to grow from 3000 to 3000. - Need scalable Architecture for performance and Capacity - 250 TB Primary will grow when Users increase - Performance Demand will grow when Users Increase• Disaster Recovery Site - Solution for effective DR solution via SyncIQ 28 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  29. 29. The Scale Out Differentiation• Tiering within Same Architecture -SmartPool Solution offered a effective tiering within same architecture Without affecting External Network Performance• Ease of Configuration and Management : - The management will remain simple due to single filesystem and OneFS operating system even as volume and complexity grows.• Redundancy and Self Healing : - The solution is self healing and resilient as it’s a distributed controller based architecture. 29 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  30. 30. Isilon Scale-Out NAS -- Benefits 30 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  31. 31. Why Isilon?Because Big Data Demands New ThinkingProduct Families Purpose-built to Optimizefor IOps, Throughput and/or $/TB.Record-breaking Scaling of Capacity andPerformance.Remarkable Simplicity. 31 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  32. 32. Thank You .SIMPLE IS SMART 32 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai

×