Your SlideShare is downloading. ×
0
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Track 3, session 3,big data infrastructure by sunil brid
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Track 3, session 3,big data infrastructure by sunil brid

625

Published on

Published in: Technology, Travel
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
625
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Big Data Infrastructure for Vertical Markets Isilon Scale out NAS 1 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 2. !!! !!!“Big Data Is Less About Size, And More About Freedom” !!! ―Techcrunch THE ERA OF !!! “Findings: „Big BIG DATA !!! Data‟ Is More Extreme Than Volume” “Big Data! It‟s Real, It‟s Real-time, and IS HERE… ― Gartner It‟s Already Changing Your “Total data: „bigger‟ World” than big data” ―IDB !!! !!! !!! ― 451 Group 2 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 3. Isilon Drives Down the Cost of Storing File-based, Unstructured Data…”Big Data” 3 Cloud Meets Big Data3 17-18 November 2011. Grand Hyatt - Mumbai
  • 4. Big Data Is Changing Enterprise Storage 90 80 70 60 Big 50 Data 40 Sources 30 20 10 0 2009 2010 2011 2012 2013 2014 File Based: 60.7% CAGR Block Based: 21.8% CAGR By 2012, 80% of all storage capacity sold will be for file-based data Source: IDC 4 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 5. Digital Storage ExplosionMobile, Personal and VirtualOver 1 Billion transistors in a single ICOver 2 Billion MP3s distributed Are technologies and ITOver 2 Billion e-mail accounts organizations ready toOver 5 Billion JPEGs shared and distributedOver 250 Million video streams/day hosted manage the resultingOver 30 TB in a single genome mapping complexity and seizeOver 20 TB in a seismic model the opportunity??By 2010 total data will exceed 1,000 Exabytes 5 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 6. 6 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 7. It‟s Day and Night Different Scale-Out Scale-Up One Large Storage pool 10+PB Shared Storage Automated Manual Linear Scalability ( Perf & Cap) Performance Bottlenecks Operational Efficiency Increasing Complexity (Multiprotocol Support ,DR,Archival) 7 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 8. Enterprises Had Deployed a Scale-Up Strategy Scalability Performance Management Availability Cost 8 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 9. Clicker QuestionDo you face challenges in your big data infrastructure.• Our biggest challenge was manageability between file systems , server , storage and network• A built up solution with isolated pieces was very hard to tune for performance• We prefer a scale out approach• Access from different clients was an issue 9 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 10. Enterprises are Transitioning to Scale-out Scalability Performance Management Availability Cost 10 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 11. Isilon Scale-Out Storage Innovation • • 11 Cloud Meets Big Data11 17-18 November 2011. Grand Hyatt - Mumbai
  • 12. Core Innovation… Isilon‟s OneFS Scale-Out Operating System 12 Cloud Meets Big Data12 17-18 November 2011. Grand Hyatt - Mumbai
  • 13. Core Innovation… Isilon‟s OneFS Scale-Out Operating System • • • • • • 13 Cloud Meets Big Data13 17-18 November 2011. Grand Hyatt - Mumbai
  • 14. Isilon‟s Scale-Out NAS Platforms S-SERIES X-SERIES NL-SERIES 14 Cloud Meets Big Data14 17-18 November 2011. Grand Hyatt - Mumbai
  • 15. Elements of Scale-out (Distributed File Systems, Clustered File Systems, etc) • • • • • • 15 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 16. 16 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 17. Isilon IQ Network Architecture Servers Servers Servers Intracluster Standard Gigabit Ethernet Isilon IQ Storage Client/Application Layer Communication Infiniband Layer Layer or GigE Layer 17 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 18. 18 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 19. Scalability & Performance Scale-out Scale-up Isilon Traditional • Scale-out • Scale-upScalability • Performance, Capacity, Both • Capacity onlyPerformance • True linear predictability • Degradation at scale “With NTAP my best day is my first day…” 19 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 20. Largest and Most Scalable File System 500X More Scalable than Traditional Storage SystemsOneFS™ can scale from 18TB to over 10,000 TB in asingle file system••• 20 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 21. VIDEO 21 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 22. SimplicityManagement & Economics File Systems Volumes Aggregates (~16TB) RAID Isilon Traditional Storage Isilon Traditional Storage • 1 volume, 1 file system • Multiple RAID, volumes, file systemsManagement • Single volume up to 10PB • Single volume up 100TB (16TB typical) • 80%+ utilization, <1FTE • 57% utilization, Multiple FTE‟sEconomics • Business agility • Silos, limitations • Investment protection • Controller swaps and relicensing 22 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 23. Clicker QuestionHow do you plan archival and DR for your big data stack.• We don‟t have a DR solution with our big data stack• We would prefer an integrated deep archival strategy with tiering 23 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 24. One File System for Multiple Tiers of Storage MB/s SmartPools™ Innovation Single Point of Management Single File System Single Volume I/Ops Tiers of Performance I/Ops Nearline Automatic Data Movement Throughput I/Ops Nearline Policy-Based Movement Throughput I/Ops Nearline Transparent Reallocation Throughput Nearline NO application-changes Throughput Throughput Nearline Investment Protection Throughput Nearline Eliminate Data Migration Scale Any Application Completely Transparent 24 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 25. Suite of Enterprise Software Applications• Combine multiple storage tiers into a single file system• Simple, scalable and flexible data protection• Policy-based client load balancing with NFS failover• Quota management and thin provisioning• Fast and flexible file-base asynchronous replication• Analytics platform to maximize performance and resource utilization• High performance wide area file and content delivery 25 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 26. Case Study - Project Background Background • Automobile Company has 3000 clients - document to be stored in electronic format • First projects for 3,000 clients and then will expand to 30,000 clients in coming years Customer Environment • ECM introduction for all unstructured business contents to create, storrage, management • DR in the event of a disaster with no loss of data • ECM Solution : Oracle UCM(Universal Content Management) Introduction of the storage infrastructure requiremnets • Main Storage: Usable 110TB / Main Storage DR : Usable 110TB • Archive Storage : Usable 140TB / Archive Storage DR : Usable 140TB • Main Introduction Storage Spec/ Main Storage DR : S200 3node + IQ72NL 3node Storage : S200 3node + IQ72NL3node • Archive Storage : IQ5000S 6node + IQ72NL 3node / Archive Storage DR : IQ5000S 6node + IQ72NL 3node 26 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 27. System Configuration Client Users : 3000 ~ 30,000 …… NAS Clients ECM Server User Network 1/10G Network 10GbE 10GbE Service Network One Filesystem One Filesystem One Filesystem One Filesystem DATASET DATASET DATASETSnapshotIQ SnapshotIQ SmartPool 아카이브 운영. SmartPool 운영.DR Local Point 아카이브.Local Point DR. (110TB) (110TB) Backup (140TB) Auto Data Bakcup Auto Data (140TB) 원본 Management 원본 Management 원본 DATASET DATASET DATASET 원본 Main Storage NAS Main Storage DR Archive Stgorage NAS Archive Storage DR SyncIQ – Replication SyncIQ – Replication 27 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 28. The Scale Out Differentiation• Client Load expected to grow from 3000 to 3000. - Need scalable Architecture for performance and Capacity - 250 TB Primary will grow when Users increase - Performance Demand will grow when Users Increase• Disaster Recovery Site - Solution for effective DR solution via SyncIQ 28 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 29. The Scale Out Differentiation• Tiering within Same Architecture -SmartPool Solution offered a effective tiering within same architecture Without affecting External Network Performance• Ease of Configuration and Management : - The management will remain simple due to single filesystem and OneFS operating system even as volume and complexity grows.• Redundancy and Self Healing : - The solution is self healing and resilient as it’s a distributed controller based architecture. 29 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 30. Isilon Scale-Out NAS -- Benefits 30 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 31. Why Isilon?Because Big Data Demands New ThinkingProduct Families Purpose-built to Optimizefor IOps, Throughput and/or $/TB.Record-breaking Scaling of Capacity andPerformance.Remarkable Simplicity. 31 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai
  • 32. Thank You .SIMPLE IS SMART 32 Cloud Meets Big Data 17-18 November 2011. Grand Hyatt - Mumbai

×