Download as PDF, PPTX
![Product Portfolio: Three Strong Pillars
Analytics
Platforms
[x]Scalers: Accelerating
Time to Insight Across
Markets and Industries
SFA: Block and File,
SSD and Disk,
Processors and Storage
Object Storage
WOS: Cloud and Data Centers,
Internet, Web and Telcos,
Massive Scale and Low Latency
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
3](https://image.slidesharecdn.com/denworth-131204110813-phpapp02/85/DDN-Product-Update-from-SC13-3-320.jpg)
![DDN Application-Defined Caching Eliminates Cluster Idle Time
In-Band File System Hint Awareness; Out-of-Band Middleware & DB Structure Caching
Out-of-Band API Control
Enables Dramatic Efficiency Gains
Application
I/O Idle
I/O Idle
Load
Operating System
File System Client
Time
CPU
Scheduler
File System ‘Hint Sniffer’ [Technical Term J]
API
Out of Band Ethernet
I/O
Load
CPU
I/O
SFX
SFX
Prefetch
Prefetch
Time
ddn.com
© 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others.
Any statements or representations around future events are subject to change.
SFX is equally valuable in telling the
system what not to cache
17](https://image.slidesharecdn.com/denworth-131204110813-phpapp02/85/DDN-Product-Update-from-SC13-17-320.jpg)
Uploaded byinside-BigData.com
DDN Product Update from SC13
The document outlines DDN's product portfolio for high-performance computing (HPC) and cloud infrastructure, emphasizing innovations in storage solutions such as the SFA and WOS platforms. It discusses the company's focus on scalability, performance, and cost efficiency, highlighting features like caching technologies and multi-protocol access capabilities. Additionally, it presents DDN's approach to addressing the challenges of data access and the evolving landscape of data-intensive computing.
More Related Content
Similar to DDN Product Update from SC13
DDN Product Update from SC13
- 1. Store Process Analyze Collaborate Archive Cloud Product Update #BurstForward Jeff Denworth VP,Marketing Invent Discover Compete ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 1
- 2. DDN End-to-End HPCCluster Expertise Applications Ex. 2x-5x 3DSRMME Gains Cluster Mgr. Dynamic Cluster Optimization (End-End) Compute SFA-E, IME Converged Systems ddn.com Networking Ethernet, InfiniBand, WAN/Cloud Schedulers SFX APIs: Prefetch, User Priority © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 2
- 3. Product Portfolio: ThreeStrong Pillars Analytics Platforms [x]Scalers: Accelerating Time to Insight Across Markets and Industries SFA: Block and File, SSD and Disk, Processors and Storage Object Storage WOS: Cloud and Data Centers, Internet, Web and Telcos, Massive Scale and Low Latency ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 3
- 4. Big Data &Cloud Infrastructure DDN 2013 Announced Product Portfolio Cloud Tiering Petascale Lustre* Storage Analytics Reference Architectures Enterprise Scale-Out File Storage Cloud Foundation WOS® 3.0 Big Data Platform Management EXAScaler™ GRIDScaler™ 10Ks of Clients 1TB/s+, HSM Linux HPC Clients NFS & CIFS ~10K Clients 1TB/s+, HSM Linux/Windows HPC Clients NFS & CIFS 32 Trillion Unique Objects GeoReplicated Cloud Storage 256 Million Objects/Second Self-Healing Cloud Parallel Boolean Search Storage Fusion Architecture™ Core Storage Platforms DirectMon SFA12Kx SFA7700 48GB/s/1.4M IOPS 1,680 Drives: 16U-2 Racks Embedded Computing 13GB/s, 400K IOPS 60 Drives in 4U; 396 Drives in 20U WOS7000 Flexible Drive Configuration SATA SAS SSD SFX Automated Flash Caching • • Read Write* • • 60 Drives in 4U Self-Contained Servers Context Commit Instant Commit * Future Release ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 4
- 5. Big Data &Cloud Infrastructure DDN 2013 Announced Product Portfolio Cloud Tiering Petascale Lustre* Storage Analytics Reference Architectures Enterprise Scale-Out File Storage Cloud Foundation Lots of Amazing Products WOS® 3.0 32 Trillion Unique Objects GeoReplicated Cloud Storage 256 Million Objects/Second Self-Healing Cloud Parallel Boolean Search GRIDScaler™ More Than We Can Cover Today... Big Data Platform Management EXAScaler™ 10Ks of Clients 1TB/s+, HSM Linux HPC Clients NFS & CIFS ~10K Clients 1TB/s+, HSM Linux/Windows HPC Clients NFS & CIFS Focus For This Talk: Storage Fusion Architecture™ Core Storage Platforms • DDN’s HPC Engineering Agenda SFA12Kx SFA7700 • SC’13 New Product Update DirectMon 48GB/s/1.4M IOPS 1,680 Drives: 16U-2 Racks Embedded Computing 13GB/s, 600K IOPS 60 Drives in 4U; 396 Drives in 20U WOS7000 Flexible Drive Configuration SATA SAS SSD SFX Automated Flash Caching • • Read Write* • • 60 Drives in 4U Self-Contained Servers Context Commit Instant Commit * Future Release ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 5
- 6. Our HPC Focus Price /Performance Infinite Research Collaboration &Data Ubiquity ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. Scale Simple Infrastructure 6
- 7. Razor-Focused On TheCost of Computing Fully-Parallel, State Machine I/O HPC’s Most Aggressive Caching Strategy Best HW Efficiency Least Electronics / perf. unit Price /Performance • HW: RAM, Flash • SW: APIs, Buffers Save on HDDs, Racks, Power Increase Cluster Efficiency Data Center Density & Storage Capacity Compute/Store Convergence • • • • ddn.com Best Slot per RU Least Electronics / perf Most Disks / system WOS = 99% platter util. © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. • • Eliminate I/O Server Layer Buffers can save $MMs 7
- 8. Breaking Down BarriersTo Data Access. Protocol Barriers Research Collaboration & Data Ubiquity Geographic Barriers Aggressively Enabling Multi-Protocol Platforms Cloud Storage Technology Can Enable Rapid Insights Single Namespace, Ubiquitous Data • Parallel File Systems • Cross-Protocol NAS • Even Object Storage DDN is actively engineering multiprotocol namespace management • • • • New: DDN GRIDScaler ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. Massive Namespace Easy OID Management Robust Sync Tools Advanced, WAN Data Protection Add Parallel File Systems, and Presto! New: DDN WOS Bridge 8
- 9. Infinite Scalability DeliversTrue Power To Your Apps Namespace Level Infinite Scale Storage Level Engineering Around POSIX Locking Real-Time Data Delivery (HDD) • • Totally Distributed Algorithms Buffering to Relieve I/O Pressure • • Non-Deterministic Data Placement & Retrieval • • ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. Guaranteed Response SLAs Eliminate Rebuild Latency Eliminate File Layout Hostpots (Buffer) Load Aware Write & Read (WOS, Buffer) 9
- 10. Sophisticated I/O Technology,Management Simplicity. One Tool; Any Platform SFA, GridScaler, ExaScaler, Clusters, Hadoop, WOS Simple Infrastructure Self-Healing Storage • • ddn.com SFA: Drive Power Cycling WOS: Data Protection Policies Always-Conform © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. Compute/Store Convergence DDN SFA Appliances eliminate I/O servers & networking to reduce systems by <90% Self-Notifying System Management Now supporting Call Home! 10
- 11. Proven Blueprints; Optimizedfor TCO Scale and Speed And more... ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 11
- 12. DDN Delivers theMost Powerful & Comprehensive End to End Stack for HPC Parallel File System Clients NAS-Attached Clients DirectMon™ Infinite Memory Engine™ Burst Buffer System Multi-Protocol Access DDN Scale-Out Parallel File Systems ReACT™ and SFX™ Storage Caching WOS™ Bridge Cloud Connector and File System Data Replicator SFA™ Storage Appliances One Integrated HPC Storage Stack from The HPC Leader ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 12
- 13. The World HPCChampion is Now Even Faster! Introducing The SFA12KX Family § § § § § World-Leading Parallel File I/O Foundation Best-In-Class IOPS & B/W for All Workloads Multi-Level Write and Read Cache, Cache APIs Deep Capacity, Leading Data Center Density Server-Virtualized To Embed HPC File Systems ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 2 New Models: SFA12KX 48GB/s SFA12KE 23GB/s 13
- 14. SFA7700: World’s FastestScale-Out Hybrid Platform DDN’s Departmental Storage Powerhouse The Speed of Flash, The Cost of Disk. The Power of SFA-OS, All in Only 4U Fully-Featured Like the SFA12KX § SFX Flash Cache and API-Enabled Caching § Next Gen DirectMon Appliance Management § World Leading Hybrid Price Performance § 13 GB/s, 400K Cache IOPS, 900TB in 12U ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. SFA7700 14
- 15. DDN Revolutionary CachingBoosts HPC Application Performance Write Distribution for Multi-Disciplinary HPC Cluster Sampling from ORNL Testing Caching Systems Are HPC’s Shock Absorbers • DeBlends Unaligned Writes • Preserve Small I/O Performance • Smoothes Out Mixed I/O Inconsistency • Significantly Improves Storage Utilization Source: ORNL 70% of I/Os are <512KB, 50% are <16KB 58% write I/Os vs. 42% read I/Os ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 15
- 16. SFX Improves ApplicationPerformance by Up to 700% Context Commit API In-Band - Out Of Band Usage & Priority-Based Caching Aligned Writes To Flash Partial Writes to RAM ReACT DRAM Cache SFX Write* Instant Commit SFX Read HDD Tier Cache Flush * 2014 Cache Warm ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 16
- 17. DDN Application-Defined CachingEliminates Cluster Idle Time In-Band File System Hint Awareness; Out-of-Band Middleware & DB Structure Caching Out-of-Band API Control Enables Dramatic Efficiency Gains Application I/O Idle I/O Idle Load Operating System File System Client Time CPU Scheduler File System ‘Hint Sniffer’ [Technical Term J] API Out of Band Ethernet I/O Load CPU I/O SFX SFX Prefetch Prefetch Time ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. SFX is equally valuable in telling the system what not to cache 17
- 18. DDN Boosts FileSystem Speed, Access and Reliability Our Value-Add Approach To Accelerate I/O, Minimize TCO & Facilitate Collaboration. GRIDScaler™ EXAScaler™ ddn.com § § § § § Open-Platform File System Agenda Data Ubiquity: NAS, Parallel IO, Cloud Caching Tools To Accelerate I/O, APIs Collaborate and Archive Via The Cloud Easy To Manage Any Platform, Any Scale © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 18
- 19. WOS 3.0: TheWorld’s Most Scalable Web Storage Simple, Hyper-Efficient Object Storage from The Performance Leader 32 Trillion Object Scalability 10TB/s 256M throughput objects/sec Up To 64 Remote Sites API, REST, NAS Access 99.99999%+ Best-In-Class Cloud Availability ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 19
- 20. WOS Delivers TrueCross-Campus and Global Collaboration Compute Grid WOS™ Storage Cloud Site 1 Compute Grid Site 3 • Eliminate Research Data Silos • ‘Push Model’ Data Distribution Compute • Synchronize Independent File Grid Systems for Simple Replication • Access Data From A Variety of Site 4 Protocols Compute Grid Site 2 ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 20
- 21. DirectMon Next GenerationMonitoring Tool • • • • • • • ddn.com New Web Interface Deep Statistical Monitoring SFA Configuration Tools ‘Scaler Dashboard & Config Log & Config Management Call Home Enabled in Q4 Scale-Out Cluster Manager © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 21
- 22. DDN Delivers theMost Powerful & Comprehensive End to End Stack for HPC Parallel File System Clients NAS-Attached Clients DirectMon™ Infinite Memory Engine™ Burst Buffer System Multi-Protocol Access DDN Scale-Out Parallel File Systems ReACT™ and SFX™ Storage Caching WOS™ Bridge Cloud Connector and File System Data Replicator SFA™ Storage Appliances One Integrated HPC Storage Stack from The HPC Leader ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 22
- 23. Store Process Analyze Collaborate Archive Cloud #BurstForward Invent Discover Compete ddn.com © 2013 DataDirectNetworks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. Exascale I/O Platform Technology Introduction
- 24. Evolution of Data-IntensiveComputing CPU Average Top500 Cluster Concurrency 2010 12 Threads per Node 13,000 Cores 2018 (est.) O(1,000) Threads per Node 57,772,000 Cores 4,400x Increase ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 24
- 25. File Systems WillNot Survive Hyper-Concurrency File System Degradation vs. Concurrency Million and Billion Core Computers Need Smarter Solutions To Scale Applications and Meet Uptime Requirements Concurrency Degree (Percent) 100 90 80 <Today> <~2015> <Exascale> 70 60 50 40 30 Wall-Clock Degradation 20 10 0 1 10 100 1000 10000 100000 Source: DDN Testing; Exascale Research ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 25
- 26. NVRAM Presents NewArchitectural Opportunity NVRAM Is A Viable HPC Tier, Today $/MB/s: $0.29 (91% less) $/GB: $1 (1000% more) W/GB/s: 0.003 (98% less) And Performance Gains Outpace HDD 1E+10 Widening GHz 100000000 106 to 108 1000000 10000 $/MB/s: $3.19 $/GB: $0.1 W/GB/s: 0.13 IOPS 100 IOPS 1 2000 HDDs (IOPS) ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 2010 Flash 2018 CPU Operations 26
- 27. Store Process Analyze Collaborate Archive Cloud #BurstForward Invent Discover Compete ddn.com © 2013 DataDirectNetworks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 27
- 28. Introducing a RevolutionaryExascale/Everyscale Architecture Leveraging 15 Years of HPC Expertise, Engineering & Deployment Leadership • • • • • Dramatically Reduces HPC Computing Cost Enhances Performance & Power Usage by Orders of Magnitude Reduces Network & Server Amount & Complexity by Orders of Magnitude Patented Algorithms Deliver Unlimited Scale Liberates Applications From File System Bottlenecks ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 28
- 29. Next-Generation Distributed Memory Systemfrom the Worldwide HPC Leader • • • • • • ddn.com Speed Up Cluster I/O 1000x Divide Number of Disk Drives by a Factor of 10 Divide Rack Space by a Factor of 10 Divide Power Needs by a Factor of 10 Break Through POSIX Scaling Barriers Achieve Unlimited File System Scalability © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 29
- 30. Infinite Memory EngineDelivers Unparalleled Scale IME Metadata Scaling (24 Concurrent Threads) Frags/Sec (Million) 200 150 Linear Scalability! 100 50 0 0 1 2 3 Proven IME Scaling IM 'Max' ddn.com 4 5 Number of IME Nodes 6 7 8 9 200 PFLOPS Bid Requirement Trinity-Pessimistic-Req © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 30
- 31. The Infinite MemoryAdvantage Limitless Scalability Patented DDN Algorithms Fully POSIX & HPC Compatible No Application Modifications Scale-Out Data Protection Distributed Erasure Coding Non-Deterministic System Write Anywhere, No Layout Needed Integrated With File Systems Designed to Accelerate Lustre*, GPFS No Code Modification Needed Writes: Fast. Reads: They’re Fast Too. No other system offers both at scale. ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 31
- 32. IME Completes DDN’sHPC-Leading Caching Strategy The Ultimate I/O Deblender API SFA Storage Fusion Architecture Control RAM 7μs SFX Flash Tiering SSD 50μs HDD 17ms ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 32
- 33. IME HW SavingsGrow As Moore’s Law Advances IME Savings Grow Greater over Time Average HPC Cluster Example (2018) Machine Size/Checkpoint 100M Threads, 2PB Checkpoint in 5m 100% Disk Only Node Local IME + SFA 80% 7 TB/s 8 TB/s 60% 33,400 3,240 40% 100 12 $33M $7M The Growing HDD Gap Peak Performance Spinning Disks Racks 20% ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 0% 62% 80% 2016 Storage Acquisition Cost 2018 33
- 34. Store Process Analyze Collaborate Archive Cloud Live Demo RunningIn Denver! #BurstForward Invent Discover Compete ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 34
- 35. Worldwide Debut!! Running LiveIn DDN’s SC’13 Booth. Node Local Buffer Storage Topology Hybrid Compute & Storage 14 x Commodity Compute Nodes Each Node with 7 Low-Cost, Node-Local SSDs High-Speed Persistence Layer File System Replay to SFA7700 w/ Lustre* Near-Perfect Disk Utilization; Fully-Ordered I/O ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 35
- 36. Demo Comparative Testing:Shared Writes 14 IME Clients (Dell nodes); 98 SSDs Linear Cluster Scaling CLUSTER LEVEL TESTING DDN GRIDScaler™ IME( overall) 6,225 Concurrent Write Requests 43 GB/s 49 GB/s 12,250,000 Concurrent Write Requests 17 MB/s 49 GB/s DDN GRIDScaler™ (per SSD) IME (per SSD) 62.5 Concurrent Write Requests 438 MB/s 500 MB/s 125,000 Concurrent Write Requests 170 KB/s 500 MB/s DISK LEVEL TESTING SSDs behind a PFS don’t help IME is at line rate to scale with SSD rates ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 2,669x More Metadata Scalability Simulates System 1/4 th the size of The Average Top500 in 2018. Avg. 2018 Top500 Cluster Concurrency 57,772,000 Cores (est) 36
- 37. From Exascale toEveryscale, DDN Is THE HPC Leader The Worldwide HPC Storage Leader ▶ Liberating Applications From Infrastructure & File System Scaling Limits ▶ The Most Advanced HPC Stack To Minimize The Cost of Computing ▶ Unleashing Global Collaboration ▶ Simplifying Scale-Out for Everyscale ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 37
- 38. See You OnThe Floor... • We Have Terrific Speakers Lined Up Today! • Over 90 People From DDN At SC’13 • CTOs & Engineers Are Here In Force • Discussing Anything/Everything HPC • ExaScale to EveryScale • Geek Out On Our $M Demos • WOS, SFA, IME, Lustre*, Analytics, Etc. • Plan The Future of HPC In Our Whisper Suites See You @ Booth 1307 ddn.com © 2013 DataDirect Networks, Inc. * Other names and brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 38