James Coomer gave this talk at the DDN User Group at SC17. "Our technological and market leadership comes from our long-term investments in leading-edge research and development, our relentless focus on solving our customers’ end-to-end data and information management challenges, and the excellence of our employees around the globe, all relentlessly focused on delivering the highest levels of satisfaction to our customers. To meet these ever-increasing requirements, users are rapidly adopting DDN’s best-of-breed high-performance storage solutions for end-to-end data management from data creation and persistent storage to active archives and the Cloud." Watch the video: https://insidehpc.com/2017/12/video-ddn-applied-technologies-performance-use-cases/ Learn more: http://ddn.com Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1. DDN Storage | © 2017 DDN Storage DDN Confidential
DDN Applied
Technologies
DDN
Performance
and Use Cases

2. DDN Storage | © 2017 DDN Storage DDN Confidential
DDN
Life Sciences Machine Learning Oil and Gas
Full range of IO types: small file
random, sequential, workflows
Secure, tenanted, cloud ready
Cost-free data manipulation and
partitioning
Heavy Read Intensive workloads
Strong requirement for single
client performance
Squeeze more jobs into Compute
RTM, Kirchoff Migration, bring
extreme cases of large datasets and
sub optimal data reuse
Economic data at scale
Monitoring and user management

3. DDN Confidential DDN Storage | © 2017 DDN Storage DDN Confidential
Integrated File
System Appliances
SCALER™
Large Scale Cloud
& Archive Object
WOS®
SSD and NVMe
Based Scale-Out
IME™
Fast and Scalable
Data Platforms
SFA™
High Density, Power
Efficient SSD, SAS, SATA
& File System Platform
Fastest Time to Insight
Scale in GB/s, IOPS, TB
Mixed Workloads
Collaboration and
Distribution, Private
and Public Cloud
Acceleration of I/O,
Workflow, File System
and Applications
DDN | Software Defined, Flash, Hybrid and Cloud Storage

4. DDN Storage | © 2017 DDN Storage DDN Confidential
SFA | Embedded Appliances
EXAScaler and GRIDScaler - Parallel Filesystems Simplified
Up to 40GB/s Throughput
Millions of IOPS
ReACTTM for Mixed-IO
Mirrored Cache for small IO
Custom Full-Fabric Architecture
768 PCIe Lanes
OPA/EDR/FDR
10/40/100GbE
SFAOS Real-Time Parity
Engine
DirectProtect
Real-time Integrity
Partial rebuilds
Up to 1752 Devices
Over 20PB in 2 floor tiles
Hybrid SSD/HDD Architecture
SCALE UP or OUT DATA PROTECTION SCALABLE FABRICS EXTREME PERFORMANCE

5. DDN Storage | © 2017 DDN Storage DDN Confidential
Big Data NoSQL
Analytics
Supercomputing Multi-Physics Machine Learning Workflows Adaptive Mesh
Refinement
Checkpointing
Flash Economics
Achieve more with less. Less
space, less power, less cost.
Harness the economics of
Flash at Scale
Faster Insight
Performance like you’ve never
seen it. The fastest data
platform you can get.
Specifically designed for Flash
Smarter Productivity
Limitless Scaling. Improve
efficiency of compute, storage and
networking. Eliminate storage
bottlenecks.
$
SCALE-OUT FLASH
IME | Scale-Out Flash for Improved Price/Performance

6. DDN Storage | © 2017 DDN Storage DDN Confidential
Scalable Lustre
Monitoring
Lustre Persistent
Client Cache
Compressing copytool
for genomic data
Audit Logs
EXAScaler ZFS
QoS Planner:
Integration and
manageability of Lustre
QoS.
Secure multi-tenant
openstack storage
EXAScaler DDN
INDIANA
UNIVERSITY
Collaborations

7. DDN Storage | © 2017 DDN Storage DDN Confidential
EXAScaler DDN
Quota Accounting
CPU/memory usage
Bandwidth
#RPC vs IO size
IOPs Top User by Bandwidth Top User by Metadata
IME Bandwidth, IOPs, summary of PFS operations
► Highly Scalable to 1000’s of clients tested in
the field
► Advanced and relevant monitoring features
► Monitor Lustre File Systems
► Integration with DDN Hardware
► extended features: JOBSTATS, IME Integration
► Proven scalability to thousands of nodes and users)

8. DDN Storage | © 2017 DDN Storage DDN Confidential
EXAScaler DDN
0
50,000
100,000
150,000
200,000
250,000
300,000
4 8 16 32 64 128
ops/sec
Number of Threads
File Creation :mds-survey on ldiskfs
1 x MDS and 1 x MDT(2 x RAID1 SSD)
default
patch
72 10
400
4000
0
500
1000
1500
2000
2500
3000
3500
4000
4500
1
10
100
1000
10000
100000
1000000
10000000
SSD Based HDD HDD 40 pools Projected HDD
Read Random 4K IOPS on ES14KX
Number of Devices
Projected Performance for
Rotational Devices
MEASURED SSD Performance
4K IOPS (Log scale)
Number of
Devices

9. DDN Storage | © 2017 DDN Storage DDN Confidential
1 Million files per job are
common within life science
Sequencers & Microscopes
pumping out TB’s per hour
Diverse experience of user base
à improved monitoring
Full range of IO types: small
random, sequential, workflows
Secure, tenanted, cloud ready
EXAScaler
HDD Capacity
at scale
Scalable
Monitoring

10. DDN Storage | © 2017 DDN Storage DDN Confidential
Autonomous Vehicles | Machine Learning at Scale DDN
…
…
► Lustre Performance at Web Scale
► Over 175 Ethernet-Attached ES7KX
► Over 1.6TB/s Aggregate Performance
► Economical approach to very large capacities > 170PB
► Flat namespace performance for each Region
Ingest/Compression Preprocessing – Sub Sampling and Training – Machine Learning

11. DDN Storage | © 2017 DDN Storage DDN Confidential
Autonomous Vehicles | Machine Learning at Scale DDN
► Deploy Flash at Scale for Machine Learning
► Deploy in-sample image sets into scale-out flash
► Optimised Capacity/Large HDD/best economics
► IME extreme client performance to each node

12. DDN Storage | © 2017 DDN Storage DDN Confidential
IME | Reservoir Simulation Study
0
0.5
1
1.5
2
2.5
3
3.5
1 3 5 7 9 2 4 6 8 10
IO TIME (MIN)
WRITERS PER NODE
50M MODEL (50 WRITERS)
IME (IO) IME (SYNC) GPFS
0
2
4
6
8
10
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
IO TIME (MIN)
WRITERS PER NODE
200M MODEL (200 WRITERS)
0
20
40
60
80
100
9 9 9 9
IO TIME (MIN)
WRITERS PER NODE
200M MODEL (200 WRITERS)
INTENSIVE IO – 2
CONCURRENT JOBS
Reservoir simulation
Large Capacity GS
► Real-world reservoir simulation study comparing large HDD-based
GRIDScaler against a 4-node IME240 system
► Varying job count (1,2), Model Size (50M, 100M, 200M), number of
writers (50,100,200), etc across 50 compute nodes
► 4-5x IO time reductions with IME – speedup improves with concurrency
GRIDScaler
x5
x4
x2.5

13. DDN Storage | © 2017 DDN Storage DDN Confidential
DDN
Big Data NoSQL
Analytics
Supercomputing Multi-Physics Machine Learning Workflows Adaptive Mesh
Refinement
Checkpointing
DDN Applied Technologies
Delivering the Potential of Flash to Applications