In this deck, Gabor Samu from IBM describes the new IBM Platform Data Manager for LSF. IBM Platform Data Manager for LSF takes control of data transfers to help organizations improve data throughput and lower costs by minimizing wasted compute cycles and conserving disk space. Platform Data Manager automates the transfer of data used by application workloads running on IBM Platform LSF clusters and the cloud, bringing frequently used data closer to compute resources by storing it in a smart, managed cache that can be shared among users and workloads. Watch the video presentation: http://wp.me/p3RLHQ-enN Learn more: http://www-03.ibm.com/systems/platformcomputing/products/lsf/datamanager.html Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

1. © 2015 IBM Corporation
IBM Platform Data Manager for LSF
Intelligent, Managed Data Staging
Gábor Samu, Portfolio Marketing Manager
Software Defined Infrastructure
gsamu@ca.ibm.com

2. © 2015 IBM Corporation2
The IBM Platform LSF Family
IBM
Platform
RTM
IBM
Platform
Analytics
IBM
Platform
Process
Manager
IBM
Platform
Application
Center
IBM
Platform
Dynamic
Cluster
IBM
Platform
License
Scheduler
IBM
Platform
Session
Scheduler
Platform
LSF
Scheduling
Efficiency
MapReduce
Accelerator
Platform HPC
Docker
Connector
Hadoop
Connector
IBM
Platform
Data
Manager
Platform MPI
Platform PCM

3. © 2015 IBM Corporation3
Challenges with data management in HPC environments
• Data is not available on the compute resources when needed
• Transfers of data done “in-band” – idle CPUs waiting for transfers
• Wasted bandwidth, storage on duplicate transfers
• What is the state of data transfers?
• Single user copying the same data repeatedly
• Multiple users transferring the same data repeatedly
Compute power requires data to
operate on

4. © 2015 IBM Corporation4
Intelligent, managed data staging
§ Workload independent, managed data
transfers
§ Eliminate duplicate transfers
§ Lower storage costs
§ Visibility of data transfer traffic
IBM Platform Data Manager for LSF
Whether in the cloud or working locally, ensure that your
data is in the right location at the right time with
intelligent caching and out-of-band transfers, helping to
reduce costs and overall time to solution.

5. © 2015 IBM Corporation5
Unique data staging capabilities
• Fully integrated with IBM Platform LSF
• Managed movement of data within and between Platform LSF clusters, with control over policies and
priority.
• Control over out-of-band movement of data
• Preventing wasted compute cycles.
• Eliminate redundant transfers with intelligent caching of data.
• Data affinity
• For environments consisting of multiple IBM Platform LSF clusters, factor in data availability in
scheduling
• Configurable file transfer mechanism
• Administrators may configure IBM Platform Data Manager to use the underlying file transfer
mechanism (e.g. scp, gridftp, IBM Aspera)

6. © 2015 IBM Corporation6
Data affinity – a closer look
Cache
(file TUV789)
Platform LSF
Cluster C
Platform Data
Manager
Cache
(file XYZ456)
Platform LSF
Cluster B
Platform Data
Manager
Cache
(file XABCD123)
Platform LSF
Cluster D
Platform Data
Manager
Cache
Platform LSF
Cluster A
Platform Data
Manager
Platform Data Manager makes
data availability a factor when
forwarding to remote clusters!
My job requires
file XABCD123
Job forwarded to Cluster D,
where requested data file is
cached

7. © 2015 IBM Corporation7
IBM Spectrum Scale (Persistent Data Source)
Flash (Data Manager Cache)
COMPUTE HOSTS
Transfer Job
Platform LSF job asks
for input data file on
slow shared file system
Platform Data Manager
pre-stages the file into
fast flash cache before
job execution
Job accesses the data
file from the cache and
writes its output to
flash
Platform Data manager
drains the output from
the cache into
persistent storage after
job execution.
HPC perspective – Burst Buffers
Platform LSF
&
Platform Data
Manager
Transfer Job

8. © 2015 IBM Corporation8
Thank you
For more information:
http://www.ibm.com/systems/platformcomputing/products/lsf