In this presentation from the Dell booth at SC13, Erik Deumans from the University of Florida describes how Big Data gets handled for HPC at the University. "As science drives a rapidly growing need for storage, existing environments face increasing pressure to expand capabilities while controlling costs. Many researchers, scientists and engineers find that they are outgrowing their current system, but fear their organizations may be too small to cover the cost and support needed for more storage. Join these experts for a lively discussion on how you can take control and solve the HPC data deluge." Watch the video presentation: http://insidehpc.com/2013/12/03/panel-discussion-solving-hpc-data-deluge/

1. Big
Data
is
like
HPC
all
over
again
• From
serial,
to
SMP
parallel,
to
MPI
parallel,
to
GPUs
• Big
Data
grows
from
– File
systems,
to
– Shared
parallel
file
systems,
to
– Hadoop
Distributed
File
System
– Distributed
data
repositories
connected
by
Globus-­‐Online-­‐like
data
transfers
Nov 20, 2013
1

2. We
have
been
there…
Programming
and
control
High-­‐performance
compu5ng
Big
Data
analy5cs
serial
Single
processor,
point-­‐to-­‐point
communicaIon
Single-­‐file
data
analysis
Dropbox-­‐style
upload
and
synchronizaIon
Tightly
coupled
Shared-­‐memory
mulI-­‐
processors
Parallel
file-­‐system
across
site
(Lustrre,
GPFS,…)
Loosely
coupled
Distributed
memory
clusters
Map-­‐reduce,
workflows,
explicit
data
movement
(Globus
Online,…)
Nov 20, 2013
2