Inder Monga, Chin Guok, Eric Pouyoul: ESnet Ilya Baldin, Paul Ruth: RENCI Simon Patton, Craig E. Tull: Berkeley Lab

1. 1 Beyond Today’s Internet • March 25, 2015
Beyond
Today’s
Internet
Experiencing
a
Smart
Future
Science DMZ as a Service:
Creating Science Super-
Facilities with GENI
Inder Monga, Chin Guok, Eric Pouyoul: ESnet
Ilya Baldin, Paul Ruth: RENCI
Simon Patton, Craig E. Tull: Berkeley Lab

2. 2 Beyond Today’s Internet • March 25, 2015
Inder
Monga
Ilya
Baldine
Simon
Pa2on
Eric
Pouyoul
Chin
Guok
Craig
Tull

3. 3 Beyond Today’s Internet • March 25, 2015
DOE
Facili*es

4. 4 Beyond Today’s Internet • March 25, 2015
Emerging Trend:
Super Facilities, Coupled by Networks
Experimental
facili*es
are
being
transformed
by
new
detectors,
advanced
mathema*cs,
robo*cs,
automa*on,
advanced
networks.

5. 5 Beyond Today’s Internet • March 25, 2015

6. 6 Beyond Today’s Internet • March 25, 2015
• Complexity
of
scien*fic
discovery
increasing
• Data
volumes
are
increasing
>
Moore’s
Law
• Fewer
large
facili*es,
but
global
scien*fic
popula*on
Automated
coupling
of
compute,
storage
with
networks
cri*cal
to
increasing
science
produc*vity

7. 7 Beyond Today’s Internet • March 25, 2015 Slide
from
Craig
E
Tull,
LBL

8. 8 Beyond Today’s Internet • March 25, 2015
Synchrotrons shed new light onto Sciences
Very diverse Science Impacts

9. 9 Beyond Today’s Internet • March 25, 2015
Super-Facility Illustrative Data Flow
Data collection
Transfer to NERSC
Analysis and modeling on NERSC supercomputers:
HipGISAXS simulation HipRMC fitting
FFT
Compare
start
with
random
system
move
parCcle
random
Autotuning
On-the-fly
calibration,
processing
Combining:
GIXSGUI, dpdak + …
Real-time
access via web portal

10. 10 Beyond Today’s Internet • March 25, 2015
Global data movement infrastructure
has challenging end-to-end requirements
Physical
pipe
that
leaks
water
at
rate
of
.0046%
by
volume.
è
è
Network
‘pipe’
that
drops
packets
at
rate
of
.0046%.
è
è
Result
100%
of
data
transferred,
slowly,
at
<<5%
opCmal
speed.
Result
99.9954%
of
water
transferred,
at
“line
rate.”
essenCally
fixed
determined
by
speed
of
light
Through
careful
engineering,
we
can
minimize
packet
loss.
AssumpCons:
10Gbps
TCP
flow,
80ms
RTT.
See
Eli
Dart,
Lauren
Rotman,
Brian
Tierney,
Mary
Hester,
and
Jason
Zurawski.
The
Science
DMZ:
A
Network
Design
Pa`ern
for
Data-­‐
Intensive
Science.
In
Proceedings
of
the
IEEE/ACM
Annual
SuperCompuBng
Conference
(SC13),
Denver
CO,
2013.

11. 11 Beyond Today’s Internet • March 25, 2015
End-­‐to-­‐end
architecture
is
cri*cal
Science
DMZ
has
three
key
components,
all
required:
• “FricCon
free”
network
path
– Highly
capable
network
devices
(wire-­‐speed,
deep
queues)
– Virtual
circuit
connecCvity
opCon
– Security
policy
and
enforcement
specific
to
science
workflows
– Located
at
or
near
site
perimeter
if
possible
• Dedicated,
high-­‐performance
Data
Transfer
Nodes
(DTNs)
– Hardware,
operaCng
system,
libraries
all
opCmized
for
transfer
– Includes
opCmized
data
transfer
tools
such
as
Globus
Online
and
GridFTP
• Performance
measurement/test
node
– perfSONAR
• Note:
General
Atomics
Science
DMZ
first
in
world
(we
think)
where
IPv6
was
chosen
for
performance
reasons.
Details
at
h`p://fasterdata.es.net/science-­‐dmz/
©
2013
Wikipedia

12. 12 Beyond Today’s Internet • March 25, 2015
Representative Science DMZ

13. 13 Beyond Today’s Internet • March 25, 2015
• Current implementations deploy dedicated DTNs in the Science DMZ
– Manual configuration and tuning
– Sharing is scheduled, many times manually
Dedicated vs. Virtual resources

14. 14 Beyond Today’s Internet • March 25, 2015
Dedicated
vs.
Virtual
resources
• GENI
provides
a
distributed
solware-­‐defined
infrastructure
(SDI)
– Compute
+
Storage
+
Network

15. 15 Beyond Today’s Internet • March 25, 2015
• GENI provides a distributed software-defined infrastructure (SDI)
– Compute + Storage + Network
• GENI racks may be deployed on-campus or in provider networks close
to the campus
• ‘Science DMZ as a service’
– Applications can provision a virtual ‘Science DMZ’ as and when needed
Dedicated vs. Virtual resources
Programmable
infrastructure
to
enable
end-­‐users
to
create
dynamic
‘fric*on-­‐free’
infrastructures
without
advanced
knowledge/training

16. 16 Beyond Today’s Internet • March 25, 2015
Today’s Demonstration:
Real-time data processing and vis. workflow
h`p://portal.nersc.gov/project/als/sc14/
Data
from
ALS
Experiment
SPADE
instance
@
Server
at
Argonne
ExoGENI
SPADE
VM
@
Starlight,
Chicago
ESnet
ExoGENI
SPADE
VM
@
Oakland,
California
Compute
Cluster
NERSC,
LBL
AL2S,
ESnet
• WAN-­‐op*mized
data
transfer
nodes
and
a
network
slice
created
programma*cally
(Science
DMZ
as
a
service)
• Applica*on
workflow
instan*ated
to
stage
data
at
the
GENI
rack
on
Science
DMZ
slice
• Data
is
moved
op*mally
across
the
WAN1
1
Earlier
work,
like
Phoebus,
have
instanCated
the
value
of
this
approach

17. 17 Beyond Today’s Internet • March 25, 2015
Microtomography of High Temperature
Materials under stress
Set
collected
by
materials
scienCst
Rob
Ritchie,
LBNL/UCB

18. 18 Beyond Today’s Internet • March 25, 2015
Summary and Future
• Many
science
applicaCons
are
now
building
coupled
workflows
across
the
WAN
to
Ce
together
specialized
science
instruments
for
‘big
data’
discovery
• Programmability
extends
to
the
enCre
‘Solware-­‐defined
infrastructure’
stack
as
illustrated
by
GENI
racks
• WAN
‘Infrastructure
as
a
Service’
soluCons
will
become
pervasive,
as
programmability
and
virtualizaCon
moves
outside
the
data
center
domain.
• We
think
of
the
‘network’
as
an
‘instrument’
that
enables
scienCfic
discovery
• Should
there
be
a
disCncCon
between
a
network
and
a
data
center?
• What
will
the
next
generaCon
WAN
look
like?

19. 19 Beyond Today’s Internet • March 25, 2015
THANK
YOU
Acknowledgements:
DOE
ASCR
Funding
and
Support,
Jason
Lee,
Brent
Draney
and
NERSC
networking
team,
Dula
Parkinson
(ALS),
Linda
Winkler
and
Argonne
Networking
Team
(ANL)
imonga@es.net,
ibaldin@renci.org,
cetull@lbl.gov