Presenting - Distem!

1. an introduction to
Distem – http://distem.gforge.inria.fr/ 1 / 16

2. Imagine you want to build a race car
Distem – http://distem.gforge.inria.fr/ 2 / 16

3. But you want it to work well
on both dry tracks and wet tracks
Distem – http://distem.gforge.inria.fr/ 3 / 16

4. You could use a simulation :
model the car and the track
and compute how they would interact
Distem – http://distem.gforge.inria.fr/ 4 / 16

5. Or you could use a real world experiment :
build a car, and try it on a real circuit
Distem – http://distem.gforge.inria.fr/ 5 / 16

6. Or you could use a real world experiment :
build a car, and try it on a real circuit
But sometimes, rain is unlikely
Distem – http://distem.gforge.inria.fr/ 5 / 16

7. You could move to another race track . . .
But what if you could change the weather ?
Distem – http://distem.gforge.inria.fr/ 6 / 16

8. You could move to another race track . . .
But what if you could change the weather ?
That’s what Distem does :
Take a platform, and alter it to suit
your experimentation needs
Distem – http://distem.gforge.inria.fr/ 6 / 16

9. An emulator for distributed systems
Take your real application
Run it on a cluster
And use Distem to alter the platform
so it matches the experimental conditions you need
Heterogeneous nodes
Long distance networks
Grid, Cloud, P2P features
...
Distem – http://distem.gforge.inria.fr/ 7 / 16

10. Ideal complement to other tools
SimGrid Grid’5000
Toolkit for the simulation of Large-scale platform
distributed applications in for experiments on
heterogeneous environments distributed systems
1700 nodes, 7000 cores
Fully reconfigurable
SMPI GRAS
SimDag MSG
Grid’5000
GRE: GRAS in situ
SMURF
SimIX network proxy
SimIX
"POSIX−like" API on a virtual platform
SURF
virtual platform simulator
XBT
Distem – http://distem.gforge.inria.fr/ 8 / 16

11. Ideal complement to other tools
SimGrid Grid’5000
Toolkit for the simulation of Large-scale platform
distributed applications in for experiments on
heterogeneous environments distributed systems
1700 nodes, 7000 cores
Fully reconfigurable
SMPI GRAS
SimDag MSG
Grid’5000
GRE: GRAS in situ
SMURF
SimIX network proxy
SimIX
"POSIX−like" API on a virtual platform
SURF
virtual platform simulator
XBT
Distem can run on Grid’5000
Distem – http://distem.gforge.inria.fr/ 8 / 16

12. What can Distem do for you ?
Introduce heterogeneity in an homogeneous cluster
How does your app perform when some nodes are slower ?
Emulate complex network topologies
How does your app perform on a Grid ? on a slower network ?
Distem – http://distem.gforge.inria.fr/ 9 / 16

13. Introducing heterogeneity
Distem splits real nodes into several virtual nodes
with a different number of cores
with different CPU performance
CPU cores
0 1 2 3 4 5 6 7
CPU performance
VN 1 VN 2 VN 3 Virtual node 4
Distem – http://distem.gforge.inria.fr/ 10 / 16

14. Emulating network topologies
Emulate several local networks linked together
Control available bandwidth and latency on each interface
n1 n4
if0 ← 0
1 ms if
1 M Mb 00
, 2 s→
bp ps
,3 ps
s, 0m Kb 00m
30
ms s 10 ,1
→ ← bps
K
←5 Mbps, 10ms ←4 Mbps, 12ms 20
10 Mbps, 5ms→
if0 n3 if1
6 Mbps, 16ms→ ←
ms 20
,3 51 0
ps → 2 K Kbp
0 Mb 1ms bp s,
10 s, s, 30
40 ms
← Mbp ms
0 →
if0 10 if0
n2 n5
Distem – http://distem.gforge.inria.fr/ 11 / 16

15. Distem internals
Distem uses modern Linux features
to steal resources from applications :
LXC, cgroups, cpufreq, iptables, traffic control
Node 3 Node 3
Switch Switch
Node 2 Node 2
Node 1 Node 1
Distem – http://distem.gforge.inria.fr/ 12 / 16

16. User interfaces
Command-line interface – distem command
Easy to use
Harder to automate
No access to more advanced features
Ruby library
Easy to automate
Access to all features
/ Easy to use if you known Ruby
REST API
Can be used from any language
Requires REST knowledge
Distem – http://distem.gforge.inria.fr/ 13 / 16

17. Overview of a Distem instance
Pnode 2
distemd
Coordinator
ST
RE
& Pnode 1
REST
User’s machine distemd
Uses the command line interface,
the Ruby client library
Pnode 3
or a REST client
distemd
Starts and controls other Pnodes
Keeps the global state of the platform
Is the gateway to Pnodes and Vnodes
Vnodes
Distem – http://distem.gforge.inria.fr/ 14 / 16

18. Example experiment : SCP vs Rsync
Transfer 50 files (total : 5 MB)
Available bandwidth : 10 Mbps
Varying network latency
SCP
Rsync
Transfer time (s)
20
10
0
0 20 40 60 80 100
Emulated latency (ms)
Distem – http://distem.gforge.inria.fr/ 15 / 16

19. More information ?
http://distem.gforge.inria.fr/
Distem – http://distem.gforge.inria.fr/ 16 / 16