an introduction toDistem – http://distem.gforge.inria.fr/   1 / 16
Imagine you want to build a race car         Distem – http://distem.gforge.inria.fr/   2 / 16
But you want it to work wellon both dry tracks and wet tracks        Distem – http://distem.gforge.inria.fr/   3 / 16
You could use a simulation :    model the car and the trackand compute how they would interact         Distem – http://dis...
Or you could use a real world experiment :   build a car, and try it on a real circuit              Distem – http://distem...
Or you could use a real world experiment :   build a car, and try it on a real circuit      But sometimes, rain is unlikel...
You could move to another race track . . .But what if you could change the weather ?             Distem – http://distem.gf...
You could move to another race track . . .But what if you could change the weather ?        That’s what Distem does :    T...
An emulator for distributed systems            Take your real application                 Run it on a cluster        And u...
Ideal complement to other tools            SimGrid               Grid’5000 Toolkit for the simulation of                  ...
Ideal complement to other tools            SimGrid               Grid’5000 Toolkit for the simulation of                  ...
What can Distem do for you ?Introduce heterogeneity in an homogeneous clusterHow does your app perform when some nodes are...
Introducing heterogeneityDistem splits real nodes into several virtual nodes   with a different number of cores   with dif...
Emulating network topologiesEmulate several local networks linked togetherControl available bandwidth and latency on each ...
Distem internals    Distem uses modern Linux features   to steal resources from applications :LXC, cgroups, cpufreq, iptab...
User interfacesCommand-line interface – distem command     Easy to use  Harder to automate  No access to more advanced fea...
Overview of a Distem instance                                                                                   Pnode 2   ...
Example experiment : SCP vs Rsync                         Transfer 50 files (total : 5 MB)                         Availabl...
More information ?http://distem.gforge.inria.fr/        Distem – http://distem.gforge.inria.fr/   16 / 16
Upcoming SlideShare
Loading in …5
×

Distem

687 views
510 views

Published on

Presenting - Distem!

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
687
On SlideShare
0
From Embeds
0
Number of Embeds
17
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Distem

  1. 1. an introduction toDistem – http://distem.gforge.inria.fr/ 1 / 16
  2. 2. Imagine you want to build a race car Distem – http://distem.gforge.inria.fr/ 2 / 16
  3. 3. But you want it to work wellon both dry tracks and wet tracks Distem – http://distem.gforge.inria.fr/ 3 / 16
  4. 4. You could use a simulation : model the car and the trackand compute how they would interact Distem – http://distem.gforge.inria.fr/ 4 / 16
  5. 5. Or you could use a real world experiment : build a car, and try it on a real circuit Distem – http://distem.gforge.inria.fr/ 5 / 16
  6. 6. Or you could use a real world experiment : build a car, and try it on a real circuit But sometimes, rain is unlikely Distem – http://distem.gforge.inria.fr/ 5 / 16
  7. 7. You could move to another race track . . .But what if you could change the weather ? Distem – http://distem.gforge.inria.fr/ 6 / 16
  8. 8. You could move to another race track . . .But what if you could change the weather ? That’s what Distem does : Take a platform, and alter it to suit your experimentation needs Distem – http://distem.gforge.inria.fr/ 6 / 16
  9. 9. An emulator for distributed systems Take your real application Run it on a cluster And use Distem to alter the platformso it matches the experimental conditions you need Heterogeneous nodes Long distance networks Grid, Cloud, P2P features ... Distem – http://distem.gforge.inria.fr/ 7 / 16
  10. 10. Ideal complement to other tools SimGrid Grid’5000 Toolkit for the simulation of Large-scale platform distributed applications in for experiments onheterogeneous environments distributed systems 1700 nodes, 7000 cores Fully reconfigurable SMPI GRAS SimDag MSG Grid’5000 GRE: GRAS in situ SMURF SimIX network proxy SimIX "POSIX−like" API on a virtual platform SURF virtual platform simulator XBT Distem – http://distem.gforge.inria.fr/ 8 / 16
  11. 11. Ideal complement to other tools SimGrid Grid’5000 Toolkit for the simulation of Large-scale platform distributed applications in for experiments onheterogeneous environments distributed systems 1700 nodes, 7000 cores Fully reconfigurable SMPI GRAS SimDag MSG Grid’5000 GRE: GRAS in situ SMURF SimIX network proxy SimIX "POSIX−like" API on a virtual platform SURF virtual platform simulator XBT Distem can run on Grid’5000 Distem – http://distem.gforge.inria.fr/ 8 / 16
  12. 12. What can Distem do for you ?Introduce heterogeneity in an homogeneous clusterHow does your app perform when some nodes are slower ?Emulate complex network topologiesHow does your app perform on a Grid ? on a slower network ? Distem – http://distem.gforge.inria.fr/ 9 / 16
  13. 13. Introducing heterogeneityDistem splits real nodes into several virtual nodes with a different number of cores with different CPU performance CPU cores 0 1 2 3 4 5 6 7 CPU performance VN 1 VN 2 VN 3 Virtual node 4 Distem – http://distem.gforge.inria.fr/ 10 / 16
  14. 14. Emulating network topologiesEmulate several local networks linked togetherControl available bandwidth and latency on each interface n1 n4 if0 ← 0 1 ms if 1 M Mb 00 , 2 s→ bp ps ,3 ps s, 0m Kb 00m 30 ms s 10 ,1 → ← bps K ←5 Mbps, 10ms ←4 Mbps, 12ms 20 10 Mbps, 5ms→ if0 n3 if1 6 Mbps, 16ms→ ← ms 20 ,3 51 0 ps → 2 K Kbp 0 Mb 1ms bp s, 10 s, s, 30 40 ms ← Mbp ms 0 → if0 10 if0 n2 n5 Distem – http://distem.gforge.inria.fr/ 11 / 16
  15. 15. Distem internals Distem uses modern Linux features to steal resources from applications :LXC, cgroups, cpufreq, iptables, traffic control Node 3 Node 3 Switch Switch Node 2 Node 2 Node 1 Node 1 Distem – http://distem.gforge.inria.fr/ 12 / 16
  16. 16. User interfacesCommand-line interface – distem command Easy to use Harder to automate No access to more advanced featuresRuby library Easy to automate Access to all features / Easy to use if you known RubyREST API Can be used from any language Requires REST knowledge Distem – http://distem.gforge.inria.fr/ 13 / 16
  17. 17. Overview of a Distem instance Pnode 2 distemd Coordinator ST RE & Pnode 1 REST User’s machine distemdUses the command line interface, the Ruby client library Pnode 3 or a REST client distemd Starts and controls other Pnodes Keeps the global state of the platform Is the gateway to Pnodes and Vnodes Vnodes Distem – http://distem.gforge.inria.fr/ 14 / 16
  18. 18. Example experiment : SCP vs Rsync Transfer 50 files (total : 5 MB) Available bandwidth : 10 Mbps Varying network latency SCP RsyncTransfer time (s) 20 10 0 0 20 40 60 80 100 Emulated latency (ms) Distem – http://distem.gforge.inria.fr/ 15 / 16
  19. 19. More information ?http://distem.gforge.inria.fr/ Distem – http://distem.gforge.inria.fr/ 16 / 16

×