Your SlideShare is downloading. ×
Distem
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Distem

326
views

Published on

Presenting - Distem!

Presenting - Distem!

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
326
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. an introduction toDistem – http://distem.gforge.inria.fr/ 1 / 16
  • 2. Imagine you want to build a race car Distem – http://distem.gforge.inria.fr/ 2 / 16
  • 3. But you want it to work wellon both dry tracks and wet tracks Distem – http://distem.gforge.inria.fr/ 3 / 16
  • 4. You could use a simulation : model the car and the trackand compute how they would interact Distem – http://distem.gforge.inria.fr/ 4 / 16
  • 5. Or you could use a real world experiment : build a car, and try it on a real circuit Distem – http://distem.gforge.inria.fr/ 5 / 16
  • 6. Or you could use a real world experiment : build a car, and try it on a real circuit But sometimes, rain is unlikely Distem – http://distem.gforge.inria.fr/ 5 / 16
  • 7. You could move to another race track . . .But what if you could change the weather ? Distem – http://distem.gforge.inria.fr/ 6 / 16
  • 8. You could move to another race track . . .But what if you could change the weather ? That’s what Distem does : Take a platform, and alter it to suit your experimentation needs Distem – http://distem.gforge.inria.fr/ 6 / 16
  • 9. An emulator for distributed systems Take your real application Run it on a cluster And use Distem to alter the platformso it matches the experimental conditions you need Heterogeneous nodes Long distance networks Grid, Cloud, P2P features ... Distem – http://distem.gforge.inria.fr/ 7 / 16
  • 10. Ideal complement to other tools SimGrid Grid’5000 Toolkit for the simulation of Large-scale platform distributed applications in for experiments onheterogeneous environments distributed systems 1700 nodes, 7000 cores Fully reconfigurable SMPI GRAS SimDag MSG Grid’5000 GRE: GRAS in situ SMURF SimIX network proxy SimIX "POSIX−like" API on a virtual platform SURF virtual platform simulator XBT Distem – http://distem.gforge.inria.fr/ 8 / 16
  • 11. Ideal complement to other tools SimGrid Grid’5000 Toolkit for the simulation of Large-scale platform distributed applications in for experiments onheterogeneous environments distributed systems 1700 nodes, 7000 cores Fully reconfigurable SMPI GRAS SimDag MSG Grid’5000 GRE: GRAS in situ SMURF SimIX network proxy SimIX "POSIX−like" API on a virtual platform SURF virtual platform simulator XBT Distem can run on Grid’5000 Distem – http://distem.gforge.inria.fr/ 8 / 16
  • 12. What can Distem do for you ?Introduce heterogeneity in an homogeneous clusterHow does your app perform when some nodes are slower ?Emulate complex network topologiesHow does your app perform on a Grid ? on a slower network ? Distem – http://distem.gforge.inria.fr/ 9 / 16
  • 13. Introducing heterogeneityDistem splits real nodes into several virtual nodes with a different number of cores with different CPU performance CPU cores 0 1 2 3 4 5 6 7 CPU performance VN 1 VN 2 VN 3 Virtual node 4 Distem – http://distem.gforge.inria.fr/ 10 / 16
  • 14. Emulating network topologiesEmulate several local networks linked togetherControl available bandwidth and latency on each interface n1 n4 if0 ← 0 1 ms if 1 M Mb 00 , 2 s→ bp ps ,3 ps s, 0m Kb 00m 30 ms s 10 ,1 → ← bps K ←5 Mbps, 10ms ←4 Mbps, 12ms 20 10 Mbps, 5ms→ if0 n3 if1 6 Mbps, 16ms→ ← ms 20 ,3 51 0 ps → 2 K Kbp 0 Mb 1ms bp s, 10 s, s, 30 40 ms ← Mbp ms 0 → if0 10 if0 n2 n5 Distem – http://distem.gforge.inria.fr/ 11 / 16
  • 15. Distem internals Distem uses modern Linux features to steal resources from applications :LXC, cgroups, cpufreq, iptables, traffic control Node 3 Node 3 Switch Switch Node 2 Node 2 Node 1 Node 1 Distem – http://distem.gforge.inria.fr/ 12 / 16
  • 16. User interfacesCommand-line interface – distem command Easy to use Harder to automate No access to more advanced featuresRuby library Easy to automate Access to all features / Easy to use if you known RubyREST API Can be used from any language Requires REST knowledge Distem – http://distem.gforge.inria.fr/ 13 / 16
  • 17. Overview of a Distem instance Pnode 2 distemd Coordinator ST RE & Pnode 1 REST User’s machine distemdUses the command line interface, the Ruby client library Pnode 3 or a REST client distemd Starts and controls other Pnodes Keeps the global state of the platform Is the gateway to Pnodes and Vnodes Vnodes Distem – http://distem.gforge.inria.fr/ 14 / 16
  • 18. Example experiment : SCP vs Rsync Transfer 50 files (total : 5 MB) Available bandwidth : 10 Mbps Varying network latency SCP RsyncTransfer time (s) 20 10 0 0 20 40 60 80 100 Emulated latency (ms) Distem – http://distem.gforge.inria.fr/ 15 / 16
  • 19. More information ?http://distem.gforge.inria.fr/ Distem – http://distem.gforge.inria.fr/ 16 / 16