Your SlideShare is downloading. ×
0
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
VCOC BonFIRE presentation at FIRE Engineering Workshop 2012
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

VCOC BonFIRE presentation at FIRE Engineering Workshop 2012

314

Published on

VCOC experiment in BonFIRE European Project (http://www.bonfire-project.eu) results. It shows a general architecture with fault-tolerance to use in fistributed Cloud environments and the usage of …

VCOC experiment in BonFIRE European Project (http://www.bonfire-project.eu) results. It shows a general architecture with fault-tolerance to use in fistributed Cloud environments and the usage of application performance indicators to trigger cluster elasticity. More information at www.cesga.es.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
314
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Building service testbeds on FIREExperimenting Virtual Clusters on distributed Cloud environments using BonFIREA. Gómez, L.M. Carril, R. Valin, J.C. Mouriño, C. Cotelo FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012
  • 2. Overview  Objective of experiment  Experiment Set-up  Virtual Cluster Architecture  Experiments about time-to-deploy  Elasticity experiment  Fault-tolerance experiment  Conclusions2 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 2
  • 3. Objective of Experiment Research the feasibility of using multiple Cloud environments for the provision of Services which need a large pool of CPUs (as High Throughput Computing). Why3 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 3
  • 4. Objective of Experiment (II)  Experiments about time-to-deploy the virtual infrastructure.  Experiments about elasticity triggered by application performance.  Experiments about fault-tolerant virtual infrastructure.4 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 4
  • 5. EXPERIMENT SET-UP5 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 5
  • 6. BonFIRE InfrastructureINRIA: Vendor Freq. (GHz) Cores RAM (GB) HLRS: Vendor Freq. (GHz) Cores RAM (GB) Intel 2.33 2*2 4 Intel 3.2 2*2 2 AMD 1,7 2*12 48 Intel 2.66 2*2 8 Intel 2,5 2*4 32 AMD 2.6 4*12 196 Intel 2.93 2*4 24 AMD 2 2 4 Intel I7 2.53 2 4 Intel I7 2.1 4 8 Intel Atom 1 2 AMD 1.65 2 2 T56N Cloud Manager: OpenNebula 3.0 Source: http://www.bonfire-project.eu/infrastructure (Nov. 2012) 6 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 6
  • 7. VCOC Experiment Agent  Manages multiple experiment executions.  Works with the Experiment Manager.  Controls deployed resources.  Stores Experiment data.7 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 7
  • 8. Experiment Agent Workflow Cluster ready8 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 8
  • 9. Elasticity Engine  Takes information about application performance  Get data about queues  Calculates Nr. VMs to fulfill demanded performance  Adds new VMs to Experiment9 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 9
  • 10. VCOC Interfaces Monitoring GUI dashboard Identity LDAP Portal Server (Used by Portal,VCOC Experiment Agent Experiment Manager, API Resource Manageruses Experiment Manager and Testbeds) Experiment Manager Portal UsageElasticity Engine uses OCCI Monitoring Reservation Accounting API AuthZResource Manager (Used by RM to authorize resource Resource Manager creation) OCCI Reservation Message Read/ Write Queue Enactor OCCI Reservation SSH Testbed SSH Monitoring Monitoring SSH Gateway GUI API VM Gateway (Monitoring VM VM Aggregator) 10 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 10
  • 11. VIRTUAL CLUSTER ARCHITECTURE11 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 11
  • 12. VC Configuration on BonFIRE12 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 12
  • 13. Simple VC Configuration13 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 13
  • 14. Distributed VC14 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 14
  • 15. Fault-tolerance VC configuration15 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 15
  • 16. Name of the Managed Experiment JSON VC Example { "name": "Virtual Cluster Experiment", Maximum elapsed time for the Experiment in minutes "description": "Virtual Cluster", "duration": 180, Description of the experiment "resources": [ { "compute": { "name": "master", Location for deployment "description": "Virtual Cluster Master", "instanceType": "small", "locations": ["fr-inria"], Main Storage Resource "resources": [ { "storage": "@VirtualClusterMaster-vcocdemo-v0.1"}, { "storage": "@vcocdemo_volume-1GB-empty"}, Master Virtual Machine Configuration { "network": "@BonFIRE WAN"} ], "contexts": [ Additional Storage Resource { } ] Network } }, { "compute": { "name": "client-1", "description": "Virtual Cluster Client", "instanceType": "small", "locations": ["fr-inria"], "resources": [ { "storage": "@VirtualClusterClient-vcocdemo-v0.1"}, { "network": "@BonFIRE WAN"} Client Virtual Machine Description ], "contexts": [ { } ] } }16 ] } VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 16
  • 17. EXPERIMENTS ABOUT TIME-TO-DEPLOY17 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 17
  • 18. Experiment data  5 repetitions Maximum  Representation: Box plot Average Median Minimum18 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 18
  • 19. Deployment time vs VC sizeBonFIRE site: INRIA Experiment Manager Start VMsReserved: 2 physical nodes with2x12 cores AMD Opteron 1.7 GHzand 48 GB RAMVC Configuration:• Simple VC• One master (small + 4GB storage)• n-1 CEs (small) Conf. OGS Undeployment19 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 19
  • 20. Deployment time vs storage size Without cacheBonFIRE site: INRIAReserved: 2 physical nodes with2x12 cores AMD Opteron 1.7 GHzand 48 GB RAMVC Configuration: Simple VC 16VM One master (small + x GB storage) 15 CEs (small) With Cache 20 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 20
  • 21. Two clusters in one site Second cluster delayed because the deployment of first VC. BonFIRE site: INRIA Reserved: 2 physical nodes with 2x12 cores AMD Opteron First Virtual Cluster 1.7 GHz and 48 GB RAM VC Configuration: 4 VM 8 VM  Simple VC  One master (small + 4GB storage)  n-1 CEs (small) Second Virtual Cluster21 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 21
  • 22. Application execution. One vs Two sites VC Conf.: Distributed VC (_dist) BonFIRE sites:  INRIA: Master + CEs  HLRS: CEs Deployment time decreases. Two sites faster than one site. But because second site has better CPUs. Impact of deployment ~ 10% total time. 22 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 22
  • 23. Adding a new VM Client VC Conf.: Simple VC BonFIRE sites:  INRIA: Master + CEs Using BonFIRE Resource Manager Very fast due to cache 23 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 23
  • 24. ELASTICITY EXPERIMENT24 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 24
  • 25. Horizontal elasticity Monitoring application performance works. We have modified software to produce information more frequently. Execution with deadline. Elasticity works.25 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 25
  • 26. FAULT TOLERANCE EXPERIMENT WITH ELASTICITY26 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 26
  • 27. Virtual Cluster SYNC27 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 27
  • 28. Fault tolerance  BonFIRE sites:  HLRS (Master + 4 CEs)  INRIA (Shadow + 4 CEs)  Demanded performance (500H/s)  Fault simulated putting HLRS VMs in CANCEL.  INRIA Shadow took control of cluster.  Elasticity worked, demanding more CEs to INRIA.28 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 28
  • 29. CONCLUSIONS29 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 29
  • 30. Conclusions  BonFIRE provides infrastructure for experiments about new concepts and services on Cloud.  Distributed VC can be used to speed up HTC applications.  Elasticity engine based on application performance for HTC works.  High QoS can be provided in VC using distributed VC + elasticity.30 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 30
  • 31. Acknowledgment To all BonFIRE team, specially to:  Maxence Dunnewind (INRIA)  Michael Gienger (HLRS)  Ally Hume (EPCC)  Kostas Kavoussanakis (EPCC)  David García Pérez (Atos Research)31 VCOC, FIRE Engineering Workshop, Ghent, Nov. 6th – 7th 2012 BonFIRE 31
  • 32. Building service testbeds on FIREThank you for your attention Project contact: bonfire@bonfire-project.eu Experiment Contact: agomez@cesga.es

×