Your SlideShare is downloading. ×
VC-Migration: Live Migration of Virtual Clusters in the Cloud Kejiang Ye, Xiaohong Jiang, Ran Ma, Fengxi Yan     CCNT Lab,...
Outline   Motivations   VC-Migration Framework   Virtual Cluster Performance       Cross-Domain Virtual Cluster      ...
Outline   Motivations   VC-Migration Framework   Virtual Cluster Performance       Cross-Domain Virtual Cluster      ...
Motivations   Currently, live migration of virtual machine    s has become a key ingredient behind the    management acti...
Motivations   Virtual Cluster is a group of virtual machines conf    igured for a common purpose, such as high perfor    ...
Motivations   Live migration of virtual clusters (VC) faces s    everal new challenges:       Huge Amount of Data      ...
Contributions Describe a framework VC-Migration to co  ntrol the migration of virtual clusters. Perform a series of expe...
Outline   Motivations   VC-Migration Framework   Virtual Cluster Performance       Cross-Domain Virtual Cluster      ...
VC-Migration Framework   Framework Design                        GRID 2012               Sep. 20-23, 2012 Beijing, China
VC-Migration Framework   VC-Migration Scenario                         GRID 2012                Sep. 20-23, 2012 Beijing,...
VC-Migration Framework   VC-Migration Strategies                                                   a. Concurrent         ...
Outline   Motivations   VC-Migration Framework   Virtual Cluster Performance       Cross-Domain Virtual Cluster      ...
Virtual Cluster Performance   Experimental Configuration       Virtual Cluster Configuration            Dell T710 Serve...
Virtual Cluster Performance   Cross-Domain Virtual Cluster                         GRID 2012                Sep. 20-23, 2...
Virtual Cluster Performance   Virtual Cluster Scalability                           GRID 2012                  Sep. 20-23...
Outline   Motivations   VC-Migration Framework   Virtual Cluster Performance       Cross-Domain Virtual Cluster      ...
Dynamic Migration Strategies of Virtual Cluster   Effects of Memory Configuration                         GRID 2012      ...
Dynamic Migration Strategies of Virtual Cluster   Migration Scalability                    Overall Time                  ...
Dynamic Migration Strategies of Virtual Cluster   Concurrent Migration with Various Granularities                        ...
Dynamic Migration Strategies of Virtual Cluster   Mutual Migration                           GRID 2012                  S...
Outline   Motivations   VC-Migration Framework   Virtual Cluster Performance       Cross-Domain Virtual Cluster      ...
Multi-VC Migration Strategies   Live Migration of Homogeneous Virtual    Clusters                         GRID 2012      ...
Multi-VC Migration Strategies   Live Migration of Heterogeneous Virtua    l Clusters                          GRID 2012  ...
Multi-VC Migration Strategies   Master Node Migration vs. Slave Node    Migration                         GRID 2012      ...
Experimental Findings   The main contradiction of VC migration is the large amount of image    data and the limited netwo...
Outline   Motivations   VC-Migration Framework   Virtual Cluster Performance       Cross-Domain Virtual Cluster      ...
Related Work    Virtual Cluster Performance Analysis        MPI Cluster [Huang et al., ICS’06; Ye et al., HPCC’10; Merge...
Outline   Motivations   VC-Migration Framework   Virtual Cluster Performance       Cross-Domain Virtual Cluster      ...
Conclusion We have studied the live migration perfor  mance and overheads of virtual clusters fr  om the experimental per...
Future Work Optimize the migration mechanism in the  hypervisor to improve the migration efficie  ncy of virtual clusters...
Q&AThank you!          GRID 2012 Sep. 20-23, 2012 Beijing, China
Upcoming SlideShare
Loading in...5
×

Grid2012 VC-Migration: Live Migration of Virtual Clusters in the Cloud

627

Published on

2012 ACM/IEEE International Conference on Grid Computing

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
627
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Grid2012 VC-Migration: Live Migration of Virtual Clusters in the Cloud"

  1. 1. VC-Migration: Live Migration of Virtual Clusters in the Cloud Kejiang Ye, Xiaohong Jiang, Ran Ma, Fengxi Yan CCNT Lab, College of Computer Science Zhejiang University, China GRID 2012 Sep. 20-23, 2012 Beijing, China
  2. 2. Outline Motivations VC-Migration Framework Virtual Cluster Performance  Cross-Domain Virtual Cluster  Virtual Cluster Scalability Dynamic Migration Strategies of Virtual Cluster  Effects of Memory Configuration  Migration Scalability  Concurrent Migration with Various Granularities  Mutual Migration Multi-VC Migration Strategies  Live Migration of Homogeneous Virtual Clusters  Live Migration of Heterogeneous Virtual Clusters  Master Node Migration vs. Slave Node Migration Related Work Conclusion & Future Work GRID 2012 Sep. 20-23, 2012 Beijing, China
  3. 3. Outline Motivations VC-Migration Framework Virtual Cluster Performance  Cross-Domain Virtual Cluster  Virtual Cluster Scalability Dynamic Migration Strategies of Virtual Cluster  Effects of Memory Configuration  Migration Scalability  Concurrent Migration with Various Granularities  Mutual Migration Multi-VC Migration Strategies  Live Migration of Homogeneous Virtual Clusters  Live Migration of Heterogeneous Virtual Clusters  Master Node Migration vs. Slave Node Migration Related Work Conclusion & Future Work GRID 2012 Sep. 20-23, 2012 Beijing, China
  4. 4. Motivations Currently, live migration of virtual machine s has become a key ingredient behind the management activities of cloud computing system to achieve the goals of:  Load balancing  Energy saving  Failure recovery  System maintenance  … GRID 2012 Sep. 20-23, 2012 Beijing, China
  5. 5. Motivations Virtual Cluster is a group of virtual machines conf igured for a common purpose, such as high perfor mance computing or parallel computing, with asso ciated  Storage resource  Operating system  Software environment  Communication protocol  Network configuration Two notable features of virtual cluster  Large scale  Intensive communication GRID 2012 Sep. 20-23, 2012 Beijing, China
  6. 6. Motivations Live migration of virtual clusters (VC) faces s everal new challenges:  Huge Amount of Data  Limitation of Network Bandwidth  Intensive Communication between VMs  Synchronous Latency  Complex VC Migration Strategies It is necessary to study this new migration sc enario to investigate the overheads & bottlen ecks. GRID 2012 Sep. 20-23, 2012 Beijing, China
  7. 7. Contributions Describe a framework VC-Migration to co ntrol the migration of virtual clusters. Perform a series of experiments to unders tand the performance bottleneck and over heads of virtual cluster migration:  Performance characterization of virtual cluster;  Dynamic migration strategies of virtual cluster s;  Migration of multiple virtual clusters; GRID 2012 Sep. 20-23, 2012 Beijing, China
  8. 8. Outline Motivations VC-Migration Framework Virtual Cluster Performance  Cross-Domain Virtual Cluster  Virtual Cluster Scalability Dynamic Migration Strategies of Virtual Cluster  Effects of Memory Configuration  Migration Scalability  Concurrent Migration with Various Granularities  Mutual Migration Multi-VC Migration Strategies  Live Migration of Homogeneous Virtual Clusters  Live Migration of Heterogeneous Virtual Clusters  Master Node Migration vs. Slave Node Migration Related Work Conclusion & Future Work GRID 2012 Sep. 20-23, 2012 Beijing, China
  9. 9. VC-Migration Framework Framework Design GRID 2012 Sep. 20-23, 2012 Beijing, China
  10. 10. VC-Migration Framework VC-Migration Scenario GRID 2012 Sep. 20-23, 2012 Beijing, China
  11. 11. VC-Migration Framework VC-Migration Strategies a. Concurrent Migration with Various Granularity b. Mutual Migration (a) (b) c. Homogeneous Multi-VC Migration d. Heterogeneous Multi-VC Migration (c) GRID 2012 (d) Sep. 20-23, 2012 Beijing, China
  12. 12. Outline Motivations VC-Migration Framework Virtual Cluster Performance  Cross-Domain Virtual Cluster  Virtual Cluster Scalability Dynamic Migration Strategies of Virtual Cluster  Effects of Memory Configuration  Migration Scalability  Concurrent Migration with Various Granularities  Mutual Migration Multi-VC Migration Strategies  Live Migration of Homogeneous Virtual Clusters  Live Migration of Heterogeneous Virtual Clusters  Master Node Migration vs. Slave Node Migration Related Work Conclusion & Future Work GRID 2012 Sep. 20-23, 2012 Beijing, China
  13. 13. Virtual Cluster Performance Experimental Configuration  Virtual Cluster Configuration  Dell T710 Server, with 2 Quad-core 64bit Xeon processors a nd 32 GB DRAM.  CentOS 5.6 with kernel version 2.6.18-238.12.1.e15xen in D omain 0, and Xen 3.3.1 as the hypervisor.  VM (Guest OS) with Ubuntu 8.10, 1 VCPU & 256 MB DRAM.  MPI version is MPICH 2.1.0.8  All the VM images are stored on a separate NFS storage ser ver  Benchmarks  HPC Challenge Benchmark Suite (HPCC) as the virtual clust er workloads: HPL, DGEMM, STREAM, PTRANS, RandomA ccess, FFT, Communication bandwidth and latency  Virt-LM Benchmark: to measure the migration performance a nd overheads. [Huang et al., ICPE2011] GRID 2012 Sep. 20-23, 2012 Beijing, China
  14. 14. Virtual Cluster Performance Cross-Domain Virtual Cluster GRID 2012 Sep. 20-23, 2012 Beijing, China
  15. 15. Virtual Cluster Performance Virtual Cluster Scalability GRID 2012 Sep. 20-23, 2012 Beijing, China
  16. 16. Outline Motivations VC-Migration Framework Virtual Cluster Performance  Cross-Domain Virtual Cluster  Virtual Cluster Scalability Dynamic Migration Strategies of Virtual Cluster  Effects of Memory Configuration  Migration Scalability  Concurrent Migration with Various Granularities  Mutual Migration Multi-VC Migration Strategies  Live Migration of Homogeneous Virtual Clusters  Live Migration of Heterogeneous Virtual Clusters  Master Node Migration vs. Slave Node Migration Related Work Conclusion & Future Work GRID 2012 Sep. 20-23, 2012 Beijing, China
  17. 17. Dynamic Migration Strategies of Virtual Cluster Effects of Memory Configuration GRID 2012 Sep. 20-23, 2012 Beijing, China
  18. 18. Dynamic Migration Strategies of Virtual Cluster Migration Scalability Overall Time Average Time GRID 2012 Sep. 20-23, 2012 Beijing, China
  19. 19. Dynamic Migration Strategies of Virtual Cluster Concurrent Migration with Various Granularities GRID 2012 Sep. 20-23, 2012 Beijing, China
  20. 20. Dynamic Migration Strategies of Virtual Cluster Mutual Migration GRID 2012 Sep. 20-23, 2012 Beijing, China
  21. 21. Outline Motivations VC-Migration Framework Virtual Cluster Performance  Cross-Domain Virtual Cluster  Virtual Cluster Scalability Dynamic Migration Strategies of Virtual Cluster  Effects of Memory Configuration  Migration Scalability  Concurrent Migration with Various Granularities  Mutual Migration Multi-VC Migration Strategies  Live Migration of Homogeneous Virtual Clusters  Live Migration of Heterogeneous Virtual Clusters  Master Node Migration vs. Slave Node Migration Related Work Conclusion & Future Work GRID 2012 Sep. 20-23, 2012 Beijing, China
  22. 22. Multi-VC Migration Strategies Live Migration of Homogeneous Virtual Clusters GRID 2012 Sep. 20-23, 2012 Beijing, China
  23. 23. Multi-VC Migration Strategies Live Migration of Heterogeneous Virtua l Clusters GRID 2012 Sep. 20-23, 2012 Beijing, China
  24. 24. Multi-VC Migration Strategies Master Node Migration vs. Slave Node Migration GRID 2012 Sep. 20-23, 2012 Beijing, China
  25. 25. Experimental Findings The main contradiction of VC migration is the large amount of image data and the limited network bandwidth. The virtual machines belonging to the same virtual cluster should be deployed together as far as possible to reduce the communication a nd synchronization latency across different physical machines. Virtual cluster has good scalability and is suitable for the high perfor mance computing and parallel computing tasks. When a virtual cluster needs to be migrated, it is important to select a suitable concurrent migration granularity. Large concurrent granul arity will decrease the VC performance dramatically. Mutual migration should be avoided due to the long overall migratio n time. The migration of slave node incurs relatively less overhead compar ed to the master node. So it should give a priority to the slave migrat ion. Migration order is important when multiple virtual clusters need to b e migrated. The long-time cross-domain virtual cluster will decrease the overall performance of applications. GRID 2012 There is a big optimization20-23, 2012 in the live migration of virtual clust Sep. space Beijing, China
  26. 26. Outline Motivations VC-Migration Framework Virtual Cluster Performance  Cross-Domain Virtual Cluster  Virtual Cluster Scalability Dynamic Migration Strategies of Virtual Cluster  Effects of Memory Configuration  Migration Scalability  Concurrent Migration with Various Granularities  Mutual Migration Multi-VC Migration Strategies  Live Migration of Homogeneous Virtual Clusters  Live Migration of Heterogeneous Virtual Clusters  Master Node Migration vs. Slave Node Migration Related Work Conclusion & Future Work GRID 2012 Sep. 20-23, 2012 Beijing, China
  27. 27. Related Work  Virtual Cluster Performance Analysis  MPI Cluster [Huang et al., ICS’06; Ye et al., HPCC’10; Merge n et al., SIGOPS Oper. Syst. Rev., 2006]  MapReduce Cluster [Ibrahim et al., CloudCom’09]  Live Migration of Single VM  Pre-copy technique [Clark et al., NSDI’05; Nelson et al., US ENIX’05]  Post-copy technique [Hines et al., VEE’09]  Whole system migration [Luo et al., Cluster’08]  Live Migration of Multiple VM  Resource reservation method [Ye et al., Cloud’10]  Avoid the data de-duplication for concurrent migraiton [Deshpande et al., HPDC’11; Al-Kiswany et al. HPDC’11]However, all the above work didn’t solve the problem of live migration of virtual GRID 2012cluster in which the intensive Sep. 20-23, 2012 Beijing, China communication overheads can affect the migrationperformance.
  28. 28. Outline Motivations VC-Migration Framework Virtual Cluster Performance  Cross-Domain Virtual Cluster  Virtual Cluster Scalability Dynamic Migration Strategies of Virtual Cluster  Effects of Memory Configuration  Migration Scalability  Concurrent Migration with Various Granularities  Mutual Migration Multi-VC Migration Strategies  Live Migration of Homogeneous Virtual Clusters  Live Migration of Heterogeneous Virtual Clusters  Master Node Migration vs. Slave Node Migration Related Work Conclusion & Future Work GRID 2012 Sep. 20-23, 2012 Beijing, China
  29. 29. Conclusion We have studied the live migration perfor mance and overheads of virtual clusters fr om the experimental perspective and inve stigated different VC migration strategies. Experimental results reveal some new dis coveries, based on which we can propose several optimization principles to improve t he migration performance of virtual cluster s. GRID 2012 Sep. 20-23, 2012 Beijing, China
  30. 30. Future Work Optimize the migration mechanism in the hypervisor to improve the migration efficie ncy of virtual clusters Design efficient migration algorithms for th e virtual clusters. GRID 2012 Sep. 20-23, 2012 Beijing, China
  31. 31. Q&AThank you! GRID 2012 Sep. 20-23, 2012 Beijing, China

×