SlideShare a Scribd company logo
Live Migration with
 AMD-V™ Extended
Migration Technology




 Publication # 43781 Revision: 3.00
 Issue Date:   April 2008




                  Advanced Micro Devices
43781 Rev. 3.00 April 2008              Live Migration with AMD-V™ Extended Migration Technology




                               © 2007, 2008 Advanced Micro Devices, Inc. All   rights reserved.

                   The contents of this document are provided in connection with Advanced
                   Micro Devices, Inc. (“AMD”) products. AMD makes no representations or
                   warranties with respect to the accuracy or completeness of the contents of this
                   publication and reserves the right to make changes to specifications and
                   product descriptions at any time without notice. The information contained
                   herein may be of a preliminary or advance nature and is subject to change
                   without notice. No license, whether express, implied, arising by estoppel, or
                   otherwise, to any intellectual property rights are granted by this publication.
                   Except as set forth in AMD’s Standard Terms and Conditions of Sale, AMD
                   assumes no liability whatsoever, and disclaims any express or implied
                   warranty, relating to its products including, but not limited to, the implied
                   warranty of merchantability, fitness for a particular purpose, or infringement of
                   any intellectual property right.
                   AMD’s products are not designed, intended, authorized or warranted for use as
                   components in systems intended for surgical implant into the body, or in other
                   applications intended to support or sustain life, or in any other application in
                   which the failure of AMD’s product could create a situation where personal
                   injury, death, or severe property or environmental damage may occur. AMD
                   reserves the right to discontinue or make changes to its products at any time
                   without notice.




  Trademarks
  AMD, the AMD Arrow logo, AMD Opteron, and combinations thereof, AMD-V are trademarks of
  Advanced Micro Devices, Inc.
  Microsoft and Windows are registered trademarks of Microsoft Corporation.
  Other product names used in this publication are for identification purposes only and may be trademarks of their
  respective companies.
43781 Rev. 3.00 April 2008                        Live Migration with AMD-V™ Extended Migration Technology


Contents
Revision History ...............................................................................................................................5
   1.1         Introduction........................................................................................................................6
   1.2         System Virtualization ........................................................................................................6
   1.3         Virtual Machine Migration ................................................................................................7
   1.4         Determining Processor Features ........................................................................................8
   1.5         VMM Basics ....................................................................................................................10
   1.6         Problems with Live Migration.........................................................................................10
       1.6.1          Backward-Compatibility Problem ...........................................................................11
       1.6.2          Forward-Compatibility Problem..............................................................................11
   1.7         AMD-V™ Extended Migration Technology...................................................................11
       1.7.1          Backward-Compatibility Assurance ........................................................................11
          1.7.1.1         Controlling Information Returned through CPUID.............................................11

          1.7.1.2         CPUID Override Model Specific Registers.........................................................12

       1.7.2          Forward-Compatibility Assurance...........................................................................16
       1.7.3          Feature-Disable Bits.................................................................................................17
   1.8         Conclusions......................................................................................................................17
   1.9         References........................................................................................................................17




Chapter 1                                                       Contents                                                                          3
43781 Rev. 3.00 April 2008             Live Migration with AMD-V™ Extended Migration Technology


List of Figures
Figure 1. Live Virtual Machine Migration on AMD Processors .......................................................8
Figure 2. Live Migration Support in Current AMD Processors ......................................................16




Chapter 1                                     List of Figures                                                    4
43781 Rev. 3.00 April 2008       Live Migration with AMD-V™ Extended Migration Technology


Revision History
      Date         Revision                               Description
 4/2008              3.00     Initial public release.




Chapter 1                              Revision History                                5
43781 Rev. 3.00 April 2008        Live Migration with AMD-V™ Extended Migration Technology


Chapter 1                     Live Migration with AMD-V™
                              Extended Machine Migration
This document describes virtual machine migration.

1.1         Introduction
Virtual Machine (VM) migration is a capability increasingly used in today’s enterprise
environments. With live migration, a Virtual Machine Monitor (VMM) moves a running VM
nearly instantaneously from one server to another and seamlessly to the end user while
maintaining guest uptime guarantees. However, if the processors running in the computers in the
pre- and post-migration environment are not identical, live migration can result in unexpected
behavior of the guest software. Since the introduction of AMD64 processor technology in 2003,
AMD has worked closely with virtualization software developers to define the functionality
necessary to ensure that live migration is possible across a broad range of AMD64 processors.
This document discusses the basics of live migration and describes the functionality currently
available from AMD to enable virtualization software vendors to support live migration across a
broad range of AMD64 processors. Figure 1 on page 8 illustrates live virtual machine migration
on AMD processors.

1.2         System Virtualization
System virtualization is the ability to abstract and pool resources on a physical platform. This
abstraction decouples software from hardware and enables multiple operating system images to
run concurrently on a single physical platform without interfering with each other.

System virtualization has existed for decades on mainframes. In the past, industry-standard x86-
based machines with their limited computing resources and challenges in correctly virtualizing the
Instruction Set Architecture (ISA)[1,2] did not lend themselves well to the technique. However,
with the increase in computing resources of x86-based servers in recent years and with new
processor extensions such as those found in AMD-V™[3], x86-based virtualization is seeing
widespread adoption and is rapidly becoming a pervasive and integral capability in enterprise
environments[4].

Virtualization can increase utilization of computing resources by consolidating the workloads
running on many physical systems into virtual machines running on a single physical system.
Virtual machines can be provisioned on-demand, replicated, and migrated.

VM migration, which is the ability to move a virtual machine from one physical server to another
under VMM control, is a capability increasingly utilized in today’s enterprise environments.

This document discusses how AMD-V™ Extended Migration Technology enables virtual machine
migration between AMD64 processors.

                 Live Migration with AMD-V™ Extended Machine Migration                             6
43781 Rev. 3.00 April 2008        Live Migration with AMD-V™ Extended Migration Technology


1.3         Virtual Machine Migration
A virtual machine can be migrated in the following ways:

   •   Static migration. The VM is shut down using OS-supported methods; its static VM
        image is moved to another VMM and restarted.
   •   Cold migration. The VM is suspended using OS-supported or VMM-supported methods.
        The suspended VM image is moved to a VMM on a different machine and resumed.
   •   Live migration. The VMM moves a running VM instance nearly instantaneously from
        one server to another. Live Migration allows for dynamic load balancing of virtualized
        resource pools, hardware maintenance without downtime, and dynamic failover support.


In the following sections, software running in the VM is called “guest software.”

As long as the hardware in the pre- and post-migration environment is identical, guest software
should behave in exactly the same way before and after the migration. Challenges can arise,
however, when guest software runs in a different hardware environment after a migration.

Note that even though a VMM presents a virtual platform to guest software, there can be certain
interfaces, depending on VMM design, which guest software can directly use to determine
underlying hardware’s capabilities, discussed further in Section 1.6 on page 10.

After the reboot/restart following a static migration, guest software should go through its platform
discovery phase and be able to adjust to any differences in underlying (virtual) hardware.

Following a cold migration, guest software may continue to maintain an identical view of the pre-
and post-migration hardware. When suspended using OS-supported methods, some operating
systems re-scan the hardware upon resume. Depending on their policies and the hardware
differences between current and previous hardware, the OSes may refuse to resume and require a
reboot.

After a live migration, guest software continues to maintain an identical view of the pre- and post-
migration hardware. This document discusses how AMD64 processors provide support for a
VMM to hide differences in software-visible processor features during live and cold migration.
For simplicity, “live migration” is used to refer to both migration methods.




                 Live Migration with AMD-V™ Extended Machine Migration                             7
43781 Rev. 3.00 April 2008       Live Migration with AMD-V™ Extended Migration Technology




Figure 1. Live Virtual Machine Migration on AMD Processors



1.4        Determining Processor Features
Software should use CPUID to determine processor features and use this information to select
appropriate code paths. CPUID is an x86 instruction that can be executed from kernel and
applications. The following example shows CPUID called from an application.




                Live Migration with AMD-V™ Extended Machine Migration                          8
43781 Rev. 3.00 April 2008       Live Migration with AMD-V™ Extended Migration Technology

//Example 1: Console application on Windows XP using
CPUID instruction to determine //processor revision and
feature information.

#include "stdafx.h"
#include <stdio.h>

int _tmain (int argc, _TCHAR* argv[])
{

       int version;
       int features_1;
       int features_2;

       __asm
       {
            mov eax, 1

              cpuid

              mov version, eax
              mov features_1, ecx
              mov features_2, edx
       }

       printf ("Processor version: 0x%x features_1: 0x%08x
       features_2: 0x%08xn", version,
       features_1, features_2);

       return 0;
}

However, a small number of existing commercial software applications determines the presence of
certain processor features using non-standard methods (such as using a try-catch code block to
determine whether certain instructions are supported) or assumes the presence of certain features
without testing for them. This software can cause indeterminate behavior under live migration
scenarios. Note that such software is equally likely to fail in non-virtualized environments with
different processors.




                Live Migration with AMD-V™ Extended Machine Migration                          9
43781 Rev. 3.00 April 2008        Live Migration with AMD-V™ Extended Migration Technology


1.5         VMM Basics
To understand the implications of VMM design on VM migration, it is useful to briefly review
various VMM design alternatives.

It is important to note that for performance reasons, a VMM may only interfere in the execution of
guest software when the guest code performs privileged operations [1,2].

The first method is called para-virtualization, in which the guest’s kernel is modified to
cooperate with the VMM when performing privileged operations. The second method is called
binary translation, in which the VMM modifies the guest’s binary image at runtime to get
processor control when guest software attempts to perform a privileged operation [2]. The VMM
can then emulate the privileged operation and return control to guest software. The third method is
hardware-assisted virtualization, in which the VMM uses processor extensions such as
AMD-V™ to intercept and emulate privileged operations in the guest.

1.6         Problems with Live Migration
In the following sections, the processor in pre-migration environment is called the “current”
processor. The processor in post-migration environment is called the “target” processor.

When performing a live migration, a VM is suspended; its current (guest and VMM) state is
captured, moved to another VMM, and resumed on it. The VMM preserves the state and identity
of virtual devices after the migration. This process happens quickly and seamlessly to the end user
and maintains guest uptime guarantees.

As indicated previously, when hardware in the pre- and post-migration environment is different,
live migration can result in unexpected guest behavior. After a live migration, there is no
indication to guest software that it is now running on a different server with potentially different
processor features. This fundamentally different programming environment may not interact well
with existing software design. For example, a guest application may use CPUID instruction at
startup to determine that the processor supports SSE3 instructions, then live migrate to another
server and consequently fail when attempting to use those instructions on a target processor that
does not support SSE3.

To help ensure that guest software does not malfunction after a live migration, the virtualization
infrastructure should address the problems shown in the following subsections.




                 Live Migration with AMD-V™ Extended Machine Migration                               10
43781 Rev. 3.00 April 2008        Live Migration with AMD-V™ Extended Migration Technology

1.6.1       Backward-Compatibility Problem
The backward-compatibility problem happens when the target processor does not implement a
feature that is implemented on the current processor. In the case of this problem, after live
migration, use of such a feature may result in unexpected software behavior. For example, use of
an instruction not implemented on the target processor results in a #UD fault.

1.6.2       Forward-Compatibility Problem
The forward-compatibility problem happens when the target processor implements features not
supported on the current processor. This situation could be problematic in cases where the target
processor uses an opcode (to implement an instruction) that is not defined on the current
processor, and software on the current processor expects use of this undefined opcode to generate
a #UD fault. In such cases, after a live migration, software observes a different processor behavior,
that is, it does not observe #UD fault upon use of the (previously undefined) opcode.

Fortunately, a very small number of existing commercial software depends on such processor
behavior. AMD actively works with its software partners to discourage use of such behaviors.

1.7         AMD-V™ Extended Migration Technology
Since the introduction of AMD64 processor technology in 2003, AMD has worked closely with
virtualization software developers to define the functionality necessary so that live migration is
possible across a broad range of products. AMD-V™ Extended Migration Technology provides
various capabilities to address problems associated with Live Migration.

1.7.1       Backward-Compatibility Assurance
To assure backward compatibility, the VMM can intercept the #UD fault and emulate the faulting
instruction. However, this method of emulating the faulting instruction is not generally acceptable
because it adds to the complexity of the VMM and may also degrade the performance of the guest.

A simpler and more widely adopted solution by many VMM vendors is to expose to the guest
software only those features that are supported on all processors in the resource pool. This solution
essentially means establishing the lowest common denominator of processor features in the
resource pool.

1.7.1.1     Controlling Information Returned through CPUID

When new features are introduced on AMD processors, their presence is indicated by unique bits
in the information returned by the CPUID instruction. Properly written software relies only on this
information to determine processor features.

To ensure proper behavior during VM migration, a VMM must control information returned to
guest software as a result of the CPUID instruction.


                 Live Migration with AMD-V™ Extended Machine Migration                               11
43781 Rev. 3.00 April 2008       Live Migration with AMD-V™ Extended Migration Technology

A VMM using hardware-assisted virtualization (AMD-V™) (such as Xen HVM or Microsoft®
Virtual Server) can use the CPUID intercept to return the appropriate bits whenever guest software
executes that instruction.

A VMM using binary translation (such as VMware ESX) or para-virtualization (such as Xen) may
need to handle this differently depending on whether CPUID was executed by guest’s kernel or
guest’s application. In the case of CPUID instruction executed by the guest’s kernel, the VMM
can binary-translate or para-virtualize that instance of the instruction and return appropriate
information to the guest.

For a CPUID instruction executed within the guest application’s context (where binary translation
or para-virtualization techniques are not feasible), the VMM requires a way to return the
appropriate CPUID information to that guest application. Note that with binary translation and
para-virtualization, it is the user-mode CPUID features that the VMM must control. Without this
ability to control the user-mode CPUID features, all processors in the resource pool must support
the same features, which significantly reduces flexibility in utilizing resources in environments
supporting Live Migration.

The AMD CPUID Override Model Specific Registers (MSRs), as described in Section 1.7.1.2,
provide the VMM the ability to control CPUID information returned to guest applications.

1.7.1.2    CPUID Override Model Specific Registers

Starting with AMD Family 0Fh revision C AMD Opteron™ processors, AMD provides CPUID
Override MSRs to allow a VMM (or OS) to specify a subset of information that the CPUID
instruction returns. This is accomplished by setting the appropriate MSRs as described below.




                 Live Migration with AMD-V™ Extended Machine Migration                          12
43781 Rev. 3.00 April 2008        Live Migration with AMD-V™ Extended Migration Technology


MSR Type               CPUID Features Register Override
MSR Description        Provides control over features reported in CPUID function 0000_0001 in ECX
                       and EDX.
MSR Index              C001_1004h
Bits                   Description
63:32                  Features. R/W, function 1, ECX
31:0                   Features. R/W, function 1, EDX


MSR Type               Extended CPUID Features Register Override
MSR Description        Provides control over features reported in CPUID function 8000_0001 in ECX
                       and EDX.
MSR Index              C001_1005h
Bits                   Description
63:32                  Features. R/W, function 1, ECX
31:0                   Features. R/W, function 1, EDX


MSR Type               Logical Processor Count (K8 only)
MSR Description        MSR is provided for backward-compatibility with K8 processors.
MSR Index              C001_100Dh
Bits                   Description
63:32                  Reserved
31:0                   Logical Processor Count; R/W


MSR Type               Processor Name String Registers
MSR Description        These registers hold the CPUID name string in ASCII. The state of these registers
                       is returned by CPUID instruction Functions 8000_0004, 8000_0003, 8000_0002.
                       Each register contains a block of 8 ASCII characters; the least significant byte
                       corresponds to the first ASCII character of the block; the most-significant byte
                       corresponds to the last character of the block. MSR C001_0030h provides the first
                       block of the name string; MSR C001_0035h contains the last block of the name
                       string.
MSR Index              C001_0030h – C001_0035h
Bits                   Description
63:0                   CpuNameString. R/W.




                  Live Migration with AMD-V™ Extended Machine Migration                              13
43781 Rev. 3.00 April 2008     Live Migration with AMD-V™ Extended Migration Technology

When the VMM or OS does not override CPUID return information using this mechanism, the
processor returns CPUID information based on the native feature set.

The following code sample shows how a VMM (or OS) can hide reporting of SSE4A instructions
on a quad-core AMD Opteron™ processor.

       /*
        * Example 2: Use MSR C001_1005 to clear bit 6
       (SSE4A) reported in ECX after CPUID
        * Function 8000_0001
       */


              /*
              Read current value of the CPUID Override MSR
              C001_1005.
               After RDMSR completes, EDX:EAX contains the
              64bit MSR value. EDX is loaded with the high 32
              bits of the MSR and EAX is loaded with the low 32
              bits. The high 32 bits of this MSR are returned
              in ECX after CPUID Function 8000_0001
              */

                     MOV CX, 0xC0011005h
                     RDMSR

              /*
              Clear bit 6 (SSE4A) of EDX register
              */

                     ANDL EDX, 0xFFFFFFBFh

              /*
              Write the new EDX:EAX value into CPUID override
              MSR
              */

                     WRMSR




                Live Migration with AMD-V™ Extended Machine Migration                     14
43781 Rev. 3.00 April 2008      Live Migration with AMD-V™ Extended Migration Technology

The following code sample shows how a VMM (or OS) can hide reporting of RDTSCP instruction
on AMD Opteron™ processor, rev. F.



       /*
        * Example 3: Use MSR C001_1005 to clear bit 27
       (RDTSCP) reported in EDX after
        * CPUID Function 8000_0001
       */

              /*
              Read current value of the CPUID Override MSR
              C001_1005.
               After RDMSR completes, EDX:EAX contains the
              64bit MSR value. EDX is loaded with the high 32
              bits of the MSR and EAX is loaded with the low 32
              bits. The low 32 bits of this MSR are returned in
              EDX after CPUID Function 8000_0001
              */
              /*
              Write the new EDX:EAX value into CPUID override
              MSR. AMD Opteron™ Rev-F Processors require a 32
              bit password in EDI. Contact AMD to get the
              password.
              */

              MOV EDI, <PASSWORD>

                     MOV CX, 0xC0011005h
                     RDMSR

              /*
              Clear bit 27 (RDTSCP) of EAX              register
              */

                     ANDL EAX, 0xF7FFFFFFh

                     WRMSR


Note that overriding CPUID return information using these MSRs does not disable the respective
features of the processor. On AMD Opteron™ processors, a password must be passed through the
EDI register before executing RDMSR and WRMSR to set the CPUID overrides. This password
can be obtained from AMD.



                Live Migration with AMD-V™ Extended Machine Migration                      15
43781 Rev. 3.00 April 2008         Live Migration with AMD-V™ Extended Migration Technology

Figure 2. shows Live Migration support in current AMD processors:




 AMD Opteron ™                          AMD Opteron ™                           AMD Opteron ™
  Processor – Rev C                      Processor – Rev E                       Processor – Rev F
 Models 840, 840 EE, 842, 846           Models 860, 860 HE, 865, 870,           Models 8212, 8220, 8218,
  HE, 850 …                              875, 890, …                             8222 SE, …
 Models 240, 242, 244, 246,             Models 260 HE, 265, 275 HE,             Models 2220,2212, 2214,
  246 HE, 248, …                         280, 290, …                             2210, …
 Models 140, 140EE, 146HE,              Models 165, 185, …                      Models 1222 SE, 1214,
  148, 150, …                                                                    1210, …


 SSE
                                       SSE3                                                             SVM
 MMX
                                                                                                   RDTSCP
 SSE2                        AMD-V Extended Migration Technology
                                                                                                    SKINIT
                                                                                          CMPXCHG16B

           Nested Paging             AMD Opteron™                 SSE4A
                                       Quad-Core                  POPCNT
                             IBS
                                       (AMD Family 10h
                       OSVW               codenamed
                                                                  LZCNT
                                         “Barcelona”)
                1 GB Paging                                       MONITOR/MWAIT




Figure 2. Live Migration Support in Current AMD Processors

If guest software does not use CPUID to determine processor features and, instead, relies on
deprecated methods (such as using certain code sequences to detect implementation differences
and using the result to identify processor features), then these overrides may be of little use.
Fortunately, AMD field surveys indicate that such software is very limited in today’s enterprise
environments.

1.7.2       Forward-Compatibility Assurance
In Section 1.6.2 on page 11, we discussed forward-compatibility problems. To avoid these
problems, the VMM needs a method to disable new instructions until all the processors in the
resource pool implement them. Disabling new instructions is required to ensure that software
continues to observe the same processor behavior (such as expected #UD faults) in forward-
migration scenarios.


                 Live Migration with AMD-V™ Extended Machine Migration                             16
43781 Rev. 3.00 April 2008        Live Migration with AMD-V™ Extended Migration Technology

1.7.3       Feature-Disable Bits
On future processors, AMD may support feature-disable bits to disable certain instructions or
instruction sets. Such feature-disable bits could be exposed through an MSR-based interface.

AMD believes that the long-term solution for forward-compatibility assurance is encouraging
developers to write well-behaved, “live-migration friendly” code and adding appropriate support
in VMMs to handle forward-compatibility problems.

1.8         Conclusions
In this document, we have described in detail the potential problems that can cause guest software
to malfunction after a Live Migration. We also discussed how AMD64 processors are designed to
provide the functionality needed to support the VMM in addressing these problems.

AMD-V Extended Migration Technology allows a VMM to safely migrate a VM between various
AMD64 processor revisions. AMD-V provides IT administrators an unprecedented flexibility in
deploying, maintaining, and upgrading servers in Live Migration environments based on AMD64
processors.

Readers are cautioned about instances of legacy software that rely on non-standard methods to
determine processor features. However, such instances are limited and should diminish over time.

AMD encourages IT administrators to work with their VMM software vendors in determining the
level of VMM’s support for AMD-V™ Extended Migration Technology. The respective vendors
can provide guidance when configuring Live Migration resource pools with AMD processors[5].

1.9         References
1. Popek, G. J.; Goldberg, R. P. “Formal Requirements for Virtualizable Third Generation
   Architectures,” Communications of the ACM; 17(7): 412 –421.
2. Adams, K.; Agesen, O. “Comparison of Software and Hardware Techniques for x86
   Virtualization.” ASPLOS.
3. AMD. AMD64 Architecture Programmer’s Manual, Vol. 2.
4. IDC. “Impact of Virtualization Software on Operating Environments.” IDC Technology Assessment.
5. VMware. “VMotion CPU Compatibility Requirements for AMD Processors.”




                 Live Migration with AMD-V™ Extended Machine Migration                              17

More Related Content

What's hot

Svc manual clx-6260_series_eng
Svc manual clx-6260_series_engSvc manual clx-6260_series_eng
Svc manual clx-6260_series_eng
Charles Camenzuli
 
Dir 636 l_a1_manual_v1_00_en_uk
Dir 636 l_a1_manual_v1_00_en_ukDir 636 l_a1_manual_v1_00_en_uk
Dir 636 l_a1_manual_v1_00_en_uk
István Budai
 
01 I P My View V1.0
01   I P My View V1.001   I P My View V1.0
01 I P My View V1.0
mckeeg
 
P21gv31
P21gv31P21gv31
P21gv31
lexavid
 
DVT-Manual
DVT-ManualDVT-Manual
DVT-Manual
Jason Weber
 
Set Up Security and Integration with DataPower XI50z
Set Up Security and Integration with DataPower XI50zSet Up Security and Integration with DataPower XI50z
Set Up Security and Integration with DataPower XI50z
Sarah Duffy
 
Wm4 0 quickstartguideissue1
Wm4 0 quickstartguideissue1Wm4 0 quickstartguideissue1
Wm4 0 quickstartguideissue1
Advantec Distribution
 
Returnil 2011
Returnil 2011Returnil 2011
Returnil 2011
Rose Banioki
 
White Paper: EMC Infrastructure for VMware Cloud Environments
White Paper: EMC Infrastructure for VMware Cloud Environments  White Paper: EMC Infrastructure for VMware Cloud Environments
White Paper: EMC Infrastructure for VMware Cloud Environments
EMC
 
Introduction to Virtual Desktop Architectures
Introduction to Virtual Desktop ArchitecturesIntroduction to Virtual Desktop Architectures
Introduction to Virtual Desktop Architectures
RingCube Technologies, Inc.
 
N68 Manual de la Boar
N68 Manual de la BoarN68 Manual de la Boar
N68 Manual de la Boar
erikadanielatorresdelgado
 
Asrock
AsrockAsrock
Asrock
IVAN MONTES
 
White Paper: EMC Infrastructure for Microsoft Private Cloud
White Paper: EMC Infrastructure for Microsoft Private Cloud White Paper: EMC Infrastructure for Microsoft Private Cloud
White Paper: EMC Infrastructure for Microsoft Private Cloud
EMC
 
Manual HP 7300E MT I52400 4G 1TB W7 PRO 64B
Manual HP 7300E MT I52400 4G 1TB W7 PRO 64BManual HP 7300E MT I52400 4G 1TB W7 PRO 64B
Manual HP 7300E MT I52400 4G 1TB W7 PRO 64B
Internity.es
 
White Paper: EMC Compute-as-a-Service — EMC Ionix IT Orchestrator, VCE Vblock...
White Paper: EMC Compute-as-a-Service — EMC Ionix IT Orchestrator, VCE Vblock...White Paper: EMC Compute-as-a-Service — EMC Ionix IT Orchestrator, VCE Vblock...
White Paper: EMC Compute-as-a-Service — EMC Ionix IT Orchestrator, VCE Vblock...
EMC
 
AltiGen Max In Sight Manual
AltiGen Max In Sight  ManualAltiGen Max In Sight  Manual
AltiGen Max In Sight Manual
CTI Communications
 
Guide server virtualization_deployment
Guide server virtualization_deploymentGuide server virtualization_deployment
Guide server virtualization_deployment
abanggeh
 
IBM Data Center Networking: Planning for Virtualization and Cloud Computing
IBM Data Center Networking: Planning for Virtualization and Cloud ComputingIBM Data Center Networking: Planning for Virtualization and Cloud Computing
IBM Data Center Networking: Planning for Virtualization and Cloud Computing
IBM India Smarter Computing
 
Deployment guide series tivoli provisioning manager for os deployment v5.1 sg...
Deployment guide series tivoli provisioning manager for os deployment v5.1 sg...Deployment guide series tivoli provisioning manager for os deployment v5.1 sg...
Deployment guide series tivoli provisioning manager for os deployment v5.1 sg...
Banking at Ho Chi Minh city
 
migrating from pic to m3
migrating from pic to m3migrating from pic to m3
migrating from pic to m3
Wassim Smati
 

What's hot (20)

Svc manual clx-6260_series_eng
Svc manual clx-6260_series_engSvc manual clx-6260_series_eng
Svc manual clx-6260_series_eng
 
Dir 636 l_a1_manual_v1_00_en_uk
Dir 636 l_a1_manual_v1_00_en_ukDir 636 l_a1_manual_v1_00_en_uk
Dir 636 l_a1_manual_v1_00_en_uk
 
01 I P My View V1.0
01   I P My View V1.001   I P My View V1.0
01 I P My View V1.0
 
P21gv31
P21gv31P21gv31
P21gv31
 
DVT-Manual
DVT-ManualDVT-Manual
DVT-Manual
 
Set Up Security and Integration with DataPower XI50z
Set Up Security and Integration with DataPower XI50zSet Up Security and Integration with DataPower XI50z
Set Up Security and Integration with DataPower XI50z
 
Wm4 0 quickstartguideissue1
Wm4 0 quickstartguideissue1Wm4 0 quickstartguideissue1
Wm4 0 quickstartguideissue1
 
Returnil 2011
Returnil 2011Returnil 2011
Returnil 2011
 
White Paper: EMC Infrastructure for VMware Cloud Environments
White Paper: EMC Infrastructure for VMware Cloud Environments  White Paper: EMC Infrastructure for VMware Cloud Environments
White Paper: EMC Infrastructure for VMware Cloud Environments
 
Introduction to Virtual Desktop Architectures
Introduction to Virtual Desktop ArchitecturesIntroduction to Virtual Desktop Architectures
Introduction to Virtual Desktop Architectures
 
N68 Manual de la Boar
N68 Manual de la BoarN68 Manual de la Boar
N68 Manual de la Boar
 
Asrock
AsrockAsrock
Asrock
 
White Paper: EMC Infrastructure for Microsoft Private Cloud
White Paper: EMC Infrastructure for Microsoft Private Cloud White Paper: EMC Infrastructure for Microsoft Private Cloud
White Paper: EMC Infrastructure for Microsoft Private Cloud
 
Manual HP 7300E MT I52400 4G 1TB W7 PRO 64B
Manual HP 7300E MT I52400 4G 1TB W7 PRO 64BManual HP 7300E MT I52400 4G 1TB W7 PRO 64B
Manual HP 7300E MT I52400 4G 1TB W7 PRO 64B
 
White Paper: EMC Compute-as-a-Service — EMC Ionix IT Orchestrator, VCE Vblock...
White Paper: EMC Compute-as-a-Service — EMC Ionix IT Orchestrator, VCE Vblock...White Paper: EMC Compute-as-a-Service — EMC Ionix IT Orchestrator, VCE Vblock...
White Paper: EMC Compute-as-a-Service — EMC Ionix IT Orchestrator, VCE Vblock...
 
AltiGen Max In Sight Manual
AltiGen Max In Sight  ManualAltiGen Max In Sight  Manual
AltiGen Max In Sight Manual
 
Guide server virtualization_deployment
Guide server virtualization_deploymentGuide server virtualization_deployment
Guide server virtualization_deployment
 
IBM Data Center Networking: Planning for Virtualization and Cloud Computing
IBM Data Center Networking: Planning for Virtualization and Cloud ComputingIBM Data Center Networking: Planning for Virtualization and Cloud Computing
IBM Data Center Networking: Planning for Virtualization and Cloud Computing
 
Deployment guide series tivoli provisioning manager for os deployment v5.1 sg...
Deployment guide series tivoli provisioning manager for os deployment v5.1 sg...Deployment guide series tivoli provisioning manager for os deployment v5.1 sg...
Deployment guide series tivoli provisioning manager for os deployment v5.1 sg...
 
migrating from pic to m3
migrating from pic to m3migrating from pic to m3
migrating from pic to m3
 

Viewers also liked

2016 legal seminar for credit professionals
2016 legal seminar for credit professionals2016 legal seminar for credit professionals
2016 legal seminar for credit professionals
Kegler Brown Hill + Ritter
 
Healthcare Costs And Performance in the OECD
Healthcare Costs And Performance in the OECDHealthcare Costs And Performance in the OECD
Healthcare Costs And Performance in the OECD
Alex Rascanu
 
Introducing BlackBerry 10 [Indonesian Version]
Introducing BlackBerry 10 [Indonesian Version]Introducing BlackBerry 10 [Indonesian Version]
Introducing BlackBerry 10 [Indonesian Version]
Khomeini Mujahid
 
2012 oslo-talk
2012 oslo-talk2012 oslo-talk
2012 oslo-talk
c.titus.brown
 
La comunicazione-del-vino-ai-tempi-di-facebook
La comunicazione-del-vino-ai-tempi-di-facebookLa comunicazione-del-vino-ai-tempi-di-facebook
La comunicazione-del-vino-ai-tempi-di-facebook
Slawka G. Scarso
 
Private Work: How to Secure a Fair Contract + Get Paid
Private Work: How to Secure a Fair Contract + Get PaidPrivate Work: How to Secure a Fair Contract + Get Paid
Private Work: How to Secure a Fair Contract + Get Paid
Kegler Brown Hill + Ritter
 
16th Annual Professional Responsibility Seminar
16th Annual Professional Responsibility Seminar16th Annual Professional Responsibility Seminar
16th Annual Professional Responsibility Seminar
Kegler Brown Hill + Ritter
 
Romairone, Gregorio
Romairone, GregorioRomairone, Gregorio
Romairone, Gregorio
Gregorio
 
Exporting from the United States: Key Legal Considerations
Exporting from the United States: Key Legal ConsiderationsExporting from the United States: Key Legal Considerations
Exporting from the United States: Key Legal Considerations
Kegler Brown Hill + Ritter
 
Growing Through China: A Comprehensive Look at Market Opportunities
Growing Through China: A Comprehensive Look at Market Opportunities Growing Through China: A Comprehensive Look at Market Opportunities
Growing Through China: A Comprehensive Look at Market Opportunities
Kegler Brown Hill + Ritter
 
2014 mmg-talk
2014 mmg-talk2014 mmg-talk
2014 mmg-talk
c.titus.brown
 
What Is Eric
What Is EricWhat Is Eric
What Is Eric
lillynewby
 
Cloudxp keynote 19 sept pvu
Cloudxp keynote 19 sept pvuCloudxp keynote 19 sept pvu
Cloudxp keynote 19 sept pvu
Piet van Vugt
 
Thara
TharaThara
Thara
mk2886
 
Trabajos Verano 2º Eso 2009
Trabajos Verano 2º Eso 2009Trabajos Verano 2º Eso 2009
Trabajos Verano 2º Eso 2009
guest5bbe75
 
11i Logs
11i Logs11i Logs
Contiguity Principle
Contiguity PrincipleContiguity Principle
Contiguity Principle
jnpletcher
 
Dock It Customer Intro 14 Aug 09
Dock It Customer Intro 14 Aug 09Dock It Customer Intro 14 Aug 09
Dock It Customer Intro 14 Aug 09
DockIT
 

Viewers also liked (20)

2016 legal seminar for credit professionals
2016 legal seminar for credit professionals2016 legal seminar for credit professionals
2016 legal seminar for credit professionals
 
Healthcare Costs And Performance in the OECD
Healthcare Costs And Performance in the OECDHealthcare Costs And Performance in the OECD
Healthcare Costs And Performance in the OECD
 
Introducing BlackBerry 10 [Indonesian Version]
Introducing BlackBerry 10 [Indonesian Version]Introducing BlackBerry 10 [Indonesian Version]
Introducing BlackBerry 10 [Indonesian Version]
 
2012 oslo-talk
2012 oslo-talk2012 oslo-talk
2012 oslo-talk
 
La comunicazione-del-vino-ai-tempi-di-facebook
La comunicazione-del-vino-ai-tempi-di-facebookLa comunicazione-del-vino-ai-tempi-di-facebook
La comunicazione-del-vino-ai-tempi-di-facebook
 
CG borodino
CG borodinoCG borodino
CG borodino
 
Private Work: How to Secure a Fair Contract + Get Paid
Private Work: How to Secure a Fair Contract + Get PaidPrivate Work: How to Secure a Fair Contract + Get Paid
Private Work: How to Secure a Fair Contract + Get Paid
 
16th Annual Professional Responsibility Seminar
16th Annual Professional Responsibility Seminar16th Annual Professional Responsibility Seminar
16th Annual Professional Responsibility Seminar
 
Romairone, Gregorio
Romairone, GregorioRomairone, Gregorio
Romairone, Gregorio
 
Exporting from the United States: Key Legal Considerations
Exporting from the United States: Key Legal ConsiderationsExporting from the United States: Key Legal Considerations
Exporting from the United States: Key Legal Considerations
 
Growing Through China: A Comprehensive Look at Market Opportunities
Growing Through China: A Comprehensive Look at Market Opportunities Growing Through China: A Comprehensive Look at Market Opportunities
Growing Through China: A Comprehensive Look at Market Opportunities
 
2014 mmg-talk
2014 mmg-talk2014 mmg-talk
2014 mmg-talk
 
What Is Eric
What Is EricWhat Is Eric
What Is Eric
 
RealTimeSchool
RealTimeSchoolRealTimeSchool
RealTimeSchool
 
Cloudxp keynote 19 sept pvu
Cloudxp keynote 19 sept pvuCloudxp keynote 19 sept pvu
Cloudxp keynote 19 sept pvu
 
Thara
TharaThara
Thara
 
Trabajos Verano 2º Eso 2009
Trabajos Verano 2º Eso 2009Trabajos Verano 2º Eso 2009
Trabajos Verano 2º Eso 2009
 
11i Logs
11i Logs11i Logs
11i Logs
 
Contiguity Principle
Contiguity PrincipleContiguity Principle
Contiguity Principle
 
Dock It Customer Intro 14 Aug 09
Dock It Customer Intro 14 Aug 09Dock It Customer Intro 14 Aug 09
Dock It Customer Intro 14 Aug 09
 

Similar to Live Migration with AMD-V™ Extended Migration Technology

Amd V Nested Paging
Amd V Nested PagingAmd V Nested Paging
Amd V Nested Paging
jworth
 
AMD-V™ Nested Paging
AMD-V™ Nested PagingAMD-V™ Nested Paging
AMD-V™ Nested Paging
James Price
 
DTI Custom Publication, 2011
DTI Custom Publication, 2011DTI Custom Publication, 2011
DTI Custom Publication, 2011
joeldeer
 
poweredge R7525 installation service manual.pdf
poweredge R7525 installation service manual.pdfpoweredge R7525 installation service manual.pdf
poweredge R7525 installation service manual.pdf
psyberdude1
 
Ibm smart cloud entry+ for system x administrator guide
Ibm smart cloud entry+ for system x administrator guideIbm smart cloud entry+ for system x administrator guide
Ibm smart cloud entry+ for system x administrator guide
IBM India Smarter Computing
 
P25 gv30
P25 gv30P25 gv30
P25 gv30
Clayton Croko
 
Moving to IBM PureFlex System x86-to-x86 Migration
Moving to IBM PureFlex System x86-to-x86 MigrationMoving to IBM PureFlex System x86-to-x86 Migration
Moving to IBM PureFlex System x86-to-x86 Migration
IBM India Smarter Computing
 
M2 m ehs6t_hardware
M2 m ehs6t_hardwareM2 m ehs6t_hardware
M2 m ehs6t_hardware
MarianoPenna
 
Service manual-acer-travel mate-5730-extensa-5630-series
Service manual-acer-travel mate-5730-extensa-5630-seriesService manual-acer-travel mate-5730-extensa-5630-series
Service manual-acer-travel mate-5730-extensa-5630-series
Daniel Blanco
 
Secure Management of Access to Privileged Accounts
Secure Management of Access to Privileged AccountsSecure Management of Access to Privileged Accounts
Secure Management of Access to Privileged Accounts
Hitachi ID Systems, Inc.
 
Secure Management of Privileged Passwords
Secure Management of Privileged PasswordsSecure Management of Privileged Passwords
Secure Management of Privileged Passwords
Hitachi ID Systems, Inc.
 
ESM 5.2 Patch 2 Release Notes
ESM 5.2 Patch 2 Release NotesESM 5.2 Patch 2 Release Notes
ESM 5.2 Patch 2 Release Notes
Protect724
 
Forwarding Connector User;s Guide for 5.1.7.6151 and 6154
Forwarding Connector User;s Guide for 5.1.7.6151 and 6154Forwarding Connector User;s Guide for 5.1.7.6151 and 6154
Forwarding Connector User;s Guide for 5.1.7.6151 and 6154
Protect724
 
Siemens catalog hmi-tp070_e
Siemens catalog hmi-tp070_eSiemens catalog hmi-tp070_e
Siemens catalog hmi-tp070_e
Dien Ha The
 
Siemens catalog hmi-tp070 touch panel
Siemens catalog hmi-tp070 touch panelSiemens catalog hmi-tp070 touch panel
Siemens catalog hmi-tp070 touch panel
Dien Ha The
 
Manual x58 20 super computer
Manual x58 20 super computerManual x58 20 super computer
Manual x58 20 super computer
liourte
 
Netbackup intallation guide
Netbackup intallation guideNetbackup intallation guide
Netbackup intallation guide
rajan981
 
H61 m vs
H61 m vsH61 m vs
H61 m vs
Marcos Daniel
 
Esm rel notes_6.0cp2
Esm rel notes_6.0cp2Esm rel notes_6.0cp2
Esm rel notes_6.0cp2
Protect724v3
 
Cloud Forms Iaa S V2wp 6299847 0411 Dm Web 4
Cloud Forms Iaa S V2wp 6299847 0411 Dm Web 4Cloud Forms Iaa S V2wp 6299847 0411 Dm Web 4
Cloud Forms Iaa S V2wp 6299847 0411 Dm Web 4
Yusuf Hadiwinata Sutandar
 

Similar to Live Migration with AMD-V™ Extended Migration Technology (20)

Amd V Nested Paging
Amd V Nested PagingAmd V Nested Paging
Amd V Nested Paging
 
AMD-V™ Nested Paging
AMD-V™ Nested PagingAMD-V™ Nested Paging
AMD-V™ Nested Paging
 
DTI Custom Publication, 2011
DTI Custom Publication, 2011DTI Custom Publication, 2011
DTI Custom Publication, 2011
 
poweredge R7525 installation service manual.pdf
poweredge R7525 installation service manual.pdfpoweredge R7525 installation service manual.pdf
poweredge R7525 installation service manual.pdf
 
Ibm smart cloud entry+ for system x administrator guide
Ibm smart cloud entry+ for system x administrator guideIbm smart cloud entry+ for system x administrator guide
Ibm smart cloud entry+ for system x administrator guide
 
P25 gv30
P25 gv30P25 gv30
P25 gv30
 
Moving to IBM PureFlex System x86-to-x86 Migration
Moving to IBM PureFlex System x86-to-x86 MigrationMoving to IBM PureFlex System x86-to-x86 Migration
Moving to IBM PureFlex System x86-to-x86 Migration
 
M2 m ehs6t_hardware
M2 m ehs6t_hardwareM2 m ehs6t_hardware
M2 m ehs6t_hardware
 
Service manual-acer-travel mate-5730-extensa-5630-series
Service manual-acer-travel mate-5730-extensa-5630-seriesService manual-acer-travel mate-5730-extensa-5630-series
Service manual-acer-travel mate-5730-extensa-5630-series
 
Secure Management of Access to Privileged Accounts
Secure Management of Access to Privileged AccountsSecure Management of Access to Privileged Accounts
Secure Management of Access to Privileged Accounts
 
Secure Management of Privileged Passwords
Secure Management of Privileged PasswordsSecure Management of Privileged Passwords
Secure Management of Privileged Passwords
 
ESM 5.2 Patch 2 Release Notes
ESM 5.2 Patch 2 Release NotesESM 5.2 Patch 2 Release Notes
ESM 5.2 Patch 2 Release Notes
 
Forwarding Connector User;s Guide for 5.1.7.6151 and 6154
Forwarding Connector User;s Guide for 5.1.7.6151 and 6154Forwarding Connector User;s Guide for 5.1.7.6151 and 6154
Forwarding Connector User;s Guide for 5.1.7.6151 and 6154
 
Siemens catalog hmi-tp070_e
Siemens catalog hmi-tp070_eSiemens catalog hmi-tp070_e
Siemens catalog hmi-tp070_e
 
Siemens catalog hmi-tp070 touch panel
Siemens catalog hmi-tp070 touch panelSiemens catalog hmi-tp070 touch panel
Siemens catalog hmi-tp070 touch panel
 
Manual x58 20 super computer
Manual x58 20 super computerManual x58 20 super computer
Manual x58 20 super computer
 
Netbackup intallation guide
Netbackup intallation guideNetbackup intallation guide
Netbackup intallation guide
 
H61 m vs
H61 m vsH61 m vs
H61 m vs
 
Esm rel notes_6.0cp2
Esm rel notes_6.0cp2Esm rel notes_6.0cp2
Esm rel notes_6.0cp2
 
Cloud Forms Iaa S V2wp 6299847 0411 Dm Web 4
Cloud Forms Iaa S V2wp 6299847 0411 Dm Web 4Cloud Forms Iaa S V2wp 6299847 0411 Dm Web 4
Cloud Forms Iaa S V2wp 6299847 0411 Dm Web 4
 

More from James Price

The Next Generation of Intel: The Dawn of Nehalem
The Next Generation of Intel: The Dawn of NehalemThe Next Generation of Intel: The Dawn of Nehalem
The Next Generation of Intel: The Dawn of Nehalem
James Price
 
Intel® Xeon® Processor 5500 Series
Intel® Xeon® Processor 5500 SeriesIntel® Xeon® Processor 5500 Series
Intel® Xeon® Processor 5500 Series
James Price
 
Intel® Virtualization Technology & Parallels Bring Native Graphics Innovation...
Intel® Virtualization Technology & Parallels Bring Native Graphics Innovation...Intel® Virtualization Technology & Parallels Bring Native Graphics Innovation...
Intel® Virtualization Technology & Parallels Bring Native Graphics Innovation...
James Price
 
Alternative Architecture Overview 44956 A
Alternative Architecture Overview 44956 AAlternative Architecture Overview 44956 A
Alternative Architecture Overview 44956 A
James Price
 
IDC Tech Spotlight: From Silicon To Cloud
IDC Tech Spotlight: From Silicon To CloudIDC Tech Spotlight: From Silicon To Cloud
IDC Tech Spotlight: From Silicon To Cloud
James Price
 
Virtualizing the Next Generation of Server Workloads with AMD™
Virtualizing the Next Generation of Server Workloads with AMD™Virtualizing the Next Generation of Server Workloads with AMD™
Virtualizing the Next Generation of Server Workloads with AMD™
James Price
 
AMD Putting Server Virtualization to Work
AMD Putting Server Virtualization to WorkAMD Putting Server Virtualization to Work
AMD Putting Server Virtualization to Work
James Price
 
DataCore Solutions Overview
DataCore Solutions OverviewDataCore Solutions Overview
DataCore Solutions Overview
James Price
 
AMD Virtualization -- Take Charge
AMD Virtualization -- Take ChargeAMD Virtualization -- Take Charge
AMD Virtualization -- Take Charge
James Price
 
DataCore Corporate Introduction
DataCore Corporate IntroductionDataCore Corporate Introduction
DataCore Corporate Introduction
James Price
 
Advanced Site Recovery -- Technical Briefing
Advanced Site Recovery -- Technical BriefingAdvanced Site Recovery -- Technical Briefing
Advanced Site Recovery -- Technical Briefing
James Price
 
Windows Server 2008 Hyper-V
Windows Server 2008 Hyper-VWindows Server 2008 Hyper-V
Windows Server 2008 Hyper-V
James Price
 

More from James Price (12)

The Next Generation of Intel: The Dawn of Nehalem
The Next Generation of Intel: The Dawn of NehalemThe Next Generation of Intel: The Dawn of Nehalem
The Next Generation of Intel: The Dawn of Nehalem
 
Intel® Xeon® Processor 5500 Series
Intel® Xeon® Processor 5500 SeriesIntel® Xeon® Processor 5500 Series
Intel® Xeon® Processor 5500 Series
 
Intel® Virtualization Technology & Parallels Bring Native Graphics Innovation...
Intel® Virtualization Technology & Parallels Bring Native Graphics Innovation...Intel® Virtualization Technology & Parallels Bring Native Graphics Innovation...
Intel® Virtualization Technology & Parallels Bring Native Graphics Innovation...
 
Alternative Architecture Overview 44956 A
Alternative Architecture Overview 44956 AAlternative Architecture Overview 44956 A
Alternative Architecture Overview 44956 A
 
IDC Tech Spotlight: From Silicon To Cloud
IDC Tech Spotlight: From Silicon To CloudIDC Tech Spotlight: From Silicon To Cloud
IDC Tech Spotlight: From Silicon To Cloud
 
Virtualizing the Next Generation of Server Workloads with AMD™
Virtualizing the Next Generation of Server Workloads with AMD™Virtualizing the Next Generation of Server Workloads with AMD™
Virtualizing the Next Generation of Server Workloads with AMD™
 
AMD Putting Server Virtualization to Work
AMD Putting Server Virtualization to WorkAMD Putting Server Virtualization to Work
AMD Putting Server Virtualization to Work
 
DataCore Solutions Overview
DataCore Solutions OverviewDataCore Solutions Overview
DataCore Solutions Overview
 
AMD Virtualization -- Take Charge
AMD Virtualization -- Take ChargeAMD Virtualization -- Take Charge
AMD Virtualization -- Take Charge
 
DataCore Corporate Introduction
DataCore Corporate IntroductionDataCore Corporate Introduction
DataCore Corporate Introduction
 
Advanced Site Recovery -- Technical Briefing
Advanced Site Recovery -- Technical BriefingAdvanced Site Recovery -- Technical Briefing
Advanced Site Recovery -- Technical Briefing
 
Windows Server 2008 Hyper-V
Windows Server 2008 Hyper-VWindows Server 2008 Hyper-V
Windows Server 2008 Hyper-V
 

Recently uploaded

BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
Neo4j
 
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Muhammad Ali
 
The Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF GuideThe Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF Guide
Shiv Technolabs
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
maigasapphire
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
Anant Gupta
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
Ivanti
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
Safe Software
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
Priyanka Aash
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
aslasdfmkhan4750
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
bellared2
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
The importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT StandardizationThe importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT Standardization
Axel Rennoch
 
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and DisadvantagesBLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
SAI KAILASH R
 

Recently uploaded (20)

BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
 
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
Litestack talk at Brighton 2024 (Unleashing the power of SQLite for Ruby apps)
 
The Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF GuideThe Role of IoT in Australian Mobile App Development - PDF Guide
The Role of IoT in Australian Mobile App Development - PDF Guide
 
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Russian Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
The importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT StandardizationThe importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT Standardization
 
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and DisadvantagesBLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
BLOCKCHAIN TECHNOLOGY - Advantages and Disadvantages
 

Live Migration with AMD-V™ Extended Migration Technology

  • 1. Live Migration with AMD-V™ Extended Migration Technology Publication # 43781 Revision: 3.00 Issue Date: April 2008 Advanced Micro Devices
  • 2. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology © 2007, 2008 Advanced Micro Devices, Inc. All rights reserved. The contents of this document are provided in connection with Advanced Micro Devices, Inc. (“AMD”) products. AMD makes no representations or warranties with respect to the accuracy or completeness of the contents of this publication and reserves the right to make changes to specifications and product descriptions at any time without notice. The information contained herein may be of a preliminary or advance nature and is subject to change without notice. No license, whether express, implied, arising by estoppel, or otherwise, to any intellectual property rights are granted by this publication. Except as set forth in AMD’s Standard Terms and Conditions of Sale, AMD assumes no liability whatsoever, and disclaims any express or implied warranty, relating to its products including, but not limited to, the implied warranty of merchantability, fitness for a particular purpose, or infringement of any intellectual property right. AMD’s products are not designed, intended, authorized or warranted for use as components in systems intended for surgical implant into the body, or in other applications intended to support or sustain life, or in any other application in which the failure of AMD’s product could create a situation where personal injury, death, or severe property or environmental damage may occur. AMD reserves the right to discontinue or make changes to its products at any time without notice. Trademarks AMD, the AMD Arrow logo, AMD Opteron, and combinations thereof, AMD-V are trademarks of Advanced Micro Devices, Inc. Microsoft and Windows are registered trademarks of Microsoft Corporation. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies.
  • 3. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology Contents Revision History ...............................................................................................................................5 1.1 Introduction........................................................................................................................6 1.2 System Virtualization ........................................................................................................6 1.3 Virtual Machine Migration ................................................................................................7 1.4 Determining Processor Features ........................................................................................8 1.5 VMM Basics ....................................................................................................................10 1.6 Problems with Live Migration.........................................................................................10 1.6.1 Backward-Compatibility Problem ...........................................................................11 1.6.2 Forward-Compatibility Problem..............................................................................11 1.7 AMD-V™ Extended Migration Technology...................................................................11 1.7.1 Backward-Compatibility Assurance ........................................................................11 1.7.1.1 Controlling Information Returned through CPUID.............................................11 1.7.1.2 CPUID Override Model Specific Registers.........................................................12 1.7.2 Forward-Compatibility Assurance...........................................................................16 1.7.3 Feature-Disable Bits.................................................................................................17 1.8 Conclusions......................................................................................................................17 1.9 References........................................................................................................................17 Chapter 1 Contents 3
  • 4. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology List of Figures Figure 1. Live Virtual Machine Migration on AMD Processors .......................................................8 Figure 2. Live Migration Support in Current AMD Processors ......................................................16 Chapter 1 List of Figures 4
  • 5. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology Revision History Date Revision Description 4/2008 3.00 Initial public release. Chapter 1 Revision History 5
  • 6. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology Chapter 1 Live Migration with AMD-V™ Extended Machine Migration This document describes virtual machine migration. 1.1 Introduction Virtual Machine (VM) migration is a capability increasingly used in today’s enterprise environments. With live migration, a Virtual Machine Monitor (VMM) moves a running VM nearly instantaneously from one server to another and seamlessly to the end user while maintaining guest uptime guarantees. However, if the processors running in the computers in the pre- and post-migration environment are not identical, live migration can result in unexpected behavior of the guest software. Since the introduction of AMD64 processor technology in 2003, AMD has worked closely with virtualization software developers to define the functionality necessary to ensure that live migration is possible across a broad range of AMD64 processors. This document discusses the basics of live migration and describes the functionality currently available from AMD to enable virtualization software vendors to support live migration across a broad range of AMD64 processors. Figure 1 on page 8 illustrates live virtual machine migration on AMD processors. 1.2 System Virtualization System virtualization is the ability to abstract and pool resources on a physical platform. This abstraction decouples software from hardware and enables multiple operating system images to run concurrently on a single physical platform without interfering with each other. System virtualization has existed for decades on mainframes. In the past, industry-standard x86- based machines with their limited computing resources and challenges in correctly virtualizing the Instruction Set Architecture (ISA)[1,2] did not lend themselves well to the technique. However, with the increase in computing resources of x86-based servers in recent years and with new processor extensions such as those found in AMD-V™[3], x86-based virtualization is seeing widespread adoption and is rapidly becoming a pervasive and integral capability in enterprise environments[4]. Virtualization can increase utilization of computing resources by consolidating the workloads running on many physical systems into virtual machines running on a single physical system. Virtual machines can be provisioned on-demand, replicated, and migrated. VM migration, which is the ability to move a virtual machine from one physical server to another under VMM control, is a capability increasingly utilized in today’s enterprise environments. This document discusses how AMD-V™ Extended Migration Technology enables virtual machine migration between AMD64 processors. Live Migration with AMD-V™ Extended Machine Migration 6
  • 7. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology 1.3 Virtual Machine Migration A virtual machine can be migrated in the following ways: • Static migration. The VM is shut down using OS-supported methods; its static VM image is moved to another VMM and restarted. • Cold migration. The VM is suspended using OS-supported or VMM-supported methods. The suspended VM image is moved to a VMM on a different machine and resumed. • Live migration. The VMM moves a running VM instance nearly instantaneously from one server to another. Live Migration allows for dynamic load balancing of virtualized resource pools, hardware maintenance without downtime, and dynamic failover support. In the following sections, software running in the VM is called “guest software.” As long as the hardware in the pre- and post-migration environment is identical, guest software should behave in exactly the same way before and after the migration. Challenges can arise, however, when guest software runs in a different hardware environment after a migration. Note that even though a VMM presents a virtual platform to guest software, there can be certain interfaces, depending on VMM design, which guest software can directly use to determine underlying hardware’s capabilities, discussed further in Section 1.6 on page 10. After the reboot/restart following a static migration, guest software should go through its platform discovery phase and be able to adjust to any differences in underlying (virtual) hardware. Following a cold migration, guest software may continue to maintain an identical view of the pre- and post-migration hardware. When suspended using OS-supported methods, some operating systems re-scan the hardware upon resume. Depending on their policies and the hardware differences between current and previous hardware, the OSes may refuse to resume and require a reboot. After a live migration, guest software continues to maintain an identical view of the pre- and post- migration hardware. This document discusses how AMD64 processors provide support for a VMM to hide differences in software-visible processor features during live and cold migration. For simplicity, “live migration” is used to refer to both migration methods. Live Migration with AMD-V™ Extended Machine Migration 7
  • 8. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology Figure 1. Live Virtual Machine Migration on AMD Processors 1.4 Determining Processor Features Software should use CPUID to determine processor features and use this information to select appropriate code paths. CPUID is an x86 instruction that can be executed from kernel and applications. The following example shows CPUID called from an application. Live Migration with AMD-V™ Extended Machine Migration 8
  • 9. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology //Example 1: Console application on Windows XP using CPUID instruction to determine //processor revision and feature information. #include "stdafx.h" #include <stdio.h> int _tmain (int argc, _TCHAR* argv[]) { int version; int features_1; int features_2; __asm { mov eax, 1 cpuid mov version, eax mov features_1, ecx mov features_2, edx } printf ("Processor version: 0x%x features_1: 0x%08x features_2: 0x%08xn", version, features_1, features_2); return 0; } However, a small number of existing commercial software applications determines the presence of certain processor features using non-standard methods (such as using a try-catch code block to determine whether certain instructions are supported) or assumes the presence of certain features without testing for them. This software can cause indeterminate behavior under live migration scenarios. Note that such software is equally likely to fail in non-virtualized environments with different processors. Live Migration with AMD-V™ Extended Machine Migration 9
  • 10. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology 1.5 VMM Basics To understand the implications of VMM design on VM migration, it is useful to briefly review various VMM design alternatives. It is important to note that for performance reasons, a VMM may only interfere in the execution of guest software when the guest code performs privileged operations [1,2]. The first method is called para-virtualization, in which the guest’s kernel is modified to cooperate with the VMM when performing privileged operations. The second method is called binary translation, in which the VMM modifies the guest’s binary image at runtime to get processor control when guest software attempts to perform a privileged operation [2]. The VMM can then emulate the privileged operation and return control to guest software. The third method is hardware-assisted virtualization, in which the VMM uses processor extensions such as AMD-V™ to intercept and emulate privileged operations in the guest. 1.6 Problems with Live Migration In the following sections, the processor in pre-migration environment is called the “current” processor. The processor in post-migration environment is called the “target” processor. When performing a live migration, a VM is suspended; its current (guest and VMM) state is captured, moved to another VMM, and resumed on it. The VMM preserves the state and identity of virtual devices after the migration. This process happens quickly and seamlessly to the end user and maintains guest uptime guarantees. As indicated previously, when hardware in the pre- and post-migration environment is different, live migration can result in unexpected guest behavior. After a live migration, there is no indication to guest software that it is now running on a different server with potentially different processor features. This fundamentally different programming environment may not interact well with existing software design. For example, a guest application may use CPUID instruction at startup to determine that the processor supports SSE3 instructions, then live migrate to another server and consequently fail when attempting to use those instructions on a target processor that does not support SSE3. To help ensure that guest software does not malfunction after a live migration, the virtualization infrastructure should address the problems shown in the following subsections. Live Migration with AMD-V™ Extended Machine Migration 10
  • 11. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology 1.6.1 Backward-Compatibility Problem The backward-compatibility problem happens when the target processor does not implement a feature that is implemented on the current processor. In the case of this problem, after live migration, use of such a feature may result in unexpected software behavior. For example, use of an instruction not implemented on the target processor results in a #UD fault. 1.6.2 Forward-Compatibility Problem The forward-compatibility problem happens when the target processor implements features not supported on the current processor. This situation could be problematic in cases where the target processor uses an opcode (to implement an instruction) that is not defined on the current processor, and software on the current processor expects use of this undefined opcode to generate a #UD fault. In such cases, after a live migration, software observes a different processor behavior, that is, it does not observe #UD fault upon use of the (previously undefined) opcode. Fortunately, a very small number of existing commercial software depends on such processor behavior. AMD actively works with its software partners to discourage use of such behaviors. 1.7 AMD-V™ Extended Migration Technology Since the introduction of AMD64 processor technology in 2003, AMD has worked closely with virtualization software developers to define the functionality necessary so that live migration is possible across a broad range of products. AMD-V™ Extended Migration Technology provides various capabilities to address problems associated with Live Migration. 1.7.1 Backward-Compatibility Assurance To assure backward compatibility, the VMM can intercept the #UD fault and emulate the faulting instruction. However, this method of emulating the faulting instruction is not generally acceptable because it adds to the complexity of the VMM and may also degrade the performance of the guest. A simpler and more widely adopted solution by many VMM vendors is to expose to the guest software only those features that are supported on all processors in the resource pool. This solution essentially means establishing the lowest common denominator of processor features in the resource pool. 1.7.1.1 Controlling Information Returned through CPUID When new features are introduced on AMD processors, their presence is indicated by unique bits in the information returned by the CPUID instruction. Properly written software relies only on this information to determine processor features. To ensure proper behavior during VM migration, a VMM must control information returned to guest software as a result of the CPUID instruction. Live Migration with AMD-V™ Extended Machine Migration 11
  • 12. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology A VMM using hardware-assisted virtualization (AMD-V™) (such as Xen HVM or Microsoft® Virtual Server) can use the CPUID intercept to return the appropriate bits whenever guest software executes that instruction. A VMM using binary translation (such as VMware ESX) or para-virtualization (such as Xen) may need to handle this differently depending on whether CPUID was executed by guest’s kernel or guest’s application. In the case of CPUID instruction executed by the guest’s kernel, the VMM can binary-translate or para-virtualize that instance of the instruction and return appropriate information to the guest. For a CPUID instruction executed within the guest application’s context (where binary translation or para-virtualization techniques are not feasible), the VMM requires a way to return the appropriate CPUID information to that guest application. Note that with binary translation and para-virtualization, it is the user-mode CPUID features that the VMM must control. Without this ability to control the user-mode CPUID features, all processors in the resource pool must support the same features, which significantly reduces flexibility in utilizing resources in environments supporting Live Migration. The AMD CPUID Override Model Specific Registers (MSRs), as described in Section 1.7.1.2, provide the VMM the ability to control CPUID information returned to guest applications. 1.7.1.2 CPUID Override Model Specific Registers Starting with AMD Family 0Fh revision C AMD Opteron™ processors, AMD provides CPUID Override MSRs to allow a VMM (or OS) to specify a subset of information that the CPUID instruction returns. This is accomplished by setting the appropriate MSRs as described below. Live Migration with AMD-V™ Extended Machine Migration 12
  • 13. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology MSR Type CPUID Features Register Override MSR Description Provides control over features reported in CPUID function 0000_0001 in ECX and EDX. MSR Index C001_1004h Bits Description 63:32 Features. R/W, function 1, ECX 31:0 Features. R/W, function 1, EDX MSR Type Extended CPUID Features Register Override MSR Description Provides control over features reported in CPUID function 8000_0001 in ECX and EDX. MSR Index C001_1005h Bits Description 63:32 Features. R/W, function 1, ECX 31:0 Features. R/W, function 1, EDX MSR Type Logical Processor Count (K8 only) MSR Description MSR is provided for backward-compatibility with K8 processors. MSR Index C001_100Dh Bits Description 63:32 Reserved 31:0 Logical Processor Count; R/W MSR Type Processor Name String Registers MSR Description These registers hold the CPUID name string in ASCII. The state of these registers is returned by CPUID instruction Functions 8000_0004, 8000_0003, 8000_0002. Each register contains a block of 8 ASCII characters; the least significant byte corresponds to the first ASCII character of the block; the most-significant byte corresponds to the last character of the block. MSR C001_0030h provides the first block of the name string; MSR C001_0035h contains the last block of the name string. MSR Index C001_0030h – C001_0035h Bits Description 63:0 CpuNameString. R/W. Live Migration with AMD-V™ Extended Machine Migration 13
  • 14. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology When the VMM or OS does not override CPUID return information using this mechanism, the processor returns CPUID information based on the native feature set. The following code sample shows how a VMM (or OS) can hide reporting of SSE4A instructions on a quad-core AMD Opteron™ processor. /* * Example 2: Use MSR C001_1005 to clear bit 6 (SSE4A) reported in ECX after CPUID * Function 8000_0001 */ /* Read current value of the CPUID Override MSR C001_1005. After RDMSR completes, EDX:EAX contains the 64bit MSR value. EDX is loaded with the high 32 bits of the MSR and EAX is loaded with the low 32 bits. The high 32 bits of this MSR are returned in ECX after CPUID Function 8000_0001 */ MOV CX, 0xC0011005h RDMSR /* Clear bit 6 (SSE4A) of EDX register */ ANDL EDX, 0xFFFFFFBFh /* Write the new EDX:EAX value into CPUID override MSR */ WRMSR Live Migration with AMD-V™ Extended Machine Migration 14
  • 15. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology The following code sample shows how a VMM (or OS) can hide reporting of RDTSCP instruction on AMD Opteron™ processor, rev. F. /* * Example 3: Use MSR C001_1005 to clear bit 27 (RDTSCP) reported in EDX after * CPUID Function 8000_0001 */ /* Read current value of the CPUID Override MSR C001_1005. After RDMSR completes, EDX:EAX contains the 64bit MSR value. EDX is loaded with the high 32 bits of the MSR and EAX is loaded with the low 32 bits. The low 32 bits of this MSR are returned in EDX after CPUID Function 8000_0001 */ /* Write the new EDX:EAX value into CPUID override MSR. AMD Opteron™ Rev-F Processors require a 32 bit password in EDI. Contact AMD to get the password. */ MOV EDI, <PASSWORD> MOV CX, 0xC0011005h RDMSR /* Clear bit 27 (RDTSCP) of EAX register */ ANDL EAX, 0xF7FFFFFFh WRMSR Note that overriding CPUID return information using these MSRs does not disable the respective features of the processor. On AMD Opteron™ processors, a password must be passed through the EDI register before executing RDMSR and WRMSR to set the CPUID overrides. This password can be obtained from AMD. Live Migration with AMD-V™ Extended Machine Migration 15
  • 16. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology Figure 2. shows Live Migration support in current AMD processors: AMD Opteron ™ AMD Opteron ™ AMD Opteron ™ Processor – Rev C Processor – Rev E Processor – Rev F Models 840, 840 EE, 842, 846 Models 860, 860 HE, 865, 870, Models 8212, 8220, 8218, HE, 850 … 875, 890, … 8222 SE, … Models 240, 242, 244, 246, Models 260 HE, 265, 275 HE, Models 2220,2212, 2214, 246 HE, 248, … 280, 290, … 2210, … Models 140, 140EE, 146HE, Models 165, 185, … Models 1222 SE, 1214, 148, 150, … 1210, … SSE SSE3 SVM MMX RDTSCP SSE2 AMD-V Extended Migration Technology SKINIT CMPXCHG16B Nested Paging AMD Opteron™ SSE4A Quad-Core POPCNT IBS (AMD Family 10h OSVW codenamed LZCNT “Barcelona”) 1 GB Paging MONITOR/MWAIT Figure 2. Live Migration Support in Current AMD Processors If guest software does not use CPUID to determine processor features and, instead, relies on deprecated methods (such as using certain code sequences to detect implementation differences and using the result to identify processor features), then these overrides may be of little use. Fortunately, AMD field surveys indicate that such software is very limited in today’s enterprise environments. 1.7.2 Forward-Compatibility Assurance In Section 1.6.2 on page 11, we discussed forward-compatibility problems. To avoid these problems, the VMM needs a method to disable new instructions until all the processors in the resource pool implement them. Disabling new instructions is required to ensure that software continues to observe the same processor behavior (such as expected #UD faults) in forward- migration scenarios. Live Migration with AMD-V™ Extended Machine Migration 16
  • 17. 43781 Rev. 3.00 April 2008 Live Migration with AMD-V™ Extended Migration Technology 1.7.3 Feature-Disable Bits On future processors, AMD may support feature-disable bits to disable certain instructions or instruction sets. Such feature-disable bits could be exposed through an MSR-based interface. AMD believes that the long-term solution for forward-compatibility assurance is encouraging developers to write well-behaved, “live-migration friendly” code and adding appropriate support in VMMs to handle forward-compatibility problems. 1.8 Conclusions In this document, we have described in detail the potential problems that can cause guest software to malfunction after a Live Migration. We also discussed how AMD64 processors are designed to provide the functionality needed to support the VMM in addressing these problems. AMD-V Extended Migration Technology allows a VMM to safely migrate a VM between various AMD64 processor revisions. AMD-V provides IT administrators an unprecedented flexibility in deploying, maintaining, and upgrading servers in Live Migration environments based on AMD64 processors. Readers are cautioned about instances of legacy software that rely on non-standard methods to determine processor features. However, such instances are limited and should diminish over time. AMD encourages IT administrators to work with their VMM software vendors in determining the level of VMM’s support for AMD-V™ Extended Migration Technology. The respective vendors can provide guidance when configuring Live Migration resource pools with AMD processors[5]. 1.9 References 1. Popek, G. J.; Goldberg, R. P. “Formal Requirements for Virtualizable Third Generation Architectures,” Communications of the ACM; 17(7): 412 –421. 2. Adams, K.; Agesen, O. “Comparison of Software and Hardware Techniques for x86 Virtualization.” ASPLOS. 3. AMD. AMD64 Architecture Programmer’s Manual, Vol. 2. 4. IDC. “Impact of Virtualization Software on Operating Environments.” IDC Technology Assessment. 5. VMware. “VMotion CPU Compatibility Requirements for AMD Processors.” Live Migration with AMD-V™ Extended Machine Migration 17