/VM

                Tsuyoshi Ozawa
                Twitter oza_x86




2009   11   9                     1
2009   11   9   2
2009   11   9   3
2009   11   9   4
2009   11   9   5
KVM


2009   11   9         6
for x86

2009   11   9             7
Linux on Real Machine   Linux on Virtual Machine




                          (=OS)
2009   11   9                        ...
2009   11   9   9
x86

                      (Shadow Page Table)




2009   11   9                               10
CPU

                            0x1000   0x11000
                  Virtual                      Physical
                ...
•

                •


2009   11   9       12
•
                •


2009   11   9       13
• OS
                •      0x1000   0x11000

                •      0x2000   0x12000

                         ...      ....
•
                   0x1000   0x11000


            • OS   0x2000   0x12000

                     ...      ...



2009   1...
•
                                          ?

                •
                                swap in
                 ...
1.
                             CR3

                2.               MMU

                3.   TLB               CPU
    ...
CR3
                •

                • Control Register   3


2009   11   9                                  18
•                      MMU


                • Memory Management Unit
2009   11   9                                19
• Translation Look-aside Buffer   TLB
                •


                • CPU      SRAM


2009   11   9                 ...
Virtual Address 0xc0003120        3



                       20bit        12bit     4.

                          CR3
   ...
2009   11   9   22
VM


2009   11   9        23
2009   11   9   24
VM


2009   11   9        25
VM




                • OS
2009   11   9           26
VM




                Virtual
                Machine
                Monitor
                           VMM
            ...
VMM

                •                 3         VMM

                • Robert P. Goldberg. ,Architectural
               ...
Type I VMM

                •

                • Xen,VMware ESX
                                   VMM
                   ...
Type II VMM
                • OS


                • Qemu      Proc       VMM
                 LilyVM
                 Boc...
Hybrid    VMM

                • Type I + Type II
                •
                • VMware Proc VMM
                  Wo...
Hybrid    VMM


            •
                         VMM
                         OS     VMM
                          H...
VM


2009   11   9        33
2

                • Shadow Page Table ( SPT )
                • Nested Page Table (Extended Page Table)


2009   11   9  ...
Shadow Page Table

                •     OS
                               Shadow
                      OS

              ...
SPT
                Hybrid   VMM




                         OS      VMM
                           Hardware

2009   11  ...
SPT
                          OS
                          VM
       Guest side        vTLB   vCR3

            Host side ...
SPT
                                create page table
                          OS                      Guest
            ...
SPT
                          OS             Guest
                                       page table
                     ...
SPT
                          OS             Guest
                                       page table
                     ...
SPT
                          OS             Guest
                                       page table
                     ...
SPT
                          OS                   Guest
                                             page table
         ...
SPT
                  OS             Guest
                               page table
                                     ...
SPT
                  OS             Guest
                               page table
                                     ...
SPT
                  OS             Guest
                               page table
                                     ...
2009   11   9   46
SPT
                  OS             Guest
                               page table
                                     ...
SPT
                  OS             Guest
                               page table
                                     ...
SPT
                  OS             Guest
                               page table
                                     ...
SPT

                •       OS




                • VMM        SPT




2009   11   9                      50
SPT

                • Pre-Validation
                • Virtual TLB
                • Virtual TLB + Memory Trace


2009   ...
Pre-validation

                • CR3
                •            PT

                • SPT


2009   11   9              ...
Pre-validation
                                create page table
                          OS                      Guest
 ...
Pre-validation
                          OS             Guest
                                       page table
          ...
Pre-validation
                          OS             Guest
                                       page table
          ...
Pre-validation
                          OS             Guest
                                       page table
          ...
Pre-validation
                          OS             Guest
                                       page table
          ...
Pre-validation
                          OS                   Guest
                                             page tabl...
Pre-validation
                  OS              Guest
                                page table
                        ...
SPT
                  OS             Guest
                               page table
                                     ...
Pre-validation
                •
                    ( CR3    )   SPT




                •

2009   11   9                ...
Virtual TLB
                • CR3
                    SPT 1

                •       TLB
                            SPT

...
Virtual TLB
                                create page table
                          OS                      Guest
    ...
Virtual TLB
                          OS             Guest
                                       page table
             ...
Virtual TLB
                          OS             Guest
                                       page table
             ...
Virtual TLB
                          OS             Guest
                                       page table
             ...
Virtual TLB
                          OS             Guest
                                       page table
             ...
Virtual TLB
                          OS                   Guest
                                             page table
 ...
Virtual TLB
                          VMM
                          OS                   Guest
                           ...
Virtual TLB
                  OS             Guest
                               page table
                             ...
Virtual TLB


                  OS             Guest
                               page table
                           ...
Virtual TLB


                  OS             Guest
                               page table
                           ...
Virtual TLB


                  OS             Guest
                               page table
                           ...
Virtual TLB


                  OS             Guest
                               page table
                           ...
Virtual TLB


                  OS             Guest
                               page table   2
                       ...
Virtual TLB


                         OS            Guest
                                     page table
               ...
Virtual TLB

                                                    OS
                         OS            Guest
         ...
Virtual TLB


                         OS            Guest
                                     page table
               ...
Virtual TLB

                •
                    •
                    •
                •          SPT



2009   11   9...
Virtual TLB

                •   TLB

                    •   CR3

                    •   invlpg

                •      ...
SPT

                • TLB
                    SPT

                •         OS   PT SPT




2009   11   9               ...
Memory Trace

                •             OS   PT



                •             OS
                                  ...
• x86
                 •
                   CPU

                 • Linux KVM


2009   11   9                  83
kvm-88/kernel/x86/paging_tmpl.h



2009   11   9                                     84
Nested Page Table
                (Extended Page Table)
                •

                    0x1000   0x11000   0x21000
...
NPT/EPT
                • AMD    CPU
                              CPU

                • Intel CPU
                      ...
• VM
                 • SPT
                   • Pre-validation
                   • Virtual TLB
                 • NPT/EP...
Upcoming SlideShare
Loading in...5
×

Memory Virtualization

3,337

Published on

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,337
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
116
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Transcript of "Memory Virtualization"

  1. 1. /VM Tsuyoshi Ozawa Twitter oza_x86 2009 11 9 1
  2. 2. 2009 11 9 2
  3. 3. 2009 11 9 3
  4. 4. 2009 11 9 4
  5. 5. 2009 11 9 5
  6. 6. KVM 2009 11 9 6
  7. 7. for x86 2009 11 9 7
  8. 8. Linux on Real Machine Linux on Virtual Machine (=OS) 2009 11 9 8
  9. 9. 2009 11 9 9
  10. 10. x86 (Shadow Page Table) 2009 11 9 10
  11. 11. CPU 0x1000 0x11000 Virtual Physical Memory Memory 0x2000 0x12000 ... ... 2009 11 9 11
  12. 12. • • 2009 11 9 12
  13. 13. • • 2009 11 9 13
  14. 14. • OS • 0x1000 0x11000 • 0x2000 0x12000 ... ... 2009 11 9 14
  15. 15. • 0x1000 0x11000 • OS 0x2000 0x12000 ... ... 2009 11 9 15
  16. 16. • ? • swap in Physical swap out Memory 2009 11 9 16
  17. 17. 1. CR3 2. MMU 3. TLB CPU CR3 x86 CPU MMU TLB 2009 11 9 17
  18. 18. CR3 • • Control Register 3 2009 11 9 18
  19. 19. • MMU • Memory Management Unit 2009 11 9 19
  20. 20. • Translation Look-aside Buffer TLB • • CPU SRAM 2009 11 9 20
  21. 21. Virtual Address 0xc0003120 3 20bit 12bit 4. CR3 2. MMU Physical Memory TLB 3. 1.TLB 2009 11 9 21
  22. 22. 2009 11 9 22
  23. 23. VM 2009 11 9 23
  24. 24. 2009 11 9 24
  25. 25. VM 2009 11 9 25
  26. 26. VM • OS 2009 11 9 26
  27. 27. VM Virtual Machine Monitor VMM Hardware 2009 11 9 27
  28. 28. VMM • 3 VMM • Robert P. Goldberg. ,Architectural Principles for Virtual Computer Systems. 2009 11 9 28
  29. 29. Type I VMM • • Xen,VMware ESX VMM Hardware 2009 11 9 29
  30. 30. Type II VMM • OS • Qemu Proc VMM LilyVM Bochs OS Hardware 2009 11 9 30
  31. 31. Hybrid VMM • Type I + Type II • • VMware Proc VMM Workstation Linux KVM OS VMM Hardware 2009 11 9 31
  32. 32. Hybrid VMM • VMM OS VMM Hardware 2009 11 9 32
  33. 33. VM 2009 11 9 33
  34. 34. 2 • Shadow Page Table ( SPT ) • Nested Page Table (Extended Page Table) 2009 11 9 34
  35. 35. Shadow Page Table • OS Shadow OS • OS Shadow 2009 11 9 35
  36. 36. SPT Hybrid VMM OS VMM Hardware 2009 11 9 36
  37. 37. SPT OS VM Guest side vTLB vCR3 Host side VMM Hardware TLB CR3 2009 11 9 37
  38. 38. SPT create page table OS Guest page table VM Guest side vTLB vCR3 Host side VMM Hardware TLB CR3 2009 11 9 38
  39. 39. SPT OS Guest page table VM Guest side vTLB vCR3 Host side VMM ... Hardware TLB CR3 2009 11 9 39
  40. 40. SPT OS Guest page table VM Guest side vTLB vCR3 VMM Host side VMM Hardware TLB CR3 2009 11 9 40
  41. 41. SPT OS Guest page table VM Guest side vTLB vCR3 VMM Host side VMM PT Hardware TLB CR3 2009 11 9 41
  42. 42. SPT OS Guest page table VM Guest side vTLB vCR3 create SPT Host side VMM Shadow page table Hardware TLB CR3 2009 11 9 42
  43. 43. SPT OS Guest page table Guest Physical VM Memory vTLB vCR3 VMM Shadow page table OS vCR3 Host Physical Hardware Memory TLB CR3 2009 11 9 43
  44. 44. SPT OS Guest page table Guest Physical VM Memory vTLB vCR3 Hook VMM Shadow VMM page table Host Physical Hardware Memory TLB CR3 2009 11 9 44
  45. 45. SPT OS Guest page table Guest SPT Physical VM Memory vTLB vCR3 CR3 VMM Shadow page table Host Physical Hardware Memory TLB CR3 2009 11 9 45
  46. 46. 2009 11 9 46
  47. 47. SPT OS Guest page table Guest Physical VM Memory vTLB vCR3 VMM Shadow page table Host Physical Hardware Memory TLB CR3 2009 11 9 47
  48. 48. SPT OS Guest page table Guest Physical VM Memory vTLB vCR3 VMM Shadow page table Host Physical Hardware Memory TLB CR3 2009 11 9 48
  49. 49. SPT OS Guest page table Guest Physical VM Memory vTLB vCR3 VMM Shadow page table Host Physical Hardware Memory TLB CR3 2009 11 9 49
  50. 50. SPT • OS • VMM SPT 2009 11 9 50
  51. 51. SPT • Pre-Validation • Virtual TLB • Virtual TLB + Memory Trace 2009 11 9 51
  52. 52. Pre-validation • CR3 • PT • SPT 2009 11 9 52
  53. 53. Pre-validation create page table OS Guest page table VM Guest side vTLB vCR3 Host side VMM Hardware TLB CR3 2009 11 9 53
  54. 54. Pre-validation OS Guest page table VM Guest side vTLB vCR3 OS Host side VMM PT Hardware TLB CR3 2009 11 9 54
  55. 55. Pre-validation OS Guest page table VM Guest side vTLB vCR3 CR3 Host side VMM Hardware TLB CR3 2009 11 9 55
  56. 56. Pre-validation OS Guest page table VM Guest side vTLB vCR3 VMM Host side VMM Hardware TLB CR3 2009 11 9 56
  57. 57. Pre-validation OS Guest page table VM Guest side vTLB vCR3 VMM Host side VMM PT Hardware TLB CR3 2009 11 9 57
  58. 58. Pre-validation OS Guest page table VM Guest side vTLB vCR3 create SPT Host side VMM Shadow page table Hardware TLB CR3 2009 11 9 58
  59. 59. Pre-validation OS Guest page table Guest SPT Physical VM Memory vTLB vCR3 CR3 VMM Shadow page table Host Physical Hardware Memory TLB CR3 2009 11 9 59
  60. 60. SPT OS Guest page table Guest Physical VM Memory vTLB vCR3 VMM Shadow page table Host SPT Physical Hardware Memory TLB CR3 2009 11 9 60
  61. 61. Pre-validation • ( CR3 ) SPT • 2009 11 9 61
  62. 62. Virtual TLB • CR3 SPT 1 • TLB SPT • TLB VMM 2009 11 9 62
  63. 63. Virtual TLB create page table OS Guest page table VM Guest side vTLB vCR3 Host side VMM Hardware TLB CR3 2009 11 9 63
  64. 64. Virtual TLB OS Guest page table VM Guest side vTLB vCR3 OS Host side VMM PT Hardware TLB CR3 2009 11 9 64
  65. 65. Virtual TLB OS Guest page table VM Guest side vTLB vCR3 CR3 Host side VMM Hardware TLB CR3 2009 11 9 65
  66. 66. Virtual TLB OS Guest page table VM Guest side vTLB vCR3 VMM Host side VMM Hardware TLB CR3 2009 11 9 66
  67. 67. Virtual TLB OS Guest page table VM Guest side vTLB vCR3 VMM Host side VMM PT Hardware TLB CR3 2009 11 9 67
  68. 68. Virtual TLB OS Guest page table VM Guest side vTLB vCR3 create SPT Host side VMM Shadow page table Hardware 1st stage TLB CR3 2009 11 9 68
  69. 69. Virtual TLB VMM OS Guest page table VM Guest side 1 vTLB vCR3 create SPT Host side VMM Shadow page table Hardware 1st stage TLB CR3 2009 11 9 69
  70. 70. Virtual TLB OS Guest page table Guest SPT Physical VM Memory vTLB vCR3 CR3 VMM Shadow page table Host 1st stage Physical Hardware Memory TLB CR3 2009 11 9 70
  71. 71. Virtual TLB OS Guest page table Guest Physical VM Memory vTLB vCR3 VMM Shadow page table Host 1st stage Physical Hardware Memory TLB CR3 2009 11 9 71
  72. 72. Virtual TLB OS Guest page table Guest Physical VM Memory vTLB vCR3 VMM Shadow page table Host 1st stage Physical Hardware Memory TLB CR3 2009 11 9 72
  73. 73. Virtual TLB OS Guest page table Guest Physical VM Memory vTLB vCR3 VMM Shadow page table Host 1st stage Physical Hardware Memory TLB CR3 2009 11 9 73
  74. 74. Virtual TLB OS Guest page table Guest Physical VM Memory vTLB vCR3 VMM VMM Shadow page table PT Host 1st stage Physical Hardware Memory TLB CR3 2009 11 9 74
  75. 75. Virtual TLB OS Guest page table 2 Guest Physical VM Memory vTLB vCR3 Shadow VMM page table Host 2nd stage Physical Hardware Memory TLB CR3 2009 11 9 75
  76. 76. Virtual TLB OS Guest page table Guest Physical VM Memory vTLB vCR3 Shadow VMM page table Host 2nd stage Physical Hardware Memory TLB CR3 ( ) CPU 2009 11 9 76
  77. 77. Virtual TLB OS OS Guest page table Guest Physical VM Memory vTLB vCR3 Shadow VMM page table Host 2nd stage Physical Hardware Memory TLB CR3 ( ) CPU 2009 11 9 77
  78. 78. Virtual TLB OS Guest page table Guest Physical VM Memory vTLB vCR3 Shadow VMM page table Host 2nd stage Physical Hardware Memory TLB CR3 ( ) CPU 2009 11 9 78
  79. 79. Virtual TLB • • • • SPT 2009 11 9 79
  80. 80. Virtual TLB • TLB • CR3 • invlpg • OS • PTE 2009 11 9 80
  81. 81. SPT • TLB SPT • OS PT SPT 2009 11 9 81
  82. 82. Memory Trace • OS PT • OS SPT (Trace) 2009 11 9 82
  83. 83. • x86 • CPU • Linux KVM 2009 11 9 83
  84. 84. kvm-88/kernel/x86/paging_tmpl.h 2009 11 9 84
  85. 85. Nested Page Table (Extended Page Table) • 0x1000 0x11000 0x21000 0x2000 0x12000 0x42000 ... ... 2009 11 9 85
  86. 86. NPT/EPT • AMD CPU CPU • Intel CPU CPU(i7 ) • VMM • SPT 2009 11 9 86
  87. 87. • VM • SPT • Pre-validation • Virtual TLB • NPT/EPT 2009 11 9 87
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×