Logical Domains
CMT Tech Day
January 6th,2010

Orgad Kimchi
ISV Engineering
LDoms Product Engineering


Agenda



   LDoms Architecture
   LDoms 1.2 -What's new
   Ops Center 2.5 integration
   Comp...
LDoms Product Engineering



 Virtualize Everywhere!
       Quickly (re)deploy new                                   Provi...
LDoms Product Engineering



Consolidation
●   Low server utilization rate in the data center
    –   Consolidation is the...
LDoms Product Engineering



Logical Domains
●   SPARC based
●   Para-virtualized OS         Control
                     ...
LDoms Product Engineering



LDoms Fundamentals
●   Each virtual machine appears as an entirely
    independent machine
● ...
LDoms Product Engineering



Roles for Logical Domains
●   Control Domain – The logical Domain manger
    runs in this dom...
LDoms Product Engineering



LDoms Manager
●   Software to manage LDoms
●   To create, configure, monitor and control
    ...
LDoms Product Engineering



LDoms Manager Architecture
                                                                  ...
LDoms Product Engineering



LDoms and CPU
 ●   SPARC CMT hardware strands are
     partitioned to logical domains as “vir...
LDoms Product Engineering



LDoms and Memory
 ●   Hypervisor reserves a small portion of system
     memory for its own u...
LDoms Product Engineering



LDoms Virtualized I/O Architecture
                Logical Domain A                     Servi...
LDoms Product Engineering



LDoms 1.1 Released in Dec 2008
Domain Mobility                    Multiple I/O domains
Fully ...
LDoms Product Engineering



Virtual I/O Dynamic Reconfiguration
 Add/Remove virtual I/O services and devices
 without reb...
LDoms Product Engineering



Network Hybrid I/O
 Network virtualized I/O path:
 Guest domain ↔ service domain ↔ physical N...
LDoms Product Engineering



Network Hybrid I/O 1/3
                                   Physical Network




              ...
LDoms Product Engineering



Network Hybrid I/O 2/3
                                   Physical Network




              ...
LDoms Product Engineering



Network Hybrid I/O 3/3
                                   Physical Network




              ...
LDoms Product Engineering



VLAN (802.1q) Support
 ●   Add VLAN Support to Virtual Network I/O
 ●   Ethernet packets swit...
LDoms Product Engineering



What's New in LDOM 1.2
●   Support for jumbo frames
●   Support for configuring domain depend...
LDoms Product Engineering



Jumbo Frames
●   The Logical Domains virtual switch (vsw) and
     virtual network (vnet) dev...
LDoms Product Engineering



Domain Dependencies
●   You can use the Logical Domains Manager to
      establish dependency...
LDoms Product Engineering



Autorecovery of configurations
●    Copy of the current configuration is
      automatically ...
LDoms Product Engineering



Export a Virtual Disk Backend Multiple Times
A virtual disk backend can be exported multiple
...
LDoms Product Engineering



Domain Mobility
 •Cold and Live migration can migrate between
 different system and CPU types...
LDoms Product Engineering



Warm Migration 1/6 - Initialization
                                virtual disk backend
    ...
LDoms Product Engineering




Warm Migration 2/6 – New Guest Creation
                                virtual disk backend...
LDoms Product Engineering




Warm Migration 3/6 – Shrink Source Guest
                              virtual disk backend
...
LDoms Product Engineering



Warm Migration 4/6 – State Transfer
                                virtual disk backend
    ...
LDoms Product Engineering




Warm Migration 5/6 – Target Guest Resume
                              virtual disk backend
...
LDoms Product Engineering




Warm Migration 6/6 – Completion & Cleanup
                               virtual disk backen...
LDoms Product Engineering



Physical-to-Virtual Migration Tool ( P2V)

The Logical Domains P2V Migration Tool automatical...
LDoms Product Engineering



Logical Domains Physical-to-Virtual Migration Tool cont..
The conversion from a physical syst...
LDoms Product Engineering



Logical Domains Configuration Assistant
●   The Logical Domains Configuration Assistant
     ...
LDoms Product Engineering


                    Ops Center Solution Set
Comprehensive management for physical and virtual
...
LDoms Product Engineering


               Virtualization Management
  For Solaris Zones and Logical Domains
Life-cycle mg...
LDoms Product Engineering



LDoms vs Containers
Feature                LDoms                           Containers
Platfor...
LDoms Product Engineering




LDoms Engineering Roadmap




                                38
LDoms Product Engineering



Competitive Differentiators
●   No-Cost, Built-in Virtualization
●   LDoms redundancy mechani...
Thank you!

    http://sun.com/ldoms
http://blogs.sun.com/vreality
Upcoming SlideShare
Loading in...5
×

Logical Domains

2,127

Published on

Scalable CMT deployment using LDOMs
CMT Tech Day
06 jan 2010, Herzliya

Published in: Technology, Business
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,127
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
0
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Logical Domains

  1. 1. Logical Domains CMT Tech Day January 6th,2010 Orgad Kimchi ISV Engineering
  2. 2. LDoms Product Engineering Agenda LDoms Architecture LDoms 1.2 -What's new Ops Center 2.5 integration Comparison LDoms vs Zones LDoms Roadmap 2
  3. 3. LDoms Product Engineering Virtualize Everywhere! Quickly (re)deploy new Provide DR2 capabilities applications Client Service Data Center Management W E EMA C E e R IL R R Developer b App P App Mail M P Databa Databa Application Network Serv Serv Serv se se er er er OS Server Maximize system resources Storage 3
  4. 4. LDoms Product Engineering Consolidation ● Low server utilization rate in the data center – Consolidation is the major driver to reduce server footprints and better utilize the server capacity – Goal: cut IT hardware spending and ongoing support and administration cost while streamlining the deployment of new IT services 10% Utilisation 20% Time 10% 4
  5. 5. LDoms Product Engineering Logical Domains ● SPARC based ● Para-virtualized OS Control Domain LDom 2 LDom 3 ● Level 1 Hypervisor Solaris Solaris 10 Nevada App ldm App App App App App ldmd Zone Zone 1 Zone 2 Hypervisor MD MD MD Hardware CPU CPU CPU CPU Shared CPU, Memory, IO Mem Mem Mem I/O 5
  6. 6. LDoms Product Engineering LDoms Fundamentals ● Each virtual machine appears as an entirely independent machine ● Own kernel, patches, tuning parameters ● Own user accounts, administrators ● Own disks ● Own console and OBP ● Own network interfaces, MAC & IP addresses ● Each domain can start, stop and reboot independently of each other 6
  7. 7. LDoms Product Engineering Roles for Logical Domains ● Control Domain – The logical Domain manger runs in this domain,create and mange other logical domains ,allocate resources. ● Service Domain - Provides virtual device services to other domains ,vdisk,vconsole. ● I/O Domain – Direct ownership of and direct access to physical I/O devices such as,network card in a PCI EXPRESS controller. ● Guest Domain -Non I/O domain that consumes virtual device services. 7
  8. 8. LDoms Product Engineering LDoms Manager ● Software to manage LDoms ● To create, configure, monitor and control domains ● Maps Logical Domains to physical resources ● Build and store MDs in the hypervisor ● Currently implemented as a Solaris daemon + CLI ● Daemon: ldmd ; CLI: ldm 8
  9. 9. LDoms Product Engineering LDoms Manager Architecture Control Plane Guest Control Domain Domain ldm ldmd Domain LDom LDom services Sequencer controller HV control Domain Constraint services DB Hypervisor Logical Domain Channel HV Control Point Service Processor 9
  10. 10. LDoms Product Engineering LDoms and CPU ● SPARC CMT hardware strands are partitioned to logical domains as “virtual CPU's” ● Ability to dynamically add/remove virtual CPU's while the guest OS is running (aka CPU DR) ● Unlike x64 Virtualization offerings which time-slice CPU resources across virtual machines 10
  11. 11. LDoms Product Engineering LDoms and Memory ● Hypervisor reserves a small portion of system memory for its own use ● Remaining system memory is allocated to logical domains by the LDoms Manager ● LDoms Manager allocates memory in blocks from physical pool; assigns guest Real Address (RA) for each block ● SPARC/Solaris requires 1GB minimum memory 11
  12. 12. LDoms Product Engineering LDoms Virtualized I/O Architecture Logical Domain A Service Domain App Device Driver App /pci@B/qlc@6 App App Virtual Device Nexus Driver Service /pci@B Virtual Device Privileged Driver Virtual Nexus Hyper Hypervisor Logical Domain Channel Privileged I/O MMU Hardware I/O PCI Root Bridge PCI B 12
  13. 13. LDoms Product Engineering LDoms 1.1 Released in Dec 2008 Domain Mobility Multiple I/O domains Fully Supported XML CPU dynamic Interface reconfiguration Virtual I/O Dynamic DVD and ISO image Reconfiguration installation support NIU Hybrid IO Enhanced vdisk support: format, devid, SCSI, I/O VLANs (802.1q) timeout Oracle Certification Sun Cluster support 13
  14. 14. LDoms Product Engineering Virtual I/O Dynamic Reconfiguration Add/Remove virtual I/O services and devices without rebooting vds, vsw, vdisk, vnet, vcc No CLI changes but effect is immediate Examples: # ldm add-vdisk vdiskN diskN@primary-vds0 ldg1 # ldm add-vnet vnetN primary-vsw0 ldg1 vdiskN and vnetN are immediately available in domain ldg1 A device cannot be removed if it is in use 14
  15. 15. LDoms Product Engineering Network Hybrid I/O Network virtualized I/O path: Guest domain ↔ service domain ↔ physical NIC Network hybrid I/O path: Guest domain ↔ physical NIC Except broadcast and multicast Better performance and scalability No overhead of the service domain virtual switch Hardware Requirements: UltraSPARC T2 based system 10Gb ethernet XAUI adapter (nxge interface) 15
  16. 16. LDoms Product Engineering Network Hybrid I/O 1/3 Physical Network XAUI Adapter Guest Domain Service Domain Guest Domain nxge0 vnet vsw vnet (non-hybrid) (hybrid) LDC Hypervisor A non-hybrid vnet sends/receives all packets through the service domain 16
  17. 17. LDoms Product Engineering Network Hybrid I/O 2/3 Physical Network XAUI Adapter Guest Domain Service Domain Guest Domain nxge0 vnet vsw vnet (non-hybrid) (hybrid) LDC Hypervisor broadcast & multicast A hybrid vnet sends/receives only broadcast and multicast packets through the service domain 17
  18. 18. LDoms Product Engineering Network Hybrid I/O 3/3 Physical Network XAUI Adapter Guest Domain Service Domain Guest Domain nxge0 vnet vsw vnet (non-hybrid) (hybrid) LDC A hybrid vnet sends/receives unicast packets directly to/from the NIU card using dedicated DMA channels 18
  19. 19. LDoms Product Engineering VLAN (802.1q) Support ● Add VLAN Support to Virtual Network I/O ● Ethernet packets switching based on VLAN IDs ● Support Added to vsw and vnet ● vnet and vsw can now service multiple subnets ● Features Similar to a Physical Switch with VLAN ● Support untagged and tagged mode ● VLAN IDs are Assigned with the ldm CLI 19
  20. 20. LDoms Product Engineering What's New in LDOM 1.2 ● Support for jumbo frames ● Support for configuring domain dependencies ● Support for auto recovery of configurations ● Support for export of same backend multiple times ● Support for physical-to-virtual migration tool ● Support for configuration assistant tools 20
  21. 21. LDoms Product Engineering Jumbo Frames ● The Logical Domains virtual switch (vsw) and virtual network (vnet) devices can now support Ethernet frames with payload sizes larger than 1500 bytes. ● This change results in these drivers being able to increase network throughput. # ldm add-vsw net-dev=e1000g0 mtu=9000 primary-vsw0 primary # ldm add-vnet vnet01 primary-vsw0 ldom1 # ifconfig vsw0 plumb # ifconfig vsw0 192.168.1.100/24 up 21
  22. 22. LDoms Product Engineering Domain Dependencies ● You can use the Logical Domains Manager to establish dependency relationships between domains. ● Domain that has one or more domains that depend on it is called a master domain. ● Domain that depends on another domain is called a slave domain. 22
  23. 23. LDoms Product Engineering Autorecovery of configurations ● Copy of the current configuration is automatically saved on the control domain whenever the Logical Domains configuration is changed. ● The autosave operation occurs immediately, even in the following situations: · When the new configuration is not explicitly saved on the SP · When the actual configuration change is not made until the affected domain reboots 23
  24. 24. LDoms Product Engineering Export a Virtual Disk Backend Multiple Times A virtual disk backend can be exported multiple times either through the same or different virtual disk servers. ● Can then be assigned to either the same or different guest domains. ● It should not be exported with the exclusive (excl) option. Specifying the excl option will only allow exporting the backend once ● The backend can be safely exported multiple times as a read-only device with the ro option. 24
  25. 25. LDoms Product Engineering Domain Mobility •Cold and Live migration can migrate between different system and CPU types •Warm Migration requires same system and CPU type •Cold Migration operation is fast •Live Migration requires OS support (aka cooperative guest support) •Time to migrate a domain is largely determined by >Type of migration being performed >Network speed >Size of guest image (Warm Migration) 25
  26. 26. LDoms Product Engineering Warm Migration 1/6 - Initialization virtual disk backend (NFS file or shared disk) System A System B Guest Control Control Domain Domain Domain vdsk vds vds vnet vsw vsw Physical Network ldmd ldmd Memory Memory Memory CPUs CPUs CPUs ldmd A check with ldmd B if migration is possible 26
  27. 27. LDoms Product Engineering Warm Migration 2/6 – New Guest Creation virtual disk backend (NFS file or shared disk) System A System B Guest Control Control Guest Domain Domain Domain Domain vdsk vds vds vdsk vnet vsw vsw vnet Physical Network ldmd ldmd Memory Memory Memory Memory CPUs CPUs CPUs 1 CPU ldmd B creates and binds a similar domain with 1 CPU 27
  28. 28. LDoms Product Engineering Warm Migration 3/6 – Shrink Source Guest virtual disk backend (NFS file or shared disk) System A System B Guest Control Control Guest Domain Domain Domain Domain vdsk vds vds vdsk vnet vsw vsw vnet Physical Network ldmd ldmd Memory Memory Memory Memory 1 CPU CPUs CPUs 1 CPU ldmd A removes all CPUs but one on the source guest 28
  29. 29. LDoms Product Engineering Warm Migration 4/6 – State Transfer virtual disk backend (NFS file or shared disk) System A System B Guest Control Control Guest Domain Domain Domain Domain vdsk vds vds vdsk vnet vsw vsw vnet Physical Network ldmd ldmd transfer memory Memory Memory and Memory Memory 1 CPU CPUs cpu state CPUs 1 CPU ldmd A suspends the last CPU and transfers state 29
  30. 30. LDoms Product Engineering Warm Migration 5/6 – Target Guest Resume virtual disk backend (NFS file or shared disk) System A System B Guest Control Control Guest Domain Domain Domain Domain vdsk vds vds vdsk vnet vsw vsw vnet Physical Network ldmd ldmd Memory Memory Memory Memory 1 CPU CPUs CPUs 1 CPU ldmd B resumes the target guest with one cpu 30
  31. 31. LDoms Product Engineering Warm Migration 6/6 – Completion & Cleanup virtual disk backend (NFS file or shared disk) System A System B Control Control Guest Domain Domain Domain vds vds vdsk vsw vsw vnet Physical Network ldmd ldmd Memory Memory Memory CPUs CPUs CPUs ldmd B add other cpus – ldmd A destroy the source guest 31
  32. 32. LDoms Product Engineering Physical-to-Virtual Migration Tool ( P2V) The Logical Domains P2V Migration Tool automatically converts an existing physical system to a virtual system that runs in a logical domain on a chip multithreading (CMT) system. The source system can be any of the following: · Any sun4u SPARC system that runs at least the Solaris 8 Operating System · Any sun4v system that runs the Solaris 10 OS, but does not run in a logical domain 32
  33. 33. LDoms Product Engineering Logical Domains Physical-to-Virtual Migration Tool cont.. The conversion from a physical system to a virtual system is performed in the following phases: · Collection phase. Runs on the physical source system. collect creates a file system image of the source system based on the configuration information that it collects about the source system. · Preparation phase. Runs on the control domain of the target system. prepare creates the logical domain on the target system based on the configuration information collected in the collect phase. · Conversion phase. Runs on the control domain of the target system. In the convert phase, the created logical domain is converted into a logical domain that runs the Solaris 10 OS by using the standard Solaris upgrade process. 33
  34. 34. LDoms Product Engineering Logical Domains Configuration Assistant ● The Logical Domains Configuration Assistant leads you through the configuration of a logical domain by setting basic properties. ● After gathering the configuration data, the Configuration Assistant creates a configuration that is suitable for booting as a logical domain. ● The Configuration Assistant is available as both a graphical user interface (GUI) and terminal-based tool, ldmconfig. 34
  35. 35. LDoms Product Engineering Ops Center Solution Set Comprehensive management for physical and virtual systems CLI BUI API Asset Management Virtualization Provisioning Patching Monitoring Management Management Policy Management Resource Management Zone Zone Zones Zone Zone Zone Zone Operating System Solaris Solaris Linux Windows Solaris Solaris Virtual Machine LDom1 LDom2 Hypervisor Logical Domain Physical Server SPARC Server SPARC Server X86 Server X86 Server X86 Server Page 35
  36. 36. LDoms Product Engineering Virtualization Management For Solaris Zones and Logical Domains Life-cycle mgt: create, delete, configure, boot, shutdown, etc. Monitoring: CPU, memory, file systems, historical data, aggregate utilization LDom resource pools: policies, auto restart Migration: Warm migration (LDoms), cold migration (Zones) Page 36
  37. 37. LDoms Product Engineering LDoms vs Containers Feature LDoms Containers Platform choice T series X M and T series Cost Free to download Built in Solaris Better isolation via multiple Security service domain Resource Flexibility CPU ,Memory ,I/O OS Flexibility Different kernel Branded Zone ( 8,9 ,Linux ) Scalability Hardware limit Hardware limit Performance Single kernel Redundancy Multi service domain Simplicity Don't need to install pkg Migration Warm /Cold Cold only P2V Solaris 8,9,10 Solaris 8,9 10 (plate spin) Management Ops center 2.5 Ops center 2.5 37
  38. 38. LDoms Product Engineering LDoms Engineering Roadmap 38
  39. 39. LDoms Product Engineering Competitive Differentiators ● No-Cost, Built-in Virtualization ● LDoms redundancy mechanism enables your domains to be more resilient than competing technologies. ● Other Hypervisors adds more substantial performance hit compared to LDoms. ● Investment Protection ● You can run most older applications ( Solaris 8 & 9 ) unmodified on a new CMT system. 39
  40. 40. Thank you! http://sun.com/ldoms http://blogs.sun.com/vreality

×