Hello! <ul><ul><li>Visual comparison of Unix-like systems
Virtualisation </li></ul></ul>
Primary Subsystems <ul><ul><li>Process Management </li></ul></ul><ul><ul><ul><li>Manages all active processes and threads ...
Performs translations of physical memory locations to virtual memory locations </li></ul><li>File Subsystem </li><ul><li>V...
Drives system configuration process </li></ul></ul></ul>
Terms <ul><ul><li>Process : A UNIX process includes the user’s address space, stack, and process control block.
User-level threads : Implemented through a threads library in the address space of a process, these threads are invisible ...
Lightweight processes : A lightweight process (LWP) can be viewed as a mapping between user level threads and kernel threa...
Kernel threads : These are the fundamental entities that can be scheduled and dispatched to run on one of the system proce...
Kernel Entry Hardware System call interface lightweight syscalls User Level Kernel Level I / O Process Management Subsyste...
UNIX v3 304KB
OpenSolaris kernel 226.1MB
System Call Interface . .. Virtual File System Framework Kernel Services Clocks & Timers etc Hardware – SPARC / x86 / x64 ...
HP-UX kernel 68.4MB
HP-UX big picture Kernel Threads VAS Pregion Pregion UAREA user structure kernel stack File Descriptor Table VFS Inode Cac...
FreeBSD kernel 368.2MB
XNU <ul><ul><li>Basis of Mac OS X, iPhone, etc. Made up of three main parts:
Mach 3.0 kernel code </li><ul><li>kernel threads, virtual memory management, inter-process communication, interrupt manage...
XNU kernel 41.5MB
XNU kernel User User libraries libc BSD I/O Kit Mach Platform Expert Xnu (kernel)
Microkernel architecture Hardware Hardware Accounts File Systems Networking Devices Virtual Memory Process Mgmt Virtual Me...
VMS (virtual memory) Solaris DEC OSF/1 -> DIGITAL UNIX HP-UX Tru64 influences System III ULTRIX Mach AIX XENIX OSF/1 Tru64...
Mach in Tru64 Process Mgmt Memory Mgmt UNIX Mach Process Mgmt Memory Mgmt IPC Mach System Calls by Select System Services ...
Tru64 kernel 109MB
Tru64 kernel Events and Interrupts I/O Subsystem vm_object Special Files Mount Table nameidata Files Buffer Cache proc uta...
AdvFS <ul><ul><li>http://advfs.sourceforge.net </li><ul><li>Code drop of AdvFS </li></ul></ul></ul>
Linux kernel 413.4MB
Linux kernel Process Kernel modules Audit IPC System Call interface Devices SELinux Memory File, I/O Networking
 
Tasks <ul><ul><li>A process, or task, in Linux is represented by a task_struct data structure. The task_struct data struct...
Most services are implemented in the kernel, with the exception of many networking functions. Thus Linux kernel is relativ...
Linux provides a unique solution in that it does not recognize a distinction between threads and processes. Using a mechan...
Context of a Process Process structure Shared  memory Heap space Data Text Virtual  Address  Space
Context of a task Attributes Instructions Private storage Thread structure Thread stack User structure  with register  con...
Plan9 kernel 8.9MB
Threading models Hardware   Kernel Space Kernel Threads User Space kernel scheduler kernel scheduler kernel scheduler KT K...
Scheduling lowest highest longest processor priority waiting scheduler
Context Switches <ul><ul><li>Save register context of current thread. </li></ul></ul><ul><ul><li>Restore register context ...
Thread Scheduling Contention Scopes Userspace  scheduler in  pthread library User Space Kernel Space &quot;unbound&quot; w...
Scheduler queues CPU0   CPU1   CPU2   CPU3   per-processor run queues
Books
<ul><ul><li>A process, or task, in Linux is represented by a task_struct data structure. The task_struct data structure co...
Most services are implemented in the kernel, with the exception of many networking functions. Thus Linux kernel is relativ...
Linux provides a unique solution in that it does not recognize a distinction between threads and processes. Using a mechan...
How Server Virtualization Works <ul><ul><li>Strong isolation between VMs
Dynamic resizing & migration
Rapid VM instantiation & cloning
Integrated workload management </li></ul></ul>OEL RHEL MS Physical Server Physical CPUs Hypervisor/VMM Virtual machine mon...
Basic Virtualization
 
KVM Architecture   Native   Linux kernel is the VMM (kvm module loaded) Xen is different. Xen copy code from linux kernel ...
Intel and AMD Virtualization technologies <ul><ul><li>Intel: </li></ul></ul><ul><ul><ul><li>VT-x (IA-32)
Upcoming SlideShare
Loading in...5
×

Visual comparison of Unix-like systems & Virtualisation

4,057

Published on

This is a presentation on SFD 2010 of Dalian China by Mark Smith who is lead, Unix & Open Source Professions.

0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,057
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
0
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide
  • There are four primary subsystems covered in this course, which are responsible for the majority of the kernel activity. Process Management The Process Management Subsystem is responsible for managing all of the running processes and threads on the system, maintaining information about process and thread resources, and scheduling threads on the CPU or CPUs. Memory Management Memory is managed by the kernel in a way that allows each process to have its own virtual address space. The Memory Management Subsystem creates an appearance that the sum of process address space is greater than the amount of physical memory in the system. This is the theory behind the virtual address space concept. File Systems The File Subsystem is responsible for managing data on various types of disks. The File Subsystem has provisions to manage different types of file system disk layouts. The ability to do this comes from a level of abstraction known as the vnode layer.
  • This slide shows the &amp;quot;Big Picture&amp;quot; of some of the primary kernel data structures. Previous slides have already shown how the process management and memory structures are linked together through the VAS and pregions. In this diagram, we also see how the process management structures are linked to the file system structures through the File Descriptor Table. Each entry in this table is mapped to an entry in the System File Table. We can complete the picture of the process by using the information contained in the System File Table entries. Each of these structures will be discussed in greater detail in later parts of this course.
  • The program code The global variables used by the program Contains process- specific information such as the process&apos; state, its threads, signals, size, scheduling priority Address range into which the process context is logically mapped Available range of addresses stored in the virtual address space; can be dynamically allocated by process for private data storage Available range of addresses stored in the virtual address space; can be allocated by process, populated with data, then shared with other processes virtual address space heap space shared memory
  • The program code specifically for this thread to execute (e.g., a function) Define thread-specific characteristics Contains thread-specific information such as the scheduling priority, states, signals, and cpu usage. Contains a program counter user structure with register context private storage Area for data; comparable to process heap space, except that unlike the process heap space, thread&apos;s private storage is not shared with other process threads Stack space used by thread to call functions
  • Transcript of "Visual comparison of Unix-like systems & Virtualisation"

    1. 1. Hello! <ul><ul><li>Visual comparison of Unix-like systems
    2. 2. Virtualisation </li></ul></ul>
    3. 3. Primary Subsystems <ul><ul><li>Process Management </li></ul></ul><ul><ul><ul><li>Manages all active processes and threads </li></ul><li>Memory Management </li><ul><li>Allocates memory for processes at run-time
    4. 4. Performs translations of physical memory locations to virtual memory locations </li></ul><li>File Subsystem </li><ul><li>Virtual File System Layer </li></ul></ul></ul><ul><ul><ul><ul><li>Abstracts and manages data for different file system types </li></ul><li>Physical File System Layer </li><ul><li>Manages access for specific file systems of various types </li></ul></ul><li>I/O Subsystem </li><ul><li>Manages system I/O resources, drivers, and data structures
    5. 5. Drives system configuration process </li></ul></ul></ul>
    6. 6. Terms <ul><ul><li>Process : A UNIX process includes the user’s address space, stack, and process control block.
    7. 7. User-level threads : Implemented through a threads library in the address space of a process, these threads are invisible to the OS. A user-level thread (ULT) is a user-created unit of execution within a process.
    8. 8. Lightweight processes : A lightweight process (LWP) can be viewed as a mapping between user level threads and kernel threads. Each LWP supports ULT and maps to one kernel thread. LWPs are scheduled by the kernel.
    9. 9. Kernel threads : These are the fundamental entities that can be scheduled and dispatched to run on one of the system processors. </li></ul></ul>
    10. 10. Kernel Entry Hardware System call interface lightweight syscalls User Level Kernel Level I / O Process Management Subsystem Interpreter User program Libraries Virtual File System Filesystem File Cache LVM device drivers IPC Scheduling Hardware control Memory Management Subsystem
    11. 11. UNIX v3 304KB
    12. 12. OpenSolaris kernel 226.1MB
    13. 13. System Call Interface . .. Virtual File System Framework Kernel Services Clocks & Timers etc Hardware – SPARC / x86 / x64 Hardware Address Translation (HAT) Network ing framework& services . .. Bus & Nexus Drivers sd ssd Processes & Threads t i m e s h a r e Scheduler i n t e r a c t i v e r e a l t i m e f a i r s h a r e f i x e d p r i o r i t y ZFS NFS ProcFS SpecFS TCP/IP Resource managem ent & cont rols Memory Management Virtual Memory Kernel Memory Allocation Resource management & controls Solaris kernel
    14. 14. HP-UX kernel 68.4MB
    15. 15. HP-UX big picture Kernel Threads VAS Pregion Pregion UAREA user structure kernel stack File Descriptor Table VFS Inode Cache System File Table UAREA user structure kernel stack Pregion Pregion fstore Arena Data Structures VFD DBD bstore File cache Process structures Region vnode vnode FS private data kthread kthread kthread kthread proc_t proc_t proc_t proc_t Memory
    16. 16. FreeBSD kernel 368.2MB
    17. 17. XNU <ul><ul><li>Basis of Mac OS X, iPhone, etc. Made up of three main parts:
    18. 18. Mach 3.0 kernel code </li><ul><li>kernel threads, virtual memory management, inter-process communication, interrupt management, real-time support, console I/O </li></ul><li>FreeBSD 5.x kernel code </li><ul><li>Process model, Permissions, System calls, Sockets, VFS, System V IPC </li></ul><li>I/O Kit </li><ul><li>Device driver model </li></ul></ul></ul>
    19. 19. XNU kernel 41.5MB
    20. 20. XNU kernel User User libraries libc BSD I/O Kit Mach Platform Expert Xnu (kernel)
    21. 21. Microkernel architecture Hardware Hardware Accounts File Systems Networking Devices Virtual Memory Process Mgmt Virtual Memory Process Mgmt IPC Kernel Microkernel User-Space Servers Applications Applications Traditional (Monolithic) Kernel Microkernel UNIX
    22. 22. VMS (virtual memory) Solaris DEC OSF/1 -> DIGITAL UNIX HP-UX Tru64 influences System III ULTRIX Mach AIX XENIX OSF/1 Tru64 UNIX SCO UNIX (System V) Tandem NonStop-UX, Integrity XC SunOS UNIX (Bell Labs) SCO XENIX System V BSD
    23. 23. Mach in Tru64 Process Mgmt Memory Mgmt UNIX Mach Process Mgmt Memory Mgmt IPC Mach System Calls by Select System Services UNIX System Calls by Applications Other Services: File Systems Buffer Caches UNIX IPC and Networking I/O Framework Security and Accounting Etc...
    24. 24. Tru64 kernel 109MB
    25. 25. Tru64 kernel Events and Interrupts I/O Subsystem vm_object Special Files Mount Table nameidata Files Buffer Cache proc utask task thread uthread port Kernel Stack run_queue pcb User Stack wait queue vm_page vm_map vm_map_entry pmap pte Virtual File System Process Virtual Memory System Calls Alpha Architecture Vnode Table File Table Networking Subsystem PALcode
    26. 26. AdvFS <ul><ul><li>http://advfs.sourceforge.net </li><ul><li>Code drop of AdvFS </li></ul></ul></ul>
    27. 27. Linux kernel 413.4MB
    28. 28. Linux kernel Process Kernel modules Audit IPC System Call interface Devices SELinux Memory File, I/O Networking
    29. 30. Tasks <ul><ul><li>A process, or task, in Linux is represented by a task_struct data structure. The task_struct data structure contains information in a number of categories: </li><ul><li>Processes in Linux are both containers and schedulable entities; processes can share address space and system resources, making processes effectively usable as threads.
    30. 31. Most services are implemented in the kernel, with the exception of many networking functions. Thus Linux kernel is relatively big in size.
    31. 32. Linux provides a unique solution in that it does not recognize a distinction between threads and processes. Using a mechanism similar to lightweight processes, user-level threads are mapped into kernel-level processes. Multiple user-level threads that constitute a single user-level process are mapped into Linux kernel-level processes that share the same group ID. This enables these processes to share resources such as files and memory and to avoid the need for a context switch when the scheduler switches among processes in the same group. </li></ul></ul></ul>
    32. 33. Context of a Process Process structure Shared memory Heap space Data Text Virtual Address Space
    33. 34. Context of a task Attributes Instructions Private storage Thread structure Thread stack User structure with register context
    34. 35. Plan9 kernel 8.9MB
    35. 36. Threading models Hardware Kernel Space Kernel Threads User Space kernel scheduler kernel scheduler kernel scheduler KT KT KT KT UT UT UT UT UT scheduler KT UT UT UT scheduler KT UT UT User Process and User Threads CPU CPU CPU CPU CPU KT file management subsystem I/O management subsystem Process & memory mgmt subsystems network management subsystem
    36. 37. Scheduling lowest highest longest processor priority waiting scheduler
    37. 38. Context Switches <ul><ul><li>Save register context of current thread. </li></ul></ul><ul><ul><li>Restore register context of highest priority thread. </li></ul></ul>UAREA processor kthread structure User structure UAREA kthread structure User structure PCB PCB
    38. 39. Thread Scheduling Contention Scopes Userspace scheduler in pthread library User Space Kernel Space &quot;unbound&quot; with userspace scheduler. Here, we &quot;bind&quot; to kernel threads. Bound threads
    39. 40. Scheduler queues CPU0 CPU1 CPU2 CPU3 per-processor run queues
    40. 41. Books
    41. 42. <ul><ul><li>A process, or task, in Linux is represented by a task_struct data structure. The task_struct data structure contains information in a number of categories: </li><ul><li>Processes in Linux are both containers and schedulable entities; processes can share address space and system resources, making processes effectively usable as threads.
    42. 43. Most services are implemented in the kernel, with the exception of many networking functions. Thus Linux kernel is relatively big in size.
    43. 44. Linux provides a unique solution in that it does not recognize a distinction between threads and processes. Using a mechanism similar to lightweight processes, user-level threads are mapped into kernel-level processes. Multiple user-level threads that constitute a single user-level process are mapped into Linux kernel-level processes that share the same group ID. This enables these processes to share resources such as files and memory and to avoid the need for a context switch when the scheduler switches among processes in the same group. </li></ul></ul></ul>Virtualisation
    44. 45. How Server Virtualization Works <ul><ul><li>Strong isolation between VMs
    45. 46. Dynamic resizing & migration
    46. 47. Rapid VM instantiation & cloning
    47. 48. Integrated workload management </li></ul></ul>OEL RHEL MS Physical Server Physical CPUs Hypervisor/VMM Virtual machine monitor OS Virtual Machines Physical Server Hypervisor/VMM Virtual machine monitor OS Physical Server Hypervisor/VMM OS Physical I/O Physical memory File definitions Oracle Database 11g Oracle Database 10g R2 Oracle Database 10g R1
    48. 49. Basic Virtualization
    49. 51. KVM Architecture Native Linux kernel is the VMM (kvm module loaded) Xen is different. Xen copy code from linux kernel ; became a VMM itself ; the real linux kernel is going to &quot;sleep&quot; and xen kernel do everything by itself (a lot of maintenance work !! When new linux kernel release, xen need port a lot code ) Guest mode ,each vm looks as a normal linux process
    50. 52. Intel and AMD Virtualization technologies <ul><ul><li>Intel: </li></ul></ul><ul><ul><ul><li>VT-x (IA-32)
    51. 53. VT-i (IA-64)
    52. 54. VT-d
    53. 55. VT-C </li></ul><li>AMD: </li><ul><li>AMD-V codename &quot;Pacifica&quot; </li></ul></ul></ul>Virtualization Layer Expand or Contract Virtual Processors Virtual Memory Virtual I/O Storage Virtual Machine 1 Expand or Contract Physical System Guest OS 1 Applications
    54. 56. Intel Virtualization Technologies <ul><ul><li>VT-x (IA-32) and VT-i (IA-64) are Intel virtualization technologies : </li></ul></ul><ul><ul><ul><li>Implemented at CPU level </li></ul><li>VT-d is Intel technology for directed I/O </li><ul><li>Implemented at chipset level
    55. 57. Support both VT-x and VT-I </li></ul><li>VT-C (connectivity) </li><ul><li>Implemented on I/O cards
    56. 58. Reduce CPU activity </li></ul></ul></ul>
    57. 59. virtio
    58. 60. Before
    59. 61. After
    60. 62. Xen overview <ul><ul><li>Fully virtualized </li></ul></ul><ul><ul><ul><li>Requires no modifications to the operating system
    61. 63. Requires trapping and emulating all privileged instructions
    62. 64. Intel and AMD are doing hardware virtualization </li></ul></ul></ul><ul><ul><ul><ul><li>Support for virtualization of CPU, MMU, and memory
    63. 65. Allows virtualization of unmodified guests
    64. 66. I/O performance might still be an issue </li></ul></ul></ul></ul><ul><ul><li>Xen 3.0 is a hypervisor (virtual machine monitor) </li></ul></ul><ul><ul><ul><li>Creates and launches virtual machines
    65. 67. Keeps machines isolated
    66. 68. Provides memory management and CPU scheduling
    67. 69. I/O interrupt processing
    68. 70. Event and Memory Channel between domains </li></ul></ul></ul>
    69. 71. Xen 3.0 architecture
    70. 72. VMware <ul><ul><li>Virtualization leader </li></ul></ul><ul><ul><ul><li>Efficient proprietary hypervisors
    71. 73. Mature management tools and eco-system </li></ul><li>x86 Standard focus </li></ul></ul>HP Proliant USB key Hypervisor VM VM
    72. 74. Virtual Private Servers (VPS) Managers <ul><ul><li>Single instance of </li></ul></ul><ul><ul><ul><li>OS Kernel </li></ul><li>Unique per workload </li><ul><li>File System
    73. 75. User Database
    74. 76. IP addresses
    75. 77. Process namespace </li></ul></ul></ul>Hardware Operating System Workload 1 Workload 2 Workload 3
    76. 78. Classification of Virtual Machine Monitors (by implementation) <ul><ul><li>Hosted virtual machines </li></ul></ul><ul><ul><ul><li>VMM runs in user mode
    77. 79. Virtual machines run as processes under the host OS
    78. 80. All device access performed by host OS </li></ul><li>Example: VMware server (former GSX), HP IVM </li></ul></ul>Hardware Host OS VMM Guest OS 1 Guest OS 2 App App App App App
    79. 81. Classification of Virtual Machine Monitors (by implementation) <ul><ul><li>Unhosted virtual machines </li></ul></ul><ul><ul><ul><li>VMM is part of the host kernel </li></ul></ul></ul><ul><ul><ul><ul><li>Boots on native hardware </li></ul><li>Device drivers needed for all devices </li></ul><li>Ex: Xen, Vmware ESX </li></ul></ul>Hardware VMM Guest OS 1 Guest OS 3 Guest OS 2 App App App App App App
    80. 82. Some Common Virtual Machine Environments User Mode Linux (UML) Unhosted Hosted VMware ESX Hyper-V VMware Server/WS MS Virtual PC HP IVM, KVM Fully virtualized Xen Para-virtualized
    81. 83. Managing virtual machines Several options on Fedora/RHEL, Ubuntu, etc. to manage virtual machines: <ul><ul><li>virsh
    82. 84. xend
    83. 85. xm
    84. 86. virt-manager </li></ul></ul>
    85. 87. Thank you
    86. 88. Hyper-threading Hardware Layer Kernel Layer User Layer H/W Threading Off H/W Threading On LCPUs disabled H/W Threading On LCPUs enabled CPU CPU CPU CPU CPU Core Core Core HT HT HT HT LCPU LCPU LCPU LCPU

    ×