Your SlideShare is downloading. ×
0
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Ashish pandey huawei osi_days2011_cgroups_understanding_better
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Ashish pandey huawei osi_days2011_cgroups_understanding_better

765

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
765
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  1. Security Level:Cgroups:Understanding better!Ashish Pandey, Liuyang, Roopesh Keeppattu, Sanil Kumar D. www.huawei.com OSI Days, November, 2011, Bangalore HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential
  2. Agenda •Setting the context… - Background and Cgroups Definition - Significance of Cgroups • Cgroups: Deeper View - Features and Design - Cgroups Subsystems - Cgroups Implementation •Cgroups Use cases - Usage Illustration - Sample Use cases •Cgroups : Challenges and Future - Key Challenges - FutureHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 2
  3. Background  Clustering and Aggregation of Resources help in Fairness in Large Systems  Predictability in Performance depends on the predictability in the availability of resource Isolation and Aggregation of Resources for specific activities can improve the predictability of the performance for the specific activities. This also helps in Resource Tracking, Control and Management  Examples : Cloud, Virtualization, Containers Clou dHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 3
  4. Cgroups Definition Cgroups helps for resource isolation and resource control Cgroups: Process Cgroups are like process container or Container process groups in Linux kernel Provide a mechanism for aggregating / partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behaviour P P Associate a set of tasks with a set of P P P P P P P parameters for one or more subsystems Cgroups help tuning the system to make best use of available hardware and network resources.HUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 4
  5. Significance of Cgroups  As the systems are becoming more complex with dynamic resource needs, better resource management is inevitable to get optimized outout.  Resource planning is better than dynamic resource allocation  Cgroups with its tree-structured hierarchy and subsystems achieves resource planning for the native system Mainly 2 types of Subsystems supported: •Isolation and Special Controllers : cpuset, namespace, freezer, device, checkpoint/restart •Resource Controllers : cpu(scheduler), memory, disk I/O, network The significance of Cgroups is that it helps to implement the essential resource groups hierarchically in kernel to enable better resource management. Every task running in the system is in exactly one of the Cgroups in this hierarchyHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 5
  6. Agenda •Setting the context… - Background and Cgroups Definition - Significance of Cgroups • Cgroups: Deeper View - Features and Design - Cgroups Subsystems - Cgroups Implementation •Cgroups Use cases - Usage Illustration - Sample Use cases •Cgroups : Challenges and Future - Key Challenges - FutureHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 6
  7. Features and Design Resource Isolation Resource Control Resource Tracking • Isolate the resources for • Controls the resource for • Resource Tracking to specific similar task groups usage limits, freezing measure how much • Separate Cpuset, Namespace, groups or check pointing resources certain systems Device for groups and restarting use for e.g. billing purposes • Freezer, Checkpoint / • State of the resource is Restart, Cpuacct, CPU, tracked Memory, Disk I/O and Network I/O Hierarchical based grouping of similar tasks in terms of resources is the design principle behind CgroupsHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 7
  8. Cgroups Subsystems  Modules which makes use of task grouping facilities provided by Cgroups to treat groups of tasks in particular way  Resource controllers allocating different levels of system resources to different Cgroups.HUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 8
  9. Cgroups Subsystems Details Subsystem Type Description cpuset Isolation Use cpuset to tie processes to system subsets of CPU For showing private view of system to processes in cgroup; ns Isolation mainly used for OS level virtualization cpu Control Shares CPU bandwidth between groups cpuacct Accounting Groups tasks using cgroup and accounts the CPU usage memory Control Limits memory and memory related parameters devices Isolation Decides the access to devices by tasks in cgroup freezer Control Freezes/unfreezes tasks in a group. Used for HPC clusters Identification of packets originating from a specific cgroup and net_cls Control actions on the packet. controls and monitors access to I/O on block devices by tasks in blk_io Control cgroupHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 9
  10. Cgroups Implementation  VFS based filesystem model implementation  Each task in the system has a reference-counted pointer to a css_set  A css_set contains a set of reference-counted pointers to cgroup_subsys_state objects, one for each cgroup subsystem registered in the system  A cgroup hierarchy filesystem can be mounted for browsing and manipulation from user space  You can list all the tasks (by pid) attached to any cgroup [cat /cgroup/tasks OR cat /cgroup/<group>/tasks]  Files: • kernel/cgroup.c • include/linux/cgroup.h • include/linux/cgroup_subsys.h • +various subsystems implementations  cgroups requires a few, simple hooks into the rest of the kernel, these hooks are not in performance critical paths of the kernel. • In init/main.c : To initialize C groups at boot time. • In fork and exit calls : To attach and detach a task from cgroups A new file system, of type "cgroup" should be mounted, to enable browsing and modifying the all the registered Cgroups subsystemHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 10
  11. Cgroups : How to write a new subsystem APIs to be implemented by a new subsystemMajor modifications to be done: •init/Kconfig: kernel configuration parameters (general setup) •include/linux/cgroup_subsys.h: cgroup subsystem definition •kernel/cgroup_example.c: cgroup subsystem implementation [Implement Kernel APIs as per cgroup_subsys] •kernel/Makefile: Makefile of the core kernel components •Add the appropriate hooks into the kernel As Cgroups provide clean kernel APIs , new subsystem addition and management are simplerHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 11
  12. Agenda •Setting the context… - Background and Cgroups Definition - Significance of Cgroups • Cgroups: Deeper View - Features and Design - Cgroups Subsystems - Cgroups Implementation •Cgroups Use cases - Usage Illustration - Sample Use cases •Cgroups : Challenges and Future - Key Challenges - FutureHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 12
  13. Usage Illustration mount -t tmpfs none /sys/fs/cgroup mount -t cgroup -ocpuset cpuset mkdir /sys/fs/cgroup/memory /sys/fs/cgroup/cpuset mount -t cgroup none /sys/fs/cgroup/memory -o cd /sys/fs/cgroup/cpuset memory mkdir Charlie #Make the new group and move bash into it cd Charlie mkdir /sys/fs/cgroup/memory/0 /bin/echo 2-3 > cpuset.cpus echo $$ > /sys/fs/cgroup/memory/0/tasks /bin/echo 1 > cpuset.mems #Since now were in the 0 cgroup, we can alter the /bin/echo $$ > tasks memory limit: sh echo 4M > # The subshell sh is now running in /sys/fs/cgroup/memory/0/memory.limit_in_bytes cpuset Charlie cat /sys/fs/cgroup/memory/0/memory.limit_in_bytes # The next line should display 4194304 /Charlie #We can check the usage: cat /proc/self/cpuset cat /sys/fs/cgroup/memory/0/memory.usage_in_bytes 1216512HUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 13
  14. Usage Illustration #An entry is added using mkdir /sys/fs/cgroup/freezer devices.allow, and removed using mount -t cgroup -ofreezer freezer devices.deny. /sys/fs/cgroup/freezer For #instance mkdir /sys/fs/cgroup/freezer/0 echo c 1:3 mr > echo $some_pid > /sys/fs/cgroup/freezer/0/tasks /sys/fs/cgroup/1/devices.allow #to get status of the freezer subsystem : #allows cgroup 1 to read and mknod cat /sys/fs/cgroup/freezer/0/freezer.state the device usually known as dev/null. THAWED #Doing #to freeze all tasks in the container : echo a > echo FROZEN > /sys/fs/cgroup/freezer/0/freezer.state /sys/fs/cgroup/1/devices.deny #at /sys/fs/cgroup/freezer/0/freezer.state #will remove the default a *:* rwm FROZEN entry. Doing #to unfreeze all tasks in the container : echo a > echo THAWED > /sys/fs/cgroup/freezer/0/freezer.state /sys/fs/cgroup/1/devices.allow cat /sys/fs/cgroup/freezer/0/freezer.state #will add the a *:* rwm entry to the THAWED whitelist.HUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 14
  15. Sample Use cases Without CPU cgroup subsystem With CPU cgroup subsystem (10 tasks in “multimedia” and 5 tasks in “browser”) (10 tasks in “multimedia” and 5 tasks in “browser”) Cgroups can provide better resource control and managementHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 15
  16. Sample Use cases (contd.)  A large university server with various users - students, professors, system tasks etc. The resource planning for this server could be along the following lines with cgroups Users Professor Student CPU (0-3) Disk (50%) Memory(50%) N/W CPU((4-7) Disk(50%) Memory(50%) N/W WWW WWW NFS(25%) Others(10%) NFS(25%) Others(10%) browsing(20%) browsing(10%)  Consider a batch job management system which requires start and stop sets of tasks in order to schedule the resources of a machine .This kind of program is often used on HPC clusters to schedule access to the cluster as a whole Batch Processing System Batch Batch Process1 Process2 Can be Cpu(0-4) Disk(50%) Memory(50%) No Freeze Cpu(5-8) Disk(40%) Memory(30%) freezedHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 16
  17. Agenda •Setting the context… - Background and Cgroups Definition - Significance of Cgroups • Cgroups: Deeper View - Features and Design - Cgroups Subsystems - Cgroups Implementation •Cgroups Use cases - Usage Illustration - Sample Use cases •Cgroups : Challenges and Future - Key Challenges - FutureHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 17
  18. Key Challenges  Throughput intensive system, Cgroups may add performance overhead  Its not currently possible to bind a new subsystem to an active cgroup hierarchy, or to unbind a subsystem from an active cgroup hierarchy  Though Cgroups acts like lighter virtualization, the same can be applied only to native system  Large multi-core systems, the linear performance scalability is needed to utilise the resource betterHUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 18
  19. Future…  Throughput optimization  Virtualization Possibilities and Lower Level Resource Abstraction  Tuning and Development for Large Multi-core systems  Multicore clustering support and Specialized Subsystems and …shaping as an inevitable component in linux kernel for effective resource control and management…HUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 19
  20. Ashish Pandey (ashish.pandey@huawei.com) Roopesh Keeppattu (roopesh@huawei.com) Liuyang(liuyang@huawei.com) Sanil Kumar D. (sanil@huawei.com)HUAWEI TECHNOLOGIES CO., LTD. OSI Days Nov 2011 20

×