Your SlideShare is downloading. ×
0
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Presentation 14 09_2012
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Presentation 14 09_2012

429

Published on

Open-Extension Container (OXC) Framework in Linux

Open-Extension Container (OXC) Framework in Linux

Published in: Technology, News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
429
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Design and Development of Real-TimeMulti-Processor Bandwidth Control Mechanisms in General Purpose Operating Systems Youcheng Sun Graduate Program in Computer Science and Engineering September 14, 2012 Y. Sun (GPCSE) Master thesis September 14, 2012 1 / 26
  • 2. Outline1 What is an ideal CPU bandwidth control mechanism for Linux?2 A brief background3 The OXC framework4 Game time!5 Conclusion Y. Sun (GPCSE) Master thesis September 14, 2012 2 / 26
  • 3. Outline1 What is an ideal CPU bandwidth control mechanism for Linux?2 A brief background3 The OXC framework4 Game time!5 Conclusion Y. Sun (GPCSE) Master thesis September 14, 2012 3 / 26
  • 4. Four golden rules Hierarchical CPU bandwidth distribution Tasks in a Linux system can be organized in a flat or hierarchy. In modern systems, the hierarchyical way is preferred. A good mechanism should be able to distribute the CPU bandwidths hierarchically. Serve all kinds of tasks In Linux, there are different types of tasks that scheduled by different schedulers. A good mechanism should be able to serve all kinds of tasks. Support multi-processor platforms Provide real-time guaruntee Linux is used to serve various scenarios, among which some have temporal requirements. A good mechanism should be able to provide real-time guarantee. Y. Sun (GPCSE) Master thesis September 14, 2012 4 / 26
  • 5. Before our work, no existing mechanisms can satisfyall four golden rules Previous work includes RT throttling, CFS bandwdith control, AQuoSA, IRMOS real-time framework, etc. One common failure is rule 2. Y. Sun (GPCSE) Master thesis September 14, 2012 5 / 26
  • 6. Before our work, no existing mechanisms can satisfyall four golden rules Previous work includes RT throttling, CFS bandwdith control, AQuoSA, IRMOS real-time framework, etc. One common failure is rule 2. Each previous mechanism can only work for a specific type of tasks in Linux. Y. Sun (GPCSE) Master thesis September 14, 2012 5 / 26
  • 7. Before our work, no existing mechanisms can satisfyall four golden rules Previous work includes RT throttling, CFS bandwdith control, AQuoSA, IRMOS real-time framework, etc. One common failure is rule 2. Each previous mechanism can only work for a specific type of tasks in Linux.In our work, a framework that satisfies all golden rules isproposed. Y. Sun (GPCSE) Master thesis September 14, 2012 5 / 26
  • 8. Outline1 What is an ideal CPU bandwidth control mechanism for Linux?2 A brief background3 The OXC framework4 Game time!5 Conclusion Y. Sun (GPCSE) Master thesis September 14, 2012 6 / 26
  • 9. Resource reservation The Constant Bandwidth Server (CBS) algorithm is used to reserve CPU bandwidth in the work. A CBS is characterized by an ordered pair (Q, P), where Q is called maximum budget and P is period. A CBS can reserve Q every P time units from a CPU. Y. Sun (GPCSE) Master thesis September 14, 2012 7 / 26
  • 10. Linux schedulingLinux adapts modular scheduling design. Each scheduler is aninstance of the struct sched_class.s t r u c t sched_class { const s t r u c t sched_class ∗n e x t ; void (∗ enqueue_task ) ( s t r u c t r q ∗rq , s t r u c t t a s k _ s t r u c t ∗p , i n t f l a g s ) ; void (∗ dequeue_task ) ( s t r u c t r q ∗rq , s t r u c t t a s k _ s t r u c t ∗p , i n t f l a g s ) ; void (∗ y i e l d _ t a s k ) ( s t r u c t r q ∗r q ) ; bool (∗ y i e l d _ t o _ t a s k ) ( s t r u c t r q ∗rq , s t r u c t t a s k _ s t r u c t ∗p , b o o l preempt ) ; void (∗ check_preempt_curr ) ( s t r u c t r q ∗rq , s t r u c t t a s k _ s t r u c t ∗p , i n t f l a g s ) ; s t r u c t t a s k _ s t r u c t ∗ (∗ p i c k _ n e x t _ t a s k ) ( s t r u c t r q ∗r q ) ; void (∗ p u t _ p r e v _ t a s k ) ( s t r u c t r q ∗rq , s t r u c t t a s k _ s t r u c t ∗p ) ; ...}; These hooks are all scheduling operations for a scheduler. Y. Sun (GPCSE) Master thesis September 14, 2012 8 / 26
  • 11. Linux schedulingLinux adapts modular scheduling design. Each scheduler is aninstance of the struct sched_class.s t r u c t sched_class { const s t r u c t sched_class ∗n e x t ; void (∗ enqueue_task ) ( struct rq *rq , s t r u c t t a s k _ s t r u c t ∗p , i n t f l a g s ) ; void (∗ dequeue_task ) ( struct rq *rq , s t r u c t t a s k _ s t r u c t ∗p , i n t f l a g s ) ; void (∗ y i e l d _ t a s k ) ( struct rq *rq ) ; b o o l (∗ y i e l d _ t o _ t a s k ) ( struct rq *rq , s t r u c t t a s k _ s t r u c t ∗p , b o o l preempt ) ; void (∗ check_preempt_curr ) ( struct rq *rq , s t r u c t t a s k _ s t r u c t ∗p , i n t f l a g s ) ; s t r u c t t a s k _ s t r u c t ∗ (∗ p i c k _ n e x t _ t a s k ) ( struct rq *rq ) ; void (∗ p u t _ p r e v _ t a s k ) ( struct rq *rq , s t r u c t t a s k _ s t r u c t ∗p ) ; ...}; These hooks are all scheduling operations for a scheduler. Y. Sun (GPCSE) Master thesis September 14, 2012 9 / 26
  • 12. The Linux scheduling is runqueue centered Every hook in a scheduling class deals with the struct rq. A struct rq instance is called runqueue in Linux. The struct rq is a per CPU structure. That is, there is a runqueue per CPU in Linux. A scheduling system is built around the runqueue in each CPU. Different per CPU scheduling systems cooperate with each other through task migration strategy defined by schedulers. Y. Sun (GPCSE) Master thesis September 14, 2012 10 / 26
  • 13. The scheme of per CPU scheduling in Linux Y. Sun (GPCSE) Master thesis September 14, 2012 11 / 26
  • 14. Outline1 What is an ideal CPU bandwidth control mechanism for Linux?2 A brief background3 The OXC framework4 Game time!5 Conclusion Y. Sun (GPCSE) Master thesis September 14, 2012 12 / 26
  • 15. The Open Extension Container (OXC) structure The idea of ox container is simple: any data structure that contains a runqueue inside is regarded as the ox container Y. Sun (GPCSE) Master thesis September 14, 2012 13 / 26
  • 16. Pass an ox container’s local runqueue to a schedulingoperation For a scheduler, it needs a runqueue so as to deal with its tasks. This runqueue can be the per CPU runqueue or an ox container’s local runqueue. When an ox container’s local runqueue is used by a scheduler, tasks will run in this local runqueue. Now, besides per CPU scheduling system, there is the per ox container scheduling system in Linux. Y. Sun (GPCSE) Master thesis September 14, 2012 14 / 26
  • 17. Per oxc scheduling + per CPU schedulingcoexist in Linux Y. Sun (GPCSE) Master thesis September 14, 2012 15 / 26
  • 18. The oxc scheduling can be structured in a hierarchy Y. Sun (GPCSE) Master thesis September 14, 2012 16 / 26
  • 19. To allocate CPU bandwidth for an ox container Different schedulers can use an ox container to enqueue, operate and dequeue their tasks. Suppose a fraction of CPU bandiwdth is allocated to an ox container, all kinds of tasks inside the container will use it as if running on a less powerful CPU. Y. Sun (GPCSE) Master thesis September 14, 2012 17 / 26
  • 20. To allocate CPU bandwidth for an ox container Different schedulers can use an ox container to enqueue, operate and dequeue their tasks. Suppose a fraction of CPU bandiwdth is allocated to an ox container, all kinds of tasks inside the container will use it as if running on a less powerful CPU.Based on this idea, the oxc (scheduling) framework is developed tocontrol CPU bandwidth allocated to tasks, which can be any type, ina real-time way. Y. Sun (GPCSE) Master thesis September 14, 2012 17 / 26
  • 21. A concrete ox container that can reserve CPUbandwidth This an ox container that can reserve bandwidth from a CPU according to CBS rules. oxc_runtime and oxc_period are the maximum budget and pe- s t r u c t oxc_rq { riod parameters in CBS. unsigned long o x c _ n r _ r u n n i n g ; int oxc_throttled ; u64 oxc_runtime ; ktime_t oxc_period ; u64 o x c _ d e a d l i n e ; u64 oxc_time ; u64 o x c _ s t a r t _ t i m e ; struct hrtimer oxc_period_timer ; raw_spinlock_t oxc_runtime_lock ; b o o l oxc_needs_resync ; s t r u c t r q rq_ ; s t r u c t r q ∗r q ; s t r u c t rb_node rb_node ; }; Y. Sun (GPCSE) Master thesis September 14, 2012 18 / 26
  • 22. A concrete ox container that can reserve CPUbandwidth This an ox container that can reserve bandwidth from a CPU oxc_runtime and oxc_period are the maximum budget and pe- according to CBS rules. riod parameters in CBS. s t r u c t oxc_rq { unsigned long o x c _ n r _ r u n n i n g ; int oxc_throttled ; u64 oxc_runtime ; ktime_t oxc_period ; u64 o x c _ d e a d l i n e ; u64 oxc_time ; u64 o x c _ s t a r t _ t i m e ; struct hrtimer oxc_period_timer ; raw_spinlock_t oxc_runtime_lock ; b o o l oxc_needs_resync ; s t r u c t r q rq_ ; the local runqueue of an ox con- s t r u c t r q ∗r q ; s t r u c t rb_node rb_node ; tainer }; Y. Sun (GPCSE) Master thesis September 14, 2012 18 / 26
  • 23. SMP support An ox container can only reserve CPU bandwidth from a CPU. cpus_allowed specifies the CPU that bandwidths are reserved from. The hyper ox container structure is defined for reserving bandwidths from multiple processors. s t r u c t hyper_oxc_rq { cpumask_var_t cpus_allowed ; s t r u c t oxc_rq ∗∗ oxc_rq ; }; Y. Sun (GPCSE) Master thesis September 14, 2012 19 / 26
  • 24. SMP support An ox container can only reserve CPU bandwidth from a cpus_allowed specifies the CPU that bandwidths are reserved from. CPU. The hyper ox container structure is defined for reserving bandwidths from multiple processors. s t r u c t hyper_oxc_rq { The ox containers to reserve band- cpumask_var_t cpus_allowed ; widths from corresponding CPUs s t r u c t oxc_rq ∗∗ oxc_rq ; }; Y. Sun (GPCSE) Master thesis September 14, 2012 19 / 26
  • 25. Outline1 What is an ideal CPU bandwidth control mechanism for Linux?2 A brief background3 The OXC framework4 Game time!5 Conclusion Y. Sun (GPCSE) Master thesis September 14, 2012 20 / 26
  • 26. Experiment set-upsCandidates : RT throttling : CPU bandwidth control work, without real-time guarantee, for RT tasks. CFS bandwidth control : CPU bandwidth control work, without real-time guarantee, for CFS tasks. oxc control : Our CPU bandwidth control work, with real-time guarantee, for any type of tasks.Experiment rules: Two tbench connections will be set up in the system; each connection is dedicated to one CPU in a dual processor platform. The CPU bandwidth allocated to tbench traffic is restricted using the above work individually. Y. Sun (GPCSE) Master thesis September 14, 2012 21 / 26
  • 27. Results: oxc control vs. RT throttling Y. Sun (GPCSE) Master thesis September 14, 2012 22 / 26
  • 28. Results: oxc control vs. CFS bandwidth control Y. Sun (GPCSE) Master thesis September 14, 2012 23 / 26
  • 29. Results Given that our work is still a prototype, such results are very encouraging. Y. Sun (GPCSE) Master thesis September 14, 2012 24 / 26
  • 30. Results Given that our work is still a prototype, such results are very encouraging. In addition: we can provide real-time gurantee ;-) Y. Sun (GPCSE) Master thesis September 14, 2012 24 / 26
  • 31. Results Given that our work is still a prototype, such results are very encouraging. In addition: we can provide real-time gurantee ;-)What I am really trying to sell :Our one-for-all solution for CPU bandwidth control in Linuxis feasible!!! Y. Sun (GPCSE) Master thesis September 14, 2012 24 / 26
  • 32. Outline1 What is an ideal CPU bandwidth control mechanism for Linux?2 A brief background3 The OXC framework4 Game time!5 Conclusion Y. Sun (GPCSE) Master thesis September 14, 2012 25 / 26
  • 33. Conclusion In mainline Linux there are two schedulers. Numerous schedulers have been or will be implemented based on Linux for various reasons. There exists CPU bandwidth control work dealing with some of these schedulers. But each such mechanism can only work with for one scheduler (one specific type of tasks). Y. Sun (GPCSE) Master thesis September 14, 2012 26 / 26
  • 34. Conclusion In mainline Linux there are two schedulers. Numerous schedulers have been or will be implemented based on Linux for various reasons. There exists CPU bandwidth control work dealing with some of these schedulers. But each such mechanism can only work with for one scheduler (one specific type of tasks). The oxc framework can provide real-time CPU bandwidth control for tasks of all these schedulers! The project is just started; more work is coming. Y. Sun (GPCSE) Master thesis September 14, 2012 26 / 26

×