Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
EC
Uploaded byEric Chou
140 views

Learn and Play with sched_ext (scx) @ COSCUP2026

Explore CPU scheduling fundamentals, challenges, and how sched_ext enables customizable, efficient scheduling via BPF in Linux kernels, improving workload management and system performance.

Technology◦
Download to read offline
Learn (and Play with) sched_ext Cheng-Yang Chou, Po-Ying Chiu 2026/08/09
System SWE @ MTK GitHub: EricccTaiwan LinkedIn: Eric Chou MS @ NCKUEE GitHub: charliechiou LinkedIn: Po-Ying Chiu 聲明：以下內容為個人研究分享,與工作無關,不代表任職公司立場。 2
Outline ● What is CPU scheduling ● Why sched_ext ● How sched_ext ● 第 N 次貢獻 Linux 核心 : link 3
Outline ● What is CPU scheduling ● Why sched_ext ● How sched_ext ● 第 N 次貢獻 Linux 核心 : link 4
What is CPU scheduling? ● Which task ○ 下一個 runnable task 是誰? ● Which CPU ○ task 應該靠近哪個 cache / NUMA node ? ● When ○ wakeup 後要不要立刻搶 CPU? ● How long ○ time slice 多長? when to preempt? 5
What is CPU scheduling? sched_ext 提供 Local / Global / Custom DiSpatch Queue 6 CPU 2 CPU 1 CPU 2 Task 2 Task 1 Task 1 Task 2 SCX_DSQ_GLOBAL SCX_DSQ_LOCAL dsq_1 dsq_2 HEAD TAIL HEAD TAIL Task 3 Task 4 unbalance ->steal lock SCX_DSQ_LOCAL CPU 1 SCX_DSQ_LOCAL
Outline ● What is CPU scheduling ● Why sched_ext ● How sched_ext ● 第 N 次貢獻 Linux 核心 : link 7
Why sched_ext? 同一套 scheduling policy 難以服務所有場景 1. Tradeoff / Workload 目標互斥 a. locality 少數 task v.s. balance 不讓 CPU idle b. RPC / interactive 在意低延遲 v.s. batch workload 在意全部跑完，而非立刻回應 2. 局部修正的反效果 a. 同一個 heuristic 在某些 workload 是最佳化，在另一個 workload 就變成 regression, e.g. CFS 中的 sched_balance_newidle() [1] 3. 硬體越來越複雜 a. SMT sibling contention, NUMA locality, LLC scharing, big.LITTLE/ asymmetric CPU capacity, power, thermal, cgroup isolation 都會影響，該不該搬動 task 、搬去哪顆 CPU 的決策 4. Kernel Upstream 迭代速度太慢 8
9 We can do in <6.12 kernel ● set CPU affinity (tasket -c [cpu] ) ● set task priority ( nice -n [prio]) But we cannot ● modify tasks’ virtual runtime ● assign tasks’ timeslice Why sched_ext?
Outline ● What is CPU scheduling ● Why sched_ext ● How sched_ext ● 第 N 次貢獻 Linux 核心 : link 10
sched_ext : user space + kernel space 1. BPF side (user space) : scheduling policy iteration a. 快速驗證 b. 針對特定 workload customized [2] c. attatch/detach BPF schedulers 2. Kernel side : mechanism and recovery a. task ownership : ops_state (NONE, QUEUEING, QUEUED, DISPATCHING) b. DSQ c. class switching d. fallback to CFS 11
[PATCHSET v7] sched: Implement BPF extensible scheduler class [3] 12
一些重點 1. task policy 是 user-visible 設定 => userspace 可以設定 a. e.g. sched_setscheduler(pid, SCHED_EXT, &param), sched_getscheduler(pid) b. SCHED_NORMAL, SCHED_BATCH, SCHED_EXT 2. task sched_class 是 kernel implementation => userspace 不能設定 a. fair_sched_class, rt_sched_class, ext_sched_class 3. runqueue / rq 是每顆 CPU 的排程狀態 a. per-CPU satae, local queue, current task 4. CPU executions a. pick next task, set next task, run 13
一些重點 14 Q: 沒有 load BPF scheduler, 但 sched_setscheduler(pid, SCHED_EXT, …) ? A: p->policy = sched_ext 但 p->sched_class = fair, 所以仍是由 CFS 排程 bool task_should_scx(struct task_struct *p) { if (!scx_enabled() || unlikely(scx_ops_enable_state() == SCX_OPS_DISABLING)) return false; if (READ_ONCE(scx_switching_all)) return true; return p->policy == SCHED_EXT; }
一些重點 15 Q: 沒有 load BPF scheduler, 但 sched_setscheduler(pid, SCHED_EXT, …) ? A: p->policy = sched_ext 但 p->sched_class = fair, 所以仍是由 CFS 排程
try_to_wake_up() select_task_rq_scx() choose target CPU hint ops.selet_cpu BPF callback skip enqueue *ops.enqueue() is not called Wakeup Path scx_bpf_dispatch() direct from select_cpu Enqueue Path enqueue_task_scx() mark task as runable/queued ops.enqueue() Dispach Queues SCX_DSQ_GLOBAL / custom DSQ (scx_bpf_create_dsq) SCX_DSQ_LOCAL per-CPU FIFO CPU Pull & Run schedule() CPU needs task balance_scx() 1. local DSQ? 2. consume global DSQ ? 3. ops.dipatch() -> BPF flush_dispatch_ buf() pick_next_task_scx() take first task from scx.local_dsq set_next_task_scx() 16 Wakeup RUN do_enqueue_task() 決定走 BPF / local / global / ddsp Task: From wakeup to run CPU always consume task from local DSQ enqueue_task_scx() do_enqueue_task()
Task: From wakeup to run CPU always consume task from local DSQ SCX DSQs CPU0 local DSQ rq->scx custom / global DSQ T1 T2 T3 T4 T5 SCX core pick_next_task_scx() local ready ops.dispatch CPU0 struct rq set rq->curr T1 running rq->scx.local_dsq T2 T3 T4 T5 refill local dsq 17
kernel/tools/sched_ext/scx_simple.bpf.c 18 idle CPU exists? // is_idle Yes => - select_cpu dispatches to SCX_DSQ_LOCAL - enqueue() skipped - CPU can pick from local DSQ NO => - enqueue() dispatches to SCX_DSQ_GLOBAL - balance_scx pulls global to local DSQ - CPU picks from local DSQ
struct sched_ext_ops 概念上類似於 sched_class 的 callbacks , implement in BPF schedulers 19
Attach BPF scheduler 20 load BPF obj -> struct_ops sched_ext_ops // 動態替換 sched_ext_ops callback table -> bpf_sched_ext_ops.reg -> bpf_scx_reg() // bpf_link 管理 scheduler lifecycle -> scx_ops_enable() user space kernel space
Detach BPF scheduler 21 Detach bpf_link -> bpf_scx_unreg() -> scx_ops_disable (SCX_EXIT_UNREG) -> scx_ops_disable_workfn() user space kernel space
sched_ext stub into core.c 1. Taks free : __put_task_struct() -> sched_ext_free() 2. Fork lifecycle hooks: a. sched_fork() -> scx_pre_fork() / scx_cancel_fork() b. sched_cgroup_fork() -> scx_fork(p) c. sched_post_fork() -> scx_post_fork() d. error path -> scx_cancel_fork() 3. scheduler class iteration a. for_each_class() -> for_each_active_class() b. for_class_range() -> for_balance_class_range() 4. Idle tracking a. set_next_task_idle() / put_prev_task_idle() -> scx_update_idle (rq, true/false) 5. scheduler init a. sched_init() -> init_sched_ext_class() 22
SCX_OPS_ENABLE() 23 1. 全系統只能有一個 SCX scheduler => v7.3 不是了 2. global state init a. scx_ops_helper, scx_root_kobj, scx_exit_info, scx_ops = *ops, exit_kind=SCX_EXIT_NONE, DISABLED -> PREPPING 3. 凍住 CPU hotplug (cpus_read_lock/unlock), 呼叫 ops.init() 4. validate_ops / allocate dispatch buffer / watchdog_timeout 5. close fork race (scx_fork_rwsem) , 打開 scx root path 6. init existing task, scx_task_iter: a. task state : SCX_TASK_NONE -> SCX_TASK_INIT -> SCX_TASK_READY 7. commit point : 切 sched class (preempt_disable) a. state : PREPPING -> ENABLINEG b. 對每個 task, 從 rq 取下 -> 換 class -> 放回 rq c. task state : READY -> ENABLED 8. publish : ENABLING -> ENABLED a. release scx_fork_rwsem / cpus_read_lock b. 掛上 sysfs, 發 uevent
SCX_OPS_DISABLE() 24 scx_ops_disable -> schedule_scx_ops_disable_work() -> kthread_queue_work(scx_ops_helper, &scx_ops_disable_work) ⇣ DEFINE_KTHREAD_WORK(scx_ops_disable_work, scx_ops_disable_workfn) scx_ops_disable_workfn() -> scx_ops_bypass()
SCX_OPS_BYPASS() 25 1. 完全不能信任 scheduler => 三個不能用: a. mutex / rwsem — 可能被餓死的 task 持有 b. static_branch_enable/disable() — jump label 內部也要拿 mutex c. cpus_read_lock() — 所以走 for_each_possible_cpu,不是 online 2. atomic_inc/dec_return(&scx_ops_bypass_depth) a. 只有 0 <-> 1 才真正動作 3. counter 一變 1 => 全域立即生效(scx_ops_bypassing()) a. ops.enqueue() 被忽略 => 一律進 global DSQ b. ops.dispatch() 被忽略 => 只 consume global DSQ c. slice 不再被信任 => 每個 tick 強制 resched,current 轉到隊尾 4. walk 每個 rq 的 runnable_list,cycle deq -> enq a. 停在 custom DSQ 的 task 沒人會再撈,要主動搬回 global DSQ b. task 會被加回 list 尾端 => 必須用 safe_reverse 迭代 5. global DSQ FIFO + 每 tick 輪轉 => 不在需要 BPF scheduler ● iter: run job ● next: 下一個 move tail ● orange: 已 run Head A B C next iter Head A C B next iter Head C B A next iter Head C B A
SCX_OPS_DISABLE_WORKFN() 26 1. 確認死因:cmpxchg(exit_kind, kind -> SCX_EXIT_DONE) a. 讀到 NONE(期間已有新 scheduler 註冊)或 DONE(已拆過) => 直接 return 2. scx_ops_bypass(true):在拿任何鎖之前,先保證 forward progress a. 持鎖的 task 可能正被壞掉的 scheduler 餓死 => 不先 bypass 會卡死在自己想修的 bug 上 3. state:ENABLED -> DISABLING 4. 現在才敢 block:mutex_lock(scx_ops_enable_mutex), percpu_down_write(scx_fork_rwsem), cpus_read_lock 5. 還原每個 task,scx_task_iter: a. 從 rq 取下 -> 換回原 sched class -> 放回 rq -> scx_ops_exit_task() b. TASK_DEAD 不切 class,但仍要 exit_task,否則 ops.exit_task() 永遠不會被呼叫 6. 關閘:static_branch_disable(全部)+ synchronize_rcu() a. 此時已無 task 在 SCX 上,才能安全關閘並 flush 飛行中的 callback 7. 收尾: ops.exit(ei) / kobject_del / destroy_dsq / free a. state:DISABLING -> DISABLED b. scx_ops_bypass(false)
Outline ● What is CPU scheduling ● Why sched_ext ● How sched_ext ● 第 N 次貢獻 Linux 核心 : link 27
N ≈ 50 (希望明年 N = 100 28
Reference [1] : https://www.facebook.com/share/p/1Frf2AwMfY/ [2] : https://engineering.fb.com/2026/07/13/ml-applications/modernizing-the-meta-ads-s ervice-with-an-open-source-kernel-scheduler/ [3] : https://lore.kernel.org/all/20240618212056.2833381-1-tj@kernel.org/ 29
Q&A Thanks for listening