[linux 6.6.y] Backport sched_ext from OpenAnolis by LFRon · Pull Request #2172 · deepin-community/kernel

LFRon · 2026-09-24T16:27:07Z

sched_ext（可扩展调度器）是Linux内核引入的一种新型CPU调度框架，允许开发者利用eBPF技术编写自定义调度策略，无需修改内核源码即可动态替换默认调度器。它以内核模块形式运行，但高度依赖用户空间工具协同，该PR从Linux 6.18 LTS内核backport下了完整的sched_ext实现，可直接与scx和scx-loader用户空间组件一起工作。
要使用sched_ext，需要内核和用户空间同时安装scx和scx-loader。首先，内核必须启用CONFIG_SCHED_CLASS_EXT配置，集成scx核心支持。其次，用户空间需安装scx软件包，它提供了多种现成的eBPF调度器程序（如scx_simple等）；同时必须安装scx-loader，作为用户态守护进程，负责将选定的调度器安全地加载到内核、处理生命周期管理及故障回退。只有内核组件与用户空间的scx、scx-loader完整配合，才能灵活切换调度策略，实现性能调优。

deepin-ci-robot · 2026-09-24T16:27:23Z

Hi @LFRon. Thanks for your PR. 😃

deepin-ci-robot · 2026-09-24T16:27:30Z

Hi @LFRon. Thanks for your PR.

I'm waiting for a deepin-community member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

deepin-ci-robot · 2026-09-24T16:27:40Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign lanlanxiyiji for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

deepin/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

sourcery-ai

Sorry @LFRon, your pull request is larger than the review limit of 150,000 diff characters

Copilot

🟡 Changes recommended

It breaks the documented task_struct kABI and contains selftest and scheduler-tool bugs that can hide failures or corrupt memory.

5 open findings

What changed in this PR

Backports sched_ext and its BPF infrastructure, tooling, examples, and selftests to the 6.6 kernel tree.

Changes:

Adds the sched_ext scheduling class and scheduler integration.
Backports required BPF, struct_ops, arena, and sparse-vmalloc support.
Adds sched_ext tools, documentation, example schedulers, and selftests.

File	Description
`tools/testing/selftests/sched_ext/util.h`	Declares test utilities.
`tools/testing/selftests/sched_ext/util.c`	Implements file utilities.
`tools/testing/selftests/sched_ext/test_example.c`	Adds framework example test.
`tools/testing/selftests/sched_ext/select_cpu_vtime.c`	Adds vtime dispatch test.
`tools/testing/selftests/sched_ext/select_cpu_vtime.bpf.c`	Implements vtime test scheduler.
`tools/testing/selftests/sched_ext/select_cpu_dispatch.c`	Tests direct CPU dispatch.
`tools/testing/selftests/sched_ext/select_cpu_dispatch.bpf.c`	Implements dispatch test scheduler.
`tools/testing/selftests/sched_ext/select_cpu_dispatch_dbl_dsp.c`	Tests duplicate dispatch failure.
`tools/testing/selftests/sched_ext/select_cpu_dispatch_dbl_dsp.bpf.c`	Triggers duplicate dispatch.
`tools/testing/selftests/sched_ext/select_cpu_dispatch_bad_dsq.c`	Tests invalid DSQ handling.
`tools/testing/selftests/sched_ext/select_cpu_dispatch_bad_dsq.bpf.c`	Dispatches to invalid DSQ.
`tools/testing/selftests/sched_ext/select_cpu_dfl.c`	Tests default CPU selection.
`tools/testing/selftests/sched_ext/select_cpu_dfl.bpf.c`	Implements default-selection scheduler.
`tools/testing/selftests/sched_ext/select_cpu_dfl_nodispatch.c`	Tests no-dispatch selection mode.
`tools/testing/selftests/sched_ext/select_cpu_dfl_nodispatch.bpf.c`	Implements no-dispatch scheduler.
`tools/testing/selftests/sched_ext/scx_test.h`	Defines test framework API.
`tools/testing/selftests/sched_ext/runner.c`	Runs sched_ext tests.
`tools/testing/selftests/sched_ext/reload_loop.c`	Adds reload stress test.
`tools/testing/selftests/sched_ext/prog_run.c`	Tests BPF_PROG_RUN integration.
`tools/testing/selftests/sched_ext/prog_run.bpf.c`	Implements syscall BPF test.
`tools/testing/selftests/sched_ext/minimal.c`	Tests minimal scheduler loading.
`tools/testing/selftests/sched_ext/minimal.bpf.c`	Defines minimal scheduler.
`tools/testing/selftests/sched_ext/maybe_null.c`	Tests nullable callback arguments.
`tools/testing/selftests/sched_ext/maybe_null.bpf.c`	Implements valid nullable callbacks.
`tools/testing/selftests/sched_ext/maybe_null_fail_yld.bpf.c`	Tests invalid yield dereference.
`tools/testing/selftests/sched_ext/maybe_null_fail_dsp.bpf.c`	Tests invalid dispatch dereference.
`tools/testing/selftests/sched_ext/maximal.c`	Tests maximal scheduler loading.
`tools/testing/selftests/sched_ext/maximal.bpf.c`	Defines all scheduler callbacks.
`tools/testing/selftests/sched_ext/init_enable_count.c`	Verifies lifecycle callback counts.
`tools/testing/selftests/sched_ext/init_enable_count.bpf.c`	Counts lifecycle callbacks.
`tools/testing/selftests/sched_ext/hotplug.c`	Tests CPU hotplug behavior.
`tools/testing/selftests/sched_ext/hotplug.bpf.c`	Implements hotplug scheduler.
`tools/testing/selftests/sched_ext/hotplug_test.h`	Shares hotplug test flags.
`tools/testing/selftests/sched_ext/exit.c`	Tests scheduler exit paths.
`tools/testing/selftests/sched_ext/exit.bpf.c`	Triggers scheduler exits.
`tools/testing/selftests/sched_ext/exit_test.h`	Defines exit test cases.
`tools/testing/selftests/sched_ext/enq_select_cpu_fails.c`	Tests prohibited helper usage.
`tools/testing/selftests/sched_ext/enq_select_cpu_fails.bpf.c`	Calls selection helper from enqueue.
`tools/testing/selftests/sched_ext/enq_last_no_enq_fails.c`	Tests invalid ENQ_LAST configuration.
`tools/testing/selftests/sched_ext/enq_last_no_enq_fails.bpf.c`	Defines invalid scheduler flags.
`tools/testing/selftests/sched_ext/dsp_local_on.c`	Tests remote local-DSQ dispatch.
`tools/testing/selftests/sched_ext/dsp_local_on.bpf.c`	Implements local-on dispatch.
`tools/testing/selftests/sched_ext/ddsp_vtimelocal_fail.c`	Tests invalid vtime dispatch.
`tools/testing/selftests/sched_ext/ddsp_vtimelocal_fail.bpf.c`	Triggers local vtime failure.
`tools/testing/selftests/sched_ext/ddsp_bogus_dsq_fail.c`	Tests bogus DSQ fallback.
`tools/testing/selftests/sched_ext/ddsp_bogus_dsq_fail.bpf.c`	Dispatches to bogus DSQ.
`tools/testing/selftests/sched_ext/create_dsq.c`	Tests DSQ lifecycle.
`tools/testing/selftests/sched_ext/create_dsq.bpf.c`	Creates and destroys DSQs.
`tools/testing/selftests/sched_ext/config`	Enables required test config.
`tools/testing/selftests/sched_ext/.gitignore`	Ignores generated test files.
`tools/testing/selftests/bpf/progs/verifier_spin_lock.c`	Updates verifier expectation.
`tools/testing/selftests/bpf/progs/test_global_func5.c`	Updates verifier message.
`tools/testing/selftests/bpf/progs/test_global_func_ctx_args.c`	Handles s390 context types.
`tools/testing/selftests/bpf/progs/task_kfunc_failure.c`	Updates kfunc failure expectation.
`tools/testing/selftests/bpf/progs/cgrp_kfunc_failure.c`	Updates cgroup verifier expectation.
`tools/testing/selftests/bpf/prog_tests/log_fixup.c`	Adjusts log truncation tests.
`tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c`	Registers test struct_ops types.
`tools/testing/selftests/bpf/bpf_test_no_cfi/bpf_test_no_cfi.c`	Tests missing CFI stubs.
`tools/sched_ext/scx_simple.c`	Adds simple scheduler frontend.
`tools/sched_ext/scx_simple.bpf.c`	Implements simple scheduler.
`tools/sched_ext/scx_show_state.py`	Displays sched_ext state.
`tools/sched_ext/scx_qmap.c`	Adds qmap scheduler frontend.
`tools/sched_ext/scx_flatcg.h`	Defines flat-cgroup state.
`tools/sched_ext/scx_central.c`	Adds central scheduler frontend.
`tools/sched_ext/include/scx/user_exit_info.h`	Defines exit reporting API.
`tools/sched_ext/include/scx/common.h`	Adds userspace helper API.
`tools/sched_ext/include/bpf-compat/gnu/stubs.h`	Adds BPF build compatibility stub.
`tools/sched_ext/.gitignore`	Ignores generated scheduler files.
`tools/Makefile`	Integrates sched_ext tools.
`tools/include/uapi/linux/bpf.h`	Synchronizes BPF UAPI.
`net/xfrm/xfrm_interface_bpf.c`	Uses kfunc definition guards.
`net/socket.c`	Uses BPF hook guards.
`net/netfilter/nf_nat_bpf.c`	Uses kfunc definition guards.
`net/netfilter/nf_conntrack_bpf.c`	Uses kfunc definition guards.
`net/ipv4/tcp_cong.c`	Updates congestion-control replacement.
`net/ipv4/fou_bpf.c`	Uses kfunc definition guards.
`net/ipv4/bpf_tcp_ca.c`	Modernizes TCP struct_ops registration.
`net/core/xdp.c`	Uses kfunc definition guards.
`net/core/filter.c`	Updates common helper availability.
`net/bpf/test_run.c`	Uses kfunc definition guards.
`net/bpf/bpf_dummy_struct_ops.c`	Registers dummy struct_ops.
`mm/vmalloc.c`	Adds sparse vmalloc mapping.
`Makefile`	Propagates tool verbosity.
`MAINTAINERS`	Adds sched_ext ownership.
`lib/dump_stack.c`	Includes sched_ext diagnostics.
`kernel/watchdog.c`	Ejects stalled BPF schedulers.
`kernel/trace/bpf_trace.c`	Updates tracing BPF helpers.
`kernel/sched/syscalls.c`	Integrates SCHED_EXT policy.
`kernel/sched/stop_task.c`	Updates scheduler class interface.
`kernel/sched/rt.c`	Updates RT class interface.
`kernel/sched/pelt.h`	Declares non-fair load updates.
`kernel/sched/pelt.c`	Implements non-fair load updates.
`kernel/sched/idle.c`	Integrates sched_ext idle tracking.
`kernel/sched/ext.h`	Declares sched_ext scheduler hooks.
`kernel/sched/debug.c`	Reports sched_ext task state.
`kernel/sched/deadline.c`	Updates deadline class interface.
`kernel/sched/cpufreq_schedutil.c`	Integrates sched_ext CPU targets.
`kernel/sched/build_policy.c`	Builds sched_ext implementation.
`kernel/sched/autogroup.c`	Updates task migration calls.
`kernel/Kconfig.preempt`	Adds sched_ext configuration.
`kernel/fork.c`	Integrates sched_ext task lifecycle.
`kernel/events/core.c`	Uses program sleepable state.
`kernel/cgroup/rstat.c`	Uses BPF hook guards.
`kernel/cgroup/cgroup.c`	Preserves interrupt state in iterators.
`kernel/bpf/trampoline.c`	Uses program sleepable state.
`kernel/bpf/tnum.c`	Adds subregister composition.
`kernel/bpf/map_iter.c`	Uses kfunc definition guards.
`kernel/bpf/Makefile`	Builds BPF arena support.
`kernel/bpf/local_storage.c`	Simplifies sequence output.
`kernel/bpf/helpers.c`	Extends helpers and kfuncs.
`kernel/bpf/hashtab.c`	Simplifies sequence output.
`kernel/bpf/disasm.c`	Decodes new BPF instructions.
`kernel/bpf/cpumask.c`	Adds cpumask weight kfunc.
`kernel/bpf/core.c`	Adds arena and JIT support.
`kernel/bpf/cgroup.c`	Consolidates common helpers.
`kernel/bpf/bpf_struct_ops_types.h`	Removes static struct_ops list.
`kernel/bpf/bpf_iter.c`	Updates iterator behavior.
`kernel/bpf/arraymap.c`	Simplifies sequence output.
`init/Kconfig`	Adds EXT group scheduling.
`init/init_task.c`	Initializes sched_ext task state.
`include/uapi/linux/sched.h`	Exposes SCHED_EXT policy.
`include/uapi/linux/bpf.h`	Extends BPF UAPI.
`include/trace/events/sched_ext.h`	Adds sched_ext tracepoint.
`include/linux/vmalloc.h`	Exposes sparse vmalloc API.
`include/linux/tnum.h`	Declares subregister composition.
`include/linux/sched/task.h`	Updates fork scheduler API.
`include/linux/sched.h`	Embeds sched_ext task state.
`include/linux/filter.h`	Adds arena instruction modes.
`include/linux/cgroup.h`	Refactors iterator flags.
`include/linux/btf.h`	Extends kfunc and struct_ops APIs.
`include/linux/btf_ids.h`	Adds kfunc BTF sets.
`include/linux/bpf_types.h`	Registers arena map type.
`include/asm-generic/vmlinux.lds.h`	Orders sched_ext class.
`drivers/tty/sysrq.c`	Reserves sched_ext reset key.
`Documentation/scheduler/index.rst`	Links sched_ext documentation.
`Documentation/bpf/kfuncs.rst`	Documents kfunc guard macros.
`Documentation/bpf/cpumasks.rst`	Documents cpumask weight.
`arch/x86/configs/deepin_x86_desktop_defconfig`	Enables sched_ext by default.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+#ifdef CONFIG_SCHED_CLASS_EXT
+	struct sched_ext_entity		scx;
+#endif


+	cpuset = CPU_ALLOC(skel->rodata->nr_cpu_ids);
+	SCX_BUG_ON(!cpuset, "Failed to allocate cpuset");
+	CPU_ZERO(cpuset);
+	CPU_SET(skel->rodata->central_cpu, cpuset);
+	SCX_BUG_ON(sched_setaffinity(0, sizeof(cpuset), cpuset),
+		   "Failed to affinitize to central CPU %d (max %d)",
+		   skel->rodata->central_cpu, skel->rodata->nr_cpu_ids - 1);
+	CPU_FREE(cpuset);


+	if (UEI_ECODE_RESTART(ecode))
+		goto restart;


+		}
+	}
+
+	for (i = 0; i < __scx_num_tests; i++) {


+	ret = sprintf(buf, "%lu", val);
+	if (ret < 0)
+		return ret;
+
+	if (write_text(path, buf, sizeof(buf)) <= 0)
+		return -1;


…igrated When a running task is migrated to another CPU, the stop_task is used to preempt the running task and migrate it. This, expectedly, invokes ops.cpu_release(). If the BPF scheduler then calls scx_bpf_reenqueue_local(), it re-enqueues all tasks on the local DSQ including the task which is being migrated. This creates an unnecessary re-enqueue of a task which is about to be deactivated and re-activated for migration anyway. It can also cause confusion for the BPF scheduler as scx_bpf_task_cpu() of the task and its allowed CPUs may not agree while migration is pending. Signed-off-by: Tejun Heo <tj@kernel.org> Fixes: 245254f ("sched_ext: Implement sched_ext_ops.cpu_acquire/release()") Acked-by: David Vernet <void@manifault.com> (cherry picked from commit e7a6395) Signed-off-by: LFRon <ronforever@outlook.com>

Move struct balance_callback definition upward so that it's visible to class-specific rq struct definitions. This will be used to embed a struct balance_callback in struct scx_rq. No functional changes. Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: David Vernet <void@manifault.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> (cherry picked from commit fc28311) Signed-off-by: LFRon <ronforever@outlook.com>

task_linked_on_dsq() exists as a helper because it used to test both the rbtree and list nodes. It now only tests the list node and the list node will soon be used for something else too. The helper doesn't improve anything materially and the naming will become confusing. Open-code the list node testing and remove task_linked_on_dsq() Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: David Vernet <void@manifault.com> (cherry picked from commit d6a0591) Signed-off-by: LFRon <ronforever@outlook.com>

@rf

sched_ext often needs to migrate tasks across CPUs right before execution and thus uses the balance path to dispatch tasks from the BPF scheduler. balance_scx() is called with rq locked and pinned but is passed @rf and thus allowed to unpin and unlock. Currently, @rf is passed down the call stack so the rq lock is unpinned just when double locking is needed. This creates unnecessary complications such as having to explicitly manipulate lock pinning for core scheduling. We also want to use dispatch_to_local_dsq_lock() from other paths which are called with rq locked but unpinned. rq lock handling in the dispatch path is straightforward outside the migration implementation and extending the pinning protection down the callstack doesn't add enough meaningful extra protection to justify the extra complexity. Unpin and repin rq lock from the outer balance_scx() and drop @rf passing and lock pinning handling from the inner functions. UP is updated to call balance_one() instead of balance_scx() to avoid adding NULL @rf handling to balance_scx(). AS this makes balance_scx() unused in UP, it's put inside a CONFIG_SMP block. No functional changes intended outside of lock annotation updates. Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: David Vernet <void@manifault.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Andrea Righi <righi.andrea@gmail.com> (cherry picked from commit 3cf78c5) Signed-off-by: LFRon <ronforever@outlook.com>

…KEUP SCX_RQ_BALANCING is used to mark that the rq is currently in balance(). Rename it to SCX_RQ_IN_BALANCE and add SCX_RQ_IN_WAKEUP which marks whether the rq is currently enqueueing for a wakeup. This will be used to implement direct dispatching to local DSQ of another CPU. Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: David Vernet <void@manifault.com> (cherry picked from commit f47a818) Signed-off-by: LFRon <ronforever@outlook.com>

@rf

In ops.dispatch(), SCX_DSQ_LOCAL_ON can be used to dispatch the task to the local DSQ of any CPU. However, during direct dispatch from ops.select_cpu() and ops.enqueue(), this isn't allowed. This is because dispatching to the local DSQ of a remote CPU requires locking both the task's current and new rq's and such double locking can't be done directly from ops.enqueue(). While waking up a task, as ops.select_cpu() can pick any CPU and both ops.select_cpu() and ops.enqueue() can use SCX_DSQ_LOCAL as the dispatch target to dispatch to the DSQ of the picked CPU, the BPF scheduler can still do whatever it wants to do. However, while a task is being enqueued for a different reason, e.g. after its slice expiration, only ops.enqueue() is called and there's no way for the BPF scheduler to directly dispatch to the local DSQ of a remote CPU. This gap in API forces schedulers into work-arounds which are not straightforward or optimal such as skipping direct dispatches in such cases. Implement deferred enqueueing to allow directly dispatching to the local DSQ of a remote CPU from ops.select_cpu() and ops.enqueue(). Such tasks are temporarily queued on rq->scx.ddsp_deferred_locals. When the rq lock can be safely released, the tasks are taken off the list and queued on the target local DSQs using dispatch_to_local_dsq(). v2: - Add missing return after queue_balance_callback() in schedule_deferred(). (David). - dispatch_to_local_dsq() now assumes that @rq is locked but unpinned and thus no longer takes @rf. Updated accordingly. - UP build warning fix. Signed-off-by: Tejun Heo <tj@kernel.org> Tested-by: Andrea Righi <righi.andrea@gmail.com> Acked-by: David Vernet <void@manifault.com> Cc: Dan Schatzberg <schatzberg.dan@gmail.com> Cc: Changwoo Min <changwoo@igalia.com> (cherry picked from commit 5b26f7b) Signed-off-by: LFRon <ronforever@outlook.com>

…eues Because there was no way to directly dispatch to the local DSQ of a remote CPU from ops.enqueue(), scx_qmap skipped looking for an idle CPU on !wakeup enqueues. This restriction was removed and sched_ext now allows SCX_DSQ_LOCAL_ON verdicts for direct dispatches. Factor out pick_direct_dispatch_cpu() from ops.select_cpu() and use it to direct dispatch from ops.enqueue() on !wakeup enqueues. Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: David Vernet <void@manifault.com> Cc: Dan Schatzberg <schatzberg.dan@gmail.com> Cc: Changwoo Min <changwoo@igalia.com> Cc: Andrea Righi <righi.andrea@gmail.com> (cherry picked from commit 1edab90) Signed-off-by: LFRon <ronforever@outlook.com>

[ Upstream commit 6ebc503 ] may_goto uses an additional 8 bytes on the stack, which causes the interpreters[] array to go out of bounds when calculating index by stack_size. 1. If a BPF program is rewritten, re-evaluate the stack size. For non-JIT cases, reject loading directly. 2. For non-JIT cases, calculating interpreters[idx] may still cause out-of-bounds array access, and just warn about it. 3. For jit_requested cases, the execution of bpf_func also needs to be warned. So move the definition of function __bpf_prog_ret0_warn out of the macro definition CONFIG_BPF_JIT_ALWAYS_ON. Reported-by: syzbot+d2a2c639d03ac200a4f1@syzkaller.appspotmail.com Closes: https://lore.kernel.org/bpf/0000000000000f823606139faa5d@google.com/ Fixes: 011832b ("bpf: Introduce may_goto instruction") Signed-off-by: Jiayuan Chen <mrpre@163.com> Link: https://lore.kernel.org/r/20250214091823.46042-2-mrpre@163.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> (cherry picked from commit 19e6817) Signed-off-by: LFRon <ronforever@outlook.com>

[ Upstream commit bbbc02b ] The bpf_verifier_state.loop_entry state should be copied by copy_verifier_state(). Otherwise, .loop_entry values from unrelated states would poison env->cur_state. Additionally, env->stack should not contain any states with .loop_entry != NULL. The states in env->stack are yet to be verified, while .loop_entry is set for states that reached an equivalent state. This means that env->cur_state->loop_entry should always be NULL after pop_stack(). See the selftest in the next commit for an example of the program that is not safe yet is accepted by verifier w/o this fix. This change has some verification performance impact for selftests: File Program Insns (A) Insns (B) Insns (DIFF) States (A) States (B) States (DIFF) ---------------------------------- ---------------------------- --------- --------- -------------- ---------- ---------- ------------- arena_htab.bpf.o arena_htab_llvm 717 426 -291 (-40.59%) 57 37 -20 (-35.09%) arena_htab_asm.bpf.o arena_htab_asm 597 445 -152 (-25.46%) 47 37 -10 (-21.28%) arena_list.bpf.o arena_list_del 309 279 -30 (-9.71%) 23 14 -9 (-39.13%) iters.bpf.o iter_subprog_check_stacksafe 155 141 -14 (-9.03%) 15 14 -1 (-6.67%) iters.bpf.o iter_subprog_iters 1094 1003 -91 (-8.32%) 88 83 -5 (-5.68%) iters.bpf.o loop_state_deps2 479 725 +246 (+51.36%) 46 63 +17 (+36.96%) kmem_cache_iter.bpf.o open_coded_iter 63 59 -4 (-6.35%) 7 6 -1 (-14.29%) verifier_bits_iter.bpf.o max_words 92 84 -8 (-8.70%) 8 7 -1 (-12.50%) verifier_iterating_callbacks.bpf.o cond_break2 113 107 -6 (-5.31%) 12 12 +0 (+0.00%) And significant negative impact for sched_ext: File Program Insns (A) Insns (B) Insns (DIFF) States (A) States (B) States (DIFF) ----------------- ---------------------- --------- --------- -------------------- ---------- ---------- ------------------ bpf.bpf.o lavd_init 7039 14723 +7684 (+109.16%) 490 1139 +649 (+132.45%) bpf.bpf.o layered_dispatch 11485 10548 -937 (-8.16%) 848 762 -86 (-10.14%) bpf.bpf.o layered_dump 7422 1000001 +992579 (+13373.47%) 681 31178 +30497 (+4478.27%) bpf.bpf.o layered_enqueue 16854 71127 +54273 (+322.02%) 1611 6450 +4839 (+300.37%) bpf.bpf.o p2dq_dispatch 665 791 +126 (+18.95%) 68 78 +10 (+14.71%) bpf.bpf.o p2dq_init 2343 2980 +637 (+27.19%) 201 237 +36 (+17.91%) bpf.bpf.o refresh_layer_cpumasks 16487 674760 +658273 (+3992.68%) 1770 65370 +63600 (+3593.22%) bpf.bpf.o rusty_select_cpu 1937 40872 +38935 (+2010.07%) 177 3210 +3033 (+1713.56%) scx_central.bpf.o central_dispatch 636 2687 +2051 (+322.48%) 63 227 +164 (+260.32%) scx_nest.bpf.o nest_init 636 815 +179 (+28.14%) 60 73 +13 (+21.67%) scx_qmap.bpf.o qmap_dispatch 2393 3580 +1187 (+49.60%) 196 253 +57 (+29.08%) scx_qmap.bpf.o qmap_dump 233 318 +85 (+36.48%) 22 30 +8 (+36.36%) scx_qmap.bpf.o qmap_init 16367 17436 +1069 (+6.53%) 603 669 +66 (+10.95%) Note 'layered_dump' program, which now hits 1M instructions limit. This impact would be mitigated in the next patch. Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20250215110411.3236773-2-eddyz87@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> (cherry picked from commit 46ba5757a7a4714e7d3f68cfe118208822cb3d78) Signed-off-by: LFRon <ronforever@outlook.com>

Recognize 'void *p__map' kfunc argument as 'struct bpf_map *p__map'. It allows kfunc to have 'void *' argument for maps, since bpf progs will call them as: struct { __uint(type, BPF_MAP_TYPE_ARENA); ... } arena SEC(".maps"); bpf_kfunc_with_map(... &arena ...); Underneath libbpf will load CONST_PTR_TO_MAP into the register via ld_imm64 insn. If kfunc was defined with 'struct bpf_map *' it would pass the verifier as well, but bpf prog would need to type cast the argument (void *)&arena, which is not clean. Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/r/20240307031228.42896-3-alexei.starovoitov@gmail.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> [ADAPT] NOTE: only the KF_ARG_PTR_TO_MAP pieces are taken; upstream's neighbouring KF_ARG_PTR_TO_NULL/KF_ARG_PTR_TO_CONST_STR enum members and the is_kfunc_arg_const_str() dispatch belong to other features absent from this base. (cherry picked from commit 8d94f13) Signed-off-by: LFRon <ronforever@outlook.com>

Recognize return of 'void *' from kfunc as returning unknown scalar. Acked-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/r/20240307031228.42896-2-alexei.starovoitov@gmail.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> (cherry picked from commit 88d1d4a) Signed-off-by: LFRon <ronforever@outlook.com>

Currently, calling any helpers, kfuncs, or subprogs except the graph data structure (lists, rbtrees) API kfuncs while holding a bpf_spin_lock is not allowed. One of the original motivations of this decision was to force the BPF programmer's hand into keeping the bpf_spin_lock critical section small, and to ensure the execution time of the program does not increase due to lock waiting times. In addition to this, some of the helpers and kfuncs may be unsafe to call while holding a bpf_spin_lock. However, when it comes to subprog calls, atleast for static subprogs, the verifier is able to explore their instructions during verification. Therefore, it is similar in effect to having the same code inlined into the critical section. Hence, not allowing static subprog calls in the bpf_spin_lock critical section is mostly an annoyance that needs to be worked around, without providing any tangible benefit. Unlike static subprog calls, global subprog calls are not safe to permit within the critical section, as the verifier does not explore them during verification, therefore whether the same lock will be taken again, or unlocked, cannot be ascertained. Therefore, allow calling static subprogs within a bpf_spin_lock critical section, and only reject it in case the subprog linkage is global. Acked-by: Yonghong Song <yonghong.song@linux.dev> Acked-by: David Vernet <void@manifault.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20240204222349.938118-2-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> [ADAPT] NOTE: two context-only conflicts resolved: - check_func_call: only the new global-subprog-under-lock rejection is taken; this base lacks the later subprog_name() refactor, so the original "func#%d" message is kept. - BPF_EXIT: condition becomes !curframe; this base has no process_bpf_exit_full label (bpf_throw/exceptions absent), so the label is dropped. (cherry picked from commit a44b133) Signed-off-by: LFRon <ronforever@outlook.com>

Allow transferring an imbalanced RCU lock state between subprog calls during verification. This allows patterns where a subprog call returns with an RCU lock held, or a subprog call releases an RCU lock held by the caller. Currently, the verifier would end up complaining if the RCU lock is not released when processing an exit from a subprog, which is non-ideal if its execution is supposed to be enclosed in an RCU read section of the caller. Instead, simply only check whether we are processing exit for frame#0 and do not complain on an active RCU lock otherwise. We only need to update the check when processing BPF_EXIT insn, as copy_verifier_state is already set up to do the right thing. Suggested-by: David Vernet <void@manifault.com> Tested-by: Yafang Shao <laoar.shao@gmail.com> Acked-by: Yonghong Song <yonghong.song@linux.dev> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Acked-by: David Vernet <void@manifault.com> Link: https://lore.kernel.org/r/20240205055646.1112186-2-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> (cherry picked from commit 6fceea0) Signed-off-by: LFRon <ronforever@outlook.com>

Introduce two new BPF kfuncs, bpf_preempt_disable and bpf_preempt_enable. These kfuncs allow disabling preemption in BPF programs. Nesting is allowed, since the intended use cases includes building native BPF spin locks without kernel helper involvement. Apart from that, this can be used to per-CPU data structures for cases where programs (or userspace) may preempt one or the other. Currently, while per-CPU access is stable, whether it will be consistent is not guaranteed, as only migration is disabled for BPF programs. Global functions are disallowed from being called, but support for them will be added as a follow up not just preempt kfuncs, but rcu_read_lock kfuncs as well. Static subprog calls are permitted. Sleepable helpers and kfuncs are disallowed in non-preemptible regions. Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20240424031315.2757363-2-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> [ADAPT] NOTE: three context-only adaptations: - helpers.c: kfuncs placed after bpf_rcu_read_unlock and registered in common_btf_ids; upstream's bpf_throw/bpf_wq neighbors are 6.9 features absent from this base. - check_helper_call: in_sleepable(env) -> env->prog->sleepable (this base has no per-state in_sleepable field; matches the existing rcu block's form). - state copy/compare: active_preempt_lock taken; the in_sleepable field copy/compare is dropped (field absent). (cherry picked from commit fc7566a) Signed-off-by: LFRon <ronforever@outlook.com>

[ Upstream commit 46f7ed3 ] There are three situations when a program logically exits and transfers control to the kernel or another program: bpf_throw, BPF_EXIT, and tail calls. The former two check for any lingering locks and references, but tail calls currently do not. Expand the checks to check for spin locks, RCU read sections and preempt disabled sections. Spin locks are indirectly preventing tail calls as function calls are disallowed, but the checks for preemption and RCU are more relaxed, hence ensure tail calls are prevented in their presence. Fixes: 9bb00b2 ("bpf: Add kfunc bpf_rcu_read_lock/unlock()") Fixes: fc7566a ("bpf: Introduce bpf_preempt_[disable,enable] kfuncs") Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20241103225940.1408302-2-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> [ADAPT] NOTE: check_reference_leak() in this base takes no second argument (upstream's bool param arrives with a later refactor), so the call keeps the existing 1-arg form; the added checks are otherwise taken verbatim. (cherry picked from commit aa51be3) Signed-off-by: LFRon <ronforever@outlook.com>

Extract the part of check_helper_call() as a utility function allowing to query 'struct bpf_func_proto' for a specific helper function id. Acked-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20240722233844.1406874-2-eddyz87@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> [ADAPT] NOTE: the pre-image in this base differs only in older wording of the invalid/unknown-func branches; the added get_helper_proto() and its call site are taken verbatim, i.e. the post-image is identical to upstream. (cherry picked from commit 45cbc7a) Signed-off-by: LFRon <ronforever@outlook.com>

The verifier currently does not permit global subprog calls when a lock is held, preemption is disabled, or when IRQs are disabled. This is because we don't know whether the global subprog calls sleepable functions or not. In case of locks, there's an additional reason: functions called by the global subprog may hold additional locks etc. The verifier won't know while verifying the global subprog whether it was called in context where a spin lock is already held by the program. Perform summarization of the sleepable nature of a global subprog just like changes_pkt_data and then allow calls to global subprogs for non-sleepable ones from atomic context. While making this change, I noticed that RCU read sections had no protection against sleepable global subprog calls, include it in the checks and fix this while we're at it. Care needs to be taken to not allow global subprog calls when regular bpf_spin_lock is held. When resilient spin locks is held, we want to potentially have this check relaxed, but not for now. Also make sure extensions freplacing global functions cannot do so in case the target is non-sleepable, but the extension is. The other combination is ok. Tests are included in the next patch to handle all special conditions. Fixes: 9bb00b2 ("bpf: Add kfunc bpf_rcu_read_lock/unlock()") Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20250301151846.1552362-2-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> [ADAPT] NOTE: four context adaptations for this base: - check_func_call: plural active_locks/active_preempt_locks -> this base's active_lock.ptr/active_preempt_lock; the active_irq_id clause (IRQ-disabled kfuncs, 6.16) is not applicable; in_sleepable(env) -> env->prog->sleepable. - bpf_prog_aux/bpf_subprog_info fields inserted adjacent to the existing changes_pkt_data members; jit_subprogs line added without the 6.9 exception_cb/exception_boundary context. - everything else (mark_subprog_might_sleep, merge_callee_effects, visit_insn helper/kfunc marking, check_cfg, attach-target check) taken verbatim. (cherry picked from commit e2d8f56) Signed-off-by: LFRon <ronforever@outlook.com>

Verifier enforces that all iterator structs are named `bpf_iter_<name>` and that whenever iterator is passed to a kfunc it's passed as a valid PTR -> STRUCT chain (with potentially const modifiers in between). We'll need this check for upcoming changes, so instead of duplicating the logic, extract it into a helper function. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20240808232230.2848712-2-andrii@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org> (cherry picked from commit 496ddd1) Signed-off-by: LFRon <ronforever@outlook.com>

There are potentially useful cases where a specific iterator type might need to be passed into some kfunc. So, in addition to existing bpf_iter_<type>_{new,next,destroy}() kfuncs, allow to pass iterator pointer to any kfunc. We employ "__iter" naming suffix for arguments that are meant to accept iterators. We also enforce that they accept PTR -> STRUCT btf_iter_<type> type chain and point to a valid initialized on-the-stack iterator state. Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/r/20240808232230.2848712-3-andrii@kernel.org Signed-off-by: Alexei Starovoitov <ast@kernel.org> [ADAPT] NOTE: this base already carries bdbf874's PTR_TO_STACK pre-check (local 6724aea), kept as-is above the new btf_check_iter_arg() block; the rest matches upstream's post-image. (cherry picked from commit baebe9a) Signed-off-by: LFRon <ronforever@outlook.com>

[ Upstream commit bd74e23 ] Andrii spotted that process_dynptr_func's rejection of incorrect argument register type will print an error string where argument numbers are not zero-indexed, unlike elsewhere in the verifier. Fix this by subtracting 1 from regno. The same scenario exists for iterator messages. Fix selftest error strings that match on the exact argument number while we're at it to ensure clean bisection. Suggested-by: Andrii Nakryiko <andrii@kernel.org> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20241203002235.3776418-1-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org> [ADAPT] NOTE: kernel-side only; the selftest portion of this commit is not picked (this project backports kernel-side only). The process_dynptr_func pointer-type-check message hunk is skipped because that check is absent from this base (it belongs to a different, un-ported commit); the other five arg-index fixes are taken verbatim. (cherry picked from commit fdaaf92) Signed-off-by: LFRon <ronforever@outlook.com>

[ Upstream commit e4414b0 ] kernel test robot reported verifier bug [1] where the helper func pointer could be NULL due to disabled config option. As Alexei suggested we could check on that in get_helper_proto directly. Marking tail_call helper func with BPF_PTR_POISON, because it is unused by design. [1] https://lore.kernel.org/oe-lkp/202507160818.68358831-lkp@intel.com Reported-by: kernel test robot <oliver.sang@intel.com> Reported-by: syzbot+a9ed3d9132939852d0df@syzkaller.appspotmail.com Suggested-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Jiri Olsa <jolsa@kernel.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Paul Chaignon <paul.chaignon@gmail.com> Acked-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/bpf/20250814200655.945632-1-jolsa@kernel.org Closes: https://lore.kernel.org/oe-lkp/202507160818.68358831-lkp@intel.com Signed-off-by: Sasha Levin <sashal@kernel.org> (cherry picked from commit 3d429cb1278e995e22995ef117fa96d223a67e93) Signed-off-by: LFRon <ronforever@outlook.com>

In order to further simplify the code in adjust_scalar_min_max_vals all the calls to mark_reg_unknown are replaced by __mark_reg_unknown. static void mark_reg_unknown(struct bpf_verifier_env *env, struct bpf_reg_state *regs, u32 regno) { if (WARN_ON(regno >= MAX_BPF_REG)) { ... mark all regs not init ... return; } __mark_reg_unknown(env, regs + regno); } The 'regno >= MAX_BPF_REG' does not apply to adjust_scalar_min_max_vals(), because it is only called from the following stack: - check_alu_op - adjust_reg_min_max_vals - adjust_scalar_min_max_vals The check_alu_op() does check_reg_arg() which verifies that both src and dst register numbers are within bounds. Signed-off-by: Cupertino Miranda <cupertino.miranda@oracle.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Cc: Yonghong Song <yonghong.song@linux.dev> Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com> Cc: David Faust <david.faust@oracle.com> Cc: Jose Marchesi <jose.marchesi@oracle.com> Cc: Elena Zannoni <elena.zannoni@oracle.com> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com> Link: https://lore.kernel.org/r/20240506141849.185293-2-cupertino.miranda@oracle.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> (cherry picked from commit d786957) Signed-off-by: LFRon <ronforever@outlook.com>

Split range computation checks in its own function, isolating pessimitic range set for dst_reg and failing return to a single point. Signed-off-by: Cupertino Miranda <cupertino.miranda@oracle.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Cc: Yonghong Song <yonghong.song@linux.dev> Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com> Cc: David Faust <david.faust@oracle.com> Cc: Jose Marchesi <jose.marchesi@oracle.com> Cc: Elena Zannoni <elena.zannoni@oracle.com> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com> bpf/verifier: improve code after range computation recent changes. Link: https://lore.kernel.org/r/20240506141849.185293-3-cupertino.miranda@oracle.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> (cherry picked from commit 0922c78) Signed-off-by: LFRon <ronforever@outlook.com>

Range for XOR and OR operators would not be attempted unless src_reg would resolve to a single value, i.e. a known constant value. This condition is unnecessary, and the following XOR/OR operator handling could compute a possible better range. Acked-by: Eduard Zingerman <eddyz87@gmail.com> Signed-off-by: Cupertino Miranda <cupertino.miranda@oracle.com Acked-by: Eduard Zingerman <eddyz87@gmail.com> Cc: Yonghong Song <yonghong.song@linux.dev> Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com> Cc: David Faust <david.faust@oracle.com> Cc: Jose Marchesi <jose.marchesi@oracle.com> Cc: Elena Zannoni <elena.zannoni@oracle.com> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com> Link: https://lore.kernel.org/r/20240506141849.185293-4-cupertino.miranda@oracle.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> (cherry picked from commit 138cc42) Signed-off-by: LFRon <ronforever@outlook.com>

Daniel Hodges reported a kernel verifier crash when playing with sched-ext. Further investigation shows that the crash is due to invalid memory access in stacksafe(). More specifically, it is the following code: if (exact != NOT_EXACT && old->stack[spi].slot_type[i % BPF_REG_SIZE] != cur->stack[spi].slot_type[i % BPF_REG_SIZE]) return false; The 'i' iterates old->allocated_stack. If cur->allocated_stack < old->allocated_stack the out-of-bound access will happen. To fix the issue add 'i >= cur->allocated_stack' check such that if the condition is true, stacksafe() should fail. Otherwise, cur->stack[spi].slot_type[i % BPF_REG_SIZE] memory access is legal. Fixes: 2793a8b ("bpf: exact states comparison for iterator convergence checks") Cc: Eduard Zingerman <eddyz87@gmail.com> Reported-by: Daniel Hodges <hodgesd@meta.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Signed-off-by: Yonghong Song <yonghong.song@linux.dev> Link: https://lore.kernel.org/r/20240812214847.213612-1-yonghong.song@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org> (cherry picked from commit bed2eb9) Signed-off-by: LFRon <ronforever@outlook.com>

Currently bpf_get_current_pid_tgid() is allowed in tracing, cgroup and sk_msg progs while bpf_get_ns_current_pid_tgid() is only allowed in tracing progs. We have an internal use case where for an application running in a container (with pid namespace), user wants to get the pid associated with the pid namespace in a cgroup bpf program. Currently, cgroup bpf progs already allow bpf_get_current_pid_tgid(). Let us allow bpf_get_ns_current_pid_tgid() as well. With auditing the code, bpf_get_current_pid_tgid() is also used by sk_msg prog. But there are no side effect to expose these two helpers to all prog types since they do not reveal any kernel specific data. The detailed discussion is in [1]. So with this patch, both bpf_get_current_pid_tgid() and bpf_get_ns_current_pid_tgid() are put in bpf_base_func_proto(), making them available to all program types. [1] https://lore.kernel.org/bpf/20240307232659.1115872-1-yonghong.song@linux.dev/ Signed-off-by: Yonghong Song <yonghong.song@linux.dev> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/bpf/20240315184854.2975190-1-yonghong.song@linux.dev (cherry picked from commit eb166e5) Signed-off-by: LFRon <ronforever@outlook.com>

deepin inclusion category: config Enable CONFIG_SCHED_CLASS_EXT=y in deepin_x86_desktop_defconfig so the sched_ext class built by this series is present in the product kernel. All Kconfig dependencies are already satisfied: CONFIG_BPF_JIT=y CONFIG_DEBUG_INFO_BTF=y CONFIG_BPF_SYSCALL=y (Kconfig default y) CONFIG_CGROUP_SCHED=y (Kconfig default y) CONFIG_EXT_GROUP_SCHED is a hidden option (default y) and is enabled automatically along with SCHED_CLASS_EXT. This is a local config-only commit, matching the existing "deepin: config:" commits on this branch; the sched_ext implementation itself is imported verbatim from upstream. Verified by generating a fresh config from this defconfig: make O=<scratch> deepin_x86_desktop_defconfig grep CONFIG_SCHED_CLASS_EXT .config => CONFIG_SCHED_CLASS_EXT=y grep CONFIG_EXT_GROUP_SCHED .config => CONFIG_EXT_GROUP_SCHED=y Signed-off-by: LFRon <ronforever@outlook.com>

A poorly behaving BPF scheduler can live-lock the system by e.g. incessantly banging on the same DSQ on a large NUMA system to the point where switching to the bypass mode can take a long time. Turning on the bypass mode requires dequeueing and re-enqueueing currently runnable tasks, if the DSQs that they are on are live-locked, this can take tens of seconds cascading into other failures. This was observed on 2 x Intel Sapphire Rapids machines with 224 logical CPUs. Inject artifical delays while the bypass mode is switching to guarantee timely completion. While at it, move __scx_ops_bypass_lock into scx_ops_bypass() and rename it to bypass_lock. Signed-off-by: Tejun Heo <tj@kernel.org> Reported-by: Valentin Andrei <vandrei@meta.com> Reported-by: Patrick Lu <patlu@meta.com> (cherry picked from commit 62dcbab) Signed-off-by: LFRon <ronforever@outlook.com>

On 2 x Intel Sapphire Rapids machines with 224 logical CPUs, a poorly behaving BPF scheduler can live-lock the system by making multiple CPUs bang on the same DSQ to the point where soft-lockup detection triggers before SCX's own watchdog can take action. It also seems possible that the machine can be live-locked enough to prevent scx_ops_helper, which is an RT task, from running in a timely manner. Implement scx_softlockup() which is called when three quarters of soft-lockup threshold has passed. The function immediately enables the ops breather and triggers an ops error to initiate ejection of the BPF scheduler. The previous and this patch combined enable the kernel to reliably recover the system from live-lock conditions that can be triggered by a poorly behaving BPF scheduler on Intel dual socket systems. Signed-off-by: Tejun Heo <tj@kernel.org> Cc: Douglas Anderson <dianders@chromium.org> Cc: Andrew Morton <akpm@linux-foundation.org> (cherry picked from commit e32c260) Signed-off-by: LFRon <ronforever@outlook.com>

…d context 4c30f5c ("sched_ext: Implement scx_bpf_dispatch[_vtime]_from_dsq()") added four kfuncs for dispatching while iterating. They are allowed from the dispatch and unlocked contexts but two of the kfuncs were only added in the dispatch section. Add missing declarations in the unlocked section. Signed-off-by: Tejun Heo <tj@kernel.org> Fixes: 4c30f5c ("sched_ext: Implement scx_bpf_dispatch[_vtime]_from_dsq()") (cherry picked from commit 72b85bf) Signed-off-by: LFRon <ronforever@outlook.com>

…time]() In sched_ext API, a repeatedly reported pain point is the overuse of the verb "dispatch" and confusion around "consume": - ops.dispatch() - scx_bpf_dispatch[_vtime]() - scx_bpf_consume() - scx_bpf_dispatch[_vtime]_from_dsq*() This overloading of the term is historical. Originally, there were only built-in DSQs and moving a task into a DSQ always dispatched it for execution. Using the verb "dispatch" for the kfuncs to move tasks into these DSQs made sense. Later, user DSQs were added and scx_bpf_dispatch[_vtime]() updated to be able to insert tasks into any DSQ. The only allowed DSQ to DSQ transfer was from a non-local DSQ to a local DSQ and this operation was named "consume". This was already confusing as a task could be dispatched to a user DSQ from ops.enqueue() and then the DSQ would have to be consumed in ops.dispatch(). Later addition of scx_bpf_dispatch_from_dsq*() made the confusion even worse as "dispatch" in this context meant moving a task to an arbitrary DSQ from a user DSQ. Clean up the API with the following renames: 1. scx_bpf_dispatch[_vtime]() -> scx_bpf_dsq_insert[_vtime]() 2. scx_bpf_consume() -> scx_bpf_dsq_move_to_local() 3. scx_bpf_dispatch[_vtime]_from_dsq*() -> scx_bpf_dsq_move[_vtime]*() This patch performs the first set of renames. Compatibility is maintained by: - The previous kfunc names are still provided by the kernel so that old binaries can run. Kernel generates a warning when the old names are used. - compat.bpf.h provides wrappers for the new names which automatically fall back to the old names when running on older kernels. They also trigger build error if old names are used for new builds. The compat features will be dropped after v6.15. v2: Documentation updates. Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: Andrea Righi <arighi@nvidia.com> Acked-by: Changwoo Min <changwoo@igalia.com> Acked-by: Johannes Bechberger <me@mostlynerdless.de> Acked-by: Giovanni Gherdovich <ggherdovich@suse.com> Cc: Dan Schatzberg <dschatzberg@meta.com> Cc: Ming Yang <yougmark94@gmail.com> (cherry picked from commit cc26abb) Signed-off-by: LFRon <ronforever@outlook.com>

In sched_ext API, a repeatedly reported pain point is the overuse of the verb "dispatch" and confusion around "consume": - ops.dispatch() - scx_bpf_dispatch[_vtime]() - scx_bpf_consume() - scx_bpf_dispatch[_vtime]_from_dsq*() This overloading of the term is historical. Originally, there were only built-in DSQs and moving a task into a DSQ always dispatched it for execution. Using the verb "dispatch" for the kfuncs to move tasks into these DSQs made sense. Later, user DSQs were added and scx_bpf_dispatch[_vtime]() updated to be able to insert tasks into any DSQ. The only allowed DSQ to DSQ transfer was from a non-local DSQ to a local DSQ and this operation was named "consume". This was already confusing as a task could be dispatched to a user DSQ from ops.enqueue() and then the DSQ would have to be consumed in ops.dispatch(). Later addition of scx_bpf_dispatch_from_dsq*() made the confusion even worse as "dispatch" in this context meant moving a task to an arbitrary DSQ from a user DSQ. Clean up the API with the following renames: 1. scx_bpf_dispatch[_vtime]() -> scx_bpf_dsq_insert[_vtime]() 2. scx_bpf_consume() -> scx_bpf_dsq_move_to_local() 3. scx_bpf_dispatch[_vtime]_from_dsq*() -> scx_bpf_dsq_move[_vtime]*() This patch performs the second rename. Compatibility is maintained by: - The previous kfunc names are still provided by the kernel so that old binaries can run. Kernel generates a warning when the old names are used. - compat.bpf.h provides wrappers for the new names which automatically fall back to the old names when running on older kernels. They also trigger build error if old names are used for new builds. The compat features will be dropped after v6.15. v2: Comment and documentation updates. Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: Andrea Righi <arighi@nvidia.com> Acked-by: Changwoo Min <changwoo@igalia.com> Acked-by: Johannes Bechberger <me@mostlynerdless.de> Acked-by: Giovanni Gherdovich <ggherdovich@suse.com> Cc: Dan Schatzberg <dschatzberg@meta.com> Cc: Ming Yang <yougmark94@gmail.com> (cherry picked from commit 5209c03) Signed-off-by: LFRon <ronforever@outlook.com>

…_move[_vtime]*() In sched_ext API, a repeatedly reported pain point is the overuse of the verb "dispatch" and confusion around "consume": - ops.dispatch() - scx_bpf_dispatch[_vtime]() - scx_bpf_consume() - scx_bpf_dispatch[_vtime]_from_dsq*() This overloading of the term is historical. Originally, there were only built-in DSQs and moving a task into a DSQ always dispatched it for execution. Using the verb "dispatch" for the kfuncs to move tasks into these DSQs made sense. Later, user DSQs were added and scx_bpf_dispatch[_vtime]() updated to be able to insert tasks into any DSQ. The only allowed DSQ to DSQ transfer was from a non-local DSQ to a local DSQ and this operation was named "consume". This was already confusing as a task could be dispatched to a user DSQ from ops.enqueue() and then the DSQ would have to be consumed in ops.dispatch(). Later addition of scx_bpf_dispatch_from_dsq*() made the confusion even worse as "dispatch" in this context meant moving a task to an arbitrary DSQ from a user DSQ. Clean up the API with the following renames: 1. scx_bpf_dispatch[_vtime]() -> scx_bpf_dsq_insert[_vtime]() 2. scx_bpf_consume() -> scx_bpf_dsq_move_to_local() 3. scx_bpf_dispatch[_vtime]_from_dsq*() -> scx_bpf_dsq_move[_vtime]*() This patch performs the third set of renames. Compatibility is maintained by: - The previous kfunc names are still provided by the kernel so that old binaries can run. Kernel generates a warning when the old names are used. - compat.bpf.h provides wrappers for the new names which automatically fall back to the old names when running on older kernels. They also trigger build error if old names are used for new builds. - scx_bpf_dispatch[_vtime]_from_dsq*() were already wrapped in __COMPAT macros as they were introduced during v6.12 cycle. Wrap new API in __COMPAT macros too and trigger build errors on both __COMPAT prefixed and naked usages of the old names. The compat features will be dropped after v6.15. Signed-off-by: Tejun Heo <tj@kernel.org> Acked-by: Andrea Righi <arighi@nvidia.com> Acked-by: Changwoo Min <changwoo@igalia.com> Acked-by: Johannes Bechberger <me@mostlynerdless.de> Acked-by: Giovanni Gherdovich <ggherdovich@suse.com> Cc: Dan Schatzberg <dschatzberg@meta.com> Cc: Ming Yang <yougmark94@gmail.com> (cherry picked from commit 5cbb302) Signed-off-by: LFRon <ronforever@outlook.com>

Returns a high-performance monotonically non-decreasing clock for the current CPU. The clock returned is in nanoseconds. It provides the following properties: 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() frequently to account for execution time and track tasks' runtime properties. Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which eventually reads a hardware timestamp counter -- is neither performant nor scalable. scx_bpf_now() aims to provide a high-performance clock by using the rq clock in the scheduler core whenever possible. 2) High enough resolution for the BPF scheduler use cases: In most BPF scheduler use cases, the required clock resolution is lower than the most accurate hardware clock (e.g., rdtsc in x86). scx_bpf_now() basically uses the rq clock in the scheduler core whenever it is valid. It considers that the rq clock is valid from the time the rq clock is updated (update_rq_clock) until the rq is unlocked (rq_unpin_lock). 3) Monotonically non-decreasing clock for the same CPU: scx_bpf_now() guarantees the clock never goes backward when comparing them in the same CPU. On the other hand, when comparing clocks in different CPUs, there is no such guarantee -- the clock can go backward. It provides a monotonically *non-decreasing* clock so that it would provide the same clock values in two different scx_bpf_now() calls in the same CPU during the same period of when the rq clock is valid. An rq clock becomes valid when it is updated using update_rq_clock() and invalidated when the rq is unlocked using rq_unpin_lock(). Let's suppose the following timeline in the scheduler core: T1. rq_lock(rq) T2. update_rq_clock(rq) T3. a sched_ext BPF operation T4. rq_unlock(rq) T5. a sched_ext BPF operation T6. rq_lock(rq) T7. update_rq_clock(rq) For [T2, T4), we consider that rq clock is valid (SCX_RQ_CLK_VALID is set), so scx_bpf_now() calls during [T2, T4) (including T3) will return the rq clock updated at T2. For duration [T4, T7), when a BPF scheduler can still call scx_bpf_now() (T5), we consider the rq clock is invalid (SCX_RQ_CLK_VALID is unset at T4). So when calling scx_bpf_now() at T5, we will return a fresh clock value by calling sched_clock_cpu() internally. Also, to prevent getting outdated rq clocks from a previous scx scheduler, invalidate all the rq clocks when unloading a BPF scheduler. One example of calling scx_bpf_now(), when the rq clock is invalid (like T5), is in scx_central [1]. The scx_central scheduler uses a BPF timer for preemptive scheduling. In every msec, the timer callback checks if the currently running tasks exceed their timeslice. At the beginning of the BPF timer callback (central_timerfn in scx_central.bpf.c), scx_central gets the current time. When the BPF timer callback runs, the rq clock could be invalid, the same as T5. In this case, scx_bpf_now() returns a fresh clock value rather than returning the old one (T2). [1] https://github.com/sched-ext/scx/blob/main/scheds/c/scx_central.bpf.c Signed-off-by: Changwoo Min <changwoo@igalia.com> Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Tejun Heo <tj@kernel.org> (cherry picked from commit 3a9910b) Signed-off-by: LFRon <ronforever@outlook.com>

scx_bpf_now() is added to the header files so the BPF scheduler can use it. Signed-off-by: Changwoo Min <changwoo@igalia.com> Acked-by: Andrea Righi <arighi@nvidia.com> Signed-off-by: Tejun Heo <tj@kernel.org> (cherry picked from commit 2e1ce39) Signed-off-by: LFRon <ronforever@outlook.com>

The BPF struct_ops previously only allowed one page of trampolines. Each function pointer of a struct_ops is implemented by a struct_ops bpf program. Each struct_ops bpf program requires a trampoline. The following selftest patch shows each page can hold a little more than 20 trampolines. While one page is more than enough for the tcp-cc usecase, the sched_ext use case shows that one page is not always enough and hits the one page limit. This patch overcomes the one page limit by allocating another page when needed and it is limited to a total of MAX_IMAGE_PAGES (8) pages which is more than enough for reasonable usages. The variable st_map->image has been changed to st_map->image_pages, and its type has been changed to an array of pointers to pages. Signed-off-by: Kui-Feng Lee <thinker.li@gmail.com> Link: https://lore.kernel.org/r/20240224223418.526631-3-thinker.li@gmail.com Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org> (cherry picked from commit 187e2af)

deepin-ci-robot added the do-not-merge/work-in-progress label Sep 24, 2026

deepin-ci-robot added the needs-ok-to-test label Sep 24, 2026

deepin-ci-robot requested review from BLumia and chenchongbiao September 24, 2026 16:27

LFRon force-pushed the linux-6.6.y-backport-sched_ext branch from 1f9a677 to 3f0e2aa Compare September 26, 2026 11:28

LFRon marked this pull request as ready for review September 27, 2026 08:24

deepin-ci-robot removed the do-not-merge/work-in-progress label Sep 27, 2026

sourcery-ai Bot reviewed Sep 27, 2026

View reviewed changes

deepin-ci-robot requested a review from opsiff September 27, 2026 08:24

LFRon force-pushed the linux-6.6.y-backport-sched_ext branch 3 times, most recently from 491e6aa to 08bda32 Compare September 30, 2026 05:53

LFRon force-pushed the linux-6.6.y-backport-sched_ext branch from 08bda32 to 71bc97f Compare October 11, 2026 09:25

LFRon changed the title ~~[linux 6.6.y] [Upstream] Backport sched_ext from linux-6.18.y~~ [linux 6.6.y] Backport sched_ext from OpenAnolis Oct 11, 2026

Avenger-285714 requested review from Avenger-285714 and Cryolitia and a balanced review from Copilot October 12, 2026 01:05

Copilot started reviewing on behalf of Avenger-285714 October 12, 2026 01:06

View session

Copilot AI reviewed Oct 12, 2026

View reviewed changes

LFRon force-pushed the linux-6.6.y-backport-sched_ext branch from 71bc97f to 5b9da57 Compare October 19, 2026 11:31

htejun added 7 commits October 22, 2026 14:23

mrpre and others added 29 commits October 22, 2026 15:36

LFRon force-pushed the linux-6.6.y-backport-sched_ext branch from 5b9da57 to 3f88320 Compare October 22, 2026 08:50

Navigation Menu

Repository navigation

[linux 6.6.y] Backport sched_ext from OpenAnolis - #2172

[linux 6.6.y] Backport sched_ext from OpenAnolis#2172
LFRon wants to merge 294 commits into
deepin-community:linux-6.6.yfrom
LFRon:linux-6.6.y-backport-sched_ext

LFRon commented Sep 24, 2026

Uh oh!

deepin-ci-robot commented Sep 24, 2026

Uh oh!

deepin-ci-robot commented Sep 24, 2026

Uh oh!

deepin-ci-robot commented Sep 24, 2026

Uh oh!

sourcery-ai Bot left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

LFRon commented Sep 24, 2026

Uh oh!

deepin-ci-robot commented Sep 24, 2026

Uh oh!

deepin-ci-robot commented Sep 24, 2026

Uh oh!

deepin-ci-robot commented Sep 24, 2026

Uh oh!

sourcery-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants