Most scheduler tuning advice on the internet is cargo cult. Numbers get copied between blog posts until nobody remembers which kernel version they applied to. Here is the short version: measure first, and touch three things at most.
1. Find out whether you are actually waiting on the CPU
Pressure Stall Information answers this directly, and it takes one file read:
cat /proc/pressure/cpu
some avg10=0.42 avg60=0.38 total=9182736If avg10 sits near zero while your service still misses its latency budget, the scheduler is not your problem. Look at I/O or at a lock.
2. Isolation beats tuning
The single most effective change for a latency-sensitive process is giving it CPUs nobody else can use. Not a priority — a reservation:
# reserve cores 2-3 for one service
sudo systemctl set-property api.service AllowedCPUs=2-3
sudo systemctl set-property system.slice AllowedCPUs=0-1Two lines, no kernel command line changes, and it survives a reboot because it is declarative.
3. The knobs worth knowing
- sched_min_granularity_ns — the floor on a timeslice. Raising it favours throughput and hurts interactivity.
- sched_wakeup_granularity_ns — how much better a waking task must be before it preempts. Lower is more responsive and more context switches.
- sched_migration_cost_ns — how long a task is considered cache-hot. Raise it if perf shows migrations dominating.
Change one, run the workload, and record the p99 with the value in the commit message. A tuning change without a before-and-after number is a guess that outlived the person who made it.