如何屏蔽Linux调度程序中的CPU(防止将线程调度到该CPU上)?
可以使用sched_setaffinity
将线程固定到cpu,从而提高性能(在某些情况下)
It is possible to use sched_setaffinity
to pin a thread to a cpu, increasing performance (in some situations)
从linux手册页:
限制进程在单个CPU上运行也可以避免 性能损失是由以下情况导致的缓存失效引起的: 进程停止在一个CPU上执行,然后重新开始执行 一个不同的CPU
Restricting a process to run on a single CPU also avoids the performance cost caused by the cache invalidation that occurs when a process ceases to execute on one CPU and then recommences execution on a different CPU
此外,如果我希望获得更实时的响应,则可以将该线程的调度程序策略更改为SCHED_FIFO
,并将优先级提高到某个较高的值(最大为sched_get_priority_max
),这意味着有问题的线程准备就绪时,应始终抢占其CPU上运行的任何其他线程.
Further, if I desire a more real-time response, I can change the scheduler policy for that thread to SCHED_FIFO
, and up the priority to some high value (up to sched_get_priority_max
), meaning the thread in question should always pre-empt any other thread running on its cpu when it becomes ready.
但是,在这一点上,运行在CPU上的线程可能已经驱除了大部分实时线程的1级缓存条目.
However, at this point, the thread running on the cpu which the real-time thread just pre-empted will possibly have evicted much of the real-time thread's level-1 cache entries.
我的问题如下:
- 是否可以防止调度程序将任何线程调度到给定的cpu上? (例如:要么完全从调度程序中隐藏CPU,要么以其他方式隐藏)
- 是否有一些绝对必须能够在该CPU上运行的线程? (例如:内核线程/中断线程)
- 如果我需要在该CPU上运行内核线程,那么要使用什么合理的最大优先级值,这样我才不会饿死内核线程?
答案是使用 python cpuset实用程序,可以轻松配置它们.
The answer is to use cpusets. The python cpuset utility makes it easy to configure them.
基本概念
3个cpusets
-
root
:存在于所有配置中,并包含所有cpus(非屏蔽) -
system
:包含用于系统任务的cpus-需要运行但不是重要"( unshielded )的 -
user
:包含用于重要"任务的cpus-我们要在实时"模式(屏蔽) 中运行的cpus
-
root
: present in all configurations and contains all cpus (unshielded) -
system
: contains cpus used for system tasks - the ones which need to run but aren't "important" (unshielded) -
user
: contains cpus used for "important" tasks - the ones we want to run in "realtime" mode (shielded)
shield
命令管理这3个cpuset.
The shield
command manages these 3 cpusets.
在设置过程中,它将所有可移动任务移动到非屏蔽cpuset(system
)中,在拆卸期间,将所有可移动任务移动到root
cpuset中.
设置后,该子命令使您可以将任务移至 shield (user
)cpuset中,此外,还可以将特殊任务(内核线程)从root
移至system
(因此移出user
cpuset).
During setup it moves all movable tasks into the unshielded cpuset (system
) and during teardown it moves all movable tasks into the root
cpuset.
After setup, the subcommand lets you move tasks into the shield (user
) cpuset, and additionally, to move special tasks (kernel threads) from root
to system
(and therefore out of the user
cpuset).
命令:
首先,我们创建一个盾牌.自然,防护罩的布局将取决于机器/任务.例如,假设我们有一台4核非NUMA机器:我们想将 3核专用于屏蔽层,而将 1核用于不重要的任务;因为它不是NUMA,所以我们不需要指定任何内存节点参数,并且让内核线程在root
cpuset中运行(即:跨所有cpus)
First we create a shield. Naturally the layout of the shield will be machine/task dependent. For example, say we have a 4-core non-NUMA machine: we want to dedicate 3 cores to the shield, and leave 1 core for unimportant tasks; since it is non-NUMA we don't need to specify any memory node parameters, and we leave the kernel threads running in the root
cpuset (ie: across all cpus)
$ cset shield --cpu 1-3
某些内核线程(那些未绑定到特定CPU的线程)可以移入system
cpuset. (通常,移动已绑定到特定CPU的内核线程不是一个好主意)
Some kernel threads (those which aren't bound to specific cpus) can be moved into the system
cpuset. (In general it is not a good idea to move kernel threads which have been bound to a specific cpu)
$ cset shield --kthread on
现在让我们列出屏蔽(user
)或非屏蔽(system
)cpusets中正在运行的内容:(-v
表示详细信息,它将列出进程名称)(添加第二个-v
以显示更多内容80个字符)
Now let's list what's running in the shield (user
) or unshielded (system
) cpusets: (-v
for verbose, which will list the process names) (add a 2nd -v
to display more than 80 characters)
$ cset shield --shield -v
$ cset shield --unshield -v -v
如果我们要停止屏蔽(拆卸)
If we want to stop the shield (teardown)
$ cset shield --reset
现在让我们在屏蔽中执行一个过程('--'
之后的命令传递给要执行的命令,而不传递给cset
)
Now let's execute a process in the shield (commands following '--'
are passed to the command to be executed, not to cset
)
$ cset shield --exec mycommand -- -arg1 -arg2
如果我们已经有一个正在运行的进程想要移入屏蔽(请注意,我们可以通过传递逗号分隔的列表或范围来移动多个进程(即使存在间隙,该范围内的任何进程也会移动) )
If we already have a running process which we want to move into the shield (note we can move multiple processes by passing a comma separated list, or ranges (any process in the range will be moved, even if there are gaps))
$ cset shield --shield --pid 1234
$ cset shield --shield --pid 1234,1236
$ cset shield --shield --pid 1234,1237,1238-1240
高级概念
cset set/proc
-这些使您可以更好地控制cpusets
cset set/proc
- these give you finer control of cpusets
设置
创建,调整,重命名,移动和销毁cpusets
Create, adjust, rename, move and destroy cpusets
命令
使用cpus 1-3创建一个cpuset,使用NUMA节点1并将其命名为"my_cpuset1"
Create a cpuset, using cpus 1-3, use NUMA node 1 and call it "my_cpuset1"
$ cset set --cpu=1-3 --mem=1 --set=my_cpuset1
将"my_cpuset1"更改为仅使用cpus 1和cpus
Change "my_cpuset1" to only use cpus 1 and 3
$ cset set --cpu=1,3 --mem=1 --set=my_cpuset1
销毁一个cpuset
Destroy a cpuset
$ cset set --destroy --set=my_cpuset1
重命名现有的cpuset
Rename an existing cpuset
$ cset set --set=my_cpuset1 --newname=your_cpuset1
创建分层的cpuset
Create a hierarchical cpuset
$ cset set --cpu=3 --mem=1 --set=my_cpuset1/my_subset1
列出现有的CPU集(级别1的深度)
List existing cpusets (depth of level 1)
$ cset set --list
列出现有的cpuset及其子对象
List existing cpuset and its children
$ cset set --list --set=my_cpuset1
列出所有现有的cpusets
List all existing cpusets
$ cset set --list --recurse
过程
管理线程和进程
命令
列出在cpuset中运行的任务
List tasks running in a cpuset
$ cset proc --list --set=my_cpuset1 --verbose
在cpuset中执行任务
Execute a task in a cpuset
$ cset proc --set=my_cpuset1 --exec myApp -- --arg1 --arg2
执行任务
$ cset proc --toset=my_cpuset1 --move --pid 1234
$ cset proc --toset=my_cpuset1 --move --pid 1234,1236
$ cset proc --toset=my_cpuset1 --move --pid 1238-1340
移动任务及其所有同级兄弟
Moving a task and all its siblings
$ cset proc --move --toset=my_cpuset1 --pid 1234 --threads
将所有任务从一个cpuset移至另一个
Move all tasks from one cpuset to another
$ cset proc --move --fromset=my_cpuset1 --toset=system
将未固定的内核线程移动到cpuset
Move unpinned kernel threads into a cpuset
$ cset proc --kthread --fromset=root --toset=system
强行将内核线程(包括固定到特定cpu的线程)移到cpuset中(注意:这可能对系统造成可怕的后果-确保您知道自己在做什么)
Forcibly move kernel threads (including those that are pinned to a specific cpu) into a cpuset (note: this may have dire consequences for the system - make sure you know what you're doing)
$ cset proc --kthread --fromset=root --toset=system --force
层次结构示例
我们可以使用分层cpusets创建优先分组
We can use hierarchical cpusets to create prioritised groupings
- 使用1个cpu(0)创建一个
system
cpuset - 用1个cpu(1)创建一个
prio_low
cpuset - 创建一个具有2 cpus(1-2)的
prio_met
cpuset - 创建一个具有3 cpus(1-3)的
prio_high
cpuset - 创建一个包含所有4个cpus(0-3)的
prio_all
cpuset(请注意与root相同;保持与root的分隔是一种很好的做法)
- Create a
system
cpuset with 1 cpu (0) - Create a
prio_low
cpuset with 1 cpu (1) - Create a
prio_met
cpuset with 2 cpus (1-2) - Create a
prio_high
cpuset with 3 cpus (1-3) - Create a
prio_all
cpuset with all 4 cpus (0-3) (note this the same as root; it is considered good practice to keep a separation from root)
要实现上述目的,请创建prio_all,然后在prio_all下创建子集prio_high,等等
To achieve the above you create prio_all, and then create subset prio_high under prio_all, etc
$ cset set --cpu=0 --set=system
$ cset set --cpu=0-3 --set=prio_all
$ cset set --cpu=1-3 --set=/prio_all/prio_high
$ cset set --cpu=1-2 --set=/prio_all/prio_high/prio_med
$ cset set --cpu=1 --set=/prio_all/prio_high/prio_med/prio_low