nagai/kernel
3
0
Fork 0
forked from mirrors/linux
Code Pull requests Activity

cgroup: Avoid false cacheline sharing of read mostly rstat_cpu

Browse source 
The rstat_cpu and also rstat_css_list of the cgroup structure are read
mostly variables. However, they may share the same cacheline as the
subsequent rstat_flush_next and *bstat variables which can be updated
frequently.  That will slow down the cgroup_rstat_cpu() call which is
called pretty frequently in the rstat code. Add a CACHELINE_PADDING()
line in between them to avoid false cacheline sharing.

A parallel kernel build on a 2-socket x86-64 server is used as the
benchmarking tool for measuring the lock hold time. Below were the lock
hold time frequency distribution before and after the patch:

      Run time        Before patch       After patch
      --------        ------------       -----------
       0-01 us         9,928,562          9,820,428
      01-05 us           110,151             50,935
      05-10 us               270                 93
      10-15 us               273                146
      15-20 us               135                 76
      20-25 us                 0                  2
      25-30 us                 1                  0

It can be seen that the patch further pushes the lock hold time towards
the lower end.

Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
This commit is contained in:
Waiman Long 2023-11-30 15:43:27 -05:00 committed by Tejun Heo
parent d499fd418f
commit 77070eeb88
1 changed files with 7 additions and 0 deletions
repo.diff.show_diff_stats Download patch file Download diff file Expand all files Collapse all files

7
include/linux/cgroup-defs.h
View file

@ -496,6 +496,13 @@ struct cgroup {
struct cgroup_rstat_cpu __percpu *rstat_cpu;
struct list_head rstat_css_list;
/*
* Add padding to separate the read mostly rstat_cpu and
* rstat_css_list into a different cacheline from the following
* rstat_flush_next and *bstat fields which can have frequent updates.
*/
CACHELINE_PADDING(_pad_);
/*
* A singly-linked list of cgroup structures to be rstat flushed.
* This is a scratch field to be used exclusively by