As cpu-partitioning doesn't use isolcpus=, isolated CPUs can incur
very high latency due to load balancing performed by the kernel
scheduler.
This patch introduces a new configuration option called
no_rebalance_cores=. This options allow users to set a range
of CPUs to have load balancing disabled from /proc. It is
expected that users will only disable load balancing on CPUs
which run PMD threads (or vCPU threads running PMD threads),
since disabling this for all isolated CPUs can cause certain
performance problems for OpenStack.
Here are some numbers:
- Without using no_rebalance_cores=, I can easily get 40us+
in load balancing isolated CPUs:
<...>-14140 [001] 1656.121155: funcgraph_entry: + 43.927 us | rebalance_domains();
<...>-14140 [001] 1660.117150: funcgraph_entry: + 41.587 us | rebalance_domains();
<...>-14140 [001] 1658.119152: funcgraph_entry: + 41.565 us | rebalance_domains();
<...>-14140 [001] 1668.110140: funcgraph_entry: + 40.539 us | rebalance_domains();
- When using no_rebalance_cores=, the maximum latency I
get is ~ 1us:
<...>-2440 [012] 1705.078076: funcgraph_entry: 1.024 us | rebalance_domains();
<...>-2438 [008] 2021.065218: funcgraph_entry: 0.995 us | rebalance_domains();
<...>-2447 [010] 2223.057289: funcgraph_entry: 0.991 us | rebalance_domains();
IMPORTANT: 10 minutes run only.
This solution was developed by Andrew Theurer <atheurer@redhat.com>,
I'm merely integrating it into the profile.
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
b39c2af9db