In the cpu-partitioning profile, IRQ affinity change is already done
as part of the 'isolated_cores' option of the scheduler plugin, so
calling defirqaffinity is, at best, redundant. So let's remove the call.
In the realtime* profiles, it's essentially the same, except that tuna
was used instead of isolated_cores. So let's use built-in functionality
instead of tuna and drop calls to defirqaffinity.
Resolves: rhbz#1590937
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
Previously the irqbalance config was not modified if setting the affinity
of at least one IRQ failed. That does not make much sense to me - it can
result in irqbalance assigning isolated CPUs to IRQs, even if it can be
prevented.
Even if affinity cannot be changed for any of the present IRQs,
hypothetically it can happen (I think) that new hardware is hotplugged,
and the affinity of its IRQ can be changed. So let's always modify the
irqbalance config for good measure.
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
Also fixed cpu-partitioning profile script not to break irqbalance
configuration file in case of defirqaffinity failure.
Resolves: rhbz#1536476
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
Helper state file for cpu-partitioning script was moved to /run/tuned.
Code for flags manipulation has been simplified.
The /run/tuned directory is explicitly cleared upon uninstallation
of Tuned to get rid of all leftovers / helper files before the reboot.
Added STORAGE_PERSISTENT variable pointing to /var/lib/tuned into
functions to be consistent.
Related: rhbz#1497182
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
As cpu-partitioning doesn't use isolcpus=, isolated CPUs can incur
very high latency due to load balancing performed by the kernel
scheduler.
This patch introduces a new configuration option called
no_rebalance_cores=. This options allow users to set a range
of CPUs to have load balancing disabled from /proc. It is
expected that users will only disable load balancing on CPUs
which run PMD threads (or vCPU threads running PMD threads),
since disabling this for all isolated CPUs can cause certain
performance problems for OpenStack.
Here are some numbers:
- Without using no_rebalance_cores=, I can easily get 40us+
in load balancing isolated CPUs:
<...>-14140 [001] 1656.121155: funcgraph_entry: + 43.927 us | rebalance_domains();
<...>-14140 [001] 1660.117150: funcgraph_entry: + 41.587 us | rebalance_domains();
<...>-14140 [001] 1658.119152: funcgraph_entry: + 41.565 us | rebalance_domains();
<...>-14140 [001] 1668.110140: funcgraph_entry: + 40.539 us | rebalance_domains();
- When using no_rebalance_cores=, the maximum latency I
get is ~ 1us:
<...>-2440 [012] 1705.078076: funcgraph_entry: 1.024 us | rebalance_domains();
<...>-2438 [008] 2021.065218: funcgraph_entry: 0.995 us | rebalance_domains();
<...>-2447 [010] 2223.057289: funcgraph_entry: 0.991 us | rebalance_domains();
IMPORTANT: 10 minutes run only.
This solution was developed by Andrew Theurer <atheurer@redhat.com>,
I'm merely integrating it into the profile.
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
KSM is enabled by default when the qemu-kvm-common-rhev package is
installed. We don't want KSM to run in KVM-RT nor cpu-partitioning
hosts, as it can cause spikes or packet drops.
This series disables KSM in those profiles when they are activated.
Likewise, we re-enable KSM when the profile is de-activated in
case it's enabled by default in systemd.
Luiz Capitulino (3):
profiles: functions: add functions to disable/enable KSM
realtime-virtual-host: disable KSM
cpu-partitioning: disable KSM
profiles/cpu-partitioning/script.sh | 2 ++
profiles/functions | 37 ++++++++++++++++++++++++++++++++
profiles/realtime-virtual-host/script.sh | 3 +++
3 files changed, 42 insertions(+)
--
2.9.5
Return-Path: lcapitulino@redhat.com
Received: from zmta04.collab.prod.int.phx2.redhat.com (LHLO
zmta04.collab.prod.int.phx2.redhat.com) (10.5.81.11) by
zmail22.collab.prod.int.phx2.redhat.com with LMTP; Tue, 3 Oct 2017 15:54:02
-0400 (EDT)
Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
by zmta04.collab.prod.int.phx2.redhat.com (Postfix) with ESMTP id 46F2FD0310
for <jskarvad@mail.corp.redhat.com>; Tue, 3 Oct 2017 15:54:02 -0400 (EDT)
Received: by smtp.corp.redhat.com (Postfix)
id 3FBC860F82; Tue, 3 Oct 2017 19:54:02 +0000 (UTC)
Delivered-To: jskarvad@redhat.com
Received: from localhost (ovpn-117-159.phx2.redhat.com [10.3.117.159])
by smtp.corp.redhat.com (Postfix) with ESMTP id 3A51A60F81;
Tue, 3 Oct 2017 19:53:59 +0000 (UTC)
From: Luiz Capitulino <lcapitulino@redhat.com>
To: jskarvad@redhat.com
Cc: kvm-rt@redhat.com
Subject: [PATCH 1/3] profiles: functions: add functions to disable/enable KSM
Date: Tue, 3 Oct 2017 15:53:50 -0400
Message-Id: <20171003195352.22910-2-lcapitulino@redhat.com>
In-Reply-To: <20171003195352.22910-1-lcapitulino@redhat.com>
References: <20171003195352.22910-1-lcapitulino@redhat.com>
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12
They will be used by the next commits.
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Users can switch to Tuned if they need process blacklisting
(i.e. marking of processes which are allowed to run on
isolated cores). This will be very probably made the default
in next Tuned release.
Related: rhbz#1403309
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
Plugin scheduler now can do cores isolation on its own, Tuna is
not needed for it, e.g.:
[scheduler]
isolated_cores=2-4
ps_blacklist=.*pmd.*;.*PMD.*;^DPDK;.*qemu-kvm.*
It will isolate cores 2-4, it will ignore processes which matched
ps_blacklist regexes. Multiple regexes can be separated by ';'.
Quoted semicolon, i.e. '\;' is taken literally.
It also supports 'ps_whitelist' which is by default set to '.*'.
It takes all processes which matches ps_whitelist than it removes
those which matches ps_blacklist and move them out of the
isolated_cores. When the profile is unloaded it allows all matching
processes to run on all cores.
It changes processes affinities, threads affinities, IRQs affinities
and it sets default_smp_affinity for IRQs.
Also cpu-partitioning profile has been switched from Tuna to this
mechanism.
Resolves: rhbz#1403309
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
With systemd full rollback is done if Tuned is terminated by user or profile
is changed. It is not done if Tuned is terminated due to system
shutdown / reboot. Without systemd full rollback is done only if profile
is changed.
Renamed profile_switch parameter to full_rollback where appropriate.
Also fixed various checks for command not found.
Resolves: rhbz#1421286
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
This adds 00-tuned-pre-udev.sh script into initrd image which is getting
tuned.cpumask kernel command line parameter and sets workqueue affinity
accordingly.
Resolves: rhbz#1395899
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
Otherwise, a process running on isolated CPUs will be
interrupted by timers or unwanted VM exists.
Resolves: rhbz#1395855
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
The profile is delivered in the new tuned-profiles-cpu-partitioning subpackage.
Also moved defirqaffinity.py to the base package to allow other profiles
to use it.
Resolves: rhbz#1359956
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
