Before the `scheduler` plugin, `tuna` was used for CPU isolation.
There is a leftover use in the verification of the `realtime` profile.
Removing that, the dependency on the `tuna` package, and now-empty
scripts from the realtime profiles.
closes#594
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
Just explain what isolated_cores= is for and recommend to only have one
isolated_cores= line to avoid wrong system configuration.
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
The ktimer_lockless_check interface was removed from Red Hat kernels.
Drop the tuned configuration as well.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
The PREEMPT_RT upstream tree [1] is reintroducing a per-CPU kthread for
handling timer expiry named ktimers/${cpu}, and RHEL-RT is following this
approach [2].
The recommended priority tuning for that kthread is to simply match that of
ksoftirqd, which is what is implemented here.
Note that ktimersoftd is the predecessor of ktimers, and isn't present on
RT kernels newer than v5.0.19-rt11 / RHEL-RT 7.
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=2122220
Resolves: rhbz#2122220
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
The calc_isolated_cores function expands to the list of cores to
isolate. It accepts optional argument which specifies how many cores
from each socket reserve for housekeeping. If not specified, one core
from each socket is reserved for housekeeping.
Example:
Machine with 2 sockets, each 4 cores, using the following user variable
configuration file, e.g. /etc/tuned/realtime-variables.conf:
isolated_cores=${f:calc_isolated_cores:2}
It will expand to:
isolated_cores=2, 3, 6, 7
I.e. cores 0, 1 and 4, 5 will be used for housekeeping.
Also updated the TuneD profiles to use this builtin function for
isolated_cores preset.
Resolves: rhbz#2093847
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
* add "^\[" to the start of the regexes, so they only match
the intended kernel threads, and not processes that have
the name of a kthread in their cmdline.
* remove unneccessary ".*" at start/end of regexes
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
The 'nx_huge_pages' option, will force any executable page mapping to be
performed at 4KiB granularity and requires any existing overlapping
huge-page mapping to be also split in 4KiB chunks[1]. This is due to a HW
bug that doesn't affect our NFV use-cases.
The way a huge-page mapping is dropped is by simply deleting the EPT
entry and flushing TLB caches on all affected CPUs. Next vCPU access to
that address will trigger an EPT_VIOLATION, which will jump into the
host for it to handle the page-fault. In our specific case, this means
all isolated CPUs running oslat will hit an EPT_VIOLATION almost exactly
at the same time. Which is bad enough already, but, given our systems
might have a huge number of isolated CPUs, will also create a lot of
contention over the KVM MMU lock. This has been observed to trigger
~100us latency spikes while testing with oslat.
So let's disable the 'kvm.nx_huge_pages' module option.
[1] see kernel commit b7e8c8303ff28
Resolves: rhbz#1976825
Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
I/O APIC, one of intel's interrupt controllers, can only change the
affinity on an IRQ line during an EOI (end of interrupt)[1]. In other
words, on boot, although tuned will set a preferred IRQ affinity, it
might not be applied until the next time the interrupt is triggered.
This might be right away, or hours later while oslat or a
latency-sensitive workload is running.
To mitigate this set the 'irqaffinity='kernel command-line[2] option to
match the non-isolated CPU mask when running the realtime-virtual-*
profiles. This way we'll get the right affinity regardless of the
situation stated above.
[1] For more reference see kernel's ioapic_ack_level() function and the
IRQD_SETAFFINITY_PENDING flag.
[2] Actually, this is why irqaffinity was introduced in the first place,
see kernel commit fbf198030e0b0.
Resolves: rhbz#1974820
Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Enable 'isolate_managed_irq' parameter by default. It is important
for -realtime systems. When it is enabled, kernel directs interrupts
to the housekeeping CPUs and thus reduces latencies for the isolated
ones.
Fixes: RHBZ#1820626
For realtime KVM host and guests it is important to use housekeeping
CPUs for processing interrupts (IRQs) and network packet transmission.
So that isolated CPUs are used for -realtime tasks assigned to them.
The 'isolate_managed_irq' and 'netdev_queue_count' variables help
to configure kernels towards this end. Users can define these
variables via tuned(8) configuration files.
Fixes: RHBZ#1817827
tuned(8) realtime-virtual-{host|guest} profiles set Kernel and
KVM module parameters via script.sh plugin. These parameters are
to be verified in a verify() function, invoked by tuned-adm verify
command.
This patch updates verify() functions to validate KVM module
parameters.
It moves kernel parameters to the tuned.conf file under the
[sysfs] & [sysctl] plugin sections. And removes call to
disable_ksm function, no longer required.
Fixes: RHBZ#1947858
Like the cpu-partitioning profile, exclude all 'pmd' threads
to avoid repinning ovs-vswitchd pmds.
Resolves: rhbz#1861767
Signed-off-by: Christophe Fontaine <cfontain@redhat.com>
The LAPIC advancement improvement is minimal (3 or 4 us) and its
calculation has shown to be problematic under certain
scenarios (for example if qemu-kvm is not installed).
Remove it.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
RHEL will include a backport of the kernel patch
"do not raise timer softirq unconditionally (spinlockless version)"
(https://lkml.org/lkml/2019/4/15/1215), but disabled by default.
Enable it in realtime-virtual-{host/guest} profiles as it decreases
maximum cyclictest latency.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
So that QEMU uses the in kernel irqchip.
Original patch provided by Luiz Capitulino <lcapitulino@redhat.com>.
Resolves: rhbz#1554458
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
Disabling the ksm and ksmtuned services during boot seems to cause
problems, so do it only once when the profile is first applied and
reenable the services only when full rollback is required.
Resolves: rhbz#1622239
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
The rt-entsk application, part of newer rt-setup packages,
worksaround a latency issue with static key
IPI's. What it does it:
/*
* Open a socket, and enable timestamping on it.
*
* This is to avoid Chrony from changing timestamping
* user count from 0->1 and vice-versa, causing
* static key enable/disable IPIs.
*
*/
Start/stop the systemctl service from the realtime-virtual-host
and realtime-virtual-guest profiles.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
As noted in the bugzilla ticket
https://bugzilla.redhat.com/show_bug.cgi?id=1554851
The QEMU I/O thread can interrupt the time measurement
of the timer. To avoid this problem, only
pin the vCPU thread.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
Reviewed-and-Tested-by: Luiz Capitulino <lcapitulino@redhat.com>
v2:
- Use unix sockets (Luiz)
- Proper numeric output (Luiz)
In the cpu-partitioning profile, IRQ affinity change is already done
as part of the 'isolated_cores' option of the scheduler plugin, so
calling defirqaffinity is, at best, redundant. So let's remove the call.
In the realtime* profiles, it's essentially the same, except that tuna
was used instead of isolated_cores. So let's use built-in functionality
instead of tuna and drop calls to defirqaffinity.
Resolves: rhbz#1590937
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
The code was completely broken. Jaroslav and I think the purpose
of the code was to check that the files
/sys/module/kvm/parameters/kvmclock_periodic_sync
/sys/module/kvm_intel/parameters/ple_gap
contain "0", which we think is the (at least intended) result of
profiles/functions:setup_kvm_mod_low_latency(). So I'm fixing it
to do that.
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
script.sh has been using variable names in different ways,
in different scopes: when using variables at all. Fix all that.
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Profile loading is broken since commit b532e9291d, due to
a small typo in the new function name.
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
KSM is enabled by default when the qemu-kvm-common-rhev package is
installed. We don't want KSM to run in KVM-RT nor cpu-partitioning
hosts, as it can cause spikes or packet drops.
This series disables KSM in those profiles when they are activated.
Likewise, we re-enable KSM when the profile is de-activated in
case it's enabled by default in systemd.
Luiz Capitulino (3):
profiles: functions: add functions to disable/enable KSM
realtime-virtual-host: disable KSM
cpu-partitioning: disable KSM
profiles/cpu-partitioning/script.sh | 2 ++
profiles/functions | 37 ++++++++++++++++++++++++++++++++
profiles/realtime-virtual-host/script.sh | 3 +++
3 files changed, 42 insertions(+)
--
2.9.5
Return-Path: lcapitulino@redhat.com
Received: from zmta04.collab.prod.int.phx2.redhat.com (LHLO
zmta04.collab.prod.int.phx2.redhat.com) (10.5.81.11) by
zmail22.collab.prod.int.phx2.redhat.com with LMTP; Tue, 3 Oct 2017 15:54:02
-0400 (EDT)
Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
by zmta04.collab.prod.int.phx2.redhat.com (Postfix) with ESMTP id 46F2FD0310
for <jskarvad@mail.corp.redhat.com>; Tue, 3 Oct 2017 15:54:02 -0400 (EDT)
Received: by smtp.corp.redhat.com (Postfix)
id 3FBC860F82; Tue, 3 Oct 2017 19:54:02 +0000 (UTC)
Delivered-To: jskarvad@redhat.com
Received: from localhost (ovpn-117-159.phx2.redhat.com [10.3.117.159])
by smtp.corp.redhat.com (Postfix) with ESMTP id 3A51A60F81;
Tue, 3 Oct 2017 19:53:59 +0000 (UTC)
From: Luiz Capitulino <lcapitulino@redhat.com>
To: jskarvad@redhat.com
Cc: kvm-rt@redhat.com
Subject: [PATCH 1/3] profiles: functions: add functions to disable/enable KSM
Date: Tue, 3 Oct 2017 15:53:50 -0400
Message-Id: <20171003195352.22910-2-lcapitulino@redhat.com>
In-Reply-To: <20171003195352.22910-1-lcapitulino@redhat.com>
References: <20171003195352.22910-1-lcapitulino@redhat.com>
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12
They will be used by the next commits.
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Tuna is already executed in the realtime profile which is included by
the realtime-virtual-host/guest profiles, there is no need to execute
it twice. Also fixed minor whitespace inconsistency.
Related: rhbz#1472840
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
With systemd full rollback is done if Tuned is terminated by user or profile
is changed. It is not done if Tuned is terminated due to system
shutdown / reboot. Without systemd full rollback is done only if profile
is changed.
Renamed profile_switch parameter to full_rollback where appropriate.
Also fixed various checks for command not found.
Resolves: rhbz#1421286
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
This doesn't have effect on kernels build with CONFIG_RCU_NOCB_CPU_ALL=y.
Resolves: rhbz#1334479
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
We create the /etc/modprobe.d/kvm.rt.tuned.conf file at
every boot. However, there's nothing that guarantees that
the kvm module will be loaded after tuned. Indeed, it's
possible to reproduce a race where the kvm module is loaded
before tuned. When this happens, the kvm module will be
loaded with incorrect parameters as the kvm.rt.tuned.conf
file doesn't exist.
The solution to this problem is to create the kvm.rt.tuned.conf
file only if it doesn't exist and delete it when changing
profiles.
Resolves: rhbz#1292117
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
profiles/realtime-virtual-host/script.sh | 26 +++++++++++++++++---------
1 file changed, 17 insertions(+), 9 deletions(-)
Added new tuned-adm functionality 'profile_info'. It has optional parameter
'profile_name'. If called without parameter, active profile is used.
It outputs name of the profile it is querying, profile summary, and profile
description. If there is no profile summary or description available it
outputs empty string(s).
Extended tuned-adm 'list' functionality. It now displays profile summary
in the list (if available).
Added summary to profiles.
Extended D-Bus API to support the new feature and for better Cockpit support:
- Added 'profile_info' method. It gets profile_name argument and it queries
the requested profile for information. If profile_name is empty string it
queries the active/selected profile. It returns tuple of the following
format (status, profile_name, summary, description), where status is
boolean. If it is False, the query mechanism failed and the other fields
don't contain valid data
- Added 'profiles2' method. It's extension of 'profiles' method. It returns
list of tuples. Tuples have following format:
(profile_name, profile_summary)
Related: rhbz#1228356
Fixes: #46
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
