1
0
Fork 0
Commit graph

1090 commits

Author SHA1 Message Date
Ondřej Lysoněk
98254efae0 scheduler: Refactor
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-07-02 23:14:41 +02:00
Ondřej Lysoněk
c170b69602 realtime-virtual-host: Fix verification in script.sh
The code was completely broken. Jaroslav and I think the purpose
of the code was to check that the files
/sys/module/kvm/parameters/kvmclock_periodic_sync
/sys/module/kvm_intel/parameters/ple_gap
contain "0", which we think is the (at least intended) result of
profiles/functions:setup_kvm_mod_low_latency(). So I'm fixing it
to do that.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-07-02 23:14:34 +02:00
Ondřej Lysoněk
f733e6e666 scheduler: isolated_cores: Fix checking CPUs are valid
Fix checking that the list of CPUs specified in isolated_cores
contains only CPUs that are present on the machine.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-07-02 23:14:11 +02:00
Jaroslav Škarvada
91c89db361
new release (2.10.0-rc.1)
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
2018-06-11 20:42:24 +02:00
Jaroslav Škarvada
f6777c189b
Merge pull request #105 from olysonek/tuned_gui
tuned-gui: Fix a traceback
2018-06-11 18:29:43 +02:00
Jaroslav Škarvada
8d7cb858f2
Merge pull request #104 from olysonek/log_to_tuned_adm
Print log excerpt in tuned-adm when changing the profile
2018-06-11 18:28:57 +02:00
Ondřej Lysoněk
0a37ff6373 tuned-adm: Don't print superfluous newline after log messages
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-11 14:40:42 +02:00
Ondřej Lysoněk
c37b6220d4 tuned-gui: Fix a traceback
Resolves: rhbz#1536835
Fixes #95

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-11 09:59:57 +02:00
Ondřej Lysoněk
6c1df704ce tuned-adm: Print log excerpt when changing profile
Print messages logged during a profile switch to stderr (this affects
the "profile" and "auto_profile" commands). By default messages with
log level ERROR and higher are printed. This can be changed using the
--loglevel command line option. Valid values are debug, info, warn,
error, console, none ('none' can be used to disable the log printing).
E.g.:
tuned-adm --loglevel info profile powersave

Log printing cannot be used when --async is used.

Resolves: rhbz#1538745

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-11 09:56:14 +02:00
Ondřej Lysoněk
829c7de36c Support sending logs over DBus
Add two DBus calls:
log_capture_start(log_level, timeout)
 * This will instruct Tuned to create a new log handler, which will
   start writing log messages with level 'log_level' and higher to
   a buffer. The log levels are standard Python log levels (e.g.
   logging.INFO) or Tuned's custom log level LOG_LEVEL_CONSOLE which
   is defined as 60. The handler will be destroyed after 'timeout' seconds
   if contents of the buffer are not collected using log_capture_finish()
   before the timeout. If 'timeout' <= 0, log messages will be collected
   for as long as it takes before log_capture_finish() is called (use
   with care, so that you don't fill up memory with Tuned logs). This
   call returns a string ID of the log handler, a token, which should
   be passed to log_capture_finish().

log_capture_finish(token)
 * This will return (as a string) log messages collected by a log handler
   associated with the 'token'. It will also destroy the log handler.

These calls are privileged. They can only be called by the root user or
by a user logged in on a local console (just like the change-profile DBus
calls). This restriction is meant to prevent ordinary users from forcing
Tuned to allocate an insane amount of memory for the log buffers and crash
it (or other processes). In the future we may wish to replace this
restriction with a configurable policy which determines how many (and how
big) log buffers can a user with a given UID create. We may also want to
destroy the log handler when the original caller disconnects from DBus.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-11 09:23:20 +02:00
Ondřej Lysoněk
1be86c857a systemd: Change log level of message about running dracut
Change the log level of the message about running dracut to CONSOLE.
It is an important message which should always be logged. It informs
the user that they may be required to do something.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-11 08:21:13 +02:00
Ondřej Lysoněk
bd91f06d17 log: Introduce log level CONSOLE
This level has the highest severity. It is meant for important messages
about situations which may require user intervention. These messages
will be shown to the user on the console when running tuned-adm (this is
implemented in a followup commit).

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-11 08:13:23 +02:00
Ondřej Lysoněk
6473d2082b Drop TunedAdminException, it's not raised anywhere
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-11 08:10:07 +02:00
Jaroslav Škarvada
64517b7785
oracle: updated the profile to be in sync with KCS 39188
Resolves: rhbz#1447323

Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
2018-06-06 17:42:54 +02:00
Jaroslav Škarvada
4fa1b8a4bb
profiles: added mssql profile
Resolves: rhbz#1442122

Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
2018-06-05 17:55:34 +02:00
Jaroslav Škarvada
097db409d3
Merge pull request #103 from olysonek/bootloader
Bootloader plugin fixes
2018-06-05 17:39:17 +02:00
Jaroslav Škarvada
028240d565
Merge pull request #102 from olysonek/scheduler
Rework and improve the scheduler plugin
2018-06-05 16:48:35 +02:00
Ondřej Lysoněk
70d73ed639 bootloader: Patch all GRUB2 config files
Patch all existing GRUB2 config files, not just the first in order.

Based on a patch from Ryan Blakley <rblakley@redhat.com>.

Resolves: rhbz#1556990

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-05 13:42:20 +02:00
Ondřej Lysoněk
de5c443205 scheduler: Do true rollback of processes' affinity
Properly revert affinity of processes whose affinity was changed
due to the isolated_cores setting.

Resolves: rhbz#1512295
https://bugzilla.redhat.com/show_bug.cgi?id=1512295#c19

Known regression: reverting affinity of processes started after
Tuned was started does not work as one might expect. See the following
reproducer (run on a 4 core machine):

$ mkdir /etc/tuned/test
$ cat > /etc/tuned/test/tuned.conf << EOF
[scheduler]
isolated_cores=1
EOF
$ cat > a.c << EOF
 #include <unistd.h>

int main(void)
{
    pause();
    return 0;
}
EOF
$ gcc a.c
$ systemctl start tuned
$ ./a.out &
$ systemctl stop tuned
$ taskset -p $(pgrep a.out)
pid 9950's current affinity mask: d  <<< *maybe* should be "f"

The affinity of the ./a.out process is 0xd after tuned is stopped, not
0xf as one might expect. This is because after starting tuned, first
the affinity of the shell session is set to the non-isolated cores (0xd).
Then when ./a.out starts, it inherits that affinity. That is, tuned
doesn't explicitly change affinity of the process. The affinity gets
inherited. So tuned will not change affinity of the process upon rollback,
because it hasn't ever touched that process. And currently, tuned would
not event know what the affinity should be reverted to.

It is unclear to me at this point whether we should attempt to address
this issue, or if we should leave it be. Properly fixing it would
require tracing where processes get their affinity from.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 16:07:04 +02:00
Ondřej Lysoněk
9f02e3f23d scheduler: Store saved affinity as a bitmask rather than list
Store saved CPU affinity of processes as a bitmask rather than
a CPU list. It should be much more memory-efficient.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 15:50:57 +02:00
Ondřej Lysoněk
f8119fa6c6 scheduler: Merge (get|set)_affinity.* methods
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 15:50:57 +02:00
Ondřej Lysoněk
32aab15530 scheduler: Do true rollback of IRQs affinity
Do true rollback of IRQs affinity. Previously the affinity of IRQs was
set to all cores on rollback.

Related: rhbz#1512295
https://bugzilla.redhat.com/show_bug.cgi?id=1512295#c19

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 15:50:26 +02:00
Ondřej Lysoněk
ae53766153 scheduler: Use separate storage for scheduling parameters
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 15:03:25 +02:00
Ondřej Lysoněk
3ec410fdb1 scheduler: Refactor
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 15:01:45 +02:00
Ondřej Lysoněk
74e8725905 scheduler: Use only one global CPU map
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 15:00:50 +02:00
Ondřej Lysoněk
7bb6ff4d4d scheduler: group.* options: Process threads as well
Previously threads were not touched when processing group.* options.

Reproducer:
$ cat > a.c << EOF
 #define _GNU_SOURCE
 #include <pthread.h>
 #include <unistd.h>

void *foo(void *unused)
{
	pause();
	return NULL;
}

int main(void)
{
	pthread_t t;

	pthread_create(&t, NULL, foo, NULL);
	pthread_setname_np(t, "a.out");
	pause();
	return 0;
}
EOF
$ gcc a.c -pthread
$ mkdir /etc/tuned/test
$ cat > /etc/tuned/test/tuned.conf << EOF
[scheduler]
group.foo=00:1:a.out
EOF
$ ./a.out &
$ tuned-adm profile test
$ for proc in /proc/$(pgrep a.out)/task/*; do
	taskset -p $(basename $proc);
done
pid 29052's current affinity mask: 1
pid 29053's current affinity mask: f  <<< should be 1

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 14:59:06 +02:00
Ondřej Lysoněk
c1bc257563 scheduler: Replace get_process with _get_cmdline
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 14:58:13 +02:00
Ondřej Lysoněk
7da6e4adc9 scheduler: Use procfs to list processes
Use procfs instead of the ps command to list processes.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 14:55:16 +02:00
Ondřej Lysoněk
b4ddf47448 scheduler: Add a function for checking if a task is a kthread
Add a function for checking if a task is a kernel thread.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 14:54:38 +02:00
Ondřej Lysoněk
00cda573e3 scheduler: Use schedutils in _set_rt() and _get_rt()
Rewrite _set_rt() and _get_rt() to use schedutils instead of
command line tools.

Behaviour change: previously if the scheduling policy was set to "*"
in a profile, e.g.
[scheduler]
group.foo=0:*:1:1:a.out

then Tuned would set the scheduling policy to SCHED_OTHER on RHEL-7,
and to SCHED_RR on Fedora (at least Fedora 27). This is due to the
behaviour of the "chrt" tool. It behaved this way despite what it
says in the following commit, which says the scheduler will not be
changed if set to "*":
https://github.com/redhat-performance/tuned/commit/bf42cb9ba3c

This commit changes/fixes that, so that the scheduling policy is
in fact not changed if "*" is specified.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 14:28:17 +02:00
Ondřej Lysoněk
ea896e3501 scheduler: Tune scheduler and affinity independently
Set scheduler policy and affinity independently so that when reading
original state of one of the parameters fails, we can still set
the other one.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:39:02 +02:00
Ondřej Lysoněk
bfc670dffc scheduler: Don't tune a process if reading original parameters failed
We should not touch a process if we cannot later revert to it's
original state.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:38:04 +02:00
Ondřej Lysoněk
5ad1f42beb scheduler: Make instance._sched_cfg a local variable
It's not needed anywhere else.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:37:13 +02:00
Ondřej Lysoněk
474a6eff90 scheduler: Always store PIDs as integers
Always store PIDs as integers as that is their natural form.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:32:58 +02:00
Ondřej Lysoněk
7ec58f779a scheduler: Improve code readability
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:19:02 +02:00
Ondřej Lysoněk
7e3819b446 scheduler: Refactor
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:18:21 +02:00
Ondřej Lysoněk
a0c906bff1 scheduler: Drop no_error argument in _set_rt()
I believe it was originally added so that errors are not logged
when we fail to set the scheduling parameters of a short-lived
process which has already disappeared. This is no longer necessary,
because we always check if the process has disappeared, and log
debug messages instead of errors if it has.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:14:25 +02:00
Ondřej Lysoněk
a897259eec scheduler: Drop no_error argument in _set_affinity()
I believe it was originally added so that errors are not logged
when we fail to set the affinity of a short-lived process which
has already disappeared. This is no longer necessary, because we
always check if the process has disappeared, and log debug messages
instead of errors if it has.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:12:24 +02:00
Ondřej Lysoněk
673d253ac1 scheduler: Drop no_error argument in _get_affinity()
I believe it was originally added so that errors are not logged
when we fail to read the affinity of a short-lived process which
has already disappeared. This is no longer necessary, because we
always check if the process has disappeared, and log debug messages
instead of errors if it has.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:10:10 +02:00
Ondřej Lysoněk
8e68e22757 Use the errno attribute of an exception instead of indices
Obtaining the errno from an exception using e[0] is not
possible in Python3.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:09:06 +02:00
Ondřej Lysoněk
02541dfb5c scheduler: Catch OSError in addition to IOError
In python3, functions such as open() or readline() raise OSError
instead of IOError.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:07:38 +02:00
Ondřej Lysoněk
625d219f9e scheduler: Make storage global for the whole plugin
It doesn't make much sense to have multiple instances of the scheduler
plugin, so let's make storage global for the plugin.

Later we should make Tuned report an error if the user defines multiple
instances of the scheduler plugin.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 13:05:25 +02:00
Ondřej Lysoněk
5bfb596a7c Unify storage keys
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 12:52:08 +02:00
Ondřej Lysoněk
e9fcd4525d Fix storage access
Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 12:50:08 +02:00
Ondřej Lysoněk
f3f05e426f scheduler: Prevent a traceback
Add a new method for restoring original CPU affinity of processes
_restore_ps_affinity() and call that in _instance_init() instead of
_instance_unapply_static(). _instance_unapply_static() touches
instance._terminate, which does not yet exist at that point
(_instance_init always gets a fresh instance object).

The reason this was not a problem in the past is that the true branch
of "if len(instance._scheduler_original) > 0:" was never executed,
because instance._scheduler_original was never correctly saved
to storage. The next commit in this patch series fixes that.

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 12:48:06 +02:00
Ondřej Lysoněk
e511d70d40 scheduler: Fix reverting CPU affinity in process groups
Previously CPU affinity of tasks created while Tuned was running
was not correctly reverted on Tuned shutdown.

Reproducer (on a 4 core machine):
$ mkdir /etc/tuned/test
$ cat > /etc/tuned/test/tuned.conf << EOF
[scheduler]
group.foo=00:1:a.out
EOF
$ cat > a.c << EOF
 #include <unistd.h>

int main(void)
{
	pause();
	return 0;
}
EOF
$ gcc a.c
$ systemctl start tuned
$ ./a.out &
$ systemctl stop tuned
$ taskset -p $(pgrep a.out)
pid 9950's current affinity mask: 1  <<< should be "f"

Known issue: if you run the above reproducer with the config below
after this commit is applied, the affinity of the task after stopping
tuned will be 0xd, not 0xf. This is because after starting tuned, first
the affinity of the shell session is set to the non-isolated cores (0xd).
Then when ./a.out starts, it inherits that affinity. Then when tuned tunes
the affinity of the ./a.out process, it remembers 0xd as its old affinity
rather than 0xf. It is unclear to me at this point whether we should do
anything about this issue. Properly fixing it would require tracing where
processes get their affinity from.

[scheduler]
group.foo=00:1:a.out
ps_blacklist=.*a.out.*
isolated_cores=1

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-06-04 12:47:12 +02:00
Ondřej Lysoněk
586b61773a Use symlinks in /etc/ to access GRUB2 config files
We can now be more sure that we won't modify another operating
system's config file.

Related: rhbz#1556990

Signed-off-by: Ondřej Lysoněk <olysonek@redhat.com>
2018-05-18 15:42:35 +02:00
Jaroslav Škarvada
72c94b3614
plugin_scheduler: whitelist/blacklist processed also for thread names
Resolves: rhbz#1512295

Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
2018-05-18 15:04:18 +02:00
Jaroslav Škarvada
e0c73cbbfa
bootloader: added check for petitboot
Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
2018-05-16 15:57:47 +02:00
Jaroslav Škarvada
90cfb257a9
Merge pull request #100 from olysonek/energy_perf_policy
Fix setting EPB on Linux 4.13+, support specifying alternative EPB values
2018-05-14 10:44:55 +02:00