Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix journalctl --follow with non-matching filter #257

Open
wants to merge 25 commits into
base: main
Choose a base branch
from

Conversation

dtardon
Copy link
Member

@dtardon dtardon commented Apr 10, 2024

Also fixes a bunch of other journalctl issues with combinations of options.

Resolves: RHEL-30567

@github-actions github-actions bot added pr/mention tracker/missing Formerly needs-bz pr/needs-ci Formerly needs-ci pr/needs-review Formerly needs-review labels Apr 10, 2024
Copy link

github-actions bot commented Apr 10, 2024

Commit validation

Tracker - Multiple trackers found

The following commits meet all requirements

commit upstream
964960b - mount-util: add umount_and_free() helper systemd/systemd@a789f72
539019e - journalctl: use DEFINE_MAIN_FUNCTION() macro systemd/systemd@9556e79
f53fa49 - journalctl: fix output when --lines is used with --grep systemd/systemd@db46919
302992b - journalctl: fix output when --until is used with --lines systemd/systemd@81fb537
660fb97 - journalctl: fix output when --since is used with --lines systemd/systemd@f582695
cdb4104 - journalctl: fix when --since, --until and --lines are used altogether systemd/systemd@5d2ab01
b834366 - journalctl: fix when --grep is used with --follow systemd/systemd@c673fd5
48f67ed - journalctl: make --follow work with --merge again systemd/systemd@e47622a
97b5411 - test: check if we can use --merge with --follow systemd/systemd@c11c50a
7d95fdd - journalctl: always initialize global variables systemd/systemd@4f0165f
81b27b0 - journalctl: use correct variable to check if --since is specified systemd/systemd@20e933a
c15028b - journalctl: fix --no-tail handling systemd/systemd@3f2203f
551c56b - journalctl: split out action_list_fields() systemd/systemd@2ce9a07
d606ae5 - journalctl: split out update_cursor() systemd/systemd@8c12d35
1230414 - journalctl: split out show() systemd/systemd@e4c4a9d
a4db3a3 - journalctl: replace ppoll() loop with sd_event_loop() systemd/systemd@713342d
01bc275 - journalctl: also update cursor with --follow systemd/systemd@24d633e
53d98ca - test: add testcase for 'journalctl --follow --cursor-file=' systemd/systemd@fb35fea
8501075 - journalctl: fix --follow with non-matching filter systemd/systemd@80ec3db
d4363bc - test: make TEST-04 stable once again systemd/systemd@57130ca
430fa10 - journalctl: do not add io event source for stdout if it is a file systemd/systemd@f882a98
1d9008d - journalctl: support --lines=+N for showing the oldest N entries systemd/systemd@8d6791d
ca0a656 - journalctl: don't skip over messages not matching the cursor systemd/systemd@4207a55
8160d9f - journalctl: make --until work again with --after-cursor and --lines systemd/systemd@cb2be36
614ea41 - test: add test case for issue #31776 systemd/systemd@bf99542

Follow-up detection

Failed

🔴 Some follow-up commits for this Pull Request were detected in upstream
🔴 Some mentions of commits from this Pull Request were detected in upstream

Follow-ups

commit follow-up
1d9008d - journalctl: support --lines=+N for showing the oldest N entries systemd/systemd@7b5ff43
systemd/systemd@f2e2c93

Commit mentions

commit mention
430fa10 - journalctl: do not add io event source for stdout if it is a file systemd/systemd@8cb0008

Tracker validation

🔴 Missing tracker or Unknown tracker type; type: 'unknown'


Pull Request validation

Failed

🔴 Failed or pending checks - build (CLANG_RELEASE, auto)[failure],build (CLANG, gcrypt)[failure],build (CLANG, auto)[failure],build (CLANG_ASAN_UBSAN, auto)[failure],build (clang, 14, lld, openssl)[failure],build (CLANG_ASAN_UBSAN_NO_DEPS, auto)[failure],build (clang, 15, bfd, auto)[failure],build (clang, 13, mold, gcrypt)[failure] Failed or pending statuses - CentOS CI (CentOS Stream 9 + sanitizers)[failure]
🔴 Review - Missing review from a member (1 required)

@jamacku jamacku added this to the RHEL-9.5.0 milestone Apr 12, 2024
@msekletar
Copy link
Member

Hi @dtardon, it seems that followup detection is correct. Can you please double-check and backport missing followups if appropriate. Also, some commits are missing JIRA reference. Thank you!

poettering and others added 21 commits August 5, 2024 10:52
(cherry picked from commit a789f72)

Related: RHEL-30567
(cherry picked from commit 9556e79)

Related: RHEL-30567
Previously, we skip the entries before arg_lines
unconditionally, which doesn't behave correctly
when used with --grep. After this commit, when
a pattern is specified, we don't skip the entries
early, but rely on the count of the lines shown
to tell us when to stop. To achieve that we would
have to search backwards instead.

Fixes #25147

(cherry picked from commit db46919)

Related: RHEL-30567
Before this commit, when --lines is specified, we jump to the tail and
search afterwards from there, thus breaking --until if used together.

After this commit:
If both --until and any of --reverse and --lines is specified, things get
a little tricky. We seek to the place of --until first. If only --reverse or
--reverse and --lines is specified, we search backwards and let the output
counter handle --lines for us. If only --lines is used, we just jump backwards
arg_lines and search afterwards from there.

(cherry picked from commit 81fb537)

Related: RHEL-30567
Before this commit, if --since is used with --lines=N,
we seek to the place of --since and search afterwards
there, resulting in outputing the first N lines.

After this commit, we only do the above if --since is used without
--reverse and --lines. Otherwise we seek to the tail first and check
if the entry is within the range of --since later.

(cherry picked from commit f582695)

Related: RHEL-30567
This is a follow-up for #26669 (81fb537).

After the mentioned commit, we stopped checking if the
entry is within the range of --until if --lines is used.

However, when --since, --until and --lines=N are used
altogether, and the number of lines between --since
and --until is smaller than N, we would seek to --since
later (f582695).
This breaks the assumption that if --lines is set,
the boundary is never exceeded because the counter of
outputs gets us covered.

(cherry picked from commit 5d2ab01)

Related: RHEL-30567
Follow-up for #25147 (db46919)

--follow sets arg_lines to 10, which breaks
--grep as the latter implies --reverse.
So let's not set --reverse if --follow is used.

(cherry picked from commit c673fd5)

Related: RHEL-30567
Set --boot with --follow only if it's not already set and if --merge is
not used, since it's not compatible with --boot.

Follow-up to 2dd9285.
Resolves: #24565

(cherry picked from commit e47622a)

Related: RHEL-30567
Provides coverage for #24565.

(cherry picked from commit c11c50a)

Related: RHEL-30567
That's not necessary, as they are initialized with zero, but for safety
and readability.

(cherry picked from commit 4f0165f)

Related: RHEL-30567
(cherry picked from commit 20e933a)

Related: RHEL-30567
Fixes a bug introduced by 62f21ec.

(cherry picked from commit 3f2203f)

Related: RHEL-30567
No functional change, just refactoring.

(cherry picked from commit 2ce9a07)

Related: RHEL-30567
No functional change, just refactoring.

(cherry picked from commit 8c12d35)

Related: RHEL-30567
No functional changes, just refactoring.

(cherry picked from commit e4c4a9d)

Related: RHEL-30567
No functional change, just refactoring.

(cherry picked from commit 713342d)

Related: RHEL-30567
Fixes #26746.

(cherry picked from commit 24d633e)

Related: RHEL-30567
Also, add a FIXME comment to illustrate the issue uncovered after by
7a4ee86.

(cherry picked from commit fb35fea)

Related: RHEL-30567
When there is no matching entry stored in journal, then initial call of
`sd_journal_previous()` following `sd_journal_seek_tail()` returns
zero, and does not move the read pointer.
In the main loop, on every journal event, we call `sd_journal_next()`,
even though the current location is tail, and it takes no effect.

In such a case, we need to call `sd_journal_previous()` instead of
`sd_journal_next()`.

(cherry picked from commit 80ec3db)

Resolves: RHEL-30567
Wait a bit if necessary for the cursor file to appear.

Follow-up fb35fea.

(cherry picked from commit 57130ca)

Related: RHEL-30567
Fixes a bug introduced by 713342d.

Fixes #28636.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=2228089.

(cherry picked from commit f882a98)

Related: RHEL-30567
YHNdnzj and others added 4 commits August 5, 2024 10:52
After f582695, the wrong behavior
occurred when --since= and --lines= are both specified is fixed.
However, it seems that the old behavior is already being somewhat
widely used, and the function itself makes sense, i.e. to allow --lines=
to output the first N journal entries.

Therefore, let's support prefixing the number for --lines= with '+',
and provide such functionality.

Related: #28746
(cherry picked from commit 8d6791d)

Related: RHEL-30567
When --after-cursor=/--cursor-file= is used together with a journal
filter, we still skipped over the first matching entry even if it wasn't
the entry the cursor points at, thus missing one "valid" entry
completely. Let's fix this by checking if the entry cursor after seeking
matches the user provided cursor, and skip to the next entry only when
the cursors match.

Resolves: #30288
(cherry picked from commit 4207a55)

Related: RHEL-30567
Fixes a regression introduced by 81fb537.

If one of the cursor option is specified, we first seek to the cursor position.
So, the current position may be out of the time range specified by --until,
and we need to verify the timestamp of the current position.

Fixes #31776.

Co-authored-by: Reid Wahl <[email protected]>
(cherry picked from commit cb2be36)

Related: RHEL-30567
(cherry picked from commit bf99542)

Related: RHEL-30567
@jamacku jamacku modified the milestones: RHEL-9.5.0, RHEL-9.6.0 Aug 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr/follow-up pr/mention pr/needs-ci Formerly needs-ci pr/needs-review Formerly needs-review tracker/missing Formerly needs-bz
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants