Execution aborted due to libsystemd assertion

[jr0me]

Description

The agent is stopped due to a failing assertion coming from libsystemd through JournalLog::SeekTail.

See log output:

Assertion 'm->type == MATCH_AND_TERM' failed at src/libsystemd/sd-journal/sd-journal.c:665, function find_location_for_match(). Aborting.

Thread 27 "wazuh-agent" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffefe00640 (LWP 72968)]
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737217824320) at ./nptl/pthread_kill.c:44
44      ./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737217824320) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737217824320) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737217824320, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff7a42476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7a287f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007ffff7f552e7 in ?? () from /lib/x86_64-linux-gnu/libsystemd.so.0
#6  0x00007ffff7f3d61f in ?? () from /lib/x86_64-linux-gnu/libsystemd.so.0
#7  0x00007ffff7f3d21a in ?? () from /lib/x86_64-linux-gnu/libsystemd.so.0
#8  0x00007ffff7f3d15c in ?? () from /lib/x86_64-linux-gnu/libsystemd.so.0
#9  0x00007ffff7f3d21a in ?? () from /lib/x86_64-linux-gnu/libsystemd.so.0
#10 0x00007ffff7f3d15c in ?? () from /lib/x86_64-linux-gnu/libsystemd.so.0
#11 0x00007ffff7f3d9ab in ?? () from /lib/x86_64-linux-gnu/libsystemd.so.0
#12 0x000055555606f92b in JournalLog::SeekTail (this=0x7fffe0022c80)
    at /home/user/Dev/wazuh-agent/src/modules/logcollector/src/journald_reader/src/journal_log.cpp:55
#13 0x0000555556063526 in logcollector::JournaldReader::Run(_ZN12logcollector14JournaldReader3RunEv.Frame *) (frame_ptr=0x7fffe0026f90)
    at /home/user/Dev/wazuh-agent/src/modules/logcollector/src/journald_reader/src/journald_reader.cpp:48
#14 0x00005555556940f3 in std::__n4861::coroutine_handle<void>::resume (this=0x7fffe0026fa0) at /usr/include/c++/13/coroutine:135
#15 0x00005555556af723 in boost::asio::detail::awaitable_frame_base<boost::asio::any_io_executor>::resume (this=0x7fffe0026fa0)
    at /home/user/Dev/wazuh-agent/build/vcpkg_installed/x64-linux/include/boost/asio/impl/awaitable.hpp:501
#16 0x00005555556aebdd in boost::asio::detail::awaitable_thread<boost::asio::any_io_executor>::pump (this=0x7fffefdff430)
    at /home/user/Dev/wazuh-agent/build/vcpkg_installed/x64-linux/include/boost/asio/impl/awaitable.hpp:770
#17 0x00005555556b62bd in boost::asio::detail::awaitable_async_op_handler<void (), boost::asio::any_io_executor>::operator()() (
    this=0x7fffefdff430) at /home/user/Dev/wazuh-agent/build/vcpkg_installed/x64-linux/include/boost/asio/impl/awaitable.hpp:804
#18 0x00005555556b5fc8 in boost::asio::detail::binder0<boost::asio::detail::awaitable_async_op_handler<void (), boost::asio::any_io_executor> >::operator()() (this=0x7fffefdff430)
    at /home/user/Dev/wazuh-agent/build/vcpkg_installed/x64-linux/include/boost/asio/detail/bind_handler.hpp:56
#19 0x00005555556b6188 in boost::asio::detail::executor_function::complete<boost::asio::detail::binder0<boost::asio::detail::awaitable_async_op_handler<void (), boost::asio::any_io_executor> >, std::allocator<void> >(boost::asio::detail::executor_function::impl_base*, bool) (

This is happening in main 19719ce on Ubuntu 22, after a freash clean up, with inventory and logcollector enabled and cmake build type set to Debug. Using the mock server in http mode.

This configuration yml is used:

agent:
  thread_count: 4
  server_url: http://localhost:27000
  retry_interval: 30s
  verification_mode: none
events:
  batch_interval: 10s
  batch_size: 1MB
inventory:
  enabled: true
  interval: 10s
  scan_on_start: true
  hardware: true
  system: true
  networks: true
  packages: true
  ports: true
  ports_all: false
  processes: false
  hotfixes: true
logcollector:
  enabled: true
  localfiles:
    - /var/log/auth.log
  reload_interval: 10s
  read_interval: 500ms
  journald:
    - field: "_SYSTEMD_UNIT"
      value: "cron.service"
      exact_match: true
      ignore_if_missing: true
    - field: "SYSLOG_IDENTIFIER"
      value: "systemd"
      exact_match: false
      ignore_if_missing: true
    - conditions:
        - field: "_COMM"
          value: "cat"
          exact_match: true
        - field: "PRIORITY"
          value: "3|4"
          exact_match: true
      ignore_if_missing: true

The error seems to go away when setting the PRIORITY condition's exact match to false.

The journald configuration was taken from the documentation in docs. See: https://github.com/wazuh/wazuh-agent/tree/main/docs/ref/modules/logcollector