From 275252aad4ab23ebc566cb30b0d03859810e5d36 Mon Sep 17 00:00:00 2001 From: Andrew Hayworth Date: Fri, 31 Oct 2025 16:01:57 -0500 Subject: [PATCH 1/4] bin: run additional validation for --dry-run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit There are certain kinds of invalid configurations that pass the `--dry-run` check, but fail to start at run time. This commit addresses the issue by re-using some of the hot reload validation logic. For example, this config is trivially invalid, passes `--dry-run`, and fails at runtime: ``` pipeline: inputs: - name: dummy tag: test invalid_property_that_does_not_exist: some_value outputs: - name: stdout match: '*' ``` ``` zsh ❮ fluent-bit --dry-run -c ~/tmp/fbconfig-bad-property.yaml Fluent Bit v4.1.1 * Copyright (C) 2015-2025 The Fluent Bit Authors * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd * https://fluentbit.io ______ _ _ ______ _ _ ___ __ | ___| | | | | ___ (_) | / | / | | |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __/ /| | `| | | _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| | | | | | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /\___ |__| |_ \_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ |_(_)___/ configuration test is successful zsh ❮ fluent-bit -c ~/tmp/fbconfig-bad-property.yaml Fluent Bit v4.1.1 * Copyright (C) 2015-2025 The Fluent Bit Authors * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd * https://fluentbit.io ______ _ _ ______ _ _ ___ __ | ___| | | | | ___ (_) | / | / | | |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __/ /| | `| | | _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| | | | | | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /\___ |__| |_ \_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ |_(_)___/ [2025/10/31 16:05:55.268532000] [ info] [fluent bit] version=4.1.1, commit=, pid=21037 [2025/10/31 16:05:55.268808000] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128 [2025/10/31 16:05:55.269140000] [ info] [simd ] disabled [2025/10/31 16:05:55.269147000] [ info] [cmetrics] version=1.0.5 [2025/10/31 16:05:55.269407000] [ info] [ctraces ] version=0.6.6 [2025/10/31 16:05:55.269476000] [error] [config] dummy: unknown configuration property 'invalid_property_that_does_not_exist'. The following properties are allowed: samples, dummy, metadata, rate, interval_sec, interval_nsec, copies, start_time_sec, start_time_nsec, fixed_timestamp, flush_on_startup, and test_hang_on_exit. [2025/10/31 16:05:55.269486000] [ help] try the command: fluent-bit -i dummy -h [2025/10/31 16:05:55.269515000] [error] [engine] input initialization failed ``` With this commit, we can see that the additional validation from hot reload catches the error right away: ``` zsh ❯ bin/fluent-bit --dry-run -c ~/tmp/fbconfig-bad-property.yaml Fluent Bit v4.2.0 * Copyright (C) 2015-2025 The Fluent Bit Authors * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd * https://fluentbit.io ______ _ _ ______ _ _ ___ __ | ___| | | | | ___ (_) | / | / | | |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __/ /| | `| | | _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| | | | | | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /\___ |__| |_ \_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ |_(_)___/ [2025/10/31 16:07:50.402568000] [error] [config] dummy: unknown configuration property 'invalid_property_that_does_not_exist'. The following properties are allowed: samples, dummy, metadata, rate, interval_sec, interval_nsec, copies, start_time_sec, start_time_nsec, fixed_timestamp, flush_on_startup, and test_hang_on_exit. [2025/10/31 16:07:50.402792000] [ help] try the command: bin/fluent-bit -i dummy -h [2025/10/31 16:07:50.402800000] [error] [reload] check properties for input plugins is failed ``` (The logs of course now say `[reload]`, which is a little misleading... we can clean that up, if desired). Signed-off-by: Andrew Hayworth --- src/fluent-bit.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/fluent-bit.c b/src/fluent-bit.c index be5685a1b9e..de45a6967ba 100644 --- a/src/fluent-bit.c +++ b/src/fluent-bit.c @@ -1436,6 +1436,11 @@ static int flb_main_run(int argc, char **argv) #endif if (config->dry_run == FLB_TRUE) { + ret = flb_reload_property_check_all(config); + if (ret != 0) { + exit(EXIT_FAILURE); + } + fprintf(stderr, "configuration test is successful\n"); flb_init_env(); flb_cf_destroy(cf_opts); From c496edc59fdabe6ddb9234f1872da461604300e9 Mon Sep 17 00:00:00 2001 From: Andrew Hayworth Date: Mon, 3 Nov 2025 10:21:43 -0600 Subject: [PATCH 2/4] bin: ensure we clean up after error and successful dry-run We need to perform the same partial initialization and releasing of resources whether or not the dry-run was successful, so this commit ensures that we call those functions before exiting either way. Signed-off-by: Andrew Hayworth --- src/fluent-bit.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/fluent-bit.c b/src/fluent-bit.c index de45a6967ba..17b4843b972 100644 --- a/src/fluent-bit.c +++ b/src/fluent-bit.c @@ -1437,14 +1437,16 @@ static int flb_main_run(int argc, char **argv) if (config->dry_run == FLB_TRUE) { ret = flb_reload_property_check_all(config); - if (ret != 0) { - exit(EXIT_FAILURE); - } - fprintf(stderr, "configuration test is successful\n"); + /* At this point config test is done, so clean up after ourselves */ flb_init_env(); flb_cf_destroy(cf_opts); flb_destroy(ctx); + + if (ret != 0) { + exit(EXIT_FAILURE); + } + fprintf(stderr, "configuration test is successful\n"); exit(EXIT_SUCCESS); } From d7d2b0603360c00a9ed3a529098bfaa0fdb7368e Mon Sep 17 00:00:00 2001 From: Andrew Hayworth Date: Mon, 10 Nov 2025 09:34:08 -0600 Subject: [PATCH 3/4] tests: runtime_shell: test additional dry-run validation This adds a runtime_shell test that runs fluent-bit in dry-run mode and ensures we are running the additional validation recently added. Specifically, we test that configuration property validation is happening, which is borrowed from the hot reload code and was not previously used for dry-run validation. Signed-off-by: Andrew Hayworth --- .../runtime_shell/dry_run_invalid_property.sh | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100755 tests/runtime_shell/dry_run_invalid_property.sh diff --git a/tests/runtime_shell/dry_run_invalid_property.sh b/tests/runtime_shell/dry_run_invalid_property.sh new file mode 100755 index 00000000000..ee48c8693b8 --- /dev/null +++ b/tests/runtime_shell/dry_run_invalid_property.sh @@ -0,0 +1,64 @@ +#!/bin/sh + +# Setup environment if not already set +if [ -z "$FLB_BIN" ]; then + FLB_ROOT=${FLB_ROOT:-$(cd $(dirname $0)/../.. && pwd)} + FLB_BIN=${FLB_BIN:-$FLB_ROOT/build/bin/fluent-bit} +fi + +echo "Using Fluent Bit at: $FLB_BIN" + +# Create a temporary YAML config file with an invalid property +cat > /tmp/dry_run_invalid_property.yaml << EOL +service: + log_level: debug + flush: 1 +pipeline: + inputs: + - name: dummy + tag: test + invalid_property_that_does_not_exist: some_value + outputs: + - name: stdout + match: '*' +EOL + +echo "Running Fluent Bit with --dry-run and invalid property config..." +echo "YAML Config:" +cat /tmp/dry_run_invalid_property.yaml + +# Redirect stdout and stderr to a file for analysis +OUTPUT_FILE="/tmp/dry_run_invalid_property_output.txt" +$FLB_BIN --dry-run -c /tmp/dry_run_invalid_property.yaml > $OUTPUT_FILE 2>&1 + +# Check exit code - we expect it to fail +EXIT_CODE=$? +echo "Fluent Bit --dry-run exited with code: $EXIT_CODE" + +# Show the output +echo "Output file content:" +cat $OUTPUT_FILE + +# Check if the output contains an error about the unknown configuration property +UNKNOWN_PROPERTY=$(grep -c "unknown configuration property 'invalid_property_that_does_not_exist'" $OUTPUT_FILE || true) +RELOAD_ERROR=$(grep -c "check properties for input plugins is failed" $OUTPUT_FILE || true) + +# Clean up +echo "Cleaning up..." +rm -f /tmp/dry_run_invalid_property.yaml +rm -f $OUTPUT_FILE + +# Check results - we expect: +# 1. Fluent Bit to fail (non-zero exit code) +# 2. Error message about unknown configuration property +# 3. Error message from reload validation +if [ "$EXIT_CODE" -ne 0 ] && [ "$UNKNOWN_PROPERTY" -gt 0 ] && [ "$RELOAD_ERROR" -gt 0 ]; then + echo "Test passed: Fluent Bit --dry-run correctly detected invalid property and failed" + exit 0 +else + echo "Test failed: Fluent Bit --dry-run should detect invalid properties and fail" + echo "Exit code: $EXIT_CODE (expected non-zero)" + echo "Unknown property message count: $UNKNOWN_PROPERTY (expected > 0)" + echo "Reload error message count: $RELOAD_ERROR (expected > 0)" + exit 1 +fi From 31924ee14fe6eb7b7154dba6f8e02fcda263ad67 Mon Sep 17 00:00:00 2001 From: Andrew Hayworth Date: Mon, 10 Nov 2025 10:20:10 -0600 Subject: [PATCH 4/4] tests: runtime_shell: actually run dry_run_invalid_property.sh This was an oversight on my part - we actually need to list this as a test for it to be run. Signed-off-by: Andrew Hayworth --- tests/runtime_shell/CMakeLists.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/runtime_shell/CMakeLists.txt b/tests/runtime_shell/CMakeLists.txt index 30ae87a6b96..11ca3f1c2c9 100644 --- a/tests/runtime_shell/CMakeLists.txt +++ b/tests/runtime_shell/CMakeLists.txt @@ -5,6 +5,7 @@ configure_file( set(UNIT_TESTS_SH custom_calyptia.sh + dry_run_invalid_property.sh in_dummy_expect.sh in_tail_expect.sh in_http_tls_expect.sh