Is your feature request related to a problem? Please describe.
I want to use one shared Fluent Bit Tail input configuration across multiple containerized workloads, but not every workload has the same log file paths.
For example, one workload may write:
while another may also write optional component logs under:
/var/log/components/*/*.log
If the shared Tail config includes both paths, workloads that do not create the optional component log directory can emit noisy missing-path or unmatched-glob errors, even though the absence of those files is expected.
Describe the solution you'd like
Add an opt-in Tail input option such as:
ignore_missing_path: true
Example:
pipeline:
inputs:
- name: tail
path: /var/log/app/*.log,/var/log/components/*/*.log
tag: app_logs
ignore_missing_path: true
When enabled, Fluent Bit should not emit an error if a configured path or glob does not currently match any files. The Tail input should still refresh normally and begin tailing matching files if they are created later, according to existing behavior such as refresh_interval.
The default should preserve current behavior, for example:
ignore_missing_path: false
Describe alternatives you've considered
I have considered maintaining separate Fluent Bit configs per workload type, but that creates duplication and makes config drift more likely.
I have also considered creating placeholder directories/files so optional globs always match something, but that feels like a workaround and adds artificial files only to satisfy the Tail input.
Another option is generating the Tail config dynamically at container startup, but that adds startup scripting and makes the effective runtime config differ from the checked-in config.
Additional context
This would be useful for containerized and orchestrated environments where a single logging sidecar or shared Fluent Bit image is reused across heterogeneous workloads.
Some log paths are mandatory, while others are optional depending on enabled features, plugins, or workload type. Having an explicit ignore_missing_path option would make optional log paths easier to model without duplicating configs or adding placeholder files.
Is your feature request related to a problem? Please describe.
I want to use one shared Fluent Bit Tail input configuration across multiple containerized workloads, but not every workload has the same log file paths.
For example, one workload may write:
while another may also write optional component logs under:
If the shared Tail config includes both paths, workloads that do not create the optional component log directory can emit noisy missing-path or unmatched-glob errors, even though the absence of those files is expected.
Describe the solution you'd like
Add an opt-in Tail input option such as:
Example:
When enabled, Fluent Bit should not emit an error if a configured path or glob does not currently match any files. The Tail input should still refresh normally and begin tailing matching files if they are created later, according to existing behavior such as
refresh_interval.The default should preserve current behavior, for example:
Describe alternatives you've considered
I have considered maintaining separate Fluent Bit configs per workload type, but that creates duplication and makes config drift more likely.
I have also considered creating placeholder directories/files so optional globs always match something, but that feels like a workaround and adds artificial files only to satisfy the Tail input.
Another option is generating the Tail config dynamically at container startup, but that adds startup scripting and makes the effective runtime config differ from the checked-in config.
Additional context
This would be useful for containerized and orchestrated environments where a single logging sidecar or shared Fluent Bit image is reused across heterogeneous workloads.
Some log paths are mandatory, while others are optional depending on enabled features, plugins, or workload type. Having an explicit
ignore_missing_pathoption would make optional log paths easier to model without duplicating configs or adding placeholder files.