in_tail: Add data reliability note

[jinyongchoi]

This commit adds a 'Data Reliability and Recovery' hint to the Tail input plugin documentation.

It clarifies the behavior of the database offset mechanism during unexpected shutdowns (e.g., system crash, power loss). Specifically, it explains that while Fluent Bit guarantees at-least-once delivery, there is a possibility of slight offset lag and minimal data duplication upon recovery. This ensures users understand that no data is lost even in these scenarios.

fluent/fluent-bit#11269

Summary by CodeRabbit

• Documentation
  • Added guidance on data reliability and recovery for the Tail input, describing at-least-once delivery, restart-from-last-committed-checkpoint behavior, and potential minimal re-ingestion after unexpected shutdowns (added in two locations).
  • Clarified unicode.encoding guidance: noted that "auto" may fail in some environments, can misguess encoding, and recommended explicitly using UTF-16LE or UTF-16BE when guessing is unreliable.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Added documentation to the Tail input describing data reliability and recovery semantics (at-least-once delivery, restart resumption from last committed checkpoint, and possible minimal re-ingestion after unexpected shutdowns) and clarified unicode.encoding auto-detection limitations with a recommendation to prefer explicit UTF-16LE/UTF-16BE variants when encoding guessing is uncertain. The data reliability block and the unicode.encoding guidance were inserted in two locations within the file.

Changes

Cohort / File(s)	Summary
Documentation Updates pipeline/inputs/tail.md	Added informational blocks on data reliability and recovery (at-least-once delivery, restart resume from last committed checkpoint, potential minimal re-ingestion after crashes) in two places; expanded guidance on unicode.encoding auto-detection limitations in both occurrences and recommended explicit UTF-16LE/UTF-16BE usage when unsure.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title directly addresses the main change - adding a data reliability note to the in_tail plugin documentation, which matches the PR objectives perfectly.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

📜 Recent review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between dfca97e and c017872.

📒 Files selected for processing (1)

• pipeline/inputs/tail.md

🚧 Files skipped from review as they are similar to previous changes (1)

• pipeline/inputs/tail.md

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7fb7ce2 and 72849b6.

📒 Files selected for processing (1)

• pipeline/inputs/tail.md (1 hunks)

🧰 Additional context used

🪛 GitHub Check: runner / vale

pipeline/inputs/tail.md

[failure] 100-100:
[vale] reported by reviewdog 🐶
[FluentBit.MayMightCan] Use 'can' for permissions or 'might' for possibility.

Raw Output:
{"message": "[FluentBit.MayMightCan] Use 'can' for permissions or 'might' for possibility.", "location": {"path": "pipeline/inputs/tail.md", "range": {"start": {"line": 100, "column": 114}}}, "severity": "ERROR"}

---

[failure] 98-98:
[vale] reported by reviewdog 🐶
[FluentBit.MayMightCan] Use 'can' for permissions or 'might' for possibility.

Raw Output:
{"message": "[FluentBit.MayMightCan] Use 'can' for permissions or 'might' for possibility.", "location": {"path": "pipeline/inputs/tail.md", "range": {"start": {"line": 98, "column": 138}}}, "severity": "ERROR"}

---

[failure] 98-98:
[vale] reported by reviewdog 🐶
[FluentBit.Latin] Use 'for example' instead of 'e.g.,'.

Raw Output:
{"message": "[FluentBit.Latin] Use 'for example' instead of 'e.g.,'.", "location": {"path": "pipeline/inputs/tail.md", "range": {"start": {"line": 98, "column": 26}}}, "severity": "ERROR"}

🪛 LanguageTool

pipeline/inputs/tail.md

[style] ~98-~98: ‘lag behind’ might be wordy. Consider a shorter alternative.
Context: ...ivery. The database offset may slightly lag behind the actual processed position if an **u...

(EN_WORDINESS_PREMIUM_LAG_BEHIND)

[jinyongchoi]

Following our discussion, I have updated the documentation to address the data reliability behavior during unexpected shutdowns (related to fluent/fluent-bit#11269).

Summary of changes:
I added a Hint block in the tail input plugin documentation under the Database file section. This update clarifies that:

1. Fluent Bit guarantees at-least-once delivery.
2. In scenarios of unexpected shutdowns (e.g., system crash, power loss), the database offset might slightly lag behind the actual processed position.
3. Upon restart, the system resumes from the last committed checkpoint. This ensures no data is lost, although it may result in the re-ingestion of a minimal amount of data (duplication).

This addition ensures that users are aware of this behavior as a known corner case, as suggested.

Thanks!

[esmerel]

looks pretty good to me also