From 25fb041a4a8a25ef74af8affaa0c3ff4d4a31adc Mon Sep 17 00:00:00 2001 From: Ra'Jiska Date: Sat, 1 Aug 2026 17:19:23 +0800 Subject: [PATCH 1/4] docs: administration: monitoring: on-demand-flush: Add on-demand flush to the HTTP server Signed-off-by: Ra'Jiska --- administration/monitoring.md | 1 + administration/on-demand-flush.md | 71 +++++++++++++++++++++++++++++++ 2 files changed, 72 insertions(+) create mode 100644 administration/on-demand-flush.md diff --git a/administration/monitoring.md b/administration/monitoring.md index 5e9a8bbd6..0f3571d5b 100644 --- a/administration/monitoring.md +++ b/administration/monitoring.md @@ -124,6 +124,7 @@ Fluent Bit exposes the following endpoints for monitoring. | `/api/v2/metrics/prometheus` | Display internal metrics per loaded plugin ready in Prometheus Server format. | Prometheus Text 0.0.4 | | `/api/v2/health` | Returns Fluent Bit health status as JSON. HTTP 200 when healthy, HTTP 500 when unhealthy. Response fields: `status` (`ok` or `error`), `errors`, `retries_failed`, `error_limit`, `retry_failure_limit`, `period_limit`. | JSON | | `/api/v2/reload` | Execute hot reloading (`POST`, `PUT`) or get the status of hot reloading (`GET`). Unsupported methods return `405 Method Not Allowed` with an `Allow: GET, POST, PUT` header. See the [hot-reloading documentation](hot-reload.md). | JSON | +| `/api/v2/flush` | Trigger an on-demand flush of currently buffered records (`POST`, `PUT`) or get the current flush count (`GET`). Unsupported methods return `405 Method Not Allowed` with an `Allow: GET, POST, PUT` header. See the [on-demand flush documentation](on-demand-flush.md). | JSON | ### V1 metrics diff --git a/administration/on-demand-flush.md b/administration/on-demand-flush.md new file mode 100644 index 000000000..822245bd2 --- /dev/null +++ b/administration/on-demand-flush.md @@ -0,0 +1,71 @@ +--- +description: Trigger an immediate flush over HTTP, independent of the periodic Flush interval +--- + +# On-demand flush + +Fluent Bit supports triggering a flush of currently buffered records on demand, over the HTTP server, independent of the configured periodic `Flush` interval. + +## Enable the HTTP server + +To get started with on-demand flush over HTTP, enable the HTTP Server in the configuration file: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +service: + http_server: on + http_listen: 0.0.0.0 + http_port: 2020 +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[SERVICE] + HTTP_Server On + HTTP_Listen 0.0.0.0 + HTTP_PORT 2020 +``` + +{% endtab %} +{% endtabs %} + +## How to flush + +After enabling the HTTP server, use one of the following methods to trigger an on-demand flush: + +### HTTP + +Use the following HTTP endpoints to trigger an on-demand flush: + +- `PUT /api/v2/flush` +- `POST /api/v2/flush` + +For using curl to trigger a flush, users must specify an empty request body as: + +```shell +curl -X POST -d '{}' localhost:2020/api/v2/flush +``` + +Obtain a count of on-demand flushes using the HTTP endpoint: + +- `GET /api/v2/flush` + +The endpoint returns `flush_now_count` as follows: + +```json +{"flush_now_count":3} +``` + +The default value of the counter is `0`. + +## Confirm a flush + +Use the `flush_now_count` returned by the `POST`/`PUT` response, or by a separate `GET /api/v2/flush` call, to confirm that a specific request resulted in a new flush. If the pipeline doesn't acknowledge the flush within the internal timeout, the `POST`/`PUT` endpoint responds with HTTP status `503` instead: + +```json +{"flush":"timeout","flush_now_count":0} +``` From 005531f74aa586259e1dae7e02fdfe0cd3cd1baf Mon Sep 17 00:00:00 2001 From: Ra'Jiska Date: Sat, 1 Aug 2026 17:19:31 +0800 Subject: [PATCH 2/4] docs: summary: Add on-demand flush entry Signed-off-by: Ra'Jiska --- SUMMARY.md | 1 + 1 file changed, 1 insertion(+) diff --git a/SUMMARY.md b/SUMMARY.md index cafaf1057..44c7e1013 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -64,6 +64,7 @@ * [Monitoring](administration/monitoring.md) * [Multithreading](administration/multithreading.md) * [Networking](administration/networking.md) +* [On-demand flush](administration/on-demand-flush.md) * [Performance tips](administration/performance.md) * [Scheduling and retries](administration/scheduling-and-retries.md) * [TLS](administration/transport-security.md) From c93cc64795a2aef9cf2181b238a955fcb0799a0d Mon Sep 17 00:00:00 2001 From: Ra'Jiska Date: Sat, 1 Aug 2026 21:12:09 +0800 Subject: [PATCH 3/4] docs: summary: Further describe flush_now_count semantics Signed-off-by: Ra'Jiska --- administration/on-demand-flush.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/administration/on-demand-flush.md b/administration/on-demand-flush.md index 822245bd2..e104bd634 100644 --- a/administration/on-demand-flush.md +++ b/administration/on-demand-flush.md @@ -64,7 +64,15 @@ The default value of the counter is `0`. ## Confirm a flush -Use the `flush_now_count` returned by the `POST`/`PUT` response, or by a separate `GET /api/v2/flush` call, to confirm that a specific request resulted in a new flush. If the pipeline doesn't acknowledge the flush within the internal timeout, the `POST`/`PUT` endpoint responds with HTTP status `503` instead: +A successful request returns HTTP status `200` with the incremented counter: + +```json +{"flush":"done","flush_now_count":3} +``` + +`flush_now_count` is a process-wide counter incremented once per on-demand flush processed by the engine. It's incremented when buffered chunks are dispatched to the output plugins. A `200` only guarantees the engine is processing chunks, not that they got pushed out or received by the outputs. Because the counter is global, it can't be used to correlate a response with a specific request when several flushes are issued concurrently. + +If the engine doesn't acknowledge the request within 2 seconds, the endpoint responds with HTTP status `503` and the counter unchanged: ```json {"flush":"timeout","flush_now_count":0} From 87100dc49bfe1c42493596c161e88b8983da07cd Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Sat, 1 Aug 2026 17:17:52 +0200 Subject: [PATCH 4/4] docs: administration: on-demand-flush: Correct response semantics and document engine behavior Validated the page against the implementation in fluent/fluent-bit#12192 and corrected four inaccuracies: - The timeout example reported flush_now_count as 0 and described the counter as unchanged, contradicting the process-wide semantics described directly above it. The 503 path packs the current global counter, which reflects other flushes. Also clarify that a 503 is a missed acknowledgement, not a cancellation: the request stays queued on the manager channel and can still be processed afterwards. - Document the previously unmentioned 500 responses. A failed dispatch to the engine triggers no flush and leaves the counter alone, while a response encoding failure occurs after the flush was already requested, so the counter can still advance. Both return an empty body rather than JSON. - Replace "must specify an empty request body as -d '{}'" with an accurate description. The handler dispatches on the request method and never reads the body, and the HTTP/1 parser accepts a bodyless POST, so no payload is required. Use an explicit http:// URL. - Add a section describing what a flush does. Pending retries are invalidated and rescheduled to run immediately before buffered chunks are dispatched, so chunks in retry backoff are sent without waiting out their timer. Signed-off-by: Eric D. Schabell --- administration/on-demand-flush.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/administration/on-demand-flush.md b/administration/on-demand-flush.md index e104bd634..e6a530e37 100644 --- a/administration/on-demand-flush.md +++ b/administration/on-demand-flush.md @@ -44,10 +44,10 @@ Use the following HTTP endpoints to trigger an on-demand flush: - `PUT /api/v2/flush` - `POST /api/v2/flush` -For using curl to trigger a flush, users must specify an empty request body as: +The endpoint ignores the request body, so no payload is required. To trigger a flush with curl: ```shell -curl -X POST -d '{}' localhost:2020/api/v2/flush +curl -X POST http://localhost:2020/api/v2/flush ``` Obtain a count of on-demand flushes using the HTTP endpoint: @@ -62,6 +62,10 @@ The endpoint returns `flush_now_count` as follows: The default value of the counter is `0`. +## What a flush does + +An on-demand flush performs two steps. Fluent Bit first invalidates any pending retries and reschedules them to run immediately, so chunks waiting in retry backoff are sent without waiting out their backoff timer. Tasks that are already running are left untouched. Fluent Bit then dispatches the currently buffered chunks to the output plugins, which is the same work the periodic `Flush` interval performs. + ## Confirm a flush A successful request returns HTTP status `200` with the incremented counter: @@ -72,8 +76,12 @@ A successful request returns HTTP status `200` with the incremented counter: `flush_now_count` is a process-wide counter incremented once per on-demand flush processed by the engine. It's incremented when buffered chunks are dispatched to the output plugins. A `200` only guarantees the engine is processing chunks, not that they got pushed out or received by the outputs. Because the counter is global, it can't be used to correlate a response with a specific request when several flushes are issued concurrently. -If the engine doesn't acknowledge the request within 2 seconds, the endpoint responds with HTTP status `503` and the counter unchanged: +If the engine doesn't acknowledge the request within 2 seconds, the endpoint responds with HTTP status `503`. The reported counter is the current process-wide value, which doesn't include this request: ```json -{"flush":"timeout","flush_now_count":0} +{"flush":"timeout","flush_now_count":2} ``` + +A `503` means the engine didn't acknowledge the request in time, not that the flush was cancelled. The request stays queued and the engine can still process it later, incrementing the counter after the response was sent. + +The endpoint responds with HTTP status `500` and an empty body in two cases. If the request can't be delivered to the engine, no flush is triggered and the counter isn't incremented. If the JSON response can't be encoded, the flush was already requested and might have completed, so the counter can still advance. A `GET` request returns `500` with an empty body under the same encoding failure.