Skip to content

Synchronous socket writes add one-second delays after EAGAIN #12115

Description

@stondo

Bug Report

Describe the bug

The synchronous socket writer in fd_io_write() retries a nonblocking
send() or sendto() after EAGAIN/EWOULDBLOCK by sleeping for one full
second.

When a large response fills the socket buffer repeatedly, each temporary
backpressure event adds another second to response completion even if the
socket becomes writable much sooner.

The behavior is present in Fluent Bit 5.0.2 and current master.

To Reproduce

  1. Create a nonblocking socket pair.
  2. Set the writer's send buffer to 4096 bytes.
  3. Write a 16 KiB payload through flb_io_fd_write().
  4. Start a reader thread that waits 20 ms before draining the peer.
  5. Measure the synchronous write duration.

A focused internal regression test reproduces the behavior. With the current
implementation, the one-second retries make the test exceed its 900 ms bound.
After replacing the blind sleep with a writability wait, the same write
completes in approximately 20-30 ms.

Expected behavior

After EAGAIN/EWOULDBLOCK, the synchronous writer should wait for POLLOUT
with the existing one-second upper bound and retry as soon as the socket is
writable. The existing bounded retry behavior should remain in place.

Your Environment

  • Version used: 5.0.2 and master at fd5ea1f0d503
  • Operating systems: embedded Linux arm64 and Debian 12 x86_64 test container
  • Affected path: synchronous network writes in src/flb_io.c
  • Observed with: a large prometheus_exporter metrics response

Additional context

The issue was isolated while testing a large metrics response over a direct
local network connection.

Before the fix, complete responses of approximately 2.69 MiB took 16-17
seconds. A syscall trace correlated the delay with sixteen EAGAIN results and
sixteen one-second sleeps in the HTTP worker.

With a bounded POLLOUT wait, a traced 2,716,787-byte response completed in
0.252 seconds. The worker encountered 23 EAGAIN results, performed 23
writability waits, and performed zero one-second sleeps.

The proposed regression test fails with the old implementation and passes with
the writability wait.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions