Azure Logs Ingestion API output doesn't split chunks into multiple requests to stay under payload size limit

[hashtagchris]

Bug Report

Describe the bug
Azure Monitor's Logs Ingestion API has a maximum payload size of 1MiB. The azure_logs_ingestion output sends one http request per flushed chunk. A chunk serialized to json may exceed 1MiB, resulting in persistent 413 responses.

Enabling compression reduces the chances of the maximum payload size being exceeded, but does not eliminate it.

To Reproduce

1. In Azure, create a log analytics workspace, direct data collection rule and a service principal for RBAC.
2. Run Fluent Bit with the below config.
3. Monitor Fluent Bit output.

service:
  flush: 10
  grace: 2
  log_level: warn

pipeline:
  inputs:
    - name: dummy
      tag: two_mb_uncompressed
      dummy: '{"SeverityText":"Info","Body":"Fluent Bit full chunk sent uncompressed to Azure Monitor","Attributes":{"repro":"azure-logs-ingestion-2mb-uncompressed"}}'
      samples: 300
      copies: 100
      rate: 100
      fixed_timestamp: true
      flush_on_startup: true

  outputs:
    - name: azure_logs_ingestion
      match: two_mb_uncompressed
      table_name: MyFluentBitLogs
      dce_url: ${DCE_URL}
      dcr_id: ${DCR_ID}
      tenant_id: ${TENANT_ID}
      client_id: ${CLIENT_ID}
      client_secret: ${CLIENT_SECRET}
      time_key: TimeGenerated
      time_generated: true
      compress: false
      retry_limit: no_retries

Expected behavior
Given individual log records with a json representation under 1MiB (possibly compressed), the azure_logs_ingestion output plugin would split a chunk into 1-N segments that each have a payload size under the limit.

If one or more of the requests to the Logs Ingestion API fail, the plugin returns FLB_RETRY or FLB_ERROR to FB core. If retries are enabled, some log records might be ingested multiple times in the case of transient failures.

Screenshots

Fluent Bit v5.1.2
* Copyright (C) 2015-2026 The Fluent Bit Authors
* Fluent Bit is a CNCF graduated project under the Fluent organization
* https://fluentbit.io

______ _                  _    ______ _ _           _____  __  
|  ___| |                | |   | ___ (_) |         |  ___|/  | 
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   _|___ \ `| | 
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / /   \ \ | | 
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V //\__/ /_| |_
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/ \____(_)___/


[2026/09/14 22:30:43.368] [ warn] [output:azure_logs_ingestion:azure_logs_ingestion.0] http_status=413:
{"error":{"code":"ContentLengthLimitExceeded","message":"Maximum allowed content length: 1048576 bytes (1 MB). Provided content length: 2646001 bytes."}}
[2026/09/14 22:30:47.625] [ warn] [output:azure_logs_ingestion:azure_logs_ingestion.0] http_status=413:
{"error":{"code":"ContentLengthLimitExceeded","message":"Maximum allowed content length: 1048576 bytes (1 MB). Provided content length: 2646001 bytes."}}

Your Environment

• Version used: v5.1.2 (homebrew)
• Configuration:
• Environment name and version (e.g. Kubernetes? What version?):
• Server type and version:
• Operating System and version: macOS Tahoe (Darwin Kernel Version 25.6.0)
• Filters and plugins: dummy, azure_logs_ingestion

Additional context
GitHub's observability team spotted this bug while we evaluated significantly increasing Fluent Bit's average chunk size to reduce request overhead. Ideally request payloads are slightly under 1MiB. This bug hasn't impacted us yet.

https://learn.microsoft.com/en-us/azure/azure-monitor/fundamentals/service-limits#logs-ingestion-api

dotnet client's segmentation of the incoming log stream:
https://github.com/Azure/azure-sdk-for-net/blob/dc8ee2a68d951f812b1413bb74f659b65edc1d3c/sdk/monitor/Azure.Monitor.Ingestion/src/LogsIngestionClient.cs#L124-L125

Related: #1938