Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elasticsearch Output does not allow setting retry_on_conflict to 0 with data stream set to true #16716

Open
BenB196 opened this issue Nov 21, 2024 · 0 comments

Comments

@BenB196
Copy link

BenB196 commented Nov 21, 2024

Logstash information:

Please include the following information:

  1. Logstash version: 8.15.4
  2. Logstash installation source: Docker
  3. How is Logstash being run: Kubernetes

Plugins installed: (bin/logstash-plugin list --verbose)

JVM (e.g. java -version):

If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:

  1. JVM version (java -version)
  2. JVM installation source (e.g. from the Operating System's package manager, from source, etc).
  3. Value of the LS_JAVA_HOME environment variable if set.

OS version (uname -a if on a Unix-like system):

Description of the problem including expected versus actual behavior:

When setting an Elasticsearch output with data_stream => true, you cannot set retry_on_conflict => 0.

This causes issues where integrations intentionally set _id fields and causes Elasticsearch to retry even when it is known that it will fail.

Steps to reproduce:

Please include a minimal but complete recreation of the problem,
including (e.g.) pipeline definition(s), settings, locale, etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

  1. Create Elasticsearch output
  2. Set data_stream => true
  3. Set retry_on_conflict => 0
  4. Observe log message stating that retry_on_conflict => 0 is invalid for data streams.

Provide logs (if relevant):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant