otelcol: fix retry mechanism #5188
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The v0.80 upgrade introduced two new settings to the common retry mechanism shared between components. One of these new settings, `multiplier`, specifies the rate at which a backoff is applied. We did not convert this new field after upgrading, leading the multiplier to be set to the Go default (zero). After the first retry, the backoff period would be multiplied by zero for each subsequent retry, causing retries to happen in a hot loop.
rfratto
added a commit
to rfratto/agent
that referenced
this pull request
Sep 14, 2023
…y blocks Follow up on grafana#5188.
ptodev
pushed a commit
that referenced
this pull request
Sep 21, 2023
* otelcol: fix retry mechanism The v0.80 upgrade introduced two new settings to the common retry mechanism shared between components. One of these new settings, `multiplier`, specifies the rate at which a backoff is applied. We did not convert this new field after upgrading, leading the multiplier to be set to the Go default (zero). After the first retry, the backoff period would be multiplied by zero for each subsequent retry, causing retries to happen in a hot loop. * otelcol.exporter.loadbalancing: properly set defaults
ptodev
added a commit
that referenced
this pull request
Sep 22, 2023
* otelcol.processor.discovery should not modify its targets attribute (#5170) Fixed a bug where `otelcol.processor.discovery` could modify the `targets` passed by an upstream component. --------- Co-authored-by: Paschalis Tsilias <[email protected]> * otelcol: fix retry mechanism (#5188) * otelcol: fix retry mechanism The v0.80 upgrade introduced two new settings to the common retry mechanism shared between components. One of these new settings, `multiplier`, specifies the rate at which a backoff is applied. We did not convert this new field after upgrading, leading the multiplier to be set to the Go default (zero). After the first retry, the backoff period would be multiplied by zero for each subsequent retry, causing retries to happen in a hot loop. * otelcol.exporter.loadbalancing: properly set defaults * Rename component stability from "alpha" to "experimental". (#5200) * Rename stability from "alpha" to "experimental". This complies with Agent stability naming conventions. * Add an experimental stability block for "otelcol.processor.span" and "otelcol.connector.spanmetrics". * Sync loki.source.file with promtail (#5245) * Sync loki.source.file with promtail * changelog * Fix scraped targets metric (#5263) * Remove main branch changes from the Changelog * Add v0.36.2 to the changelog * Change date of v0.36.2 from 2023-09-21 to 2023-09-22 --------- Co-authored-by: Paschalis Tsilias <[email protected]> Co-authored-by: Robert Fratto <[email protected]> Co-authored-by: Piotr <[email protected]>
rfratto
added a commit
that referenced
this pull request
Sep 22, 2023
…y blocks (#5219) Follow up on #5188. Co-authored-by: Paschalis Tsilias <[email protected]> Co-authored-by: Paulin Todev <[email protected]>
hainenber
pushed a commit
to hainenber/agent
that referenced
this pull request
Sep 23, 2023
…y blocks (grafana#5219) Follow up on grafana#5188. Co-authored-by: Paschalis Tsilias <[email protected]> Co-authored-by: Paulin Todev <[email protected]>
github-actions
bot
added
the
frozen-due-to-age
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The v0.80 upgrade introduced two new settings to the common retry mechanism shared between components. One of these new settings,
multiplier, specifies the rate at which a backoff is applied.We did not convert this new field after upgrading, leading the multiplier to be set to the Go default (zero). After the first retry, the backoff period would be multiplied by zero for each subsequent retry, causing retries to happen in a hot loop.
A follow up PR should expose these new settings to the user rather than hard coding them at the new upstream defaults. I've split the additions of the new fields and fixing the defaults across two separate PRs in case we want to release this in a patch.