(Backport to r321) rules: Add new RuleEvaluationTimeSum field to groups (#15672)
#807
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* feat(ruler): Add new `RuleEvaluationTimeSum` field to groups Coupled with a metric: `rule_group_last_rule_duration_sum_seconds` This will give us more observability into how fast a group runs with or without concurrency Signed-off-by: Julien Duchesne <[email protected]> * Update rules/group.go Co-authored-by: gotjosh <[email protected]> Signed-off-by: Julien Duchesne <[email protected]> Signed-off-by: Julien Duchesne <[email protected]> * Apply suggestions from code review Co-authored-by: gotjosh <[email protected]> Signed-off-by: Julien Duchesne <[email protected]> Signed-off-by: Julien Duchesne <[email protected]> * Remove `in seconds`. A duration is a duration Signed-off-by: Julien Duchesne <[email protected]> --------- Signed-off-by: Julien Duchesne <[email protected]> Signed-off-by: Julien Duchesne <[email protected]> Co-authored-by: gotjosh <[email protected]>
julienduchesne
changed the title
RuleEvaluationTimeSum field to groups (#15672)RuleEvaluationTimeSum field to groups (#15672)
julienduchesne
changed the title
RuleEvaluationTimeSum field to groups (#15672)RuleEvaluationTimeSum field to groups (#15672)
colega
approved these changes
Dec 17, 2024
colega
added a commit
to grafana/mimir
that referenced
this pull request
Dec 17, 2024
This includes: - grafana/mimir-prometheus#807 - grafana/mimir-prometheus#806 Signed-off-by: Oleg Zaytsev <[email protected]>
4 tasks
julienduchesne
added a commit
to grafana/mimir
that referenced
this pull request
Dec 17, 2024
* Update mimir-prometheus weekly-r321 This includes: - grafana/mimir-prometheus#807 - grafana/mimir-prometheus#806 Signed-off-by: Oleg Zaytsev <[email protected]> * Rule Concurrency: Prevent flapping of concurrency (#10189) * Rule Concurrency: Prevent flapping of concurrency Iterates on #8146 The `isGroupAtRisk` function only uses the group's last evaluation time as a metric However, if the concurrency of the group causes the group's eval time to lower to less than the threshold, this will flap between enabling concurrency and disabling it on every run In this PR, a condition is added to also sum up the last evaluation time of each rule to compare against the threshold * Linting * Use the new `evaluationRuleTimeSum` field from the group * Linting * Add changelog + metric * Apply suggestions from code review Co-authored-by: gotjosh <[email protected]> * Unrevert crypto * Fix typo in changelog --------- Co-authored-by: gotjosh <[email protected]> --------- Signed-off-by: Oleg Zaytsev <[email protected]> Co-authored-by: Julien Duchesne <[email protected]> Co-authored-by: gotjosh <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
RuleEvaluationTimeSumfield to groupsCoupled with a metric:
rule_group_last_rule_duration_sum_secondsThis will give us more observability into how fast a group runs with or without concurrency