controller: export wallclock lag metrics also for storage collections #30568
+270
−128
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
teskje
force-pushed
the
storage-wallclock-metrics
branch
from
0b85fcb
to
36834e8
Compare
In preparation of having the storage controller export wallclock lag metrics too, this commit factors out the common infrastructure from `mz-compute-client` and moves it into `mz-cluster-client`.
This commit makes minor changes to the code structure around `CollectionState` initialization in the storage controller. This removes some redundancy, but more importantly makes it easier to attach `WallclockLagMetrics` to the `CollectionState` in the next commit.
This commit wires the `ControllerMetrics` through to the storage controller, creates `WallclockLagMetrics` objects in the `CollectionState`s/`ExportState`s of all storage collections, and uses those to update the wallclock lag metrics during every maintenance call.
teskje
force-pushed
the
storage-wallclock-metrics
branch
from
36834e8
to
6ce30eb
Compare
|
MitigationsCompleting required mitigations increases Resilience Coverage.
Risk Summary:The pull request has a high-risk score of 81, driven by the predictors "Sum Bug Reports Of Files" and "Delta of Executable Lines." Historically, pull requests with these predictors are 115% more likely to cause a bug compared to the repository baseline. The repository's observed bug trend remains steady. Note: The risk score is not based on semantic analysis but on historical predictors of bug occurrence in the repository. The attributes above were deemed the strongest predictors based on that history. Predictors and the score may change as the PR evolves in code, time, and review activity. Bug Hotspots:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR rewires the existing
mz_dataflow_wallclock_lag_secondsmetric so that it also includes storage collections. To this end, a newControllerMetricstype is introduced to define metrics that are exported by both the compute and the storage controller, and the wallclock lag metrics are moved there. TheControllerMetricstype is then passed to both controllers, so they can export wallclock lag metrics for their respective collections.Note that in contrast to compute collections, the wallclock lag for storage collections is not per replica (as we are comparing with the global persist frontier here), and in some cases not even per cluster (as not all storage collections are associated with clusters). As a result, the
replica_idlabel is always empty for storage collections, and theinstance_idlabel is sometimes empty.Motivation
Part of https://github.com/MaterializeInc/database-issues/issues/8235
Checklist
$T ⇔ Proto$Tmapping (possibly in a backwards-incompatible way), then it is tagged with aT-protolabel.