Add support for multi-run graphing in ilab / Crucible backend #120
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GET localhost:8000/api/v1/ilab/runs?benchmark=ilab will query the ilab.crucible OpenSearch instance and return a list of ilab benchmark runs.
Add Crucible readme file. Cleanups and refactoring
Also added the option to override the default graph title generator using the new `Graph.title` field.
This cleans up my direct API call to get the run's periods for graphing, to use a separate action and a reducer. I also experimented with trying to improve error diagnosis by looking at some of the error responses to "toast" instead of just saying something went wrong.
Add a Crucible `close` method, and use a FastAPI yield dependency to ensure every API connection is closed cleanly.
+ other review feedback
+ add some method documentation + misc review feedback
Multigraph API failed if more than one `Graph` element specified the same run; fix to be smarter about missing run IDs. This also contains experimental code to expose per-iteration param values, which doesn't quite work but doesn't seem to hurt anything.
(And `/api/v1/ilab/runs` reports iterations in numerical order.)
Move the legend up off the graph (although 1.5 is arbitrary and maybe not ideal -- is there a more dynamic way to do this). Also, I fixed the name of a `.less` class earlier, but just happened to notice the corresponding use...
Move "unique parameters" accordion back up with parameters using the new expansion toggles.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Type of change
Description
This draft PR is backed by PR #117 to add support for overlaying metric graphs across multiple runs. It'll be rebased on
mainand opened for review after #117 is merged.Related Tickets & Documents
Primarily PANDA-600.
Checklist before requesting a review
Testing
InstructLab CPT is using a persistent Crucible controller system in RDU3, tied to a 4-way L40S test system. The data store (a private OpenSearch instance) contains a set of Crucible runs capturing both training and SDG runs.