Lorenze/adding guardrails #3711
Closed
+525
−78
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Added a new `guardrails` field to the Task class to allow for multiple guardrail functions or string descriptions for output validation. - Implemented a model validator to ensure guardrails are processed correctly, supporting both callable functions and string descriptions. - Updated the task execution flow to invoke guardrails sequentially, handling validation and retry logic. - Added comprehensive tests for various guardrail scenarios, including sequential processing, validation failures, and mixed return types. This update improves the flexibility and robustness of task output validation, ensuring better control over task execution outcomes.
- Updated the type annotation for the `guardrails` field in the Task class from `list` to `Sequence` to enhance type safety and compatibility with various sequence types. - This change improves type checking and aligns with best practices for handling callable functions and string descriptions in guardrail validation. This update contributes to the overall robustness and maintainability of the Task class.
- Updated the Task class to ensure that guardrails can only be used when an agent is provided, improving error handling and validation. - Introduced a smart task factory function to automatically assign a mock agent when guardrails are present, maintaining backward compatibility. - Updated tests to utilize the new smart task factory, ensuring proper functionality with and without guardrails. This update enhances the robustness of task execution and guardrail integration, ensuring better control over task validation outcomes.
| if self.guardrails is not None and ( | ||
| not isinstance(self.guardrails, (list, tuple)) or len(self.guardrails) > 0 | ||
| ): | ||
| if self.agent is None: |
There was a problem hiding this comment.
I think the agent check is too early maybe, only string-based guardrails need an agent, right?
| @model_validator(mode="after") | ||
| def ensure_guardrails_is_list_of_callables(self) -> "Task": | ||
| guardrails = [] | ||
| if self.guardrails is not None and ( |
There was a problem hiding this comment.
did you want to check for empty lists here?
|
covered by #3728 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.