fix(guardrails_ai): add valid alias to action results (#1578)

[ichbinlucaskim]

Related Issues

• Fixes bug: GuardrailsAI integration: NeMo expects result["valid"] but GuardrailsAI returns validation_passed #1578

Summary

This PR fixes a mismatch between the GuardrailsAI actions and the
GuardrailsAI Colang flows:

• Colang flows (nemoguardrails/library/guardrails_ai/flows.co and flows.v1.co)
  expect $result["valid"] when deciding whether to block.
• GuardrailsAI actions (validate_guardrails_ai_input /
  validate_guardrails_ai_output) previously returned only
  {"validation_result": <ValidationOutcome>} without a valid key.
• At runtime, the raw action return is stored in $result, while
  output_mapping is only used by is_output_blocked(). As a result,
  $result contained validation_result (with validation_passed in
  GuardrailsAI ≥ 0.3) but no valid key, leading to KeyError: 'valid'
  when flows evaluated $result["valid"].

What this PR changes

1. GuardrailsAI actions

   File: nemoguardrails/library/guardrails_ai/actions.py

   • validate_guardrails_ai_input and validate_guardrails_ai_output now:

     result = validate_guardrails_ai(validator, text, **joined_parameters)
     valid = guardrails_ai_validation_mapping(result)
     return {**result, "valid": valid}

   • validate_guardrails_ai continues to return a dict with a single
     validation_result key, so passing its result into
     guardrails_ai_validation_mapping is safe.

   This makes $result["valid"] available in Colang flows while preserving
   validation_result unchanged.

2. Tests

   Files:

   • tests/test_guardrails_ai_actions.py
   • tests/test_guardrails_ai_e2e_actions.py

   Updates:

   • Add unit tests to assert that validate_guardrails_ai_input and
     validate_guardrails_ai_output return both validation_result and
     valid, with valid reflecting the GuardrailsAI validation_passed
     outcome.
   • Align test_validation_mapping_e2e with the contract of
     guardrails_ai_validation_mapping (returns a boolean).
   • Add an e2e-style test to verify that the actions return a dict with
     validation_result and valid suitable for use in Colang flows.

3. Docs

   File: docs/user-guides/community/guardrails-ai.md

   • Add a short “Result Format in Colang Flows” section explaining that
     GuardrailsAI actions return a dict stored in $result with:
     • validation_result: raw GuardrailsAI outcome.
     • valid: boolean derived from validation_passed, intended for
       conditions like if not $result["valid"].

4. Changelog

   File: CHANGELOG.md

   • Under ## [0.20.0] → ### 🐛 Bug Fixes:

     - *(guardrails_ai)* Add `valid` alias to action results for Colang flows

Rationale

During investigation I checked the current develop branch (actions, Colang
flows, and related tests/commits) to confirm this is not intentional behavior
and that there is no existing adapter writing valid into $result.

There were two possible fixes:

• Option A: change the Colang flows to read validation_passed
  directly (e.g. $result["validation_result"].validation_passed).
• Option B: keep the flows as-is and add a backward-compatible adapter
  in the GuardrailsAI action layer, so actions return validation_result
  plus a valid boolean.

This PR implements Option B to maintain backward compatibility with
existing and custom flows that already use $result["valid"], while
still aligning with GuardrailsAI’s validation_passed field under the
hood.

Testing

Local tests (virtualenv):

python -m pytest tests/test_guardrails_ai_actions.py -v
python -m pytest tests/test_guardrails_ai_e2e_actions.py -v

All GuardrailsAI unit tests pass. GuardrailsAI e2e tests that do not
depend on external validators/services pass; the rest are skipped when
those validators are not available.

Description

Checklist

• I've read the CONTRIBUTING guidelines.
• I've updated the documentation if applicable.
• I've added tests if applicable.
• @mentions of the person or team responsible for reviewing proposed changes.
  • @okhleif-10

[github-actions]

Documentation preview

https://nvidia-nemo.github.io/Guardrails/review/pr-1611

[greptile-apps]

Greptile Overview

Greptile Summary

This PR fixes a critical runtime issue where Colang flows expected $result["valid"] but GuardrailsAI actions only returned {"validation_result": ...}, causing KeyError exceptions.

Key Changes:

• Modified validate_guardrails_ai_input and validate_guardrails_ai_output to return both validation_result and valid keys
• The valid key is a boolean derived from validation_result.validation_passed using the existing guardrails_ai_validation_mapping function
• Added comprehensive unit and e2e tests verifying the new return format
• Corrected existing test expectations for guardrails_ai_validation_mapping (returns boolean, not dict)
• Updated documentation to explain the result format for Colang flows
• Added changelog entry under bug fixes

The implementation maintains backward compatibility by preserving the original validation_result while adding the valid alias required by Colang flows.

Confidence Score: 5/5

• This PR is safe to merge - it fixes a critical bug with a minimal, well-tested solution
• The fix is straightforward, well-tested, maintains backward compatibility, and directly addresses the KeyError issue in Colang flows. All changes are additive (adding the valid key) without removing existing functionality.
• No files require special attention - all changes are clean and well-structured

Important Files Changed

Filename	Overview
nemoguardrails/library/guardrails_ai/actions.py	Added valid key to action return dicts to fix Colang flow compatibility
tests/test_guardrails_ai_actions.py	Added unit tests to verify both validation_result and valid keys are returned
tests/test_guardrails_ai_e2e_actions.py	Fixed validation mapping test and added e2e test for Colang flow compatibility

Sequence Diagram

sequenceDiagram
    participant Flow as Colang Flow
    participant Action as validate_guardrails_ai_input/output
    participant Core as validate_guardrails_ai
    participant Mapping as guardrails_ai_validation_mapping
    participant Guard as GuardrailsAI Guard

    Flow->>Action: Call action with validator, text
    Action->>Core: validate_guardrails_ai(validator, text, params)
    Core->>Guard: guard.validate(text, metadata)
    Guard-->>Core: ValidationResult (with validation_passed)
    Core-->>Action: {"validation_result": ValidationResult}
    Action->>Mapping: guardrails_ai_validation_mapping(result)
    Mapping-->>Action: boolean (validation_passed)
    Action-->>Flow: {"validation_result": ..., "valid": boolean}
    Note over Flow: Can now access $result["valid"]<br/>without KeyError

Loading

[greptile-apps]

_{3 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!