Skip to content

feat: Add Pillar Security guardrail integration #12791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

eagle-p
Copy link

@eagle-p eagle-p commented Jul 20, 2025

Add Pillar Security guardrail integration

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • I have added a screenshot of my new test passing locally
image - [x] My PR passes all unit tests on [`make test-unit`](https://docs.litellm.ai/docs/extras/contributing_code) - [x] My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature

Changes

Overview

This PR implements a comprehensive Pillar Security guardrail integration for LiteLLM, providing advanced security features including prompt injection protection, PII detection, secret detection, and content moderation.

Core Features Implemented

⚙️ Technical Features:

  • Multi-mode Execution: Support for pre_call, during_call, and post_call modes
  • Configurable Actions: block (raise exception) or monitor (log only) on flagged content
  • Session Tracking: Comprehensive conversation context analysis via metadata
  • Async Implementation: Full async/await support for optimal performance
  • Error Handling: Robust error handling with custom exception types

Files Added

Core Implementation:

  • litellm/proxy/guardrails/guardrail_hooks/pillar/pillar.py - Main PillarGuardrail class with async hooks
  • litellm/proxy/guardrails/guardrail_hooks/pillar/__init__.py - Module initialization and registry
  • litellm/types/proxy/guardrails/guardrail_hooks/pillar.py - Pydantic configuration model

Testing:

  • tests/guardrails_tests/test_pillar_guardrails.py - Comprehensive test suite (623 lines) with:
    • Configuration validation tests
    • Pre/during/post-call hook tests
    • Clean and flagged content scenarios
    • Error handling and edge cases
    • Payload and session tracking tests
    • Mocked API responses for reliable testing

Documentation:

  • docs/my-website/docs/proxy/guardrails/pillar_security.md - Complete user guide (408 lines) with:
    • Quick start guide and setup instructions
    • Multiple configuration examples (security vs. performance trade-offs)
    • API reference and usage examples
    • Troubleshooting and support information

Files Modified

Configuration Integration:

  • litellm/types/guardrails.py - Added PILLAR enum value and PillarGuardrailConfigModel
  • docs/my-website/sidebars.js - Added Pillar documentation to sidebar navigation

Configuration Examples

Recommended Dual-Mode Configuration:

guardrails:
  - guardrail_name: "pillar-dual-protection"
    litellm_params:
      guardrail: pillar
      mode: [pre_call, post_call]            # Monitor both input and output
      api_key: os.environ/PILLAR_API_KEY
      api_base: os.environ/PILLAR_API_BASE
      on_flagged_action: "monitor"           # Log threats but allow requests
      default_on: true

Alternative Configurations:

  • Input-only blocking for maximum security
  • Low-latency monitoring for performance-critical applications
  • Full monitoring for analytics and compliance

Integration Approach

  • Follows LiteLLM Patterns: Uses established guardrail patterns and base classes
  • Environment Variable Support: Supports PILLAR_API_KEY, PILLAR_API_BASE, PILLAR_ON_FLAGGED_ACTION
  • Metadata Integration: Uses LiteLLM's metadata system for session tracking
  • Async-First: Built with async/await for optimal performance
  • Error Resilience: Graceful degradation on API failures with detailed error reporting

Testing Coverage

  • Configuration Tests: Validates setup with/without API keys
  • Hook Integration Tests: Tests pre_call, moderation, and post_call hooks
  • Content Scenarios: Safe content, prompt injection, secrets detection
  • Error Handling: API failures, malformed responses, network issues
  • Edge Cases: Empty messages, missing fields, malformed data
  • Session Management: User ID and session ID extraction and tracking

Implements comprehensive LLM security guardrails using Pillar Security API with support for prompt injection detection, PII/secret detection, content moderation, and multi-mode execution (pre_call, during_call, post_call). Includes complete documentation, testing, and configurable actions on flagged content.
Copy link

vercel bot commented Jul 20, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
litellm ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 21, 2025 7:22am

Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.
Restructure PillarGuardrailConfigModel to properly inherit from GuardrailConfigModel[T]
and resolve return type compatibility issue in get_config_model method.
@eagle-p
Copy link
Author

eagle-p commented Jul 21, 2025

Ready to be merged :)

@krrishdholakia
Copy link
Contributor

Have you manually qa'ed this to confirm it works @eagle-p ?

@eagle-p
Copy link
Author

eagle-p commented Jul 22, 2025

Hi Krish!
Thank you for your time and amazing work!!
I manually QA’ed all key flows and everything looks good 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants