Skip to content

feat: add an install-time switch to run without LiteLLM - #1301

Closed
dylan-openhands wants to merge 1 commit into
mainfrom
dj/litellm-flag
Closed

dylan-openhands wants to merge 1 commit into
mainfrom
dj/litellm-flag

Conversation

@dylan-openhands

@dylan-openhands dylan-openhands commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Description

Adds an "Enable LiteLLM gateway" admin-console toggle (enable_litellm, default on). Off means no LiteLLM at all: no proxy pod or database, no LITE_LLM_* env, no litellm-config init container, no budget cronjob or upgrade hooks, no LiteLLM status informers, and no LiteLLM check in the TLS preflight. Only ENABLE_LEGACY_LITELLM=false remains.

With it off, BYOK is forced on and the selected provider seeds the default model for new users, called directly: the first line of its models list plus the key entered in the admin console (read from litellm-env-secrets). This covers Anthropic, OpenAI, Gemini, DeepSeek, Mistral, Groq, OpenRouter and Custom. Bedrock, Vertex AI and Azure need LiteLLM and show a notice instead. LiteLLM-only fields (admin console, proxy hostname, LiteLLM DB name, header forwarding, budget upgrade checks) are hidden.

Chart:

  • litellm.enabled (default true) selects no-gateway mode; true with litellm-helm.enabled: false still means external LiteLLM. The render fails if litellm.enabled: false while litellm-helm.enabled: true.
  • defaultLlm.* sets OPENHANDS_LLM_PROVIDER_ROUTE=direct and OPENHANDS_DEFAULT_LLM_MODEL/BASE_URL/API_KEY.

Requires the enterprise image from OpenHands/enterprise#538 (stack #522). Merge after that stack ships in an enterprise release and the chart's image tag is bumped to it; otherwise the toggle is available with an app that ignores ENABLE_LEGACY_LITELLM.

Helm Chart Checklist

  • I have tested the chart upgrade path from the previous version
  • I have verified backwards compatibility with existing values.yaml configurations
  • I have updated the chart's README.md if there are any breaking changes or new required values

Additional Notes

  • Defaults are unchanged: with LiteLLM on, the chart renders byte-identical to main for both bundled and external LiteLLM.
  • Switching an existing install from on to off works (orgs and their Default profile move to the install default on next load; verified on the fleet). Off to on isn't supported yet.
  • Tests: helm unittest charts/openhands (39 suites / 196 tests), pytest scripts/test_*.py (307 passed).

@dylan-openhands
dylan-openhands marked this pull request as ready for review October 12, 2026 17:41
@dylan-openhands

Copy link
Copy Markdown
Contributor Author

This PR has become stale, will re-open or re-assess as needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: feat A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant