Skip to content

feat: enhance model validation and error handling#243

Open
kerwin2046 wants to merge 8 commits into
agegr:mainfrom
kerwin2046:feat/model-validation-v2
Open

feat: enhance model validation and error handling#243
kerwin2046 wants to merge 8 commits into
agegr:mainfrom
kerwin2046:feat/model-validation-v2

Conversation

@kerwin2046

Copy link
Copy Markdown

Bug: models.json schema validation is all-or-nothing — one invalid model entry silently drops ALL custom providers
Reproduction:

  1. Defines multiple custom providers in ~/.pi/agent/models.json, e.g., new-provider-xx (3 models),new-provider-2xx (3 models), new-provider-3xx (1 model)
  2. One of the model entries has an empty id field: { "id": "" }
  3. The entire model dropdown in the UI shows zero custom models — only built-in providers appear
  4. No error or warning is shown anywhere
    Root cause:
    ModelConfig.load() in pi-coding-agent uses Typebox schema validation on the entire models.json file. If ANY single field across ANY provider violates the schema (e.g., Type.String({ minLength: 1 }) rejects ""), the whole file is discarded and ModelConfig returns an empty providers Map. ModelRuntime.rebuildProviders() then has no custom providers to register, so modelRuntime.getAvailable() returns only builtins — silently.
    The relevant code path:
    models.json → ModelConfig.load() → validateModelsConfig.Check(parsed) → false → new ModelConfig(new Map(), error)
    ↘ true → new ModelConfig(parsedProviders)
    Why it's insidious:
  • The error is stored in ModelConfig.error and exposed via ModelRuntime.getError(), but neither /api/models nor the UI ever read it
  • Users see "model dropdown is empty for my custom providers" with no clue why
  • A typo in one model definition poisons every other correctly-configured provider in the same file
  • The failure is silent and undebuggable without reading server logs
    Fix (pi-web side, defense-in-depth):
  • PUT /api/models-config: Inline validation before writing — checks each model's id is a non-empty string, provider structure is valid. Returns HTTP 400 with a clear message like Provider "new-provider-xx", model 无法使用 ChatGPT Plus 订阅作为模型供应商 #3: "id" must be a non-empty string. The ModelsConfig modal shows this inline in its footer.
  • GET /api/models: Checks modelRuntime.getError() specifically for schema/parse errors (Invalid models.json schema: / Failed to parse models.json:). Only these block model loading; availability/composition errors still allow partial results. Returns the error as configError in the response for upstream consumers.
image

kerwin2046 and others added 8 commits July 24, 2026 15:43
- Added validation for provider models and models.json structure to ensure correct data format.
- Implemented error reporting for invalid configurations, allowing for partial model loading despite configuration errors.
- Updated ModelsData interface to include optional configError field for better error tracking.
feat: enhance model validation and error handling
- Added validation for provider models and models.json structure to ensure correct data format.
- Implemented error reporting for invalid configurations, allowing for partial model loading despite configuration errors.
- Updated ModelsData interface to include optional configError field for better error tracking.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant