Skip to content

feat(models): add LiteLLM SDK model provider - #1169

Open
prodmanpd wants to merge 1 commit into
ScrapeGraphAI:pre/betafrom
prodmanpd:feat/add-litellm-provider
Open

prodmanpd wants to merge 1 commit into
ScrapeGraphAI:pre/betafrom
prodmanpd:feat/add-litellm-provider

Conversation

@prodmanpd

Copy link
Copy Markdown

Summary

Adds LiteLLM as a model provider, so a graph can run on any provider the LiteLLM SDK supports (Anthropic, Bedrock, Vertex AI, Azure, Gemini, Groq, ...) with "model": "litellm/<provider>/<model>". LiteLLM is an optional extra (pip install scrapegraphai[litellm]) and is imported lazily, so the base install is unchanged.

Changes

  • scrapegraphai/models/litellm.py: LiteLLM chat model on litellm.completion / acompletion, with drop_params=True, provider env var credentials when api_key is not set, and usage and cost in the response metadata.
  • abstract_graph.py: litellm route and context window lookup.
  • custom_callback.py: uses the cost a model reports when it is not in the price table.
  • pyproject.toml / uv.lock: litellm>=1.94.0,<1.105.0 extra. litellm's minimums also bump openai (2.13 to 2.54), aiohttp, boto3/botocore, s3transfer, pydantic-settings and yarl in the shared lock.
  • test-suite.yml: adds the new test file to the CI list.
  • Example in examples/smart_scraper_graph/litellm/, README note, tests/test_litellm_model.py (23 tests).

Tests

  • CI unit test list: 118 passed (23 new). The new file also passes on litellm 1.94.0 and without litellm installed.
  • ruff, isort, black clean on changed files.
  • Live SmartScraperGraph with a Pydantic schema on litellm/vertex_ai/gemini-2.5-flash:
llm class   : LiteLLM
model_token : 1048576 (defaulted: False)
result      : {'products': [{'name': 'Hammer', 'price': 12.5}, {'name': 'Wrench', 'price': 8.0}, {'name': 'Screwdriver set', 'price': 19.99}]}
  • Live invoke and ainvoke on gpt-4.1-mini: cost split into prompt and completion, summed correctly by the callback.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant