-
-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LiteLLM Minor Fixes & Improvements (01/16/2025) - p2 #7828
base: main
Are you sure you want to change the base?
Conversation
allows http calls to work
* Update azure.py Added optional parameter azure ad token provider * Added parameter to main.py * Found token provider arg location * Fixed embeddings * Fixed ad token provider --------- Co-authored-by: Krish Dholakia <[email protected]>
get v0 out for sync azure gpt route to begin with
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
model does not support it
Does this PR allow calling of a Bedrock model via a REST endpoint with a Bearer token? |
i'm confused @sterankin that's always been possible - https://docs.litellm.ai/docs/providers/bedrock#3-test-it |
google ai studio raises errors
@krrishdholakia - that seems to require an AWS Account ID and secret - however I have my bedrock behind an API Gateway with an authorizer so I need to be able to call the endpoint with a Bearer Token - we are not hitting bedrock directly. Api Gateway - > Lambda -> Bedrock Is it possible to use LiteLLM to call a custom url and add custom header as well as Bearer token for Auth? for example like this:
litellm.exceptions.AuthenticationError: litellm.AuthenticationError: BedrockException Invalid Authentication - Unable to locate credentials |
Yes that should already work @sterankin I believe we have a test for this as well. Let me add the flow to our docs. I think there's a minor change you need to make in your example |
Thanks for the help @krrishdholakia - I look forward to seeing what change is needed I couldnt see it in the docs but maybe they need updated. |
Hi @krrishdholakia did you find how my example should be changed? |
Hey @sterankin added it here - https://docs.litellm.ai/docs/providers/bedrock#calling-via-proxy |
Does this solve your problem? @sterankin |
@krrishdholakia Thankyou for updating the docs here: https://docs.litellm.ai/docs/providers/bedrock#calling-via-proxy However in the code you have the line:
What should client be set to in the example? Thanks |
@sterankin that's from my test - forgot to clean it up, done now. It was from this test -
|
I kep getting this error:
Why is it trying to use aws auth and boto3 when I am passing a Bearer token to the API for auth? |
@sterankin can we track this on a separate ticket - i believe we currently assume you have credentials, just need to route the call via proxy. in your case, does your proxy contain the aws credentials? |
Title
Relevant issues
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes
[REQUIRED] Testing - Attach a screenshot of any new tests passing locally
If UI changes, send a screenshot/GIF of working UI fixes