[Bug]: Is there a way to disable automatic conversion of input URLs to base64?

[icefox57]

What happened?

I encountered a problem when using litellm with Google Gemini models. My workflow involves distinguishing between smaller media files (processed as base64) and larger ones (uploaded to LeanCloud, with the URL passed to litellm). This distinction is made to avoid hitting the 20MB payload size limit, as documented.

However, even when I pass a URL (pointing to the uploaded file on LeanCloud) instead of base64 data, litellm still attempts to convert the file to base64 internally. This results in the following error for large files:

Environment
• litellm version: 1.55.4
• Python version: 3.9.x
• OS: Windows Server 2022

Relevant log output

litellm.llms.vertex_ai.common_utils.VertexAIError: {
  "error": {
    "code": 400,
    "message": "Request payload size exceeds the limit: 20971520 bytes.",
    "status": "INVALID_ARGUMENT"
  }
}

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.55.4

Twitter / LinkedIn details

No response

[krrishdholakia]

when i last checked gemini on google ai studio didn't allow url's to be passed, checking now

[krrishdholakia]

hmm i don't see file uri's working -

do you have a working curl for the gemini api working with url input? @icefox57

[icefox57]

I am using the liteLLM Python SDK for the Gemini API, and the input format I’m using is based on the documentation.

my logs confirm that the image was successfully uploaded, and I received a valid URL

2024-12-19 07:54:13,995 - INFO - Generated images for model 'gemini/gemini-1.5-flash': [{'type': 'image_url', 'image_url': 'https://lc-gluttony.s3.amazonaws.com/TpXoUG1Bhubl/BL1kACGnzeDV8dsvvQicjhr237GopzW9/41ce25a8-39c5-49be-b3db-09f9f4aa7c01.mp4'}]

[krrishdholakia]

@icefox57 your request is to disable url -> b64 but based on our testing the gemini api (google ai studio) doesn't accept url's

if you have a script calling the google ai studio endpoint with a url - https://ai.google.dev/gemini-api/docs

please can you share it, so we can work against that

[krrishdholakia]

Converting this to a discussion as it doesn't (currently) seem like a bug on litellm