-
Notifications
You must be signed in to change notification settings - Fork 7k
feat: Make the models.dev domain configurable for offline environments #9258
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Conversation
|
Hey! Your PR title Please update it to start with one of:
Where See CONTRIBUTING.md for details. |
|
The following comment was made by an LLM, it may be inaccurate: No duplicate PRs found |
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
| return JSON.parse(json) as Record<string, Provider> | ||
| } | ||
| const json = await fetch("https://models.dev/api.json").then((x) => x.text()) | ||
| const url = Bun.env.MODELS_DEV_URL || "https://models.dev" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we have a single definition somewhere so we dont need the || everywhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
perhaps global or flag?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That does sound cleaner, I will look how other configurable things that can be overridden in environment variables handle this in the project.
Maybe the "env-or-constant" approach skips some other places where things can be configured.
|
Would it be possible to support file:///path/to/api.json? So I can download or define my own api.json file. |
I think that is supported by bun, but I'll try to figure out if it actually works: In your example |
Great! Thanks a lot~ |
What does this PR do?
There are several issues and pull requests (#8860 #4959 #8830 #8814 #8642 #7313) that discuss the issue that the models.dev domain can't be reached in offline environments. Even if you would use a local Ollama instance, the project tries to fetch
https://models.dev/api.json.The solution I propose is making the models.dev URL configurable, so you can use something in your local network to host the
api.jsonfile. A typical corporate environment might have products like Sonatype Nexus Repository that allow for proxying of specific files on the internet. It would be nice if aMODELS_DEV_URLcould be configured to contain the path to an internal host with the right files.How did you verify your code works?
I put my computer in airplane mode, configured the
MODELS_DEV_URLenvironment variable tohttp://localhost:8000/, ranpython3 -m http.serverin a directory withapi.json, and tried to use a local model.Fixes: #7313