-
Notifications
You must be signed in to change notification settings - Fork 9
Initial auto model list implementation #41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Conversation
| local function get_model_list() | ||
| local headers = { | ||
| ["Content-Type"] = "application/json", | ||
| ["Authorization"] = "Bearer "..API_KEY, | ||
| } | ||
|
|
||
| local body_tbl = {} | ||
| local requestId = http.send_request(API_MODELS_URL, "GET", headers, body_tbl) | ||
|
|
||
| local response | ||
| local error | ||
|
|
||
| -- ugly 5s busy timeout | ||
| local sec = tonumber(os.clock() + 5); | ||
| while not response and not error and ( os.clock() < sec ) do | ||
| log.info("waiting for models response"); | ||
| response, error = http.check_response(requestId) | ||
| end | ||
|
|
||
| log.info("models response " .. inspect(response)) | ||
|
|
||
| local resultPairs = {} | ||
| if response and response.data then | ||
| log.info("models list array result" .. inspect(response.data)) | ||
| for i, v in ipairs(response.data) do | ||
| local pair = { v["id"], v["id"] } | ||
| log.info("adding the pair: " .. inspect(pair)) | ||
| table.insert(resultPairs, pair) | ||
| end | ||
| elseif error then | ||
| log.error("models error" .. inspect(error)) | ||
| else | ||
| log.info("models probably timeout") | ||
| end | ||
| log.info("models final table", inspect(resultPairs)) | ||
| return resultPairs | ||
| end | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This reminds me i need to add model list sanitization by default. I keep forgetting.
This returns... like 200+ models with gemini, nvidia combined.
| local API_URL = "http://127.0.0.1:8000/v1/chat/completions" | ||
| local API_MODELS_URL = "http://127.0.0.1:8000/v1/models" | ||
| local API_KEY = config.PROXY_API_KEY |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Honestly, this is a perfect thing to add to advanced settings in MCM - Base URL(API_URL in code). Maybe in the future. including the key
Fetch the model list from the proxy at the game start (with 5s timeout).
Option is not yet used, just shown in MCM menu.
Hardcoded to use the proxy provider for now.