Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Automatic Model List Retrieval for SiliconCloud LLM API Node #135

Merged
merged 14 commits into from
Oct 16, 2024

Conversation

lcolok
Copy link
Contributor

@lcolok lcolok commented Sep 12, 2024

Implement Automatic Model List Retrieval for SiliconCloud LLM API Node

Description

This PR implements the functionality of automatically retrieving the model list for the "☁️BizyAir SiliconCloud LLM API" node. By querying the https://api.siliconflow.cn/v1/models interface, it dynamically updates the model dropdown options, thereby enhancing user experience and allowing users to access real-time information about available models.

Changes

The automatic model list retrieval for the "☁️BizyAir SiliconCloud LLM API" node is implemented through frontend-backend interaction:

Backend:

  • Added a method to fetch the latest model list from the SiliconFlow API
  • Implemented a new POST endpoint /bizyair/get_silicon_cloud_models as the interface for the frontend to retrieve the model list

Frontend:

  • Automatically sends a request to the newly added backend endpoint to fetch the latest model list when the node is created
  • Dynamically updates the node's model selector options after receiving the model list

Frontend-Backend Interaction:

  • When a user creates a SiliconCloud LLM API node in ComfyUI, the frontend automatically triggers a request to the backend
  • The backend receives the request and calls the SiliconFlow API to get the latest model list
  • The backend returns the retrieved model list to the frontend
  • The frontend receives the data and updates the node's model selector in real-time, presenting the latest available models to the user

This implementation ensures that users see the most up-to-date model list every time they create a node, without the need for manual updates or application restarts.

Current Limitations

Currently, the API response does not include information about model categories or whether models are free or not. This limits our ability to effectively filter models. As a result, all available models are displayed without categorization.

Future Improvements

SiliconFlow has indicated that model filtering functionality will be supported in the near future. Once this feature becomes available, we can further improve this implementation:

  • Specifically filter out large language models
  • Add descriptions for whether models are free or paid, such as adding the prefix "(Free)" to model names to identify free models

image


为SiliconCloud LLM API节点实现自动获取模型列表功能

描述

本 PR 为"☁️BizyAir SiliconCloud LLM API"节点实现了自动获取模型列表的功能。通过查询 https://api.siliconflow.cn/v1/models 接口,动态更新模型下拉选项,从而提升了用户体验,使用户能够实时了解可用的模型。

变更

为"☁️BizyAir SiliconCloud LLM API"节点实现了自动获取模型列表的功能,主要通过前后端的联动实现:

后端:

  • 新增了从 SiliconFlow API 获取最新模型列表的方法
  • 实现了新的 POST 端点 /bizyair/get_silicon_cloud_models,作为前端获取模型列表的接口

前端:

  • 在节点创建时,自动向后端新增的端点发送请求获取最新模型列表
  • 获取到模型列表后,动态更新节点的模型选择器选项

前后端联动:

  • 当用户在 ComfyUI 中创建 SiliconCloud LLM API 节点时,前端自动触发对后端的请求
  • 后端接收请求后,调用 SiliconFlow API 获取最新模型列表
  • 后端将获取到的模型列表返回给前端
  • 前端接收到数据后,实时更新节点的模型选择器,呈现给用户最新的可用模型

这种实现确保了每次创建节点时,用户都能看到最新的模型列表,无需手动更新或重启应用。

当前局限性

目前,API 返回的信息不包含模型分类或是否免费等信息,这限制了我们有效筛选模型的能力。因此,所有可用模型都会被显示,没有进行分类。

未来改进

SiliconFlow 表示近期将支持模型筛选功能。一旦这个功能可用,我们可以进一步改进此实现:

  • 专门筛选出大语言模型
  • 补充对模型为免费还是付费的描述,比如沿用之前在模型名称前面加上(Free)这种字符串标识来标识免费模型

@doombeaker
Copy link
Collaborator

非常感谢你的工作,做了我们一直想做但是没做的事情。

有个小小的建议,现在 model list 已经可以通过加 type 参数过滤出语言模型了,可否同步更新上:

import requests

url = "https://api.siliconflow.cn/v1/models"

querystring = {"type":"text"}

headers = {"Authorization": "Bearer xxxx"}

response = requests.request("GET", url, headers=headers, params=querystring)

print(response.text)

llm.py Outdated Show resolved Hide resolved
llm.py Outdated Show resolved Hide resolved
@doombeaker doombeaker merged commit daa35e6 into siliconflow:master Oct 16, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants