Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Default Options Value Override Feature #1859

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

MotorBottle
Copy link
Contributor

@MotorBottle MotorBottle commented Oct 1, 2024

已实现

实现默认参数的覆盖,可以修改一些参数的默认值而无需用户主动请求指定该参数的值。使用方法为在default主题的渠道设置中,以json格式指定需要覆盖参数的模型以及其对应的参数和值。

几个问题

  1. 目前只针对默认主题添加了此功能,其他主题可以正常使用但如需使用新功能需要进一步适配
  2. 目前支持默认覆盖的参数有限,如需针对一些模型的特定参数进行覆盖,可能需要用户手动添加。目前已支持的参数:temperature, max_tokens, top_p, frequency_penalty, presence_penalty, stop, n, stream, num_ctx。如需添加其它参数需前往relay/controller/text.go编辑applyParameterOverrides函数。
  3. 没有接触过golang,代码可能存在不完善的部分,如有问题请feel free指出

close #1753

我已确认该 PR 已自测通过,相关截图如下:
image
image

Copy link

codecov bot commented Oct 1, 2024

Codecov Report

Attention: Patch coverage is 0% with 100 lines in your changes missing coverage. Please review.

Project coverage is 1.20%. Comparing base (7e51b04) to head (b2cc54f).

Files with missing lines Patch % Lines
relay/controller/text.go 0.00% 83 Missing ⚠️
model/channel.go 0.00% 11 Missing ⚠️
relay/meta/relay_meta.go 0.00% 5 Missing ⚠️
middleware/distributor.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##            main   #1859      +/-   ##
========================================
- Coverage   1.21%   1.20%   -0.01%     
========================================
  Files        145     145              
  Lines      12235   12328      +93     
========================================
  Hits         149     149              
- Misses     12072   12165      +93     
  Partials      14      14              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@MotorBottle MotorBottle changed the title Add Default Options Value Override in Panel Add Default Options Value Override Feature Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Default Options Value Override in Panel
1 participant