Stop Posit AI models from truncating long replies at 4,096 tokens - #82
Merged
Conversation
Models without a declared output limit (GLM-5.2 today) sent no max_tokens, so the server's 4,096 default truncated long replies. The fetcher now applies a 16,384 fallback before the capabilities spread, so explicit per-model entries (Kimi K3: 131,072) still win.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When you chat with a Posit AI model that doesn't declare its own output limit — GLM 5.2 today — a long reply cuts off mid-thought at exactly 4,096 tokens. The assistant never tells the server how much output to allow, so the server's small default applies and the response ends with a "length" stop reason.
The assistant now sends a default limit of 16,384 output tokens for every Posit AI model, matching the baseline it already uses for other providers. Models that declare their own limit, such as Kimi K3 at 131,072, keep it.
If a model is new or otherwise unknown to the assistant, the 16,384 default applies instead of the server's. No server changes are required. The new value takes effect on the next model-list refresh; cached model lists refresh within an hour.