[FEATURE] ModelConfig CRD should provide field for context size

[marians]

📋 Prerequisites

• I have searched the existing issues to avoid creating a duplicate
• By submitting this issue, you agree to follow our Code of Conduct

📝 Feature Summary

The ModelConfig spec should provide a field to store the LLM's context window size/limit. This can be used to inform users and to control compaction.

❓ Problem Statement / Motivation

As an agent user in an interactive session, I want to know how much of the model's context window is already used, so I can decide whether there is room for more token intensive operations, or whether compaction is required.

For a context compaction mechanism, it's relevant to trigger compaction well before the context size limit is reached.

kagent currently has no way to store the context size limit, with the exception of Ollama-specific options (see #1218) .

💡 Proposed Solution

The ModelConfig CRD should provide a specific, provider-agnostic spec field.

#1173 proposed something similar, but had a broader scope and got closed as not planned.

🔄 Alternatives Considered

No response

🎯 Affected Service(s)

None

📚 Additional Context

No response

🙋 Are you willing to contribute?

• I am willing to submit a PR for this feature

[marians]

Some design detail, since I'm offering to implement this.

Placement: top-level, not per-provider

spec.contextWindow, a sibling of spec.model — not inside the per-provider blocks.

The existing token fields are per-provider for a good reason: each SDK names and shapes its output budget differently (maxTokens for OpenAI/Anthropic/Azure, maxCompletionTokens for OpenAI, maxOutputTokens for Gemini/Bedrock). The context window is the opposite case — it's a property of the model itself, identical in meaning for every provider, and no provider API accepts it as a request parameter. Duplicating it into nine provider blocks would give nine places to set the same number.

apiVersion: kagent.dev/v1alpha2
kind: ModelConfig
metadata:
  name: sonnet
spec:
  model: claude-sonnet-4-6
  provider: Anthropic
  contextWindow: 200000     # new: total tokens the model accepts per request
  anthropic:
    maxTokens: 8192         # existing: output budget, unchanged

Suggested semantics: total tokens per request as the provider counts them (prompt + tool schemas + completion budget), optional, +kubebuilder:validation:Minimum=1, no default. Declared by the operator, never inferred. kagent does not verify it against the provider — a wrong value is the operator's problem, exactly like maxTokens today.

Consumers

Three, in rough order of value:

1. Compaction threshold as a fraction. Today agent.spec.contextConfig.compaction.tokenThreshold is an absolute prompt-token count, so you must know the window and do the arithmetic yourself — and nothing stops you putting tokenThreshold: 150000 on an agent pointed at a 32k model. With a declared window:

spec:
  contextConfig:
    compaction:
      tokenThresholdPercent: 70   # of the model's contextWindow

tokenThreshold stays supported and wins if both are set. This is the motivation in the issue body — triggering compaction well before the limit — expressed in the only unit that survives a model swap.

2. Interactive display. "63% of context used" needs a measured numerator and a declared denominator. A2A: report turn-total token usage on the terminal status update #2303 (turn-total token usage on the A2A terminal status) is the numerator; this issue is the denominator. Neither is sufficient alone, which is worth noting since they'll likely be reviewed separately.

3. Preflight accounting. [BUG] Asking broad questions result in prompt too long. #1877 (broad questions → Bedrock 400 ValidationException) was closed as not planned, but the fix proposed in its comments required "the real Bedrock model window" to estimate input + tool payload + output budget before the call. That input doesn't exist today. This field supplies it, without committing this issue to implementing the preflight itself.

Scope limits, stated up front

• It's declarative, not enforcing. No admission-time rejection of oversized requests in this change.
• No model catalog. Auto-populating known windows per model id is a plausible follow-up — the /models handler currently tracks only {Name, FunctionCalling} — but it's a separate discussion, and hardcoded model metadata ages badly. Manual and explicit first.

Open question: Ollama

num_ctx in spec.ollama.options already sets the server-side window (#1218 / #1245), so for Ollama the two could disagree — contextWindow describing the model while num_ctx configures the runtime. Options: leave them independent and document it, warn on mismatch, or default num_ctx from contextWindow when unset.

Change surface

If this direction is acceptable I'd send: field + validation on ModelConfigSpec (v1alpha2 only), CRD regeneration, tokenThresholdPercent resolved against it in the agent translator, unit tests for the resolution and precedence rules, plus an E2E for the new CRD field per CONTRIBUTING.

Happy to split the ModelConfig field and the compaction consumer into two PRs if that reviews more easily.