diff --git a/src/app/docs/kagent/supported-providers/azure-ai-foundry/page.mdx b/src/app/docs/kagent/supported-providers/azure-ai-foundry/page.mdx index caa6ed3b..99d57b61 100644 --- a/src/app/docs/kagent/supported-providers/azure-ai-foundry/page.mdx +++ b/src/app/docs/kagent/supported-providers/azure-ai-foundry/page.mdx @@ -12,7 +12,7 @@ export const metadata = { ## Configuring Azure AI Foundry -> Foundry chat models and memory embeddings require the Go agent runtime (`runtime: go`). Chat models must be available through Foundry's OpenAI-compatible chat completions API. Claude and other models are not yet supported. +> **Note:** Foundry chat models and memory embeddings require the Go agent runtime (`runtime: go`). Chat models must be available through Foundry's OpenAI-compatible chat completions API. Claude and other models are not yet supported. The following steps use API key authentication. To authenticate without an API key, see [Workload Identity](#workload-identity). @@ -61,7 +61,7 @@ spec: systemMessage: "You are a helpful assistant." ``` -4. To create the agent declaratively, save the preceding manifests as `foundry-model.yaml` and `foundry-agent.yaml`, then apply them to your cluster. +4. Save the manifests from steps 2 and 3 as `foundry-model.yaml` and `foundry-agent.yaml`, then apply them to your cluster. ```bash kubectl apply -f foundry-model.yaml @@ -96,7 +96,7 @@ The runtime chooses a credential based on the fields in the `ModelConfig`. ### Workload Identity -Omit `apiKeySecret` and `apiKeyPassthrough` to use Azure Workload Identity. For local development, the runtime will attempt to authenticate with your Azure CLI login. +Omit `apiKeySecret` and `apiKeyPassthrough` to use Azure Workload Identity. For local development, the runtime tries to authenticate using your Azure CLI login. > **Note:** The runtime validates Azure credentials at startup. If credentials cannot be resolved, the agent does not become ready. @@ -259,4 +259,4 @@ For the complete memory and embedding configuration and model requirements, see - **The `ModelConfig` reports `Accepted=False`:** Check whether the required `endpointFrom` ConfigMap and key exist with `kubectl describe modelconfig MODEL_CONFIG_NAME --namespace NAMESPACE`. - **The agent fails to become ready with a Workload Identity credential error:** Confirm the pod label and ServiceAccount annotation, and verify that the federated credential subject matches the ServiceAccount used by the pod. -- **Foundry returns `401 Unauthorized` or `403 Forbidden`:** Confirm that the managed identity has the `Cognitive Services User` role on the Foundry resource, or that the configured API key has access to the resource. \ No newline at end of file +- **Foundry returns `401 Unauthorized` or `403 Forbidden`:** Confirm that the managed identity has the `Cognitive Services User` role on the Foundry resource, or that the configured API key has access to the resource. diff --git a/src/app/docs/kagent/supported-providers/azure-openai/page.mdx b/src/app/docs/kagent/supported-providers/azure-openai/page.mdx index b2b65460..800e932a 100644 --- a/src/app/docs/kagent/supported-providers/azure-openai/page.mdx +++ b/src/app/docs/kagent/supported-providers/azure-openai/page.mdx @@ -65,7 +65,7 @@ After you apply the `ModelConfig`, you can select it from the **Model** dropdown > **Note:** Azure Workload Identity requires the Go agent runtime (`runtime: go`). -Omit `apiKeySecret` and `apiKeyPassthrough` to use Azure Workload Identity. For local development, the runtime will attempt to authenticate with your Azure CLI login. +Omit `apiKeySecret` and `apiKeyPassthrough` to use Azure Workload Identity. For local development, the runtime tries to authenticate using your Azure CLI login. > **Note:** The runtime validates Azure credentials at startup. If credentials cannot be resolved, the agent does not become ready. diff --git a/src/app/docs/kagent/supported-providers/byo-openai/page.mdx b/src/app/docs/kagent/supported-providers/byo-openai/page.mdx index 6e68c23e..e6f1e106 100644 --- a/src/app/docs/kagent/supported-providers/byo-openai/page.mdx +++ b/src/app/docs/kagent/supported-providers/byo-openai/page.mdx @@ -60,11 +60,11 @@ Good job! You added a model to kagent. Next, you can [create or update an agent] ## Self-hosted vLLM behind an OpenAI-compatible gateway -A common self-hosted pattern places an OpenAI-compatible gateway such as [Bifrost](https://github.com/maximhq/bifrost) or [LiteLLM](https://docs.litellm.ai/) in front of a [vLLM](https://docs.vllm.ai/) server (kagent → gateway → vLLM). Configure it as an OpenAI-compatible provider exactly as above, with two extra things to get right. +A common self-hosted pattern places an OpenAI-compatible gateway such as [Bifrost](https://github.com/maximhq/bifrost) or [LiteLLM](https://docs.litellm.ai/) in front of a [vLLM](https://docs.vllm.ai/) server (kagent → gateway → vLLM). Configure the gateway as an OpenAI-compatible provider, exactly as shown above, with two extra things to get right. ### Enable tool calling in vLLM -kagent sends a `tools` array with `tool_choice: "auto"` on every request. Its runtime registers a built-in `ask_user` tool on every agent, so this happens even when you configure no tools yourself. The vLLM backend **must** be launched with automatic tool choice enabled, or every agent turn fails. +kagent sends a `tools` array with `tool_choice: "auto"` on every request. kagent's runtime registers a built-in `ask_user` tool on every agent, so a `tools` array is always sent, even when you configure no tools yourself. The vLLM backend **must** be launched with automatic tool choice enabled, or every agent turn fails. ```bash vllm serve Qwen/Qwen2.5-7B-Instruct \ @@ -72,7 +72,7 @@ vllm serve Qwen/Qwen2.5-7B-Instruct \ --tool-call-parser hermes ``` -The correct `--tool-call-parser` depends on your model family. See the [vLLM tool calling docs](https://docs.vllm.ai/en/latest/features/tool_calling.html) for the current list. At the time of writing, Qwen2.5 uses `hermes` and Llama 3.1 uses `llama3_json`, but parser names change across vLLM releases, so check the docs for your model. +The correct `--tool-call-parser` depends on your model family. For example, Qwen2.5 uses `hermes` and Llama 3.1 uses `llama3_json`. Parser names change across vLLM releases, so check the [vLLM tool calling docs](https://docs.vllm.ai/en/latest/features/tool_calling.html) for your model's current parser name. ### Use the gateway's model identifier @@ -95,7 +95,7 @@ spec: ### Troubleshooting: `provider API error (status 400)` -If every agent message fails with a generic `status 400`, vLLM was almost certainly started without `--enable-auto-tool-choice` and a matching `--tool-call-parser`. Because kagent always sends `tool_choice: "auto"`, vLLM rejects the request shape until automatic tool choice is enabled. Restart vLLM with the flags above and retry. +If every agent message fails with a generic `status 400`, the most common cause is that vLLM started without `--enable-auto-tool-choice` and a matching `--tool-call-parser`. Because kagent always sends `tool_choice: "auto"`, vLLM rejects the request until automatic tool choice is enabled. Restart vLLM with the flags above and retry. ## TLS Configuration