Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 125 additions & 17 deletions components/camel-ai/camel-openai/src/main/docs/openai-component.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -256,17 +256,6 @@ from("direct:conversation")
.log("Second response: ${body}"); // Will remember "Alice"
----

=== Using Third-Party or Local OpenAI-Compatible Endpoint

.Usage example:
[source,java]
----
from("direct:local")
.setBody(constant("Hello from local LLM"))
.to("openai:chat-completion?baseUrl=http://localhost:1234/v1&model=local-model")
.log("${body}");
----

== Input Handling

The component accepts the following types of input in the message body:
Expand Down Expand Up @@ -492,17 +481,136 @@ String-valued fields are set directly. Non-string fields (numbers, booleans, obj

NOTE: This maps fields from the response message's additional properties (fields not part of the standard schema). Standard response fields like `content`, `role`, and `tool_calls` are not accessible through this option.

== Compatibility
== OpenAI-Compatible Providers

Because the component speaks the OpenAI API, you do not need a separate component to use third-party
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new section overlaps with the existing == Compatibility section further down (currently at line 495 on main), which lists the same providers. It also overlaps with === Using Third-Party or Local OpenAI-Compatible Endpoint (line 259 on main) which has a near-identical LM Studio example.

Now that this comprehensive section exists, consider:

  • Removing the brief === Using Third-Party or Local OpenAI-Compatible Endpoint section (line 259) since it's now a subset of this content
  • Trimming the == Compatibility section to avoid repeating the provider list, or merging it into this section

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching the overlap. Pushed 2264fbd which:

  • Removes === Using Third-Party or Local OpenAI-Compatible Endpoint (was line 259); its example is now covered by === LM Studio (local) in the new section.
  • Removes the == Compatibility heading along with its intro paragraph, bullet list, and NOTE block (redundant with the new provider table).
  • Preserves the caveat about provider variations from the old == Compatibility NOTE as a NOTE: in the intro of == OpenAI-Compatible Providers.
  • Promotes === Embedding Models by Provider to sit under == OpenAI-Compatible Providers, right before == Embeddings Operation, so all provider info stays grouped.

Net: -24/+3 lines, single source of truth for OpenAI-compatible provider docs.

Reported by Claude Code on behalf of Karol Krawczyk

gateways or local model servers that expose an OpenAI-compatible API. Point `baseUrl` at the provider,
set `apiKey` if it requires one, and use the same operations and options as with OpenAI.

NOTE: Ensure the provider supports the chat completions and/or embeddings endpoint format you use.
Authentication requirements and minor API variations differ between providers.

[cols="1,2,2",options="header"]
|===
| Provider | `baseUrl` | Notes

| OpenAI | `https://api.openai.com/v1` | Default; `baseUrl` can be omitted
| OpenRouter | `https://openrouter.ai/api/v1` | Multi-model gateway with provider routing and fallbacks
| Ollama | `http://localhost:11434/v1` | Local LLM server
| LM Studio | `http://localhost:1234/v1` | Local model runner
| vLLM | `http://localhost:8000/v1` | High-throughput, self-hosted serving engine
|===

=== Ollama (local)

https://ollama.com[Ollama] runs models locally and does not require an API key. Install Ollama and
pull the model used in the example below:

[source,bash]
----
ollama run llama3.2
----

[tabs]
====
Java::
+
[source,java]
----
from("direct:chat")
.setBody(constant("What is Apache Camel?"))
.to("openai:chat-completion?baseUrl=http://localhost:11434/v1&model=llama3.2");
----

YAML::
+
[source,yaml]
----
- to:
uri: openai:chat-completion
parameters:
baseUrl: http://localhost:11434/v1
model: llama3.2
userMessage: What is Apache Camel?
----
====

For local embeddings, use an embedding model such as `nomic-embed-text` (see the *Embedding Models by
Provider* table below).

=== LM Studio (local)

https://lmstudio.ai[LM Studio] serves the model currently loaded in the app. Set `model` to the
identifier shown in its UI.

[source,java]
----
from("direct:chat")
.to("openai:chat-completion?baseUrl=http://localhost:1234/v1&model=local-model");
----

=== vLLM (self-hosted)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add the vLLM command to install and run meta-llama/Llama-3.1-8B-Instruct, as in the example route? and a link to the vLLM website?

For example, locally I am using vllm-mlx a lot on mac os, can you mention this as well? For example, this is how I run it on my mac vllm-mlx serve mlx-community/Qwen3.6-35B-A3B-4bit --port 8000


https://docs.vllm.ai[vLLM] is a high-throughput LLM serving engine. Install it with `pip install vllm`
and start the model used in the example below:

[source,bash]
----
vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000
----

[source,java]
----
from("direct:chat")
.to("openai:chat-completion?baseUrl=http://localhost:8000/v1"
+ "&model=meta-llama/Llama-3.1-8B-Instruct");
----

This component works with any OpenAI API-compatible endpoint by setting the `baseUrl` parameter. This includes:
If vLLM was started with `--api-key`, pass the same value via the `apiKey` option.

- OpenAI official API (`https://api.openai.com/v1`)
- Local LLM servers (e.g., Ollama, LM Studio, LocalAI)
- Third-party OpenAI-compatible providers
On Apple Silicon, the `vllm-mlx` variant uses Apple's MLX framework and runs the same way:

[source,bash]
----
vllm-mlx serve mlx-community/Qwen2.5-7B-Instruct-4bit --port 8000
----

=== OpenRouter
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you manage to actually test it with openrouter? To be honest, when I opened the jira issue, I never used openrouter, I just supposed it would work :D if you have openrouter, and can do a quick test would be nice, otherwise, no worries, maybe @orpiske can test and review the docs (if needed) in the future


https://openrouter.ai[OpenRouter] is an OpenAI-compatible gateway that routes requests across many
model providers. Set `baseUrl` to its endpoint and select a model with a cross-provider identifier:

[source,java]
----
from("direct:chat")
.to("openai:chat-completion?baseUrl=https://openrouter.ai/api/v1"
+ "&apiKey={{openrouter.api.key}}"
+ "&model=anthropic/claude-sonnet-4-20250514");
----

==== Provider Routing

OpenRouter accepts a `provider` object in the request body to control routing order and fallbacks.
Pass it through the `additionalBodyProperty` option as a JSON value — the component parses JSON-valued
properties and adds them to the request body:

[source,yaml]
----
- to:
uri: openai:chat-completion
parameters:
baseUrl: https://openrouter.ai/api/v1
apiKey: "{{openrouter.api.key}}"
model: anthropic/claude-sonnet-4-20250514
additionalBodyProperty.provider: '{"order":["anthropic","google"],"allow_fallbacks":false}'
----

[NOTE]
====
When using local or third-party providers, ensure they support the chat completions and/or embeddings API endpoint format. Some providers may have different authentication requirements or API variations.
OpenRouter's optional attribution headers (`HTTP-Referer` and `X-Title`, which identify your app on
the OpenRouter rankings) are sent as HTTP request headers, not body properties. The component does not
currently expose a way to set custom HTTP request headers, so they cannot be configured today. They
are optional and do not affect chat completions.
====

=== Embedding Models by Provider
Expand Down