Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,4 @@ playwright-report/
*.sln
*.sw?

# Other
.env
*.db
4 changes: 4 additions & 0 deletions backend/omni/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
config*.yaml
!config.sample.yaml
*.db
.env
4 changes: 4 additions & 0 deletions backend/omni/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added

- Config can include other config files
- OpenAPI tool registry now supports `tool_servers` and auto-registers all `operationId` endpoints from each OpenAPI spec as tools

### Chnaged

- Config to be used can be defined via CONFIG_PATH environment variable
- Refactored tools contract to use request-aware registry methods: `get_tools(request)` and `run_tool(request, params)`
- Tool definitions now use OpenAI Responses API `FunctionToolParam` shape directly
- `GET /api/tools` now returns the raw tool-definition list (no `{ "tools": ... }` envelope)

## [0.0.1] - 2026-02-12

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ includes:
- path: ./src/modai/default_config.yaml

modules:
# Deactivate authentication completely for development purposes.
auth_oidc:
collision_strategy: drop
session:
Expand All @@ -11,8 +12,3 @@ modules:
user_id: "dev-user"
email: "dev@example.com"
name: "Dev User"
openapi_tool_registry:
config:
tools:
- url: http://localhost:8001/roll
method: POST
171 changes: 40 additions & 131 deletions backend/omni/docs/architecture/tools.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# Tools Architecture

## 1. Overview
- **Architecture Style**: Microservice-based tool system with a generic tool abstraction and a web layer that serves tools in OpenAI format
- **Architecture Style**: Microservice-based tool system with OpenAI-formatted tool definitions
- **Design Principles**:
- Tools are LLM-agnostic — a tool has a definition (name, description, parameters) and a run capability; neither is tied to any LLM API
- OpenAPI as one registry implementation — the OpenAPI registry fetches specs from microservices and creates tool instances that handle HTTP invocation internally
- Registry encapsulates invocation — callers do not need to know about URLs, methods, or HTTP; they just run a tool with parameters
- Web layer transforms definitions — the Tools Web Module converts tool definitions to OpenAI function-calling format
- Tool definitions are OpenAI-compatible at the registry boundary (`FunctionToolParam`)
- OpenAPI as one registry implementation — the OpenAPI registry fetches specs from microservices and executes operations over HTTP
- Registry encapsulates invocation — callers do not need to know about URLs, methods, or HTTP; they call `run_tool(request, params)`
- `ToolRegistryModule` is a plain interface (non-web)
- Central aggregation — `ToolsRouterModule` exposes `GET /api/tools`, returns tools without renaming, and dispatches `run_tool` by tool name
- Extensible — new registry implementations can plug in any tool backend (HTTP, gRPC, in-process functions, etc.) without changing callers
- **Quality Attributes**: Decoupled, language-agnostic, independently deployable, discoverable

Expand Down Expand Up @@ -118,151 +119,59 @@ The registry will build a tool definition that includes `user_id` and `order_id`

```mermaid
flowchart TD
FE[Frontend] -->|GET /api/tools| TW[Tools Web Module]
TW -->|get_tools| TR[Tool Registry Module]
TR -->|GET /openapi.json| TS1[Tool Service A]
TR -->|GET /openapi.json| TS2[Tool Service B]
FE[Frontend] -->|GET /api/tools| CTR[ToolsRouterModule]
CTR -->|aggregate get_tools| TR1[OpenAPIToolRegistry]
TR1 -->|GET /openapi.json| TS1[Tool Service A]
TR1 -->|GET /openapi.json| TS2[Tool Service B]
FE -->|POST /api/responses with tool names| CR[Chat Router]
CR --> CA[Chat Agent Module]
CA -->|get_tool_by_name| TR
CA -->|tool.run params | TS1
CA -->|tool.run params | TS2
CA -->|run_tool request+params| CTR
CTR -->|route by tool name| TR1
TR1 -->|HTTP invoke| TS1
TR1 -->|HTTP invoke| TS2
```

**Flow**:
1. Frontend calls `GET /api/tools` to discover all available tools
2. Tools Web Module asks the Tool Registry for all tools and converts their definitions to OpenAI format
2. Tool Registry returns tools as-is (already OpenAI format)
3. User selects which tools to enable for a chat session
4. Frontend sends `POST /api/responses` with tool names (as received from `GET /api/tools`)
5. When the LLM emits a `tool_call`, the Chat Agent looks up the tool by name in the registry
6. The Chat Agent runs the tool with the LLM-supplied parameters — the tool directly invokes the microservice; no registry involvement at invocation time
5. When the LLM emits a `tool_call`, the Chat Agent calls `run_tool` in the central router with the tool name
6. The central router resolves the registry by matching the tool name and delegates to the selected registry
7. The target registry resolves and invokes the operation and returns the result to the Chat Agent

## 4. Module Architecture
## 4. API Endpoints

### 4.1 Core Abstractions
- `GET /api/tools` — List all available tools across registries (tool names are returned unchanged)

**Tool Definition** — a value object with three fields:
- `name` — unique identifier derived from the OpenAPI `operationId`
- `description` — human-readable text describing what the tool does
- `parameters` — a fully-resolved JSON Schema (all `$ref` pointers inlined) describing the input

A tool definition contains enough information to construct an LLM tool call but is not tied to any specific LLM API.

**Tool** — pairs a definition with execution capability:
- Exposes its `definition` (read-only)
- Provides a `run(params)` operation that executes the tool with the given parameters and returns the result

#### Reserved `_`-prefixed keys in `params`

Callers may inject caller-supplied metadata into the `params` dict using keys prefixed with `_`. These keys are **never** forwarded to the tool microservice's JSON body — tool implementations must extract and consume them before sending the request.

Currently defined reserved keys:

| Key | Type | Description |
|---|---|---|
| `_bearer_token` | `str \| None` | Forwarded as `Authorization: Bearer <token>` HTTP header |

This convention keeps the `Tool.run` interface stable while allowing callers to pass through transport-level concerns (auth, tracing, etc.) without requiring interface changes.

### 4.2 Tool Registry Module (Plain Module)

**Purpose**: Aggregates tools from all configured sources and provides lookup by name.

**Responsibilities**:
- Return all available tools via `get_tools`
- Look up a tool by name via `get_tool_by_name`
- Handle unavailable tool services gracefully (skip with warning, don't fail)

**No module dependencies**: The registry does not depend on other modAI modules.

### 4.3 OpenAPI Tool Registry (concrete implementation)

**Purpose**: Concrete registry implementation that harvests OpenAPI specs from configured HTTP microservices.

**How it works**:
- On each call to `get_tools`, fetches `/openapi.json` from each configured service
- Extracts the tool definition from the spec:
- `operationId` → name
- `summary`/`description` → description
- Request body schema → parameters (all `$ref` resolved inline)
- Path parameters (`in: path`) from the `parameters` array are merged into the schema's `properties` and `required` lists so the LLM is told to supply them
- Each resulting tool's `run` operation:
1. Resolves `{param_name}` placeholders in the configured URL by substituting values from the supplied `params` dict
2. Sends the remaining parameters as the JSON request body
3. Makes an HTTP call to the resolved URL using the configured method

**Configuration** — each tool entry specifies:
- `url`: The full trigger endpoint URL of the tool microservice
- `method`: The HTTP method to use when invoking the tool (e.g. POST, PUT, GET)

### 4.4 Tools Web Module (Web Module)

**Purpose**: Exposes `GET /api/tools` endpoint. Transforms tool definitions into OpenAI function-calling format.

**Dependencies**: Tool Registry Module

**Responsibilities**:
- Expose `GET /api/tools` endpoint
- Call the Tool Registry to get all available tools
- Convert each tool definition to OpenAI function-calling format
- Return the transformed tool definitions to the frontend

### 4.5 Chat Agent Module

The Chat Agent Module receives a tool registry dependency. When the LLM emits a `tool_call`:
1. Extract the function name from the tool call
2. Look up the tool by name in the registry
3. Run the tool with the LLM-supplied parameters — no HTTP knowledge needed in the chat module
4. Return the result to the LLM

## 5. API Endpoints

- `GET /api/tools` — List all available tools in OpenAI function-calling format

### 5.1 List Available Tools
### 4.1 List Available Tools

**Endpoint**: `GET /api/tools`

**Purpose**: Returns all available tools in OpenAI function-calling format.
**Purpose**: Returns all available tools in OpenAI function-calling format aggregated from all configured registries.

**Tool Definition → OpenAI Transformation**:
- `name` → `function.name`
- `description` → `function.description`
- `parameters` → `function.parameters` (already resolved, no `$ref`)
The endpoint returns tool definitions in OpenAI function-tool format directly as a JSON list.

**Response Format (200 OK)**:
```json
{
"tools": [
{
"type": "function",
"function": {
"name": "calculate",
"description": "Evaluate a math expression",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "Math expression to evaluate"
}
},
"required": ["expression"]
[
{
"type": "function",
"name": "calculate",
"description": "Evaluate a math expression",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "Math expression to evaluate"
}
}
}
]
}
},
"required": ["expression"]
},
"strict": true
}
]
```

If a tool service is unreachable, it is omitted from the response and a warning is logged.

## 6. Configuration

The tool registry is configured with a list of tool microservice endpoints. Each entry has:
- `url`: The full trigger endpoint URL of the tool microservice
- `method`: The HTTP method used to invoke the tool (e.g. PUT, POST, GET)

The registry derives the base URL from `url` (strips the path) and appends `/openapi.json` to fetch the spec.

See `config.yaml` and `default_config.yaml` for concrete configuration examples.
2 changes: 1 addition & 1 deletion backend/omni/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,6 @@ check:
uv run ruff check src

# Fix code style and linting issues
check-write:
format:
uv run ruff format src
uv run ruff check --fix src
18 changes: 8 additions & 10 deletions backend/omni/src/modai/default_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -79,15 +79,13 @@ modules:
module_dependencies:
http_client: "http_client"
config:
tools: []
tool_servers: []
# Example:
# tools:
# - url: http://calculator-service:8000/calculate
# method: POST
# - url: http://web-search-service:8000/search
# method: PUT
# tool_servers:
# - url: http://calculator-service:8000/openapi.json
# - url: http://web-search-service:8000/openapi.json

tool_registry:
predefined_tool_registry:
class: modai.modules.tools.tool_registry_predefined_vars.PredefinedVariablesToolRegistryModule
module_dependencies:
delegate_registry: "openapi_tool_registry"
Expand All @@ -98,10 +96,10 @@ modules:
# variable_mappings:
# X-Session-Id: session_id

tools_web:
class: modai.modules.tools.tools_web_module.OpenAIToolsWebModule
tool_registry:
class: modai.modules.tools.tool_router.ToolsRouterModule
module_dependencies:
tool_registry: tool_registry
predefined: "predefined_tool_registry"

full_reset:
class: modai.modules.reset.reset_web_module.ResetWebModule
Expand Down
Loading
Loading