Skip to content

Commit a9b81db

Browse files
authored
Add DBNL Telemetry Exporter (#1111)
Add support for exporting telemetry to DBNL. Closes #1107 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. ## Summary by CodeRabbit * **New Features** * Added DBNL telemetry exporter to enable tracing and observability integration. * **Documentation** * Added a DBNL integration guide, new observability workflow page, and updated provider/integrations lists and table of contents to include DBNL. * **Examples** * Added DBNL configuration and a sample observability tutorial showing installation, credential setup, and how to analyze traces. Authors: - Renaud Bourassa (https://github.com/dbnl-renaud) Approvers: - Will Killian (https://github.com/willkill07) URL: #1111
1 parent 73f654f commit a9b81db

File tree

6 files changed

+226
-1
lines changed

6 files changed

+226
-1
lines changed

docs/source/extend/telemetry-exporters.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@ Examples of existing telemetry exporters include:
7878
- **Patronus**: Exports traces to Patronus via OTLP
7979
- **Galileo**: Exports traces to Galileo via OTLP
8080
- **RagaAI Catalyst**: Exports traces to RagaAI Catalyst
81+
- **DBNL**: Exports traces to DBNL via OTLP
8182

8283
## Quick Start: Your First Telemetry Exporter
8384

@@ -244,7 +245,7 @@ Specialized for OpenTelemetry-compatible services with many pre-built options:
244245
- **Use case**: OTLP-compatible backends, standard observability tools
245246
- **Base class**: `OtelSpanExporter`
246247
- **Data flow**: `IntermediateStep``Span`[Processing Pipeline]`OtelSpan` → Export
247-
- **Pre-built integrations**: Langfuse, LangSmith, OpenTelemetry Collector, Patronus, Galileo, Phoenix, RagaAI, Weave
248+
- **Pre-built integrations**: Langfuse, LangSmith, OpenTelemetry Collector, Patronus, Galileo, Phoenix, RagaAI, Weave, DBNL
248249

249250
#### Advanced Custom Exporters
250251

@@ -269,6 +270,7 @@ Before creating a custom exporter, check if your observability service is alread
269270

270271
| Service | Type | Installation | Configuration |
271272
|---------|------|-------------|---------------|
273+
| **DBNL** | `dbnl` | `pip install "nvidia-nat[opentelemetry]"` | API URL + API token + project id |
272274
| **File** | `file` | `pip install nvidia-nat` | local file or directory |
273275
| **Langfuse** | `langfuse` | `pip install "nvidia-nat[opentelemetry]"` | endpoint + API keys |
274276
| **LangSmith** | `langsmith` | `pip install "nvidia-nat[opentelemetry]"` | endpoint + API key |

docs/source/workflows/observe/index.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,7 @@ Each exporter has its own detailed configuration guide with complete setup instr
133133
- **[Catalyst](https://catalyst.raga.ai/)** - See [Observing with Catalyst](./observe-workflow-with-catalyst.md)
134134
- **Custom Exporters** - See [Adding Telemetry Exporters](../../extend/telemetry-exporters.md) for creating custom integrations
135135
- **[NVIDIA Data Flywheel Blueprint](https://build.nvidia.com/nvidia/build-an-enterprise-data-flywheel)** - See [Observing with Data Flywheel](./observe-workflow-with-data-flywheel.md)
136+
- **[DBNL](https://distributional.com/)** - See [Observing with DBNL](./observe-workflow-with-dbnl.md)
136137
- **[Dynatrace](https://dynatrace.com/)** - See [Observing with Dynatrace](./observe-workflow-with-dynatrace.md)
137138
- **File Export** - Built-in file-based tracing for local development and debugging
138139
- **[Galileo](https://galileo.ai/)** - See [Observing with Galileo](./observe-workflow-with-galileo.md)
@@ -151,6 +152,7 @@ For complete configuration examples and setup instructions, refer to the individ
151152
|----------|-----------|
152153
| Catalyst | Logging, Tracing |
153154
| Data Flywheel | Logging, Tracing |
155+
| DBNL | Logging, Tracing |
154156
| Dynatrace | Logging, Tracing |
155157
| Galileo | Logging, Tracing |
156158
| Langfuse | Logging, Tracing |
@@ -202,6 +204,7 @@ For complete information about developing and integrating custom telemetry expor
202204
203205
Observing with Catalyst <./observe-workflow-with-catalyst.md>
204206
Observing with Data Flywheel <./observe-workflow-with-data-flywheel.md>
207+
Observing with DBNL <./observe-workflow-with-dbnl.md>
205208
Observing with Dynatrace <./observe-workflow-with-dynatrace.md>
206209
Observing with Galileo <./observe-workflow-with-galileo.md>
207210
Observing with OTEL Collector <./observe-workflow-with-otel-collector.md>
Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
<!--
2+
SPDX-FileCopyrightText: Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3+
SPDX-License-Identifier: Apache-2.0
4+
5+
Licensed under the Apache License, Version 2.0 (the "License");
6+
you may not use this file except in compliance with the License.
7+
You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
-->
17+
18+
# Observing a Workflow with DBNL
19+
20+
This guide provides a step-by-step process to enable observability in a NeMo Agent toolkit workflow using DBNL for tracing. By the end of this guide, you will have:
21+
22+
- Configured telemetry in your workflow.
23+
- Ability to view traces in the DBNL platform.
24+
25+
## Step 1: Install DBNL
26+
27+
Visit [https://docs.dbnl.com/get-started/quickstart](https://docs.dbnl.com/get-started/quickstart) to install DBNL.
28+
29+
## Step 2: Create a Project
30+
31+
Create a new Trace Ingestion project in DBNL. To create a new project in DBNL:
32+
33+
1. Navigate to your DBNL deployment (e.g. <http://localhost:8080/>)
34+
2. Go to Projects > + New Project
35+
3. Name your project `nat-calculator`
36+
4. Add a LLM connection to your project
37+
5. Select Trace Ingestion as the project Data Source
38+
6. Click on Generate API Token and note down the generated **API Token**
39+
7. Note down the **Project Id** for the project
40+
41+
## Step 3: Configure Your Environment
42+
43+
Set the following environment variables in your terminal:
44+
45+
```bash
46+
# DBNL_API_URL should point to your deployment API URL (e.g. http://localhost:8080/api)
47+
export DBNL_API_URL=<your_api_url>
48+
export DBNL_API_TOKEN=<your_api_token>
49+
export DBNL_PROJECT_ID=<your_project_id>
50+
```
51+
52+
## Step 4: Install the NeMo Agent toolkit OpenTelemetry Subpackages
53+
54+
```bash
55+
# Install specific telemetry extras required for DBNL
56+
uv pip install -e '.[opentelemetry]'
57+
```
58+
59+
## Step 5: Modify NeMo Agent toolkit Workflow Configuration
60+
61+
Update your workflow configuration file to include the telemetry settings.
62+
63+
Example configuration:
64+
```yaml
65+
general:
66+
telemetry:
67+
tracing:
68+
dbnl:
69+
_type: dbnl
70+
```
71+
72+
## Step 6: Run the workflow
73+
74+
From the root directory of the NeMo Agent toolkit library, install dependencies and run the pre-configured `simple_calculator_observability` example.
75+
76+
**Example:**
77+
78+
```bash
79+
# Install the workflow and plugins
80+
uv pip install -e examples/observability/simple_calculator_observability/
81+
82+
# Run the workflow with DBNL telemetry settings
83+
# Note: you may have to update configuration settings based on your DBNL deployment
84+
nat run --config_file examples/observability/simple_calculator_observability/configs/config-dbnl.yml --input "What is 1*2?"
85+
```
86+
87+
As the workflow runs, telemetry data will start showing up in DBNL.
88+
89+
## Step 7: Analyze Traces Data in DBNL
90+
91+
Analyze the traces in DBNL. To analyze traces in DBNL:
92+
93+
1. Navigate to your DBNL deployment (e.g. http://localhost:8080/)
94+
2. Go to Projects > nat-calculator
95+
96+
For additional help, see the [DBNL docs](https://docs.dbnl.com/).

examples/observability/simple_calculator_observability/README.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,6 +251,36 @@ Transmit traces to Galileo for workflow observability.
251251
nat run --config_file examples/observability/simple_calculator_observability/configs/config-galileo.yml --input "Is 100 > 50?"
252252
```
253253

254+
### Analyze Traces with DBNL
255+
256+
[DBNL](https://www.distributional.com/) helps you understand your agent by analyzing your traces.
257+
258+
1. Install DBNL:
259+
260+
Visit [https://docs.dbnl.com/get-started/quickstart](https://docs.dbnl.com/get-started/quickstart) to install DBNL.
261+
262+
2. Create a trace ingestion project:
263+
264+
Navigate to your DBNL deployment and go to Projects > + New Project
265+
266+
Create a trace ingestion project and generate an API token
267+
268+
Take note of the API token and project id
269+
270+
3. Set your DBNL credentials:
271+
272+
```bash
273+
# DBNL_API_URL should point to your deployment API URL (e.g. http://localhost:8080/api)
274+
export DBNL_API_URL=<your_api_url>
275+
export DBNL_API_TOKEN=<your_api_token>
276+
export DBNL_PROJECT_ID=<your_project_id>
277+
```
278+
279+
4. Run the workflow
280+
281+
```bash
282+
nat run --config_file examples/observability/simple_calculator_observability/configs/config-dbnl.yml --input "Is 100 > 50?"
283+
```
254284

255285
## Configuration Files
256286

@@ -266,6 +296,7 @@ The example includes multiple configuration files for different observability pl
266296
| `config-patronus.yml` | Patronus | AI safety and compliance monitoring |
267297
| `config-catalyst.yml` | Catalyst | RagaAI Catalyst integration |
268298
| `config-galileo.yml` | Galileo | Galileo integration |
299+
| `config-dbnl.yml` | DBNL | AI product analytics |
269300

270301
## What Gets Traced
271302

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2024-2025, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
16+
17+
general:
18+
telemetry:
19+
logging:
20+
console:
21+
_type: console
22+
level: WARN
23+
file:
24+
_type: file
25+
path: ./.tmp/nat_simple_calculator.log
26+
level: DEBUG
27+
tracing:
28+
dbnl:
29+
_type: dbnl
30+
31+
function_groups:
32+
calculator:
33+
_type: calculator
34+
35+
functions:
36+
current_datetime:
37+
_type: current_datetime
38+
39+
llms:
40+
nim_llm:
41+
_type: nim
42+
model_name: meta/llama-3.1-70b-instruct
43+
temperature: 0.0
44+
max_tokens: 1024
45+
46+
workflow:
47+
_type: react_agent
48+
tool_names: [calculator, current_datetime]
49+
llm_name: nim_llm
50+
verbose: true
51+
parse_agent_response_max_retries: 3

packages/nvidia_nat_opentelemetry/src/nat/plugins/opentelemetry/register.py

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,3 +193,45 @@ async def galileo_telemetry_exporter(config: GalileoTelemetryExporter, builder:
193193
drop_on_overflow=config.drop_on_overflow,
194194
shutdown_timeout=config.shutdown_timeout,
195195
)
196+
197+
198+
class DBNLTelemetryExporter(BatchConfigMixin, TelemetryExporterBaseConfig, name="dbnl"):
199+
"""A telemetry exporter to transmit traces to DBNL."""
200+
201+
api_url: str | None = Field(description="The DBNL API URL.", default=None)
202+
api_token: str | None = Field(description="The DBNL API token.", default=None)
203+
project_id: str | None = Field(description="The DBNL project id.", default=None)
204+
205+
206+
@register_telemetry_exporter(config_type=DBNLTelemetryExporter)
207+
async def dbnl_telemetry_exporter(config: DBNLTelemetryExporter, builder: Builder):
208+
"""Create a DBNL telemetry exporter."""
209+
210+
from nat.plugins.opentelemetry import OTLPSpanAdapterExporter
211+
212+
api_token = config.api_token or os.environ.get("DBNL_API_TOKEN")
213+
if not api_token:
214+
raise ValueError("API token is required for DBNL")
215+
project_id = config.project_id or os.environ.get("DBNL_PROJECT_ID")
216+
if not project_id:
217+
raise ValueError("Project id is required for DBNL")
218+
219+
headers = {
220+
"Authorization": f"Bearer {api_token}",
221+
"x-dbnl-project-id": project_id,
222+
}
223+
224+
api_url = config.api_url or os.environ.get("DBNL_API_URL")
225+
if not api_url:
226+
raise ValueError("API url is required for DBNL")
227+
endpoint = api_url.rstrip("/") + "/otel/v1/traces"
228+
229+
yield OTLPSpanAdapterExporter(
230+
endpoint=endpoint,
231+
headers=headers,
232+
batch_size=config.batch_size,
233+
flush_interval=config.flush_interval,
234+
max_queue_size=config.max_queue_size,
235+
drop_on_overflow=config.drop_on_overflow,
236+
shutdown_timeout=config.shutdown_timeout,
237+
)

0 commit comments

Comments
 (0)