Skip to content

Latest commit

 

History

History
58 lines (46 loc) · 1.88 KB

README.md

File metadata and controls

58 lines (46 loc) · 1.88 KB
Raw

ODCL (OTel Data Collector for LLM)

ODCL (OTel Data Collector for LLM) is the tool or template to generate OpenTelemetry metrics for various LLM and LLM Applications. All implementation are based on predefined OpenTelemetry Semantic Conventions.

Requirements

  • Java 11+

Ensure that Java SDK 11+ is installed.

java -version

Installation

  1. Download the installation package:
curl -O https://github.com/instana/otel-dc/releases/download/v1.0.5/otel-dc-llm-1.0.5.tar
  1. Extract the package to the desired deployment location:
tar vxf otel-dc-llm-1.0.5.tar

Configuration

Configure otel dc

cd otel-dc-llm-1.0.5
vi config/config.yaml

The following options are required：

  • otel.agentless.mode: The connection mode of the OTel data connector, the default mode is agentless.
  • otel.backend.url: The gRPC endpoint of the Instana backend or Instana agent, that depends on agentless or not.
  • callback.interval: The time interval in seconds to post data to backend or agent.
  • otel.service.name: The Data Collector name, which can be any string that you choose.
  • otel.service.port: The listen port of Data Collector for receiving the metrics data from the instrumented applications, the default port is 8000.

Configure model price

vi config/prices.properties

Customize more price items by the following format:

<aiSystem>.<modelId>.input=0.0
<aiSystem>.<modelId>.output=0.0

The can be set to '*' to match all modelIds within the aiSystem, but this has a lower priority than items with a specific modelId specified.

Run ODCL

Run the Data Collector with the following command according to your current system:

nohup ./bin/otel-dc-llm &

Reference

If your platform is not supported by the pre-built binaries or if you prefer to compile from source, you can follow the steps in DEVELOP.md