Add python telemetry hopper to land telemetry data in Azure Monitor #249
+376
−14
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3 tasks
aditjha-msft
force-pushed
the
user/aditjha/pythonAzureMonitor
branch
2 times, most recently
from
ee91462 to
f866cfc
Compare
aditjha-msft
added a commit
that referenced
this pull request
May 30, 2025
This PR introduces support for using the OTEL Go sdk to instrument the tool using OTEL standards and practices. It provides initialization logic needed for a global tracer to be used across the tool's execution. For starters, the PR adds a customization time span, with more telemetry to be added in follow up PRs that help with data collection tied to the tool's KPIs. The PR also adds logic to disable telemetry, and uses a no-op tracer when `--disable-telemetry` is passed in. This follows OTEL standardized method of disabling telemetry collection in a clean manner. As part of telemetry initialization, the `telemetry.go` file has been added to set up the OTEL tracer and the OTLP grpc exporter. The exporter will send data to a Python service listening at the local port, which will then forward the data to Azure Monitor. In the effort of splitting up PRs in chunks, the PR for the complement Python telemetry service is here: #249. --- ### **Checklist** - [x] Tests added/updated - [x] Documentation updated (if needed) - [x] Code conforms to style guidelines
aditjha-msft
force-pushed
the
user/aditjha/pythonAzureMonitor
branch
from
7b45fd3 to
2a3dc66
Compare
cwize1
requested changes
Jun 2, 2025
| @@ -0,0 +1,101 @@ | |||
| asgiref==3.8.1 | |||
There was a problem hiding this comment.
Why are there so many packages in this file?
toolkit/tools/go.mod
Outdated
| gonum.org/v1/gonum v0.15.0 | ||
| gopkg.in/alecthomas/kingpin.v2 v2.2.6 | ||
| gopkg.in/ini.v1 v1.67.0 | ||
| gopkg.in/yaml.v3 v3.0.1 | ||
| ) | ||
|
|
||
| require ( |
There was a problem hiding this comment.
Why are there changes to the go.mod file?
| @@ -6,7 +6,18 @@ ARG BASE_IMAGE="mcr.microsoft.com/azurelinux/base/core:3.0" | |||
| FROM ${BASE_IMAGE} | |||
| RUN tdnf update -y && \ | |||
| tdnf install -y qemu-img rpm coreutils util-linux systemd openssl \ | |||
| sed createrepo_c squashfs-tools cdrkit parted e2fsprogs dosfstools \ | |||
| xfsprogs zstd veritysetup grub2 grub2-pc systemd-ukify binutils lsof | |||
| sed createrepo_c squashfs-tools cdrkit parted e2fsprogs dosfstools \ | |||
| RUN python3 -m venv /opt/telemetry-venv && \ | ||
| /opt/telemetry-venv/bin/pip install --no-cache-dir -r /usr/local/bin/requirements.txt | ||
|
|
||
| RUN chmod +x /usr/local/bin/entrypoint.sh && \ |
There was a problem hiding this comment.
We don't need this for the other script files added to the container.
Note: Git will save/restore the execute bit (but not any other permission).
|
|
||
| # Start telemetry service if enabled | ||
| if [[ "$ENABLE_TELEMETRY" == "true" ]]; then | ||
| /opt/telemetry-venv/bin/python /usr/local/bin/telemetry_hopper.py > /dev/null 2>&1 || true & |
| from opentelemetry.proto.trace.v1.trace_pb2 import Span as ProtoSpan | ||
| from opentelemetry.proto.common.v1.common_pb2 import KeyValue | ||
|
|
||
| DEFAULT_GRPC_PORT = 4317 |
There was a problem hiding this comment.
It might be good to pass the port number in as a param so that there is only 1 source of truth for what this value is.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces the Python telemetry hopper which will be used in the container scenario to forward telemetry from image customizer to Azure Monitor. This approach is needed for a variety of reasons, most importantly because Azure Monitor doesn't provide a Go SDK for instrumentation, and other solutions come with security and auth challenges. See design doc here: Image Customizer OTel Integration.docx
This will work alongside PR: #241
if --disable-telemetry is passed in to the docker command, the telemetry hopper will not be started and collection and export will be disabled:
Checklist