[Ratelimit Processor] Instrument the ratelimiter service with telemetry metrics (#562)

* first effort to instrument

Signed-off-by: Andreas Gkizas <[email protected]>

* adding initial implementation for metrics provider in ratelimiter

Signed-off-by: Andreas Gkizas <[email protected]>

* updating test to read projectid from metadata

Signed-off-by: Andreas Gkizas <[email protected]>

* license update

Signed-off-by: Andreas Gkizas <[email protected]>

* update with correct attributes as fucntions

Signed-off-by: Andreas Gkizas <[email protected]>

* changing Meter namespace

Signed-off-by: Andreas Gkizas <[email protected]>

* make goproto

Signed-off-by: Andreas Gkizas <[email protected]>

* run with metadata

Signed-off-by: Andreas Gkizas <[email protected]>

* run with metadata and remove unneeded funcsions

Signed-off-by: Andreas Gkizas <[email protected]>

* make go generate

Signed-off-by: Andreas Gkizas <[email protected]>

* make lint

Signed-off-by: Andreas Gkizas <[email protected]>

* adding err for projectID

Signed-off-by: Andreas Gkizas <[email protected]>

* license update

Signed-off-by: Andreas Gkizas <[email protected]>

* remove files

Signed-off-by: Andreas Gkizas <[email protected]>

* fixing tests

Signed-off-by: Andreas Gkizas <[email protected]>

* make fmt

Signed-off-by: Andreas Gkizas <[email protected]>

* make default projectID empty

Signed-off-by: Andreas Gkizas <[email protected]>

* make gogenerate

Signed-off-by: Andreas Gkizas <[email protected]>

* make gogenerate

Signed-off-by: Andreas Gkizas <[email protected]>

* Update processor/ratelimitprocessor/gubernator.go

Co-authored-by: Vignesh Shanmugam <[email protected]>

* abstract the metadata function and use WithErrorReason

Signed-off-by: Andreas Gkizas <[email protected]>

* Update processor/ratelimitprocessor/gubernator.go

Co-authored-by: kaiyan-sheng <[email protected]>

* Update processor/ratelimitprocessor/gubernator.go

Co-authored-by: kaiyan-sheng <[email protected]>

* moving to common function and changed labels

Signed-off-by: Andreas Gkizas <[email protected]>

* adding to local and also duraton metric

Signed-off-by: Andreas Gkizas <[email protected]>

* fix reason values and rerun make generate

* Add limit_threshold and throttle_behavior attrs

* run  make license-update

* add test for limit threshold

* fix CodeGen and error message

* downgrade mdatagen to v0.124.0 to avoid missing import in generated file

* change mdatagen to v0.123.0

* remove project_id attr

* return errTooManyRequests when throttling

* Add goimports to fix the generated files

* add go install for goimports

* remove processor_id and reason field

* remove reason field when throttled in local.go

* fix errors

Signed-off-by: Andreas Gkizas <[email protected]>

* fix errors

Signed-off-by: Andreas Gkizas <[email protected]>

* fix errors

Signed-off-by: Andreas Gkizas <[email protected]>

* fix test

* move `metadata.NewTelemetryBuilder` to factory.go

* fix tests

* fix local_test.go

* remove attributes

* generate new documentation.md

* adding test with AssertEqualRatelimitRequests

Signed-off-by: Andreas Gkizas <[email protected]>

* add comment for the goimports workaround

---------

Signed-off-by: Andreas Gkizas <[email protected]>
Co-authored-by: Vignesh Shanmugam <[email protected]>
Co-authored-by: kaiyan-sheng <[email protected]>

Author: 3 people

.github/workflows/checks.yml

@@ -162,7 +162,7 @@ jobs:
         run: |
           make -j2 gogenerate && make -j2 license-update
           if [[ -n $(git status -s) ]]; then
-            echo 'Generated code is out of date, please run "make generate" and commit the changes in this PR.'
+            echo 'Generated code is out of date, please run "make gogenerate && make license-update" and commit the changes in this PR.'
             exit 1
           fi
   unittest-matrix:

Makefile

@@ -65,6 +65,8 @@ gotidy:
 
 .PHONY: gogenerate
 gogenerate:
+	# This is a workaround for a bug in mdatagen upstream: https://github.com/open-telemetry/opentelemetry-collector/issues/13069
+	@command -v goimports >/dev/null 2>&1 || $(MAKE) -B install-tools
 	@$(MAKE) $(FOR_GROUP_TARGET) TARGET="generate"
 	@$(MAKE) $(FOR_GROUP_TARGET) TARGET="fmt"
 

Makefile.Common

@@ -75,6 +75,7 @@ lint: $(LINT) misspell
 .PHONY: generate
 generate: $(MDATAGEN)
 	$(GOCMD) generate ./...
+	goimports -w ./
 
 .PHONY: moddownload
 moddownload:

connector/elasticapmconnector/generated_package_test.go

@@ -11,6 +11,7 @@ require (
 	go.opentelemetry.io/build-tools/multimod v0.22.0
 	go.opentelemetry.io/collector/cmd/builder v0.125.0
 	go.opentelemetry.io/collector/cmd/mdatagen v0.125.0
+	golang.org/x/tools v0.31.0
 	golang.org/x/vuln v1.1.4
 )
 
@@ -236,7 +237,6 @@ require (
 	golang.org/x/sys v0.32.0 // indirect
 	golang.org/x/telemetry v0.0.0-20240522233618-39ace7a40ae7 // indirect
 	golang.org/x/text v0.24.0 // indirect
-	golang.org/x/tools v0.31.0 // indirect
 	google.golang.org/genproto/googleapis/rpc v0.0.0-20250219182151-9fdb1cabc7b2 // indirect
 	google.golang.org/grpc v1.72.0 // indirect
 	google.golang.org/protobuf v1.36.6 // indirect

extension/apmconfigextension/generated_package_test.go

@@ -33,5 +33,6 @@ import (
 	_ "go.opentelemetry.io/build-tools/multimod"
 	_ "go.opentelemetry.io/collector/cmd/builder"
 	_ "go.opentelemetry.io/collector/cmd/mdatagen"
+	_ "golang.org/x/tools/cmd/goimports"
 	_ "golang.org/x/vuln/cmd/govulncheck"
 )