How to exclude /actuator/health endpoint span

[alvin562]

We have a java service that we automatically instrumented with opentelemetry and everythings worked pretty well. However, one issue we have is we are seeing a lot of spans to /actuator/health endpoint (essentially a health check from our ELB). Does the java instrumentation provide any way we can exclude these spans from being emitted?

We tried looking into specific properties/env variables but couldn't find anything. Any feedback would be greatly appreciated. Thanks!

[trask]

(linking to #6597)

[alvin562]

Thanks @trask. Is there any other way we can somehow exclude this span while we await #6597? Unfortunately this is causing a flood of spans which we are trying to suppress. Oddly enough we are able to ignore such spans in Node.js by just using ignoreIncomingRequestHook I'm wondering if something similar or perhaps a workaround we can do in java.

[alvin562]

Actually nevermind just looked into the sampler in #6597, we can try and use that in the meantime. @trask To make sure my understandings correct should we create an entirely new project extension just like here? Once thats done should we import the opentelemetry-java-contrib project to use the RuleBasedRoutingSampler which we can then create our own Sampler from with our own rules? Please let me know if my understandings correct.

[alvin562]

Or specifically did we have to create an entire new extension project from scratch or would we be able to use the existing example? My concern with the sample extension repo is contains all the Demo extensions which we dont actually need. So im not sure how that would affect the otel java agent once we include the jar.

[jkwatson]

transferring to the instrumentation repository

[EricBuist]

What is this thing about creating a new extension? This seems to be over-complex for something that should be the default behavior. The actuator/health is supposed to be called multiple times by a controller, e.g., Kubernetes, and that will always result in a flood of spans.

[trajano]

I agree it's kind of ridiculous. However, I think the best place at the moment for this would've been the opentelemetry-collector which would ideally provide a simple way of dropping the span based on some rules. This will offload the responsibility and extra configuration from the instrumentation to something that is meant to handle the actual routing.

Unfortunately open-telemetry/opentelemetry-collector#8801 where I tried to start doing this doesn't even work correctly.

Note I also had to cheat by using debug because open-telemetry/opentelemetry-collector#8800 is still open.

[GoncaloPT]

Hi. Was this subject dropped? Is there a way of achieving the desired filtering from opentelemetry?

[jaydeluca]

this was solved via #1060

You can customize a sampler using declarative file config as seen in this example here