Spring Boot app cannot connect to Daemon in Docker container

[haiko]

I have instrumented a Spring Boot application with XRay. I run a Docker container with a proxy to XRay using the instructions on https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon-local.html. But my application cannot connect to the XRay daemon in the docker container. The daemon in the docker container start up logs the following:

2022-02-01T10:51:23Z [Info] Initializing AWS X-Ray daemon 3.3.3
2022-02-01T10:51:23Z [Info] Using buffer memory limit of 19 MB
2022-02-01T10:51:23Z [Info] 304 segment buffers allocated
2022-02-01T10:51:23Z [Info] Using region: eu-central-1
2022-02-01T10:51:23Z [Info] HTTP Proxy server using X-Ray Endpoint : https://xray.eu-central-1.amazonaws.com
2022-02-01T10:51:23Z [Info] Starting proxy http server on 127.0.0.1:2000

My application tries to connect but fails with the following exception:

2022-02-01 11:52:59.150 ERROR 48013 --- [pool-2-thread-1] c.a.x.s.sampling.pollers.RulePoller      : Encountered error polling GetSamplingRules: 
com.amazonaws.xray.internal.XrayClientException: Could not serialize and send request.
	at com.amazonaws.xray.internal.UnsignedXrayClient.sendRequest(UnsignedXrayClient.java:142)
	at com.amazonaws.xray.internal.UnsignedXrayClient.getSamplingRules(UnsignedXrayClient.java:112)
	at com.amazonaws.xray.strategy.sampling.pollers.RulePoller.pollRule(RulePoller.java:100)
	at com.amazonaws.xray.strategy.sampling.pollers.RulePoller.lambda$start$0(RulePoller.java:72)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.net.ConnectException: Connection refused (Connection refused)

When I run the xray daemon locally on my system everything works and traces are uploaded to AWS. Anyone an idea?

[wangzlei]

By default XRay Daemon turns on UDP port 2000 for sending trace to XRay and TCP port 2000 for remote sampling rules. I guess you are running Daemon in docker for Mac, so, get error log about Sampling rule poller but still see trace in xray console. There is a known issue that docker for Mac does not map to local TCP port, please try workaround by searching google, such as docker/for-mac#770

If not using remote sampling rule, you can ignore that log.

[Gaurav-Vishvakarman]

I am also facing same kind of issue and these stack traces are getting injected under cloudwatch which in turn increasing cost down the line. Is there any way using which we can disable polling GetSamplingRules at service or x-ray daemon level?