Allow custom TraceID to be set

[eduardtm]

Hey, I would like to be able to set the TraceID on the root span to a custom value which is deterministically determined from a random value. To give you the full picture, we use the modulo sampler to deterministically determine from a UUID request id whether it should be sampled. At some point after a series of grpc and kafka processing a db insert happens. We have a database log listener which fires kafka events for certian db inserts but we don’t have any way to forward the trace headers there. We store the same request id for idempotency reasons in the database so we want to resurect a trace if the request id matches one that should be sampled. I am happy to submit a PR to allow custom trace ID generation. Do you have any suggestion on how it should be done or maybe a way to go around the issue for now ?

[codefromthecrypt]

UUID format isn't going to work. Usually people correlate external IDs with a tag. While the opentracing api does not include an abandon() api, tools like brave do (likely zipkin-go does.. haven't checked). At any rate, late sampling trace tier or span tier is problematic as there's a chance of leaking sampling status externally before you decide. you can discuss the impact of leaking status here as it is going to affect any instrumentation receiving B3 headers, not just opentracing https://gitter.im/openzipkin/zipkin

…

On Sat, Nov 24, 2018 at 2:16 AM Eduard Mucilianu ***@***.***> wrote: Hey, I would like to be able to set the TraceID on the root span to a custom value which is deterministically determined from a random value. To give you the full picture, we use the modulo sampler to deterministically determine from a UUID request id whether it should be sampled. At some point after a series of grpc and kafka processing a db insert happens. We have a database log listener which fires kafka events for certian db inserts but we don’t have any way to forward the trace headers there. We store the same request id for idempotency reasons in the database so we want to resurect a trace if the request id matches one that should be sampled. I am happy to submit a PR to allow custom trace ID generation. Do you have any suggestion on how it should be done or maybe a way to go around the issue for now ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#120>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAD61xfN9uMYDaK-9l9uSKJe67gDNcf5ks5uyEl8gaJpZM4YxDP2> .

[eduardtm]

Thanks for the reply, I wasn’t suggesting using an UUID as the TraceID but to derive the 64 bits or 128 bits of the trace id from an UUID, don’t see what the problem is with that (they only need to be globally unique right ?). Also, I wasn’t suggesting to do late sampling, we sample at the entry point and later on, at db read prior to kafka, we can see deterministically if something was sampled and determine what trace id it had. Regarding the abandon, I suspect that if any client/span calls it on a trace id then the collector should just discard or ignore the remainging trace spans right ? even if another process still thinks it should send them. I am not that familiar with the latter two parts but they might be unrelated to my problem.

What I am suggesting is instead of enforcing the randomID() in tracer.go if there is no trace id, first check for a traceID tag or better, take in on the global Tracer a function traceIDGenerator that takes in the span startoptions or just the tags and returns a trace id. We can obviously document if there are any issues with it but I don’t see why not allow clients to supply it as long as we document that it needs to be random so that we gain flexibility.

[codefromthecrypt]

the main thing here is thinking through the problem. I think you are, though I think you are missing something significant. The collector is not stateful. If someone changes their mind later, it will not throw out spans that were referred to as sampled before someone later changed their mind to not. This causes confusion in other words.

What you are asking for is to be able to pre-ordain IDs, which is a feature opentracing has pushed back on until extremely recently, and you are asking to do that in an opentracing related repo! Ex opentracing/opentracing-java#212

You can still ask for something here, but my opinion is that intentionally creating a race on trace ID has known problems, and creating an api for a practice with known problems is not always popular. I think I've said enough on this topic. Hope it adds color. I'll leave it to others to decide.

[codefromthecrypt]

sorry I forgot some things related to this discussion:

in zipkin-go, you can change the algo. however you have no explicit parameterization which I think is what's being requested. https://github.com/openzipkin/zipkin-go/blob/049fbe2d48ed0f64ede7f076a38848c63b80682d/idgenerator/idgenerator.go

the closest lifecycle is startSpan.. there is room for flexibility, but it inputs are still needed up front
https://github.com/openzipkin/zipkin-go/blob/8a54c368aafc1d1e936c8c33a17dda2ec42e675b/tracer.go#L70

For example, in propagation like stackdriver and aws, you can extract only a trace ID from headers. that would work, but again I think you are asking to map or defer the trace ID. My thinking can be wrong, but anyway I think now (for real :) ) you have everything in my head... which is hopefully helpful!

[basvanbeek]

With the above concerns @adriancole has raised and us not wanting to diverge even more from our zipkin-go native implementation I'm going to close this one.

[eduardtm]

what zipkin-go does with the idGenerator interface is what I would need yes, the only difference is that the startSpanOptions would have to be passed in and the tracer intialiser would have to take it in similar to how it takes the sampler. The race condition from the rand generator occurs only if people would badly implement it, not in the use case I present as there won’t be a seeded rand there to use. I know that you’ve closed it, but could you guys say whether you would be open for a PR or I should go on with a fork ?

Thanks

[basvanbeek]

Yes we are not considering this feature for this library so if you must have this feature, a fork will do.

Main reason is if we accept this, it will be our responsibility to maintain and deal with the fallout as described above.

Another reason is we want to see this repo not diverge even more from our native zipkin-go library as eventually we expect to have a bridge on top of zipkin-go for opentracing consumers to take them into the Zipkin v2 world.