VizTracer as a thread local?

[seagrine]

I was playing around with VizTracer (awesome lib, btw!) and saw the error:

Warning! Overwrite tracer! You should not have two VizTracer recording at the same time!

I was trying to have one VizTracer recording per thread but it appears there can only be one VizTracer recording per process.

Is that true? It would be really nice to be able to scope the VizTracer to a single thread since several popular libraries use threads to concurrently handle separate tasks, especially where there's a lot of IO.

[gaogaotiantian]

Why per thread when VizTracer already supports multi-thread tracing? That's the whole point - you can have all threads on the same report. If you are only interested in one of them, that's fine you can focus on that one, but you'll also know the exact timeline for others to infer the interaction between the threads.

[seagrine]

Only because I want to save the traces for each thread/task separately.

[gaogaotiantian]

It's very difficult for me to imagine a valid use case for this, but to answer your question, not you can't do that directly with VizTracer. However, the information is there so you can always filter the data after trace is collected, it's just json.

[seagrine]

Well, I'll illustrate the use case, maybe you'll be able to help me figure out a good approach.
I have a thread pool that is acting as a task processor for a queue of tasks (actually there are many such thread pools across many processes, but we can pretend there is just one).
Each task has a set of parameters, such as "who triggered it" or "what data objects it manipulates"
I want to enable tracing for just the tasks that are triggered by X, and save them to disk somewhere, but not for any of the other tasks.

[gaogaotiantian]

Okay that makes sense. However because of the Python mechanics, it's impossible (very difficult) to enable tracing on certain threads but not others. Especially in the case where it needs to be dynamically determined. I would still suggest the after mortem method - filter the json.

Record all the thread ids that you want to log, but enable viztracer for the whole time. Then in your collected trace file, filter the "traceEvents" field based on the tid. You'll have what you need - it should be a very simple script if you are familiar with json.

[seagrine]

That makes sense. Unfortunately that solution would mean that we need to accept the performance hit and risks across all tasks. We only want to trace a tiny fraction of the tasks.