Single-threaded magic-trace bottleneck?

[shoffmeister]

I dared trying -full-execution, despite (because of ?!) the stern warnings.

htop below is trying to tell me that magic-trace consumes one CPU, and that perf apparently is only on 35%-ish busy. I have ample physical memory still available, the disk (SSD) is bored (glances).

Naively I now kick off some 3-4 additional, totally imaginary, worker threads on magic-trace to unstall perf script and everything goes brrrr, is faster by a factor of 3.

Merry be the naïve, because they don't know whether the underlying data structures can be parallelized? Sure, there is a singular timeline, but perhaps some aspects can be parallelized and then joined (eventually) onto that single timeline?

Perhaps I am hitting a design trade-off, i.e. "don't record for six seconds, always try to work with triggers?"

[shoffmeister]

Ooops, the Firefox tab just crashed on http://fedora:8080 and -serve doesn't want to respond to Chrome at this time.

:)

[shoffmeister]

And Chrome is telling me:

Oops! Your WASM trace processor ran out of memory
The in-memory representation of the trace is too big for the browser memory limits (typically 2GB per tab).

I mean, clearly, 17 GB of .fxt ... what's that these days?

BTW, it might be interesting to note the Perfetto build instructions describing that a local dev server can be conveniently launched at http://localhost:10000/ via ui/run-dev-server

[Xyene]

magic-trace is ~entirely bottlenecked on parsing textual output from perf script, because that's the interface to IPT that is available ~everywhere. Our text parsing is slower than perf's text generation.

Implementing #32 would be the right way to address this, but it requires a recent perf so wouldn't be usable by everyone.