Include runtime in profiles?

[casperisfine]

I haven't dug too much on this issue, but one complaint I got from people used to StackProf/Speedscope was the lack of durations in flamegraph and such.

The overall profile duration is displayed, but individual sample don't have the information, so you kinda have to compute overall_time * percent to have an idea if what you are looking at is 10m or 100ms.

I know Vernier is not really collecting this information, and probably shouldn't as it would be costly, but I think it could be estimated by simply doing this division?

It seems that it's what the Firefox profiler is doing, e.g. this public profile: https://share.firefox.dev/341Msw7

But not with profile done by vernier:

cc @dalehamel

[dalehamel]

Dug into this a bit, it seems to be generated by this: https://github.com/firefox-devtools/profiler/blob/4ba5982a5dd1a5e46d78b3155a2ef337e14bcc13/src/components/tooltip/CallNode.js#L495, per https://github.com/firefox-devtools/profiler/blob/4ba5982a5dd1a5e46d78b3155a2ef337e14bcc13/src/components/tooltip/CallNode.js#L125-L127

Confirmed this by forcing the "hover" state in dev tools:

The difference seems to be this block of code:

https://github.com/firefox-devtools/profiler/blob/4ba5982a5dd1a5e46d78b3155a2ef337e14bcc13/src/components/flame-graph/Canvas.js#L397-L405

And that seems to originate here https://github.com/firefox-devtools/profiler/blob/main/src/profile-logic/call-tree.js#L42

I think that something is up when computing the call tree timings, which is resulting in this value being null. I haven't been able to determine what is missing yet, however.

[dalehamel]

This is also interesting... https://github.com/firefox-devtools/profiler/blob/main/src/profile-logic/call-tree.js#L341-L379

[dalehamel]

Ok I think I've figured it out... bizarrely, the difference between the two profiles seems to be that the threads in the javascript example you gave @casperisfine are missing the "weight" field, it is null.

By nulling out the weight fields in a vernier sample, it now calculates the totals 🤨

vernier_no_weights.json.gz

Here is the original:

orig.json.gz

Obviously now the "samples" count is very wrong, i guess this has something to do with how vernier is storing the sample weights in the "weights" field (which seems intuitive...)

[joshuay03]

Ok I think I've figured it out... bizarrely, the difference between the two profiles seems to be that the threads in the javascript example you gave @casperisfine are missing the "weight" field, it is null.

See https://github.com/tenderlove/profiler/blob/979dca8b83a9bde8afb1636a847d4eda6882ee28/src/profile-logic/call-tree.js#L673-L699 which is used by the FlameGraph component.

[jhawthorn]

We could remove the "weights". Currently that allows us to do "RLE" on consecutive samples by incrementing the weight instead of making a larger array. I think that helps us keep the profile sizes down quite a bit, especially as most threads in most profiles will be doing nothing most of the time, so it would be great if we had a solution which had the best of both worlds.