Statistics and Metrics

[qinsoon]

In GitLab by @steveblackburn on Feb 25, 2020, 12:28

I'd like to have a systematic approach to statistics and logging.

Some principles:

• A run-time option but with near-zero overhead when not activated (done)
• Overheads to be absolutely minimal when activated, IO deferred to end of program. (done)
• Assume powerful post analysis (so minimal analysis at run-time, OK to be verbose and allow analysis later)
• Per-thread logs that might be aligned afterward via clock timestamps
  • For example, each worker thread should stamp the start and end of each work packet (thread ID, clock, work-type, start/end), and start and end of any idle time. This allows detailed breakdown of where work is performed (both concurrent and gc-time).
• Key algorithmic decisions should be logged (eg full heap collection triggered due to ...)
• Possibly provide a (compile-time?) switch which allows logging in real time (with overhead) for debugging purposes.
• Should integrate well with performance counters, so for example, it should be simple to get performance counts for all worker thread stats.
• Need to output in a form that is trivially parsed and processed

[qinsoon]

Most of the items mentioned in the issue are done by:

• The eBPF tracing points and post-analysis scripts.
• The analysis feature.
• The standard Rust logging framework.
• Perf counters are supported for gc/mutator phase and for work packets.

There are some related issues:

• Stats cleanup #224
• Allow different output formats for stats #1063

We can close this issue now.

[wks]

#805 Heap dumping for debugging GC or VM binding

This is related, too. Currently, the VM binding can print logs when scanning objects, and that will help the GC developers see which objects are visited during GC. #805 further extends this capability to generated a trace that can be easily parsed for off-line analysis.