Skip to content

Conversation

@rylev
Copy link
Member

@rylev rylev commented Jul 29, 2021

This adds more figures to the summary.

Note: I've purposefully haven't worked on finishing the visual presentation.

Screenshot 2021-07-29 at 15 05 15

The figures include:

  • all: all test results including insignificant and filtered out results
  • filtered: whatever has not been filtered out by a filter
  • scenario kinds: the various scenario kinds there are (full, incr-full, incr-unchanged, incr-patched) regardless if they have been filtered.

Also a border (red or green) is applied around any summary element that has more improvements or regressions than unchanged results. This is an attempt to draw attention to summary elements that are "significant".

Questions:

  • which summary elements are most useful?
  • Is it useful to highlight summary elements that show a "significant" number of improvements or regressions? If so, should it be how it's currently done (if there are more improvement or regression results than unchanged) or by some percentage of the total number of results? If by percentage, which?
  • Are we missing any good summary figures?
  • Should we try to summarize the summary (i.e., attempt to interpret the results for the viewer so that they do not have to)?

Fixes #935, fixes #188, fixes #953

@rylev rylev requested a review from Mark-Simulacrum July 29, 2021 13:14
@Mark-Simulacrum
Copy link
Member

Might also fix #953, not sure -- I had forgotten about this when opening that issue.

@rylev
Copy link
Member Author

rylev commented Aug 4, 2021

This fixes #953

I've removed the explicit summaries for the different scenario kinds. We can add those back if we want, but for now that information can be retrieved by explicitly filtering.

image

@rylev rylev merged commit 79aa3ce into rust-lang:master Aug 4, 2021
@rylev rylev deleted the bigger-summary branch August 4, 2021 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

summary counters off Summary line on comparison page does not update when filtering comparisons should include "summary" numbers

2 participants