Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EPIC: Improve the contents and organization of release summary report #649

Open
5 tasks
smlambert opened this issue Apr 19, 2022 · 3 comments
Open
5 tasks
Assignees
Labels
enhancement New feature or request epic

Comments

@smlambert
Copy link
Contributor

smlambert commented Apr 19, 2022

We need to improve the release summary report to make it easier to navigate by triagers during a release.

  • Organize the report by platform.
    The release summary report does not organize all the test failures of the same platform together. Since the purpose of the report is to make it easier for triagers to review and triage and they do that by platform, we should group all results of a single platform together.

  • Put a summary table at the top of the report
    Creating a table with a column for triagers to be able to annotate as they verify and work through the triage to mark the platform as completed and ready to release would be good. This should also include the platforms that pass, not just ones with failures in them. Some examples of reformatting the report in this way can be found here: AQAvit Triage for January 2023: jdk-11.0.18+10 aqa-tests#4256 and AQAvit Triage for January 2023: jdk8u362-b09 aqa-tests#4258

  • Consider removing some information from the main report.
    Some types of tests seem to include the java -version info, others do not and it's quite cluttered when it is included in the main report. We can consider removing this or making it available as a comment at the bottom of the platform section.

  • Consider what rerun information is needed in the summary report as per EPIC: Improve the contents and organization of release summary report #649 (comment)

  • Currently, the release summary report summarizes test targets, when available, we should also aim to include details about test cases for the test suites at supply that information (openjdk, jck) as per Enhancement: Include listing of test case failures within the test target when it is available #841

@smlambert smlambert added the enhancement New feature or request label Apr 19, 2022
@smlambert smlambert changed the title Organize release summary report by platform Improve organization of release summary report Mar 24, 2023
@sophia-guo
Copy link
Contributor

With rerun-build enabled if the rerun build fails there will be rerun failure information same as others:
Screenshot 2024-01-15 at 11 38 11 AM

If the rerun build succeeds there will be no any further information.
Screenshot 2024-01-15 at 11 29 32 AM

When doing the triage it will be taken as rerun build as default, which is not correct.

@smlambert smlambert changed the title Improve organization of release summary report EPIC: Improve the contents and organization of release summary report Mar 4, 2024
@smlambert
Copy link
Contributor Author

adoptium/aqa-tests#5121 - consider adding a 'one-liner' statement in the result summary that includes the metrics shared in the grid view, Included test summary total overall test targets executed, passed, failed, skipped) and pass percentage.

Screenshot 2024-03-04 at 2 19 21 PM

@smlambert
Copy link
Contributor Author

Once a PR for adoptium/TKG#507 lands, if a test target fails and if there is a change in machine configuration as indicated in TKG console output, include mention of the change in result summary report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request epic
Projects
Status: Todo
Development

No branches or pull requests

2 participants