Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve changelog generation #27744

Merged
merged 18 commits into from
Dec 30, 2020
Merged

Improve changelog generation #27744

merged 18 commits into from
Dec 30, 2020

Conversation

oandregal
Copy link
Member

@oandregal oandregal commented Dec 15, 2020

Part of #27448

This PR tries to improve the changelog generated for each release, so it needs less manual intervention. These are the sections the changelog would produce with this PR's changes:

Section Labels
Features [Type] Feature
Enhancements [Type] Enhancement
New APIs [Type] New API
Bug Fixes [Type] Bug, [Type] Regression, also tries to infer from the title when it contains variations of "fix" or "bug"
Performance [Type] Performance
Experiments [Type] Experimental, [Block] Navigation, [Block] Query, [Block] Post Comments Count, [Block] Post Comments Form, [Block] Post Comments, [Block] Post Featured Image, [Block] Post Hierarchical Terms, [Block] Post Title, [Block] Site Logo, [Feature] Full Site Editing, Global Styles, [Feature] Navigation Screen, [Feature] Widgets Screen
Documentation [Type] Documentation
Code Quality [Type] Code Quality
Tools [Package] Dependency Extraction Webpack Plugin, [Package] Jest Puppeteer aXe, [Package] E2E Tests, [Package] E2E Test Utils, [Package] Env, [Package] ESLint plugin, [Package] stylelint config, [Package] Project management automation, '[Type] Project Management, [Package] Scripts, [Type] Build Tooling, Automated Testing
Security [Type] Security
Breaking Change [Type] Breaking Change
Various PRs that haven't been classified by any other tag or title inference.

What I've done

  • I've taken a look at the release posts on the first page of make core.
  • Reviewed the sections to understand whether the PRs were there because they had the proper labels or not (hence required manual curation). I wanted to grow an intuition on where things stood.
  • My impression is that:
    • Some sections were working fine (PRs had labels related to the section most of the time): bug fixes, documentation, code quality, etc.
    • Build tooling/tools section: this could work better if we used more labels to classify them (see table above).
    • Experiments section: this could work better if we organized the PRs according to phase 2 milestones (see). See the new sections "FSE: ..." and "Screen: ..." (note that while the changelog generation creates new sections for them, they could be part of the "experiments" section in the release post easily see "Next Steps").
    • Mobile section: I realized we remove all the PRs related to mobile, so these were grouped under their own section to inspect and remove them easily.
    • Some other sections require more thought and can be iterated later: features, enhancements, new APIs, etc.

How to test

  • Run ./bin/commander.js changelog and review what the 9.7 would look like.
  • Run ./bin/commander.js changelog -m "Gutenberg 9.6" and compare with the release post for 9.6. Does it feel like a good match?
  • Run ./bin/commander.js changelog -m "Gutenberg 9.5" and compare with the release post for 9.5. Does it feel like a good match?
  • Run ./bin/commander.js changelog -m "Gutenberg 9.4" and compare with the release post for 9.4. Does it feel like a good match?

Next steps

The next step I'd like to address is the ability to group PRs within the same section. For example, group all PRs that affect the same block in the "bug fixes" section, or perhaps convert "FSE: ...", "Screen: ..." sections to groups within the "Experiments" category, etc.

I'd like to address this separately but the changes here should already be useful in reducing the amount of time involved in preparing the release post.

@oandregal oandregal self-assigned this Dec 15, 2020
@oandregal oandregal added the [Package] Project management automation /packages/project-management-automation label Dec 15, 2020
@github-actions
Copy link

github-actions bot commented Dec 15, 2020

Size Change: 0 B

Total Size: 1.28 MB

ℹ️ View Unchanged
Filename Size Change
build/a11y/index.js 1.14 kB 0 B
build/annotations/index.js 3.81 kB 0 B
build/api-fetch/index.js 3.42 kB 0 B
build/autop/index.js 2.83 kB 0 B
build/blob/index.js 665 B 0 B
build/block-directory/index.js 8.71 kB 0 B
build/block-directory/style-rtl.css 943 B 0 B
build/block-directory/style.css 942 B 0 B
build/block-editor/index.js 129 kB 0 B
build/block-editor/style-rtl.css 11.3 kB 0 B
build/block-editor/style.css 11.2 kB 0 B
build/block-library/blocks/archives/editor-rtl.css 120 B 0 B
build/block-library/blocks/archives/editor.css 119 B 0 B
build/block-library/blocks/audio/editor-rtl.css 118 B 0 B
build/block-library/blocks/audio/editor.css 118 B 0 B
build/block-library/blocks/audio/style-rtl.css 152 B 0 B
build/block-library/blocks/audio/style.css 152 B 0 B
build/block-library/blocks/block/editor-rtl.css 211 B 0 B
build/block-library/blocks/block/editor.css 211 B 0 B
build/block-library/blocks/button/editor-rtl.css 513 B 0 B
build/block-library/blocks/button/editor.css 513 B 0 B
build/block-library/blocks/button/style-rtl.css 488 B 0 B
build/block-library/blocks/button/style.css 488 B 0 B
build/block-library/blocks/buttons/editor-rtl.css 275 B 0 B
build/block-library/blocks/buttons/editor.css 275 B 0 B
build/block-library/blocks/buttons/style-rtl.css 346 B 0 B
build/block-library/blocks/buttons/style.css 346 B 0 B
build/block-library/blocks/calendar/style-rtl.css 249 B 0 B
build/block-library/blocks/calendar/style.css 249 B 0 B
build/block-library/blocks/categories/editor-rtl.css 135 B 0 B
build/block-library/blocks/categories/editor.css 135 B 0 B
build/block-library/blocks/categories/style-rtl.css 132 B 0 B
build/block-library/blocks/categories/style.css 132 B 0 B
build/block-library/blocks/code/style-rtl.css 141 B 0 B
build/block-library/blocks/code/style.css 141 B 0 B
build/block-library/blocks/columns/editor-rtl.css 239 B 0 B
build/block-library/blocks/columns/editor.css 239 B 0 B
build/block-library/blocks/columns/style-rtl.css 467 B 0 B
build/block-library/blocks/columns/style.css 466 B 0 B
build/block-library/blocks/cover/editor-rtl.css 440 B 0 B
build/block-library/blocks/cover/editor.css 438 B 0 B
build/block-library/blocks/cover/style-rtl.css 1.27 kB 0 B
build/block-library/blocks/cover/style.css 1.27 kB 0 B
build/block-library/blocks/embed/editor-rtl.css 529 B 0 B
build/block-library/blocks/embed/editor.css 529 B 0 B
build/block-library/blocks/embed/style-rtl.css 419 B 0 B
build/block-library/blocks/embed/style.css 419 B 0 B
build/block-library/blocks/file/editor-rtl.css 246 B 0 B
build/block-library/blocks/file/editor.css 245 B 0 B
build/block-library/blocks/file/style-rtl.css 288 B 0 B
build/block-library/blocks/file/style.css 289 B 0 B
build/block-library/blocks/freeform/editor-rtl.css 2.49 kB 0 B
build/block-library/blocks/freeform/editor.css 2.49 kB 0 B
build/block-library/blocks/gallery/editor-rtl.css 692 B 0 B
build/block-library/blocks/gallery/editor.css 693 B 0 B
build/block-library/blocks/gallery/style-rtl.css 1.11 kB 0 B
build/block-library/blocks/gallery/style.css 1.11 kB 0 B
build/block-library/blocks/group/editor-rtl.css 364 B 0 B
build/block-library/blocks/group/editor.css 364 B 0 B
build/block-library/blocks/group/style-rtl.css 117 B 0 B
build/block-library/blocks/group/style.css 117 B 0 B
build/block-library/blocks/heading/editor-rtl.css 174 B 0 B
build/block-library/blocks/heading/editor.css 174 B 0 B
build/block-library/blocks/heading/style-rtl.css 137 B 0 B
build/block-library/blocks/heading/style.css 137 B 0 B
build/block-library/blocks/html/editor-rtl.css 324 B 0 B
build/block-library/blocks/html/editor.css 325 B 0 B
build/block-library/blocks/image/editor-rtl.css 738 B 0 B
build/block-library/blocks/image/editor.css 737 B 0 B
build/block-library/blocks/image/style-rtl.css 510 B 0 B
build/block-library/blocks/image/style.css 511 B 0 B
build/block-library/blocks/latest-comments/editor-rtl.css 201 B 0 B
build/block-library/blocks/latest-comments/editor.css 200 B 0 B
build/block-library/blocks/latest-comments/style-rtl.css 315 B 0 B
build/block-library/blocks/latest-comments/style.css 315 B 0 B
build/block-library/blocks/latest-posts/editor-rtl.css 183 B 0 B
build/block-library/blocks/latest-posts/editor.css 183 B 0 B
build/block-library/blocks/latest-posts/style-rtl.css 568 B 0 B
build/block-library/blocks/latest-posts/style.css 567 B 0 B
build/block-library/blocks/list/editor-rtl.css 129 B 0 B
build/block-library/blocks/list/editor.css 129 B 0 B
build/block-library/blocks/list/style-rtl.css 127 B 0 B
build/block-library/blocks/list/style.css 127 B 0 B
build/block-library/blocks/media-text/editor-rtl.css 240 B 0 B
build/block-library/blocks/media-text/editor.css 240 B 0 B
build/block-library/blocks/media-text/style-rtl.css 579 B 0 B
build/block-library/blocks/media-text/style.css 577 B 0 B
build/block-library/blocks/more/editor-rtl.css 479 B 0 B
build/block-library/blocks/more/editor.css 479 B 0 B
build/block-library/blocks/navigation-link/editor-rtl.css 438 B 0 B
build/block-library/blocks/navigation-link/editor.css 440 B 0 B
build/block-library/blocks/navigation-link/style-rtl.css 747 B 0 B
build/block-library/blocks/navigation-link/style.css 745 B 0 B
build/block-library/blocks/navigation/editor-rtl.css 1.31 kB 0 B
build/block-library/blocks/navigation/editor.css 1.31 kB 0 B
build/block-library/blocks/navigation/style-rtl.css 205 B 0 B
build/block-library/blocks/navigation/style.css 205 B 0 B
build/block-library/blocks/nextpage/editor-rtl.css 440 B 0 B
build/block-library/blocks/nextpage/editor.css 440 B 0 B
build/block-library/blocks/paragraph/editor-rtl.css 161 B 0 B
build/block-library/blocks/paragraph/editor.css 161 B 0 B
build/block-library/blocks/paragraph/style-rtl.css 279 B 0 B
build/block-library/blocks/paragraph/style.css 279 B 0 B
build/block-library/blocks/post-author/editor-rtl.css 255 B 0 B
build/block-library/blocks/post-author/editor.css 255 B 0 B
build/block-library/blocks/post-author/style-rtl.css 229 B 0 B
build/block-library/blocks/post-author/style.css 230 B 0 B
build/block-library/blocks/post-comments-form/style-rtl.css 293 B 0 B
build/block-library/blocks/post-comments-form/style.css 293 B 0 B
build/block-library/blocks/post-content/editor-rtl.css 187 B 0 B
build/block-library/blocks/post-content/editor.css 187 B 0 B
build/block-library/blocks/post-excerpt/editor-rtl.css 134 B 0 B
build/block-library/blocks/post-excerpt/editor.css 134 B 0 B
build/block-library/blocks/post-featured-image/editor-rtl.css 387 B 0 B
build/block-library/blocks/post-featured-image/editor.css 386 B 0 B
build/block-library/blocks/post-featured-image/style-rtl.css 149 B 0 B
build/block-library/blocks/post-featured-image/style.css 149 B 0 B
build/block-library/blocks/preformatted/style-rtl.css 119 B 0 B
build/block-library/blocks/preformatted/style.css 119 B 0 B
build/block-library/blocks/pullquote/editor-rtl.css 231 B 0 B
build/block-library/blocks/pullquote/editor.css 231 B 0 B
build/block-library/blocks/pullquote/style-rtl.css 359 B 0 B
build/block-library/blocks/pullquote/style.css 359 B 0 B
build/block-library/blocks/query-loop/editor-rtl.css 142 B 0 B
build/block-library/blocks/query-loop/editor.css 141 B 0 B
build/block-library/blocks/query-loop/style-rtl.css 361 B 0 B
build/block-library/blocks/query-loop/style.css 363 B 0 B
build/block-library/blocks/query/editor-rtl.css 210 B 0 B
build/block-library/blocks/query/editor.css 210 B 0 B
build/block-library/blocks/quote/editor-rtl.css 121 B 0 B
build/block-library/blocks/quote/editor.css 121 B 0 B
build/block-library/blocks/quote/style-rtl.css 215 B 0 B
build/block-library/blocks/quote/style.css 214 B 0 B
build/block-library/blocks/rss/editor-rtl.css 245 B 0 B
build/block-library/blocks/rss/editor.css 246 B 0 B
build/block-library/blocks/rss/style-rtl.css 329 B 0 B
build/block-library/blocks/rss/style.css 328 B 0 B
build/block-library/blocks/search/editor-rtl.css 213 B 0 B
build/block-library/blocks/search/editor.css 213 B 0 B
build/block-library/blocks/search/style-rtl.css 384 B 0 B
build/block-library/blocks/search/style.css 386 B 0 B
build/block-library/blocks/separator/editor-rtl.css 151 B 0 B
build/block-library/blocks/separator/editor.css 151 B 0 B
build/block-library/blocks/separator/style-rtl.css 281 B 0 B
build/block-library/blocks/separator/style.css 281 B 0 B
build/block-library/blocks/shortcode/editor-rtl.css 547 B 0 B
build/block-library/blocks/shortcode/editor.css 547 B 0 B
build/block-library/blocks/site-logo/editor-rtl.css 251 B 0 B
build/block-library/blocks/site-logo/editor.css 251 B 0 B
build/block-library/blocks/site-logo/style-rtl.css 166 B 0 B
build/block-library/blocks/site-logo/style.css 166 B 0 B
build/block-library/blocks/social-link/editor-rtl.css 211 B 0 B
build/block-library/blocks/social-link/editor.css 211 B 0 B
build/block-library/blocks/social-links/editor-rtl.css 749 B 0 B
build/block-library/blocks/social-links/editor.css 749 B 0 B
build/block-library/blocks/social-links/style-rtl.css 1.37 kB 0 B
build/block-library/blocks/social-links/style.css 1.37 kB 0 B
build/block-library/blocks/spacer/editor-rtl.css 321 B 0 B
build/block-library/blocks/spacer/editor.css 321 B 0 B
build/block-library/blocks/spacer/style-rtl.css 107 B 0 B
build/block-library/blocks/spacer/style.css 107 B 0 B
build/block-library/blocks/subhead/editor-rtl.css 148 B 0 B
build/block-library/blocks/subhead/editor.css 148 B 0 B
build/block-library/blocks/subhead/style-rtl.css 134 B 0 B
build/block-library/blocks/subhead/style.css 134 B 0 B
build/block-library/blocks/table/editor-rtl.css 530 B 0 B
build/block-library/blocks/table/editor.css 530 B 0 B
build/block-library/blocks/table/style-rtl.css 433 B 0 B
build/block-library/blocks/table/style.css 433 B 0 B
build/block-library/blocks/tag-cloud/editor-rtl.css 162 B 0 B
build/block-library/blocks/tag-cloud/editor.css 162 B 0 B
build/block-library/blocks/tag-cloud/style-rtl.css 145 B 0 B
build/block-library/blocks/tag-cloud/style.css 145 B 0 B
build/block-library/blocks/template-part/editor-rtl.css 644 B 0 B
build/block-library/blocks/template-part/editor.css 645 B 0 B
build/block-library/blocks/text-columns/editor-rtl.css 146 B 0 B
build/block-library/blocks/text-columns/editor.css 146 B 0 B
build/block-library/blocks/text-columns/style-rtl.css 209 B 0 B
build/block-library/blocks/text-columns/style.css 209 B 0 B
build/block-library/blocks/verse/editor-rtl.css 118 B 0 B
build/block-library/blocks/verse/editor.css 118 B 0 B
build/block-library/blocks/verse/style-rtl.css 137 B 0 B
build/block-library/blocks/verse/style.css 137 B 0 B
build/block-library/blocks/video/editor-rtl.css 547 B 0 B
build/block-library/blocks/video/editor.css 548 B 0 B
build/block-library/blocks/video/style-rtl.css 241 B 0 B
build/block-library/blocks/video/style.css 241 B 0 B
build/block-library/common-rtl.css 940 B 0 B
build/block-library/common.css 937 B 0 B
build/block-library/editor-rtl.css 8.87 kB 0 B
build/block-library/editor.css 8.87 kB 0 B
build/block-library/index.js 149 kB 0 B
build/block-library/style-rtl.css 8.48 kB 0 B
build/block-library/style.css 8.48 kB 0 B
build/block-library/theme-rtl.css 789 B 0 B
build/block-library/theme.css 790 B 0 B
build/block-serialization-default-parser/index.js 1.88 kB 0 B
build/block-serialization-spec-parser/index.js 3.06 kB 0 B
build/blocks/index.js 48.1 kB 0 B
build/components/index.js 170 kB 0 B
build/components/style-rtl.css 15.4 kB 0 B
build/components/style.css 15.4 kB 0 B
build/compose/index.js 11 kB 0 B
build/core-data/index.js 15.2 kB 0 B
build/data-controls/index.js 829 B 0 B
build/data/index.js 8.98 kB 0 B
build/date/index.js 31.8 kB 0 B
build/deprecated/index.js 768 B 0 B
build/dom-ready/index.js 571 B 0 B
build/dom/index.js 4.95 kB 0 B
build/edit-navigation/index.js 11.1 kB 0 B
build/edit-navigation/style-rtl.css 881 B 0 B
build/edit-navigation/style.css 885 B 0 B
build/edit-post/index.js 306 kB 0 B
build/edit-post/style-rtl.css 6.47 kB 0 B
build/edit-post/style.css 6.45 kB 0 B
build/edit-site/index.js 24.5 kB 0 B
build/edit-site/style-rtl.css 3.91 kB 0 B
build/edit-site/style.css 3.91 kB 0 B
build/edit-widgets/index.js 26.3 kB 0 B
build/edit-widgets/style-rtl.css 3.1 kB 0 B
build/edit-widgets/style.css 3.1 kB 0 B
build/editor/editor-styles-rtl.css 476 B 0 B
build/editor/editor-styles.css 478 B 0 B
build/editor/index.js 42.6 kB 0 B
build/editor/style-rtl.css 3.84 kB 0 B
build/editor/style.css 3.84 kB 0 B
build/element/index.js 4.63 kB 0 B
build/escape-html/index.js 735 B 0 B
build/format-library/index.js 6.75 kB 0 B
build/format-library/style-rtl.css 547 B 0 B
build/format-library/style.css 548 B 0 B
build/hooks/index.js 2.27 kB 0 B
build/html-entities/index.js 622 B 0 B
build/i18n/index.js 3.57 kB 0 B
build/is-shallow-equal/index.js 698 B 0 B
build/keyboard-shortcuts/index.js 2.54 kB 0 B
build/keycodes/index.js 1.93 kB 0 B
build/list-reusable-blocks/index.js 3.11 kB 0 B
build/list-reusable-blocks/style-rtl.css 476 B 0 B
build/list-reusable-blocks/style.css 476 B 0 B
build/media-utils/index.js 5.31 kB 0 B
build/notices/index.js 1.86 kB 0 B
build/nux/index.js 3.42 kB 0 B
build/nux/style-rtl.css 671 B 0 B
build/nux/style.css 668 B 0 B
build/plugins/index.js 2.54 kB 0 B
build/primitives/index.js 1.43 kB 0 B
build/priority-queue/index.js 789 B 0 B
build/redux-routine/index.js 2.84 kB 0 B
build/reusable-blocks/index.js 2.92 kB 0 B
build/rich-text/index.js 13.4 kB 0 B
build/server-side-render/index.js 2.77 kB 0 B
build/shortcode/index.js 1.7 kB 0 B
build/token-list/index.js 1.27 kB 0 B
build/url/index.js 3.02 kB 0 B
build/viewport/index.js 1.86 kB 0 B
build/warning/index.js 1.14 kB 0 B
build/wordcount/index.js 1.22 kB 0 B

compressed-size-action

@youknowriad
Copy link
Contributor

Accessibility

I don't think we need an accessibility category in the changelog but we can use this to group items inside "bugs" or "enhancements".

Experimental

Sometimes something is both experimental and a bug fix (or enhancement), I believe we should give priority to "Experimental" in this case. Does the tool do that?

FSE: * sections

These should be subsections of the "Experimental" section, is this the case? Same for screen navigation and widgets

Breaking change

This doesn't mean anything for me in terms of changelog and I believe we should remove it.

Mobile

These should be removed from the changelog entirely, mobile apps have their own process for releases.

Thanks for your work here.

@oandregal
Copy link
Member Author

Accessibility, Experimental, and Breaking Change now work as you suggested. In case you want to take a look and suggest other changes, this is how the categorization mechanism works: if a PR has many labels, the one that has a lower index in the LABEL_MAPPING object will be chosen ― it fallbacks to "Various" if the PR doesn't have any label present in LABEL_MAPPING.

"FSE: ..." and "Screen: ...". With the current setup, it's straightforward to make these sub-headings within the experimental section (add one #) or make them a group (remove the ###) within the section. Would it be ok to land this as it is and iterate to add grouping in a subsequent PR?

Mobile. Now that they are grouped within a category, deleting them is straightforward. I didn't remove it in case the release manager wants to double check them.

Thoughts? Good to go and iterate?

@youknowriad
Copy link
Contributor

Would it be ok to land this as it is and iterate to add grouping in a subsequent PR?

I think in the previous release changelogs, we did group these in experiments but manually. It would be good if we can stay consistent.

Mobile

I believe the previous version of the changelog command did remove these.

--

I'll give this a try in a few minutes.

@youknowriad
Copy link
Contributor

Just gave this a test and here's what I don't like:

  • There's too many top level sections, I definitely think we should move the "FSE*" and screens under experiment.
  • The removal of "accessibility" means these items are dispatched in the other sections, what would be cool is to group them (sub lists) under the parent sections (bug, enhancement).
  • I like the renaming of "Tools" instead of "Build Tools".
  • Mobile I still think we should remove as it doesn't bring any value to plugin users (these changelogs are meant for plugin releases)

It seems there's an opportunity to do better in terms of grouping based on "blocks" and things like that but that's something for later.

@oandregal oandregal force-pushed the try/improve-changelog-match branch from 5e968ff to 27eca61 Compare December 30, 2020 13:23
@oandregal
Copy link
Member Author

@youknowriad addressed your suggestions, so as it is, it should reduce the amount of time required to re-categorize issues.

'[Block] Post Featured Image': 'Experiments',
'[Block] Post Hierarchical Terms': 'Experiments',
'[Block] Post Title': 'Experiments',
'[Block] Site Logo': 'Experiments',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just tested the PR and it works great. as a follow-up we might want to group more things automatically for instance, create a sublist for all FSE related block PRs

``

Experiments:

  • Full Site Editing Blocks:
    • entry 1
    • entry 2

There are more things that could use this "sub grouping".

@oandregal oandregal merged commit ab33643 into master Dec 30, 2020
@oandregal oandregal deleted the try/improve-changelog-match branch December 30, 2020 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Package] Project management automation /packages/project-management-automation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants