Skip to content

PR Feedback

Jump to bottom
Michael Hladky edited this page Sep 3, 2024 · 2 revisions

Original content here: https://github.com/code-pushup/cli/issues/478

User story

As a user, I want to get feedback on my pull requests, which tells me how the proposed changes will impact my Code PushUp reports. Specifically, I want a comment to be posts a report diff summary in Markdown. The diff should list which categories and audits have improved or regressed, and by how much.

As a paying customer, I also want to view the full diff in the portal UI. The PR comment should include a link to the portal diff page. I also want to be able to access this diff from the portal directly, by selecting source and target Git refs (branches/commits/tags).

Architecture

Free version

graph TD;
  user(["🤓 User"])
  pr(["📑 Pull Request"])
  comment(["🗨️ PR comment"])

  user -- creates --> pr
  user -- "👁️‍🗨️ views" --> comment
  pr -- triggers --> ghAction
  postComment -- creates --> comment

  subgraph ghAction ["🏗️ code-pushup/github-action"]

  collectSrc[["git checkout $GITHUB_HEAD_REF\ncode-pushup collect"]]
  collectTgt[["git checkout $GITHUB_BASE_REF\ncode-pushup collect"]]
  compare[["code-pushup compare source-report.json target-report.json"]]
  postComment[["github.issues.createComment({ ... })"]]

  srcReport("source-report.json")
  tgtReport("target-report.json")
  diffJson("reports-diff.json")
  diffMd("reports-diff.md")

  collectSrc --> srcReport
  collectTgt --> tgtReport
  srcReport & tgtReport --> compare
  compare --> diffJson & diffMd
  diffMd --> postComment
  
  end
Loading

Paid version

graph TD;
  user(["🤓 User"])
  pr(["📑 Pull Request"])
  comment(["🗨️ PR comment"])
  ghAction{{"🏗️ code-pushup/github-action"}}
  portal[("🌈 Portal")]

  user -. "💰 pays for" .-> portal
  user -- creates --> pr
  user -- "👁️‍🗨️ views" --> comment
  pr -- triggers --> ghAction
  ghAction -- "🗘 uploads to\n& downloads from" --> portal
  ghAction -- creates --> comment
  comment -- "🔗 links to" --> portal
Loading

The GitHub Action will accept optional parameters for interacting with portal. The flow will then differ from the free version in a few ways:

  • Since it's likely that the target branch (e.g. main) already has an uploaded report, we can download it from the portal API instead of running collect again. This report caching should speed up CI runs.
  • Because we want the reports to be accessible in the portal as well (primarily via link in PR comment), all reports collected by the GitHub Action are also uploaded to the portal (autorun instead of collect).
  • The Markdown comment will have an additional link added, for accessing the full diff in the portal.

In order to compare any two reports by branch/commit/etc., the portal API will implement the diffing logic as well. In this case, the diff could be more complex, e.g. to compare which issues were introduced or fixed (would be too much detail to include in MD comment).

Acceptance criteria

Free features:

  • code-pushup/cli#482
  • code-pushup/cli#559
  • code-pushup/cli#575
  • code-pushup/cli#605

Paid features:

  • portal
    • code-pushup/portal#268
    • code-pushup/portal#332
    • Report download in portal-client
    • code-pushup/portal#376
  • CLI and other
    • CLI option to download cached report instead of collect
    • #681
    • GitHub Action portal connection (download and upload reports)

Follow-up improvements

  • use report artifact as cache
Clone this wiki locally