Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Display performance measurement results as custom metrics #3491

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

k2tzumi
Copy link

@k2tzumi k2tzumi commented Oct 5, 2024

The author should do the following, if applicable

  • Add tests
  • Run tests
  • bun run format:fix && bun run lint:fix to format the code
  • Add TSDoc/JSDoc to document the code

What's this all about?

This will be an improvement in type checking performance monitoring added by the following Pull Request.
#3406

In conjunction with octocov, type checking performance will be displayed as Pull Request comments and CI summary as follows

Comment on lines +12 to +13
comment:
if: is_pull_request
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete this line if you do not need it to appear as a comment in every Pull Request.
You can also check the summary only.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you think we should enable or disable it? It's cool that the performance measuring result is posted to PR as a comment, but currently, it is only a TypeScript type performance. Many PRs are not related to the type definition, so it may not be important for them.

CC: @m-shaka

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

People often ignore such an automated comment, so it may not be a problem

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We believe that the comment display itself should always be displayed when measuring performance. Displaying comments makes people aware of performance.

Many PRs are not related to the type definition, so it may not be important for them.

I thought that the essential answer to this question is that we should consider not nudging the perf-measures-type-check-on-pr Job for PRs that do not affect TypeScript type performance.

It is possible to specify a path base for the execution condition of a GHA job, but is the following the only files that affect TypeScript type performance?

  • src/client/*.ts
  • perf-measures/**/*
    If the source itself that measures performance is changed, it is also necessary to run the job

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@m-shaka @k2tzumi Thank you for the comment!

We believe that the comment display itself should always be displayed when measuring performance. Displaying comments makes people aware of performance.

I see!

It is possible to specify a path base for the execution condition of a GHA job, but is the following the only files that affect TypeScript type performance?

We have to add the following files that affect application type definitions:

  • src/types.ts
  • src/hono-base.ts
  • src/request.ts
  • src/context.ts
  • src/validator/validator.ts - it can be used in the RPC-mode.

@m-shaka

What do you think of this?

Copy link

codecov bot commented Oct 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.58%. Comparing base (9986b47) to head (9d9c15a).
Report is 54 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3491   +/-   ##
=======================================
  Coverage   95.58%   95.58%           
=======================================
  Files         155      155           
  Lines        9357     9357           
  Branches     2749     2733   -16     
=======================================
  Hits         8944     8944           
  Misses        413      413           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yusukebe
Copy link
Member

Hi @k2tzumi

Sorry for the late reply. This is great! But please wait a bit for the discussion in PR #3443 to be finished.

@k2tzumi
Copy link
Author

k2tzumi commented Nov 2, 2024

@yusukebe

There is a discussion going on for #3443, is there anything we should address in our PR here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants