Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Add Ecosystem tests #521

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

CI: Add Ecosystem tests #521

wants to merge 6 commits into from

Conversation

bkchr
Copy link
Contributor

@bkchr bkchr commented Dec 26, 2024

This adds the ecosystem tests as a CI job.

  • Does not require a CHANGELOG entry

@bkchr
Copy link
Contributor Author

bkchr commented Dec 28, 2024

@xlc any idea why the ecosystem tests are failing: https://github.com/polkadot-fellows/runtimes/actions/runs/12518886514/job/34922591260?pr=521 ?

@xlc
Copy link
Contributor

xlc commented Jan 3, 2025

The error is Error handling request: 'RpcError: -32008: Response is too big: Exceeded max limit of 1048576 so it is requesting some big value from RPC and failed. There are a few similar intermittent errors in the nightly runs of the ecosystem tests and I haven't got the chance to investigate on exactly what state caused the error and why it is only an Intermittent error (rerun fixes the error most of the time).
Will spend some time on this. Meanwhile, I will suggest setup some retry logic. e.g. https://github.com/open-web3-stack/polkadot-ecosystem-tests/blob/f6b76c9201dfca0c1f95e14840ccfed2029efd02/.github/workflows/update-known-good.yml#L105

@bkchr
Copy link
Contributor Author

bkchr commented Jan 3, 2025

The error is Error handling request: 'RpcError: -32008: Response is too big: Exceeded max limit of 1048576 so it is requesting some big value from RPC and failed. There are a few similar intermittent errors in the nightly runs of the ecosystem tests and I haven't got the chance to investigate on exactly what state caused the error and why it is only an Intermittent error (rerun fixes the error most of the time).

Maybe we need a dedicated RPC server with a higher response limit.

@bkchr
Copy link
Contributor Author

bkchr commented Jan 3, 2025

What is weird is that it already fails on getting the runtime version:

getRuntimeVersion(at?: BlockHash): RuntimeVersion:: -32603: Internal RpcError: -32008: Response is too big: Exceeded max limit of 1048576

@xlc
Copy link
Contributor

xlc commented Jan 3, 2025

it is possible to set rpc endpoints via env vars and we can use github action secrets to avoid expose internal rpcs to public

@bkchr
Copy link
Contributor Author

bkchr commented Jan 3, 2025

What is also weird is that the default max response size is 15MIB. Meaning they have decreased the size quite a lot?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants