Create DINFRA_milestone_2.md

[rvalle]

Milestone Delivery Checklist

• The milestone-delivery-template.md has been copied and updated.
• This pull request is being made by the same account as the accepted application.
• I have disclosed any and all sources of reused code in the submitted repositories and have done my due diligence to meet its license requirements.
• In case of acceptance, an invoice must be submitted and the payment will be transferred to the Polkadot/fiat account provided in the application.
• The delivery is according to the Guidelines for Milestone Deliverables.

Link to the application pull request: w3f/Grants-Program#1622 and later updated payment destination in w3f/Grants-Program#1884

[keeganquigley]

Thanks for the delivery @rvalle starting my evaluation now.

[rvalle]

thanks @keeganquigley let me know if you need anything.

[rvalle]

@keeganquigley how is it going? did you get any troubles starting the whole cluster locally?

[keeganquigley]

Hi @rvalle apologies for the delay., and thanks for your patience since this project takes some time to set up. So far all the Docker packages are working and I can run it to simulate the deployment cycles, but am still running into some failing tests.

yarn docker:zombietest is failing:

WARNING: Image for service zombietest was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating dinfra_zombietest_1 ... error

ERROR: for dinfra_zombietest_1  Cannot start service zombietest: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/home/ubuntu/dinfra/parachain/zombienet/config.yaml" to rootfs at "/home/zombienet/config.yaml": mount /home/ubuntu/dinfra/parachain/zombienet/config.yaml:/home/zombienet/config.yaml (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

ERROR: for zombietest  Cannot start service zombietest: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/home/ubuntu/dinfra/parachain/zombienet/config.yaml" to rootfs at "/home/zombienet/config.yaml": mount /home/ubuntu/dinfra/parachain/zombienet/config.yaml:/home/zombienet/config.yaml (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: Encountered errors while bringing up the project.
error Command failed with exit code 1.

yarn build fails:

> ogate
$ yarn build
$ nest build

> acs-reactor
warning package.json: No license field
$ yarn build
warning package.json: No license field
$ yarn build:prepare
warning package.json: No license field
$ bash -c 'mkdir -p deps && cp -a ../dinfra-client-python deps && . ./venv/bin/activate && pip install -r requirements.txt'
cp: cannot stat '../dinfra-client-python': No such file or directory
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 1
Command: /home/ubuntu/.nvm/versions/node/v18.13.0/bin/node
Arguments: /home/ubuntu/.nvm/versions/node/v18.13.0/lib/node_modules/yarn/lib/cli.js run build:workspace
Directory: /home/ubuntu/dinfra/packages/acs-reactor
Output:

info Visit https://yarnpkg.com/en/docs/cli/workspaces for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I'm running:
Ubuntu 20.04 x86-64
Node v18.13.0
python 3.8 venv

[rvalle]

Hi @keeganquigley

Good to see that you got the cluster to run. Building the docker containers is more reliable, as there are no local dependencies, than building and testing locally.

Note that the local build/test steps already run on Continuous Integration so we can always check the .gitlabci as source of truth, and the output of pipelines as for what to expect.

• The first error is complaining about dinfra-client-python missing. This is the OpenAPI client code, which is automatically generated by an step before, it has a dependency on client tools docker image openapi-generator. This step was also required to build the docker containers, so this step has already worked in your environment. Lets force all the required steps, from project root do:

    - yarn install
    - yarn build:client
    - yarn build
    - yarn test

• In the second error looks like the zombietest cannot mount the test script which is down the parachain directory. Note that this directory is a git submodule, you can force its correct initialization buy doing yarn git:init from project root. Also note that when a docker mount fails to find a file it interprets it as a mount point and creates a directory that may now be in the way, check /home/ubuntu/dinfra/parachain/zombienet/config.yaml. after ensuring its not in the way try:

  - yarn git:init
  - yarn docker:build
  - yarn docker:zombietest

Your environment looks good to me. But I think you had some regression in your environment because some steps required for the docker build are now failing. I guess the root cause is that the project is very demanding in terms of disk space... perhaps you had to delete the project and then re-install it later, or perhaps you run out of disk space at some point and subsequent attempts failed.

Let me know if this helps. When in doubt use the gitlabci as guide ensuring that the parachain submodule is checked out.

[keeganquigley]

Thanks for your help @rvalle looks great now. Everything works and the zombienet tests run now. The above commands worked; I will update the eval accordingly.

zombietest_1   | │                                                           Test Results                                                            │
zombietest_1   | └───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
zombietest_1   | ┌──────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
zombietest_1   | │ 12/18/2023, 10:51:20 PM      │ ✅ alice: is up (63ms)                                                                             │
zombietest_1   | └──────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────��┘
zombietest_1   | ┌──────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
zombietest_1   | │ 12/18/2023, 10:51:20 PM      │ ✅ bob: is up (45ms)                                                                               │
zombietest_1   | └──────────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
zombietest_1   | ┌──────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
zombietest_1   | │ 12/18/2023, 10:51:20 PM      │ ✅ alice: reports node_roles is 4 (1ms)                                                            │
zombietest_1   | └──────────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
zombietest_1   | ┌──────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
zombietest_1   | │ 12/18/2023, 10:51:20 PM      │ ✅ alice: reports sub_libp2p_is_major_syncing is 0 (0ms)                                           │
zombietest_1   | └──────────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
zombietest_1   | ┌──────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
zombietest_1   | │ 12/18/2023, 10:51:20 PM      │ ✅ bob: log line matches "Imported #[0-9]+" within 30 seconds (1ms)                                │
zombietest_1   | └──────────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
zombietest_1   | ┌──────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
zombietest_1   | │ 12/18/2023, 10:51:20 PM      │ ✅ alice: log line matches "Imported #[0-9]+" within 30 seconds (1ms)                              │
zombietest_1   | └──────────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘
zombietest_1   | ┌──────────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────┐
zombietest_1   | │ 12/18/2023, 10:51:20 PM      │ ✅ collator01: is up (10ms)                                                                        │
zombietest_1   | └──────────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────┘

[keeganquigley]

Everything is working now and you can find my final eval here. Great work and congrats!

[github-actions]

🪙 Please fill out the invoice form in order to initiate the payment process. Thank you!

[github-actions]

We noticed that this is the last milestone of your project. Congratulations on completing your grant! 🎊

So, where to from here? First of all, you should join our Grants Community chat, if you haven't already, so we can stay in touch.
If you are looking for continuative support for your project, there are quite a few options. The main goal of the W3F grants program is to support research as well as early-stage technical projects. If your project still falls under one of those categories, you might want to apply for a follow-up grant. However, depending on your goals and project status, there are other support programs in our ecosystem that might be better suited as the next step. For example, projects with a Business Case/Token should look into the Substrate Builders Program or VC Funding and Common Good projects have a good chance of receiving Treasury Funding. If you are looking for guidance, the team at https://substrate.io/ecosystem/square-one/ has compiled a list of ecosystem support sources and are happy to help you navigate it.

For a more comprehensive list, see our Alternative Funding page. Let us know if you have any questions regarding the above. We are more than happy to point you to additional resources and help you determine the best course of action.
Lastly, we hope your W3F grant was a success and we want to thank you for being part of the journey!

[rvalle]

@keeganquigley thanks for your review.
Glad to see my comments were useful.
Yes, I have now joined the chat.
I will review the different options going forward.

[keeganquigley]

Yes sorry, I forgot to mention that I did try to join the private Element chat but it kept giving me an error and wouldn't let me join for some reason, sorry about that.

[rvalle]

thanks for the info @keeganquigley most probably our homeserver needs an update or something. will review it.