Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add yarn berry support #679

Draft
wants to merge 23 commits into
base: develop
Choose a base branch
from

Conversation

favna
Copy link

@favna favna commented Nov 29, 2023

Thanks for contributing to the buildpack. To speed up the process of reviewing your pull request please provide us with:

  • A short explanation of the proposed change:

I am making a first attempt at adding Yarn berry support to the CF NodeJS buildpack. This isn't yet finished because unit tests still need to be updated/added. I'm creating a Draft PR because I'm not all too familiar with Golang (admittedly big chunks of this PR were written with Copilot assistance) and I would like some early feedback. Some points I am particularly unsure about:

  1. How to generate code with mockgen, exactly. I am developing on a MacBook and I have installed Go through Brew but I cannot figure out how to now install and use Mockgen. When I click the context menu buttons in VSCode it tells me mockgen is not on my PATH.. I managed to figure this out by using JetBrains GoLand IDE which had commands to run mockgen. Bliss JetBrains 🙏!
  2. For the functions getYarnVersion and isYarnLocalCacheEnabled how to process the stdout of the commands and assign them to variables so they can be used for the return of the function. The current code was CoPilot generated but I have my strong suspicions it is incorrect because I am not using y.Command.Run like the Build function (now doBuildClassic) is. I Managed to figure this out through writing unit tests and some println's. Again bliss JetBrains GoLand for making it easy to run specific suits repeatedly.

This PR is sort of related to #229 but also in general it would be great if the CF NodeJS buildpack supported Yarn Berry because Yarn v1 is well and thoroughly deprecated by now and Yarn has recently released Yarn v4. There are many, many advantages to upgrading and there is practically no reason to still stick with Yarn v1.

Lastly I also completely separated checking s.IsVendored from the switch case below it because previously when a project was vendored it would always run npm, regardless of package manager used. That completely broke projects that were using Yarn, at least Yarn Berry. I had temporarily monkey-patched the Go code in a fork that we were using to alleviate this.

  • An explanation of the use cases your change solves

Support for Yarn Berry (v2/v3/v4/future). This is something that the buildpack straight-up didn't support before and it should.

  • I have viewed signed and have submitted the Contributor License Agreement

  • I have made this pull request to the develop branch

  • I have added an integration test

Copy link

linux-foundation-easycla bot commented Nov 29, 2023

CLA Signed

The committers listed above are authorized under a signed CLA.

@favna favna force-pushed the feat/add-yarn-berry-support branch from 67172e2 to 300405e Compare November 29, 2023 16:21
@brayanhenao
Copy link
Member

@thitch97 Could you take a look on this PR?

Also, @favna thanks for your contribution, check the Unit/integration tests that are failing.

@favna
Copy link
Author

favna commented Dec 4, 2023

@brayanhenao I would love to, but I'm not sure how I can without mockgen working and as mentioned I'm not sure how to set it up on my system. Any guidance would be much appreciated. I tried joining your Slack with the link in the readme, but it just gives me an error saying I don't have an account on the slack.

@favna
Copy link
Author

favna commented Dec 5, 2023

@brayanhenao I have fixed the existing and added new unit tests. I will now start looking at integration tests.

Edit 2: Integration tests are running locally through Docker but I also seem to be hitting more platform setup issues but I see that GH has started now as well so hopefully they pass in the action runner.

@favna favna force-pushed the feat/add-yarn-berry-support branch 2 times, most recently from 55b53cd to 86be654 Compare December 5, 2023 16:45
@brayanhenao brayanhenao marked this pull request as ready for review December 5, 2023 17:03
@brayanhenao brayanhenao self-assigned this Dec 5, 2023
@brayanhenao brayanhenao marked this pull request as draft December 5, 2023 17:03
@favna
Copy link
Author

favna commented Dec 5, 2023

Very weird that unit tests failed just now in actions, they worked on my machine. I changed the way global cache is disabled now for Yarn and they still run fine locally:

ishare-1701797017

@brayanhenao
Copy link
Member

Hi @favna! Running both the unit and integration suites should not give you problems. All you need is Docker or a Cloud Foundry setup and Golang on the machine where you'll run the tests.

If you're hitting some errors in the configuration, paste the logs here so I can figure out what is happening.

For the integration tests, no worries if you don't have a cf environment. You can go the Docker route instead. Use the command:

./scripts/integration.sh --platform docker --github-token <your-github-token>

@favna
Copy link
Author

favna commented Dec 5, 2023

@brayanhenao Added some logging for unit tests in an attempt to find out why they are failing on GitHub Actions but not locally. As for integration tests, I ran them with docker platform and used tee to dump both STDOUT and STDERR to a file, this is the file: integration.log

Copy link
Member

@TisVictress TisVictress left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@favna This looks like a great start. However, I'm not sure why the decision was made to add support for berry in the paketo buildpack only and not in CF. Before you continue working on this, let me ask around and get back to you.

@favna
Copy link
Author

favna commented Dec 6, 2023

I think the commit above finally fixes unit tests in GitHub Actions. Thanks to the run before that I was able to find out there were ANSI color escape codes so a quick prompt to Copilot later I added TERM=dumb to the env. If unit tests finally pass after that one I have 1 more commit pending to clean up all the logging and the manual writing to .yarnrc.yml in tests that I temporarily added.

@favna
Copy link
Author

favna commented Dec 11, 2023

Any updates available @TisVictress ?

@favna favna force-pushed the feat/add-yarn-berry-support branch 2 times, most recently from 5e54f0e to 7ce2586 Compare December 12, 2023 09:25
@favna
Copy link
Author

favna commented Dec 12, 2023

I temporarily created a branch on my fork that I could use to slam GitHub Actions and find a solution for unit tests. Bunch of builds later I have finally resolved the dastardly issue so I pushed the final clean changes to this branch. You can see that unit tests and integration tests both passed here: https://github.com/favna/nodejs-buildpack/actions/runs/7179209025?pr=1 so once approved for this PR I assume they will also pass here.

favna added a commit to favna/nodejs-buildpack that referenced this pull request Dec 12, 2023
Working on my other PR (cloudfoundry#679) I noticed the `actions/checkout` and `actions/setup-go` this repository uses are using older version. This PR updates them.
@favna favna mentioned this pull request Dec 12, 2023
3 tasks
@favna favna force-pushed the feat/add-yarn-berry-support branch from 7ce2586 to 33d5f9d Compare January 2, 2024 10:53
@favna
Copy link
Author

favna commented Jan 2, 2024

New year, new changes. Best wishes for 2024. @brayanhenao, @TisVictress can you give me any updates on the review process of this PR?

favna added a commit to favna/nodejs-buildpack that referenced this pull request Jan 2, 2024
Working on my other PR (cloudfoundry#679) I noticed the `actions/checkout` and `actions/setup-go` this repository uses are using older version. This PR updates them.
@brayanhenao
Copy link
Member

Hi @favna,

We appreciate your effort in contributing to the project. Currently, supporting Yarn Berry isn't a top priority in our roadmap, and we don't have the capacity to thoroughly review the pull request at the moment.

Given the significant impact such changes can have on users of the Buildpack, we want to make sure any adjustments are carefully considered before implementation.

When we have the time or this change is prioritized we will be updating this PR with the respective comments.

Thank you for your understanding.

@brayanhenao brayanhenao removed their assignment Jan 11, 2024
@favna favna force-pushed the feat/add-yarn-berry-support branch from 33d5f9d to 1f297e4 Compare January 30, 2024 08:07
favna and others added 21 commits May 3, 2024 11:05
Signed-off-by: Jeroen Claassens <[email protected]>
Signed-off-by: Jeroen Claassens <[email protected]>
Signed-off-by: Jeroen Claassens <[email protected]>
Signed-off-by: Jeroen Claassens <[email protected]>
…nsure it won't have ANSI color escape codes

Signed-off-by: Jeroen Claassens <[email protected]>
Yarn in PnP mode strictly speaking does not use node_modules so this way the log line doesn't "lie" about what it may have found
@favna favna force-pushed the feat/add-yarn-berry-support branch from 5001e50 to 2d4c4f4 Compare May 3, 2024 09:05
@bvincent
Copy link

bvincent commented Jul 2, 2024

Thanks @favna, do you have any plan to release this feature @TisVictress & @brayanhenao ?

@Gerg Gerg requested review from robdimsdale, a team, arjun024, ForestEckhardt and Gerg and removed request for thitch97 October 18, 2024 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants