V2 task taking over an hour when using npm private registry. Log files are huge #1441
Comments
|
By the way, this is an Azure Artifact feed and I am using Yarn Berry, v4.5.1, I also tried 3.8.6 with no avail. |
|
@eli-gc thanks for the report.
I too wish this could be disabled as it does cause a lot of log noise, but unfortunately this is not configurable within Dependabot CLI, so it cannot be hidden currently. This was not so much of an issue in V1 because it did not use Dependabot CLI (which caused other issues). it is hard to say if the volume of logs is the performance issue or the contents of your package/lock file is. If you are able to provide a copy, I will investigate further. |
|
Let me check if I can post those. There is actually only 1 package that is hosted in our private Azure artifact feed, the rest are all public packages. If there is a way to only search the private feed for the one package and grab the others from public sources, maybe I could test that. |
If it helps, I really only need the
There certainly is, but that is something you would need to configure in your project source files, not Dependabot. Here is an example of how this could be done: In the same directory as your registry=https://registry.npmjs.org/ # by default, fetch packages from the public feed
@myorg:registry=https://pkgs.dev.azure.com/myorg/_packaging/myfeed/npm/registry/ # for anything in the "myorg" namespace, fetch packages from my private feedThen in your {
"dependencies": {
"@myorg/my-private-package": "~1.0",
"node": "~22.0"
}
}When running |
|
I'll try those changes, thanks! Here is the packages.json I am using, the only private package we use is the @3nickels/data-modules.
|
|
Here is the yarn.lock file. |
|
After looking at the logs again, it seems like it is just repeating itself. I can see GET to the same package multiple times. Don't really see any errors in the logs though so I'm not sure why it is retrying. |
|
I am getting the same yarn looping behavior.... I want to move to v2 because I also fetch nuget dependencies from ADO Artifacts..... I guess the good news is that once the 60 minute timeout of the pipeline kicked in, PRs were created. I do wonder why the yarn dependabot logic is running too much though. |
|
Oh interesting, I don't even get PRs created. |
I wager your problem is on the It might need to be: [...]
token: ${{ORGANIZATION}}:${{NPM_TOKEN}}Looks like your organization is But note this doesn't solve the looping behavior which the original issue is about. |
|
@epicstar the README says to use # Artifactory private feed using PAT
artifactory:
type: nuget-feed
url: https://artifactory.com/api/nuget/v3/myfeed
token: PAT:${{ MY_DEPENDABOT_ARTIFACTORY_PAT }} |
|
Yes, that works for nuget feeds, but the README isn't explicit about how to authenticate on Azure NPM feeds since the authentication isn't merely PAT Token based on the NPM feeds. And for more info: yarnpkg/berry#316 (comment) I understand that these are instructions on yarn berry's .yarnrc.yml and not dependabot. I could be wrong. |
|
I actually do have that in my .yarnrc.yml, but maybe I need to put it in my dependabot config as well. I'll give that a try. Thanks for the suggestion. |
|
Tried it but didn't seem to make a difference. Still just takes forever and times out after an hour. |
|
Yup, mine ran today and I got the same issue... This time no PRs made. |
|
For people interested in my log file... |
|
Just an update on this, I haven't managed to reproduce the issue yet, but do want to mention that there are known issues reported in dependabot-core around NPM updates stalling and causing timeouts that you might want to contribute to if you believe they are related. See:
This issue is still on my radar to look in to, but there is no obvious or straight forward fix, so progress is slow. |
|
Seems like it is indeed a dependabot/dependabot-core issue and maybe not your side., From the comments, it's related to mixing dependabot groups (which I want to use) and a monorepo. THanks for looking into the issue @rhyskoedijk. |
|
I was wondering if it is possible to extend the timeout to over 60 minutes? I tried in my pipeline yaml, and dependabot is still timing out at 60 minutes. steps:
- task: dependabot@2
timeoutInMinutes: 180
inputs:
abandonUnwantedPullRequests: false
setAutoComplete: true
githubConnection: blah
azureDevopsAccessToken: $(DEVOPS_PAT)
env:
AZURE_ARTIFACTS_PAT: $(DEVOPS_PAT)There's nothing in the documentation that suggests that can change the timeout on the |
|
@epicstar Dependabot does not have a timeout, the pipeline does. You need to configure the timeout on the pipeline, there are some limitations if you are not using paid agents. jobs:
- job: dependabot
displayName: Dependabot
timeoutInMinutes: 180 # max 3 hours
steps:
# ...snip... |
|
Ok thank you that explains why my timeout isn't working. I should research the pipeline args and caveats more. I apologize. |
|
I tested with 1.39 and it still seems to be printing the whole yarn.lock. Seems to be less lines but the file size of the log is about the same as before. I am still getting timeout after an hour. |
|
@eli-gc unfortunately I'm not sure if the timeout issue can be resolved in the scope of this project; My understanding is that this is an issue in dependabot-core. See:
If you believe your issue is not the same as these and you are able to upload or send me a copy of your task logs, I will investigate further to see if there are any issues in the scope of this project. |
|
From #1518, I have supplied a minimum repo that reproduces this exact issue on Azure DevOps by running over 60 minutes. The same exact code in Github runs in less than 16 minutes. |
|
@epicstar thanks, I will investigate further and see what is different between the two environments |
|
I'll get you logs ASAP. It's been 2 hours and the dependabot job is still running ☠️ |
|
Same logs posted from #1518 are here. Github Dependabot seems to only hone in on the dependencies declared in the monorepo, while this version seems to hone in on the dependencies from the whole yarn.lock multiple times. In addition, Dependabot in Azure DevOps ran for 3 hours before timing out (I probably should've put it on 6 hours in retrospect). |
|
I removed the private registry and it is still timing out so seems to be unrelated to that. Works fine on github. |
|
Tried it with the latest version. 2.40.1110 without private registry. It seems like it gets stuck when trying to submit a PR. I truncated the logs below and attached the full version. |
Describe the bug
I recently migrated to the V2 task and it is timing out after my timeout limit (60 minutes). Using the V1 task only takes 3 minutes. It seems for every package, it prints out the entire contents of the yarn lock file. I'm not sure if the performance issues are related to that but the log file is huge, 134.2 MB.
Categorization
I don't think this is a permission issue because it looks to be authenticating.
Configurations
Pipeline:
dependabot.yml
Logs and screenshots
Logs in the beginning of the run. I had to truncate the {"data"} section because it exceeded the character limit. I think this part is potentially causing my poor performance. It seems to be printing all of the yarn.lock or something.
The text was updated successfully, but these errors were encountered: