-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
401 Unauthorized
when looking for entrypoint/cmd of an image hosted on a private AWS ECR with v3.6.0
#13947
Comments
401 Unauthorized
when looking for entrypoint/cmd of an image hosted on AWS ECR with v3.6.0401 Unauthorized
when looking for entrypoint/cmd of an image hosted on a private AWS ECR with v3.6.0
@tico24, could this be related to the helm chart changes rather than the controller? I'm not aware of controller differences that might cause this. |
The controller yaml doesn't change, nor do any controller permissions. |
@tooptoop4 I tried with a random secret but it didn't change anything. |
Is this only occurring with Amazon ECR, or is anyone seeing it with a different container registry? |
@Sirz3chs you said:
Which mechanism was your controller getting permissions to access ECR before you upgraded to 3.6? |
I believe this should be managed magically by the use of I'll put up a speculative fix. |
I have put up something that might work in #14008 - if anyone needs help testing it reach out to me here or in Slack. It won't get merged without confirmation that it helps. |
@Joibel I didn't grant any specific permissions to the controller. Only the EKS nodes have the necessary rights to pull ECR images. |
google/go-containerregistry#1950 / tektoncd/pipeline#7698 have details on the cause |
Pre-requisites
:latest
image tag (i.e.quay.io/argoproj/workflow-controller:latest
) and can confirm the issue still exists on:latest
. If not, I have explained why, in detail, in my description below.What happened? What did you expect to happen?
I'm upgrading from
v3.5.12
tov3.6.0
using the official Helm chart. After the upgrade, workflows with images hosted on ECR fail to start.I always have this error:
I precise that my image does contains an
ENTRYPOINT
. I tried to give IAM permissions to read the ECR repository to the workflow-controller with IRSA but it doesn't work.Version(s)
v3.6.0
Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
Any simple workflow whose image is hosted on a private ECR repository.
Logs from the workflow controller
Logs from in your workflow's wait container
The text was updated successfully, but these errors were encountered: