Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[REGRESSION]: Docker@2 fails - "docker inspect" requires at least 1 argument #20506

Closed
4 of 7 tasks
YodaDaCoda opened this issue Oct 3, 2024 · 7 comments · May be fixed by microsoft/azure-pipelines-tasks-common-packages#400
Closed
4 of 7 tasks

[REGRESSION]: Docker@2 fails - "docker inspect" requires at least 1 argument #20506

YodaDaCoda opened this issue Oct 3, 2024 · 7 comments · May be fixed by microsoft/azure-pipelines-tasks-common-packages#400
Labels
Area: Release Area:RM RM task team bug regression This used to work, but a change in the service/tasks broke it. triage

Comments

@YodaDaCoda
Copy link

YodaDaCoda commented Oct 3, 2024 •
edited
Loading

New issue checklist

Task name

Docker@2

Breaking task version

2.246.0

Last working task version

2.243.1

Regression Description

In v2.243.1 the docker inspect command fails, but by this point the image is already pushed to the container registry and the failing command still lets the build succeed. As of 2.246.0 the build fails (but the image is still pushed).

Environment type (Please select at least one enviroment where you face this issue)

  • Self-Hosted
  • Microsoft Hosted
  • VMSS Pool
  • Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

ubuntu-latest

Relevant log output

2024-10-03T06:03:05.9529821Z [command]/usr/bin/docker history --format createdAt:{{.CreatedAt}}; layerSize:{{.Size}}; createdBy:{{.CreatedBy}}; layerId:{{.ID}} --no-trunc ***/rtba-sf:5611
2024-10-03T06:03:05.9592865Z ##[debug]which '/usr/bin/docker'
2024-10-03T06:03:05.9627085Z ##[debug]found: '/usr/bin/docker'
2024-10-03T06:03:05.9629246Z ##[debug]/usr/bin/docker arg: push
2024-10-03T06:03:05.9635753Z ##[debug]/usr/bin/docker arg: ***/rtba-sf:38264fe495ddcc9a5525d19b1a3b4ef60c005c66
2024-10-03T06:03:05.9636888Z ##[debug]DOCKER_HOST=undefined
2024-10-03T06:03:05.9637633Z ##[debug]exec tool: /usr/bin/docker
2024-10-03T06:03:05.9638374Z ##[debug]arguments:
2024-10-03T06:03:05.9639039Z ##[debug]   push
2024-10-03T06:03:05.9639834Z ##[debug]   ***/rtba-sf:38264fe495ddcc9a5525d19b1a3b4ef60c005c66
2024-10-03T06:03:05.9640837Z [command]/usr/bin/docker push ***/rtba-sf:38264fe495ddcc9a5525d19b1a3b4ef60c005c66
2024-10-03T06:03:05.9826806Z createdAt:2024-10-03T06:01:19Z; layerSize:118MB; createdBy:RUN /bin/sh -c     apt-get update
2024-10-03T06:03:05.9829017Z     apt-get install --yes --no-install-recommends ca-certificates jq git python3 python3-junitparser python3-dotenv
2024-10-03T06:03:05.9830106Z     apt-get clean
2024-10-03T06:03:05.9831131Z     rm -rf /var/lib/apt/lists/*
2024-10-03T06:03:05.9832086Z     rm -rf /var/cache/apt/archives/*
2024-10-03T06:03:05.9832912Z  # buildkit; layerId:sha256:cdff2dec137d8d3e46ca2ffffc3cfa6011eede4c277ae1895b30b2a6ea95673d
2024-10-03T06:03:05.9833977Z createdAt:2024-10-03T06:01:19Z; layerSize:0B; createdBy:ENV PATH=/app/node_modules/.bin:/app/volta/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin; layerId:<missing>
2024-10-03T06:03:05.9842462Z createdAt:2024-10-03T06:01:19Z; layerSize:0B; createdBy:ENV VOLTA_HOME=/app/volta; layerId:<missing>
2024-10-03T06:03:05.9843018Z createdAt:2024-10-03T06:01:19Z; layerSize:0B; createdBy:ENV BASH_ENV=~/.bashrc; layerId:<missing>
2024-10-03T06:03:05.9843564Z createdAt:2024-10-03T06:00:59Z; layerSize:1.11GB; createdBy:COPY /app /app # buildkit; layerId:<missing>
2024-10-03T06:03:05.9844090Z createdAt:2023-11-28T09:02:19Z; layerSize:0B; createdBy:/bin/sh -c #(nop)  CMD ["/bin/bash"]; layerId:<missing>
2024-10-03T06:03:05.9844769Z createdAt:2023-11-28T09:02:19Z; layerSize:70.3MB; createdBy:/bin/sh -c #(nop) ADD file:9627edfd854222fb9117755e0e89c54a01ba3477dffb8137693b12c586d970b8 in / ; layerId:<missing>
2024-10-03T06:03:05.9845409Z createdAt:2023-11-28T09:02:18Z; layerSize:0B; createdBy:/bin/sh -c #(nop)  LABEL org.opencontainers.image.version=23.04; layerId:<missing>
2024-10-03T06:03:05.9846020Z createdAt:2023-11-28T09:02:18Z; layerSize:0B; createdBy:/bin/sh -c #(nop)  LABEL org.opencontainers.image.ref.name=ubuntu; layerId:<missing>
2024-10-03T06:03:05.9846586Z createdAt:2023-11-28T09:02:18Z; layerSize:0B; createdBy:/bin/sh -c #(nop)  ARG LAUNCHPAD_BUILD_ARCH; layerId:<missing>
2024-10-03T06:03:05.9847121Z createdAt:2023-11-28T09:02:17Z; layerSize:0B; createdBy:/bin/sh -c #(nop)  ARG RELEASE; layerId:<missing>
2024-10-03T06:03:05.9863846Z ##[debug]Exit code 0 received from tool '/usr/bin/docker'
2024-10-03T06:03:05.9868144Z ##[debug]STDIO streams have closed for tool '/usr/bin/docker'
2024-10-03T06:03:05.9900394Z ##[debug]which '/usr/bin/docker'
2024-10-03T06:03:05.9906076Z ##[debug]found: '/usr/bin/docker'
2024-10-03T06:03:05.9906591Z ##[debug]/usr/bin/docker arg: inspect
2024-10-03T06:03:05.9907073Z ##[debug]/usr/bin/docker arg: ["-f","{{.RootFS.Layers}}"]
2024-10-03T06:03:05.9907544Z ##[debug]DOCKER_HOST=undefined
2024-10-03T06:03:05.9908011Z ##[debug]exec tool: /usr/bin/docker
2024-10-03T06:03:05.9908455Z ##[debug]arguments:
2024-10-03T06:03:05.9908882Z ##[debug]   inspect
2024-10-03T06:03:05.9909798Z ##[debug]   -f
2024-10-03T06:03:05.9910271Z ##[debug]   {{.RootFS.Layers}}
2024-10-03T06:03:05.9910734Z [command]/usr/bin/docker inspect -f {{.RootFS.Layers}}
2024-10-03T06:03:06.0052082Z The push refers to repository [***/rtba-sf]
2024-10-03T06:03:06.0167048Z "docker inspect" requires at least 1 argument.
2024-10-03T06:03:06.0175952Z See 'docker inspect --help'.
2024-10-03T06:03:06.0176080Z 
2024-10-03T06:03:06.0176374Z Usage:  docker inspect [OPTIONS] NAME|ID [NAME|ID...]
2024-10-03T06:03:06.0176505Z 
2024-10-03T06:03:06.0177205Z Return low-level information on Docker objects
2024-10-03T06:03:06.0193151Z ##[debug]Exit code 1 received from tool '/usr/bin/docker'
2024-10-03T06:03:06.0195692Z ##[debug]STDIO streams have closed for tool '/usr/bin/docker'
2024-10-03T06:03:06.0216149Z ##[error]"docker inspect" requires at least 1 argument.
2024-10-03T06:03:06.0227447Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=f5b2684b-f64c-4b7f-b478-eaa957d0bc5f;]"docker inspect" requires at least 1 argument.
2024-10-03T06:03:06.0229315Z ##[error]See 'docker inspect --help'.
2024-10-03T06:03:06.0230625Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=f5b2684b-f64c-4b7f-b478-eaa957d0bc5f;]See 'docker inspect --help'.
2024-10-03T06:03:06.0232204Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=f5b2684b-f64c-4b7f-b478-eaa957d0bc5f;]
2024-10-03T06:03:06.0233120Z ##[error]Usage:  docker inspect [OPTIONS] NAME|ID [NAME|ID...]
2024-10-03T06:03:06.0234176Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=f5b2684b-f64c-4b7f-b478-eaa957d0bc5f;]Usage:  docker inspect [OPTIONS] NAME|ID [NAME|ID...]
2024-10-03T06:03:06.0235373Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=f5b2684b-f64c-4b7f-b478-eaa957d0bc5f;]
2024-10-03T06:03:06.0236589Z ##[error]Return low-level information on Docker objects
2024-10-03T06:03:06.0237136Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=f5b2684b-f64c-4b7f-b478-eaa957d0bc5f;]Return low-level information on Docker objects
2024-10-03T06:03:06.0237762Z ##[debug]task result: Failed
2024-10-03T06:03:06.0238538Z ##[error]Unhandled: The process '/usr/bin/docker' failed with exit code 1
2024-10-03T06:03:06.0239126Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=f5b2684b-f64c-4b7f-b478-eaa957d0bc5f;]Unhandled: The process '/usr/bin/docker' failed with exit code 1
2024-10-03T06:03:06.0249183Z ##[debug]Processed: ##vso[task.complete result=Failed;]Unhandled: The process '/usr/bin/docker' failed with exit code 1
2024-10-03T06:03:06.0251757Z ##[error]Error: The process '/usr/bin/docker' failed with exit code 1
    at ExecState._setResult (/home/vsts/work/_tasks/Docker_e28912f1-0114-4464-802a-a3a35437fd16/2.246.0/node_modules/azure-pipelines-task-lib/toolrunner.js:1249:25)
    at ExecState.CheckComplete (/home/vsts/work/_tasks/Docker_e28912f1-0114-4464-802a-a3a35437fd16/2.246.0/node_modules/azure-pipelines-task-lib/toolrunner.js:1232:18)
    at ChildProcess.<anonymous> (/home/vsts/work/_tasks/Docker_e28912f1-0114-4464-802a-a3a35437fd16/2.246.0/node_modules/azure-pipelines-task-lib/toolrunner.js:1160:19)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)
2024-10-03T06:03:06.0253714Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=f5b2684b-f64c-4b7f-b478-eaa957d0bc5f;]Error: The process '/usr/bin/docker' failed with exit code 1
    at ExecState._setResult (/home/vsts/work/_tasks/Docker_e28912f1-0114-4464-802a-a3a35437fd16/2.246.0/node_modules/azure-pipelines-task-lib/toolrunner.js:1249:25)
    at ExecState.CheckComplete (/home/vsts/work/_tasks/Docker_e28912f1-0114-4464-802a-a3a35437fd16/2.246.0/node_modules/azure-pipelines-task-lib/toolrunner.js:1232:18)
    at ChildProcess.<anonymous> (/home/vsts/work/_tasks/Docker_e28912f1-0114-4464-802a-a3a35437fd16/2.246.0/node_modules/azure-pipelines-task-lib/toolrunner.js:1160:19)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)

Full task logs with system.debug enabled

Unable to copy-paste full logs as it makes the body of the issue too long. See attachments.

UNSUCCESSFUL RUN

docker-build-and-push-failed.log

SUCCESSFUL RUN

docker-build-and-push-successful.log

Repro steps

azure pipelines step

          - task: [email protected]
            displayName: Build an image
            inputs:
              repository: $(imageName)
              command: buildAndPush
              Dockerfile: .ci/docker/Dockerfile
              buildContext: "."
              tags: |
                $(Build.BuildId)
                $(Build.SourceVersion)
              containerRegistry: $(registryConnection)

Dockerfile

# builder image, used to install volta and node dependencies
FROM ubuntu:23.04 as builder
ENV BASH_ENV ~/.bashrc
ENV VOLTA_HOME /app/volta
ENV PATH $VOLTA_HOME/bin:$PATH
COPY package.json package-lock.json /app/
COPY .ci/npm-install.sh .ci/link-plugins.sh /app/.ci/
RUN <<EOF
    set -e # exit on error
    set -x # print each command

    # apt install dependancies & cleanup
    apt-get update
    apt-get install --yes --no-install-recommends curl ca-certificates
    apt-get clean
    rm -rf /var/lib/apt/lists/*
    rm -rf /var/cache/apt/archives/*

    # install & setup volta
    curl https://get.volta.sh | bash
    volta setup
    # allow volta to download/update node version in PRs (which run in limited user context)
    chmod 777 /app/volta/tmp /app/volta/tools/image/node /app/volta/tools/inventory /app/volta/tools/inventory/node

    # link sfdx config dir to /app
    mkdir -p /app/sf
    mkdir -p ~/.local/share
    ln -s /app/sf ~/.local/share/sf

    # install node dependencies
    # this also triggers volta to download the pinned node version
    cd /app
    PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true /app/.ci/npm-install.sh

    # clean up
    npm cache clean --force
    rm -rf /app/.ci
EOF

# final image, contains volta and node_modules from builder, adds git, python3 and junitparser
FROM ubuntu:23.04
COPY --from=builder /app /app
ENV BASH_ENV ~/.bashrc
ENV VOLTA_HOME /app/volta
ENV PATH /app/node_modules/.bin:$VOLTA_HOME/bin:$PATH
RUN <<EOF
    apt-get update
    apt-get install --yes --no-install-recommends ca-certificates jq git python3 python3-junitparser python3-dotenv
    apt-get clean
    rm -rf /var/lib/apt/lists/*
    rm -rf /var/cache/apt/archives/*
EOF
@YodaDaCoda YodaDaCoda added bug regression This used to work, but a change in the service/tasks broke it. labels Oct 3, 2024
@YodaDaCoda YodaDaCoda mentioned this issue Oct 3, 2024
Open
@v-schhabra v-schhabra removed Area: Artifacts Area: ABTT Akvelon Build Tasks Team area of work Task: Bash labels Oct 3, 2024
@v-schhabra
Copy link
Contributor

v-schhabra commented Oct 3, 2024 •
edited
Loading

Hi @YodaDaCoda
Could you please test the pipeline and check if you are still having issues or not?
We did task override from 2.246.0 to 2.240.4 version which was the last successful version.

@YodaDaCoda
Copy link
Author

I'll have a look. Yesterday when I raised this issue I was using docker@2 and it was running v2.246.0.

I had a look at the azure-pipelines-tasks-common-packages library and have submitted a PR that I'm confident would resolve the issue properly. I can see from the git blame on the file that attempts have been made...

@YodaDaCoda
Copy link
Author

docker@2 is executing 2.246.0 for me on an ms-hosted agent.

@v-schhabra
Copy link
Contributor

@YodaDaCoda Could you pls delete the DockerV2 task from agent folder and then try creating a new release?

@YodaDaCoda
Copy link
Author

The agents are hosted by azure, we're not hosting our own agents and don't have access to delete anything from the agent folder.

I've reverted to [email protected] for now.

@v-schhabra
Copy link
Contributor

@YodaDaCoda Could you pls delete the DockerV2 task from agent folder and then try creating a new release?

Sorry I thought you are using self-hosted agents.

@v-schhabra
Copy link
Contributor

Issue has been fixed in this PR #20516
I hope you ae not facing any issue now.

@v-schhabra v-schhabra added the Area:RM RM task team label Nov 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Area: Release Area:RM RM task team bug regression This used to work, but a change in the service/tasks broke it. triage
Projects
None yet
Milestone
No milestone
2 participants
@YodaDaCoda @v-schhabra