Dockerfile cant write to docker volumes on github build agents.

[PeterKnealeCMRI]

Unable to write to volumes when running on github build agents

When running VEP within a docker container upon a github build agent I was unable to write output files to the mounted volume.

More specifically, I encounted the below error when attempting to set the --output-file parameter to a location that was a volume that was shared between the docker host and container such as /output/......

I believe this is due to the usage of the USER keyword within the dockerfile which is not supported by github
See

  Ensure your Dockerfile does not set the USER instruction, otherwise you will not be able to access GITHUB_WORKSPACE.
  https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#file-systems

Checking a few other common dockerfiles, postgres/rabbit etc I find that they dont use the USER keyword in their dockerfiles

Additional information

• I tried running as password-less sudo with no luck
• I tried One solution is to use the option :Z within the Docker -v option (only from docker 1.7.0): as described in documentation
• I tried An other solution is to change the read/write access of the mounted volume ($HOME/vep_data): as described in documentation

System

• VEP version: 109.3 (and latest)
• VEP Cache version: 109.3
• Perl version: Whatever is installed within the docker container
• OS: [e.g. MacOS, Ubuntu, CentOS] Ubuntu 22.04
• tabix installed ? No idea

Full VEP command line

docker run \
  --entrypoint vep \
  -v $(pwd)/vep:/data \
  ensemblorg/ensembl-vep:latest \
    --fork 4 \
    --biotype \
    --canonical \
    --distance 5000 \
    --species homo_sapiens \
    --merged \
    --use_given_ref \
    --symbol \
    --assembly GRCh38 \
    --cache \
    --offline \
    --no_stats \
    --buffer_size 3500 \
    --force_overwrite \
    --input_file /data/xxxx.acceptor.vcf \
    --output_file /data/xxxx.acceptor.vep

Full error message

-------------------- EXCEPTION --------------------
95 MSG: ERROR: Could not write to output file /data/xxxx.acceptor.vep

Data files (if applicable)

They include:

• The input file
• The output file
• The custom file(s)
  You can provide a sample of them

I'll provide a repro if you'd like further confirmation.

Workaround

• Ensure you run the container with a name
  --name vep_donor

• Set the output file to a path that is not associated with volume.
  --output_file /opt/vep/xxxx.vep

• Copy out the file after the container stops using docker cp
docker cp vep_donor:opt/vep/xxxx.vep .

[nakib103]

@PeterKnealeCMRI ,

Thanks for flagging this issue. We will look more into and get back to you with more info.

Best regards,
Nakib