1.11.0 works while 1.12.0 fails for apptainer+wget ( wget: unable to resolve host address 'ftp.sra.ebi.ac.uk' )

[lindenb]

Description of the bug

Hi all,

the workflow works with 1.11.0

$ cat ids.csv 
ERR861313


rm -rf work && module load apptainer && nextflow run nf-core/fetchngs -r 1.11.0 -profile apptainer,glicid -resume --input ids.csv --outdir output                                                  




Nextflow 24.10.2 is available - Please consider updating your version to it
N E X T F L O W  ~  version 24.10.0
Launching `https://github.com/nf-core/fetchngs` [high_elion] DSL2 - revision: 04ee5031a4 [1.11.0]
WARN: Access to undefined parameter `monochromeLogs` -- Initialise it to a default value eg. `params.monochromeLogs = some_value`


------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/fetchngs v1.11.0-g04ee503
------------------------------------------------------
Core Nextflow options
  revision       : 1.11.0
  runName        : high_elion
  containerEngine: apptainer
  launchDir      : /home/me/src/faqbioinfo/src/nextflow/20241209.nextflow.glicid
  workDir        : /home/me/src/faqbioinfo/src/nextflow/20241209.nextflow.glicid/work
  projectDir     : /scratch/nautilus/users/me/.nextflow/assets/nf-core/fetchngs
  userName       : me
  profile        : apptainer,glicid
  configFiles    : 

Input/output options
  input          : ids.csv
  outdir         : output

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/fetchngs for your analysis please cite:

* The pipeline
  https://doi.org/10.5281/zenodo.5070524

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/fetchngs/blob/master/CITATIONS.md
------------------------------------------------------
[55/965386] Submitted process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (ERR861313)
[c2/753446] Submitted process > NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:CUSTOM_SRATOOLSNCBISETTINGS (ncbi-settings)
[86/0e9954] Submitted process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP (1)
[b5/3f465b] Submitted process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX941081_ERR861313)
[e1/30c030] Submitted process > NFCORE_FETCHNGS:SRA:SRA_TO_SAMPLESHEET (ERX941081_ERR861313)
[4c/29cba8] Submitted process > NFCORE_FETCHNGS:SRA:SRA_MERGE_SAMPLESHEET
[50/f90882] Submitted process > NFCORE_FETCHNGS:SRA:MULTIQC_MAPPINGS_CONFIG
-[nf-core/fetchngs] Pipeline completed successfully-
WARN: =============================================================================
  Please double-check the samplesheet that has been auto-created by the pipeline.

  Public databases don't reliably hold information such as strandedness
  information, controls etc

  All of the sample metadata obtained from the ENA has been appended
  as additional columns to help you manually curate the samplesheet before
  running nf-core/other pipelines.
===================================================================================

but it fails to run with 1.12:

$ rm -rf work && module load apptainer && nextflow run nf-core/fetchngs -r 1.12.0 -profile apptainer,glicid -resume --input ids.csv --outdir output

Nextflow 24.10.2 is available - Please consider updating your version to it
N E X T F L O W  ~  version 24.10.0
Launching `https://github.com/nf-core/fetchngs` [elegant_kimura] DSL2 - revision: 8ec2d934f9 [1.12.0]                                                                                                             
WARN: Access to undefined parameter `monochromeLogs` -- Initialise it to a default value eg. `params.monochromeLogs = some_value`
                                                    
                                                    
------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'                                                        
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,                                                                                                                                                                 
                                        `._,._,'
  nf-core/fetchngs v1.12.0-g8ec2d93
------------------------------------------------------
Core Nextflow options                 
  revision       : 1.12.0                                                                                
  runName        : elegant_kimura
  containerEngine: apptainer
  launchDir      : /home/me/src/faqbioinfo/src/nextflow/20241209.nextflow.glicid 
  workDir        : /home/me/src/faqbioinfo/src/nextflow/20241209.nextflow.glicid/work
  projectDir     : /scratch/nautilus/users/me/.nextflow/assets/nf-core/fetchngs
  userName       : me
  profile        : apptainer,glicid
  configFiles    : 
                                                    
Input/output options                                                                                                                                                                                              
  input          : ids.csv                                                                               
  outdir         : output
                                                    
!! Only displaying parameters that differ from the pipeline defaults !!                                                                                                                                           
------------------------------------------------------
If you use nf-core/fetchngs for your analysis please cite:
                                                                                                                                                                                                                  
* The pipeline
  https://doi.org/10.5281/zenodo.5070524                                                                                                                                                                          

* The nf-core framework                   
  https://doi.org/10.1038/s41587-020-0439-x        
                                                                                                         
* Software dependencies                                                                                  
  https://github.com/nf-core/fetchngs/blob/master/CITATIONS.md
------------------------------------------------------                 
[55/965386] Submitted process > NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (ERR861313)
[86/0e9954] Submitted process > NFCORE_FETCHNGS:SRA:SRA_RUNINFO_TO_FTP (1)
[70/7ed288] Submitted process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX941081_ERR861313)
[70/7ed288] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX941081_ERR861313)` terminated with an error exit status (4) -- Execution is retried (1)
[cc/a62795] Re-submitted process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX941081_ERR861313)
[cc/a62795] NOTE: Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX941081_ERR861313)` terminated with an error exit status (4) -- Execution is retried (2)
[12/064087] Re-submitted process > NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX941081_ERR861313)
ERROR ~ Error executing process > 'NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX941081_ERR861313)'
                                                    
Caused by:                                          
  Process `NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP (ERX941081_ERR861313)` terminated with an error exit status (4)                                                                                                      
                                                                                                         
                                                    
Command executed:                                                                                        
                                                                                                         
  wget \ 
      -t 5 -nv -c -T 60 \                                                                                                                                                                                         
      -O ERX941081_ERR861313_1.fastq.gz \                                                                                                                                                                         
      ftp.sra.ebi.ac.uk/vol1/fastq/ERR861/ERR861313/ERR861313_1.fastq.gz   
                                   
  echo "1a4b9bfe9ddab71db795a9f4a2aee3f1  ERX941081_ERR861313_1.fastq.gz" > ERX941081_ERR861313_1.fastq.gz.md5                                                                                                    
  md5sum -c ERX941081_ERR861313_1.fastq.gz.md5                                                                                                                                                                    
                                                    
  wget \                                            
      -t 5 -nv -c -T 60 \                                                                                
      -O ERX941081_ERR861313_2.fastq.gz \       
      ftp.sra.ebi.ac.uk/vol1/fastq/ERR861/ERR861313/ERR861313_2.fastq.gz                                 
                                              
  echo "795a85530c9bcd6d14a3eea64f3d425c  ERX941081_ERR861313_2.fastq.gz" > ERX941081_ERR861313_2.fastq.gz.md5                                                                                                    
  md5sum -c ERX941081_ERR861313_2.fastq.gz.md5  
                                   
  cat <<-END_VERSIONS > versions.yml                                                                     
  "NFCORE_FETCHNGS:SRA:SRA_FASTQ_FTP":
      wget: $(echo $(wget --version | head -n 1 | sed 's/^GNU Wget //; s/ .*$//'))                       
  END_VERSIONS                   
                                                    
Command exit status:                                                                                                                                                                                              
  4                                                                                                                                                                                                               
                                                                                                         
Command output:                               
  (empty)                          
                                                    
Command error:                                      
  WARNING: Skipping mount /opt/software/glicid/containers/apptainer/var/apptainer/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container                                                
  wget: unable to resolve host address 'ftp.sra.ebi.ac.uk'                                               
                                                    
Work dir:                                           
  /home/me/src/faqbioinfo/src/nextflow/20241209.nextflow.glicid/work/12/0640876d906c4b7b8a84c73590f526                                                                                   
                                                                                                         
Container:                                                                                               
  /scratch/nautilus/users/me/.singularity/cache/quay.io-biocontainers-wget-1.20.1.img                                                                                                    
                                                    
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`                                                                                                 

 -- Check '.nextflow.log' file for details
-[nf-core/fetchngs] Pipeline completed with errors-
WARN: =============================================================================                      
  Please double-check the samplesheet that has been auto-created by the pipeline.                        
                                                                                                         
  Public databases don't reliably hold information such as strandedness
  information, controls etc                                                                              
                                                                                                         
  All of the sample metadata obtained from the ENA has been appended                   
  as additional columns to help you manually curate the samplesheet before                                                                                                                                        
  running nf-core/other pipelines.                                                                       
===================================================================================                                                                                                                               

in my local nextflow.config:

profiles {

        glicid {

                singularity {
                        envWhitelist="HTTP_PROXY,HTTPS_PROXY,FTP_PROXY,http_proxy,https_proxy,ftp_proxy"
                        autoMounts = true
                        }


                process {
                        executor="slurm"
                        clusterOptions = "--qos=short"
                        queue = "standard"
                        cache = "lenient"
                        errorStrategy = "retry"
                        maxRetries = 3
                        time = "24h"

                        /** at the time of writing I didn't check all the labels */

                        withLabel:process_single {
                                queue = {"standard"}
                                cpus={1}
                                memory = {2.GB  * task.attempt }
                        }
(...)
                }
        }
}

Command used and terminal output

see above

Relevant files

No response

System information

No response