dxCompiler 2.5.0

• Docker image dependencies that are DNAnexus platform files are included in applet's bundledDepends
• Adds -instanceTypeSelection compiler option to allow disabling compile-type instance type selection
• Adds -defaultInstanceType option
• Adds support for new London region, aws:eu-west-2-g
• Fixes -projectWideReuse
• Fixes evaluation of outputs when one declaration depends on another

Dependency updates

dxCommon 0.8.0

• Adds getTargetDir methods to LocalizationDisambiguator
• Fixes use of localizationDir together with force in SafeLocalizationDisambiguator
• Adds FileSource.resolveDirectory as a separate method from resolve

dxApi 0.10.0

• Removes price-based selection of instance types in favor of rank-based selection
• Fixes parsing of non-file-type default values that are reference-type links
• Fixes parsing of parameter defaults/suggestions/choices that are of type Hash

dxFileAccessProtocols 0.5.0

• Implements resolveDirectory method

wdlTools 0.17.0

• Breaking Change Eval.applyMap is changed to Eval.applyAll and takes a Vector rather than Map argument. This is done to ensure the expressions are evaluated in order in case there are dependencies between them.
• Fixes parsing of runtime.returnCodes

dxCompiler 2.4.10

• Fix issue with using File declarations in scatter/conditional blocks
• Allow scattering on an empty array
• Update dxda version

dxCompiler 2.4.9

• Files that are generated by calling functions in worklfow expressions are now uploaded to the platform
• dxni now makes optional any input parameter for which the native app(let) has a default value
• Jobs now fail on out-of-disk errors
• Updated dxda and dxfuse to latest versions
• Updated dxCommon and dxApi dependencies

dxCompiler 2.4.8

• Fixes issue where an optional variable inside a conditional could have an inferred type of T??, which is illegal and results in a runtime error
• Specifies operating system when selecting optimal instance type
• Fixes issue with references between output variables
• Uses an output stage if there is an output parameter with a literal value (DNAnexus output parameters do not support default values)
• Fixes issue where compiling with -execTree [pretty|json] did not print the workflow tree
• Adds support for specifying app metadata in the hints section in WDL development version
• All compile-time calls to findDataObjects now search the context project for any file with no project specified
• All runtime dxAPI.describeFilesBulk calls, which invoke the platform's system.findDataObjects, now search in the job/analysis workspace first before searching the source project(s).
  • The call is used to search and describe input files for the analysis and to search and describe output files during the outputs reorganization of the analysis (in its default implementation, not the customized one). If the files cannot be found in the workspace container they are looked for in the project that qualifies file ID (e.g. in project_A if file_B's path is project_A:file_B)
• Better handles insufficient permissions when requesting instance type price list

Dependency updates

dxApi 0.0.6

• Adds option to DxApi.describeFilesBulk to search first in the workspace container
• DxApi.resolveDataObject now searches in the current workspace and/or project if the project is not specified explicitly. The call is used to find one data object at a time, it's not used in bulk resolution.
• Refactors DxFindDataObjects to use separate DxFindDataObjectsConstraints class for specifying constraints
• Uses the currently select project ID as the workspace ID when not running in a job
• Better handles insufficient permissions when requesting instance type price list

dxCompiler 2.4.7

• Fixes issue with using struct types in workflow outputs
• Fixes issue with array with optional item type

dxCompiler 2.4.6

• Fixes regression in WDL code generator where long expressions within placeholders are line-wrapped incorrectly

dxCompiler 2.4.5

• An applet that contains multiple scatter or conditional blocks will now have a name that is the concatenation of all the block names
• Fixes multiple issues with WDL code generator
• Fixes issue with referencing struct fields/call outputs in declarations within nested blocks

Warning: we discovered a regression in this release that affects tasks with long command blocks. Please wait until the next release if this applies to your WDLs.

dxCompiler 2.4.4

• Escapes WDL strings in generated code
• Fixes issues with using expressions in placeholder option values
• Fixes error when evaluating array element access for an optional value
• Fixes localization of files in identically named folders in different projects
• Fixes localization of files with the same name in the same folder

dxCompiler 2.4.3

• Fixes an issue where a file input from an different project than where the workflow is compiled is localized to an invalid path
• File downloads (including Docker images) no longer fail when retried after a previous failure
• Adds the -waitOnUpload compiler option, which causes all file uploads to block until they complete
• Fixes an issue where tasks with outputs that are collections of files (e.g. Array[File]) are compiled with an incorrect default input value
• Fixes an issue where using a field of a struct as a call input causes a runtime error, e.g.

  struct MyStruct {
    String s
  }
  workflow wf {
    input {
      MyStruct my
    }
    call mytask { input: s = my.s }
  }
  ``

dxCompiler 2.4.2

• -imports now works correctly - import statements can reference files that are relative to an import directory rather than the main document
• Coercion from Object to Map is now allowed - this enables the result of read_json to be assigned to a variable of type Map
• Strings with escape sequences are now processed correctly - to preserve escape sequences in a string, they need to be "double-escaped"
  • For example, to pass a regular expression containing a tab character to the second argument of the sub function:

    String s1 = "hello\tBob"
    String s2 = sub(s1, "\\t", " ")

  • Another common example is passing a read group to bwa mem. Whether the string is defined in WDL, in a JSON input file, or on the command line, it needs to be "double-escaped":

    String rg = "@RG\\tID:${sample_name}\\tSM:${sample_name}\\tLB:${sample_name}\\tPL:ILLUMINA"
    
    command <<<
    bwa -R "~{rg}" ...
    >>>

• Fixes an issue where indentation is stripped out of some commands
• The default reorg applet now polls the analysis until it is fully updated before proceeding. For custom reorg applets, see the updated code in the example.
• Fixes an issue with nested scatters that reference private variables from outer scopes, e.g.

  workflow wf {
    input {
      Array[String] samples
      Array[File] files
    } 
    scatter (i in range(length(samples))) {
      scatter (j in range(length(files))) {
        call mytask { 
          input: sample = samples[i], file = files[j]
        }
      }  
    }
  }