revise: relaxes requirement of exact WDL version in imports

[claymcleod]

This PR relaxes the requirement that importing documents must match the WDL version of the importing document exactly.

Overview

I've been doing quite a lot of thinking towards preparing for better package management in WDL. One of the important bits of getting this right is going to be the asynchronous nature with which code is developed and distributed. The current restriction in the specification makes it incredibly difficult to build a cogent ecosystem of packages.

The problem

To illustrate this, consider a task imported_task that is written in WDL v1.2 by Team A and workflow workflow_foo that is written in WDL v1.2 by Team B. Upon the release of WDL v1.3, Team B may want to update workflow_foo to take advantage of some new language features. However, Team A is slow to update imported_task to WDL v1.3. Because WDL has this restriction, Team B is deadlocked unless they fork and reimplement imported_task simply to change the version of the document that contains it.

This hardship does not seem necessary; because of the backward compatibility of all WDL v1.x releases, the WDL v1.2 code of imported_task should compile just fine when imported into the WDL v1.3 context of workflow_foo (provided that the execution engine supports both WDL v1.2 and v1.3).

The solution

In my eyes, there is nothing stopping from simply loosening the requirements and allowing engines that support WDL v1.3 to import documents that have the same major version and a minor version that is less than or equal to the importing document.

Is this a breaking change?

I've given this some thought, and I can't see how loosening this restriction would be considered a breaking change. Previous versions of WDL can continue to uphold the restriction, and WDL v1.3 can simply drop it without consequence I believe. There is at least one additional constraint on implementers that this changes would introduce (though it is not related to the backwards compatibility of the spec).

Technically speaking, a WDL engine today could support, say, WDL v1.2 without needing to support prior versions of WDL v1.x. With the relaxing of this requirement, execution engines of WDL v1.3+ will be required to support WDL v1.x. I don't view this as a negative requirement, nor am I aware of any execution engines that this would practically affect negatively.

Conclusion

Though the change is small, consider this the discussion ground for this concept more broadly.

---

Before submitting this PR, please make sure:

• You have added a few sentences describing the PR here.
• You have added yourself or the appropriate individual as the assignee.
• You have updated the README.md or other documentation to account for these changes (when appropriate).
• You have updated the CHANGELOG.md describing the change and linking back to your pull request.
• You have read and agree to the CONTRIBUTING.md document.
• You have added or updated relevant example WDL tests to the specification.
  • See the guide for more details.

[patmagee]

I think there should be a mention of the effective version of the wdl document. Ie all imported docs are treated as the version of the first imported document.

[claymcleod]

I can definitely point it out though, I don't know if this is a requirement rather than a liberty that an execution engine might take to make their lives easier. If, for example, an execution engine wants to parse the various versions individually according to the defined version in the document and then stitch them together internally using its abstract representation, I see no issue with that either.

[patmagee]

This is a great idea and I think will make things a lot more flexible and reusable. IIRC the restriction did not enter the language for any reason other than Cromwell being unable to handle different versions at the time. Whether that is still true or not I am unsure

[peterhuene]

I kept typing out a long comment about why sprocket only enforces a major version match as there's technically no reason a 1.1 document can't make use of a 1.2 minor version document's exports, provided the exported items are representable/usable from 1.1. The hope was this would lower the barrier to adoption of newer minor versions.

But then I kept typing out counterexamples and really it was putting the onus on the WDL authors to ensure compatibility from the previous minor versions (e.g. don't use a Directory input in 1.2, don't use an enum input in 1.3, etc). Which would lead to probably just confusion, poor diagnostics, and a barrier to entry anyway.

Ultimately I settled on the change you have in this PR and we should fix sprocket to additionally enforce the minor version maximum.