Deterministic build?

[kkimdev]

It seems like we can specify a specific git revision in fips.yml with rev when we import dependencies, but the dependencies of the imported library is up to the library's fips.yml.

Is there a good way to ensure specific reversions so that we can get deterministic build if needed?

For example in Rust, Cargo.lock file maintains a snapshot of dependency versions https://github.com/rust-lang-nursery/rls/blob/master/Cargo.lock and it's encurraged to check-in Git repository for reproduciability.

[floooh]

It should be possible to override 'sub-dependencies' by listing them with their required versions in your top-level fips.yml before the dependency which imports those sub-dependencies.

Basically, fips should resolve dependencies in the order they are listed in fips.yml, depth-first, and it will not touch dependency-directories that already exists.

It's a bit more work than the automatic lock-file snapshotting, but it should allow for the same level of control. If you try this let me know if this works, I did some changes to the dependency resolution fairly recently to ensure the right import-order, but the whole thing is a bit more complex than I'd like, there may be subtle bugs in it.

[kkimdev]

Thanks for the quick answer! Yeah, I just tried and it seems to work. I'll use this workaround for now. Though I think ideally snapshot support will be great.

For now:

• We need to manually resolve all the sub-dependencies. Not only cumbersome but also error-prone as dependencies can be changed over time.
• No easy way to track branch and get everything updated.

In my mind, the main usecase for package pinning is to maintain "the last known working build" while keeping everything up-to-date. I think cargo and yarn's approach is close to ideal. Basically, specifying version requirement (http://doc.crates.io/specifying-dependencies.html) in the main config file Cargo.toml and having a dependency snapshot file Cargo.lock updated only on cargo update command.

I think adding a snapshot file support shouldn't be too bad. As a first step, I'd say even just outputting [package name, git url, rev]s as a file on fips update is already useful, even if fips does not use that file. And as a second step, maybe we can add an option, something like ./fips update --from-snapshot.

What do you think about this approach?

[kkimdev]

Also, I guess the snapshot file format does not have to be different from fips.yml. The only difference will be that it enumerates all the sub-dependencies and also have rev: revision pinning for every package.

Does fips have an option to load a custom file rather than the default fips.yml?

[floooh]

Yes I agree that a proper snapshot file would be better, especially together with "fips update" (which may be does the wrong thing at the moment). fips.yml is currently hardcoded, but I think the lock-file should not be a plain copy of fips.yml. fips.yml is more like a general project-config file, for instance besides imports (and exports) you can also put general values in there which can then be read by custom verb- or code-generator scripts.

[kkimdev]

FYI: WIP pull request #147