Reintegrate repo-specific Nix code into iohk-nix

[deepfire]

@deepfire commented on Thu Nov 07 2019

Terminology

• iohk-nix :: The "standard library" of IOHK-specific Nix code intended to simplify usage of haskell.nix by the IOHK client repositories.
• client repository :: repositories using the IOHK Nix infrastructure, such as cardano-ledger, ouroboros-network and cardano-node.

The problem

After the initial introduction of iohk-nix into the build machinery of IOHK's Haskell-code-carrying repositories grew, these repositories started inevitably diverging -- that is, accumulating local, repository-specific Nix changes, extending the functionality provided in iohk-nix.

Some of those extensions are truly repository-specific, but not all are -- some extensions are conceptually repetitions of the same thing from the other repository, and so they contribute to the overall complexity of the system in an excessive way.

Proposed solution

Overall, we should revisit the way how the repository-specific Nix build system extensions are defined -- with the aim of making it as minimal and declarative as possible.

To do this effectively, we would need to perform a systematic review of what Nix extensions are defined in the client repositories, for what purpose, and based on that picture, define a more declarative and comprehensive interface between iohk-nix and client repositories -- and then implement that across all of IOHK repositories.

Time frame

This is not an immediate problem, but rather one of the sort that reduces the overall efficiency of the development process.

This appears to define the planning horizon for this work as vague "middle-term".

Development resources

Due to systemic nature of the problem, one would expect that this would require dedicated expert effort the DevOps team.

---

@angerman commented on Thu Nov 07 2019

As if a while ago, Haskell.nix is just an overlay and as such orthogonal to iohk-nix as well as nixpkgs.

To use haskell.nix all the consumer would do is:

with import <nixpkgs> (import <haskell-nix>); 
haskell-nix.function args;

As such I believe this issue might already be resolved with the Haskell.nix overlays?

---

@deepfire commented on Thu Nov 07 2019

@angerman, thank you!

I've updated the issue to reflect that haskell.nix is no longer part of the problem.

[disassembler]

I had a discussion with @rvl last night, and the new haskell.nix gets rid of the need for having haskell.nix abstraction in iohk-nix. The intended plan is:

1. Use niv in all repositories for source management
2. Pin haskell.nix (and potentially stackage/hackage override versions), iohk-nix (which will only have nix helper functions shared across team and environment configuration)
3. Use a nixpkgs overlay to get haskell.nix.
4. Implement one of haskell.nix functions to build using stack our cabal.

Since niv is already done on cardano-explorer, I intend to use it as a model for what other teams need to do.