Print maximum ghc version in ebuild comment

[solpeth]

Background

Haskell packages are famously/notoriously strict on dependency version ranges. This improves predictability of correct compilation and operation, but increases the maintenance burden to ensure that the fragile/non-existent ABI stability is appropriately dealt with when bumping package versions.

All packages are required to have version bounds on their dependencies in hackage. We reflect this in our ebuilds generated by hackport, except for GHC.

Problem

We have very few resources in the Gentoo Haskell project. Most haskell work occurs here rather than in ::gentoo. No haskell packages are stabilised as a rule. Because of how easy it is to break packages with dependency version changes, we have to methodically and meticulously work through dependency trees when bumping packages.

The most complex package to bump is GHC itself.

GHC includes not only the compiler, but core libraries. It depends on these libraries for its internal operation. These core libraries should almost never be upgraded by Portage to a different version, as it may break GHC's operation or confuse cabal when building other packages that depend on these libraries. Thus, we must keep the GHC version and a bunch of non-upgradeable packages tightly matched.

Some packages are not visible to Portage (some are due to historical changes in bundled libraries, e.g. some packages were once not bundled but now are, and vice versa, leading to ::haskell maintainers marking such packages as dummy libraries that defer to GHC's bundled libraries rather than emerging a second copy of the library from hackage). Hackport will determine the minimum GHC version for any given haskell package based on the versions of the bundled libraries that the package depends on. But it does not determine a maximum version.

This is by and large intended behaviour to avoid adding significantly to the workload of loosening upper bounds. However, this also means that we have no visibility of package incompatibility with newer GHC versions. This means we must, by hand and with our limited resources, try to compile the entire ::haskell repository on a newer GHC version to determine if it is reasonably safe to bump the GHC version for all users.

Current GHC bumping process

1. Build a list of key packages (easy)
2. Build 75% of the repository (very time-consuming)

With such limited resources, step 2 takes a very long time. ::haskell only recently keyworded a version of GHC that is not even supported upstream anymore. This is too slow.

Proposed GHC bumping process

1. Build a list of key packages (unchanged)
2. Rely on upstream dependency data to determine compatibility for the rest of the repository

My proposed solution to step 2 is to have hackport insert an easily-greppable comment in the ebuilds which marks the highest ghc version the package is compatible with. While not perfect, it has the benefit of:

1. Avoiding adding to the maintenance burden of adding the GHC upper bound in RDEPEND
2. Avoiding having to build the whole repository to get to 75% success.

We should be able to rely on upstream reports for compatibility. Many projects use CI tools to automate this. Hackage itself checks packages automatically against a range of GHC versions. Where a Gentoo-specific quirk contributes to a compatibility issue that should not exist, we can deal with that directly with our limited resources.

Pros

1. Much faster bumping of GHC versions to remain relevant and up to date to an upstream-supported version at the very least. We effectively eliminate the 75% build requirement while minimising unintended breakage. Although we are for all intents and purposes the place to go to for haskell packaging in Gentoo Linux, we are also not ::gentoo, and some (but not too much) breakage should be occasionally expected and acceptable. We should be able to deal with such breakage quickly and directly when it does arise
2. Freeing up resources for other tasks, e.g. improving tooling and syncing to ::gentoo.

Cons

1. We are relying on upstream being correct in their determination of GHC compatibility, rather than directly verifying this ourselves
2. This will not help in the immediate sense, as ebuilds will need to be commented by hackport when otherwise updated.

Mechanism of Operation in Hackport

1. Determine maximum GHC version allowed by the package (see the current logic in determining the minimum GHC version, which may be able to be re-used for this purpose)
2. Print a comment (perhaps near the top of the ebuild like hackport flags) of a specific format which is easily greppable by developers, and perhaps can be easily parsed by automated scripts.

[solpeth]

@hololeap if I start on this I'll push a branch we can work on together. Otherwise if you do the same I'll hack on that if needed.

[solpeth]

See https://github.com/gentoo-haskell/hackport/blob/c1df6dccf7a5db73e1dc419a6cc55ca9ae37171c/src/Portage/GHCCore.hs, e.g.:

hackport/src/Portage/GHCCore.hs

Lines 118 to 148 in c1df6dc

	packageBuildableWithGHCVersion
	:: GenericPackageDescription
	-> FlagAssignment
	-> (DC.CompilerInfo, InstalledPackageIndex)
	-> Either [Cabal.Dependency] (PackageDescription, FlagAssignment)
	packageBuildableWithGHCVersion pkg user_specified_fas (compiler_info, pkgIndex) = trace_failure $
	finalizePD user_specified_fas defaultComponentRequestedSpec (dependencySatisfiable pkgIndex) platform compiler_info [] pkg
	where trace_failure v = case v of
	(Left deps) -> trace (unwords ["rejecting dep:" , show_compiler compiler_info
	, "as", show_deps deps
	, "were not found."
	]
	) v
	_ -> trace (unwords ["accepting dep:" , show_compiler compiler_info
	]
	) v
	show_deps = show . map prettyShow
	show_compiler (DC.CompilerInfo { DC.compilerInfoId = CompilerId GHC v }) = "ghc-" ++ prettyShow v
	show_compiler c = show c
	-- | Given a 'GenericPackageDescription' it returns the miminum GHC version
	-- to build a package, and a list of core packages to that GHC version.
	minimumGHCVersionToBuildPackage :: GenericPackageDescription -> FlagAssignment -> Maybe ( DC.CompilerInfo
	, [Cabal.PackageName]
	, PackageDescription
	, FlagAssignment
	, InstalledPackageIndex)
	minimumGHCVersionToBuildPackage gpd user_specified_fas =
	listToMaybe [ (cinfo, packageNamesFromPackageIndex pix, pkg_desc, picked_flags, pix)
	| g@(cinfo, pix) <- ghcs
	, Right (pkg_desc, picked_flags) <- return (packageBuildableWithGHCVersion gpd user_specified_fas g)]

[solpeth]

For context: I have considered the idea of having a CI job running in the background to automate the building process against newer GHC versions. I have not put it forward because I have not looked into its feasibility at all, and is still resource-heavy.

Nonetheless, if someone wanted to get such an automated process set up and operational, that would help in the immediate sense.

I still think it would be useful for hackport to encode maximum version information where it is available, at least as a back up.