Test lower bounds specified in main dependencies

[jdawang]

Description

This is a feature that allows users to specify packages that they want to test lower bounds of. If specified in an edgetest environment, edgetest will:

1. Setup the environment, like for the upgrade use case.
2. Install the lower bounds of packages, where the lower bounds of packages are parsed from the main dependencies, only with the >= specifier. This is the initial implementation and we can handle extras later.
3. Run the unit tests, as in the upgrade case.
4. Report if tests passed or failed.

Currently, there is no:

• Parsing of extras.
• Parsing of any requirement specifiers that do not contain >=, as this would require a search.
• Automatic updating requirement files as this would require a search.
• Default environments for each dependency and one with all requirements as for the upgrade case.
• Handling if a package is specified with extras in the dependencies and without in lower or vice versa. Might want to add this in right now, but can also leave for later and document.

So, no testing of lower bounds will be done unless explicitly asked for by the user.

There are also some changes that I made to get it to work:

• Schema for edgetests.envs environment now has a new section lower that is mutually exclusive with `upgrade.
• When setting up the environment, edgetest will now soft fail and report setup unsuccessful for that environment instead of just exiting with an exception right away. In practice, I ran into issues testing on packages with very outdated versions that they can't even be installed.
• Which leads to a new column in the report, Setup successful, which indicates whether the environment could be set up and everything installed alright.
• Another new column in the report for the lowered packages.

Closes #69

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• New and existing integration tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules

[fdosani]

@jdawang This is great! Thanks for the effort. I have one question, but I think your docs actually answer my question. Great experimental feature!

[ak-gupta]

Great work! A couple optional fixes but I appreciate the extensive testing and documentation you put into this.

[ak-gupta]

Documentation note: reading the utilities, this is a list of the packages to be lowered alongside the lower bound, correct? I.e.

[
    "pandas==1.5.1",
]

instead of

["pandas",]

Should we add that to the docs where lower is an input? It's different than the format of upgrade.

[jdawang]

Yeah that's a good catch. In the config, it's just like upgrade, but in TestPackage, the config is already parsed to provide the package to be lowered alongside the lower bound.

[ak-gupta]

Yeah not a huge deal since it's not user facing. Might save us some confusion down the road though.

[ak-gupta]

I'm finding this loop a bit difficult to read. What if we split it out a bit? Something along the lines of

Suggested change

	output["envs"][-1]["lower"] = "\n".join(
	f"{pkg_name}=={lower_bound}"
	for pkg_name, lower_bound in get_lower(
	config.get("options", "install_requires")
	).items()
	if _isin_case_dashhyphen_ins(
	pkg_name, output["envs"][-1]["lower"].split("\n")
	)
	and lower_bound is not None
	# TODO: Emit warning if lower_bound is None but package is in lower
	# TODO: Parse through extra requirements as well to get lower bounds
	)
	output["envs"][-1]["lower"] = ""
	for pkg_name, lower_bound in get_lower(config.get("options", "install_requires").items():
	if (
	_isin_case_dashhyphen_ins(pkg_name, output["envs"][-1]["lower"].split("\n")) and lower_bound is not None
	):
	output["envs"][-1]["lower"] += f"{pkg_name}=={lower_bound}\n"
	# TODO: Emit warning if lower_bound is None but package is in lower
	# TODO: Parse through extra requirements as well to get lower bounds

[jdawang]

Made a couple simplifying changes @fdosani and @ak-gupta and I think it is ready now.

[ak-gupta]

LGTM! Great work!