Allow to use deterministic mtime in jar

[bmwiedemann]

When updating a .jar file, we use the SOURCE_DATE_EPOCH env variable to determine the mtime value of the manifest file.

See https://reproducible-builds.org/ for why this is good.

This branch will fail if an incompatible jar version (e.g. openjdk < 17) is used. This is probably preferable to silently generating non-deterministic build results. And it keeps the code simpler.

[eli-schwartz]

I think that ideally we should be testing this in CI. We can use unittests/ to define a custom python routine that first runs some arbitrary python code (like setting os.environ :)) and then tries to build one of the sample test projects in test cases/.

[eli-schwartz]

Per the spec:

If the value is malformed, the build process SHOULD exit with a non-zero error code.

Which relying on python to catch invalid string-to-int conversions will do.

That being said, I wonder whether it's worth explicitly catching the exception and re-raising MesonException with a more specific error message?

[bmwiedemann]

IMHO, it is not worth the extra complexity to catch cases that should never happen. The usual cases are

1. the variable is not set
2. the variable is set and contains a valid integer

[bmwiedemann]

As you know the test infra, could you add such a test? For me it could take days (which I don't have) to figure out all the details.

[eli-schwartz]

As you know the test infra, could you add such a test? For me it could take days (which I don't have) to figure out all the details.

I'm happy to advise! But it's your PR, so it's also your responsibility to complete the PR.

Here is a basic java test sample project that will automatically test any commits regarding jar() in meson already: https://github.com/mesonbuild/meson/tree/master/test%20cases/java/1%20basic

Note it is literally just a sample meson.build, and the "project tests" harness attempts to configure compile and install all sample project directories in turn.

By default all non- "unit" test sample projects are run by run_project_tests.py. The unit/ directory is unsuitable for running at all without arbitrary python setup code. The run_unittests.py suite of tests runs a mixture of unit/ and regular project tests, but with added arbitrary python setup code. The test harness we use is the python stdlib unittest (you can use pytest to run it for the better logging, but the test cases remain unittest-compatible and can be run without installing additional packages besides the python interpreter itself).

Here is an example of building a sample project (as a platform agnostic test which means that in CI it will only run on "fast" platforms) with arbitrary python code run first:

meson/unittests/platformagnostictests.py

Lines 569 to 582 in 6835b6e

	def test_readonly_sourcedir(self) -> None:
	"""Test building with read-only source directory."""
	testdir = self.copy_srcdir(os.path.join(self.common_test_dir, '233 wrap case'))
	# Make the source directory and all its contents read-only recursively
	# Keep execute permission on directories
	for dir, _, files in os.walk(testdir):
	os.chmod(dir, 0o555)
	for file in files:
	filepath = os.path.join(dir, file)
	os.chmod(filepath, 0o444)
	self.init(testdir)
	self.build()

Here is the definition of self.build which can pass overridden envvars:

meson/unittests/baseplatformtests.py

Line 274 in 6835b6e

def build(self, target=None, *, extra_args=None, override_envvars=None, stderr=True):

[eli-schwartz]

For more details see https://mesonbuild.com/Contributing.html#tests (and if you really think that it will take you days to figure out how to contribute a.test case, then that is a documentation bug, so please tell us how we can improve the contributing docs!)

[bmwiedemann]

I managed to add some test and run it with
./run_unittests.py LinuxlikeTests.test_jar_manifest_mtime or pytest unittests/linuxliketests.py -k test_jar_manifest_mtime

However, I cannot find the .jar file it should create and then we would still need to check the mtime that now should stop to vary. The docs seem to only cover the basics, not these specific things. Would we just use python zip modules for the test?

[eli-schwartz]

It should be in self.builddir (or self.installdir if you run self.install()) and yes, it's preferable to use python's zip libraries rather than shelling out to the system zip utilities in /usr/bin as the latter won't work when the tests are run on Windows.

[bmwiedemann]

Thanks, that got me a bit further. Now the test fails - my guess is that the patched code is not actually run in the test. I think, it should be mesonbuild/minstall.py : run => do_install => install_targets => fix_rpath => fix_jar but not sure how to get that triggered.

[eli-schwartz]

Here is the definition of self.build which can pass overridden envvars:

And currently you are passing it to init(), which is like setting it before running meson setup -- but the environment variable is processed by meson install?

[bmwiedemann]

Indeed, the env override was needed for the install step. The test looks good now.

Is the hardcoded myprog.jar okay or should we parse the test.json similar to how platformagnostictests.py:174 does it?

And should the test move to a different file? Maybe it could also be tested on Windows?