Substitution fails during NI build when java.prefs module is not present

[Akirathan]

In enso, we are building NI from a small JDK created by jlink. On latest GraalVM CE 25.0.0, it is no longer possible to build NI from small JDK without java.prefs module on MacOS.

Minimal reproducer

Makefile:

NI_BUILDER_MODULES = "org.graalvm.nativeimage.builder,org.graalvm.nativeimage.driver,org.graalvm.nativeimage.librarysupport,org.graalvm.nativeimage.objectfile,org.graalvm.nativeimage.pointsto"
NI_BASE_MODULES = "org.graalvm.nativeimage,org.graalvm.nativeimage.base,com.oracle.graal.graal_enterprise,com.oracle.svm.svm_enterprise"
JDK_MODULES = "java.naming,java.net.http,java.rmi,jdk.attach,jdk.charsets,jdk.crypto.ec,jdk.httpserver,jdk.localedata,jdk.jdwp.agent,jdk.security.auth"
ADD_MODULES = "$(NI_BUILDER_MODULES),$(NI_BASE_MODULES),$(JDK_MODULES)"

MODULE_JARS = "$$JAVA_HOME/lib/svm/bin/../../graalvm/svm-driver.jar:$$JAVA_HOME/lib/svm/bin/../builder/native-image-base.jar:$$JAVA_HOME/lib/svm/bin/../builder/espresso-svm.jar:$$JAVA_HOME/lib/svm/bin/../builder/objectfile.jar:$$JAVA_HOME/lib/svm/bin/../builder/pointsto.jar:$$JAVA_HOME/lib/svm/bin/../builder/svm-enterprise.jar:$$JAVA_HOME/lib/svm/bin/../builder/svm.jar:$$JAVA_HOME/lib/svm/bin/../builder/svm-configure.jar:$$JAVA_HOME/lib/svm/bin/../builder/svm-capnproto-runtime.jar:$$JAVA_HOME/lib/svm/bin/../builder/svm-foreign.jar:$$JAVA_HOME/lib/svm/bin/../library-support.jar"

NI_EXE = small_jdk/lib/svm/bin/native-image

all: clean native_image

small_jdk:
	jlink --output small_jdk --module-path "$(MODULE_JARS)" --add-modules "$(ADD_MODULES)"
	cp -r $$JAVA_HOME/lib/graalvm $$JAVA_HOME/lib/svm $$JAVA_HOME/lib/static $$JAVA_HOME/lib/truffle small_jdk/lib

Main.class: Main.java
	javac Main.java

native_image: Main.class small_jdk
	$(NI_EXE) Main -o main

clean:
	rm -rf small_jdk
	rm -f Main.class
	rm -f main

Main.java:

public class Main {
    public static void main(String[] args) {
        System.out.println("Hello from Main");
    }
}

Running make on GraalVM CE 25.0.0, the following error is reported:

========================================================================================================================
GraalVM Native Image: Generating 'main' (executable)...
========================================================================================================================
[1/8] Initializing...
                                                                                    (0.0s @ 0.08GB)
Error: Substitution target for com.oracle.svm.core.posix.darwin.Target_java_util_prefs_FileSystemPreferences is not loaded. Use field `onlyWith` in the `TargetClass` annotation to make substitution only active when needed.

Running make on GraalVM CE 24.0.1 is OK.

Note that @JaroslavTulach was not possible to reproduce this error on Linux AMD64. Seems to affect only Mac (I have aarch64, but it probably also affects x64).

Context

• Encountered in Update to GraalVM 25 enso-org/enso#14019 (comment)
• The process how to create small JDK was recommended to us by @fniephaus in https://graalvm.slack.com/archives/CN9KSFB40/p1715117172381849?thread_ts=1715083564.880179&cid=CN9KSFB40
• Tracked as an issue in Investigate the need for java.prefs module in NI build enso-org/enso#14051

[oracle-contributor-agreement]

Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA).
The following contributors of this PR have not signed the OCA:

• PR author: [email protected] (@Akirathan)

To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application.

When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated.

If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public.

[Akirathan]

Because of this issue, we had to add java.prefs module to small JDK used for NI builds as a workaround. See enso-org/enso#14019 (comment). I have confirmed that with this PR applied, java.prefs can be removed from the small JDK.

[JaroslavTulach]

• PR author: [email protected] (@Akirathan)

• Pavel is making this contribution on behalf of Enso
• the same company on behalf of I am making my contributions
• looking at my mailbox I see Enso signed the company agreement in ~2022
• such a company wide OCA agreement shall cover all Enso employees
• as such this contribution by Pavel shall be OCA covered as well

[JaroslavTulach]

• The original code in question was integrated by @vjovanov as part of [GR-63031] Enable -H:Preserve=all for macOS. #11106
• Vojin, what do you think about these changes proposed by Pavel?

[vjovanov]

Looks good, thanks for fixing this issue.

I am wondering if we should do this systematically for all JDK substitutions. For example, we could ask the JDK substitutions to be targeted with a module. This would require us to change a lot of substitutions, but it would solve this problem for all other jlink related issues. @zapster WDYT?

[zapster]

I am wondering if we should do this systematically for all JDK substitutions. For example, we could ask the JDK substitutions to be targeted with a module.

If running on a custom jlinked JDK is something we want to actively support, then it sounds reasonable. However, I'm not sure how much that aligns with the goals of Project Terminus #12236. Maybe to go one step back, what issue are you solving with the "small jdk"?

This would require us to change a lot of substitutions, but it would solve this problem for all other jlink related issues. @zapster WDYT?

I believe it is possible to implement this via a special case in the AnnotationSubstitutionProcessor. We would need to have a map from classes to jdk modules. We would need create it at/before native-image build time and bake into the native-image executable. I think list could be created automatically, e.g., via a javac annotation processor, assuming that the native-image executable is built with a "full jdk", which is the case in the reproducer outlined above. Then in AnnotationSubstitutionProcessor#findTargetClass we simply ask the map and check if the module is present, right after we evaluated the onlyWith class. That way, we would not need to change any substitution.

That said, I'd like to learn more about the motivation for jlinking. There might be other issues since the native-image executable does not expect that the base modules differ at native-image build and native-image run time.

[JaroslavTulach]

Hello Josef, thanks for considering Enso use-case...

what issue are you solving with the "small jdk"?

Enso is server side application. We don't want java.desktop & co. in our GraalVM. Neither HotSpot nor native image mode (we use both). We are using hint from Fabio to built jlink version of GraalVM...

• The process how to create small JDK was recommended to us by @fniephaus in https://graalvm.slack.com/archives/CN9KSFB40/p1715117172381849?thread_ts=1715083564.880179&cid=CN9KSFB40

...yes, it would be great to let jlink be supported by GraalVM more "officially".

[vjovanov]

OK, let us merge this PR for now until we know the shape of GraalVM 25.1 better. Then we can re-iterate.

I'll put this to merge and backport to 25.0.1.

[JaroslavTulach]

We would need to have a map from classes to jdk modules.

Btw. Element.getModuleOf maybe the method you are looking for, @zapster