[BUG] Delombok not Working with Modules

[bradleylarrick]

The Lombok plugin and delombok are not working correctly with modules. I've created
a sample project at https://github.com/bradleylarrick/delombok-example.git.

I'm using Maven 3.8.1, Lombok 1.18.20 and JDK 11 (the same behavior exhibits with JDK 16).

Without the module-info.java file the delombok process works correctly. When I add
the module-info.java file the lombok plugin errors saying that the modules referenced in
module-info.java can't be found:

src\main\java\module-info.java:24: error: module not found: com.fasterxml.jackson.annotation
src\main\java\module-info.java:25: error: module not found: com.fasterxml.jackson.core
.
.
.

I get the same errors when I try running the delombok from the command line using the same arguments
the plugin uses.

I can get the command line delombok to work if I change the sourcepath to point to the lombok-enabled
code (src/main/lombok) instead if the standard code (src/main/java). This presents the problem of
requiring all classes the lombok-enabled code reference to be in the src/main/lombok path.

I tried changing the command line script to add the jar files to the module-path instead of the classpath,
but then I get the following errors on the lombok-enabled classes:

C:\projects\delombok-example\src\main\lombok\org\larrick\datagen\data\Address.java:7: error: file should be on source path, or on patch path for module
C:\projects\delombok-example\src\main\lombok\org\larrick\datagen\data\Ethnicity.java:7: error: file should be on source path, or on patch path for module
.
.
.

Please advise.

[Auties00]

t at https://github.com/bradleylarrick/delombok-example.git.

I was about to create this exact issue!
Other examples:
awhitford/lombok.maven#46
Auties00/Cobalt#24

[Rawi01]

I did some tests using your example project but was unable to compile it. I had to add lombok to the compiler plugin annotation path as mentioned on the lombok setup page and move the content of the lombok folder to the java folder to fix that. This doesn't fix the delombok goal but I was able to delombok it using the command line using java -jar lombok-1.18.20.jar delombok src/main/java -d target/generated-sources/delombok --module-path $cp.

[Auties00]

I did some tests using your example project but was unable to compile it. I had to add lombok to the compiler plugin annotation path as mentioned on the lombok setup page and move the content of the lombok folder to the java folder to fix that. This doesn't fix the delombok goal but I was able to delombok it using the command line using java -jar lombok-1.18.20.jar delombok src/main/java -d target/generated-sources/delombok --module-path $cp.

This didn't work for me, this is the command I wrote:
java --module-path C:\Users\alaut\.m2\repository\org\bouncycastle\bcpkix-jdk15on\1.68\bcpkix-jdk15on-1.68.jar;C:\Users\alaut\.m2\repository\org\bouncycastle\bcprov-jdk15on\1.68\bcprov-jdk15on-1.68.jar;C:\Users\alaut\.m2\repository\org\projectlombok\lombok\1.18.20\lombok-1.18.20.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.validation-api\1.1.1.8-jre16\jakarta.validation-api-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\undertow-websockets-jsr\1.1.1.8-jre16\undertow-websockets-jsr-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\undertow-servlet\1.1.1.8-jre16\undertow-servlet-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\undertow-core\1.1.1.8-jre16\undertow-core-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\undertow-parser-generator\1.1.1.8-jre16\undertow-parser-generator-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.servlet-api\1.1.1.8-jre16\jakarta.servlet-api-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jboss-logmanager\1.1.1.8-jre16\jboss-logmanager-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\jakarta\json\jakarta.json-api\2.0.0\jakarta.json-api-2.0.0.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.websocket-api\1.1.1.8-jre16\jakarta.websocket-api-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.xml.bind-api\1.1.1.8-jre16\jakarta.xml.bind-api-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.activation\1.1.1.8-jre16\jakarta.activation-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\org\codehaus\woodstox\stax2-api\4.2.1\stax2-api-4.2.1.jar;C:\Users\alaut\.m2\repository\org\jvnet\staxex\stax-ex\2.0.0\stax-ex-2.0.0.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\guava\1.1.1.8-jre16\guava-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.annotation-api\1.1.1.8-jre16\jakarta.annotation-api-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\fasterxml\jackson\core\jackson-databind\2.12.3\jackson-databind-2.12.3.jar;C:\Users\alaut\.m2\repository\com\fasterxml\jackson\core\jackson-annotations\2.12.3\jackson-annotations-2.12.3.jar;C:\Users\alaut\.m2\repository\com\fasterxml\jackson\core\jackson-core\2.12.3\jackson-core-2.12.3.jar;C:\Users\alaut\.m2\repository\org\junit\jupiter\junit-jupiter\5.6.3\junit-jupiter-5.6.3.jar;C:\Users\alaut\.m2\repository\org\junit\jupiter\junit-jupiter-api\5.6.3\junit-jupiter-api-5.6.3.jar;C:\Users\alaut\.m2\repository\org\opentest4j\opentest4j\1.2.0\opentest4j-1.2.0.jar;C:\Users\alaut\.m2\repository\org\junit\platform\junit-platform-commons\1.6.3\junit-platform-commons-1.6.3.jar;C:\Users\alaut\.m2\repository\org\junit\jupiter\junit-jupiter-params\5.6.3\junit-jupiter-params-5.6.3.jar;C:\Users\alaut\.m2\repository\org\junit\jupiter\junit-jupiter-engine\5.6.3\junit-jupiter-engine-5.6.3.jar;C:\Users\alaut\.m2\repository\org\apiguardian\apiguardian-api\1.1.0\apiguardian-api-1.1.0.jar;C:\Users\alaut\.m2\repository\org\junit\platform\junit-platform-engine\1.6.3\junit-platform-engine-1.6.3.jar --add-modules com.fasterxml.jackson.databind -jar C:\delombok\delombok.jar delombok C:\Users\alaut\NativeWhatsapp4j\src\main\java -d C:\Users\alaut\NativeWhatsapp4j\target\delombok

[bradleylarrick]

Just so we're on the same page, I'm not looking for help on the pom file. It is configured according to the documentation for the lombok plugin, and if you remove or rename module-info.java, it works fine. Adding module-info.java causes the failure because, it appears, the lombok plugin (and to some extent, the Delombok class) isn't designed to support modules.

Rawi's suggestion of moving all the source code to src/main/java defeats the purpose of the delombok - I don't want to delombok my entire code set (the classes in the example I provided are a small subset of my project).

My current work-around is a hack to the lombok plugin to set sourcepath = sourceDirectory (src/main/lombok), which works because the Delombok class never sees the module-info class (see the command line version of this in the delombok script). This feels like a true hack, though. It seems like the plugin needs to support setting the module-path and the sourcepath for the Delombok class so you can execute the equivalent of the command line:

java -jar lombok-1.18.20.jar delombok src/main/lombok -d target/generated-sources/delombok --module-path $cp --sourcepath src/main/java

But that doesn't work, either. It generates errors like the following:

C:\projects\delombok-example\src\main\lombok\org\larrick\datagen\data\Address.java:7: error: file should be on source path, or on patch path for module

Adding src/main/lombok to the sourcepath argument seems to actually generate delomboked code, but still generates errors like:

C:\projects\delombok-example\src\main\lombok\org\larrick\datagen\data\Address.java:9: error: package com.fasterxml.jackson.annotation is not visible

[Rawi01]

@Auties00 There are two --module-path parameters, one for java and one for delombok and you set the former but I think you need to set the latter. Btw lombok exports the lombok.extern.jackson package since 1.18.18 so I think --add-opens is no longer required.
@bradleylarrick Your example project doesn't compile if I remove the delombok plugin. I still don't get what you want to do and why.

Edit:
I think I should have read the lombok.maven documentation more carefully, I totally missed that the plugin compiles src/main/lombok and add it to the normal build path before compilation.

[Auties00]

@Auties00 There are two --module-path parameters, one for java and one for delombok and you set the former but I think you need to set the latter. Btw lombok exports the lombok.extern.jackson package since 1.18.18 so I think --add-opens is no longer required.
@bradleylarrick Your example project doesn't compile if I remove the delombok plugin.

I've done as you said but some modules are still unresolved.
Command:
java -jar C:\delombok\delombok.jar delombok C:\Users\alaut\NativeWhatsapp4j\src\main\java -d C:\Users\alaut\NativeWhatsapp4j\target\delombok --module-path C:\Users\alaut\NativeWhatsapp4j\target\test-classes;C:\Users\alaut\NativeWhatsapp4j\target\classes;C:\Users\alaut\.m2\repository\org\bouncycastle\bcpkix-jdk15on\1.68\bcpkix-jdk15on-1.68.jar;C:\Users\alaut\.m2\repository\org\bouncycastle\bcprov-jdk15on\1.68\bcprov-jdk15on-1.68.jar;C:\Users\alaut\.m2\repository\org\projectlombok\lombok\1.18.20\lombok-1.18.20.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.validation-api\1.1.1.8-jre16\jakarta.validation-api-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\undertow-websockets-jsr\1.1.1.8-jre16\undertow-websockets-jsr-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\undertow-servlet\1.1.1.8-jre16\undertow-servlet-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\undertow-core\1.1.1.8-jre16\undertow-core-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\undertow-parser-generator\1.1.1.8-jre16\undertow-parser-generator-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.servlet-api\1.1.1.8-jre16\jakarta.servlet-api-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jboss-logmanager\1.1.1.8-jre16\jboss-logmanager-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\jakarta\json\jakarta.json-api\2.0.0\jakarta.json-api-2.0.0.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.websocket-api\1.1.1.8-jre16\jakarta.websocket-api-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.xml.bind-api\1.1.1.8-jre16\jakarta.xml.bind-api-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.activation\1.1.1.8-jre16\jakarta.activation-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\org\codehaus\woodstox\stax2-api\4.2.1\stax2-api-4.2.1.jar;C:\Users\alaut\.m2\repository\org\jvnet\staxex\stax-ex\2.0.0\stax-ex-2.0.0.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\guava\1.1.1.8-jre16\guava-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\guicedee\services\jakarta.annotation-api\1.1.1.8-jre16\jakarta.annotation-api-1.1.1.8-jre16.jar;C:\Users\alaut\.m2\repository\com\fasterxml\jackson\core\jackson-databind\2.12.3\jackson-databind-2.12.3.jar;C:\Users\alaut\.m2\repository\com\fasterxml\jackson\core\jackson-annotations\2.12.3\jackson-annotations-2.12.3.jar;C:\Users\alaut\.m2\repository\com\fasterxml\jackson\core\jackson-core\2.12.3\jackson-core-2.12.3.jar;C:\Users\alaut\.m2\repository\org\junit\jupiter\junit-jupiter\5.6.3\junit-jupiter-5.6.3.jar;C:\Users\alaut\.m2\repository\org\junit\jupiter\junit-jupiter-api\5.6.3\junit-jupiter-api-5.6.3.jar;C:\Users\alaut\.m2\repository\org\opentest4j\opentest4j\1.2.0\opentest4j-1.2.0.jar;C:\Users\alaut\.m2\repository\org\junit\platform\junit-platform-commons\1.6.3\junit-platform-commons-1.6.3.jar;C:\Users\alaut\.m2\repository\org\junit\jupiter\junit-jupiter-params\5.6.3\junit-jupiter-params-5.6.3.jar;C:\Users\alaut\.m2\repository\org\junit\jupiter\junit-jupiter-engine\5.6.3\junit-jupiter-engine-5.6.3.jar;C:\Users\alaut\.m2\repository\org\apiguardian\apiguardian-api\1.1.0\apiguardian-api-1.1.0.jar;C:\Users\alaut\.m2\repository\org\junit\platform\junit-platform-engine\1.6.3\junit-platform-engine-1.6.3.jar
Error:
C:\Users\alaut\NativeWhatsapp4j\src\main\java\module-info.java:14: error: module not found: com.google.common requires com.google.common; ^ C:\Users\alaut\NativeWhatsapp4j\src\main\java\module-info.java:18: error: module not found: jakarta.validation requires jakarta.validation; ^ C:\Users\alaut\NativeWhatsapp4j\src\main\java\module-info.java:19: error: module not found: jakarta.xml.bind requires jakarta.xml.bind;

[bradleylarrick]

@Auties00 That's not my example project.
https://github.com/bradleylarrick/delombok-example.git.

[Auties00]

@Auties00 That's not my example project.
https://github.com/bradleylarrick/delombok-example.git.

Yeah, I'm using a project I'm working on as an example. Using the method that @Rawi01 suggested fixes the issue partially as now most of the modules are correctly detected

[Auties00]

@Auties00 There are two --module-path parameters, one for java and one for delombok and you set the former but I think you need to set the latter. Btw lombok exports the lombok.extern.jackson package since 1.18.18 so I think --add-opens is no longer required.
@bradleylarrick Your example project doesn't compile if I remove the delombok plugin. I still don't get what you want to do and why.

Edit:
I think I should have read the lombok.maven documentation more carefully, I totally missed that the plugin compiles src/main/lombok and add it to the normal build path before compilation.

The issue is getting weird though as java detects all the modules from the module path while lombok misses some. Maybe a similar option to --add-modules should be provided?

[Auties00]

@Auties00 That's not my example project.
https://github.com/bradleylarrick/delombok-example.git.

I've created a temporary fix:
https://github.com/Auties00/delombok-plugin
I'm deploying the plugin right now to maven central to use it in my project, it would be used like this:

            <plugin>
                <groupId>com.github.auties00</groupId>
                <artifactId>delombok-plugin</artifactId>
                <version>1.18.20</version>
                <configuration>
                    <rootDirectory>${project.build.sourceDirectory}</rootDirectory>
                    <outputDirectory>${project.build.directory}/delombok</outputDirectory>
                </configuration>
                <executions>
                    <execution>
                        <phase>process-sources</phase>
                        <goals>
                            <goal>delombok</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

[GedMarc]

Nice fix!

[bradleylarrick]

Your fix doesn't work for me. I referenced your plugin in my project and it delomboked the source files, but it didn't tell the compiler about the new source directory (below is the compiler output):

[INFO] Changes detected - recompiling the module!
[DEBUG] Classpath:
[DEBUG] C:\projects\DataGen\Generator\target\classes
[DEBUG] C:\Users\bradley.m2\repository\com\fasterxml\woodstox\woodstox-core\6.2.4\woodstox-core-6.2.4.jar
[DEBUG] Modulepath:
[DEBUG] C:\Users\bradley.m2\repository\org\projectlombok\lombok\1.18.21\lombok-1.18.21.jar
[DEBUG] C:\Users\bradley.m2\repository\commons-cli\commons-cli\1.4\commons-cli-1.4.jar
[DEBUG] C:\Users\bradley.m2\repository\org\apache\commons\commons-math3\3.6.1\commons-math3-3.6.1.jar
[DEBUG] C:\Users\bradley.m2\repository\com\fasterxml\jackson\core\jackson-annotations\2.12.3\jackson-annotations-2.12.3.jar
[DEBUG] C:\Users\bradley.m2\repository\com\fasterxml\jackson\core\jackson-core\2.12.3\jackson-core-2.12.3.jar
[DEBUG] C:\Users\bradley.m2\repository\com\fasterxml\jackson\core\jackson-databind\2.12.3\jackson-databind-2.12.3.jar
[DEBUG] C:\Users\bradley.m2\repository\com\fasterxml\jackson\dataformat\jackson-dataformat-csv\2.12.3\jackson-dataformat-csv-2.12.3.jar
[DEBUG] C:\Users\bradley.m2\repository\com\fasterxml\jackson\dataformat\jackson-dataformat-xml\2.12.3\jackson-dataformat-xml-2.12.3.jar
[DEBUG] C:\Users\bradley.m2\repository\com\fasterxml\jackson\module\jackson-module-jaxb-annotations\2.12.3\jackson-module-jaxb-annotations-2.12.3.jar
[DEBUG] C:\Users\bradley.m2\repository\jakarta\xml\bind\jakarta.xml.bind-api\2.3.2\jakarta.xml.bind-api-2.3.2.jar
[DEBUG] C:\Users\bradley.m2\repository\jakarta\activation\jakarta.activation-api\1.2.1\jakarta.activation-api-1.2.1.jar
[DEBUG] C:\Users\bradley.m2\repository\org\codehaus\woodstox\stax2-api\4.2.1\stax2-api-4.2.1.jar
[DEBUG] C:\Users\bradley.m2\repository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.12.3\jackson-datatype-jsr310-2.12.3.jar
[DEBUG] Source roots:
[DEBUG] C:\projects\DataGen\Generator\src\main\java
[DEBUG] C:\projects\DataGen\Generator\target\generated-sources\annotations

And I don't see this as a good long-term fix. It forces me to delombok my entire code set, which kind of defeats the purpose of delomboking.

[Rawi01]

@Auties00 Hmm, weird, it should be able to find the modules. I will try to reproduce that one.
@bradleylarrick Can you explain why you actaully need to delombok and why you want to limit it to some specific parts? As far as I know there are three main reasons to use delombok: removing lombok completly (thats obviously not the case here), generating javadocs (still no reason to delombok before compilation) and supporting different ecosystems (e.g. Kotlin). I have absolutly no idea why you try to delombok only a part of the program. Nevertheless I played around with the lombok maven plugin and I think that I found a way to solve your problem.

[bradleylarrick]

@Rawi01 That's a fair question. I want to apologize upfront if I gave the impression that this was an urgent issue for me -- it's not. As I mentioned before I hacked the plugin code to make my scenario work. I started using the lombok plugin in my build process, using the recommended configuration, because it seemed to improve my code coverage metrics. When I modularized my project, it broke the delombok process (as I'm learning, introducing modules breaks a lot of things). Hence the bug report.

Based on my investigation, I feel like the "correct" fix to this problem may include changes to the core Delombok class as well as the plugin (please see my 4 May 21 comment). Unfortunately I don't fully understand how the Delombok class uses the compiler to accomplish it's task so I can't provide any reasonable suggestions.

[Auties00]

t the new source director

you have to specify the new sources using maven's compiler plugin. There is an option named "compileSourceRoots", set that to the outputDirectory of my plugin and it will work

[vishvik]

Hi @bradleylarrick, @Auties00,
I, too, face this issue. Do you mind sharing how you are using Delombok with Java modules now?
I tried the @Auties00 's plugin but it is not working in my project.

[bradleylarrick]

Vishvik,
I'm not using Delombok anymore, so I can't give you any guidance.

[Auties00]

Hi @bradleylarrick, @Auties00,
I, too, face this issue. Do you mind sharing how you are using Delombok with Java modules now?
I tried the @Auties00 's plugin but it is not working in my project.

it's 100% a bad solution, but it works for me. You can look at how i use in my Cobalt library. It's just not as fast as it should be as it delomboks each file separately to avoid the modules problem

[TanHonN]

So is there a real good configuration to solve this problem now? Is it really a problem with lombok which means the lombok can't work with java module system?

[moaxcp]

What is the workaround for this on gradle? I've been using the freefair plugin and ran into the same issue when upgrading to modules. I thought I could remove delombok but it is needed for javadoc and the javadoc jar is required for publishing to maven central. This is a real mess for me.

[GedMarc]

Add lombok dependency to the annotation processor portion of the compiler plug in using
Add dependency to lombok as provided and optional,
In module info, require static lombok

I deploy a lot to maven central with Java docs this way

[moaxcp]

@GedMarc I'm asking about gradle specifically in a multimodule project with module-info.java. Javadoc will fail for missing builder classes and other issues. Delombok needs to run before javadoc but with the freefair gradle plugin delombok fails as described above. I was wondering if there is a similar plugin to what @Auties00 created for maven on gradle.

[GedMarc]

Yes for Javadoc that does happen, Gradle configures a bit differently but the structures, configurations and end result is near identical to Maven -

@moaxcp
For the module dependencies on a Javadoc run -

<plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-javadoc-plugin</artifactId>
                    <version>${maven.javadoc.version}</version>
                    <executions>
                        <execution>
                            <id>javadoc</id>
                            <phase>verify</phase>
                            <goals>
                                <goal>jar</goal>
                            </goals>
                            <configuration>
                                <detectJavaApiLink>true</detectJavaApiLink>
                                <force>true</force>
                                <offlineLinks>
                                    <offlineLink>
                                        <!--suppress UnresolvedMavenProperty -->
                                        <url>https://docs.oracle.com/javase/${jdk.release}/docs/api/</url>
                                        <location>${project.basedir}</location>
                                    </offlineLink>
                                </offlineLinks>
                                <failOnError>true</failOnError>
                                <additionalOptions>
                                    <additionalOption>-Xdoclint:none</additionalOption>
                                </additionalOptions>
                                <sourcepath>target/source-output/</sourcepath>
                                <verbose>false</verbose>
                                <additionalDependencies>
                                    <dependency>
                                        <groupId>com.google.code.findbugs</groupId>
                                        <artifactId>jsr305</artifactId>
                                        <version>${jsr305.version}</version>
                                    </dependency> 

You would want to configure the additional dependencies portion to include the modules that are required during build or source-shading,
In my case as I modularize the dependencies myself, i output the shade source to target/source-output, and then specify that as a source route for javadoc alone. Any additional or missing dependencies I specify in additionalDependencies

The gradle equivalent looks like it would be
apply plugin: 'java'

The issue of adding dependencies for gradles Javadoc’s is by adding the classpath dependency, if not shading sources
javadoc {
source = sourceSets.main.allJava
classpath = configurations.compile
include 'target/source-output/*'
}
(y)

[victorwss]

I think that I understood what is happening.

Today, I got this problem with com.fasterxml.jackson.core and com.fasterxml.jackson.databind, but not with com.fasterxml.jackson.annotation. All of them in version 2.18.1.

Inspecting the JARs' structure I found what is happening: com.fasterxml.jackson.annotation has a standard module-info.class in its root, while for com.fasterxml.jackson.core and com.fasterxml.jackson.databind, they are located in the META-INF.versions.9 folder. This means that the issue is that Delombok can't understand correctly multi-version module JAR files.

[victorwss]

@Rawi01
Is this a duplicate of #3389 (or perhaps, the other way around)?