HADOOP-19725. [JDK17] Upgrade SpotBugs Version to Support JDK 17 Compilation. #8028

slfan1989 · 2025-10-12T23:31:00Z

Description of PR

JIRA: HADOOP-19725. [JDK17] Upgrade SpotBugs Version to Support JDK 17 Compilation.

The error in sputbug is currently caused by incompatibility with JDK 17.

[java]   Error scanning org/apache/hadoop/yarn/proto/YarnServerResourceManagerRecoveryProtos$ApplicationStateDataProto$Builder for referenced classes
     [java]     java.lang.IllegalArgumentException: Unsupported class file major version 61
     [java]       At org.objectweb.asm.ClassReader.<init>(ClassReader.java:189)
     [java]       At org.objectweb.asm.ClassReader.<init>(ClassReader.java:170)
     [java]       At org.objectweb.asm.ClassReader.<init>(ClassReader.java:156)
     [java]       At edu.umd.cs.findbugs.asm.FBClassReader.<init>(FBClassReader.java:35)
     [java]       At edu.umd.cs.findbugs.classfile.engine.asm.ClassReaderAnalysisEngine.analyze(ClassReaderAnalysisEngine.java:48)
     [java]       At edu.umd.cs.findbugs.classfile.engine.asm.ClassReaderAnalysisEngine.analyze(ClassReaderAnalysisEngine.java:34)
     [java]       At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getClassAnalysis(AnalysisCache.java:261)
     [java]       At edu.umd.cs.findbugs.classfile.engine.ClassInfoAnalysisEngine.analyze(ClassInfoAnalysisEngine.java:75)
     [java]       At edu.umd.cs.findbugs.classfile.engine.ClassInfoAnalysisEngine.analyze(ClassInfoAnalysisEngine.java:38)
     [java]       At edu.umd.cs.findbugs.classfile.impl.AnalysisCache.getClassAnalysis(AnalysisCache.java:261)
     [java]       At edu.umd.cs.findbugs.FindBugs2.buildReferencedClassSet(FindBugs2.java:806)
     [java]       At edu.umd.cs.findbugs.FindBugs2.execute(FindBugs2.java:249)
     [java]       At edu.umd.cs.findbugs.FindBugs.runMain(FindBugs.java:395)
     [java]       At edu.umd.cs.findbugs.FindBugs2.main(FindBugs2.java:1231)

We upgraded spotbugs.version from 4.2.2 to 4.8.3, and spotbugs-maven-plugin.version from 4.2.0 to 4.7.3.6.

How was this patch tested?

CI.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

…ilation.

zhtttylz

+1,LGTM @slfan1989

slfan1989 · 2025-10-13T15:24:26Z

@cnauroth @szetszwo After upgrading to JDK 17, I found that spotbug could not run properly because the current version does not support JDK 17. To resolve this issue, I upgraded the versions of the two related plugins. The changes have been tested locally, and the results are as expected.

hadoop-yetus · 2025-10-13T18:01:43Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	14m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 49s		trunk passed
+1 💚	compile	15m 58s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	compile	15m 44s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
-1 ❌	mvnsite	9m 55s	/branch-mvnsite-root.txt	root in trunk failed.
+1 💚	javadoc	9m 22s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	8m 51s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	shadedclient	124m 16s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	28m 30s		the patch passed
+1 💚	compile	15m 17s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javac	15m 17s		the patch passed
+1 💚	compile	15m 43s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	15m 43s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-1 ❌	mvnsite	7m 2s	/patch-mvnsite-root.txt	root in the patch failed.
+1 💚	javadoc	9m 7s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	8m 45s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	shadedclient	53m 32s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	853m 28s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 44s		The patch does not generate ASF License warnings.
		1109m 31s

Reason	Tests
Failed junit tests	hadoop.hdfs.tools.TestDFSAdmin
	hadoop.yarn.sls.appmaster.TestAMSimulator
	hadoop.yarn.server.router.webapp.TestFederationWebApp
	hadoop.yarn.server.router.subcluster.fair.TestYarnFederationWithFairScheduler
	hadoop.yarn.server.router.webapp.TestRouterWebServicesREST
	hadoop.yarn.server.nodemanager.containermanager.logaggregation.TestLogAggregationService

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8028/1/artifact/out/Dockerfile
GITHUB PR	#8028
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname	Linux da08cd8994df 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `3b47bd1`
Default Java	Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8028/1/testReport/
Max. process+thread count	3529 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8028/1/console
versions	git=2.25.1 maven=3.9.11
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2025-10-13T23:56:12Z

I have completed the investigation of the mvnsite build issues and confirmed that the problem is related to the JDIFF module. We plan to submit a separate PR to fix and optimize this issue.

slfan1989 · 2025-10-14T11:14:02Z

@steveloughran Could you please review this PR? Thank you very much!

slfan1989 · 2025-10-15T09:36:42Z

@Hexiaoqiao Could you please review this PR? Thank you very much!

szetszwo

@slfan1989 , thanks for working on this! Please see the comments inlined.

szetszwo · 2025-10-15T15:21:41Z

pom.xml

    <dependency-check-maven.version>7.1.1</dependency-check-maven.version>
-    <spotbugs.version>4.2.2</spotbugs.version>
-    <spotbugs-maven-plugin.version>4.2.0</spotbugs-maven-plugin.version>
+    <spotbugs.version>4.8.3</spotbugs.version>


Why not 4.9.7 (or 4.8.6)?

https://mvnrepository.com/artifact/com.github.spotbugs/spotbugs

Thank you for reviewing! You made a very good point — I’ll update this PR accordingly.

spotbugs 4.9 onwards requires JDK 11+, are we ready to drop Java 8 support on trunk?

currently, trunk GitHub Actions site jobs fail due to this upgrade.

@szetszwo @slfan1989

~~@slfan1989 , we probably should move back to 4.8.x?~~

@pan3793 , Actually, SpotBugs is build tool but not a runtime library. One option is to run SpotBugs code analysis using jdk11 or above. The runtime still can be built with and run with JDK8. Would it work?

Thank you all for the discussion! I believe what @szetszwo mentioned makes sense.

Let me explain the situation:

We have upgraded the JDK version in the trunk to JDK 17 and JDK 21. Regarding the issue of mvn site failing to compile, this is a known issue and is unrelated to the SpotBugs plugin.

Here’s a detailed explanation of the two issues:

The reason for upgrading is that SpotBugs 4.2.0, as indicated by Yetus, does not support scanning for JDK 17 and above. Therefore, we upgraded to a higher version, choosing SpotBugs 4.9.7. From my perspective, upgrading to SpotBugs 4.9.7 is reasonable. Although new warning messages have appeared, I do not plan to roll back the version. Issue after upgrading to SpotBugs 4.9.7: Due to the introduction of new rules, new warning messages appeared during compilation. We have formulated a solution:

Temporary solution: We are filtering out these new warnings globally to mitigate the impact of the new filtering rules. You can refer to the related PR HADOOP-19731. Fix SpotBugs warnings introduced after SpotBugs version upgrade. #8053.

mvn site compilation failure: This issue is related to our custom annotations, which we can see in the error logs. We already have a solution for this:

Referring to PR HADOOP-19402. [JDK11] JDiff Support JDK11. #8038. we have rewritten all the annotations under hadoop-common-project/hadoop-annotations/src/main/java17/org/apache/hadoop/classification/tools/* and added support for JDIFF under JDK 17.

Currently, these two PRs are being followed up by HuaLong. In offline communication, HuaLong mentioned that he is currently on leave, so it may take some more time to complete.

I hope this makes the situation clearer!

szetszwo · 2025-10-15T15:23:00Z

pom.xml

-    <spotbugs.version>4.2.2</spotbugs.version>
-    <spotbugs-maven-plugin.version>4.2.0</spotbugs-maven-plugin.version>
+    <spotbugs.version>4.8.3</spotbugs.version>
+    <spotbugs-maven-plugin.version>4.7.3.6</spotbugs-maven-plugin.version>


Similarly, why not 4.9.7.0 (or 4.8.6.7)?

…ilation.

szetszwo

+1 the change looks good.

steveloughran

LGTM
+1

Hexiaoqiao

Thanks @slfan1989 . LGTM. +1. I think it is smooth after check spotbug release changes and other apache projects upgrade feedbacks. TBH, I am not check it with new JDK version carefully.

hadoop-yetus · 2025-10-16T09:08:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	37m 51s		trunk passed
+1 💚	compile	15m 40s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	compile	15m 43s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
-1 ❌	mvnsite	10m 13s	/branch-mvnsite-root.txt	root in trunk failed.
+1 💚	javadoc	9m 11s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	8m 47s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	shadedclient	125m 4s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	28m 20s		the patch passed
+1 💚	compile	15m 11s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javac	15m 11s		the patch passed
+1 💚	compile	15m 58s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	15m 58s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-1 ❌	mvnsite	7m 5s	/patch-mvnsite-root.txt	root in the patch failed.
+1 💚	javadoc	9m 8s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	8m 40s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	shadedclient	53m 57s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	792m 24s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 46s		The patch does not generate ASF License warnings.
		1035m 29s

Reason	Tests
Failed junit tests	hadoop.hdfs.tools.TestDFSAdmin
	hadoop.yarn.sls.appmaster.TestAMSimulator
	hadoop.yarn.server.router.webapp.TestFederationWebApp
	hadoop.yarn.server.router.subcluster.fair.TestYarnFederationWithFairScheduler
	hadoop.yarn.server.router.webapp.TestRouterWebServicesREST

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8028/2/artifact/out/Dockerfile
GITHUB PR	#8028
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname	Linux c50dc2503f2d 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c7ac33f`
Default Java	Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8028/2/testReport/
Max. process+thread count	3553 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8028/2/console
versions	git=2.25.1 maven=3.9.11
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2025-10-16T14:03:42Z

Thanks @slfan1989 . LGTM. +1. I think it is smooth after check spotbug release changes and other apache projects upgrade feedbacks. TBH, I am not check it with new JDK version carefully.

@Hexiaoqiao Many thanks for reviewing the code! The new Sputbug plugin has been tested on JDK 17 and JDK 21 and works properly.

slfan1989 · 2025-10-16T14:05:41Z

@szetszwo @steveloughran @Hexiaoqiao @zhtttylz Thank you very much for reviewing the code!

HADOOP-19725. [JDK17] Upgrade SpotBugs Version to Support JDK 17 Comp…

3b47bd1

…ilation.

github-actions bot added build trunk labels Oct 12, 2025

slfan1989 requested a review from Hexiaoqiao October 13, 2025 02:38

zhtttylz approved these changes Oct 13, 2025

View reviewed changes

slfan1989 requested review from cnauroth and szetszwo October 13, 2025 15:24

slfan1989 requested a review from steveloughran October 14, 2025 14:41

szetszwo reviewed Oct 15, 2025

View reviewed changes

HADOOP-19725. [JDK17] Upgrade SpotBugs Version to Support JDK 17 Comp…

c7ac33f

…ilation.

szetszwo approved these changes Oct 15, 2025

View reviewed changes

steveloughran approved these changes Oct 15, 2025

View reviewed changes

Hexiaoqiao approved these changes Oct 16, 2025

View reviewed changes

slfan1989 merged commit 0fddaee into apache:trunk Oct 16, 2025
1 of 5 checks passed

slfan1989 mentioned this pull request Oct 20, 2025

MAPREDUCE-7522. Fix SpotBugs issues in hadoop-mapreduce-client-app module. #8040

Open

4 tasks

HADOOP-19725. [JDK17] Upgrade SpotBugs Version to Support JDK 17 Compilation. #8028

HADOOP-19725. [JDK17] Upgrade SpotBugs Version to Support JDK 17 Compilation. #8028

Uh oh!

Conversation

slfan1989 commented Oct 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

Uh oh!

zhtttylz left a comment

Choose a reason for hiding this comment

Uh oh!

slfan1989 commented Oct 13, 2025

Uh oh!

hadoop-yetus commented Oct 13, 2025

Uh oh!

slfan1989 commented Oct 13, 2025

Uh oh!

slfan1989 commented Oct 14, 2025

Uh oh!

slfan1989 commented Oct 15, 2025

Uh oh!

szetszwo left a comment

Choose a reason for hiding this comment

Uh oh!

szetszwo Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

slfan1989 Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

pan3793 Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

szetszwo Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slfan1989 Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

szetszwo Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

szetszwo left a comment

Choose a reason for hiding this comment

Uh oh!

steveloughran left a comment

Choose a reason for hiding this comment

Uh oh!

Hexiaoqiao left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Oct 16, 2025

Uh oh!

slfan1989 commented Oct 16, 2025

Uh oh!

Uh oh!

slfan1989 commented Oct 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

slfan1989 commented Oct 12, 2025 •

edited

Loading

szetszwo Nov 4, 2025 •

edited

Loading

slfan1989 Nov 4, 2025 •

edited

Loading