Skip to content

java.lang.OutOfMemoryError: Java heap space on test-mnx-ubuntu2204-x64-2 #4538

Closed

Description

@richardlau

Two node-cross-compile runs have failed on test-mnx-ubuntu2204-x64-2 with OutOfMemoryError during git checkout:

e.g.

12:07:42  > git checkout -f c7dfcaba79481d29170b9d48f409e89197787ba4 # timeout=10
12:08:04 FATAL: Self-suppression not permitted
12:08:04 java.lang.OutOfMemoryError: Java heap space
12:08:04 Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 8.225.232.44/8.225.232.44:43584
12:08:04 		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1916)
12:08:04 		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:384)
12:08:04 		at hudson.remoting.Channel.call(Channel.java:1108)
12:08:04 		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:306)
12:08:04 		at PluginClassLoader for git-client/jdk.proxy72/jdk.proxy72.$Proxy147.withRepository(Unknown Source)
12:08:04 		at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.RemoteGitImpl.withRepository(RemoteGitImpl.java:264)
12:08:04 		at PluginClassLoader for git//hudson.plugins.git.GitSCM.printCommitMessageToLog(GitSCM.java:1428)
12:08:04 		at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1368)
12:08:04 		at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277)
12:08:04 		at hudson.scm.SCM.checkout(SCM.java:540)
12:08:04 		at hudson.model.AbstractProject.checkout(AbstractProject.java:1254)
12:08:04 		at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:648)
12:08:04 		at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:84)
12:08:04 		at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:521)
12:08:04 		at hudson.model.Run.execute(Run.java:1849)
12:08:04 		at PluginClassLoader for matrix-project//hudson.matrix.MatrixBuild.run(MatrixBuild.java:331)
12:08:04 		at hudson.model.ResourceController.execute(ResourceController.java:97)
12:08:04 		at hudson.model.Executor.run(Executor.java:456)
12:08:04 Caused: java.lang.IllegalArgumentException: Self-suppression not permitted
12:08:04 	at java.base/java.lang.Throwable.addSuppressed(Throwable.java:1101)
12:08:04 	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.withRepository(AbstractGitAPIImpl.java:27)
12:08:04 	at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.withRepository(CliGitAPIImpl.java:83)
12:08:04 	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
12:08:04 	at java.base/java.lang.reflect.Method.invoke(Method.java:565)
12:08:04 	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:986)
12:08:04 	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:961)
12:08:04 	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:907)
12:08:04 	at hudson.remoting.UserRequest.perform(UserRequest.java:225)
12:08:04 	at hudson.remoting.UserRequest.perform(UserRequest.java:50)
12:08:04 	at hudson.remoting.Request$2.run(Request.java:391)
12:08:04 	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:81)
12:08:04 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:328)
12:08:04 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
12:08:04 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
12:08:04 	at hudson.remoting.Engine$1.lambda$newThread$0(Engine.java:312)
12:08:04 	at java.base/java.lang.Thread.run(Thread.java:1474)

It's not consistent though -- other builds on the same machine in between those two succeeded.

Image

Activity

  1. sxa commented on Oct 22, 2026

    @sxa
    Member

    I'm quite surprised that 128m of heap isn't enough but just to note a few things:

    • The JVM version on this machine is 25.0.2 which is a few patch levels behind (25.0.5 is the latest)
    • The JVM has been up since June, which is generally not a problem but if there's a memory leak somewhere that might be why we're seeing it now
    • There was a PR in to update the heap size on these machines to 512m so this feels like an opportunity to apply it.
      Or just the scattergun approach of "all of the above"
  2. richardlau commented on Oct 22, 2026

    @richardlau
    MemberAuthor

    Been discussing with @sxa. Our plan is:

    • Run the Ansible scripts on the already offline in Jenkins test-mnx-ubuntu2204-x64-1. That will pick up ansible: raise jenkins agent heap on mnx x64 workspace machines #4493 to increase the heap size there and update to the most recent patch level for Java 25.
    • Bring test-mnx-ubuntu2204-x64-1 back online in Jenkins and take test-mnx-ubuntu2204-x64-2 offline.
    • Wait for test-mnx-ubuntu2204-x64-2 to quiesce and then run the Ansible scripts against it before bringing it back online.
  3. richardlau commented on Oct 22, 2026

    @richardlau
    MemberAuthor

    test-mnx-ubuntu2204-x64-1 has been updated and is now running Java 25.0.4. I had to run

    systemctl daemon-reload
    

    and then restart the agent to pick up the change from #4493 -- the Ansible playbook restarted the agent, but it did not pick up the changed parameters without the daemon-reload.

  4. richardlau commented on Oct 23, 2026

    @richardlau
    MemberAuthor

    Both mnx jenkins-workspace machines have been updated (via the current Ansible playbook on main) to the most recent JDK 25 patch release and to increase the max heap size.

  5. richardlau commented on Oct 27, 2026

    @richardlau
    MemberAuthor

    Since the updates/restarts we haven't run into this again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions