Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

harmony crash vmState=0x0005ff09 #20536

Open
pshipton opened this issue Nov 7, 2024 · 13 comments
Open

harmony crash vmState=0x0005ff09 #20536

pshipton opened this issue Nov 7, 2024 · 13 comments
Labels
arch:power comp:jit test failure

Comments

@pshipton
Copy link
Member

pshipton commented Nov 7, 2024 •
edited
Loading

Internal build
[AIX] 80 Load_Level_2.harmony.5mins.Mode187
-Xaggressive -Xgcpolicy:gencon -Xjit -Xnocompressedrefs
aix72p9-17

vmState [0x5ff09]: {J9VMSTATE_JIT} {BinaryEncoding}

30x grinder passed

j> 15:25:54 20241107 15:25:53 Runtime State Reporter IMPORTANT: 317993 tests complete, 0 currently running
j> 15:25:54 +0x63c
j> 15:25:54 +0x0
j> 15:25:54 +0x0
j> 15:25:54 Unhandled exception
j> 15:25:54 Type=Segmentation error vmState=0x0005ff09
j> 15:25:54 J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000032
j> 15:25:54 Handler1=F046DC94 Handler2=F02AB4D0
j> 15:25:54 R0=D4E055FC R1=325AA740 R2=303F9B54 R3=31E16040
j> 15:25:54 R4=FFFFFFFB R5=34691334 R6=303EF984 R7=31A145B0
j> 15:25:54 R8=15F50109 R9=15F50109 R10=EF063D40 R11=00000000
j> 15:25:54 R12=D07F0254 R13=3031CDB8 R14=325AD6F8 R15=325AD1B0
j> 15:25:54 R16=00000000 R17=D500B564 R18=00000000 R19=000001B4
j> 15:25:54 R20=000001B4 R21=EF0C05D0 R22=463B4D00 R23=EF0C02EC
j> 15:25:54 R24=EF0C0330 R25=FFFFFFFB R26=4A2FD9FC R27=31E16040
j> 15:25:54 R28=34691334 R29=31E1CF50 R30=31E1CF50 R31=31E16040
j> 15:25:54 IAR=D4DEAB18 LR=D4E055FC MSR=0000D032 CTR=D4DEAB00
j> 15:25:54 CR=4280288B FPSCR=82004000 XER=00000001 TID=00000000
j> 15:25:54 MQ=00000000
j> 15:25:54 FPR0=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR1=40a8000000000000 (f: 0.000000, d: 3.072000e+03)
j> 15:25:54 FPR2=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR3=404a800000000000 (f: 0.000000, d: 5.300000e+01)
j> 15:25:54 FPR4=41efffffffe00000 (f: 4292870144.000000, d: 4.294967e+09)
j> 15:25:54 FPR5=3fb99999a0000000 (f: 2684354560.000000, d: 1.000000e-01)
j> 15:25:54 FPR6=0000000000000001 (f: 1.000000, d: 4.940656e-324)
j> 15:25:54 FPR7=4530000000000000 (f: 0.000000, d: 1.934281e+25)
j> 15:25:54 FPR8=3ef218c5236ad745 (f: 594204480.000000, d: 1.725841e-05)
j> 15:25:54 FPR9=bf7d671ec3dcfba3 (f: 3286039552.000000, d: -7.178421e-03)
j> 15:25:54 FPR10=3f4c83068f617754 (f: 2405529344.000000, d: 8.701117e-04)
j> 15:25:54 FPR11=3fb436e526a7ec68 (f: 648539264.000000, d: 7.896263e-02)
j> 15:25:54 FPR12=3fff535660000000 (f: 1610612736.000000, d: 1.957846e+00)
j> 15:25:54 FPR13=3fc3ae44529ba88c (f: 1385932928.000000, d: 1.537557e-01)
j> 15:25:54 FPR14=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR15=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR16=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR17=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR18=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR19=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR20=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR21=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR22=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR23=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR24=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR25=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR26=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR27=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR28=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR29=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR30=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 FPR31=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 15:25:54 
j> 15:25:54 Method_being_compiled=sun/reflect/MethodAccessorGenerator.emitInvoke()V
j> 15:25:54 Target=2_90_20241107_81198 (AIX 7.2)
j> 15:25:54 CPU=ppc (32 logical CPUs) (0x100000000 RAM)
j> 15:25:54 ----------- Stack Backtrace -----------
j> 15:25:54 (0xD4E0543C [libj9jit29.so+0x103643c])
j> 15:25:54 (0xD4E052BC [libj9jit29.so+0x10362bc])
j> 15:25:54 (0xD3F4B21C [libj9jit29.so+0x17c21c])
j> 15:25:54 (0xD4F86CF4 [libj9jit29.so+0x11b7cf4])
j> 15:25:54 (0xD4F8667C [libj9jit29.so+0x11b767c])
j> 15:25:54 (0xD4F6B03C [libj9jit29.so+0x119c03c])
j> 15:25:54 (0xD4F6A948 [libj9jit29.so+0x119b948])
j> 15:25:54 (0xD4F699D0 [libj9jit29.so+0x119a9d0])
j> 15:25:54 (0xD4F66840 [libj9jit29.so+0x1197840])
j> 15:25:54 (0xD4F60714 [libj9jit29.so+0x1191714])
j> 15:25:54 (0xD0E8C434 [libj9prt29.so+0x56434])
j> 15:25:54 (0xD4F4DEC8 [libj9jit29.so+0x117eec8])
j> 15:25:54 (0xD4F4BFD8 [libj9jit29.so+0x117cfd8])
j> 15:25:54 (0xD4FBFEBC [libj9jit29.so+0x11f0ebc])
j> 15:25:54 (0xD4FBF920 [libj9jit29.so+0x11f0920])
j> 15:25:54 (0xD0E8C434 [libj9prt29.so+0x56434])
j> 15:25:54 (0xD4FBF02C [libj9jit29.so+0x11f002c])
j> 15:25:54 (0xD07EC5CC [libj9thr29.so+0x45cc])
j> 15:25:54 _pthread_body+0xe4 (0xD0579FC8 [libpthreads.a+0x3fc8])
j> 15:25:54 ---------------------------------------
@pshipton
Copy link
Member Author

pshipton commented Nov 7, 2024

@hzongaro pls take a look

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 7, 2024

Issue Number: 20536
Status: Open
Recommended Components: comp:vm, comp:gc, comp:test
Recommended Assignees: chengjin01, pshipton, babsingh

@hzongaro
Copy link
Member

hzongaro commented Nov 7, 2024

@zl-wang, could someone on the Power team take a look at this?

@zl-wang
Copy link
Contributor

zl-wang commented Nov 7, 2024

@IBMJimmyk please take a look ...

@IBMJimmyk
Copy link
Contributor

I can take a look.

@zl-wang
Copy link
Contributor

zl-wang commented Nov 8, 2024

similar issue with similar crash on LinuxBE: #20546

@IBMJimmyk
Copy link
Contributor

Regarding internal build: http://vmfarm.rtp.raleigh.ibm.com/job_output.php?id=95560368

Was the corefile, javacore and jitdump uploaded anywhere? The output says they were generated but I don't see a download link on the failing test.

@pshipton
Copy link
Member Author

pshipton commented Nov 8, 2024

It would be in http://vmfarm.rtp.raleigh.ibm.com/etc/cores/j9vm/ but I don't see it so I guess something went wrong.

@IBMJimmyk
Copy link
Contributor

That's unfortunate. The LinuxBE issue Julian linked to seems to have corefiles uploaded so I will try looking at those.

@pshipton
Copy link
Member Author

pshipton commented Nov 11, 2024 •
edited
Loading

Maybe this has the same cause? Still no diagnostics though. Opened an issue automation/issues/116.
Internal build
[AIX] 80 Load_Level_2.harmony.5mins.Mode112 -Xgcpolicy:gencon -Xjit:count=0 -Xnocompressedrefs

30x grinder 6/30 failed.

j> 16:19:31 #INFO:  No threads were activated following a resume all compilation threads call
j> 16:19:32 Unhandled exception
j> 16:19:32 Type=Segmentation error vmState=0x00020011
j> 16:19:32 J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000032
j> 16:19:32 Handler1=F0B6DC94 Handler2=F0B1D4D0
j> 16:19:32 R0=D6DC946C R1=35E24570 R2=303F98C4 R3=48D88620
j> 16:19:32 R4=FFFFFFFD R5=34D049E4 R6=303EF6F4 R7=00000001
j> 16:19:32 R8=00004C1D R9=15C20393 R10=00000004 R11=00000000
j> 16:19:32 R12=D6BDE6BC R13=A8183F71 R14=F0980C80 R15=30258174
j> 16:19:32 R16=303F9768 R17=00000000 R18=00000FE4 R19=301AC9C4
j> 16:19:32 R20=301AC9C4 R21=301AA040 R22=301AA03C R23=000029B8
j> 16:19:32 R24=45B4D270 R25=FFFFFFFD R26=47D33D48 R27=00000310
j> 16:19:32 R28=000000C4 R29=00000000 R30=302E0328 R31=48D88620
j> 16:19:32 IAR=D6BDE618 LR=D6DC946C MSR=0200D032 CTR=D6BDE600
j> 16:19:32 CR=4200022B FPSCR=82024000 XER=40000000 TID=00000000
j> 16:19:32 MQ=00000000
j> 16:19:32 FPR0=0000000082024000 (f: 2181185536.000000, d: 1.077649e-314)
j> 16:19:32 FPR1=4330080000000000 (f: 0.000000, d: 4.512396e+15)
j> 16:19:32 FPR2=41f0000000000000 (f: 0.000000, d: 4.294967e+09)
j> 16:19:32 FPR3=4330080000000000 (f: 0.000000, d: 4.512396e+15)
j> 16:19:32 FPR4=3df0000000000000 (f: 0.000000, d: 2.328306e-10)
j> 16:19:32 FPR5=4530000000000000 (f: 0.000000, d: 1.934281e+25)
j> 16:19:32 FPR6=0000000000000001 (f: 1.000000, d: 4.940656e-324)
j> 16:19:32 FPR7=4530000000000000 (f: 0.000000, d: 1.934281e+25)
j> 16:19:32 FPR8=be4a770bf9c1816f (f: 4190208256.000000, d: -1.232374e-08)
j> 16:19:32 FPR9=bf2d19e9583c1400 (f: 1480332288.000000, d: -2.220247e-04)
j> 16:19:32 FPR10=4024fc19da277be6 (f: 3660020736.000000, d: 1.049238e+01)
j> 16:19:32 FPR11=3fd55555555450ef (f: 1431589120.000000, d: 3.333333e-01)
j> 16:19:32 FPR12=3c7abc9e3b39803f (f: 993624128.000000, d: 2.319047e-17)
j> 16:19:32 FPR13=4008000000000000 (f: 0.000000, d: 3.000000e+00)
j> 16:19:32 FPR14=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR15=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR16=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR17=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR18=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR19=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR20=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR21=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR22=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR23=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR24=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR25=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR26=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR27=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR28=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR29=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR30=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 FPR31=0000000000000000 (f: 0.000000, d: 0.000000e+00)
j> 16:19:32 Target=2_90_20241111_81426 (AIX 7.2)
j> 16:19:32 CPU=ppc (32 logical CPUs) (0x100000000 RAM)
j> 16:19:32 ----------- Stack Backtrace -----------
j> 16:19:32 (0xD6DC9124 [libj9jit29.so+0x1208124])
j> 16:19:32 (0xD6DDADDC [libj9jit29.so+0x1219ddc])
j> 16:19:32 (0xD6DDA5A4 [libj9jit29.so+0x12195a4])
j> 16:19:32 (0xD0825384 [libj9hookable29.so+0x384])
j> 16:19:32 (0xD59DE2F8 [libj9vm29.so+0x8d2f8])
j> 16:19:32 (0xD71B2DF0 [libj9gc29.so+0x11df0])
j> 16:19:32 (0xD7235E40 [libj9gc29.so+0x94e40])
j> 16:19:32 (0xD72359A8 [libj9gc29.so+0x949a8])
j> 16:19:32 (0xD722C690 [libj9gc29.so+0x8b690])
j> 16:19:32 (0xD72948F8 [libj9gc29.so+0xf38f8])
j> 16:19:32 (0xD721D838 [libj9gc29.so+0x7c838])
j> 16:19:32 (0xD71D6A0C [libj9gc29.so+0x35a0c])
j> 16:19:32 (0xD71D65A0 [libj9gc29.so+0x355a0])
j> 16:19:32 (0xD737F49C [libj9gc29.so+0x1de49c])
j> 16:19:32 (0xD737EF68 [libj9gc29.so+0x1ddf68])
j> 16:19:32 (0xD721D838 [libj9gc29.so+0x7c838])
j> 16:19:32 (0xD735C320 [libj9gc29.so+0x1bb320])
j> 16:19:32 (0xD7338BAC [libj9gc29.so+0x197bac])
j> 16:19:32 (0xD7341D0C [libj9gc29.so+0x1a0d0c])
j> 16:19:32 (0xD73425C4 [libj9gc29.so+0x1a15c4])
j> 16:19:32 (0xD7340434 [libj9gc29.so+0x19f434])
j> 16:19:32 (0xD72248FC [libj9gc29.so+0x838fc])
j> 16:19:32 (0xD7221B84 [libj9gc29.so+0x80b84])
j> 16:19:32 (0xD5C85A24 [libj9jit29.so+0xc4a24])
j> 16:19:32 (0xD5C7BE04 [libj9jit29.so+0xbae04])
j> 16:19:32 (0xD5989C50 [libj9vm29.so+0x38c50])
j> 16:19:32 (0xD5972AE4 [libj9vm29.so+0x21ae4])
j> 16:19:32 (0xD2A37434 [libj9prt29.so+0x56434])
j> 16:19:32 (0xD5972920 [libj9vm29.so+0x21920])
j> 16:19:32 (0xD29C95CC [libj9thr29.so+0x45cc])
j> 16:19:32 _pthread_body+0xe4 (0xD0579FC8 [libpthreads.a+0x3fc8])
j> 16:19:32 ---------------------------------------

@pshipton pshipton mentioned this issue Nov 11, 2024
Closed
@pshipton
Copy link
Member Author

Opened #20567 for the previous comment, it seems a different issue.

@pshipton
Copy link
Member Author

There is an AIX core file saved, see https://github.ibm.com/runtimes/automation/issues/116#issuecomment-97015346 and let me know if you need help to get it.

@pshipton
Copy link
Member Author

AIX core files can be found under http://vmfarm.rtp.raleigh.ibm.com/etc/cores/tmp/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch:power comp:jit test failure
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@pshipton @hzongaro @zl-wang @IBMJimmyk