Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Since NodeJS 20.x on MacOS 15, address sanitizer fails on all tests. #56154

Open
viferga opened this issue Dec 6, 2024 · 1 comment
Open

Comments

@viferga
Copy link

viferga commented Dec 6, 2024

Version

20.x and superior

Platform

MacOS 15 (specifically on GitHub Actions `macos-15`) and ARM64

Subsystem

C++ GC Oilpan (V8)

What steps will reproduce the bug?

The steps to reproduce are difficult to describe because I am facing this in my project while embedding NodeJS: metacall/core#530

Basically this happens when compiling NodeJS with --shared, --debug and --enable-asan, here are the options of my build system:
https://github.com/metacall/core/blob/433e3107112b6d8362aa03057dae20b96ff5b1a8/cmake/FindNodeJS.cmake#L565
https://github.com/metacall/core/blob/433e3107112b6d8362aa03057dae20b96ff5b1a8/cmake/FindNodeJS.cmake#L568
https://github.com/metacall/core/blob/433e3107112b6d8362aa03057dae20b96ff5b1a8/cmake/FindNodeJS.cmake#L578

It produces the following stack trace (this stack trace is provided by metacall itself, not by NodeJS):

2024-11-06T04:19:05.3574440Z Stack trace (most recent call last) in thread 6171357184:
2024-11-06T04:19:05.3575040Z #12   Object "libsystem_pthread.dylib", at 0x195d872e3, in _pthread_start + 135
2024-11-06T04:19:05.3575720Z #11   Object "libclang_rt.asan_osx_dynamic.dylib", at 0x1048c585b, in asan_thread_start(void*) + 67
2024-11-06T04:19:05.3576420Z #10   Object "libnode_loaderd.so", at 0x107d3b163, in node_loader_impl_thread(void*) + 8199
2024-11-06T04:19:05.3577090Z #9    Object "libnode.127.dylib", at 0x11c464e5f, in node::Start(int, char**) + 1055
2024-11-06T04:19:05.3577740Z #8    Object "libnode.127.dylib", at 0x11c4653df, in node::StartInternal(int, char**) + 651
2024-11-06T04:19:05.3579200Z #7    Object "libnode.127.dylib", at 0x11c4618a3, in node::InitializeOncePerProcessInternal(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, node::ProcessInitializationFlags::Flags) + 5159
2024-11-06T04:19:05.3580730Z #6    Object "libnode.127.dylib", at 0x121d88167, in cppgc::InitializeProcess(v8::PageAllocator*, unsigned long) + 191
2024-11-06T04:19:05.3581450Z #5    Object "libsystem_platform.dylib", at 0x195dbc183, in _sigtramp + 55
2024-11-06T04:19:05.3582190Z #4    Object "libbacktrace_plugind.so", at 0x1074316ab, in backward::SignalHandling::sig_handler(int, __siginfo*, void*) + 55
2024-11-06T04:19:05.3583100Z #3    Object "libbacktrace_plugind.so", at 0x10743263f, in backward::SignalHandling::handleSignal(int, __siginfo*, void*) + 1187
2024-11-06T04:19:05.3584150Z #2    Object "libbacktrace_plugind.so", at 0x107432cab, in backward::StackTraceImpl<backward::system_tag::darwin_tag>::load_from(void*, unsigned long, void*, void*) + 415
2024-11-06T04:19:05.3585330Z #1    Object "libbacktrace_plugind.so", at 0x10743355f, in backward::StackTraceImpl<backward::system_tag::darwin_tag>::load_here(unsigned long, void*, void*) + 839
2024-11-06T04:19:05.3586770Z #0    Object "libbacktrace_plugind.so", at 0x10743ffb3, in unsigned long backward::details::unwind<backward::StackTraceImpl<backward::system_tag::darwin_tag>::callback>(backward::StackTraceImpl<backward::system_tag::darwin_tag>::callback, unsigned long) + 403

As you can see, after node::InitializeOncePerProcessInternal it initializes Oilpan with cppgc::InitializeProcess and then generates a segmentation fault. I am not sure if the problem comes from NodeJS itself or maybe V8, or a wrong configuration from my build system.

I have another CI run which fails in a different part, while compiling, I am not sure if it may be related, probably not. It may be worth it to review the issue, at least try to resolve most common issues with address sanitizer on MacOS 15 and ARM64:
job-logs.txt
https://github.com/metacall/core/actions/runs/11689087116/job/32551061003

2024-11-05T18:02:36.1975490Z ==80728==ERROR: AddressSanitizer: container-overflow on address 0x62d007874358 at pc 0x0001039ad7d4 bp 0x00016fd1c410 sp 0x00016fd1c408
2024-11-05T18:02:36.1977390Z READ of size 8 at 0x62d007874358 thread T0
2024-11-05T18:02:58.9705660Z     #0 0x1039ad7d0 in v8::internal::compiler::ControlEquivalence::RunUndirectedDFS(v8::internal::compiler::Node*) control-equivalence.cc:161
2024-11-05T18:02:58.9711730Z     #1 0x104030730 in v8::internal::compiler::CFGBuilder::Run(v8::internal::compiler::BasicBlock*, v8::internal::compiler::Node*) scheduler.cc:282
2024-11-05T18:02:58.9712830Z     #2 0x10402f384 in v8::internal::compiler::Scheduler::FuseFloatingControl(v8::internal::compiler::BasicBlock*, v8::internal::compiler::Node*) scheduler.cc:1894
2024-11-05T18:02:58.9714210Z     #3 0x1040476b0 in v8::internal::compiler::ScheduleLateNodeVisitor::ProcessQueue(v8::internal::compiler::Node*) scheduler.cc:1528
2024-11-05T18:02:58.9715110Z     #4 0x104027d20 in v8::internal::compiler::Scheduler::ScheduleLate() scheduler.cc:1855
2024-11-05T18:02:58.9716310Z     #5 0x104026130 in v8::internal::compiler::Scheduler::ComputeSchedule(v8::internal::Zone*, v8::internal::compiler::Graph*, v8::base::Flags<v8::internal::compiler::Scheduler::Flag, int>, v8::internal::TickCounter*, v8::internal::ProfileDataFromFile const*) scheduler.cc:70
2024-11-05T18:02:58.9718610Z     #6 0x103fbf14c in auto v8::internal::compiler::PipelineImpl::Run<v8::internal::compiler::ComputeSchedulePhase>() pipeline.cc:1367
2024-11-05T18:02:58.9720850Z     #7 0x103faeea8 in v8::internal::compiler::Pipeline::GenerateCodeForCodeStub(v8::internal::Isolate*, v8::internal::compiler::CallDescriptor*, v8::internal::compiler::Graph*, v8::internal::compiler::JSGraph*, v8::internal::compiler::SourcePositionTable*, v8::internal::CodeKind, char const*, v8::internal::Builtin, v8::internal::AssemblerOptions const&, v8::internal::ProfileDataFromFile const*) pipeline.cc:3240
2024-11-05T18:02:58.9723690Z     #8 0x10392c2ec in v8::internal::compiler::CodeAssembler::GenerateCode(v8::internal::compiler::CodeAssemblerState*, v8::internal::AssemblerOptions const&, v8::internal::ProfileDataFromFile const*) code-assembler.cc:175
2024-11-05T18:02:58.9725630Z     #9 0x1045bfac0 in v8::internal::(anonymous namespace)::BuildWithCodeStubAssemblerCS(v8::internal::Isolate*, v8::internal::Builtin, void (*)(v8::internal::compiler::CodeAssemblerState*), v8::internal::CallDescriptors::Key, char const*) setup-builtins-internal.cc:203
2024-11-05T18:02:58.9727130Z     #10 0x1045b2370 in v8::internal::SetupIsolateDelegate::SetupBuiltinsInternal(v8::internal::Isolate*) setup-builtins-internal.cc:353
2024-11-05T18:02:58.9728100Z     #11 0x1028f25fc in v8::internal::SetupIsolateDelegate::SetupBuiltins(v8::internal::Isolate*, bool) setup-isolate-full.cc:29
2024-11-05T18:02:58.9729160Z     #12 0x10099a8d4 in v8::internal::Isolate::Init(v8::internal::SnapshotData*, v8::internal::SnapshotData*, v8::internal::SnapshotData*, bool) isolate.cc:4446
2024-11-05T18:02:58.9730060Z     #13 0x100998e98 in v8::internal::Isolate::InitWithoutSnapshot() isolate.cc:4000
2024-11-05T18:02:58.9730800Z     #14 0x1000faccc in v8::SnapshotCreator::SnapshotCreator(v8::Isolate*, long const*, v8::StartupData const*) api.cc:565
2024-11-05T18:02:58.9736920Z     #15 0x101fa4b24 in v8::internal::CreateSnapshotDataBlobInternal(v8::SnapshotCreator::FunctionCodeHandling, char const*, v8::Isolate*) snapshot.cc:754
2024-11-05T18:02:58.9737810Z     #16 0x1000d37bc in main mksnapshot.cc:288
2024-11-05T18:02:58.9738150Z     #17 0x1848f8270  (<unknown module>)
2024-11-05T18:02:58.9738390Z 
2024-11-05T18:02:58.9738910Z 0x62d007874358 is located 24408 bytes inside of 32768-byte region [0x62d00786e400,0x62d007876400)
2024-11-05T18:02:58.9739740Z allocated by thread T0 here:
2024-11-05T18:02:59.0123980Z     #0 0x10d0ccc04 in malloc+0x94 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x54c04)
2024-11-05T18:02:59.0124790Z     #1 0x101ffdb4c in v8::internal::AllocAtLeastWithRetry(unsigned long) allocation.cc:135
2024-11-05T18:02:59.0126550Z     #2 0x10200e194 in v8::internal::AccountingAllocator::AllocateSegment(unsigned long, bool) accounting-allocator.cc:94
2024-11-05T18:02:59.0127330Z     #3 0x10200f338 in v8::internal::Zone::Expand(unsigned long) zone.cc:164
2024-11-05T18:02:59.0128410Z     #4 0x10200ef9c in v8::internal::Zone::AsanNew(unsigned long) zone.cc:55
2024-11-05T18:02:59.0129540Z     #5 0x103684a70 in std::__1::deque<v8::internal::compiler::Node*, v8::internal::RecyclingZoneAllocator<v8::internal::compiler::Node*>>::__add_back_capacity() deque:2163
2024-11-05T18:02:59.0132240Z     #6 0x10404726c in v8::internal::compiler::ScheduleLateNodeVisitor::ProcessQueue(v8::internal::compiler::Node*) scheduler.cc:1523
2024-11-05T18:02:59.0134610Z     #7 0x104027d20 in v8::internal::compiler::Scheduler::ScheduleLate() scheduler.cc:1855
2024-11-05T18:02:59.0163080Z     #8 0x104026130 in v8::internal::compiler::Scheduler::ComputeSchedule(v8::internal::Zone*, v8::internal::compiler::Graph*, v8::base::Flags<v8::internal::compiler::Scheduler::Flag, int>, v8::internal::TickCounter*, v8::internal::ProfileDataFromFile const*) scheduler.cc:70
2024-11-05T18:02:59.0167450Z     #9 0x103fbf14c in auto v8::internal::compiler::PipelineImpl::Run<v8::internal::compiler::ComputeSchedulePhase>() pipeline.cc:1367
2024-11-05T18:02:59.0173320Z     #10 0x103faeea8 in v8::internal::compiler::Pipeline::GenerateCodeForCodeStub(v8::internal::Isolate*, v8::internal::compiler::CallDescriptor*, v8::internal::compiler::Graph*, v8::internal::compiler::JSGraph*, v8::internal::compiler::SourcePositionTable*, v8::internal::CodeKind, char const*, v8::internal::Builtin, v8::internal::AssemblerOptions const&, v8::internal::ProfileDataFromFile const*) pipeline.cc:3240
2024-11-05T18:02:59.0178650Z     #11 0x10392c2ec in v8::internal::compiler::CodeAssembler::GenerateCode(v8::internal::compiler::CodeAssemblerState*, v8::internal::AssemblerOptions const&, v8::internal::ProfileDataFromFile const*) code-assembler.cc:175
2024-11-05T18:02:59.0180500Z     #12 0x1045bfac0 in v8::internal::(anonymous namespace)::BuildWithCodeStubAssemblerCS(v8::internal::Isolate*, v8::internal::Builtin, void (*)(v8::internal::compiler::CodeAssemblerState*), v8::internal::CallDescriptors::Key, char const*) setup-builtins-internal.cc:203
2024-11-05T18:02:59.0181970Z     #13 0x1045b2370 in v8::internal::SetupIsolateDelegate::SetupBuiltinsInternal(v8::internal::Isolate*) setup-builtins-internal.cc:353
2024-11-05T18:02:59.0182960Z     #14 0x1028f25fc in v8::internal::SetupIsolateDelegate::SetupBuiltins(v8::internal::Isolate*, bool) setup-isolate-full.cc:29
2024-11-05T18:02:59.0184000Z     #15 0x10099a8d4 in v8::internal::Isolate::Init(v8::internal::SnapshotData*, v8::internal::SnapshotData*, v8::internal::SnapshotData*, bool) isolate.cc:4446
2024-11-05T18:02:59.0184820Z     #16 0x100998e98 in v8::internal::Isolate::InitWithoutSnapshot() isolate.cc:4000
2024-11-05T18:02:59.0185600Z     #17 0x1000faccc in v8::SnapshotCreator::SnapshotCreator(v8::Isolate*, long const*, v8::StartupData const*) api.cc:565
2024-11-05T18:02:59.0186710Z     #18 0x101fa4b24 in v8::internal::CreateSnapshotDataBlobInternal(v8::SnapshotCreator::FunctionCodeHandling, char const*, v8::Isolate*) snapshot.cc:754
2024-11-05T18:02:59.0187580Z     #19 0x1000d37bc in main mksnapshot.cc:288
2024-11-05T18:02:59.0188440Z     #20 0x1848f8270  (<unknown module>)
2024-11-05T18:02:59.0188680Z 
2024-11-05T18:02:59.0189160Z HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
2024-11-05T18:02:59.0190310Z If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
2024-11-05T18:02:59.0191990Z SUMMARY: AddressSanitizer: container-overflow control-equivalence.cc:161 in v8::internal::compiler::ControlEquivalence::RunUndirectedDFS(v8::internal::compiler::Node*)
2024-11-05T18:02:59.0193030Z Shadow bytes around the buggy address:
2024-11-05T18:02:59.0203410Z   0x62d007874080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2024-11-05T18:02:59.0204660Z   0x62d007874100: 00 00 00 00 00 00 f7 f7 f7 00 00 00 00 00 00 f7
2024-11-05T18:02:59.0205930Z   0x62d007874180: f7 f7 00 00 00 00 00 00 f7 f7 f7 00 00 00 00 00
2024-11-05T18:02:59.0207230Z   0x62d007874200: 00 f7 f7 f7 00 00 00 00 00 00 f7 f7 f7 00 f7 f7
2024-11-05T18:02:59.0209110Z   0x62d007874280: f7 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
2024-11-05T18:02:59.0210640Z =>0x62d007874300: 00 00 00 00 00 00 fc fc fc fc fc[fc]fc fc fc fc
2024-11-05T18:02:59.0212010Z   0x62d007874380: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
2024-11-05T18:02:59.0213260Z   0x62d007874400: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
2024-11-05T18:02:59.0214680Z   0x62d007874480: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
2024-11-05T18:02:59.0216080Z   0x62d007874500: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
2024-11-05T18:02:59.0217360Z   0x62d007874580: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
2024-11-05T18:02:59.0218750Z Shadow byte legend (one shadow byte represents 8 application bytes):
2024-11-05T18:02:59.0220000Z   Addressable:           00
2024-11-05T18:02:59.0220820Z   Partially addressable: 01 02 03 04 05 06 07 
2024-11-05T18:02:59.0221770Z   Heap left redzone:       fa
2024-11-05T18:02:59.0222610Z   Freed heap region:       fd
2024-11-05T18:02:59.0223700Z   Stack left redzone:      f1
2024-11-05T18:02:59.0224470Z   Stack mid redzone:       f2
2024-11-05T18:02:59.0225320Z   Stack right redzone:     f3
2024-11-05T18:02:59.0226090Z   Stack after return:      f5
2024-11-05T18:02:59.0226810Z   Stack use after scope:   f8
2024-11-05T18:02:59.0227610Z   Global redzone:          f9
2024-11-05T18:02:59.0228530Z   Global init order:       f6
2024-11-05T18:02:59.0229230Z   Poisoned by user:        f7
2024-11-05T18:02:59.0230000Z   Container overflow:      fc
2024-11-05T18:02:59.0230700Z   Array cookie:            ac
2024-11-05T18:02:59.0231430Z   Intra object redzone:    bb
2024-11-05T18:02:59.0232240Z   ASan internal:           fe
2024-11-05T18:02:59.0233010Z   Left alloca redzone:     ca
2024-11-05T18:02:59.0233890Z   Right alloca redzone:    cb
2024-11-05T18:02:59.0295230Z ==80728==ABORTING
2024-11-05T18:02:59.0437730Z /bin/sh: line 1: 80728 Abort trap: 6           "/Users/runner/work/core/core/build/source/loaders/node_loader/sources/node-v20.18.0/out/Debug/mksnapshot" --turbo_instruction_scheduling "--target_os=mac" "--target_arch=arm64" --startup_src "/Users/runner/work/core/core/build/source/loaders/node_loader/sources/node-v20.18.0/out/Debug/obj.target/v8_snapshot/geni/snapshot.cc" --embedded_variant Default --embedded_src "/Users/runner/work/core/core/build/source/loaders/node_loader/sources/node-v20.18.0/out/Debug/obj.target/v8_snapshot/geni/embedded.S" --no-native-code-counters
2024-11-05T18:02:59.0491070Z make: *** [59267c138e2bfe7d1316b6ed917c8332f9a35c6e.intermediate] Error 134

How often does it reproduce? Is there a required condition?

It reproduces whenever I enable support for asan. It is fully reproducible always on MacOS 15 / ARM64.

What is the expected behavior? Why is that the expected behavior?

Run flawlessly with asan enabled.

What do you see instead?

Avoid bugs and segfaults with asan.

Additional information

No response

@paulincai
Copy link

@viferga May I please ask what macOS 15 you are on. 15.2 is already released and I would also like to know if you updated from 15.1 or from something before that.

We are also seeing problems with building MeteorJS projects with arm64 but the problem was reported after updating to 15.2. Developers info in 15.2 are very few. The only one other issue I could find so far was in Reddit (https://www.reddit.com/r/MacOSBeta/comments/1gn6s5a/heads_up_for_developers_152_has_some_strange/)

I don't know if your problem, the MeteorJS problem (Node 22.11.0) and the problem in this post are related in any way. I am trying to gather information at this stage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants