You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Regarding the stacktraces below, does it look to you like a deadlock with tcmalloc?
See that in the first I call dl_iterate_phdr and which in its turn calls tcmalloc and then tcmalloc locks a mutex/futex, and the second thread calls tcmalloc (probably inlined the malloc function?), and proceeds to the same function (dl_iterate_phdr) I called in the first thread.
I think the second thread also locks because it's also in tcmalloc::CentralFreeList::Populate.
The last stacktrace of each thread is of functions inside the original project (irrelevant to tcmalloc).
Thread 1 (Thread 0x7f1680ec7000 (LWP 123804)):
#0 0x00007f16b1f6715b in sys_futex (v3=0, a2=0x0, t=0x7ffc5bb09f40, v=2, o=128,
a=0x7f16ad46be48 tcmalloc::Static::pageheap_lock_) at ./src/base/linux_syscall_support.h:2485 #1 base::internal::SpinLockDelay (w=0x7f16ad46be48 tcmalloc::Static::pageheap_lock_, value=2,
loop=) at ./src/base/spinlock_linux-inl.h:84 #2 0x00007f16b1f67074 in SpinLock::SlowLock (this=0x7f16ad46be48 tcmalloc::Static::pageheap_lock_)
at src/base/spinlock.cc:119 #3 0x00007f16ad0a31d8 in Lock (this=) at src/base/spinlock.h:71 #4 SpinLockHolder (l=, this=) at src/base/spinlock.h:133 #5 tcmalloc::CentralFreeList::Populate (
this=this@entry=0x7f16ad45e380 tcmalloc::Static::central_cache_+64448)
at src/central_freelist.cc:328 #6 0x00007f16ad0a32d8 in tcmalloc::CentralFreeList::FetchFromOneSpansSafe (
this=0x7f16ad45e380 tcmalloc::Static::central_cache_+64448, N=4, start=0x7ffc5bb0a070,
end=0x7ffc5bb0a078) at src/central_freelist.cc:284 #7 0x00007f16ad0a3364 in tcmalloc::CentralFreeList::RemoveRange (
this=0x7f16ad45e380 tcmalloc::Static::central_cache_+64448, start=start@entry=0x7ffc5bb0a070,
end=end@entry=0x7ffc5bb0a078, N=4) at src/central_freelist.cc:264 #8 0x00007f16ad0a6b50 in tcmalloc::ThreadCache::FetchFromCentralCache (this=0x883c40, cl=53,
byte_size=16384, oom_handler=0x7f16ad0b45c0 <tcmalloc::cpp_throw_oom(unsigned long)>)
at src/thread_cache.cc:126 #9 0x00007f16a688f49d in void std::vector<scale::details::shared_library_address_space, std::allocatorscale::details::shared_library_address_space >::_M_emplace_back_auxscale::details::shared_library_address_space(scale::details::shared_library_address_space&&) ()
from /home/test_user/sqream-with-submodules/sqream/build/libbasic.so #10 0x00007f16a68638eb in ?? () from /home/test_user/sqream-with-submodules/sqream/build/libbasic.so #11 0x00007f1696e381bc in dl_iterate_phdr () from /lib64/libc.so.6 #12 0x00007f16a68602eb in scale::details::library_addresses() ()
...
Thread 3 (Thread 0x7f165d745700 (LWP 123862)):
#0 0x00007f16ace7154d in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x00007f16ace6ceb6 in _L_lock_941 () from /lib64/libpthread.so.0 #2 0x00007f16ace6cdaf in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x00007f1696e380ff in dl_iterate_phdr () from /lib64/libc.so.6 #4 0x00007f16970db083 in _Unwind_Find_FDE (
pc=0x7f16ad0a5132 <tcmalloc::PageHeap::New(unsigned long)+146>, bases=bases@entry=0x7f165d744118)
at ../.././libgcc/unwind-dw2-fde-dip.c:459 #5 0x00007f16970d7a46 in uw_frame_state_for (context=context@entry=0x7f165d744070,
fs=fs@entry=0x7f165d744160) at ../.././libgcc/unwind-dw2.c:1241 #6 0x00007f16970d9598 in _Unwind_Backtrace (
trace=trace@entry=0x7f16b1f65e70 <libgcc_backtrace_helper(_Unwind_Context*, void*)>,
trace_argument=trace_argument@entry=0x7f165d744320) at ../.././libgcc/unwind.inc:290 #7 0x00007f16b1f65c8a in GetStackTrace_libgcc (result=, max_depth=,
skip_count=) at src/stacktrace_libgcc-inl.h:100 #8 0x00007f16b1f66364 in GetStackTrace (result=0x843538, max_depth=30, skip_count=3)
at src/stacktrace.cc:295 #9 0x00007f16ad0a4e36 in RecordGrowth (growth=1048576) at src/page_heap.cc:618 #10 tcmalloc::PageHeap::GrowHeap (this=0x7f16ad2cb640 tcmalloc::Static::pageheap_, n=)
at src/page_heap.cc:644 #11 0x00007f16ad0a5133 in tcmalloc::PageHeap::New (this=0x7f16ad2cb640 tcmalloc::Static::pageheap_,
n=n@entry=5) at src/page_heap.cc:154 #12 0x00007f16ad0a30d5 in tcmalloc::CentralFreeList::Populate (
this=this@entry=0x7f16ad45e840 tcmalloc::Static::central_cache_+65664)
at src/central_freelist.cc:329 #13 0x00007f16ad0a32d8 in tcmalloc::CentralFreeList::FetchFromOneSpansSafe (
this=0x7f16ad45e840 tcmalloc::Static::central_cache_+65664, N=1, start=0x7f165d7444c0,
end=0x7f165d7444c8) at src/central_freelist.cc:284 #14 0x00007f16ad0a3364 in tcmalloc::CentralFreeList::RemoveRange (
this=0x7f16ad45e840 tcmalloc::Static::central_cache_+65664, start=start@entry=0x7f165d7444c0,
end=end@entry=0x7f165d7444c8, N=1) at src/central_freelist.cc:264 #15 0x00007f16ad0a6b50 in tcmalloc::ThreadCache::FetchFromCentralCache (this=0x887980, cl=54,
byte_size=20480, oom_handler=0x7f16ad0b45c0 <tcmalloc::cpp_throw_oom(unsigned long)>)
at src/thread_cache.cc:126 #16 0x00007f16a68e498c in set_status(status_information const&) ()
...
The text was updated successfully, but these errors were encountered:
Regarding the stacktraces below, does it look to you like a deadlock with tcmalloc?
See that in the first I call dl_iterate_phdr and which in its turn calls tcmalloc and then tcmalloc locks a mutex/futex, and the second thread calls tcmalloc (probably inlined the malloc function?), and proceeds to the same function (dl_iterate_phdr) I called in the first thread.
I think the second thread also locks because it's also in tcmalloc::CentralFreeList::Populate.
The last stacktrace of each thread is of functions inside the original project (irrelevant to tcmalloc).
Thread 1 (Thread 0x7f1680ec7000 (LWP 123804)):
#0 0x00007f16b1f6715b in sys_futex (v3=0, a2=0x0, t=0x7ffc5bb09f40, v=2, o=128,
a=0x7f16ad46be48 tcmalloc::Static::pageheap_lock_) at ./src/base/linux_syscall_support.h:2485
#1 base::internal::SpinLockDelay (w=0x7f16ad46be48 tcmalloc::Static::pageheap_lock_, value=2,
loop=) at ./src/base/spinlock_linux-inl.h:84
#2 0x00007f16b1f67074 in SpinLock::SlowLock (this=0x7f16ad46be48 tcmalloc::Static::pageheap_lock_)
at src/base/spinlock.cc:119
#3 0x00007f16ad0a31d8 in Lock (this=) at src/base/spinlock.h:71
#4 SpinLockHolder (l=, this=) at src/base/spinlock.h:133
#5 tcmalloc::CentralFreeList::Populate (
this=this@entry=0x7f16ad45e380 tcmalloc::Static::central_cache_+64448)
at src/central_freelist.cc:328
#6 0x00007f16ad0a32d8 in tcmalloc::CentralFreeList::FetchFromOneSpansSafe (
this=0x7f16ad45e380 tcmalloc::Static::central_cache_+64448, N=4, start=0x7ffc5bb0a070,
end=0x7ffc5bb0a078) at src/central_freelist.cc:284
#7 0x00007f16ad0a3364 in tcmalloc::CentralFreeList::RemoveRange (
this=0x7f16ad45e380 tcmalloc::Static::central_cache_+64448, start=start@entry=0x7ffc5bb0a070,
end=end@entry=0x7ffc5bb0a078, N=4) at src/central_freelist.cc:264
#8 0x00007f16ad0a6b50 in tcmalloc::ThreadCache::FetchFromCentralCache (this=0x883c40, cl=53,
byte_size=16384, oom_handler=0x7f16ad0b45c0 <tcmalloc::cpp_throw_oom(unsigned long)>)
at src/thread_cache.cc:126
#9 0x00007f16a688f49d in void std::vector<scale::details::shared_library_address_space, std::allocatorscale::details::shared_library_address_space >::_M_emplace_back_auxscale::details::shared_library_address_space(scale::details::shared_library_address_space&&) ()
from /home/test_user/sqream-with-submodules/sqream/build/libbasic.so
#10 0x00007f16a68638eb in ?? () from /home/test_user/sqream-with-submodules/sqream/build/libbasic.so
#11 0x00007f1696e381bc in dl_iterate_phdr () from /lib64/libc.so.6
#12 0x00007f16a68602eb in scale::details::library_addresses() ()
...
Thread 3 (Thread 0x7f165d745700 (LWP 123862)):
#0 0x00007f16ace7154d in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007f16ace6ceb6 in _L_lock_941 () from /lib64/libpthread.so.0
#2 0x00007f16ace6cdaf in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007f1696e380ff in dl_iterate_phdr () from /lib64/libc.so.6
#4 0x00007f16970db083 in _Unwind_Find_FDE (
pc=0x7f16ad0a5132 <tcmalloc::PageHeap::New(unsigned long)+146>, bases=bases@entry=0x7f165d744118)
at ../.././libgcc/unwind-dw2-fde-dip.c:459
#5 0x00007f16970d7a46 in uw_frame_state_for (context=context@entry=0x7f165d744070,
fs=fs@entry=0x7f165d744160) at ../.././libgcc/unwind-dw2.c:1241
#6 0x00007f16970d9598 in _Unwind_Backtrace (
trace=trace@entry=0x7f16b1f65e70 <libgcc_backtrace_helper(_Unwind_Context*, void*)>,
trace_argument=trace_argument@entry=0x7f165d744320) at ../.././libgcc/unwind.inc:290
#7 0x00007f16b1f65c8a in GetStackTrace_libgcc (result=, max_depth=,
skip_count=) at src/stacktrace_libgcc-inl.h:100
#8 0x00007f16b1f66364 in GetStackTrace (result=0x843538, max_depth=30, skip_count=3)
at src/stacktrace.cc:295
#9 0x00007f16ad0a4e36 in RecordGrowth (growth=1048576) at src/page_heap.cc:618
#10 tcmalloc::PageHeap::GrowHeap (this=0x7f16ad2cb640 tcmalloc::Static::pageheap_, n=)
at src/page_heap.cc:644
#11 0x00007f16ad0a5133 in tcmalloc::PageHeap::New (this=0x7f16ad2cb640 tcmalloc::Static::pageheap_,
n=n@entry=5) at src/page_heap.cc:154
#12 0x00007f16ad0a30d5 in tcmalloc::CentralFreeList::Populate (
this=this@entry=0x7f16ad45e840 tcmalloc::Static::central_cache_+65664)
at src/central_freelist.cc:329
#13 0x00007f16ad0a32d8 in tcmalloc::CentralFreeList::FetchFromOneSpansSafe (
this=0x7f16ad45e840 tcmalloc::Static::central_cache_+65664, N=1, start=0x7f165d7444c0,
end=0x7f165d7444c8) at src/central_freelist.cc:284
#14 0x00007f16ad0a3364 in tcmalloc::CentralFreeList::RemoveRange (
this=0x7f16ad45e840 tcmalloc::Static::central_cache_+65664, start=start@entry=0x7f165d7444c0,
end=end@entry=0x7f165d7444c8, N=1) at src/central_freelist.cc:264
#15 0x00007f16ad0a6b50 in tcmalloc::ThreadCache::FetchFromCentralCache (this=0x887980, cl=54,
byte_size=20480, oom_handler=0x7f16ad0b45c0 <tcmalloc::cpp_throw_oom(unsigned long)>)
at src/thread_cache.cc:126
#16 0x00007f16a68e498c in set_status(status_information const&) ()
...
The text was updated successfully, but these errors were encountered: