Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mold randomly segfaults #1211

Open
firasuke opened this issue Mar 1, 2024 · 10 comments
Open

mold randomly segfaults #1211

firasuke opened this issue Mar 1, 2024 · 10 comments

Comments

@firasuke
Copy link

firasuke commented Mar 1, 2024

I recently started using mold as the default linker for a custom Linux distribution I am building from scratch (the distribution uses musl libc as the default C library), and mold is producing random segfaults:

image

I am not facing any issues when using bfd.
@firasuke
Copy link
Author

firasuke commented Mar 1, 2024 •
edited
Loading

Here is the output of strace for both a successful run of mold and a segfault:

Successful Run:

execve("/usr/bin/mold", ["mold"], 0x7ffd2547d700 /* 12 vars */) = 0
arch_prctl(ARCH_SET_FS, 0x7929391957e8) = 0
set_tid_address(0x792939192d08)         = 974
brk(NULL)                               = 0x5ab0131c3000
brk(0x5ab0131c5000)                     = 0x5ab0131c5000
mmap(0x5ab0131c3000, 4096, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x5ab0131c3000
open("/usr/lib/libmimalloc.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=113512, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240g\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 323584, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7929390ab000
mmap(0x7929390b1000, 81920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x5000) = 0x7929390b1000
mmap(0x7929390c5000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x18000) = 0x7929390c5000
mmap(0x7929390c7000, 208896, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x19000) = 0x7929390c7000
mmap(0x7929390ca000, 196608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7929390ca000
close(3)                                = 0
open("/etc/ld-musl-x86_64.path", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=9, ...}) = 0
read(3, "/usr/lib\n", 9)                = 9
close(3)                                = 0
open("/usr/lib/libxxhash.so.0", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=32400, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\"\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 45056, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7929390a0000
mmap(0x7929390a2000, 28672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x1000) = 0x7929390a2000
mmap(0x7929390a9000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x7000) = 0x7929390a9000
mmap(0x7929390aa000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x7000) = 0x7929390aa000
close(3)                                = 0
open("/usr/lib/libz.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=118480, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`~\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 131072, PROT_READ, MAP_PRIVATE, 3, 0) = 0x792939080000
mmap(0x792939087000, 90112, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x6000) = 0x792939087000
mmap(0x79293909d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x1b000) = 0x79293909d000
mmap(0x79293909f000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x1c000) = 0x79293909f000
close(3)                                = 0
open("/usr/lib/libzstd.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=660488, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\246\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 671744, PROT_READ, MAP_PRIVATE, 3, 0) = 0x792938fdc000
mmap(0x792938fe6000, 622592, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x9000) = 0x792938fe6000
mmap(0x79293907e000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0xa0000) = 0x79293907e000
mmap(0x79293907f000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0xa0000) = 0x79293907f000
close(3)                                = 0
open("/usr/lib/libmimalloc.so.2", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=113512, ...}) = 0
close(3)                                = 0
open("/usr/lib/libstdc++.so.6", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=2435984, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\313\22\0\0\0\0\0"..., 960) = 960
mmap(NULL, 2465792, PROT_READ, MAP_PRIVATE, 3, 0) = 0x792938c00000
mmap(0x792938d2c000, 1134592, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x12b000) = 0x792938d2c000
mmap(0x792938e41000, 45056, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x23f000) = 0x792938e41000
mmap(0x792938e4c000, 57344, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x249000) = 0x792938e4c000
mmap(0x792938e56000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x792938e56000
close(3)                                = 0
open("/usr/lib/libgcc_s.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=653352, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20X\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 139264, PROT_READ, MAP_PRIVATE, 3, 0) = 0x792938fba000
mmap(0x792938fbf000, 106496, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x4000) = 0x792938fbf000
mmap(0x792938fd9000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x1d000) = 0x792938fd9000
mmap(0x792938fdb000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x1e000) = 0x792938fdb000
close(3)                                = 0
mprotect(0x7929390c5000, 8192, PROT_READ) = 0
mprotect(0x7929390a9000, 4096, PROT_READ) = 0
mprotect(0x79293909d000, 8192, PROT_READ) = 0
mprotect(0x79293907e000, 4096, PROT_READ) = 0
mprotect(0x792938e41000, 45056, PROT_READ) = 0
mprotect(0x792938fd9000, 8192, PROT_READ) = 0
mprotect(0x792939191000, 4096, PROT_READ) = 0
mprotect(0x5ab012d7f000, 413696, PROT_READ) = 0
getrandom("\x2d\x80\xf2\x5f\x35\xa1\x8d\xd5\x0d\xd8\x92\x00\xe8\xc8\x3e\xd3\x6c\x8e\x0d\x33\xb4\x51\x54\x5d\xfd\x7c\x1e\xfa\x47\x26\xa4\x7b", 32, GRND_NONBLOCK) = 32
open("/proc/sys/vm/overcommit_memory", O_RDONLY) = 3
read(3, "0\n", 32)                      = 2
close(3)                                = 0
access("/sys/devices/system/node/node1", R_OK) = -1 ENOENT (No such file or directory)
mmap(0x55efa000000, 1073741824, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x55efa000000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x792938fb9000
open("/usr/lib/libtcm.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/libtbbmalloc.so.2", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
ioctl(2, TIOCGWINSZ, 0x7ffe60ad4d90)    = -1 ENOTTY (Inappropriate ioctl for device)
sched_getaffinity(0, 128, [0, 1, 2, 3]) = 8
getpid()                                = 974
sched_getaffinity(974, 128, [0, 1, 2, 3]) = 8
open("/usr/lib/libiomp5.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
writev(2, [{iov_base="", iov_len=0}, {iov_base="mold: fatal: -m option is missin"..., iov_len=33}], 2mold: fatal: -m option is missing) = 33
writev(2, [{iov_base="", iov_len=0}, {iov_base="\n", iov_len=1}], 2
) = 1
exit_group(1)                           = ?
+++ exited with 1 +++

Segfault Run:

execve("/usr/bin/mold", ["mold"], 0x7ffdb816f6e0 /* 12 vars */) = 0
arch_prctl(ARCH_SET_FS, 0x71e95c4207e8) = 0
set_tid_address(0x71e95c41dd08)         = 1046
brk(NULL)                               = 0x59f445319000
brk(0x59f44531b000)                     = 0x59f44531b000
mmap(0x59f445319000, 4096, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x59f445319000
open("/usr/lib/libmimalloc.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=113512, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240g\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 323584, PROT_READ, MAP_PRIVATE, 3, 0) = 0x71e95c336000
mmap(0x71e95c33c000, 81920, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x5000) = 0x71e95c33c000
mmap(0x71e95c350000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x18000) = 0x71e95c350000
mmap(0x71e95c352000, 208896, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x19000) = 0x71e95c352000
mmap(0x71e95c355000, 196608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x71e95c355000
close(3)                                = 0
open("/etc/ld-musl-x86_64.path", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=9, ...}) = 0
read(3, "/usr/lib\n", 9)                = 9
close(3)                                = 0
open("/usr/lib/libxxhash.so.0", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=32400, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\"\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 45056, PROT_READ, MAP_PRIVATE, 3, 0) = 0x71e95c32b000
mmap(0x71e95c32d000, 28672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x1000) = 0x71e95c32d000
mmap(0x71e95c334000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x7000) = 0x71e95c334000
mmap(0x71e95c335000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x7000) = 0x71e95c335000
close(3)                                = 0
open("/usr/lib/libz.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=118480, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`~\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 131072, PROT_READ, MAP_PRIVATE, 3, 0) = 0x71e95c30b000
mmap(0x71e95c312000, 90112, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x6000) = 0x71e95c312000
mmap(0x71e95c328000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x1b000) = 0x71e95c328000
mmap(0x71e95c32a000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x1c000) = 0x71e95c32a000
close(3)                                = 0
open("/usr/lib/libzstd.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=660488, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\246\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 671744, PROT_READ, MAP_PRIVATE, 3, 0) = 0x71e95c267000
mmap(0x71e95c271000, 622592, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x9000) = 0x71e95c271000
mmap(0x71e95c309000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0xa0000) = 0x71e95c309000
mmap(0x71e95c30a000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0xa0000) = 0x71e95c30a000
close(3)                                = 0
open("/usr/lib/libmimalloc.so.2", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=113512, ...}) = 0
close(3)                                = 0
open("/usr/lib/libstdc++.so.6", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0755, st_size=2435984, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\313\22\0\0\0\0\0"..., 960) = 960
mmap(NULL, 2465792, PROT_READ, MAP_PRIVATE, 3, 0) = 0x71e95c000000
mmap(0x71e95c12c000, 1134592, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x12b000) = 0x71e95c12c000
mmap(0x71e95c241000, 45056, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x23f000) = 0x71e95c241000
mmap(0x71e95c24c000, 57344, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x249000) = 0x71e95c24c000
mmap(0x71e95c256000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x71e95c256000
close(3)                                = 0
open("/usr/lib/libgcc_s.so.1", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
fcntl(3, F_SETFD, FD_CLOEXEC)           = 0
fstat(3, {st_mode=S_IFREG|0644, st_size=653352, ...}) = 0
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20X\0\0\0\0\0\0"..., 960) = 960
mmap(NULL, 139264, PROT_READ, MAP_PRIVATE, 3, 0) = 0x71e95bfde000
mmap(0x71e95bfe3000, 106496, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 3, 0x4000) = 0x71e95bfe3000
mmap(0x71e95bffd000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x1d000) = 0x71e95bffd000
mmap(0x71e95bfff000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x1e000) = 0x71e95bfff000
close(3)                                = 0
mprotect(0x71e95c350000, 8192, PROT_READ) = 0
mprotect(0x71e95c334000, 4096, PROT_READ) = 0
mprotect(0x71e95c328000, 8192, PROT_READ) = 0
mprotect(0x71e95c309000, 4096, PROT_READ) = 0
mprotect(0x71e95c241000, 45056, PROT_READ) = 0
mprotect(0x71e95bffd000, 8192, PROT_READ) = 0
mprotect(0x71e95c41c000, 4096, PROT_READ) = 0
mprotect(0x59f444b21000, 413696, PROT_READ) = 0
getrandom("\xb1\x1a\x96\xb5\x69\x5d\x4c\xd0\xe1\x8c\x60\x50\x03\x8e\x27\x9c\x72\xaf\x43\x81\x12\x94\x8f\x84\x5e\x1a\xc4\x38\x25\x83\x57\xd2", 32, GRND_NONBLOCK) = 32
open("/proc/sys/vm/overcommit_memory", O_RDONLY) = 3
read(3, "0\n", 32)                      = 2
close(3)                                = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x71e9a272b583} ---
+++ killed by SIGSEGV +++
The process was killed by SIGSEGV: Segmentation fault

@rui314
Copy link
Owner

rui314 commented Mar 2, 2024

Can you run the executable under gdb to get a stacktrace of the crashed process?

mold doesn't do anything meaningful when being ran without an option, so this issue looks odd.

@firasuke
Copy link
Author

firasuke commented Mar 2, 2024

Can you run the executable under gdb to get a stacktrace of the crashed process?

Unfortunately I am unable to provide a stacktrace for now.

mold doesn't do anything meaningful when being ran without an option, so this issue looks odd.

It is segfaulting when it is being called by gcc in the linking stage.

@rui314
Copy link
Owner

rui314 commented Mar 2, 2024

But your screenshot shows that just running mold without any arguments can reproduce the issue, no?

@firasuke
Copy link
Author

firasuke commented Mar 2, 2024

But your screenshot shows that just running mold without any arguments can reproduce the issue, no?

Yes mold is segfaulting when being run with or without arguments, either by calling it directly, or when gcc attempts to call it in the linking stage.

@rui314
Copy link
Owner

rui314 commented Mar 3, 2024

Can you reproduce the issue if you repeatedly run mold without arguments under gdb?

@firasuke
Copy link
Author

firasuke commented Mar 3, 2024

Can you reproduce the issue if you repeatedly run mold without arguments under gdb?

Unfortunately I don't currently have gdb in the environment where I am using mold.

I am keeping this open until I can properly reproduce this under gdb.

@theoparis
Copy link

I also received a segfault with musl libc on an alpine linux docker container and a gentoo linux container. It does seem to be very random and I can't seem to easily reproduce it via a Dockerfile 😕

@rui314
Copy link
Owner

rui314 commented Oct 14, 2024

Are you sure you have enough memory? Without enough memory, mold (and any other process) can be killed by the kernel when it's low on memory. You may want to check if OOM killer didn't kick in using dmesg.

@Alcaro
Copy link

Alcaro commented Oct 14, 2024

The OOM killer uses SIGKILL, not SIGSEGV.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rui314 @Alcaro @theoparis @firasuke