Skip to content

ARM64 ASan runtime test failures #5826

@StephanTLavavej

Description

@StephanTLavavej

I attempted to enable ARM64 ASan runtime test coverage in #5824 and encountered the following failures.

Hanging tests (SEVERE)

A small number of tests appear to be hanging, causing the shards to hit their 30 minute timeouts and preventing us from gathering useful logs beyond the names of the other failing tests. Annoyingly, the names of the hanging tests aren't printed, and must be deduced by omission. Using my arcane powers, I determined that the following tests appear to be hanging (i.e. they are printed by a successful non-ASan run, and weren't printed by the hanging ASan run).

@zacklj89 merged AddressSanitizer-PR-675052 which fixed deadlocks in the atomics implementation. This was merged very recently (2025-10-14), so it's not present in the Insiders build that we're currently using (MSVC Compiler 19.50.35702). P2093R14_formatted_output uses a Windows semaphore and P2693R1_text_formatting_thread_id uses for_each(execution::par) so I suspect that all of these hangs share the same root cause and have been fixed. Zack has gotten a clean internal run of the STL's tests with ARM64 ASan, so we just need to wait for those fixes to ship.

Click to expand affected tests:

std hangs

tests/P2093R14_formatted_output
tests/P2693R1_text_formatting_thread_id

libcxx hangs

std/atomics/atomics.types.generic/atomics.types.float/fetch_add.pass.cpp
std/atomics/atomics.types.generic/atomics.types.float/fetch_sub.pass.cpp
std/atomics/atomics.types.generic/atomics.types.float/notify_all.pass.cpp

Ordinary test failures

All of the logs report "ERROR: AddressSanitizer: access-violation on unknown address".

@zacklj89 investigated, found multiple root causes, and fixed them:

it looks like the ARM64 specific hotpatch logic will overwrite a jump table in KERNELBASE when hooking legacy allocator APIs. The jump table values get changed, and when GetLocaleInfoHelper goes to use them, the original targets are incorrect and cause that AV 😅

the GH_001103_countl_zero_correctness and other odd exit code bugs were happening with intercepted functions that get called before main -- particularly with calls to strlen from inside ntdll

Click to expand test log:
name: std.tests/P0355R7_calendars_and_time_zones_formatting
output: Build setup steps:
Build steps:
Command: "C:\Program Files\Microsoft Visual Studio\18\Insiders\VC\Tools\MSVC\14.50.35717\bin\HostARM64\ARM64\cl.EXE" "C:\a\_work\1\s\tests\std\tests\P0355R7_calendars_and_time_zones_formatting\test.cpp" "-IC:\stlBuild\out\inc" "-IC:\a\_work\1\s\llvm-project\libcxx\test\support" "-IC:\a\_work\1\s\tests\std\include" "/nologo" "/Od" "/W4" "/w14061" "/w14242" "/w14265" "/w14582" "/w14583" "/w14587" "/w14588" "/w14749" "/w14841" "/w14842" "/w15038" "/w15214" "/w15215" "/w15216" "/w15217" "/w15262" "/sdl" "/WX" "/D_ENABLE_STL_INTERNAL_CHECK" "/bigobj" "/FIforce_include.hpp" "/w14365" "/w14668" "/w15267" "/D_ENFORCE_FACET_SPECIALIZATIONS=1" "/w14640" "/Zc:threadSafeInit-" "/EHsc" "/MDd" "/std:c++latest" "/permissive-" "-fsanitize=address" "/Zi" "/utf-8" "-FeC:\stlBuild\tests\std\tests\P0355R7_calendars_and_time_zones_formatting\Output\09\P0355R7_calendars_and_time_zones_formatting.exe" "-link" "-LIBPATH:C:\stlBuild\out\lib\arm64" "-LIBPATH:C:\Program Files\Microsoft Visual Studio\18\Insiders\VC\Tools\MSVC\14.50.35717\lib\arm64" "/MANIFEST:EMBED" "/debug"
Exit Code: 0 (0x0)
Standard Output:
--
test.cpp
--

Intellisense response file steps:
Test setup steps:
Test steps:
Test step failed unexpectedly.
Command: "C:\stlBuild\tests\std\tests\P0355R7_calendars_and_time_zones_formatting\Output\09\P0355R7_calendars_and_time_zones_formatting.exe"
Exit Code: 1 (0x1)
Standard Error:
--
=================================================================
==8492==ERROR: AddressSanitizer: access-violation on unknown address 0x800095da5848 (pc 0x800095da5848 bp 0x008b6159dc80 sp 0x008b6159db80 T0)
==8492==The signal is caused by a UNKNOWN memory access.
    #0 0x800095da5844  (<unknown module>)
    #1 0x7fff35da5258  (C:\Windows\System32\KERNELBASE.dll+0x180055258)
    #2 0xfe43ffff35da51b0  (<unknown module>)

==8492==Register values:
x0  = 0  x1  = 0  x2  = 0  x3  = 0
x4  = 0  x5  = 0  x6  = 0  x7  = 39
x8  = 3a  x9  = 0  x10 = 27cfb1660cc  x11 = 27cfb180a9c
x12 = 800095da5848  x13 = 0  x14 = 0  x15 = 7fff35da5708
x16 = 9e4bf4f5a7bc  x17 = 9e4bf4f5a7bc  x18 = 7ffef1e2cc94  x19 = 0
x20 = 0  x21 = 0  x22 = 0  x23 = 27cfb180a9c
x24 = 1  x25 = 1292fd0d31b0  x26 = 0  x27 = 27cfb1660cc
x28 = 0  x29 = 8b6159dc80  x30 = 7fff35da525c
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: access-violation (<unknown module>) 
==8492==ABORTING
Click to expand affected tests:

std failures

tests/Dev09_119644_compiler_option_gz
tests/Dev10_908702_string_memory_leak
tests/Dev11_0836436_get_time
tests/GH_003867_output_nan
tests/GH_005204_regex_collating_ranges
tests/GH_005236_collate_facet
tests/P0355R7_calendars_and_time_zones_formatting
tests/P0645R10_text_formatting_formatting
tests/P0645R10_text_formatting_utf8
tests/P0881R7_stacktrace
tests/P1502R1_standard_library_header_units

libcxx failures

std/input.output/iostream.format/ext.manip/get_time.pass.cpp
std/input.output/iostreams.base/ios.base/ios.base.locales/imbue.pass.cpp
std/input.output/iostreams.base/ios/basic.ios.members/imbue.pass.cpp
std/localization/locale.categories/category.collate/locale.collate.byname/compare.pass.cpp
std/localization/locale.categories/category.collate/locale.collate.byname/hash.pass.cpp
std/localization/locale.categories/category.ctype/locale.codecvt.byname/ctor_char.pass.cpp
std/localization/locale.categories/category.ctype/locale.codecvt.byname/ctor_char16_t.pass.cpp
std/localization/locale.categories/category.ctype/locale.ctype.byname/scan_not.pass.cpp
std/localization/locale.categories/category.ctype/locale.ctype.byname/tolower_1.pass.cpp
std/localization/locale.categories/category.ctype/locale.ctype.byname/tolower_many.pass.cpp
std/localization/locale.categories/category.monetary/locale.moneypunct.byname/positive_sign.pass.cpp
std/localization/locale.categories/category.time/locale.time.get.byname/date_order.pass.cpp
std/localization/locale.categories/category.time/locale.time.get.byname/get_year_wide.pass.cpp
std/localization/locale.categories/category.time/locale.time.get.byname/get_year.pass.cpp
std/localization/locale.categories/facet.numpunct/locale.numpunct.byname/decimal_point.pass.cpp
std/localization/locales/locale/locale.cons/assign.pass.cpp
std/localization/locales/locale/locale.cons/default.pass.cpp
std/localization/locales/locale/locale.cons/locale_char_pointer_cat.pass.cpp
std/localization/locales/locale/locale.cons/locale_locale_cat.pass.cpp
std/localization/locales/locale/locale.cons/locale_string_cat.pass.cpp
std/localization/locales/locale/locale.cons/string.pass.cpp
std/localization/locales/locale/locale.operators/eq.pass.cpp
std/re/re.regex/re.regex.locale/imbue.pass.cpp
std/re/re.traits/default.pass.cpp
std/re/re.traits/getloc.pass.cpp
std/re/re.traits/imbue.pass.cpp
std/re/re.traits/transform_primary.pass.cpp
std/re/re.traits/transform.pass.cpp
std/re/re.traits/translate_nocase.pass.cpp
std/time/time.cal/time.cal.md/time.cal.md.nonmembers/ostream.pass.cpp
std/time/time.cal/time.cal.mdlast/ostream.pass.cpp
std/time/time.cal/time.cal.month/time.cal.month.nonmembers/ostream.pass.cpp
std/time/time.cal/time.cal.mwd/time.cal.mwd.nonmembers/ostream.pass.cpp
std/time/time.cal/time.cal.wdlast/time.cal.wdlast.nonmembers/ostream.pass.cpp
std/time/time.cal/time.cal.ym/time.cal.ym.nonmembers/ostream.pass.cpp
std/time/time.cal/time.cal.ymd/time.cal.ymd.nonmembers/ostream.pass.cpp
std/time/time.cal/time.cal.ymdlast/time.cal.ymdlast.nonmembers/ostream.pass.cpp
std/time/time.cal/time.cal.ymwdlast/time.cal.ymwdlast.nonmembers/ostream.pass.cpp
std/time/time.clock/time.clock.local/ostream.pass.cpp
std/time/time.clock/time.clock.system/sys_time.ostream.pass.cpp
std/time/time.clock/time.clock.utc/utc_time.ostream.pass.cpp
std/time/time.hms/time.hms.nonmembers/ostream.pass.cpp
std/time/time.syn/formatter.month_weekday.pass.cpp
std/time/time.syn/formatter.month.pass.cpp
std/utilities/format/format.functions/locale-specific_form.pass.cpp

Metadata

Metadata

Assignees

No one assigned

    Labels

    ARM64Related to the ARM64 architectureASanAddress SanitizertestRelated to test code

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions