Skip to content

tests/ui/allocator/no_std-alloc-error-handler* fail when download-rustc is enabled #108767

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jyn514 opened this issue Mar 5, 2023 · 4 comments · Fixed by #110229
Closed

tests/ui/allocator/no_std-alloc-error-handler* fail when download-rustc is enabled #108767

jyn514 opened this issue Mar 5, 2023 · 4 comments · Fixed by #110229
Assignees
@jyn514
Labels
A-download-rustc Area: The `rust.download-rustc` build option. A-metadata Area: Crate metadata A-testsuite Area: The testsuite used to check the correctness of rustc C-bug Category: This is a bug. E-hard Call for participation: Hard difficulty. Experience needed to fix: A lot. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@jyn514
Copy link
Member

jyn514 commented Mar 5, 2023 •
edited
Loading

cc #81930

I tried this:

x test tests/ui/allocator/no_std-alloc-error-handler*

I expected to see this happen: The tests pass.

Instead, this happened: The tests fail:

Check compiletest suite=ui mode=ui (aarch64-unknown-linux-gnu -> aarch64-unknown-linux-gnu)

running 22 tests
iiiiiiiiiiiiiiiiiiiiFF
failures:

---- [ui] tests/ui/allocator/no_std-alloc-error-handler-default.rs stdout ----

error: test compilation failed although it shouldn't!
status: exit status: 1
command: "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/home/gh-jyn514/rust/tests/ui/allocator/no_std-alloc-error-handler-default.rs" "-Zthreads=1" "--target=aarch64-unknown-linux-gnu" "-O" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Zdeduplicate-diagnostics=no" "-Cstrip=debuginfo" "--remap-path-prefix=/home/gh-jyn514/rust/tests/ui=fake-test-src-base" "-C" "prefer-dynamic" "-o" "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/test/ui/allocator/no_std-alloc-error-handler-default/a" "-Crpath" "-Cdebuginfo=0" "-Lnative=/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/test/ui/allocator/no_std-alloc-error-handler-default/auxiliary" "-C" "panic=abort"
stdout: none
--- stderr -------------------------------
error[E0464]: multiple candidates for `rlib` dependency `libc` found
  --> fake-test-src-base/allocator/no_std-alloc-error-handler-default.rs:15:1
   |
LL | extern crate libc;
   | ^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib
   = note: candidate #2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta

error: aborting due to previous error

For more information about this error, try `rustc --explain E0464`.
------------------------------------------


---- [ui] tests/ui/allocator/no_std-alloc-error-handler-custom.rs stdout ----

error: test compilation failed although it shouldn't!
status: exit status: 1
command: "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/home/gh-jyn514/rust/tests/ui/allocator/no_std-alloc-error-handler-custom.rs" "-Zthreads=1" "--target=aarch64-unknown-linux-gnu" "-O" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Zdeduplicate-diagnostics=no" "-Cstrip=debuginfo" "--remap-path-prefix=/home/gh-jyn514/rust/tests/ui=fake-test-src-base" "-C" "prefer-dynamic" "-o" "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/test/ui/allocator/no_std-alloc-error-handler-custom/a" "-Crpath" "-Cdebuginfo=0" "-Lnative=/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/test/ui/allocator/no_std-alloc-error-handler-custom/auxiliary" "-C" "panic=abort"
stdout: none
--- stderr -------------------------------
error[E0464]: multiple candidates for `rlib` dependency `libc` found
  --> fake-test-src-base/allocator/no_std-alloc-error-handler-custom.rs:16:1
   |
LL | extern crate libc;
   | ^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib
   = note: candidate #2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta

error: aborting due to previous error

For more information about this error, try `rustc --explain E0464`.
------------------------------------------



failures:
    [ui] tests/ui/allocator/no_std-alloc-error-handler-custom.rs
    [ui] tests/ui/allocator/no_std-alloc-error-handler-default.rs

test result: FAILED. 0 passed; 2 failed; 20 ignored; 0 measured; 14535 filtered out; finished in 0.13s

These tests are pretty complicated but the error message looks correct:

error[E0464]: multiple candidates for `rlib` dependency `libc` found
  --> fake-test-src-base/allocator/no_std-alloc-error-handler-custom.rs:16:1
   |
LL | extern crate libc;
   | ^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib
   = note: candidate #2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta

I'm not sure why both files are there and particularly why they have different hashes. tar -tf build/cache/35636f9459720513ca4ed19373a4a7c9e0ea3c46//rustc-dev-nightly-aarch64-unknown-linux-gnu.tar.xz | grep liblibc shows that
rustc-dev-nightly-aarch64-unknown-linux-gnu/rustc-dev/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta is being downloaded from CI, and I guess the .rlib is coming from ensure(compile::Std) somewhere - maybe we should clear the rustlib/ directory before doing that?

Meta

HEAD is 35636f9
@jyn514 jyn514 added E-hard Call for participation: Hard difficulty. Experience needed to fix: A lot. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) C-bug Category: This is a bug. labels Mar 5, 2023
@jyn514 jyn514 mentioned this issue Mar 5, 2023
Merged
@KittyBorgX
Copy link
Member

@rustbot claim

@KittyBorgX KittyBorgX removed their assignment Mar 16, 2023
@jyn514
Copy link
Member Author

jyn514 commented Apr 9, 2023

Ok, it turns out both versions are coming from CI:

; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rust-std-nightly-x86_64-unknown-linux-gnu/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rustc-dev-nightly-x86_64-unknown-linux-gnu/rustc-dev/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta

I am ... surprised this hasn't broken anything else? I don't know why only this specific test would be failing.

@jyn514
Copy link
Member Author

jyn514 commented Apr 9, 2023 •
edited
Loading

The difference between this test and others is that somehow in other tests rustc_resolve knows ahead of time which hash it should use:

INFO rustc_metadata::creader resolving dep crate libc hash: `75267ce7b5b7c8d8` extra filename: `-68a2d9e195dd6ed2`
INFO rustc_metadata::creader resolving crate `libc`
INFO rustc_metadata::creader falling back to a load
INFO rustc_metadata::locator lib candidate: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
INFO rustc_metadata::locator rlib reading metadata from: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
INFO rustc_metadata::creader register crate `libc` (cnum = 6. private_dep = false)

for the allocator tests, it doesn't know, and gives an ambiguity error:

INFO rustc_metadata::creader resolving crate `libc`
INFO rustc_metadata::creader falling back to a load
INFO rustc_metadata::locator lib candidate: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
INFO rustc_metadata::locator lib candidate: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta
INFO rustc_metadata::locator rmeta reading metadata from: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta
INFO rustc_metadata::locator rlib reading metadata from: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
INFO rustc_metadata::creader resolving crate `core`
error[E0464]: multiple candidates for `rlib` dependency `libc` found

@jyn514
Copy link
Member Author

jyn514 commented Apr 9, 2023 •
edited
Loading

ok, here's a minimal reproducer:

#![crate_type = "lib"]
#![no_std]
#![feature(rustc_private)]
extern crate libc;

I think what's going on is for the working tests, the implicit extern crate std injected by the compiler also loads libc, because std depends on libc:

; cargo tree -p std -i libc --depth 1
libc v0.2.140
├── panic_abort v0.0.0 (/home/jyn/src/rust2/library/panic_abort)
├── std v0.0.0 (/home/jyn/src/rust2/library/std)

Then, when the compiler sees extern crate libc, it's already seen the version std depends on and resolves the ambiguity by reusing that crate.

But for the failing test, we never load std, so we don't have a pre-existing hash to default to, so it gives a hard ambiguity error.

@jyn514 jyn514 added A-metadata Area: Crate metadata A-testsuite Area: The testsuite used to check the correctness of rustc labels Apr 9, 2023
@jyn514 jyn514 self-assigned this Apr 9, 2023
@jyn514 jyn514 added the A-download-rustc Area: The `rust.download-rustc` build option. label Apr 12, 2023
@jyn514 jyn514 mentioned this issue Apr 12, 2023
Merged
bors added a commit to rust-lang-ci/rust that referenced this issue Apr 19, 2023
@bors
Auto merge of rust-lang#110229 - jyn514:download-rustc-tests, r=alber…
da48140 
…tlarsan68

Fix no_std tests that load libc from the sysroot when download-rustc is enabled

There were a series of unfortunate interactions here. Here's an MCVE of the test this fixes (committed as `tests/ui/meta/no_std-extern-libc.rs`):
```rust
#![crate_type = "lib"]
#![no_std]
#![feature(rustc_private)]
extern crate libc;
```

Before, this would give an error about duplicate versions of libc:
```
error[E0464]: multiple candidates for `rlib` dependency `libc` found
  --> fake-test-src-base/allocator/no_std-alloc-error-handler-default.rs:15:1
   |
LL | extern crate libc;
   | ^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib
   = note: candidate #2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta
```
Both these versions were downloaded from CI, but one came from the `rust-std` component and one came from `rustc-dev`:
```
; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rust-std-nightly-x86_64-unknown-linux-gnu/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rustc-dev-nightly-x86_64-unknown-linux-gnu/rustc-dev/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta
```
The fix was to only copy files from `rust-std` unless a Step explicitly requests for the `rustc-dev` components to be available by calling `builder.ensure(compile::Rustc)`.

To avoid having to re-parse the `rustc-dev.tar.xz` tarball every time, which is quite slow, this adds a new `build/host/ci-rustc/.rustc-dev-contents` cache file which stores only the names of files we need to copy into the sysroot.

This also allows reverting the hack in rust-lang#110121; now that we only copy rustc-dev on-demand, we can correctly add the `Rustc` check artifacts into the sysroot, so that this works correctly even when `download-rustc` is forced to `true` and some tool depends on a local change to `compiler`.

---

See rust-lang#108767 (comment) for why `no_std` is required for the MCVE test to fail; it's complicated and not particularly important.

Fixes rust-lang#108767.
@bors bors closed this as completed in 71f04bd Apr 19, 2023
RalfJung pushed a commit to RalfJung/miri that referenced this issue Apr 22, 2023
@bors
Auto merge of #110229 - jyn514:download-rustc-tests, r=albertlarsan68
3e72609 
Fix no_std tests that load libc from the sysroot when download-rustc is enabled

There were a series of unfortunate interactions here. Here's an MCVE of the test this fixes (committed as `tests/ui/meta/no_std-extern-libc.rs`):
```rust
#![crate_type = "lib"]
#![no_std]
#![feature(rustc_private)]
extern crate libc;
```

Before, this would give an error about duplicate versions of libc:
```
error[E0464]: multiple candidates for `rlib` dependency `libc` found
  --> fake-test-src-base/allocator/no_std-alloc-error-handler-default.rs:15:1
   |
LL | extern crate libc;
   | ^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib
   = note: candidate rust-lang#2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta
```
Both these versions were downloaded from CI, but one came from the `rust-std` component and one came from `rustc-dev`:
```
; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rust-std-nightly-x86_64-unknown-linux-gnu/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rustc-dev-nightly-x86_64-unknown-linux-gnu/rustc-dev/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta
```
The fix was to only copy files from `rust-std` unless a Step explicitly requests for the `rustc-dev` components to be available by calling `builder.ensure(compile::Rustc)`.

To avoid having to re-parse the `rustc-dev.tar.xz` tarball every time, which is quite slow, this adds a new `build/host/ci-rustc/.rustc-dev-contents` cache file which stores only the names of files we need to copy into the sysroot.

This also allows reverting the hack in rust-lang/rust#110121; now that we only copy rustc-dev on-demand, we can correctly add the `Rustc` check artifacts into the sysroot, so that this works correctly even when `download-rustc` is forced to `true` and some tool depends on a local change to `compiler`.

---

See rust-lang/rust#108767 (comment) for why `no_std` is required for the MCVE test to fail; it's complicated and not particularly important.

Fixes rust-lang/rust#108767.
@intelfx intelfx mentioned this issue Mar 10, 2024
Open
onur-ozkan added a commit to onur-ozkan/rust that referenced this issue Aug 20, 2024
@onur-ozkan
don't copy .rustc-dev-contents from CI rustc
4bf79c7 
Copying files from `.rustc-dev-contents` regressed rust-lang#108767
again. Since `rustc-src` is already included in the CI rustc sysroot, we don't need to copy these
files to have `rustc-src` component in the sysroot.

Signed-off-by: onur-ozkan <[email protected]>
onur-ozkan added a commit to onur-ozkan/rust that referenced this issue Aug 20, 2024
@onur-ozkan
don't copy .rustc-dev-contents from CI rustc
9589279 
Copying files from `.rustc-dev-contents` regressed rust-lang#108767
again. Since `rustc-src` is already included in the CI rustc sysroot, we don't need to copy these
files to have `rustc-src` component in the sysroot.

Signed-off-by: onur-ozkan <[email protected]>
onur-ozkan added a commit to onur-ozkan/rust that referenced this issue Aug 20, 2024
@onur-ozkan
don't copy .rustc-dev-contents from CI rustc
fa6d54f 
Since rust-lang#127188, copying files from `.rustc-dev-contents`
regressed rust-lang#108767 again. Since `rustc-src` is already
included in the CI rustc sysroot, we don't need to copy these files to have `rustc-src` component.

Signed-off-by: onur-ozkan <[email protected]>
@onur-ozkan onur-ozkan mentioned this issue Aug 20, 2024
Merged
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Sep 3, 2024
@matthiaskrgr
Rollup merge of rust-lang#129311 - onur-ozkan:multiple-candidates-fix…
44187d6 
…, r=Kobzol

don't copy `.rustc-dev-contents` from CI rustc

Since rust-lang#127188, copying files from `.rustc-dev-contents` regressed rust-lang#108767 again. Since `rustc-src` is already included in the CI rustc sysroot, we don't need to copy these files to have `rustc-src` component.

Blocker for rust-lang#122709
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Sep 3, 2024
@rust-timer
Unrolled build for rust-lang#129311
Loading
Loading status checks…
a19b205 
Rollup merge of rust-lang#129311 - onur-ozkan:multiple-candidates-fix, r=Kobzol

don't copy `.rustc-dev-contents` from CI rustc

Since rust-lang#127188, copying files from `.rustc-dev-contents` regressed rust-lang#108767 again. Since `rustc-src` is already included in the CI rustc sysroot, we don't need to copy these files to have `rustc-src` component.

Blocker for rust-lang#122709
@Kobzol Kobzol mentioned this issue Sep 4, 2024
Merged
@onur-ozkan onur-ozkan mentioned this issue Sep 17, 2024
Merged
workingjubilee added a commit to workingjubilee/rustc that referenced this issue Oct 7, 2024
@workingjubilee
Rollup merge of rust-lang#130479 - onur-ozkan:llvm-bitcode-linker-mul…
8cd9d95 
…tiple-candidates, r=Kobzol

skip in-tree compiler build for llvm-bitcode-linker if ci-rustc is on

Similar to rust-lang#108767, resolves the `multiple candidates` problem for ci-rustc.

See rust-lang#122709 (comment) for more context.

Blocker for rust-lang#122709.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Oct 7, 2024
@rust-timer
Unrolled build for rust-lang#130479
Loading
Loading status checks…
f46a037 
Rollup merge of rust-lang#130479 - onur-ozkan:llvm-bitcode-linker-multiple-candidates, r=Kobzol

skip in-tree compiler build for llvm-bitcode-linker if ci-rustc is on

Similar to rust-lang#108767, resolves the `multiple candidates` problem for ci-rustc.

See rust-lang#122709 (comment) for more context.

Blocker for rust-lang#122709.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jyn514 jyn514

Labels
A-download-rustc Area: The `rust.download-rustc` build option. A-metadata Area: Crate metadata A-testsuite Area: The testsuite used to check the correctness of rustc C-bug Category: This is a bug. E-hard Call for participation: Hard difficulty. Experience needed to fix: A lot. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix no_std tests that load libc from the sysroot when download-rustc is enabled jyn514/rust
2 participants
@jyn514 @KittyBorgX