Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate libs-through-symlinks and translation run-make tests to rmake #129011

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Oneirical
Copy link
Contributor

@Oneirical Oneirical commented Aug 12, 2024

Part of #121876 and the associated Google Summer of Code project.

Everything here works locally... except line 57 (the test works if it is commented out):

    rustc()
        .arg("-Ztranslate-lang=zh-CN")
        .input("test.rs")
        .sysroot("fakeroot")
        .run_fail()
        .assert_stderr_contains("this is a test message");

This will therefore initially fail CI.

try-job: aarch64-gnu
try-job: aarch64-apple
try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu

@rustbot
Copy link
Collaborator

rustbot commented Aug 12, 2024

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Aug 12, 2024
@rustbot
Copy link
Collaborator

rustbot commented Aug 12, 2024

This PR modifies tests/run-make/. If this PR is trying to port a Makefile
run-make test to use rmake.rs, please update the
run-make port tracking issue
so we can track our progress. You can either modify the tracking issue
directly, or you can comment on the tracking issue and link this PR.

cc @jieyouxu

@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Aug 14, 2024

☔ The latest upstream changes (presumably #129092) made this pull request unmergeable. Please resolve the merge conflicts.

tests/run-make/libs-through-symlinks/rmake.rs Show resolved Hide resolved
tests/run-make/libs-through-symlinks/rmake.rs Show resolved Hide resolved
tests/run-make/translation/rmake.rs Show resolved Hide resolved
tests/run-make/translation/rmake.rs Show resolved Hide resolved
// translation of rustc's error messages.
rustc()
.arg("-Ztranslate-lang=tlh")
// .input("test.rs")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remark: commented out

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Original test:

sysroot-missing:
	$(RUSTC) $< -Ztranslate-lang=tlh 2>&1 | $(CGREP) "missing locale directory"

Note how the ususal test.rs next to the section header sysroot-missing: is absent. I have no idea what the $< is slotting in.

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 16, 2024
@Oneirical Oneirical force-pushed the dont-blink-they-sys-you branch from ea3ebb5 to af4bdbf Compare August 16, 2024 14:37
@jieyouxu jieyouxu added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 16, 2024
@rust-log-analyzer
Copy link
Collaborator

The job x86_64-gnu-llvm-17 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
------
 > importing cache manifest from ghcr.io/rust-lang/rust-ci-cache:20d3b4d4a2629cbf7865cdbf92fe47512a7c96658c24253a045ff38e8075cd7fb37ca6fcadfa6e6d093333943ad24f6fc4f163ec5b74fd940de9d5bb03eb4d3b:
------
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-17]
---
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-17', '--enable-llvm-link-shared', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-17/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.thin-lto-import-instr-limit := 10
configure: change-id            := 99999999
---
  Downloaded boml v0.3.1
   Compiling boml v0.3.1
   Compiling y v0.1.0 (/checkout/compiler/rustc_codegen_gcc/build_system)
    Finished `release` profile [optimized] target(s) in 3.79s
     Running `/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-codegen/x86_64-unknown-linux-gnu/release/y test --use-system-gcc --use-backend gcc --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc --release --mini-tests --std-tests`
Using system GCC
[BUILD] example
[AOT] mini_core_hello_world
/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools/cg_gcc/mini_core_hello_world
abc
---
---- [run-make] tests/run-make/translation stdout ----

error: rmake recipe failed to complete
status: exit status: 101
command: cd "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/translation/rmake_out" && env -u RUSTFLAGS AR="ar" CC="cc" CC_DEFAULT_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64" CXX="c++" CXX_DEFAULT_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64" HOST_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools-bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" LD_LIB_PATH_ENVVAR="LD_LIBRARY_PATH" LLVM_BIN_DIR="/usr/lib/llvm-17/bin" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgputargetmca amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen codegentypes core coroutines coverage debuginfobtf debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerparallel dwp engine executionengine extensions filecheck frontendhlsl frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink lanai lanaiasmparser lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcjit orcshared orctargetprocess passes perfjitevents powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser textapi transformutils ve veasmparser vecodegen vectorize vedesc vedisassembler veinfo webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xcore xcorecodegen xcoredesc xcoredisassembler xcoreinfo xray xtensa xtensaasmparser xtensacodegen xtensadesc xtensadisassembler xtensainfo" LLVM_FILECHECK="/usr/lib/llvm-17/bin/FileCheck" NODE="/usr/bin/node" PYTHON="/usr/bin/python3" RUSTC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" RUSTDOC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" SOURCE_ROOT="/checkout" TARGET="x86_64-unknown-linux-gnu" TARGET_RPATH_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" TARGET_RPATH_ENV="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/translation/rmake_out:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/translation/rmake"
--- stderr -------------------------------
--- stderr -------------------------------
=== HAYSTACK ===
error: failed to load fluent bundle: could not read ftl file: No such file or directory (os error 2)

=== NEEDLE ===
this is a test message
thread 'main' panicked at /checkout/tests/run-make/translation/rmake.rs:88:10:

rm -f $(FAKEROOT)/share
mkdir -p $(FAKEROOT)/share/locale/zh-CN/
ln -s $(CURDIR)/working.ftl $(FAKEROOT)/share/locale/zh-CN/basic-translation.ftl
$(RUSTC) $< --sysroot $(FAKEROOT) -Ztranslate-lang=zh-CN 2>&1 | $(CGREP) "this is a test message"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remark (to myself): $< ?

@bors
Copy link
Contributor

bors commented Aug 23, 2024

☔ The latest upstream changes (presumably #129443) made this pull request unmergeable. Please resolve the merge conflicts.

@jieyouxu
Copy link
Member

@rustbot blocked (on fixing symlink cleanup in compiletest and bootstrap)

@rustbot rustbot added S-blocked Status: Blocked on something else such as an RFC or other implementation work. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 23, 2024
@jieyouxu jieyouxu added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-blocked Status: Blocked on something else such as an RFC or other implementation work. labels Sep 27, 2024
@jieyouxu
Copy link
Member

The symlink trouble I observed previously from compiletest and bootstrap likely came from somehow under non-admin mode artifacts can be produced under admin ownership, which doesn't block this PR now that run_make_support symlink handling is rectified.

@jieyouxu
Copy link
Member

jieyouxu commented Dec 25, 2024

The symlink trouble I observed previously from compiletest and bootstrap likely came from somehow under non-admin mode artifacts can be produced under admin ownership, which doesn't block this PR now that run_make_support symlink handling is rectified.

@jieyouxu Nice joke, this had to be reverted then re-fixed in #134659.

@jieyouxu
Copy link
Member

libs-through-symlink extracted to #134829.

Zalathar added a commit to Zalathar/rust that referenced this pull request Dec 28, 2024
…inks, r=lqd

Migrate `libs-through-symlink` to rmake.rs

Part of rust-lang#121876.

This PR migrates `tests/run-make/libs-through-symlink/` to use rmake.rs.

- Regression test for rust-lang#13890.
- Original fix PR is rust-lang#13903.
- Document test intent, backlink to rust-lang#13890 and fix PR rust-lang#13903.
- Fix the test logic: the `Makefile` version seems to not actually be exercising the "library search traverses symlink" logic, because the actual symlinked-to-library is present under the `$(TMPDIR)` directory tree when `bar.rs` is compiled, because the `$(RUSTC)` invocation has an implicit `-L $(TMPDIR)`. The symlink itself was actually broken, i.e. it should've been `ln -nsf $(TMPDIR)/outdir/$(NAME) $(TMPDIR)` but it used `ln -nsf outdir/$(NAME) $(TMPDIR)`. The rmake.rs version now explicitly separates the two directory trees and sets the CWD of the `bar.rs` rustc invocation so that the actual library is *not* present under its CWD tree.

I.e. it is now

```
$test_output/           # rustc foo.rs -o actual_lib_dir/libfoo.rlib
    actual_lib_dir/
        libfoo.rlib
    symlink_lib_dir/    # CWD set; rustc -L . bar.rs
        libfoo.rlib --> $test_output/actual_lib_dir/libfoo.rlib
```

Partially supersedes rust-lang#129011.
This PR is co-authored with `@Oneirical.`

r? compiler
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Dec 28, 2024
Rollup merge of rust-lang#134829 - jieyouxu:migrate-libs-through-symlinks, r=lqd

Migrate `libs-through-symlink` to rmake.rs

Part of rust-lang#121876.

This PR migrates `tests/run-make/libs-through-symlink/` to use rmake.rs.

- Regression test for rust-lang#13890.
- Original fix PR is rust-lang#13903.
- Document test intent, backlink to rust-lang#13890 and fix PR rust-lang#13903.
- Fix the test logic: the `Makefile` version seems to not actually be exercising the "library search traverses symlink" logic, because the actual symlinked-to-library is present under the `$(TMPDIR)` directory tree when `bar.rs` is compiled, because the `$(RUSTC)` invocation has an implicit `-L $(TMPDIR)`. The symlink itself was actually broken, i.e. it should've been `ln -nsf $(TMPDIR)/outdir/$(NAME) $(TMPDIR)` but it used `ln -nsf outdir/$(NAME) $(TMPDIR)`. The rmake.rs version now explicitly separates the two directory trees and sets the CWD of the `bar.rs` rustc invocation so that the actual library is *not* present under its CWD tree.

I.e. it is now

```
$test_output/           # rustc foo.rs -o actual_lib_dir/libfoo.rlib
    actual_lib_dir/
        libfoo.rlib
    symlink_lib_dir/    # CWD set; rustc -L . bar.rs
        libfoo.rlib --> $test_output/actual_lib_dir/libfoo.rlib
```

Partially supersedes rust-lang#129011.
This PR is co-authored with `@Oneirical.`

r? compiler
poliorcetics pushed a commit to poliorcetics/rust that referenced this pull request Dec 28, 2024
…inks, r=lqd

Migrate `libs-through-symlink` to rmake.rs

Part of rust-lang#121876.

This PR migrates `tests/run-make/libs-through-symlink/` to use rmake.rs.

- Regression test for rust-lang#13890.
- Original fix PR is rust-lang#13903.
- Document test intent, backlink to rust-lang#13890 and fix PR rust-lang#13903.
- Fix the test logic: the `Makefile` version seems to not actually be exercising the "library search traverses symlink" logic, because the actual symlinked-to-library is present under the `$(TMPDIR)` directory tree when `bar.rs` is compiled, because the `$(RUSTC)` invocation has an implicit `-L $(TMPDIR)`. The symlink itself was actually broken, i.e. it should've been `ln -nsf $(TMPDIR)/outdir/$(NAME) $(TMPDIR)` but it used `ln -nsf outdir/$(NAME) $(TMPDIR)`. The rmake.rs version now explicitly separates the two directory trees and sets the CWD of the `bar.rs` rustc invocation so that the actual library is *not* present under its CWD tree.

I.e. it is now

```
$test_output/           # rustc foo.rs -o actual_lib_dir/libfoo.rlib
    actual_lib_dir/
        libfoo.rlib
    symlink_lib_dir/    # CWD set; rustc -L . bar.rs
        libfoo.rlib --> $test_output/actual_lib_dir/libfoo.rlib
```

Partially supersedes rust-lang#129011.
This PR is co-authored with `@Oneirical.`

r? compiler
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
Status: In progress
Development

Successfully merging this pull request may close these issues.

5 participants