sort: fix panic on non-UTF-8 filenames with --files0-from

[ppmpreetham]

Re-opening this to fix the hidden character warning in the previous branch name. Supersedes #11592.

Fixes #9696

Description

When parsing NUL-separated filenames via --files0-from, sort previously enforced strict UTF-8 validation using std::str::from_utf8().expect(...). This caused an immediate panic when encountering valid non-UTF-8 paths. GNU sort treats filenames as raw bytes and does not require them to be UTF-8 encoded.

This PR aligns uutils with GNU behavior by removing the strict UTF-8 enforcement:

• On Unix: We now use OsStr::from_bytes to losslessly cast the raw bytes directly into an OsString, preserving arbitrary byte sequences exactly as GNU does.
• On non-Unix (e.g., Windows): We safely fall back to String::from_utf8_lossy. This prevents the program from panicking or failing on invalid UTF-8 sequences, ensuring graceful error handling across all platforms.

Testing

Reproduced the issue and verified the fix locally (both linux and windows):

$ printf "20\n10\n" > "weird$(printf '\xff')name"
$ printf "weird$(printf '\xff')name\0" > list0
$ ./target/release/coreutils sort --files0-from=list0
10
20

[cakebaker]

Can you please add a test to tests/by_util/test_sort.rs to ensure we don't regress in the future? Thanks.

[github-actions]

GNU testsuite comparison:

GNU test failed: tests/tail/tail-n0f. tests/tail/tail-n0f is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tail/inotify-dir-recreate (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tty/tty-eof (passes in this run but fails in the 'main' branch)

[ppmpreetham]

added a test for the same

[codspeed-hq]

Merging this PR will not alter performance

✅ 305 untouched benchmarks
⏩ 46 skipped benchmarks¹

---

_{Comparing ppmpreetham:fix/9692-Panics-on-Non-UTF-8 (afcd762) with main (e510449)}

Footnotes

1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[oech3]

please fix clippy

  error: unused import: `OsStringExt`
    --> src/uu/sort/src/sort.rs:42:36
     |
  42 | use std::os::unix::ffi::{OsStrExt, OsStringExt};
     |                                    ^^^^^^^^^^^
     |
     = note: `-D unused-imports` implied by `-D warnings`
     = help: to override `-D warnings` add `#[allow(unused_imports)]`

[github-actions]

GNU testsuite comparison:

Skip an intermittent issue tests/pr/bounded-memory (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/tail/pipe-f is now passing!

[RenjiSann]

Suggested change

	let filename = std::ffi::OsString::from_vec(vec![b'a', 0xFF, b'b']);
	let filename = std::ffi::OsString::from_vec(b"a\xffb".into());

Something like this would be more readable

[RenjiSann]

Remove the brackets

[RenjiSann]

Suggested change

	#[cfg(not(unix))]
	let f_str = String::from_utf8_lossy(&line);
	#[cfg(not(unix))]
	let f = OsStr::new(f_str.as_ref());
	#[cfg(not(unix))]
	let f = {
	let f_str = String::from_utf8_lossy(&line);
	OsStr::new(f_str.as_ref())
	};

Avoid writing #[cfg(not(unix))] more than once

[RenjiSann]

Not sure why you can't keep the match here

[sylvestre]

some jobs are failing btw :)