Apply runtime library workaround for snapshot toolchains on macOS #484

omochi · 2024-06-10T01:04:30Z

swiftlang/swift#73327 (comment) のハックを Carton に組み込む

動作の様子

コード署名し直すことを黄色で予告して、コマンドラインを表示しました。
また、ついでに見た目の改善としてコマンドライン実行時の不足している改行を追加したり、余計な空行を削除したりしました。

kateinoigakukun · 2024-06-10T03:39:10Z

Sources/SwiftToolchain/ToolchainInstallation.swift

+      guard fileSystem.isFile(file) else { continue }
+
+      let process = Foundation.Process()
+      process.executableURL = URL(fileURLWithPath: "/usr/bin/sudo")


We don't need sudo as long as the toolchain is installed under ~/Library/Toolchain/ if I remember correctly, and we can assume it's installed there by carton

あ、そうなの
試してみます

omochi

self comment

omochi · 2024-06-10T04:58:29Z

Sources/CartonCore/FoundationProcessEx.swift

    printsLoadingMessage: Bool = true,
    forwardExit: Bool = false
  ) throws {
    let process = Foundation.Process()
    process.executableURL = executableURL
    process.arguments = arguments
+    process.environment = environment


checkRunに環境変数の設定する口を開けました。

omochi · 2024-06-10T04:59:04Z

Sources/CartonDriver/CartonDriverCommand.swift

  let pluginArguments = try derivePackageCommandArguments(
    swiftExec: swiftExec,
    subcommand: subcommand,
    scratchPath: scratchPath.path,
    extraArguments: extraArguments
  )

+  var env: [String: String] = ProcessInfo.processInfo.environment
+  if ToolchainSystem.isSnapshotVersion(swiftPath.verison),
+     swiftPath.toolchain.extension == "xctoolchain"


mac環境かつsnapshotなら、DYLD... を設定します。

omochi · 2024-06-10T04:59:55Z

Sources/CartonHelpers/DefaultToolchain.swift

@@ -12,7 +12,9 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

-#if compiler(>=5.10)
+#if compiler(>=6.0)
+public let defaultToolchainVersion = "wasm-6.0-SNAPSHOT-2024-06-08-a"


6.0のsnapshotを追加します。
CIのSwift6.0が swift-6.0-DEVELOPMENT-SNAPSHOT-2024-06-07-a になっていて、
それと対応する wasm swift にしています。
だから1日ズレています。

omochi · 2024-06-10T05:00:26Z

Sources/CartonHelpers/Process+run.swift

@@ -94,7 +94,6 @@ extension Process {

          switch result.map(\.exitStatus) {
          case .success(.terminated(code: EXIT_SUCCESS)):
-            terminal.write("\n")


この改行が不要そうでした。

omochi · 2024-06-10T05:00:36Z

Sources/CartonHelpers/Process+run.swift

@@ -103,7 +102,7 @@ extension Process {
              terminal.write(stdoutBuffer)
            }
            terminal.write(
-              "\n`\(processName)` process finished successfully\n",


この冒頭の改行も不要だと思いました。

omochi · 2024-06-10T05:02:25Z

Sources/SwiftToolchain/ToolchainInstallation.swift

@@ -65,6 +66,8 @@ extension ToolchainSystem {
        stream: stdoutStream,
        header: "Downloading the archive"
      )
+      defer { terminal.write("\n") }


ダウンロードが完了した時
Download completed successfully
のメッセージが saving to ... の右に表示されていておかしかったので、改行を入れます。

ここはターミナルがアニメーションするループがあるので、
エラーの場合もうまくいくように
defer でここに書きます。

omochi · 2024-06-10T05:02:51Z

Sources/SwiftToolchain/ToolchainInstallation.swift

@@ -109,6 +112,32 @@ extension ToolchainSystem {
    terminal.logLookup("Unpacking the archive: ", arguments.joined(separator: " "))
    try await Process.run(arguments, terminal)

+    if ext == "pkg", Self.isSnapshotVersion(version) {
+      try await patchSnapshotForMac(path: installationPath, terminal: terminal)


インストール後にmacでsnapshotならパッチ(再署名)します。

omochi · 2024-06-10T05:03:25Z

Sources/SwiftToolchain/ToolchainInstallation.swift

+    for file in try fileSystem.traverseRecursively(binDir) {
+      guard fileSystem.isFile(file) else { continue }
+
+      try Foundation.Process.checkRun(


出力がスッキリするので Foundation.Process.checkRun を使いました。

omochi · 2024-06-10T05:04:00Z

Sources/SwiftToolchain/ToolchainManagement.swift

+    public var verison: String
+    public var swift: AbsolutePath
+    public var toolchain: AbsolutePath
+  }


従来、swiftとバージョンのタプルでしたが、
ツールチェーンディレクトリも使いたくなったので追加します。

ついでにタプルをやめて型にすることで利用側のコードを読みやすくします。

omochi · 2024-06-10T05:06:27Z

あれ、CI失敗だ

error: 'testapp': Invalid manifest (compiled with: ["/Users/runner/Library/Developer/Toolchains/swift-wasm-5.10.0-RELEASE.xctoolchain/usr/bin/swiftc", "-vfsoverlay", "/var/folders/lr/439_fwvd3m76p9vy50d57kcc0000gn/T/TemporaryDirectory.NZeeV5/vfs.yaml", "-L", "/Users/runner/Library/Developer/Toolchains/swift-wasm-5.10.0-RELEASE.xctoolchain/usr/lib/swift/pm/ManifestAPI", "-lPackageDescription", "-Xlinker", "-rpath", "-Xlinker", "/Users/runner/Library/Developer/Toolchains/swift-wasm-5.10.0-RELEASE.xctoolchain/usr/lib/swift/pm/ManifestAPI", "-target", "arm64-apple-macosx12.0", "-sdk", "/Applications/Xcode_15.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk", "-F", "/Applications/Xcode_15.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks", "-I", "/Applications/Xcode_15.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib", "-L", "/Applications/Xcode_15.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib", "-swift-version", "5", "-I", "/Users/runner/Library/Developer/Toolchains/swift-wasm-5.10.0-RELEASE.xctoolchain/usr/lib/swift/pm/ManifestAPI", "-sdk", "/Applications/Xcode_15.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk", "-package-description-version", "5.3.0", "/Users/runner/work/carton/carton/Tests/Fixtures/TestApp/Package.swift", "-Xfrontend", "-disable-implicit-concurrency-module-import", "-Xfrontend", "-disable-implicit-string-processing-module-impor
t", "-o", "/var/folders/lr/439_fwvd3m76p9vy50d57kcc0000gn/T/TemporaryDirectory.ivLEoP/testapp-manifest"])
<unknown>:0: error: unable to execute command: <unknown>
Swift/ErrorType.swift:200: Fatal error: Error raised at top level: Process failed with status 1.

omochi · 2024-06-10T05:57:53Z

Sources/CartonCore/FoundationProcessEx.swift

    printsLoadingMessage: Bool = true,
    forwardExit: Bool = false
  ) throws {
    let process = Foundation.Process()
    process.executableURL = executableURL
    process.arguments = arguments
+    if let environment {
+      process.environment = environment


環境変数を設定する口を開けました。
nil をセットすると、
継承するデフォルト挙動が消えるようなので、
nilの時はifで避けます。

参考:
https://developer.apple.com/documentation/foundation/process/1409412-environment

If this method isn’t used, the environment is inherited from the process that created the receiver

omochi · 2024-06-10T09:27:06Z

だめだあ
手元でちゃんとみてみよう

omochi · 2024-06-10T15:35:43Z

Sources/CartonFrontend/Commands/TestRunners/CommandTestRunner.swift

+    } else {
+      let programName = (program as NSString).lastPathComponent
+      if programName == "wasmtime" {
+        arguments += ["--dir", "."]


wasmtimeのpreopenの問題の対応

omochi mentioned this pull request Jun 10, 2024

Add Swift 6.0 CI for building carton for macOS #483

Merged

kateinoigakukun reviewed Jun 10, 2024

View reviewed changes

omochi commented Jun 10, 2024

View reviewed changes

omochi requested a review from kateinoigakukun June 10, 2024 05:04

omochi changed the title ~~[WIP] macにsnapshot版のツールチェーンを入れるときは9999問題を回避する作業をする~~ macにsnapshot版のツールチェーンを入れるときは9999問題を回避する作業をする Jun 10, 2024

omochi marked this pull request as ready for review June 10, 2024 05:05

omochi commented Jun 10, 2024

View reviewed changes

omochi marked this pull request as draft June 10, 2024 08:25

omochi added 14 commits June 10, 2024 23:49

patch mac snapshot

56e254e

comment

2f9acd9

set DYLD_LIBRARY_PATH if need

502ae93

reset stdin pgrp

46b6e2b

revert wip default toolchain

dcc8dec

fix test

861e5c7

desudo

d9fb1d2

avoid setting nil

8d7edd1

link RegexParser

8835ffd

swift_StringProcessing

ad8f613

preopen on wasmtime

1a408ec

fix bug in test runner option

f254601

apply patch also command mode

af42678

cap hack version

b398cfe

omochi force-pushed the patch-mac-snapshot branch from 17a436f to b398cfe Compare June 10, 2024 14:51

omochi marked this pull request as ready for review June 10, 2024 15:34

omochi commented Jun 10, 2024

View reviewed changes

kateinoigakukun approved these changes Jun 10, 2024

View reviewed changes

kateinoigakukun changed the title ~~macにsnapshot版のツールチェーンを入れるときは9999問題を回避する作業をする~~ Apply runtime library workaround for snapshot toolchains on macOS Jun 10, 2024

kateinoigakukun merged commit 79b686c into swiftwasm:main Jun 10, 2024
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apply runtime library workaround for snapshot toolchains on macOS #484

Apply runtime library workaround for snapshot toolchains on macOS #484

omochi commented Jun 10, 2024 •

edited

Loading

kateinoigakukun Jun 10, 2024

omochi Jun 10, 2024

omochi left a comment

omochi Jun 10, 2024

omochi Jun 10, 2024

omochi Jun 10, 2024

omochi Jun 10, 2024

omochi Jun 10, 2024

omochi Jun 10, 2024

omochi Jun 10, 2024

omochi Jun 10, 2024

omochi Jun 10, 2024

omochi commented Jun 10, 2024

omochi Jun 10, 2024 •

edited

Loading

omochi commented Jun 10, 2024

omochi Jun 10, 2024

Apply runtime library workaround for snapshot toolchains on macOS #484

Apply runtime library workaround for snapshot toolchains on macOS #484

Conversation

omochi commented Jun 10, 2024 • edited Loading

動作の様子

Choose a reason for hiding this comment

Choose a reason for hiding this comment

omochi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

omochi commented Jun 10, 2024

omochi Jun 10, 2024 • edited Loading

Choose a reason for hiding this comment

omochi commented Jun 10, 2024

Choose a reason for hiding this comment

omochi commented Jun 10, 2024 •

edited

Loading

omochi Jun 10, 2024 •

edited

Loading