Straightening up messy Apple compilations

[dot-asm]

I would like to make a case for instead of endlessly struggling to map all those Apple-specific environment variables, e.g. like in #661 and #662, one would simply let xcrun do all the work. I mean instead of calling composing all sorts of command line options and passing them to cc, one would simply call xcrun -sdk [iphoneos|iphonesimulator|watchos|etc.] cc -arch [arm64|x86_64|armv7]. In other words, cc-rs would have to simply map Rust target triplet to a combination of -sdk and -arch parameters.

[dot-asm]

Moving https://github.com/alexcrichton/cc-rs/pull/661#issuecomment-1067508870 here. @indygreg, @dvc94ch, @vladimir-ea.

Tangentially related to this, I've implemented Apple SDK discovery in pure Rust in https://github.com/indygreg/PyOxidizer/blob/ae848c9083c9c8563d5890f5d39a8e04a1d71832/tugger-apple/src/sdk.rs. This includes parsing the SDKSettings.json files to extract SDK metadata such as versions and which targets are supported. Like xcode-select and xcrun, it supports honoring DEVELOPER_DIR for finding the install root of SDKs. There's also some related code at https://github.com/indygreg/PyOxidizer/blob/ae848c9083c9c8563d5890f5d39a8e04a1d71832/pyoxidizer/src/environment.rs#L531 (and elsewhere in this file) for attempting to find an appropriate SDK to use given constraints. (This code should arguably be moved to the aforementioned crate/module.)

This code probably isn't suitable for inclusion in this crate as-is. But if you want me to extract it to its own crate or incorporate it into cc-rs somehow, let's talk (in another issue presumably).

As for this PR, externally specifying SDKROOT to force the Apple SDK to use seems reasonable to me. This is what I've done with PyOxidizer to allow users to override the built-in/default SDK finding. I say this without knowing how the Rust compiler internally handles SDKROOT, so I may be missing some important context! But my understanding is that build tools commonly export SDKROOT to force use of a specific SDK to avoid downstream tools performing their own SDK discovery.

[dot-asm]

But if you want me to extract it to its own crate or incorporate it into cc-rs somehow, let's talk

First of all I want to clarify that I'm just a cc-rs user responding to #663. Well, obviously in my own way, but nevertheless. #663 [among other things] effectively reads "keep it manageable" and I can't agree more.

[dot-asm]

externally specifying SDKROOT to force the Apple SDK to use seems reasonable to me.

xcrun respects externally specified SDKROOT, so what's the problem? It "kicks in" only when you don't specify one and finds one suitable for --sdk [iphoneos|phonesimulator|watchos|etc.]

I say this without knowing how the Rust compiler internally handles SDKROOT,

It doesn't matter for cc-rs. Its job is to compile suitable .o module, that's it.

[dot-asm]

Tangentially related to this, I've implemented Apple SDK discovery in pure Rust.

And the question in the context would be what would it take to implement xcrun replacement? Would it make more sense than to make cc-rs less manageable?

[Recall that the "tangent" effectively is "cross-compiling on non-MacOS host."]

[dot-asm]

xcrun respects externally specified SDKROOT, so what's the problem?

This might be formally incorrect or misleading statement. xcrun "respects" SDKROOT only if you don't specify --sdk flag. In other words, xcrun cc ... sets SDKROOT to MacOSX.sdk only if SDKROOT is not set, while xcrun --sdk macosx cc ... always sets SDKROOT to MacOSX.sdk.

[dvc94ch]

Don't think there is much point discussing this before #663 is resolved. I don't think this solution conflicts with the fairly simple and straightforward setting an environment variable solution.

[MikeChongCan]

@dvc94ch now #663 seems to be resolved. Could we move on this discussion?

I'm happy to do whatever I can to help the iteration on mobile platforms.

[dvc94ch]

So to summarize the behaviour of xcrun (what @indygreg said):

1. if SDKROOT is set return that
2. if DEVELOPER_DIR is set select an sdk from developer dir
3. use default paths from xcode to find an sdk

On linux/windows we can then set the SDKROOT or DEVELOPER_DIR and it should just work. Extracting that logic into a crate seems reasonable. I don't think we need a binary called xcrun if the resulting behaviour is the same (use same environment variables). Not sure what the policy is of cc-rs on external crates, maybe it needs to be added to cc-rs.

[jurvis]

+1 will love to see this change landed. let me know how I can help!

(rust-bitcoin/rust-secp256k1#443)

[dvc94ch]

This code probably isn't suitable for inclusion in this crate as-is. But if you want me to extract it to its own crate or incorporate it into cc-rs somehow, let's talk (in another issue presumably).

@indygreg does your offer to factor out the sdk selection logic into a separate crate still stand?

[indygreg]

What do you think about https://github.com/indygreg/toolchain-tools/tree/main/apple-sdk?

This is effectively what exists in tugger-apple before - just in its own crate with slightly better error handling and optional dependencies to jettison the JSON and plist parsing for people who don't need it.

It still needs support for SDKROOT handling. Want to send a PR?

[indygreg]

... and I guess I need to port some of PyOxidizer's code for finding an appropriate SDK to use. That should be easy enough. But it may not happen for a few days since I'm busy this week.

I also have to think about how to do targeting without dependency bloat. Right now we pull in a few dozen crates (via plist and serde_json) so we can parse SDKSettings.json and SDKSettings.plist. You need to parse these files in order to do SDK selection / targeting properly. But I imagine there's a simpler/faster solution based purely on parsing version components in directory names that could be employed instead. It isn't as robust but it likely gets the job done.