Rollup merge of #131477 - madsmtm:sdkroot-via-env-var, r=nnethercote

Apple: Always pass SDK root when linking with `cc`, and pass it via `SDKROOT` env var

Fixes https://github.com/rust-lang/rust/issues/80817, fixes https://github.com/rust-lang/rust/issues/96943, and generally simplifies our linker invocation on Apple platforms.

Part of https://github.com/rust-lang/rust/issues/129432.

### Necessary background on trampoline binaries

The developer binaries such as `/usr/bin/cc` and `/usr/bin/clang` are actually trampolines (similar in spirit to the Rust binaries in `~/.cargo/bin`) which effectively invokes `xcrun` to get the current Xcode developer directory, which allows it to find the actual binary under `/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/*`.

This binary is then launched with the following environment variables set (but none of them are set if `SDKROOT` is set explicitly):
- `SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk`
- `LIBRARY_PATH=/usr/local/lib` (appended)
- `CPATH=/usr/local/include` (appended)
- `MANPATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/share/man:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/usr/share/man:/Applications/Xcode.app/Contents/Developer/usr/share/man:/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/share/man:` (prepended)

This allows the user to type e.g. `clang foo.c` in their terminal on macOS, and have it automatically pick up a suitable Clang binary and SDK from either an installed Xcode.app or the Xcode Command Line Tools.
(It acts roughly as-if you typed `xcrun -sdk macosx clang foo.c`).

### Finding a suitable SDK

All compilation on macOS is cross-compilation using SDKs, there are no system headers any more (`/usr/include` is gone), and the system libraries are elsewhere in the file system (`/usr/lib` is basically empty). Instead, the logic for finding the SDK is handled by the `/usr/bin/cc` trampoline (see above).

But relying on the `cc` trampoline doesn't work when:
- Cross-compiling, since a different SDK is needed there.
- Invoking the linker directly, since the linker doesn't understand `SDKROOT`.
- Linking build scripts inside Xcode (see https://github.com/rust-lang/rust/issues/80817), since Xcode prepends `/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin` to `PATH`, which means `cc` refers to the _actual_ Clang binary, and we end up with the wrong SDK root specified.

Basically, we cannot rely on the trampoline at all, so the last commit removes the special-casing that was done when linking with `cc` for macOS (i.e. the most common path), so that **we now always invoke `xcrun` (if `SDKROOT` is not explicitly specified) to find the SDK root**.

Making sure this is non-breaking has a few difficulties though, namely that the user might not have Xcode installed, and that the compiler driver may not understand the `-isysroot` flag. These difficulties are explored below.

#### No Xcode

There are several compiler drivers which work without Xcode by bundling their own SDK, including `zig cc`, Nixpkgs' `clang` and Homebrew's `llvm` package. Additionally, `xcrun` is rarely available when cross-compiling from non-macOS and instead the user might provide a downloaded SDK manually with `-Clink-args=...`.

We do still want to _try_ to invoke `xcrun` if possible, since it is usually the SDK that the user wants (and if not, the environment should override `xcrun`, such as is done by Nixpkgs). But we do not want failure to invoke `xcrun` to stop the linking process. This is changed in the second-to-last commit.

#### `SDKROOT` vs. `-isysroot`

The exact reasoning why we do not always pass the SDK root when linking on macOS eludes me (the git history dead ends in rust-lang/rust#100286), but I suspect it's because we want to support compiler drivers which do not support the `-isysroot` option.

To make sure that such use-cases continue to work, we now pass the SDK root via the `SDKROOT` environment variable. This way, compiler drivers that support setting the SDK root (such as Clang and GCC) can use it, while compiler drivers that don't (presumably because they figure out the SDK in some other way) can just ignore it.

One small danger here would be if there's some compiler driver out there which works with the `-isysroot` flag, but not with the `SDKROOT` environment variable. I am not aware of any?

In a sense, this also shifts the blame; if a compiler driver does not understand `SDKROOT`, it won't work with e.g. `xcrun -sdk macosx15.0 $tool` either, so it can more clearly be argued that this is incorrect behaviour on the part of the tool.

Note also that this overrides the behaviour discussed above (`/usr/bin/cc` sets some extra environment variables), I will argue that is fine since `MANPATH` and `CPATH` is useless when linking, and `/usr/local/lib` is empty on a default system at least since macOS 10.14 (it might be filled by extra libraries installed by the user, but I'll argue that if we want it to be part of the default library search path, we should set it explicitly so that it's also set when linking with `-Clinker=ld`).

### Considered alternatives

- Invoke `/usr/bin/cc` instead of `cc`.
  - This breaks many other use-cases though where overriding `cc` in the PATH is desired.
- Look up `which cc`, and do special logic if in Xcode toolchain.
  - Seems brittle, and besides, it's not the `cc` in the Xcode toolchain that's wrong, it's the `/usr/bin/cc` behaviour that is a bit too magical.
- Invoke `xcrun --sdk macosx cc`.
  - This completely ignores `SDKROOT`, so we'd still have to parse that first to figure out if it's suitable or not, but would probably be workable.
- Maybe somehow configure the linker with extra flags such that it'll be able to link regardless of linking for macOS or e.g. iOS? Though I doubt this is possible.
- Bundle the SDK, similar to `zig-cc`.
  - Comes with it's own host of problems.

### Testing

Tested that this works with the following `-Clinker=...`:
- [x] Default (`cc`)
- [x] `/usr/bin/ld`
- [x] Actual Clang from Xcode (`/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang`)
- [x] `/usr/bin/clang` (invoked via `clang` instead of `cc`)
- [x] Homebrew's `llvm` package (ignores `SDKROOT`, uses their own SDK)
- [x] Homebrew's `gcc` package (`SDKROOT` is preferred over their own SDK)
- [x] ~Macports `clang`~ Couldn't get it to build
- [x] Macports `gcc` (`SDKROOT` is preferred over their own SDK)
- [x] Zig CC installed via. homebrew (ignores both `-isysroot` and `SDKROOT`, uses their own SDK)
- [x] Nixpkgs `clang` (ignores `SDKROOT`, uses their own SDK)
- [x] Nixpkgs `gcc` (ignores `SDKROOT`, uses their own SDK)
- [x] ~[`cosmocc`](https://github.com/jart/cosmopolitan)?~ Doesn't accept common flags (like `-arch`)

CC ```````@BlackHoleFox``````` ```````@thomcc```````

7 files changed, 115 insertions, 54 deletions

diff --git a/compiler/rustc_codegen_ssa/messages.ftl b/compiler/rustc_codegen_ssa/messages.ftl
index 3ca070acc9d..b6cfea88363 100644
--- a/compiler/rustc_codegen_ssa/messages.ftl
+++ b/compiler/rustc_codegen_ssa/messages.ftl
@@ -401,6 +401,9 @@ codegen_ssa_version_script_write_failure = failed to write version script: {$err
 
 codegen_ssa_visual_studio_not_installed = you may need to install Visual Studio build tools with the "C++ build tools" workload
 
+codegen_ssa_xcrun_about =
+    the SDK is needed by the linker to know where to find symbols in system libraries and for embedding the SDK version in the final object file
+
 codegen_ssa_xcrun_command_line_tools_insufficient =
     when compiling for iOS, tvOS, visionOS or watchOS, you need a full installation of Xcode
 
diff --git a/compiler/rustc_codegen_ssa/src/back/apple.rs b/compiler/rustc_codegen_ssa/src/back/apple.rs
index 2f68bad1695..2274450e20e 100644
--- a/compiler/rustc_codegen_ssa/src/back/apple.rs
+++ b/compiler/rustc_codegen_ssa/src/back/apple.rs
@@ -160,6 +160,10 @@ pub(super) fn add_version_to_llvm_target(
 pub(super) fn get_sdk_root(sess: &Session) -> Option<PathBuf> {
     let sdk_name = sdk_name(&sess.target);
 
+    // Attempt to invoke `xcrun` to find the SDK.
+    //
+    // Note that when cross-compiling from e.g. Linux, the `xcrun` binary may sometimes be provided
+    // as a shim by a cross-compilation helper tool. It usually isn't, but we still try nonetheless.
     match xcrun_show_sdk_path(sdk_name, sess.verbose_internals()) {
         Ok((path, stderr)) => {
             // Emit extra stderr, such as if `-verbose` was passed, or if `xcrun` emitted a warning.
@@ -169,7 +173,19 @@ pub(super) fn get_sdk_root(sess: &Session) -> Option<PathBuf> {
             Some(path)
         }
         Err(err) => {
-            let mut diag = sess.dcx().create_err(err);
+            // Failure to find the SDK is not a hard error, since the user might have specified it
+            // in a manner unknown to us (moreso if cross-compiling):
+            // - A compiler driver like `zig cc` which links using an internally bundled SDK.
+            // - Extra linker arguments (`-Clink-arg=-syslibroot`).
+            // - A custom linker or custom compiler driver.
+            //
+            // Though we still warn, since such cases are uncommon, and it is very hard to debug if
+            // you do not know the details.
+            //
+            // FIXME(madsmtm): Make this a lint, to allow deny warnings to work.
+            // (Or fix <https://github.com/rust-lang/rust/issues/21204>).
+            let mut diag = sess.dcx().create_warn(err);
+            diag.note(fluent::codegen_ssa_xcrun_about);
 
             // Recognize common error cases, and give more Rust-specific error messages for those.
             if let Some(developer_dir) = xcode_select_developer_dir() {
@@ -209,6 +225,8 @@ fn xcrun_show_sdk_path(
     sdk_name: &'static str,
     verbose: bool,
 ) -> Result<(PathBuf, String), XcrunError> {
+    // Intentionally invoke the `xcrun` in PATH, since e.g. nixpkgs provide an `xcrun` shim, so we
+    // don't want to require `/usr/bin/xcrun`.
     let mut cmd = Command::new("xcrun");
     if verbose {
         cmd.arg("--verbose");
@@ -280,7 +298,7 @@ fn stdout_to_path(mut stdout: Vec<u8>) -> PathBuf {
     }
     #[cfg(unix)]
     let path = <OsString as std::os::unix::ffi::OsStringExt>::from_vec(stdout);
-    #[cfg(not(unix))] // Unimportant, this is only used on macOS
-    let path = OsString::from(String::from_utf8(stdout).unwrap());
+    #[cfg(not(unix))] // Not so important, this is mostly used on macOS
+    let path = OsString::from(String::from_utf8(stdout).expect("stdout must be UTF-8"));
     PathBuf::from(path)
 }
diff --git a/compiler/rustc_codegen_ssa/src/back/link.rs b/compiler/rustc_codegen_ssa/src/back/link.rs
index 3ec0d900994..4ebe59dc2a7 100644
--- a/compiler/rustc_codegen_ssa/src/back/link.rs
+++ b/compiler/rustc_codegen_ssa/src/back/link.rs
@@ -3194,39 +3194,60 @@ fn add_apple_link_args(cmd: &mut dyn Linker, sess: &Session, flavor: LinkerFlavo
 }
 
 fn add_apple_sdk(cmd: &mut dyn Linker, sess: &Session, flavor: LinkerFlavor) -> Option<PathBuf> {
-    let os = &sess.target.os;
-    if sess.target.vendor != "apple"
-        || !matches!(os.as_ref(), "ios" | "tvos" | "watchos" | "visionos" | "macos")
-        || !matches!(flavor, LinkerFlavor::Darwin(..))
-    {
+    if !sess.target.is_like_darwin {
         return None;
     }
-
-    if os == "macos" && !matches!(flavor, LinkerFlavor::Darwin(Cc::No, _)) {
+    let LinkerFlavor::Darwin(cc, _) = flavor else {
         return None;
-    }
-
-    let sdk_root = sess.time("get_apple_sdk_root", || get_apple_sdk_root(sess))?;
+    };
 
-    match flavor {
-        LinkerFlavor::Darwin(Cc::Yes, _) => {
-            // Use `-isysroot` instead of `--sysroot`, as only the former
-            // makes Clang treat it as a platform SDK.
-            //
-            // This is admittedly a bit strange, as on most targets
-            // `-isysroot` only applies to include header files, but on Apple
-            // targets this also applies to libraries and frameworks.
-            cmd.cc_arg("-isysroot");
-            cmd.cc_arg(&sdk_root);
-        }
-        LinkerFlavor::Darwin(Cc::No, _) => {
-            cmd.link_arg("-syslibroot");
-            cmd.link_arg(&sdk_root);
-        }
-        _ => unreachable!(),
+    // The default compiler driver on macOS is at `/usr/bin/cc`. This is a trampoline binary that
+    // effectively invokes `xcrun cc` internally to look up both the compiler binary and the SDK
+    // root from the current Xcode installation. When cross-compiling, when `rustc` is invoked
+    // inside Xcode, or when invoking the linker directly, this default logic is unsuitable, so
+    // instead we invoke `xcrun` manually.
+    //
+    // (Note that this doesn't mean we get a duplicate lookup here - passing `SDKROOT` below will
+    // cause the trampoline binary to skip looking up the SDK itself).
+    let sdkroot = sess.time("get_apple_sdk_root", || get_apple_sdk_root(sess))?;
+
+    if cc == Cc::Yes {
+        // There are a few options to pass the SDK root when linking with a C/C++ compiler:
+        // - The `--sysroot` flag.
+        // - The `-isysroot` flag.
+        // - The `SDKROOT` environment variable.
+        //
+        // `--sysroot` isn't actually enough to get Clang to treat it as a platform SDK, you need
+        // to specify `-isysroot`. This is admittedly a bit strange, as on most targets `-isysroot`
+        // only applies to include header files, but on Apple targets it also applies to libraries
+        // and frameworks.
+        //
+        // This leaves the choice between `-isysroot` and `SDKROOT`. Both are supported by Clang and
+        // GCC, though they may not be supported by all compiler drivers. We choose `SDKROOT`,
+        // primarily because that is the same interface that is used when invoking the tool under
+        // `xcrun -sdk macosx $tool`.
+        //
+        // In that sense, if a given compiler driver does not support `SDKROOT`, the blame is fairly
+        // clearly in the tool in question, since they also don't support being run under `xcrun`.
+        //
+        // Additionally, `SDKROOT` is an environment variable and thus optional. It also has lower
+        // precedence than `-isysroot`, so a custom compiler driver that does not support it and
+        // instead figures out the SDK on their own can easily do so by using `-isysroot`.
+        //
+        // (This in particular affects Clang built with the `DEFAULT_SYSROOT` CMake flag, such as
+        // the one provided by some versions of Homebrew's `llvm` package. Those will end up
+        // ignoring the value we set here, and instead use their built-in sysroot).
+        cmd.cmd().env("SDKROOT", &sdkroot);
+    } else {
+        // When invoking the linker directly, we use the `-syslibroot` parameter. `SDKROOT` is not
+        // read by the linker, so it's really the only option.
+        //
+        // This is also what Clang does.
+        cmd.link_arg("-syslibroot");
+        cmd.link_arg(&sdkroot);
     }
 
-    Some(sdk_root)
+    Some(sdkroot)
 }
 
 fn get_apple_sdk_root(sess: &Session) -> Option<PathBuf> {
@@ -3255,7 +3276,13 @@ fn get_apple_sdk_root(sess: &Session) -> Option<PathBuf> {
             }
             "macosx"
                 if sdkroot.contains("iPhoneOS.platform")
-                    || sdkroot.contains("iPhoneSimulator.platform") => {}
+                    || sdkroot.contains("iPhoneSimulator.platform")
+                    || sdkroot.contains("AppleTVOS.platform")
+                    || sdkroot.contains("AppleTVSimulator.platform")
+                    || sdkroot.contains("WatchOS.platform")
+                    || sdkroot.contains("WatchSimulator.platform")
+                    || sdkroot.contains("XROS.platform")
+                    || sdkroot.contains("XRSimulator.platform") => {}
             "watchos"
                 if sdkroot.contains("WatchSimulator.platform")
                     || sdkroot.contains("MacOSX.platform") => {}
diff --git a/compiler/rustc_target/src/spec/base/apple/mod.rs b/compiler/rustc_target/src/spec/base/apple/mod.rs
index da9f96ce37d..ecc74264160 100644
--- a/compiler/rustc_target/src/spec/base/apple/mod.rs
+++ b/compiler/rustc_target/src/spec/base/apple/mod.rs
@@ -1,5 +1,4 @@
 use std::borrow::Cow;
-use std::env;
 use std::fmt::{Display, from_fn};
 use std::num::ParseIntError;
 use std::str::FromStr;
@@ -209,29 +208,10 @@ fn link_env_remove(os: &'static str) -> StaticCow<[StaticCow<str>]> {
     // that's only applicable to cross-OS compilation. Always leave anything for the
     // host OS alone though.
     if os == "macos" {
-        let mut env_remove = Vec::with_capacity(2);
-        // Remove the `SDKROOT` environment variable if it's clearly set for the wrong platform, which
-        // may occur when we're linking a custom build script while targeting iOS for example.
-        if let Ok(sdkroot) = env::var("SDKROOT") {
-            if sdkroot.contains("iPhoneOS.platform")
-                || sdkroot.contains("iPhoneSimulator.platform")
-                || sdkroot.contains("AppleTVOS.platform")
-                || sdkroot.contains("AppleTVSimulator.platform")
-                || sdkroot.contains("WatchOS.platform")
-                || sdkroot.contains("WatchSimulator.platform")
-                || sdkroot.contains("XROS.platform")
-                || sdkroot.contains("XRSimulator.platform")
-            {
-                env_remove.push("SDKROOT".into())
-            }
-        }
-        // Additionally, `IPHONEOS_DEPLOYMENT_TARGET` must not be set when using the Xcode linker at
+        // `IPHONEOS_DEPLOYMENT_TARGET` must not be set when using the Xcode linker at
         // "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld",
         // although this is apparently ignored when using the linker at "/usr/bin/ld".
-        env_remove.push("IPHONEOS_DEPLOYMENT_TARGET".into());
-        env_remove.push("TVOS_DEPLOYMENT_TARGET".into());
-        env_remove.push("XROS_DEPLOYMENT_TARGET".into());
-        env_remove.into()
+        cvs!["IPHONEOS_DEPLOYMENT_TARGET", "TVOS_DEPLOYMENT_TARGET", "XROS_DEPLOYMENT_TARGET"]
     } else {
         // Otherwise if cross-compiling for a different OS/SDK (including Mac Catalyst), remove any part
         // of the linking environment that's wrong and reversed.
diff --git a/src/doc/unstable-book/src/compiler-environment-variables/SDKROOT.md b/src/doc/unstable-book/src/compiler-environment-variables/SDKROOT.md
index be9ed02f54d..6d371ae289f 100644
--- a/src/doc/unstable-book/src/compiler-environment-variables/SDKROOT.md
+++ b/src/doc/unstable-book/src/compiler-environment-variables/SDKROOT.md
@@ -1,6 +1,6 @@
 # `SDKROOT`
 
 This environment variable is used on Apple targets.
-It is passed through to the linker (currently either as `-isysroot` or `-syslibroot`).
+It is passed through to the linker (currently either directly or via the `-syslibroot` flag).
 
 Note that this variable is not always respected. When the SDKROOT is clearly wrong (e.g. when the platform of the SDK does not match the `--target` used by rustc), this is ignored and rustc does its own detection.
diff --git a/tests/run-make/link-under-xcode/foo.rs b/tests/run-make/link-under-xcode/foo.rs
new file mode 100644
index 00000000000..f328e4d9d04
--- /dev/null
+++ b/tests/run-make/link-under-xcode/foo.rs
@@ -0,0 +1 @@
+fn main() {}
diff --git a/tests/run-make/link-under-xcode/rmake.rs b/tests/run-make/link-under-xcode/rmake.rs
new file mode 100644
index 00000000000..c9394feb000
--- /dev/null
+++ b/tests/run-make/link-under-xcode/rmake.rs
@@ -0,0 +1,32 @@
+//! Test that linking works under an environment similar to what Xcode sets up.
+//!
+//! Regression test for https://github.com/rust-lang/rust/issues/80817.
+
+//@ only-apple
+
+use run_make_support::{cmd, rustc, target};
+
+fn main() {
+    // Fetch toolchain `/usr/bin` directory. Usually:
+    // /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
+    let clang_bin = cmd("xcrun").arg("--find").arg("clang").run().stdout_utf8();
+    let toolchain_bin = clang_bin.trim().strip_suffix("/clang").unwrap();
+
+    // Put toolchain directory at the front of PATH.
+    let path = format!("{}:{}", toolchain_bin, std::env::var("PATH").unwrap());
+
+    // Check that compiling and linking still works.
+    //
+    // Removing `SDKROOT` is necessary for the test to excercise what we want, since bootstrap runs
+    // under `/usr/bin/python3`, which will set SDKROOT for us.
+    rustc().target(target()).env_remove("SDKROOT").env("PATH", &path).input("foo.rs").run();
+
+    // Also check linking directly with the system linker.
+    rustc()
+        .target(target())
+        .env_remove("SDKROOT")
+        .env("PATH", &path)
+        .input("foo.rs")
+        .arg("-Clinker-flavor=ld")
+        .run();
+}