illumos should put libc last in library search order

Under some conditions, the toolchain will produce a sequence of linker
arguments that result in a NEEDED list that puts libc before libgcc_s;
e.g.,

    [0]  NEEDED            0x2046ba            libc.so.1
    [1]  NEEDED            0x204723            libm.so.2
    [2]  NEEDED            0x204736            libsocket.so.1
    [3]  NEEDED            0x20478b            libumem.so.1
    [4]  NEEDED            0x204763            libgcc_s.so.1

Both libc and libgcc_s provide an unwinder implementation, but libgcc_s
provides some extra symbols upon which Rust directly depends.  If libc
is first in the NEEDED list we will find some of those symbols in libc
but others in libgcc_s, resulting in undefined behaviour as the two
implementations do not use compatible interior data structures.

This solution is not perfect, but is the simplest way to produce correct
binaries on illumos for now.

2 files changed, 19 insertions, 0 deletions

diff --git a/compiler/rustc_codegen_ssa/src/back/linker.rs b/compiler/rustc_codegen_ssa/src/back/linker.rs
index 401d379b0d1..a66de89f681 100644
--- a/compiler/rustc_codegen_ssa/src/back/linker.rs
+++ b/compiler/rustc_codegen_ssa/src/back/linker.rs
@@ -340,6 +340,14 @@ impl<'a> Linker for GccLinker<'a> {
     }
 
     fn link_dylib(&mut self, lib: Symbol, verbatim: bool, as_needed: bool) {
+        if self.sess.target.os == "illumos" && lib.as_str() == "c" {
+            // libc will be added via late_link_args on illumos so that it will
+            // appear last in the library search order.
+            // FIXME: This should be replaced by a more complete and generic
+            // mechanism for controlling the order of library arguments passed
+            // to the linker.
+            return;
+        }
         if !as_needed {
             if self.sess.target.is_like_osx {
                 // FIXME(81490): ld64 doesn't support these flags but macOS 11
diff --git a/compiler/rustc_target/src/spec/illumos_base.rs b/compiler/rustc_target/src/spec/illumos_base.rs
index 2e365d210f3..2b8e046c46b 100644
--- a/compiler/rustc_target/src/spec/illumos_base.rs
+++ b/compiler/rustc_target/src/spec/illumos_base.rs
@@ -6,6 +6,17 @@ pub fn opts() -> TargetOptions {
     late_link_args.insert(
         LinkerFlavor::Gcc,
         vec![
+            // The illumos libc contains a stack unwinding implementation, as
+            // does libgcc_s.  The latter implementation includes several
+            // additional symbols that are not always in base libc.  To force
+            // the consistent use of just one unwinder, we ensure libc appears
+            // after libgcc_s in the NEEDED list for the resultant binary by
+            // ignoring any attempts to add it as a dynamic dependency until the
+            // very end.
+            // FIXME: This should be replaced by a more complete and generic
+            // mechanism for controlling the order of library arguments passed
+            // to the linker.
+            "-lc".to_string(),
             // LLVM will insert calls to the stack protector functions
             // "__stack_chk_fail" and "__stack_chk_guard" into code in native
             // object files.  Some platforms include these symbols directly in