Rollup merge of #130630 - taiki-e:s390x-clobber-abi, r=Amanieu

Support clobber_abi and vector/access registers (clobber-only) in s390x inline assembly

This supports `clobber_abi` which is one of the requirements of stabilization mentioned in #93335.

This also supports vector registers (as `vreg`) and access registers (as `areg`) as clobber-only, which need to support clobbering of them to implement clobber_abi.

Refs:
- "1.2.1.1. Register Preservation Rules" section in ELF Application Binary Interface s390x Supplement, Version 1.6.1 (lzsabi_s390x.pdf in https://github.com/IBM/s390x-abi/releases/tag/v1.6.1)
- Register definition in LLVM:
  - Vector registers https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.td#L249
  - Access registers https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.td#L332

I have three questions:
- ~~ELF Application Binary Interface s390x Supplement says that `cc` (condition code, bits 18-19 of PSW) is "Volatile".
  However, we do not have a register class for `cc` and instead mark `cc` as clobbered unless `preserves_flags` is specified (https://github.com/rust-lang/rust/pull/111331).
  Therefore, in the current implementation, if both `preserves_flags` and `clobber_abi` are specified, `cc` is not marked as clobbered. Is this okay? Or even if `preserves_flags` is used, should `cc` be marked as clobbered if `clobber_abi` is used?~~ UPDATE: resolved https://github.com/rust-lang/rust/pull/130630#issuecomment-2367923121
- ~~ELF Application Binary Interface s390x Supplement says that `pm` (program mask, bits 20-23 of PSW) is "Cleared".
  There does not appear to be any registers associated with this in either [LLVM](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.td) or [GCC](https://github.com/gcc-mirror/gcc/blob/33ccc1314dcdb0b988a9276ca6b6ce9b07bea21e/gcc/config/s390/s390.h#L407-L431), so at this point I don't see any way other than to just ignore it. Is this okay as-is?~~ UPDATE: resolved https://github.com/rust-lang/rust/pull/130630#issuecomment-2367923121
- Is "areg" a good name for register class name for access registers? It may be a bit confusing between that and `reg_addr`, which uses the “a” constraint (https://github.com/rust-lang/rust/pull/119431)...

Note:

- GCC seems to [recognize only `a0` and `a1`](https://github.com/gcc-mirror/gcc/blob/33ccc1314dcdb0b988a9276ca6b6ce9b07bea21e/gcc/config/s390/s390.h#L428-L429), and using `a[2-15]` [causes errors](https://godbolt.org/z/a46vx8jjn).
  Given that cg_gcc has a similar problem with other architecture (https://github.com/rust-lang/rustc_codegen_gcc/issues/485), I don't feel this is a blocker for this PR, but it is worth mentioning here.
- `vreg` should be able to accept `#[repr(simd)]` types as input if the `vector` target feature added in https://github.com/rust-lang/rust/pull/127506 is enabled, but core_arch has no s390x vector type and both `#[repr(simd)]` and `core::simd` are unstable, so I have not implemented it in this PR. EDIT: And supporting it is probably more complex than doing the equivalent on other architectures... https://github.com/rust-lang/rust/pull/88245#issuecomment-905559591

cc `@uweigand`

r? `@Amanieu`

`@rustbot` label +O-SystemZ

7 files changed, 200 insertions, 12 deletions

diff --git a/compiler/rustc_codegen_gcc/src/asm.rs b/compiler/rustc_codegen_gcc/src/asm.rs
index 13a00f7e08d..a04cd4735f4 100644
--- a/compiler/rustc_codegen_gcc/src/asm.rs
+++ b/compiler/rustc_codegen_gcc/src/asm.rs
@@ -682,6 +682,11 @@ fn reg_to_gcc(reg: InlineAsmRegOrRegClass) -> ConstraintOrRegister {
             InlineAsmRegClass::S390x(S390xInlineAsmRegClass::reg) => "r",
             InlineAsmRegClass::S390x(S390xInlineAsmRegClass::reg_addr) => "a",
             InlineAsmRegClass::S390x(S390xInlineAsmRegClass::freg) => "f",
+            InlineAsmRegClass::S390x(
+                S390xInlineAsmRegClass::vreg | S390xInlineAsmRegClass::areg,
+            ) => {
+                unreachable!("clobber-only")
+            }
             InlineAsmRegClass::Err => unreachable!(),
         },
     };
@@ -757,6 +762,9 @@ fn dummy_output_type<'gcc, 'tcx>(cx: &CodegenCx<'gcc, 'tcx>, reg: InlineAsmRegCl
             S390xInlineAsmRegClass::reg | S390xInlineAsmRegClass::reg_addr,
         ) => cx.type_i32(),
         InlineAsmRegClass::S390x(S390xInlineAsmRegClass::freg) => cx.type_f64(),
+        InlineAsmRegClass::S390x(S390xInlineAsmRegClass::vreg | S390xInlineAsmRegClass::areg) => {
+            unreachable!("clobber-only")
+        }
         InlineAsmRegClass::Msp430(Msp430InlineAsmRegClass::reg) => cx.type_i16(),
         InlineAsmRegClass::M68k(M68kInlineAsmRegClass::reg) => cx.type_i32(),
         InlineAsmRegClass::M68k(M68kInlineAsmRegClass::reg_addr) => cx.type_i32(),
diff --git a/compiler/rustc_codegen_llvm/src/asm.rs b/compiler/rustc_codegen_llvm/src/asm.rs
index 7621f111fe2..82ca3f519f7 100644
--- a/compiler/rustc_codegen_llvm/src/asm.rs
+++ b/compiler/rustc_codegen_llvm/src/asm.rs
@@ -708,6 +708,9 @@ fn reg_to_llvm(reg: InlineAsmRegOrRegClass, layout: Option<&TyAndLayout<'_>>) ->
             S390x(S390xInlineAsmRegClass::reg) => "r",
             S390x(S390xInlineAsmRegClass::reg_addr) => "a",
             S390x(S390xInlineAsmRegClass::freg) => "f",
+            S390x(S390xInlineAsmRegClass::vreg | S390xInlineAsmRegClass::areg) => {
+                unreachable!("clobber-only")
+            }
             Msp430(Msp430InlineAsmRegClass::reg) => "r",
             M68k(M68kInlineAsmRegClass::reg) => "r",
             M68k(M68kInlineAsmRegClass::reg_addr) => "a",
@@ -866,6 +869,9 @@ fn dummy_output_type<'ll>(cx: &CodegenCx<'ll, '_>, reg: InlineAsmRegClass) -> &'
         Avr(AvrInlineAsmRegClass::reg_ptr) => cx.type_i16(),
         S390x(S390xInlineAsmRegClass::reg | S390xInlineAsmRegClass::reg_addr) => cx.type_i32(),
         S390x(S390xInlineAsmRegClass::freg) => cx.type_f64(),
+        S390x(S390xInlineAsmRegClass::vreg | S390xInlineAsmRegClass::areg) => {
+            unreachable!("clobber-only")
+        }
         Msp430(Msp430InlineAsmRegClass::reg) => cx.type_i16(),
         M68k(M68kInlineAsmRegClass::reg) => cx.type_i32(),
         M68k(M68kInlineAsmRegClass::reg_addr) => cx.type_i32(),
diff --git a/compiler/rustc_span/src/symbol.rs b/compiler/rustc_span/src/symbol.rs
index 8f226b26bef..e8aa129c6cd 100644
--- a/compiler/rustc_span/src/symbol.rs
+++ b/compiler/rustc_span/src/symbol.rs
@@ -411,6 +411,7 @@ symbols! {
         arbitrary_enum_discriminant,
         arbitrary_self_types,
         arbitrary_self_types_pointers,
+        areg,
         args,
         arith_offset,
         arm,
diff --git a/compiler/rustc_target/src/asm/mod.rs b/compiler/rustc_target/src/asm/mod.rs
index 4d8c5cea8a8..df13d24cb9e 100644
--- a/compiler/rustc_target/src/asm/mod.rs
+++ b/compiler/rustc_target/src/asm/mod.rs
@@ -439,7 +439,7 @@ impl InlineAsmReg {
             Self::Hexagon(r) => r.overlapping_regs(|r| cb(Self::Hexagon(r))),
             Self::LoongArch(_) => cb(self),
             Self::Mips(_) => cb(self),
-            Self::S390x(_) => cb(self),
+            Self::S390x(r) => r.overlapping_regs(|r| cb(Self::S390x(r))),
             Self::Bpf(r) => r.overlapping_regs(|r| cb(Self::Bpf(r))),
             Self::Avr(r) => r.overlapping_regs(|r| cb(Self::Avr(r))),
             Self::Msp430(_) => cb(self),
@@ -892,6 +892,7 @@ pub enum InlineAsmClobberAbi {
     AArch64NoX18,
     RiscV,
     LoongArch,
+    S390x,
 }
 
 impl InlineAsmClobberAbi {
@@ -941,6 +942,10 @@ impl InlineAsmClobberAbi {
                 "C" | "system" => Ok(InlineAsmClobberAbi::LoongArch),
                 _ => Err(&["C", "system"]),
             },
+            InlineAsmArch::S390x => match name {
+                "C" | "system" => Ok(InlineAsmClobberAbi::S390x),
+                _ => Err(&["C", "system"]),
+            },
             _ => Err(&[]),
         }
     }
@@ -1098,6 +1103,28 @@ impl InlineAsmClobberAbi {
                     f16, f17, f18, f19, f20, f21, f22, f23,
                 }
             },
+            InlineAsmClobberAbi::S390x => clobbered_regs! {
+                S390x S390xInlineAsmReg {
+                    r0, r1, r2, r3, r4, r5,
+                    r14,
+
+                    // f0-f7, v0-v7
+                    f0, f1, f2, f3, f4, f5, f6, f7,
+                    v0, v1, v2, v3, v4, v5, v6, v7,
+
+                    // Technically the left halves of v8-v15 (i.e., f8-f15) are saved, but
+                    // we have no way of expressing this using clobbers.
+                    v8, v9, v10, v11, v12, v13, v14, v15,
+
+                    // Other vector registers are volatile
+                    v16, v17, v18, v19, v20, v21, v22, v23,
+                    v24, v25, v26, v27, v28, v29, v30, v31,
+
+                    // a0-a1 are reserved, other access registers are volatile
+                    a2, a3, a4, a5, a6, a7,
+                    a8, a9, a10, a11, a12, a13, a14, a15,
+                }
+            },
         }
     }
 }
diff --git a/compiler/rustc_target/src/asm/s390x.rs b/compiler/rustc_target/src/asm/s390x.rs
index 4258b23ac57..9b31190a72b 100644
--- a/compiler/rustc_target/src/asm/s390x.rs
+++ b/compiler/rustc_target/src/asm/s390x.rs
@@ -9,6 +9,8 @@ def_reg_class! {
         reg,
         reg_addr,
         freg,
+        vreg,
+        areg,
     }
 }
 
@@ -35,11 +37,13 @@ impl S390xInlineAsmRegClass {
 
     pub fn supported_types(
         self,
-        arch: InlineAsmArch,
+        _arch: InlineAsmArch,
     ) -> &'static [(InlineAsmType, Option<Symbol>)] {
-        match (self, arch) {
-            (Self::reg | Self::reg_addr, _) => types! { _: I8, I16, I32, I64; },
-            (Self::freg, _) => types! { _: F32, F64; },
+        match self {
+            Self::reg | Self::reg_addr => types! { _: I8, I16, I32, I64; },
+            Self::freg => types! { _: F32, F64; },
+            Self::vreg => &[],
+            Self::areg => &[],
         }
     }
 }
@@ -76,6 +80,52 @@ def_regs! {
         f13: freg = ["f13"],
         f14: freg = ["f14"],
         f15: freg = ["f15"],
+        v0: vreg = ["v0"],
+        v1: vreg = ["v1"],
+        v2: vreg = ["v2"],
+        v3: vreg = ["v3"],
+        v4: vreg = ["v4"],
+        v5: vreg = ["v5"],
+        v6: vreg = ["v6"],
+        v7: vreg = ["v7"],
+        v8: vreg = ["v8"],
+        v9: vreg = ["v9"],
+        v10: vreg = ["v10"],
+        v11: vreg = ["v11"],
+        v12: vreg = ["v12"],
+        v13: vreg = ["v13"],
+        v14: vreg = ["v14"],
+        v15: vreg = ["v15"],
+        v16: vreg = ["v16"],
+        v17: vreg = ["v17"],
+        v18: vreg = ["v18"],
+        v19: vreg = ["v19"],
+        v20: vreg = ["v20"],
+        v21: vreg = ["v21"],
+        v22: vreg = ["v22"],
+        v23: vreg = ["v23"],
+        v24: vreg = ["v24"],
+        v25: vreg = ["v25"],
+        v26: vreg = ["v26"],
+        v27: vreg = ["v27"],
+        v28: vreg = ["v28"],
+        v29: vreg = ["v29"],
+        v30: vreg = ["v30"],
+        v31: vreg = ["v31"],
+        a2: areg = ["a2"],
+        a3: areg = ["a3"],
+        a4: areg = ["a4"],
+        a5: areg = ["a5"],
+        a6: areg = ["a6"],
+        a7: areg = ["a7"],
+        a8: areg = ["a8"],
+        a9: areg = ["a9"],
+        a10: areg = ["a10"],
+        a11: areg = ["a11"],
+        a12: areg = ["a12"],
+        a13: areg = ["a13"],
+        a14: areg = ["a14"],
+        a15: areg = ["a15"],
         #error = ["r11"] =>
             "The frame pointer cannot be used as an operand for inline asm",
         #error = ["r15"] =>
@@ -87,13 +137,8 @@ def_regs! {
             "c12", "c13", "c14", "c15"
         ] =>
             "control registers are reserved by the kernel and cannot be used as operands for inline asm",
-        #error = [
-            "a0", "a1", "a2", "a3",
-            "a4", "a5", "a6", "a7",
-            "a8", "a9", "a10", "a11",
-            "a12", "a13", "a14", "a15"
-        ] =>
-            "access registers are not supported and cannot be used as operands for inline asm",
+        #error = ["a0", "a1"] =>
+            "a0 and a1 are reserved for system use and cannot be used as operands for inline asm",
     }
 }
 
@@ -106,4 +151,48 @@ impl S390xInlineAsmReg {
     ) -> fmt::Result {
         write!(out, "%{}", self.name())
     }
+
+    pub fn overlapping_regs(self, mut cb: impl FnMut(S390xInlineAsmReg)) {
+        macro_rules! reg_conflicts {
+            (
+                $(
+                    $full:ident : $($field:ident)*
+                ),*;
+            ) => {
+                match self {
+                    $(
+                        Self::$full => {
+                            cb(Self::$full);
+                            $(cb(Self::$field);)*
+                        }
+                        $(Self::$field)|* => {
+                            cb(Self::$full);
+                            cb(self);
+                        }
+                    )*
+                    r => cb(r),
+                }
+            };
+        }
+
+        // The left halves of v0-v15 are aliased to f0-f15.
+        reg_conflicts! {
+            v0 : f0,
+            v1 : f1,
+            v2 : f2,
+            v3 : f3,
+            v4 : f4,
+            v5 : f5,
+            v6 : f6,
+            v7 : f7,
+            v8 : f8,
+            v9 : f9,
+            v10 : f10,
+            v11 : f11,
+            v12 : f12,
+            v13 : f13,
+            v14 : f14,
+            v15 : f15;
+        }
+    }
 }
diff --git a/src/doc/unstable-book/src/language-features/asm-experimental-arch.md b/src/doc/unstable-book/src/language-features/asm-experimental-arch.md
index 43e11b6d57d..b1c429c7676 100644
--- a/src/doc/unstable-book/src/language-features/asm-experimental-arch.md
+++ b/src/doc/unstable-book/src/language-features/asm-experimental-arch.md
@@ -51,7 +51,10 @@ This feature tracks `asm!` and `global_asm!` support for the following architect
 | CSKY         | `reg`          | `r[0-31]`                          | `r`                  |
 | CSKY         | `freg`         | `f[0-31]`                          | `f`                  |
 | s390x        | `reg`          | `r[0-10]`, `r[12-14]`              | `r`                  |
+| s390x        | `reg_addr`     | `r[1-10]`, `r[12-14]`              | `a`                  |
 | s390x        | `freg`         | `f[0-15]`                          | `f`                  |
+| s390x        | `vreg`         | `v[0-31]`                          | Only clobbers        |
+| s390x        | `areg`         | `a[2-15]`                          | Only clobbers        |
 | Arm64EC      | `reg`          | `x[0-12]`, `x[15-22]`, `x[25-27]`, `x30` | `r`            |
 | Arm64EC      | `vreg`         | `v[0-15]`                          | `w`                  |
 | Arm64EC      | `vreg_low16`   | `v[0-15]`                          | `x`                  |
@@ -90,6 +93,8 @@ This feature tracks `asm!` and `global_asm!` support for the following architect
 | CSKY         | `freg`                          | None           | `f32`,                                  |
 | s390x        | `reg`, `reg_addr`               | None           | `i8`, `i16`, `i32`, `i64`               |
 | s390x        | `freg`                          | None           | `f32`, `f64`                            |
+| s390x        | `vreg`                          | N/A            | Only clobbers                           |
+| s390x        | `areg`                          | N/A            | Only clobbers                           |
 | Arm64EC      | `reg`                           | None           | `i8`, `i16`, `i32`, `f32`, `i64`, `f64` |
 | Arm64EC      | `vreg`                          | None           | `i8`, `i16`, `i32`, `f32`, `i64`, `f64`, <br> `i8x8`, `i16x4`, `i32x2`, `i64x1`, `f32x2`, `f64x1`, <br> `i8x16`, `i16x8`, `i32x4`, `i64x2`, `f32x4`, `f64x2` |
 
@@ -157,6 +162,8 @@ This feature tracks `asm!` and `global_asm!` support for the following architect
 | CSKY         | `r15`                                   | This is the link register. |
 | CSKY         | `r[26-30]`                              | Reserved by its ABI.       |
 | CSKY         | `r31`                                   | This is the TLS register.  |
+| s390x        | `c[0-15]`                               | Reserved by the kernel. |
+| s390x        | `a[0-1]`                                | Reserved for system use. |
 | Arm64EC      | `xzr`                                   | This is a constant zero register which can't be modified. |
 | Arm64EC      | `x18`                                   | This is an OS-reserved register. |
 | Arm64EC      | `x13`, `x14`, `x23`, `x24`, `x28`, `v[16-31]` | These are AArch64 registers that are not supported for Arm64EC. |
diff --git a/tests/codegen/asm-s390x-clobbers.rs b/tests/codegen/asm-s390x-clobbers.rs
new file mode 100644
index 00000000000..45f72206bdf
--- /dev/null
+++ b/tests/codegen/asm-s390x-clobbers.rs
@@ -0,0 +1,50 @@
+//@ revisions: s390x
+//@[s390x] compile-flags: --target s390x-unknown-linux-gnu
+//@[s390x] needs-llvm-components: systemz
+
+#![crate_type = "rlib"]
+#![feature(no_core, rustc_attrs, lang_items, asm_experimental_arch)]
+#![no_core]
+
+#[lang = "sized"]
+trait Sized {}
+
+#[rustc_builtin_macro]
+macro_rules! asm {
+    () => {};
+}
+
+// CHECK-LABEL: @cc_clobber
+// CHECK: call void asm sideeffect "", "~{cc}"()
+#[no_mangle]
+pub unsafe fn cc_clobber() {
+    asm!("", options(nostack, nomem));
+}
+
+// CHECK-LABEL: @no_clobber
+// CHECK: call void asm sideeffect "", ""()
+#[no_mangle]
+pub unsafe fn no_clobber() {
+    asm!("", options(nostack, nomem, preserves_flags));
+}
+
+// CHECK-LABEL: @a2_clobber
+// CHECK: call void asm sideeffect "", "~{a2}"()
+#[no_mangle]
+pub unsafe fn a2_clobber() {
+    asm!("", out("a2") _, options(nostack, nomem, preserves_flags));
+}
+
+// CHECK-LABEL: @v0_clobber
+// CHECK: call void asm sideeffect "", "~{v0}"()
+#[no_mangle]
+pub unsafe fn v0_clobber() {
+    asm!("", out("v0") _, options(nostack, nomem, preserves_flags));
+}
+
+// CHECK-LABEL: @clobber_abi
+// CHECK: asm sideeffect "", "={r0},={r1},={r2},={r3},={r4},={r5},={r14},={f0},={f1},={f2},={f3},={f4},={f5},={f6},={f7},~{v0},~{v1},~{v2},~{v3},~{v4},~{v5},~{v6},~{v7},~{v8},~{v9},~{v10},~{v11},~{v12},~{v13},~{v14},~{v15},~{v16},~{v17},~{v18},~{v19},~{v20},~{v21},~{v22},~{v23},~{v24},~{v25},~{v26},~{v27},~{v28},~{v29},~{v30},~{v31},~{a2},~{a3},~{a4},~{a5},~{a6},~{a7},~{a8},~{a9},~{a10},~{a11},~{a12},~{a13},~{a14},~{a15}"()
+#[no_mangle]
+pub unsafe fn clobber_abi() {
+    asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
+}