Rollup merge of #60300 - mjbshaw:ffi_types, r=rkruppe

Allow null-pointer-optimized enums in FFI if their underlying representation is FFI safe

I'm not sure if this requires an RFC. I attempted to start [a discussion on internals.rust-lang.org](https://internals.rust-lang.org/t/options-ffi-safety-and-guarantees-for-abi-compatibility-with-nonnull-optimizations/9784) and when no one really objected I figured I'd go ahead and try implementing this.

This allows types like `Option<NonZeroU8>` to be used in FFI without triggering the `improper_ctypes` lint. This works by changing the `is_repr_nullable_ptr` function to consider an enum `E` to be FFI-safe if:

- `E` has no explicit `#[repr(...)]`.
- It only has two variants.
- One of those variants is empty (meaning it has no fields).
- The other variant has only one field.
- That field is one of the following:
  - `&T`
  - `&mut T`
  - `extern "C" fn`
  - `core::num::NonZero*`
  - `core::ptr::NonNull<T>`
  - `#[repr(transparent)] struct` wrapper around one of the types in this list.
- The size of `E` and its field are both known and are both the same size (implying `E` is participating in the nonnull optimization).

This logic seems consistent with [the Rust nomicon](https://doc.rust-lang.org/nomicon/repr-rust.html).

2 files changed, 2 insertions, 0 deletions

diff --git a/src/libcore/num/mod.rs b/src/libcore/num/mod.rs
index 562a7a4b3c7..932c0eaa4c7 100644
--- a/src/libcore/num/mod.rs
+++ b/src/libcore/num/mod.rs
@@ -50,6 +50,7 @@ assert_eq!(size_of::<Option<core::num::", stringify!($Ty), ">>(), size_of::<", s
                 #[derive(Copy, Clone, Eq, PartialEq, Ord, PartialOrd, Hash)]
                 #[repr(transparent)]
                 #[rustc_layout_scalar_valid_range_start(1)]
+                #[cfg_attr(not(stage0), rustc_nonnull_optimization_guaranteed)]
                 pub struct $Ty($Int);
             }
 
diff --git a/src/libcore/ptr.rs b/src/libcore/ptr.rs
index 006b1e143ee..4bb4d3ee466 100644
--- a/src/libcore/ptr.rs
+++ b/src/libcore/ptr.rs
@@ -2938,6 +2938,7 @@ impl<'a, T: ?Sized> From<NonNull<T>> for Unique<T> {
 #[stable(feature = "nonnull", since = "1.25.0")]
 #[repr(transparent)]
 #[rustc_layout_scalar_valid_range_start(1)]
+#[cfg_attr(not(stage0), rustc_nonnull_optimization_guaranteed)]
 pub struct NonNull<T: ?Sized> {
     pointer: *const T,
 }