Rollup merge of #61878 - RalfJung:pin, r=Dylan-DPC

improve pinning projection docs

This tries to improve the explanation of structural pinning and pinning projections based on [this URLO thread](https://users.rust-lang.org/t/when-is-it-safe-to-move-a-member-value-out-of-a-pinned-future/28182).

Fixes https://github.com/rust-lang/rust/issues/61272.

1 files changed, 130 insertions, 41 deletions

diff --git a/src/libcore/pin.rs b/src/libcore/pin.rs
index c5247e134c8..c063cee5227 100644
--- a/src/libcore/pin.rs
+++ b/src/libcore/pin.rs
@@ -138,10 +138,11 @@
 //! To make this work, not just moving the data is restricted; deallocating, repurposing, or
 //! otherwise invalidating the memory used to store the data is restricted, too.
 //! Concretely, for pinned data you have to maintain the invariant
-//! that *its memory will not get invalidated from the moment it gets pinned until
+//! that *its memory will not get invalidated or repurposed from the moment it gets pinned until
 //! when `drop` is called*. Memory can be invalidated by deallocation, but also by
 //! replacing a [`Some(v)`] by [`None`], or calling [`Vec::set_len`] to "kill" some elements
-//! off of a vector.
+//! off of a vector. It can be repurposed by using [`ptr::write`] to overwrite it without
+//! calling the destructor first.
 //!
 //! This is exactly the kind of guarantee that the intrusive linked list from the previous
 //! section needs to function correctly.
@@ -166,57 +167,130 @@
 //! implementation as well: if an element of your type could have been pinned,
 //! you must treat Drop as implicitly taking `Pin<&mut Self>`.
 //!
-//! In particular, if your type is `#[repr(packed)]`, the compiler will automatically
+//! For example, you could implement `Drop` as follows:
+//! ```rust,no_run
+//! # use std::pin::Pin;
+//! # struct Type { }
+//! impl Drop for Type {
+//!     fn drop(&mut self) {
+//!         // `new_unchecked` is okay because we know this value is never used
+//!         // again after being dropped.
+//!         inner_drop(unsafe { Pin::new_unchecked(self)});
+//!         fn inner_drop(this: Pin<&mut Type>) {
+//!             // Actual drop code goes here.
+//!         }
+//!     }
+//! }
+//! ```
+//! The function `inner_drop` has the type that `drop` *should* have, so this makes sure that
+//! you do not accidentally use `self`/`this` in a way that is in conflict with pinning.
+//!
+//! Moreover, if your type is `#[repr(packed)]`, the compiler will automatically
 //! move fields around to be able to drop them. As a consequence, you cannot use
 //! pinning with a `#[repr(packed)]` type.
 //!
 //! # Projections and Structural Pinning
 //!
-//! One interesting question arises when considering the interaction of pinning
-//! and the fields of a struct. When can a struct have a "pinning projection",
-//! i.e., an operation with type `fn(Pin<&Struct>) -> Pin<&Field>`?  In a
-//! similar vein, when can a generic wrapper type (such as `Vec<T>`, `Box<T>`,
-//! or `RefCell<T>`) have an operation with type `fn(Pin<&Wrapper<T>>) ->
-//! Pin<&T>`?
-//!
-//! Note: For the entirety of this discussion, the same applies for mutable references as it
-//! does for shared references.
+//! When working with pinned structs, the question arises how one can access the
+//! fields of that struct in a method that takes just `Pin<&mut Struct>`.
+//! The usual approach is to write helper methods (so called *projections*)
+//! that turn `Pin<&mut Struct>` into a reference to the field, but what
+//! type should that reference have? Is it `Pin<&mut Field>` or `&mut Field`?
+//! The same question arises with the fields of an `enum`, and also when considering
+//! container/wrapper types such as [`Vec<T>`], [`Box<T>`], or [`RefCell<T>`].
+//! (This question applies to both mutable and shared references, we just
+//! use the more common case of mutable references here for illustration.)
+//!
+//! It turns out that it is actually up to the author of the data structure
+//! to decide whether the pinned projection for a particular field turns
+//! `Pin<&mut Struct>` into `Pin<&mut Field>` or `&mut Field`. There are some
+//! constraints though, and the most important constraint is *consistency*:
+//! every field can be *either* projected to a pinned reference, *or* have
+//! pinning removed as part of the projection. If both are done for the same field,
+//! that will likely be unsound!
+//!
+//! As the author of a data structure you get to decide for each field whether pinning
+//! "propagates" to this field or not. Pinning that propagates is also called "structural",
+//! because it follows the structure of the type.
+//! In the following subsections, we describe the considerations that have to be made
+//! for either choice.
+//!
+//! ## Pinning *is not* structural for `field`
+//!
+//! It may seem counter-intuitive that the field of a pinned struct might not be pinned,
+//! but that is actually the easiest choice: if a `Pin<&mut Field>` is never created,
+//! nothing can go wrong! So, if you decide that some field does not have structural pinning,
+//! all you have to ensure is that you never create a pinned reference to that field.
+//!
+//! Fields without structural pinning may have a projection method that turns
+//! `Pin<&mut Struct>` into `&mut Field`:
+//! ```rust,no_run
+//! # use std::pin::Pin;
+//! # type Field = i32;
+//! # struct Struct { field: Field }
+//! impl Struct {
+//!     fn pin_get_field<'a>(self: Pin<&'a mut Self>) -> &'a mut Field {
+//!         // This is okay because `field` is never considered pinned.
+//!         unsafe { &mut self.get_unchecked_mut().field }
+//!     }
+//! }
+//! ```
 //!
-//! Having a pinning projection for some field means that pinning is "structural":
-//! when the wrapper is pinned, the field must be considered pinned, too.
-//! After all, the pinning projection lets us get a `Pin<&Field>`.
+//! You may also `impl Unpin for Struct` *even if* the type of `field`
+//! is not `Unpin`. What that type thinks about pinning is not relevant
+//! when no `Pin<&mut Field>` is ever created.
+//!
+//! ## Pinning *is* structural for `field`
+//!
+//! The other option is to decide that pinning is "structural" for `field`,
+//! meaning that if the struct is pinned then so is the field.
+//!
+//! This allows writing a projection that creates a `Pin<&mut Field>`, thus
+//! witnessing that the field is pinned:
+//! ```rust,no_run
+//! # use std::pin::Pin;
+//! # type Field = i32;
+//! # struct Struct { field: Field }
+//! impl Struct {
+//!     fn pin_get_field<'a>(self: Pin<&'a mut Self>) -> Pin<&'a mut Field> {
+//!         // This is okay because `field` is pinned when `self` is.
+//!         unsafe { self.map_unchecked_mut(|s| &mut s.field) }
+//!     }
+//! }
+//! ```
 //!
-//! However, structural pinning comes with a few extra requirements, so not all
-//! wrappers can be structural and hence not all wrappers can offer pinning projections:
+//! However, structural pinning comes with a few extra requirements:
 //!
-//! 1.  The wrapper must only be [`Unpin`] if all the structural fields are
+//! 1.  The struct must only be [`Unpin`] if all the structural fields are
 //!     `Unpin`. This is the default, but `Unpin` is a safe trait, so as the author of
-//!     the wrapper it is your responsibility *not* to add something like
-//!     `impl<T> Unpin for Wrapper<T>`. (Notice that adding a projection operation
+//!     the struct it is your responsibility *not* to add something like
+//!     `impl<T> Unpin for Struct<T>`. (Notice that adding a projection operation
 //!     requires unsafe code, so the fact that `Unpin` is a safe trait does not break
 //!     the principle that you only have to worry about any of this if you use `unsafe`.)
-//! 2.  The destructor of the wrapper must not move structural fields out of its argument. This
+//! 2.  The destructor of the struct must not move structural fields out of its argument. This
 //!     is the exact point that was raised in the [previous section][drop-impl]: `drop` takes
-//!     `&mut self`, but the wrapper (and hence its fields) might have been pinned before.
+//!     `&mut self`, but the struct (and hence its fields) might have been pinned before.
 //!     You have to guarantee that you do not move a field inside your `Drop` implementation.
-//!     In particular, as explained previously, this means that your wrapper type must *not*
+//!     In particular, as explained previously, this means that your struct must *not*
 //!     be `#[repr(packed)]`.
+//!     See that section for how to write `drop` in a way that the compiler can help you
+//!     not accidentally break pinning.
 //! 3.  You must make sure that you uphold the [`Drop` guarantee][drop-guarantee]:
-//!     once your wrapper is pinned, the memory that contains the
+//!     once your struct is pinned, the memory that contains the
 //!     content is not overwritten or deallocated without calling the content's destructors.
-//!     This can be tricky, as witnessed by `VecDeque<T>`: the destructor of `VecDeque<T>` can fail
-//!     to call `drop` on all elements if one of the destructors panics. This violates the
+//!     This can be tricky, as witnessed by [`VecDeque<T>`]: the destructor of `VecDeque<T>`
+//!     can fail to call `drop` on all elements if one of the destructors panics. This violates the
 //!     `Drop` guarantee, because it can lead to elements being deallocated without
 //!     their destructor being called. (`VecDeque` has no pinning projections, so this
 //!     does not cause unsoundness.)
 //! 4.  You must not offer any other operations that could lead to data being moved out of
-//!     the fields when your type is pinned. For example, if the wrapper contains an
+//!     the structural fields when your type is pinned. For example, if the struct contains an
 //!     `Option<T>` and there is a `take`-like operation with type
-//!     `fn(Pin<&mut Wrapper<T>>) -> Option<T>`,
-//!     that operation can be used to move a `T` out of a pinned `Wrapper<T>` -- which means
-//!     pinning cannot be structural.
+//!     `fn(Pin<&mut Struct<T>>) -> Option<T>`,
+//!     that operation can be used to move a `T` out of a pinned `Struct<T>` -- which means
+//!     pinning cannot be structural for the field holding this data.
 //!
-//!     For a more complex example of moving data out of a pinned type, imagine if `RefCell<T>`
+//!     For a more complex example of moving data out of a pinned type, imagine if [`RefCell<T>`]
 //!     had a method `fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut T>`.
 //!     Then we could do the following:
 //!     ```compile_fail
@@ -231,13 +305,16 @@
 //!     (using `RefCell::get_pin_mut`) and then move that content using the mutable
 //!     reference we got later.
 //!
-//! For a type like `Vec<T>`, both possibilites (structural pinning or not) make sense,
-//! and the choice is up to the author. A `Vec<T>` with structural pinning could
-//! have `get_pin`/`get_pin_mut` projections. However, it could *not* allow calling
+//! ## Examples
+//!
+//! For a type like [`Vec<T>`], both possibilites (structural pinning or not) make sense.
+//! A `Vec<T>` with structural pinning could have `get_pin`/`get_pin_mut` methods to get
+//! pinned references to elements. However, it could *not* allow calling
 //! `pop` on a pinned `Vec<T>` because that would move the (structurally pinned) contents!
 //! Nor could it allow `push`, which might reallocate and thus also move the contents.
 //! A `Vec<T>` without structural pinning could `impl<T> Unpin for Vec<T>`, because the contents
 //! are never pinned and the `Vec<T>` itself is fine with being moved as well.
+//! At that point pinning just has no effect on the vector at all.
 //!
 //! In the standard library, pointer types generally do not have structural pinning,
 //! and thus they do not offer pinning projections. This is why `Box<T>: Unpin` holds for all `T`.
@@ -249,16 +326,28 @@
 //! whether the content is pinned is entirely independent of whether the pointer is
 //! pinned, meaning pinning is *not* structural.
 //!
+//! When implementing a [`Future`] combinator, you will usually need structural pinning
+//! for the nested futures, as you need to get pinned references to them to call `poll`.
+//! But if your combinator contains any other data that does not need to be pinned,
+//! you can make those fields not structural and hence freely access them with a
+//! mutable reference even when you just have `Pin<&mut Self>` (such as in your own
+//! `poll` implementation).
+//!
 //! [`Pin<P>`]: struct.Pin.html
-//! [`Unpin`]: ../../std/marker/trait.Unpin.html
-//! [`Deref`]: ../../std/ops/trait.Deref.html
-//! [`DerefMut`]: ../../std/ops/trait.DerefMut.html
-//! [`mem::swap`]: ../../std/mem/fn.swap.html
-//! [`mem::forget`]: ../../std/mem/fn.forget.html
+//! [`Unpin`]: ../marker/trait.Unpin.html
+//! [`Deref`]: ../ops/trait.Deref.html
+//! [`DerefMut`]: ../ops/trait.DerefMut.html
+//! [`mem::swap`]: ../mem/fn.swap.html
+//! [`mem::forget`]: ../mem/fn.forget.html
 //! [`Box<T>`]: ../../std/boxed/struct.Box.html
+//! [`Vec<T>`]: ../../std/vec/struct.Vec.html
 //! [`Vec::set_len`]: ../../std/vec/struct.Vec.html#method.set_len
-//! [`None`]: ../../std/option/enum.Option.html#variant.None
-//! [`Some(v)`]: ../../std/option/enum.Option.html#variant.Some
+//! [`VecDeque<T>`]: ../../std/collections/struct.VecDeque.html
+//! [`RefCell<T>`]: ../cell/struct.RefCell.html
+//! [`None`]: ../option/enum.Option.html#variant.None
+//! [`Some(v)`]: ../option/enum.Option.html#variant.Some
+//! [`ptr::write`]: ../ptr/fn.write.html
+//! [`Future`]: ../future/trait.Future.html
 //! [drop-impl]: #drop-implementation
 //! [drop-guarantee]: #drop-guarantee