Implement span quoting for proc-macros

This PR implements span quoting, allowing proc-macros to produce spans
pointing *into their own crate*. This is used by the unstable
`proc_macro::quote!` macro, allowing us to get error messages like this:

```
error[E0412]: cannot find type `MissingType` in this scope
  --> $DIR/auxiliary/span-from-proc-macro.rs:37:20
   |
LL | pub fn error_from_attribute(_args: TokenStream, _input: TokenStream) -> TokenStream {
   | ----------------------------------------------------------------------------------- in this expansion of procedural macro `#[error_from_attribute]`
...
LL |             field: MissingType
   |                    ^^^^^^^^^^^ not found in this scope
   |
  ::: $DIR/span-from-proc-macro.rs:8:1
   |
LL | #[error_from_attribute]
   | ----------------------- in this macro invocation
```

Here, `MissingType` occurs inside the implementation of the proc-macro
`#[error_from_attribute]`. Previosuly, this would always result in a
span pointing at `#[error_from_attribute]`

This will make many proc-macro-related error message much more useful -
when a proc-macro generates code containing an error, users will get an
error message pointing directly at that code (within the macro
definition), instead of always getting a span pointing at the macro
invocation site.

This is implemented as follows:
* When a proc-macro crate is being *compiled*, it causes the `quote!`
  macro to get run. This saves all of the sapns in the input to `quote!`
  into the metadata of *the proc-macro-crate* (which we are currently
  compiling). The `quote!` macro then expands to a call to
  `proc_macro::Span::recover_proc_macro_span(id)`, where `id` is an
opaque identifier for the span in the crate metadata.
* When the same proc-macro crate is *run* (e.g. it is loaded from disk
  and invoked by some consumer crate), the call to
`proc_macro::Span::recover_proc_macro_span` causes us to load the span
from the proc-macro crate's metadata. The proc-macro then produces a
`TokenStream` containing a `Span` pointing into the proc-macro crate
itself.

The recursive nature of 'quote!' can be difficult to understand at
first. The file `src/test/ui/proc-macro/quote-debug.stdout` shows
the output of the `quote!` macro, which should make this eaier to
understand.

This PR also supports custom quoting spans in custom quote macros (e.g.
the `quote` crate). All span quoting goes through the
`proc_macro::quote_span` method, which can be called by a custom quote
macro to perform span quoting. An example of this usage is provided in
`src/test/ui/proc-macro/auxiliary/custom-quote.rs`

Custom quoting currently has a few limitations:

In order to quote a span, we need to generate a call to
`proc_macro::Span::recover_proc_macro_span`. However, proc-macros
support renaming the `proc_macro` crate, so we can't simply hardcode
this path. Previously, the `quote_span` method used the path
`crate::Span` - however, this only works when it is called by the
builtin `quote!` macro in the same crate. To support being called from
arbitrary crates, we need access to the name of the `proc_macro` crate
to generate a path. This PR adds an additional argument to `quote_span`
to specify the name of the `proc_macro` crate. Howver, this feels kind
of hacky, and we may want to change this before stabilizing anything
quote-related.

Additionally, using `quote_span` currently requires enabling the
`proc_macro_internals` feature. The builtin `quote!` macro
has an `#[allow_internal_unstable]` attribute, but this won't work for
custom quote implementations. This will likely require some additional
tricks to apply `allow_internal_unstable` to the span of
`proc_macro::Span::recover_proc_macro_span`.

3 files changed, 24 insertions, 5 deletions

diff --git a/library/proc_macro/src/bridge/mod.rs b/library/proc_macro/src/bridge/mod.rs
index c898d483a8b..355ad1f9f88 100644
--- a/library/proc_macro/src/bridge/mod.rs
+++ b/library/proc_macro/src/bridge/mod.rs
@@ -162,6 +162,8 @@ macro_rules! with_api {
                 fn join($self: $S::Span, other: $S::Span) -> Option<$S::Span>;
                 fn resolved_at($self: $S::Span, at: $S::Span) -> $S::Span;
                 fn source_text($self: $S::Span) -> Option<String>;
+                fn save_span($self: $S::Span) -> usize;
+                fn recover_proc_macro_span(id: usize) -> $S::Span;
             },
         }
     };
@@ -338,6 +340,7 @@ mark_noop! {
     &'a [u8],
     &'a str,
     String,
+    usize,
     Delimiter,
     Level,
     LineColumn,
diff --git a/library/proc_macro/src/lib.rs b/library/proc_macro/src/lib.rs
index c7f58f36154..525fd0fbe34 100644
--- a/library/proc_macro/src/lib.rs
+++ b/library/proc_macro/src/lib.rs
@@ -265,7 +265,7 @@ pub mod token_stream {
 /// Unquoting is done with `$`, and works by taking the single next ident as the unquoted term.
 /// To quote `$` itself, use `$$`.
 #[unstable(feature = "proc_macro_quote", issue = "54722")]
-#[allow_internal_unstable(proc_macro_def_site)]
+#[allow_internal_unstable(proc_macro_def_site, proc_macro_internals)]
 #[rustc_builtin_macro]
 pub macro quote($($t:tt)*) {
     /* compiler built-in */
@@ -394,6 +394,20 @@ impl Span {
         self.0.source_text()
     }
 
+    // Used by the implementation of `Span::quote`
+    #[doc(hidden)]
+    #[unstable(feature = "proc_macro_internals", issue = "27812")]
+    pub fn save_span(&self) -> usize {
+        self.0.save_span()
+    }
+
+    // Used by the implementation of `Span::quote`
+    #[doc(hidden)]
+    #[unstable(feature = "proc_macro_internals", issue = "27812")]
+    pub fn recover_proc_macro_span(id: usize) -> Span {
+        Span(bridge::client::Span::recover_proc_macro_span(id))
+    }
+
     diagnostic_method!(error, Level::Error);
     diagnostic_method!(warning, Level::Warning);
     diagnostic_method!(note, Level::Note);
diff --git a/library/proc_macro/src/quote.rs b/library/proc_macro/src/quote.rs
index 144e2d6bac4..1fd59889709 100644
--- a/library/proc_macro/src/quote.rs
+++ b/library/proc_macro/src/quote.rs
@@ -65,6 +65,7 @@ pub fn quote(stream: TokenStream) -> TokenStream {
     if stream.is_empty() {
         return quote!(crate::TokenStream::new());
     }
+    let proc_macro_crate = quote!(crate);
     let mut after_dollar = false;
     let tokens = stream
         .into_iter()
@@ -105,7 +106,7 @@ pub fn quote(stream: TokenStream) -> TokenStream {
                 ))),
                 TokenTree::Ident(tt) => quote!(crate::TokenTree::Ident(crate::Ident::new(
                     (@ TokenTree::from(Literal::string(&tt.to_string()))),
-                    (@ quote_span(tt.span())),
+                    (@ quote_span(proc_macro_crate.clone(), tt.span())),
                 ))),
                 TokenTree::Literal(tt) => quote!(crate::TokenTree::Literal({
                     let mut iter = (@ TokenTree::from(Literal::string(&tt.to_string())))
@@ -115,7 +116,7 @@ pub fn quote(stream: TokenStream) -> TokenStream {
                     if let (Some(crate::TokenTree::Literal(mut lit)), None) =
                         (iter.next(), iter.next())
                     {
-                        lit.set_span((@ quote_span(tt.span())));
+                        lit.set_span((@ quote_span(proc_macro_crate.clone(), tt.span())));
                         lit
                     } else {
                         unreachable!()
@@ -135,6 +136,7 @@ pub fn quote(stream: TokenStream) -> TokenStream {
 /// Quote a `Span` into a `TokenStream`.
 /// This is needed to implement a custom quoter.
 #[unstable(feature = "proc_macro_quote", issue = "54722")]
-pub fn quote_span(_: Span) -> TokenStream {
-    quote!(crate::Span::def_site())
+pub fn quote_span(proc_macro_crate: TokenStream, span: Span) -> TokenStream {
+    let id = span.save_span();
+    quote!((@ proc_macro_crate ) ::Span::recover_proc_macro_span((@ TokenTree::from(Literal::usize_unsuffixed(id)))))
 }