Auto merge of #119673 - petrochenkov:dialoc5, r=compiler-errors,cjgillot

macro_rules: Preserve all metavariable spans in a global side table

This PR preserves spans of `tt` metavariables used to pass tokens to declarative macros.
Such metavariable spans can then be used in span combination operations like `Span::to` to improve all kinds of diagnostics.

Spans of non-`tt` metavariables are currently kept in nonterminal tokens, but the long term plan is remove all nonterminal tokens from rustc parser and rely on the proc macro model with invisible delimiters (#114647, #67062).
In particular, `NtIdent` nonterminal (corresponding to `ident` metavariables) becomes easy to remove when this PR lands (#119412 does it).

The metavariable spans are kept in a global side table keyed by `Span`s of original tokens.
The alternative to the side table is keeping them in `SpanData` instead, but the performance regressions would be large because any spans from tokens passed to declarative macros would stop being inline and would work through span interner instead, and the penalty would be paid even if we never use the metavar span for the given original span.
(But also see the comment on `fn maybe_use_metavar_location` describing the map collision issues with the side table approach.)

There are also other alternatives - keeping the metavar span in `Token` or `TokenTree`, but associating it with `Span` itsel is the most natural choice because metavar spans are used in span combining operations, and those operations are not necessarily tied to tokens.

31 files changed, 237 insertions, 130 deletions

diff --git a/compiler/rustc_expand/src/lib.rs b/compiler/rustc_expand/src/lib.rs
index 4da86d77dc8..0e73abc9ed8 100644
--- a/compiler/rustc_expand/src/lib.rs
+++ b/compiler/rustc_expand/src/lib.rs
@@ -6,6 +6,7 @@
 #![feature(if_let_guard)]
 #![feature(let_chains)]
 #![feature(macro_metavar_expr)]
+#![feature(map_try_insert)]
 #![feature(proc_macro_diagnostic)]
 #![feature(proc_macro_internals)]
 #![feature(proc_macro_span)]
diff --git a/compiler/rustc_expand/src/mbe/transcribe.rs b/compiler/rustc_expand/src/mbe/transcribe.rs
index 434891ebc76..519e4a634d8 100644
--- a/compiler/rustc_expand/src/mbe/transcribe.rs
+++ b/compiler/rustc_expand/src/mbe/transcribe.rs
@@ -13,7 +13,7 @@ use rustc_errors::DiagnosticBuilder;
 use rustc_errors::{pluralize, PResult};
 use rustc_span::hygiene::{LocalExpnId, Transparency};
 use rustc_span::symbol::{sym, Ident, MacroRulesNormalizedIdent};
-use rustc_span::{Span, SyntaxContext};
+use rustc_span::{with_metavar_spans, Span, SyntaxContext};
 
 use smallvec::{smallvec, SmallVec};
 use std::mem;
@@ -254,7 +254,8 @@ pub(super) fn transcribe<'a>(
                         MatchedTokenTree(tt) => {
                             // `tt`s are emitted into the output stream directly as "raw tokens",
                             // without wrapping them into groups.
-                            result.push(maybe_use_metavar_location(cx, &stack, sp, tt));
+                            let tt = maybe_use_metavar_location(cx, &stack, sp, tt, &mut marker);
+                            result.push(tt);
                         }
                         MatchedNonterminal(nt) => {
                             // Other variables are emitted into the output stream as groups with
@@ -319,6 +320,17 @@ pub(super) fn transcribe<'a>(
     }
 }
 
+/// Store the metavariable span for this original span into a side table.
+/// FIXME: Try to put the metavariable span into `SpanData` instead of a side table (#118517).
+/// An optimal encoding for inlined spans will need to be selected to minimize regressions.
+/// The side table approach is relatively good, but not perfect due to collisions.
+/// In particular, collisions happen when token is passed as an argument through several macro
+/// calls, like in recursive macros.
+/// The old heuristic below is used to improve spans in case of collisions, but diagnostics are
+/// still degraded sometimes in those cases.
+///
+/// The old heuristic:
+///
 /// Usually metavariables `$var` produce interpolated tokens, which have an additional place for
 /// keeping both the original span and the metavariable span. For `tt` metavariables that's not the
 /// case however, and there's no place for keeping a second span. So we try to give the single
@@ -338,15 +350,12 @@ pub(super) fn transcribe<'a>(
 ///   These are typically used for passing larger amounts of code, and tokens in that code usually
 ///   combine with each other and not with tokens outside of the sequence.
 /// - The metavariable span comes from a different crate, then we prefer the more local span.
-///
-/// FIXME: Find a way to keep both original and metavariable spans for all tokens without
-/// regressing compilation time too much. Several experiments for adding such spans were made in
-/// the past (PR #95580, #118517, #118671) and all showed some regressions.
 fn maybe_use_metavar_location(
     cx: &ExtCtxt<'_>,
     stack: &[Frame<'_>],
-    metavar_span: Span,
+    mut metavar_span: Span,
     orig_tt: &TokenTree,
+    marker: &mut Marker,
 ) -> TokenTree {
     let undelimited_seq = matches!(
         stack.last(),
@@ -357,18 +366,44 @@ fn maybe_use_metavar_location(
             ..
         })
     );
-    if undelimited_seq || cx.source_map().is_imported(metavar_span) {
+    if undelimited_seq {
+        // Do not record metavar spans for tokens from undelimited sequences, for perf reasons.
+        return orig_tt.clone();
+    }
+
+    let insert = |mspans: &mut FxHashMap<_, _>, s, ms| match mspans.try_insert(s, ms) {
+        Ok(_) => true,
+        Err(err) => *err.entry.get() == ms, // Tried to insert the same span, still success
+    };
+    marker.visit_span(&mut metavar_span);
+    let no_collision = match orig_tt {
+        TokenTree::Token(token, ..) => {
+            with_metavar_spans(|mspans| insert(mspans, token.span, metavar_span))
+        }
+        TokenTree::Delimited(dspan, ..) => with_metavar_spans(|mspans| {
+            insert(mspans, dspan.open, metavar_span)
+                && insert(mspans, dspan.close, metavar_span)
+                && insert(mspans, dspan.entire(), metavar_span)
+        }),
+    };
+    if no_collision || cx.source_map().is_imported(metavar_span) {
         return orig_tt.clone();
     }
 
+    // Setting metavar spans for the heuristic spans gives better opportunities for combining them
+    // with neighboring spans even despite their different syntactic contexts.
     match orig_tt {
         TokenTree::Token(Token { kind, span }, spacing) => {
             let span = metavar_span.with_ctxt(span.ctxt());
+            with_metavar_spans(|mspans| insert(mspans, span, metavar_span));
             TokenTree::Token(Token { kind: kind.clone(), span }, *spacing)
         }
         TokenTree::Delimited(dspan, dspacing, delimiter, tts) => {
-            let open = metavar_span.shrink_to_lo().with_ctxt(dspan.open.ctxt());
-            let close = metavar_span.shrink_to_hi().with_ctxt(dspan.close.ctxt());
+            let open = metavar_span.with_ctxt(dspan.open.ctxt());
+            let close = metavar_span.with_ctxt(dspan.close.ctxt());
+            with_metavar_spans(|mspans| {
+                insert(mspans, open, metavar_span) && insert(mspans, close, metavar_span)
+            });
             let dspan = DelimSpan::from_pair(open, close);
             TokenTree::Delimited(dspan, *dspacing, *delimiter, tts.clone())
         }
diff --git a/compiler/rustc_span/src/lib.rs b/compiler/rustc_span/src/lib.rs
index 49ae9f16c8e..616a7ccc7c6 100644
--- a/compiler/rustc_span/src/lib.rs
+++ b/compiler/rustc_span/src/lib.rs
@@ -72,6 +72,7 @@ pub mod fatal_error;
 
 pub mod profiling;
 
+use rustc_data_structures::fx::FxHashMap;
 use rustc_data_structures::stable_hasher::{Hash128, Hash64, HashStable, StableHasher};
 use rustc_data_structures::sync::{FreezeLock, FreezeWriteGuard, Lock, Lrc};
 
@@ -98,6 +99,9 @@ mod tests;
 pub struct SessionGlobals {
     symbol_interner: symbol::Interner,
     span_interner: Lock<span_encoding::SpanInterner>,
+    /// Maps a macro argument token into use of the corresponding metavariable in the macro body.
+    /// Collisions are possible and processed in `maybe_use_metavar_location` on best effort basis.
+    metavar_spans: Lock<FxHashMap<Span, Span>>,
     hygiene_data: Lock<hygiene::HygieneData>,
 
     /// A reference to the source map in the `Session`. It's an `Option`
@@ -115,6 +119,7 @@ impl SessionGlobals {
         SessionGlobals {
             symbol_interner: symbol::Interner::fresh(),
             span_interner: Lock::new(span_encoding::SpanInterner::default()),
+            metavar_spans: Default::default(),
             hygiene_data: Lock::new(hygiene::HygieneData::new(edition)),
             source_map: Lock::new(None),
         }
@@ -168,6 +173,11 @@ pub fn create_default_session_globals_then<R>(f: impl FnOnce() -> R) -> R {
 // deserialization.
 scoped_tls::scoped_thread_local!(static SESSION_GLOBALS: SessionGlobals);
 
+#[inline]
+pub fn with_metavar_spans<R>(f: impl FnOnce(&mut FxHashMap<Span, Span>) -> R) -> R {
+    with_session_globals(|session_globals| f(&mut session_globals.metavar_spans.lock()))
+}
+
 // FIXME: We should use this enum or something like it to get rid of the
 // use of magic `/rust/1.x/...` paths across the board.
 #[derive(Debug, Eq, PartialEq, Clone, Ord, PartialOrd, Decodable)]
@@ -824,29 +834,64 @@ impl Span {
         )
     }
 
+    /// Check if you can select metavar spans for the given spans to get matching contexts.
+    fn try_metavars(a: SpanData, b: SpanData, a_orig: Span, b_orig: Span) -> (SpanData, SpanData) {
+        let get = |mspans: &FxHashMap<_, _>, s| mspans.get(&s).copied();
+        match with_metavar_spans(|mspans| (get(mspans, a_orig), get(mspans, b_orig))) {
+            (None, None) => {}
+            (Some(meta_a), None) => {
+                let meta_a = meta_a.data();
+                if meta_a.ctxt == b.ctxt {
+                    return (meta_a, b);
+                }
+            }
+            (None, Some(meta_b)) => {
+                let meta_b = meta_b.data();
+                if a.ctxt == meta_b.ctxt {
+                    return (a, meta_b);
+                }
+            }
+            (Some(meta_a), Some(meta_b)) => {
+                let meta_b = meta_b.data();
+                if a.ctxt == meta_b.ctxt {
+                    return (a, meta_b);
+                }
+                let meta_a = meta_a.data();
+                if meta_a.ctxt == b.ctxt {
+                    return (meta_a, b);
+                } else if meta_a.ctxt == meta_b.ctxt {
+                    return (meta_a, meta_b);
+                }
+            }
+        }
+
+        (a, b)
+    }
+
     /// Prepare two spans to a combine operation like `to` or `between`.
-    /// FIXME: consider using declarative macro metavariable spans for the given spans if they are
-    /// better suitable for combining (#119412).
     fn prepare_to_combine(
         a_orig: Span,
         b_orig: Span,
     ) -> Result<(SpanData, SpanData, Option<LocalDefId>), Span> {
         let (a, b) = (a_orig.data(), b_orig.data());
+        if a.ctxt == b.ctxt {
+            return Ok((a, b, if a.parent == b.parent { a.parent } else { None }));
+        }
 
-        if a.ctxt != b.ctxt {
-            // Context mismatches usually happen when procedural macros combine spans copied from
-            // the macro input with spans produced by the macro (`Span::*_site`).
-            // In that case we consider the combined span to be produced by the macro and return
-            // the original macro-produced span as the result.
-            // Otherwise we just fall back to returning the first span.
-            // Combining locations typically doesn't make sense in case of context mismatches.
-            // `is_root` here is a fast path optimization.
-            let a_is_callsite = a.ctxt.is_root() || a.ctxt == b.span().source_callsite().ctxt();
-            return Err(if a_is_callsite { b_orig } else { a_orig });
+        let (a, b) = Span::try_metavars(a, b, a_orig, b_orig);
+        if a.ctxt == b.ctxt {
+            return Ok((a, b, if a.parent == b.parent { a.parent } else { None }));
         }
 
-        let parent = if a.parent == b.parent { a.parent } else { None };
-        Ok((a, b, parent))
+        // Context mismatches usually happen when procedural macros combine spans copied from
+        // the macro input with spans produced by the macro (`Span::*_site`).
+        // In that case we consider the combined span to be produced by the macro and return
+        // the original macro-produced span as the result.
+        // Otherwise we just fall back to returning the first span.
+        // Combining locations typically doesn't make sense in case of context mismatches.
+        // `is_root` here is a fast path optimization.
+        let a_is_callsite = a.ctxt.is_root() || a.ctxt == b.span().source_callsite().ctxt();
+        Err(if a_is_callsite { b_orig } else { a_orig })
     }
 
     /// This span, but in a larger context, may switch to the metavariable span if suitable.
diff --git a/tests/coverage/no_spans.cov-map b/tests/coverage/no_spans.cov-map
index 9915fc52e6d..30171c3f319 100644
--- a/tests/coverage/no_spans.cov-map
+++ b/tests/coverage/no_spans.cov-map
@@ -1,3 +1,11 @@
+Function name: no_spans::affected_function
+Raw bytes (9): 0x[01, 01, 00, 01, 01, 1a, 1c, 00, 1d]
+Number of files: 1
+- file 0 => global file 1
+Number of expressions: 0
+Number of file 0 mappings: 1
+- Code(Counter(0)) at (prev + 26, 28) to (start + 0, 29)
+
 Function name: no_spans::affected_function::{closure#0}
 Raw bytes (9): 0x[01, 01, 00, 01, 01, 1b, 0c, 00, 0e]
 Number of files: 1
diff --git a/tests/coverage/no_spans.coverage b/tests/coverage/no_spans.coverage
index e55177698a2..b117c32ffd0 100644
--- a/tests/coverage/no_spans.coverage
+++ b/tests/coverage/no_spans.coverage
@@ -23,7 +23,7 @@
    LL|       |}
    LL|       |
    LL|       |macro_that_defines_a_function! {
-   LL|       |    fn affected_function() {
+   LL|      1|    fn affected_function() {
    LL|      1|        || ()
    LL|       |    }
    LL|       |}
diff --git a/tests/coverage/no_spans_if_not.cov-map b/tests/coverage/no_spans_if_not.cov-map
index 5277267ec1b..bc3e14eddd5 100644
--- a/tests/coverage/no_spans_if_not.cov-map
+++ b/tests/coverage/no_spans_if_not.cov-map
@@ -1,3 +1,15 @@
+Function name: no_spans_if_not::affected_function
+Raw bytes (21): 0x[01, 01, 01, 01, 00, 03, 01, 16, 1c, 01, 12, 02, 02, 0d, 00, 0f, 00, 02, 0d, 00, 0f]
+Number of files: 1
+- file 0 => global file 1
+Number of expressions: 1
+- expression 0 operands: lhs = Counter(0), rhs = Zero
+Number of file 0 mappings: 3
+- Code(Counter(0)) at (prev + 22, 28) to (start + 1, 18)
+- Code(Expression(0, Sub)) at (prev + 2, 13) to (start + 0, 15)
+    = (c0 - Zero)
+- Code(Zero) at (prev + 2, 13) to (start + 0, 15)
+
 Function name: no_spans_if_not::main
 Raw bytes (9): 0x[01, 01, 00, 01, 01, 0b, 01, 02, 02]
 Number of files: 1
diff --git a/tests/coverage/no_spans_if_not.coverage b/tests/coverage/no_spans_if_not.coverage
index 1b6bbc75a04..d235568db65 100644
--- a/tests/coverage/no_spans_if_not.coverage
+++ b/tests/coverage/no_spans_if_not.coverage
@@ -19,11 +19,11 @@
    LL|       |}
    LL|       |
    LL|       |macro_that_defines_a_function! {
-   LL|       |    fn affected_function() {
-   LL|       |        if !false {
-   LL|       |            ()
+   LL|      1|    fn affected_function() {
+   LL|      1|        if !false {
+   LL|      1|            ()
    LL|       |        } else {
-   LL|       |            ()
+   LL|      0|            ()
    LL|       |        }
    LL|       |    }
    LL|       |}
diff --git a/tests/ui/anon-params/anon-params-edition-hygiene.rs b/tests/ui/anon-params/anon-params-edition-hygiene.rs
index 607412f44c4..56e7336a7a5 100644
--- a/tests/ui/anon-params/anon-params-edition-hygiene.rs
+++ b/tests/ui/anon-params/anon-params-edition-hygiene.rs
@@ -1,3 +1,4 @@
+//@ check-pass
 //@ edition:2018
 //@ aux-build:anon-params-edition-hygiene.rs
 
@@ -8,7 +9,6 @@
 extern crate anon_params_edition_hygiene;
 
 generate_trait_2015_ident!(u8);
-// FIXME: Edition hygiene doesn't work correctly with `tt`s in this case.
-generate_trait_2015_tt!(u8); //~ ERROR expected one of `:`, `@`, or `|`, found `)`
+generate_trait_2015_tt!(u8);
 
 fn main() {}
diff --git a/tests/ui/anon-params/anon-params-edition-hygiene.stderr b/tests/ui/anon-params/anon-params-edition-hygiene.stderr
deleted file mode 100644
index 373d7c6aebb..00000000000
--- a/tests/ui/anon-params/anon-params-edition-hygiene.stderr
+++ /dev/null
@@ -1,23 +0,0 @@
-error: expected one of `:`, `@`, or `|`, found `)`
-  --> $DIR/anon-params-edition-hygiene.rs:12:1
-   |
-LL | generate_trait_2015_tt!(u8);
-   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected one of `:`, `@`, or `|`
-   |
-   = note: anonymous parameters are removed in the 2018 edition (see RFC 1685)
-   = note: this error originates in the macro `generate_trait_2015_tt` (in Nightly builds, run with -Z macro-backtrace for more info)
-help: if this is a `self` type, give it a parameter name
-   |
-LL | generate_trait_2015_tt!(self: u8);
-   |                         +++++
-help: if this is a parameter name, give it a type
-   |
-LL | generate_trait_2015_tt!(u8: TypeName);
-   |                           ++++++++++
-help: if this is a type, explicitly ignore the parameter name
-   |
-LL | generate_trait_2015_tt!(_: u8);
-   |                         ++
-
-error: aborting due to 1 previous error
-
diff --git a/tests/ui/editions/edition-keywords-2018-2018-parsing.rs b/tests/ui/editions/edition-keywords-2018-2018-parsing.rs
index e3eed775a5f..4975246fa94 100644
--- a/tests/ui/editions/edition-keywords-2018-2018-parsing.rs
+++ b/tests/ui/editions/edition-keywords-2018-2018-parsing.rs
@@ -16,7 +16,7 @@ macro_rules! local_passes_ident {
     ($i: ident) => ($i) //~ ERROR macro expansion ends with an incomplete expression
 }
 macro_rules! local_passes_tt {
-    ($i: tt) => ($i) //~ ERROR macro expansion ends with an incomplete expression
+    ($i: tt) => ($i)
 }
 
 pub fn check_async() {
@@ -34,7 +34,7 @@ pub fn check_async() {
     if passes_tt!(r#async) == 1 {} // OK
     if local_passes_ident!(async) == 1 {} // Error reported above in the macro
     if local_passes_ident!(r#async) == 1 {} // OK
-    if local_passes_tt!(async) == 1 {} // Error reported above in the macro
+    if local_passes_tt!(async) == 1 {} //~ ERROR macro expansion ends with an incomplete expression
     if local_passes_tt!(r#async) == 1 {} // OK
     module::async(); //~ ERROR expected identifier, found keyword `async`
     module::r#async(); // OK
diff --git a/tests/ui/editions/edition-keywords-2018-2018-parsing.stderr b/tests/ui/editions/edition-keywords-2018-2018-parsing.stderr
index 6f08cff433b..4bbe1597233 100644
--- a/tests/ui/editions/edition-keywords-2018-2018-parsing.stderr
+++ b/tests/ui/editions/edition-keywords-2018-2018-parsing.stderr
@@ -68,10 +68,10 @@ LL |     ($i: ident) => ($i)
    |                       ^ expected one of `move`, `|`, or `||`
 
 error: macro expansion ends with an incomplete expression: expected one of `move`, `|`, or `||`
-  --> $DIR/edition-keywords-2018-2018-parsing.rs:19:20
+  --> $DIR/edition-keywords-2018-2018-parsing.rs:37:30
    |
-LL |     ($i: tt) => ($i)
-   |                    ^ expected one of `move`, `|`, or `||`
+LL |     if local_passes_tt!(async) == 1 {}
+   |                              ^ expected one of `move`, `|`, or `||`
 
 error[E0308]: mismatched types
   --> $DIR/edition-keywords-2018-2018-parsing.rs:42:33
diff --git a/tests/ui/fmt/format-args-capture-first-literal-is-macro.rs b/tests/ui/fmt/format-args-capture-first-literal-is-macro.rs
index 5afd21a17e5..3a0b7ba4666 100644
--- a/tests/ui/fmt/format-args-capture-first-literal-is-macro.rs
+++ b/tests/ui/fmt/format-args-capture-first-literal-is-macro.rs
@@ -6,7 +6,6 @@ extern crate format_string_proc_macro;
 macro_rules! identity_mbe {
     ($tt:tt) => {
         $tt
-        //~^ ERROR there is no argument named `a`
     };
 }
 
@@ -16,6 +15,7 @@ fn main() {
     format!(identity_pm!("{a}"));
     //~^ ERROR there is no argument named `a`
     format!(identity_mbe!("{a}"));
+    //~^ ERROR there is no argument named `a`
     format!(concat!("{a}"));
     //~^ ERROR there is no argument named `a`
 }
diff --git a/tests/ui/fmt/format-args-capture-first-literal-is-macro.stderr b/tests/ui/fmt/format-args-capture-first-literal-is-macro.stderr
index 4cf3afad7b8..e399361579f 100644
--- a/tests/ui/fmt/format-args-capture-first-literal-is-macro.stderr
+++ b/tests/ui/fmt/format-args-capture-first-literal-is-macro.stderr
@@ -1,5 +1,5 @@
 error: there is no argument named `a`
-  --> $DIR/format-args-capture-first-literal-is-macro.rs:16:26
+  --> $DIR/format-args-capture-first-literal-is-macro.rs:15:26
    |
 LL |     format!(identity_pm!("{a}"));
    |                          ^^^^^
@@ -8,10 +8,10 @@ LL |     format!(identity_pm!("{a}"));
    = note: to avoid ambiguity, `format_args!` cannot capture variables when the format string is expanded from a macro
 
 error: there is no argument named `a`
-  --> $DIR/format-args-capture-first-literal-is-macro.rs:8:9
+  --> $DIR/format-args-capture-first-literal-is-macro.rs:17:27
    |
-LL |         $tt
-   |         ^^^
+LL |     format!(identity_mbe!("{a}"));
+   |                           ^^^^^
    |
    = note: did you intend to capture a variable `a` from the surrounding scope?
    = note: to avoid ambiguity, `format_args!` cannot capture variables when the format string is expanded from a macro
diff --git a/tests/ui/lint/wide_pointer_comparisons.rs b/tests/ui/lint/wide_pointer_comparisons.rs
index 37807776d2f..31369001075 100644
--- a/tests/ui/lint/wide_pointer_comparisons.rs
+++ b/tests/ui/lint/wide_pointer_comparisons.rs
@@ -110,10 +110,12 @@ fn main() {
     {
         macro_rules! cmp {
             ($a:tt, $b:tt) => { $a == $b }
-            //~^ WARN ambiguous wide pointer comparison
         }
 
+        // FIXME: This lint uses some custom span combination logic.
+        // Rewrite it to adapt to the new metavariable span rules.
         cmp!(a, b);
+        //~^ WARN ambiguous wide pointer comparison
     }
 
     {
diff --git a/tests/ui/lint/wide_pointer_comparisons.stderr b/tests/ui/lint/wide_pointer_comparisons.stderr
index 349ff467d0f..6ef117c63c5 100644
--- a/tests/ui/lint/wide_pointer_comparisons.stderr
+++ b/tests/ui/lint/wide_pointer_comparisons.stderr
@@ -421,18 +421,18 @@ LL |         std::ptr::eq(*a, *b)
    |         ~~~~~~~~~~~~~  ~   +
 
 warning: ambiguous wide pointer comparison, the comparison includes metadata which may not be expected
-  --> $DIR/wide_pointer_comparisons.rs:112:33
+  --> $DIR/wide_pointer_comparisons.rs:117:14
    |
-LL |             ($a:tt, $b:tt) => { $a == $b }
-   |                                 ^^^^^^^^
+LL |         cmp!(a, b);
+   |              ^^^^
    |
 help: use `std::ptr::addr_eq` or untyped pointers to only compare their addresses
    |
-LL |             ($a:tt, $b:tt) => { std::ptr::addr_eq($a, $b) }
-   |                                 ++++++++++++++++++  ~   +
+LL |         cmp!(std::ptr::addr_eq(a, b));
+   |              ++++++++++++++++++ ~  +
 
 warning: ambiguous wide pointer comparison, the comparison includes metadata which may not be expected
-  --> $DIR/wide_pointer_comparisons.rs:121:39
+  --> $DIR/wide_pointer_comparisons.rs:123:39
    |
 LL |             ($a:ident, $b:ident) => { $a == $b }
    |                                       ^^^^^^^^
@@ -447,7 +447,7 @@ LL |             ($a:ident, $b:ident) => { std::ptr::addr_eq($a, $b) }
    |                                       ++++++++++++++++++  ~   +
 
 warning: ambiguous wide pointer comparison, the comparison includes metadata which may not be expected
-  --> $DIR/wide_pointer_comparisons.rs:131:37
+  --> $DIR/wide_pointer_comparisons.rs:133:37
    |
 LL |             ($a:expr, $b:expr) => { $a == $b }
    |                                     ^^
diff --git a/tests/ui/macros/issue-118786.rs b/tests/ui/macros/issue-118786.rs
index cc6c751813b..97454c9de07 100644
--- a/tests/ui/macros/issue-118786.rs
+++ b/tests/ui/macros/issue-118786.rs
@@ -5,8 +5,7 @@
 macro_rules! make_macro {
     ($macro_name:tt) => {
         macro_rules! $macro_name {
-        //~^ ERROR macros that expand to items must be delimited with braces or followed by a semicolon
-        //~| ERROR macro expansion ignores token `{` and any following
+        //~^ ERROR macro expansion ignores token `{` and any following
         //~| ERROR cannot find macro `macro_rules` in this scope
             () => {}
         }
@@ -14,3 +13,4 @@ macro_rules! make_macro {
 }
 
 make_macro!((meow));
+//~^ ERROR macros that expand to items must be delimited with braces or followed by a semicolon
diff --git a/tests/ui/macros/issue-118786.stderr b/tests/ui/macros/issue-118786.stderr
index 1a8ac9340da..03e65c94ba7 100644
--- a/tests/ui/macros/issue-118786.stderr
+++ b/tests/ui/macros/issue-118786.stderr
@@ -1,13 +1,13 @@
 error: macros that expand to items must be delimited with braces or followed by a semicolon
-  --> $DIR/issue-118786.rs:7:22
+  --> $DIR/issue-118786.rs:15:13
    |
-LL |         macro_rules! $macro_name {
-   |                      ^^^^^^^^^^^
+LL | make_macro!((meow));
+   |             ^^^^^^
    |
 help: change the delimiters to curly braces
    |
-LL |         macro_rules! {$macro_name} {
-   |                      +           +
+LL | make_macro!({meow});
+   |             ~    ~
 help: add a semicolon
    |
 LL |         macro_rules! $macro_name; {
diff --git a/tests/ui/methods/method-on-ambiguous-numeric-type.stderr b/tests/ui/methods/method-on-ambiguous-numeric-type.stderr
index 060595e1d40..12427040272 100644
--- a/tests/ui/methods/method-on-ambiguous-numeric-type.stderr
+++ b/tests/ui/methods/method-on-ambiguous-numeric-type.stderr
@@ -47,8 +47,8 @@ LL |     local_bar_tt.pow(2);
    |
 help: you must specify a type for this binding, like `i32`
    |
-LL |     ($tt:tt) => { let $tt: i32 = 42; }
-   |                          +++++
+LL |     local_mac_tt!(local_bar_tt: i32);
+   |                               +++++
 
 error[E0689]: can't call method `pow` on ambiguous numeric type `{integer}`
   --> $DIR/method-on-ambiguous-numeric-type.rs:37:9
diff --git a/tests/ui/parser/issues/issue-44406.stderr b/tests/ui/parser/issues/issue-44406.stderr
index d005f116e12..78cde9b6dca 100644
--- a/tests/ui/parser/issues/issue-44406.stderr
+++ b/tests/ui/parser/issues/issue-44406.stderr
@@ -15,7 +15,7 @@ LL |         bar { baz: $rest }
 help: if `bar` is a function, use the arguments directly
    |
 LL -         bar(baz: $rest)
-LL +         bar(: $rest)
+LL +         bar($rest)
    |
 
 error: aborting due to 1 previous error
diff --git a/tests/ui/parser/issues/issue-68091-unicode-ident-after-if.stderr b/tests/ui/parser/issues/issue-68091-unicode-ident-after-if.stderr
index 8e125864b8b..a68fae1a36e 100644
--- a/tests/ui/parser/issues/issue-68091-unicode-ident-after-if.stderr
+++ b/tests/ui/parser/issues/issue-68091-unicode-ident-after-if.stderr
@@ -5,6 +5,11 @@ LL |         $($c)ö* {}
    |             ^   - if this block is the condition of the `if` expression, then it must be followed by another block
    |             |
    |             expected condition here
+...
+LL |     x!(if);
+   |     ------ in this macro invocation
+   |
+   = note: this error originates in the macro `x` (in Nightly builds, run with -Z macro-backtrace for more info)
 
 error: aborting due to 1 previous error
 
diff --git a/tests/ui/parser/issues/issue-68092-unicode-ident-after-incomplete-expr.rs b/tests/ui/parser/issues/issue-68092-unicode-ident-after-incomplete-expr.rs
index 1a90b4724d4..d489df85c44 100644
--- a/tests/ui/parser/issues/issue-68092-unicode-ident-after-incomplete-expr.rs
+++ b/tests/ui/parser/issues/issue-68092-unicode-ident-after-incomplete-expr.rs
@@ -1,9 +1,9 @@
 macro_rules! x {
     ($($c:tt)*) => {
-        $($c)ö* //~ ERROR macro expansion ends with an incomplete expression: expected expression
+        $($c)ö*
     };
 }
 
 fn main() {
-    x!(!);
+    x!(!); //~ ERROR macro expansion ends with an incomplete expression: expected expression
 }
diff --git a/tests/ui/parser/issues/issue-68092-unicode-ident-after-incomplete-expr.stderr b/tests/ui/parser/issues/issue-68092-unicode-ident-after-incomplete-expr.stderr
index 15aa62e0810..3a4b0cbf2a5 100644
--- a/tests/ui/parser/issues/issue-68092-unicode-ident-after-incomplete-expr.stderr
+++ b/tests/ui/parser/issues/issue-68092-unicode-ident-after-incomplete-expr.stderr
@@ -1,8 +1,8 @@
 error: macro expansion ends with an incomplete expression: expected expression
-  --> $DIR/issue-68092-unicode-ident-after-incomplete-expr.rs:3:13
+  --> $DIR/issue-68092-unicode-ident-after-incomplete-expr.rs:8:9
    |
-LL |         $($c)ö*
-   |             ^ expected expression
+LL |     x!(!);
+   |         ^ expected expression
 
 error: aborting due to 1 previous error
 
diff --git a/tests/ui/proc-macro/capture-macro-rules-invoke.stdout b/tests/ui/proc-macro/capture-macro-rules-invoke.stdout
index bbab08bca49..71e34119ba7 100644
--- a/tests/ui/proc-macro/capture-macro-rules-invoke.stdout
+++ b/tests/ui/proc-macro/capture-macro-rules-invoke.stdout
@@ -271,7 +271,7 @@ PRINT-BANG INPUT (DEBUG): TokenStream [
                 span: $DIR/capture-macro-rules-invoke.rs:47:19: 47:20 (#0),
             },
         ],
-        span: $DIR/capture-macro-rules-invoke.rs:15:60: 15:63 (#0),
+        span: $DIR/capture-macro-rules-invoke.rs:47:13: 47:22 (#0),
     },
     Punct {
         ch: ',',
diff --git a/tests/ui/rust-2018/edition-lint-infer-outlives-macro.fixed b/tests/ui/rust-2018/edition-lint-infer-outlives-macro.fixed
index 38d8a36bcdf..435857224ac 100644
--- a/tests/ui/rust-2018/edition-lint-infer-outlives-macro.fixed
+++ b/tests/ui/rust-2018/edition-lint-infer-outlives-macro.fixed
@@ -111,10 +111,13 @@ mod everything_outside_with_tt_inner {
 mod everything_outside_with_tt_outer {
     macro_rules! m {
         ($b:lifetime $colon:tt $a:tt) => {
-            struct Foo<$a, $b >(&$a &$b ());
+            // FIXME: replacement span is corrupted due to a collision in metavar span table.
+            // struct Foo<$a, $b $colon $a>(&$a &$b ());
+            // ^ ERROR: outlives requirements can be inferred
+            struct Bar<$a, $b>(&$a &$b ()) ;
+            //~^ ERROR: outlives requirements can be inferred
+            struct Baz<$a, $b>(&$a &$b ()) where (): Sized, ;
             //~^ ERROR: outlives requirements can be inferred
-            struct Bar<$a, $b>(&$a &$b ()) where $b $colon $a;
-            struct Baz<$a, $b>(&$a &$b ()) where (): Sized, $b $colon $a;
         }
     }
     m!('b: 'a);
@@ -123,9 +126,10 @@ mod everything_outside_with_tt_outer {
 mod everything_outside_with_tt_both {
     macro_rules! m {
         ($b:tt $colon:tt $a:tt) => {
-            struct Foo<$a, $b >(&$a &$b ());
-            //~^ ERROR: outlives requirements can be inferred
-            struct Bar<$a, $b>(&$a &$b ()) where ;
+            // FIXME: replacement span is corrupted due to a collision in metavar span table.
+            // struct Foo<$a, $b $colon $a>(&$a &$b ());
+            // ^ ERROR: outlives requirements can be inferred
+            struct Bar<$a, $b>(&$a &$b ()) ;
             //~^ ERROR: outlives requirements can be inferred
             struct Baz<$a, $b>(&$a &$b ()) where (): Sized, ;
             //~^ ERROR: outlives requirements can be inferred
diff --git a/tests/ui/rust-2018/edition-lint-infer-outlives-macro.rs b/tests/ui/rust-2018/edition-lint-infer-outlives-macro.rs
index 60eedf7b069..6c879231a16 100644
--- a/tests/ui/rust-2018/edition-lint-infer-outlives-macro.rs
+++ b/tests/ui/rust-2018/edition-lint-infer-outlives-macro.rs
@@ -111,10 +111,13 @@ mod everything_outside_with_tt_inner {
 mod everything_outside_with_tt_outer {
     macro_rules! m {
         ($b:lifetime $colon:tt $a:tt) => {
-            struct Foo<$a, $b $colon $a>(&$a &$b ());
-            //~^ ERROR: outlives requirements can be inferred
+            // FIXME: replacement span is corrupted due to a collision in metavar span table.
+            // struct Foo<$a, $b $colon $a>(&$a &$b ());
+            // ^ ERROR: outlives requirements can be inferred
             struct Bar<$a, $b>(&$a &$b ()) where $b $colon $a;
+            //~^ ERROR: outlives requirements can be inferred
             struct Baz<$a, $b>(&$a &$b ()) where (): Sized, $b $colon $a;
+            //~^ ERROR: outlives requirements can be inferred
         }
     }
     m!('b: 'a);
@@ -123,8 +126,9 @@ mod everything_outside_with_tt_outer {
 mod everything_outside_with_tt_both {
     macro_rules! m {
         ($b:tt $colon:tt $a:tt) => {
-            struct Foo<$a, $b $colon $a>(&$a &$b ());
-            //~^ ERROR: outlives requirements can be inferred
+            // FIXME: replacement span is corrupted due to a collision in metavar span table.
+            // struct Foo<$a, $b $colon $a>(&$a &$b ());
+            // ^ ERROR: outlives requirements can be inferred
             struct Bar<$a, $b>(&$a &$b ()) where $b $colon $a;
             //~^ ERROR: outlives requirements can be inferred
             struct Baz<$a, $b>(&$a &$b ()) where (): Sized, $b $colon $a;
diff --git a/tests/ui/rust-2018/edition-lint-infer-outlives-macro.stderr b/tests/ui/rust-2018/edition-lint-infer-outlives-macro.stderr
index 734ae687978..d684911be39 100644
--- a/tests/ui/rust-2018/edition-lint-infer-outlives-macro.stderr
+++ b/tests/ui/rust-2018/edition-lint-infer-outlives-macro.stderr
@@ -83,25 +83,40 @@ LL |     m!('b: 'a);
    = note: this error originates in the macro `m` (in Nightly builds, run with -Z macro-backtrace for more info)
 
 error: outlives requirements can be inferred
-  --> $DIR/edition-lint-infer-outlives-macro.rs:114:31
+  --> $DIR/edition-lint-infer-outlives-macro.rs:117:44
    |
-LL |             struct Foo<$a, $b $colon $a>(&$a &$b ());
-   |                               ^^^^^^^^^ help: remove this bound
+LL |             struct Bar<$a, $b>(&$a &$b ()) where $b $colon $a;
+   |                                            ^^^^^^^^^^^^^^^^^^ help: remove this bound
+...
+LL |     m!('b: 'a);
+   |     ---------- in this macro invocation
+   |
+   = note: this error originates in the macro `m` (in Nightly builds, run with -Z macro-backtrace for more info)
 
 error: outlives requirements can be inferred
-  --> $DIR/edition-lint-infer-outlives-macro.rs:126:31
+  --> $DIR/edition-lint-infer-outlives-macro.rs:119:61
+   |
+LL |             struct Baz<$a, $b>(&$a &$b ()) where (): Sized, $b $colon $a;
+   |                                                             ^^^^^^^^^^^^ help: remove this bound
+...
+LL |     m!('b: 'a);
+   |     ---------- in this macro invocation
    |
-LL |             struct Foo<$a, $b $colon $a>(&$a &$b ());
-   |                               ^^^^^^^^^ help: remove this bound
+   = note: this error originates in the macro `m` (in Nightly builds, run with -Z macro-backtrace for more info)
 
 error: outlives requirements can be inferred
-  --> $DIR/edition-lint-infer-outlives-macro.rs:128:50
+  --> $DIR/edition-lint-infer-outlives-macro.rs:132:44
    |
 LL |             struct Bar<$a, $b>(&$a &$b ()) where $b $colon $a;
-   |                                                  ^^^^^^^^^^^^ help: remove this bound
+   |                                            ^^^^^^^^^^^^^^^^^^ help: remove this bound
+...
+LL |     m!('b: 'a);
+   |     ---------- in this macro invocation
+   |
+   = note: this error originates in the macro `m` (in Nightly builds, run with -Z macro-backtrace for more info)
 
 error: outlives requirements can be inferred
-  --> $DIR/edition-lint-infer-outlives-macro.rs:130:61
+  --> $DIR/edition-lint-infer-outlives-macro.rs:134:61
    |
 LL |             struct Baz<$a, $b>(&$a &$b ()) where (): Sized, $b $colon $a;
    |                                                             ^^^^^^^^^^^^ help: remove this bound
diff --git a/tests/ui/span/macro-span-replacement.rs b/tests/ui/span/macro-span-replacement.rs
index e6fcfa4c709..16f4018cf3e 100644
--- a/tests/ui/span/macro-span-replacement.rs
+++ b/tests/ui/span/macro-span-replacement.rs
@@ -4,10 +4,10 @@
 
 macro_rules! m {
     ($a:tt $b:tt) => {
-        $b $a; //~ WARN struct `S` is never constructed
+        $b $a;
     }
 }
 
 fn main() {
-    m!(S struct);
+    m!(S struct); //~ WARN struct `S` is never constructed
 }
diff --git a/tests/ui/span/macro-span-replacement.stderr b/tests/ui/span/macro-span-replacement.stderr
index 5dd56342889..6248db112f8 100644
--- a/tests/ui/span/macro-span-replacement.stderr
+++ b/tests/ui/span/macro-span-replacement.stderr
@@ -1,11 +1,8 @@
 warning: struct `S` is never constructed
-  --> $DIR/macro-span-replacement.rs:7:12
+  --> $DIR/macro-span-replacement.rs:12:8
    |
-LL |         $b $a;
-   |            ^^
-...
 LL |     m!(S struct);
-   |     ------------ in this macro invocation
+   |        ^
    |
 note: the lint level is defined here
   --> $DIR/macro-span-replacement.rs:3:9
diff --git a/tests/ui/type/type-check/issue-88577-check-fn-with-more-than-65535-arguments.stderr b/tests/ui/type/type-check/issue-88577-check-fn-with-more-than-65535-arguments.stderr
index 1ef02321e15..1a3848bbcf5 100644
--- a/tests/ui/type/type-check/issue-88577-check-fn-with-more-than-65535-arguments.stderr
+++ b/tests/ui/type/type-check/issue-88577-check-fn-with-more-than-65535-arguments.stderr
@@ -1,8 +1,8 @@
 error: function can not have more than 65535 arguments
-  --> $DIR/issue-88577-check-fn-with-more-than-65535-arguments.rs:6:22
+  --> $DIR/issue-88577-check-fn-with-more-than-65535-arguments.rs:6:17
    |
 LL |         fn _f($($t: ()),*) {}
-   |                      ^
+   |                 ^^^^^^
 ...
 LL | many_args!{[_]########## ######}
    | -------------------------------- in this macro invocation
diff --git a/tests/ui/typeck/issue-116473-ice-wrong-span-variant-args.rs b/tests/ui/typeck/issue-116473-ice-wrong-span-variant-args.rs
index c319c63eda2..cc66b5fd6f2 100644
--- a/tests/ui/typeck/issue-116473-ice-wrong-span-variant-args.rs
+++ b/tests/ui/typeck/issue-116473-ice-wrong-span-variant-args.rs
@@ -58,12 +58,12 @@ macro_rules! nested2_ident {
 // instead of the enum variant
 macro_rules! nested1_tt_args_in_first_macro {
     () => (nested2_tt_args_in_first_macro!(i32, u32));
+    //~^ ERROR type arguments are not allowed on this type
 }
 
 macro_rules! nested2_tt_args_in_first_macro {
     ($arg1:tt, $arg2:tt) => (if let EnumUnit::VariantB::<$arg1, $arg2> {}
-    //~^ ERROR type arguments are not allowed on this type
-    //~| ERROR mismatched types
+    //~^ ERROR mismatched types
             = 5 { true } else { false });
 }
 
diff --git a/tests/ui/typeck/issue-116473-ice-wrong-span-variant-args.stderr b/tests/ui/typeck/issue-116473-ice-wrong-span-variant-args.stderr
index b17936ee3d3..cb7666657ef 100644
--- a/tests/ui/typeck/issue-116473-ice-wrong-span-variant-args.stderr
+++ b/tests/ui/typeck/issue-116473-ice-wrong-span-variant-args.stderr
@@ -1,10 +1,10 @@
 error[E0109]: type arguments are not allowed on this type
   --> $DIR/issue-116473-ice-wrong-span-variant-args.rs:15:51
    |
+LL |     () => (recursive_tt!(VariantB));
+   |                          -------- not allowed on this type
 LL |     ($variant:tt) => (if let EnumUnit::$variant::<i32, u32> {} = 5 { true } else { false });
-   |                                        --------   ^^^  ^^^ type argument not allowed
-   |                                        |
-   |                                        not allowed on this type
+   |                                                   ^^^  ^^^ type argument not allowed
 ...
 LL |     recursive_tt!();
    |     --------------- in this macro invocation
@@ -69,10 +69,11 @@ LL |     recursive_ident!();
 error[E0109]: type arguments are not allowed on this type
   --> $DIR/issue-116473-ice-wrong-span-variant-args.rs:38:51
    |
+LL |     () => (nested2_tt!(VariantB));
+   |                        -------- not allowed on this type
+...
 LL |     ($variant:tt) => (if let EnumUnit::$variant::<i32, u32> {} = 5 { true } else { false });
-   |                                        --------   ^^^  ^^^ type argument not allowed
-   |                                        |
-   |                                        not allowed on this type
+   |                                                   ^^^  ^^^ type argument not allowed
 ...
 LL |     nested1_tt!();
    |     ------------- in this macro invocation
@@ -136,12 +137,13 @@ LL |     nested1_ident!();
    = note: this error originates in the macro `nested2_ident` which comes from the expansion of the macro `nested1_ident` (in Nightly builds, run with -Z macro-backtrace for more info)
 
 error[E0109]: type arguments are not allowed on this type
-  --> $DIR/issue-116473-ice-wrong-span-variant-args.rs:64:58
+  --> $DIR/issue-116473-ice-wrong-span-variant-args.rs:60:44
    |
+LL |     () => (nested2_tt_args_in_first_macro!(i32, u32));
+   |                                            ^^^  ^^^ type argument not allowed
+...
 LL |     ($arg1:tt, $arg2:tt) => (if let EnumUnit::VariantB::<$arg1, $arg2> {}
-   |                                               --------   ^^^^^  ^^^^^ type argument not allowed
-   |                                               |
-   |                                               not allowed on this type
+   |                                               -------- not allowed on this type
 ...
 LL |     nested1_tt_args_in_first_macro!();
    |     --------------------------------- in this macro invocation
@@ -155,11 +157,11 @@ LL +     ($arg1:tt, $arg2:tt) => (if let EnumUnit::<$arg1, $arg2>::VariantB {}
    |
 
 error[E0308]: mismatched types
-  --> $DIR/issue-116473-ice-wrong-span-variant-args.rs:64:37
+  --> $DIR/issue-116473-ice-wrong-span-variant-args.rs:65:37
    |
 LL |     ($arg1:tt, $arg2:tt) => (if let EnumUnit::VariantB::<$arg1, $arg2> {}
    |                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected integer, found `Enum<(), ()>`
-...
+LL |
 LL |             = 5 { true } else { false });
    |               - this expression has type `{integer}`
 ...