Auto merge of #119673 - petrochenkov:dialoc5, r=compiler-errors,cjgillot

macro_rules: Preserve all metavariable spans in a global side table

This PR preserves spans of `tt` metavariables used to pass tokens to declarative macros.
Such metavariable spans can then be used in span combination operations like `Span::to` to improve all kinds of diagnostics.

Spans of non-`tt` metavariables are currently kept in nonterminal tokens, but the long term plan is remove all nonterminal tokens from rustc parser and rely on the proc macro model with invisible delimiters (#114647, #67062).
In particular, `NtIdent` nonterminal (corresponding to `ident` metavariables) becomes easy to remove when this PR lands (#119412 does it).

The metavariable spans are kept in a global side table keyed by `Span`s of original tokens.
The alternative to the side table is keeping them in `SpanData` instead, but the performance regressions would be large because any spans from tokens passed to declarative macros would stop being inline and would work through span interner instead, and the penalty would be paid even if we never use the metavar span for the given original span.
(But also see the comment on `fn maybe_use_metavar_location` describing the map collision issues with the side table approach.)

There are also other alternatives - keeping the metavar span in `Token` or `TokenTree`, but associating it with `Span` itsel is the most natural choice because metavar spans are used in span combining operations, and those operations are not necessarily tied to tokens.

4 files changed, 25 insertions, 5 deletions

diff --git a/tests/coverage/no_spans.cov-map b/tests/coverage/no_spans.cov-map
index 9915fc52e6d..30171c3f319 100644
--- a/tests/coverage/no_spans.cov-map
+++ b/tests/coverage/no_spans.cov-map
@@ -1,3 +1,11 @@
+Function name: no_spans::affected_function
+Raw bytes (9): 0x[01, 01, 00, 01, 01, 1a, 1c, 00, 1d]
+Number of files: 1
+- file 0 => global file 1
+Number of expressions: 0
+Number of file 0 mappings: 1
+- Code(Counter(0)) at (prev + 26, 28) to (start + 0, 29)
+
 Function name: no_spans::affected_function::{closure#0}
 Raw bytes (9): 0x[01, 01, 00, 01, 01, 1b, 0c, 00, 0e]
 Number of files: 1
diff --git a/tests/coverage/no_spans.coverage b/tests/coverage/no_spans.coverage
index e55177698a2..b117c32ffd0 100644
--- a/tests/coverage/no_spans.coverage
+++ b/tests/coverage/no_spans.coverage
@@ -23,7 +23,7 @@
    LL|       |}
    LL|       |
    LL|       |macro_that_defines_a_function! {
-   LL|       |    fn affected_function() {
+   LL|      1|    fn affected_function() {
    LL|      1|        || ()
    LL|       |    }
    LL|       |}
diff --git a/tests/coverage/no_spans_if_not.cov-map b/tests/coverage/no_spans_if_not.cov-map
index 5277267ec1b..bc3e14eddd5 100644
--- a/tests/coverage/no_spans_if_not.cov-map
+++ b/tests/coverage/no_spans_if_not.cov-map
@@ -1,3 +1,15 @@
+Function name: no_spans_if_not::affected_function
+Raw bytes (21): 0x[01, 01, 01, 01, 00, 03, 01, 16, 1c, 01, 12, 02, 02, 0d, 00, 0f, 00, 02, 0d, 00, 0f]
+Number of files: 1
+- file 0 => global file 1
+Number of expressions: 1
+- expression 0 operands: lhs = Counter(0), rhs = Zero
+Number of file 0 mappings: 3
+- Code(Counter(0)) at (prev + 22, 28) to (start + 1, 18)
+- Code(Expression(0, Sub)) at (prev + 2, 13) to (start + 0, 15)
+    = (c0 - Zero)
+- Code(Zero) at (prev + 2, 13) to (start + 0, 15)
+
 Function name: no_spans_if_not::main
 Raw bytes (9): 0x[01, 01, 00, 01, 01, 0b, 01, 02, 02]
 Number of files: 1
diff --git a/tests/coverage/no_spans_if_not.coverage b/tests/coverage/no_spans_if_not.coverage
index 1b6bbc75a04..d235568db65 100644
--- a/tests/coverage/no_spans_if_not.coverage
+++ b/tests/coverage/no_spans_if_not.coverage
@@ -19,11 +19,11 @@
    LL|       |}
    LL|       |
    LL|       |macro_that_defines_a_function! {
-   LL|       |    fn affected_function() {
-   LL|       |        if !false {
-   LL|       |            ()
+   LL|      1|    fn affected_function() {
+   LL|      1|        if !false {
+   LL|      1|            ()
    LL|       |        } else {
-   LL|       |            ()
+   LL|      0|            ()
    LL|       |        }
    LL|       |    }
    LL|       |}