Auto merge of #98189 - mystor:fast_ident_literal, r=eddyb

proc_macro/bridge: stop using a remote object handle for proc_macro Ident and Literal

This is the fourth part of https://github.com/rust-lang/rust/pull/86822, split off as requested in https://github.com/rust-lang/rust/pull/86822#pullrequestreview-1008655452. This patch transforms the `Ident` and `Group` types into structs serialized over IPC rather than handles.

Symbol values are interned on both the client and server when deserializing, to avoid unnecessary string copies and keep the size of `TokenTree` down. To do the interning efficiently on the client, the proc-macro crate is given a vendored version of the fxhash hasher, as `SipHash` appeared to cause performance issues. This was done rather than depending on `rustc_hash` as it is unfortunately difficult to depend on crates from within `proc_macro` due to it being built at the same time as `std`.

In addition, a custom arena allocator and symbol store was also added, inspired by those in `rustc_arena` and `rustc_span`. To prevent symbol re-use across multiple invocations of a macro on the same thread, a new range of `Symbol` names are used for each invocation of the macro, and symbols from previous invocations are cleaned-up.

In order to keep `Ident` creation efficient, a special ASCII-only case was added to perform ident validation without using RPC for simple identifiers. Full identifier validation couldn't be easily added, as it would require depending on the `rustc_lexer` and `unicode-normalization` crates from within `proc_macro`. Unicode identifiers are validated and normalized using RPC.

See the individual commit messages for more details on trade-offs and design decisions behind these patches.

4 files changed, 12 insertions, 28 deletions

diff --git a/src/test/ui/proc-macro/invalid-punct-ident-2.rs b/src/test/ui/proc-macro/invalid-punct-ident-2.rs
index 04a0a873311..151f6203439 100644
--- a/src/test/ui/proc-macro/invalid-punct-ident-2.rs
+++ b/src/test/ui/proc-macro/invalid-punct-ident-2.rs
@@ -1,17 +1,9 @@
 // aux-build:invalid-punct-ident.rs
-// rustc-env:RUST_BACKTRACE=0
-
-// FIXME https://github.com/rust-lang/rust/issues/59998
-// normalize-stderr-test "thread.*panicked.*proc_macro_server.rs.*\n" -> ""
-// normalize-stderr-test "note:.*RUST_BACKTRACE=1.*\n" -> ""
-// normalize-stderr-test "\nerror: internal compiler error.*\n\n" -> ""
-// normalize-stderr-test "note:.*unexpectedly panicked.*\n\n" -> ""
-// normalize-stderr-test "note: we would appreciate a bug report.*\n\n" -> ""
-// normalize-stderr-test "note: compiler flags.*\n\n" -> ""
-// normalize-stderr-test "note: rustc.*running on.*\n\n" -> ""
-// normalize-stderr-test "query stack during panic:\n" -> ""
-// normalize-stderr-test "we're just showing a limited slice of the query stack\n" -> ""
-// normalize-stderr-test "end of query stack\n" -> ""
+// ignore-stage1
+// only-linux
+//
+// FIXME: This should be a normal (stage1, all platforms) test in
+// src/test/ui/proc-macro once issue #59998 is fixed.
 
 #[macro_use]
 extern crate invalid_punct_ident;
diff --git a/src/test/ui/proc-macro/invalid-punct-ident-2.stderr b/src/test/ui/proc-macro/invalid-punct-ident-2.stderr
index f7e1f4bc7d3..0bd07bd649e 100644
--- a/src/test/ui/proc-macro/invalid-punct-ident-2.stderr
+++ b/src/test/ui/proc-macro/invalid-punct-ident-2.stderr
@@ -1,5 +1,5 @@
 error: proc macro panicked
-  --> $DIR/invalid-punct-ident-2.rs:19:1
+  --> $DIR/invalid-punct-ident-2.rs:11:1
    |
 LL | invalid_ident!();
    | ^^^^^^^^^^^^^^^^
diff --git a/src/test/ui/proc-macro/invalid-punct-ident-3.rs b/src/test/ui/proc-macro/invalid-punct-ident-3.rs
index aebba341625..7c22a56b6fb 100644
--- a/src/test/ui/proc-macro/invalid-punct-ident-3.rs
+++ b/src/test/ui/proc-macro/invalid-punct-ident-3.rs
@@ -1,17 +1,9 @@
 // aux-build:invalid-punct-ident.rs
-// rustc-env:RUST_BACKTRACE=0
-
-// FIXME https://github.com/rust-lang/rust/issues/59998
-// normalize-stderr-test "thread.*panicked.*proc_macro_server.rs.*\n" -> ""
-// normalize-stderr-test "note:.*RUST_BACKTRACE=1.*\n" -> ""
-// normalize-stderr-test "\nerror: internal compiler error.*\n\n" -> ""
-// normalize-stderr-test "note:.*unexpectedly panicked.*\n\n" -> ""
-// normalize-stderr-test "note: we would appreciate a bug report.*\n\n" -> ""
-// normalize-stderr-test "note: compiler flags.*\n\n" -> ""
-// normalize-stderr-test "note: rustc.*running on.*\n\n" -> ""
-// normalize-stderr-test "query stack during panic:\n" -> ""
-// normalize-stderr-test "we're just showing a limited slice of the query stack\n" -> ""
-// normalize-stderr-test "end of query stack\n" -> ""
+// ignore-stage1
+// only-linux
+//
+// FIXME: This should be a normal (stage1, all platforms) test in
+// src/test/ui/proc-macro once issue #59998 is fixed.
 
 #[macro_use]
 extern crate invalid_punct_ident;
diff --git a/src/test/ui/proc-macro/invalid-punct-ident-3.stderr b/src/test/ui/proc-macro/invalid-punct-ident-3.stderr
index 541c71d74db..a0cc5ef6e2d 100644
--- a/src/test/ui/proc-macro/invalid-punct-ident-3.stderr
+++ b/src/test/ui/proc-macro/invalid-punct-ident-3.stderr
@@ -1,5 +1,5 @@
 error: proc macro panicked
-  --> $DIR/invalid-punct-ident-3.rs:19:1
+  --> $DIR/invalid-punct-ident-3.rs:11:1
    |
 LL | invalid_raw_ident!();
    | ^^^^^^^^^^^^^^^^^^^^