Introduce `ByteSymbol`.

It's like `Symbol` but for byte strings. The interner is now used for
both `Symbol` and `ByteSymbol`. E.g. if you intern `"dog"` and `b"dog"`
you'll get a `Symbol` and a `ByteSymbol` with the same index and the
characters will only be stored once.

The motivation for this is to eliminate the `Arc`s in `ast::LitKind`, to
make `ast::LitKind` impl `Copy`, and to avoid the need to arena-allocate
`ast::LitKind` in HIR. The latter change reduces peak memory by a
non-trivial amount on literal-heavy benchmarks such as `deep-vector` and
`tuple-stress`.

`Encoder`, `Decoder`, `SpanEncoder`, and `SpanDecoder` all get some
changes so that they can handle normal strings and byte strings.

This change does slow down compilation of programs that use
`include_bytes!` on large files, because the contents of those files are
now interned (hashed). This makes `include_bytes!` more similar to
`include_str!`, though `include_bytes!` contents still aren't escaped,
and hashing is still much cheaper than escaping.

1 files changed, 4 insertions, 4 deletions

diff --git a/compiler/rustc_builtin_macros/src/concat_bytes.rs b/compiler/rustc_builtin_macros/src/concat_bytes.rs
index 92d011fb9d1..fd2d740c020 100644
--- a/compiler/rustc_builtin_macros/src/concat_bytes.rs
+++ b/compiler/rustc_builtin_macros/src/concat_bytes.rs
@@ -177,15 +177,15 @@ pub(crate) fn expand_concat_bytes(
                 Ok(LitKind::Byte(val)) => {
                     accumulator.push(val);
                 }
-                Ok(LitKind::ByteStr(ref bytes, _)) => {
-                    accumulator.extend_from_slice(bytes);
+                Ok(LitKind::ByteStr(ref byte_sym, _)) => {
+                    accumulator.extend_from_slice(byte_sym.as_byte_str());
                 }
                 _ => {
                     guar.get_or_insert_with(|| invalid_type_err(cx, token_lit, e.span, false));
                 }
             },
-            ExprKind::IncludedBytes(bytes) => {
-                accumulator.extend_from_slice(bytes);
+            ExprKind::IncludedBytes(byte_sym) => {
+                accumulator.extend_from_slice(byte_sym.as_byte_str());
             }
             ExprKind::Err(guarantee) => {
                 guar = Some(*guarantee);