| Age | Commit message (Collapse) | Author | Lines |
|---|
|
From 72 bytes to 12 bytes (on x86-64).
There are two parts to this:
- Changing various source code offsets from 64-bit to 32-bit. This is
not a problem because the rest of rustc also uses 32-bit source code
offsets. This means `Token` is no longer `Copy` but this causes no
problems.
- Removing the `RawStrError` from `LiteralKind`. Raw string literal
invalidity is now indicated by a `None` value within
`RawStr`/`RawByteStr`, and the new `validate_raw_str` function can be
used to re-lex an invalid raw string literal to get the `RawStrError`.
There is one very small change in behaviour. Previously, if a raw string
literal matched both the `InvalidStarter` and `TooManyHashes` cases,
the latter would override the former. This has now changed, because
`raw_double_quoted_string` now uses `?` and so returns immediately upon
detecting the `InvalidStarter` case. I think this is a slight
improvement to report the earlier-detected error, and it explains the
change in the `test_too_many_hashes` test.
The commit also removes a couple of comments that refer to #77629 and
say that the size of these types don't affect performance. These
comments are wrong, though the performance effect is small.
|
|
Because it's tiny and hot.
|
|
|
|
Optimize `rustc_lexer`
The `cursor.first()` method in `rustc_lexer` now calls the `chars.next()` method instead of `chars.nth_char(0)`.
This allows LLVM to optimize the code better. The biggest win is that `eat_while()` is now fully inlined and generates better assembly. This improves the lexer's performance by 35% in a micro-benchmark I made (Lexing all 18MB of code in the compiler directory). But lexing is only a small part of the overall compilation time, so I don't know how significant it is.
Big thanks to criterion and `cargo asm`.
|
|
and optimize the iterator returned by `tokenize().
This improves lexer performance by 35%
|
|
|
|
In the lexer, consider emojis to be valid identifiers and reject
them later to avoid knock down parse errors.
|
|
|
|
|
|
|
|
Co-authored-by: Niko Matsakis <niko@alum.mit.edu>
|
|
This commit denies any identifiers immediately followed by
one of three tokens `"`, `'` or `#`, which is stricter than
the requirements of RFC 3101 but may be necessary according
to the discussion at [Zulip].
[Zulip]: https://rust-lang.zulipchat.com/#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/238470099
|
|
Changes `librustc_X` to `rustc_X`, only in documentation comments.
Plain code comments are left unchanged.
Also fix incorrect file paths.
|
|
|
|
|
|
|
|
UTF8 bytes.
Fixed it by inlining it in the two places where the count is used and simplified the logic there.
|
|
Also add an Oxford comma while we're editing that line.
|
|
Lexer now discerns between regular comments and doc comments, so use that.
The change only affects the choice of reported errors.
|
|
|
