| Age | Commit message (Collapse) | Author | Lines |
|
Optimize `rustc_lexer`
The `cursor.first()` method in `rustc_lexer` now calls the `chars.next()` method instead of `chars.nth_char(0)`.
This allows LLVM to optimize the code better. The biggest win is that `eat_while()` is now fully inlined and generates better assembly. This improves the lexer's performance by 35% in a micro-benchmark I made (Lexing all 18MB of code in the compiler directory). But lexing is only a small part of the overall compilation time, so I don't know how significant it is.
Big thanks to criterion and `cargo asm`.
|
|
and optimize the iterator returned by `tokenize().
This improves lexer performance by 35%
|
|
|
|
In the lexer, consider emojis to be valid identifiers and reject
them later to avoid knock down parse errors.
|
|
|
|
r=petrochenkov"
The PR had some unforseen perf regressions that are not as easy to find.
Revert the PR for now.
This reverts commit 6ae8912a3e7d2c4c775024f58a7ba4b1aedc4073, reversing
changes made to 86d6d2b7389fe1b339402c1798edae8b695fc9ef.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Co-authored-by: Niko Matsakis <niko@alum.mit.edu>
|
|
This commit denies any identifiers immediately followed by
one of three tokens `"`, `'` or `#`, which is stricter than
the requirements of RFC 3101 but may be necessary according
to the discussion at [Zulip].
[Zulip]: https://rust-lang.zulipchat.com/#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/238470099
|
|
Changes `librustc_X` to `rustc_X`, only in documentation comments.
Plain code comments are left unchanged.
Also fix incorrect file paths.
|
|
|
|
|
|
|
|
|
|
|
|
UTF8 bytes.
Fixed it by inlining it in the two places where the count is used and simplified the logic there.
|
|
Also add an Oxford comma while we're editing that line.
|
|
Lexer now discerns between regular comments and doc comments, so use that.
The change only affects the choice of reported errors.
|
|
StringReader is an intornal abstraction which at the moment changes a
lot, so these unit tests cause quite a bit of friction.
Moving them to rustc_lexer and more ingerated-testing style should
make them much less annoying, hopefully without decreasing their
usefulness much.
Note that coloncolon tests are removed (it's unclear what those are
testing).
\r\n tests are removed as well, as we normalize line endings even
before lexing.
|
|
|