summary refs log tree commit diff
path: root/compiler/rustc_parse/src/lexer/mod.rs
AgeCommit message (Collapse)AuthorLines
2022-08-01Shrink `Token`.Nicholas Nethercote-23/+29
From 72 bytes to 12 bytes (on x86-64). There are two parts to this: - Changing various source code offsets from 64-bit to 32-bit. This is not a problem because the rest of rustc also uses 32-bit source code offsets. This means `Token` is no longer `Copy` but this causes no problems. - Removing the `RawStrError` from `LiteralKind`. Raw string literal invalidity is now indicated by a `None` value within `RawStr`/`RawByteStr`, and the new `validate_raw_str` function can be used to re-lex an invalid raw string literal to get the `RawStrError`. There is one very small change in behaviour. Previously, if a raw string literal matched both the `InvalidStarter` and `TooManyHashes` cases, the latter would override the former. This has now changed, because `raw_double_quoted_string` now uses `?` and so returns immediately upon detecting the `InvalidStarter` case. I think this is a slight improvement to report the earlier-detected error, and it explains the change in the `test_too_many_hashes` test. The commit also removes a couple of comments that refer to #77629 and say that the size of these types don't affect performance. These comments are wrong, though the performance effect is small.
2022-08-01Add a size assertion for `Token`.Nicholas Nethercote-0/+7
2022-08-01Remove `StringReader::end_src_index`.Nicholas Nethercote-5/+2
It not needed, always being set to the end of the text.
2022-08-01Improve shebang handling.Nicholas Nethercote-8/+5
Avoid doing stuff until it's necessary.
2022-05-20Remove `crate` visibility usage in compilerJacob Pratt-1/+1
2022-04-29errors: `span_suggestion` takes `impl ToString`David Wood-2/+2
Change `span_suggestion` (and variants) to take `impl ToString` rather than `String` for the suggested code, as this simplifies the requirements on the diagnostic derive. Signed-off-by: David Wood <david.wood@huawei.com>
2022-04-28rustc_ast: Harmonize delimiter naming with `proc_macro::Delimiter`Vadim Petrochenkov-9/+9
2022-04-16Rollup merge of #95859 - rainy-me:unterminated-nested-block-comment, ↵Dylan DPC-10/+50
r=petrochenkov Improve diagnostics for unterminated nested block comment close #95283 (This is my first time try to messing around with rust compiler and might get a lot of things wrong... :bow: )
2022-04-14refactor: change to use peekablerainy-me-20/+17
2022-04-13couple of clippy::complexity fixesMatthias Krüger-3/+3
2022-04-14improve diagnostics for unterminated nested block commentrainy-me-10/+53
2022-03-31Rollup merge of #95251 - GrishaVar:hashes-u16-to-u8, r=dtolnayDylan DPC-3/+1
Reduce max hash in raw strings from u16 to u8 [Relevant discussion](https://rust-lang.zulipchat.com/#narrow/stream/237824-t-lang.2Fdoc/topic/Max.20raw.20string.20delimiters)
2022-03-30Update error message & remove outdated test commentGrisha Vartanyan-3/+1
2022-03-27Make fatal DiagnosticBuilder yield neverMichael Goulet-12/+19
2022-03-02rename ErrorReported -> ErrorGuaranteedmark-2/+2
2022-02-23rustc_errors: let `DiagnosticBuilder::emit` return a "guarantee of emission".Eduard-Mihai Burtescu-2/+4
2022-02-16Adopt let_else in even more placesest31-6/+2
2021-11-23review comment: plural of emoji is emojiEsteban Kuber-1/+1
2021-11-23Account for confusable codepoints when recovering emoji identifiersEsteban Kuber-2/+13
2021-11-23Tokenize emoji as if they were valid indentifiersEsteban Kuber-0/+6
In the lexer, consider emojis to be valid identifiers and reject them later to avoid knock down parse errors.
2021-11-04Optimize literal, doc comment lint as well, extract function.Hans Kratz-39/+2
2021-11-04Create subslice as that leads to a smaller code size.Hans Kratz-3/+4
2021-11-04Optimize bidi character detection.Hans Kratz-6/+39
2021-11-01fix formattingPietro Albini-1/+1
2021-10-31Lint against RTL unicode codepoints in literals and commentsEsteban Küber-3/+37
Address CVE-2021-42574.
2021-07-31Suggest `br` if the unknown string prefix `rb` is foundFabian Wolff-3/+11
2021-07-06rename rust_2021_token_prefixes to rust_2021_prefixes_incompatible_syntaxRyan Levick-2/+2
2021-07-06rust_2021_token_prefixesRyan Levick-2/+2
2021-07-06Rename reserved_prefix lint to reserved_prefixesRyan Levick-2/+2
2021-06-26No reserved_prefix suggestion in proc macro call_site.Mara Bos-11/+13
2021-06-26Rename 'bad prefix' to 'unknown prefix'.Mara Bos-5/+5
2021-06-26Check the span's edition for the reserved prefixes.Mara Bos-15/+16
2021-06-26Improve comments for reserved prefixes.Mara Bos-1/+4
Co-authored-by: Niko Matsakis <niko@alum.mit.edu>
2021-06-26Add migration lint for reserved prefixes.Mara Bos-7/+16
2021-06-26Fix note in reserved prefix error.Mara Bos-1/+1
2021-06-26Add machine applicable suggestion to unknown prefix error.Mara Bos-5/+4
2021-06-26Reserve prefixed identifiers and string literals (RFC 3101)lrh2000-1/+31
This commit denies any identifiers immediately followed by one of three tokens `"`, `'` or `#`, which is stricter than the requirements of RFC 3101 but may be necessary according to the discussion at [Zulip]. [Zulip]: https://rust-lang.zulipchat.com/#narrow/stream/268952-edition-2021/topic/reserved.20prefixes/near/238470099
2021-05-08Make `Diagnostic::span_fatal` unconditionally raise an errorJoshua Nelson-9/+5
It had no callers which didn't immediately call `raise()`, and this unifies the behavior with `Session`.
2021-05-08Remove some unnecessary uses of `struct_span_fatal`Joshua Nelson-36/+20
All of them immediately called `emit()` then `raise()`, so they could just call `span_fatal` directly.
2021-04-08Fix outdated crate names in compiler docspierwill-1/+1
Changes `librustc_X` to `rustc_X`, only in documentation comments. Plain code comments are left unchanged. Also fix incorrect file paths.
2021-02-07Clarify error message wordingSmitty-1/+1
2021-02-06Add note about encoding when null bytes foundSmitty-0/+3
2021-02-03Handle `Span`s for byte and raw strings and add more detailEsteban Küber-10/+20
2020-11-22Fix typo in doc comment for report_too_many_hashesNicolas-1/+1
"to big" -> "too big"
2020-11-05Fix even more URLsGuillaume Gomez-1/+1
2020-09-03Rename IsJoint -> SpacingAleksey Kladov-7/+7
To match better naming from proc-macro
2020-09-03Condense StringReader's API to a single functionAleksey Kladov-29/+16
2020-09-01Don't emit trivia tokensAleksey Kladov-60/+53
2020-08-31Make StringReader privateAleksey Kladov-3/+3
After the recent refactorings, we can actually completely hide this type. It should help with #63689.
2020-08-30Remove unused functionAleksey Kladov-4/+0
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-07-04 10:14:12 +0000