about summary refs log tree commit diff
path: root/compiler/rustc_lexer/src/tests.rs
AgeCommit message (Collapse)AuthorLines
2025-07-10test(lexer): Add frontmatter unit testEd Page-2/+84
2025-07-09feat(lexer): Allow including frontmatter with 'tokenize'Ed Page-1/+2
2025-05-05Implement RFC 3503: frontmattersDeadbeef-1/+1
Supercedes #137193
2025-02-08Rustfmtbjorn3-10/+25
2024-11-25Improve `strip_shebang` testing.Nicholas Nethercote-41/+31
It's currently a bit ad hoc. This commit makes it more methodical, with pairs of match/no-match tests for all the relevant cases.
2024-09-22Reformat using the new identifier sorting from rustfmtMichael Goulet-26/+11
2024-07-29Reformat `use` declarations.Nicholas Nethercote-2/+2
The previous commit updated `rustfmt.toml` appropriately. This commit is the outcome of running `x fmt --all` with the new formatting options.
2023-04-10Revert "Don't recover lifetimes/labels containing emojis as character literals"Michael Goulet-1/+1
Reverts PR #108031 Fixes (doesnt close until beta backported) #109746 This reverts commit e3f9db5fc319c6d8eee5d47d216ea6a426070c41. This reverts commit 98b82aedba3f3f581e89df54352914b27f42c6f7. This reverts commit 380fa264132ad481e73cbbf0f3a0feefd99a1d78.
2023-02-14Update lexer lifetime test许杰友 Jieyou Xu (Joe)-1/+1
2022-08-01Shrink `Token`.Nicholas Nethercote-28/+22
From 72 bytes to 12 bytes (on x86-64). There are two parts to this: - Changing various source code offsets from 64-bit to 32-bit. This is not a problem because the rest of rustc also uses 32-bit source code offsets. This means `Token` is no longer `Copy` but this causes no problems. - Removing the `RawStrError` from `LiteralKind`. Raw string literal invalidity is now indicated by a `None` value within `RawStr`/`RawByteStr`, and the new `validate_raw_str` function can be used to re-lex an invalid raw string literal to get the `RawStrError`. There is one very small change in behaviour. Previously, if a raw string literal matched both the `InvalidStarter` and `TooManyHashes` cases, the latter would override the former. This has now changed, because `raw_double_quoted_string` now uses `?` and so returns immediately upon detecting the `InvalidStarter` case. I think this is a slight improvement to report the earlier-detected error, and it explains the change in the `test_too_many_hashes` test. The commit also removes a couple of comments that refer to #77629 and say that the size of these types don't affect performance. These comments are wrong, though the performance effect is small.
2022-03-23Reduce max hash in raw strings from u16 to u8Grisha Vartanyan-4/+4
2022-03-16Add test for >65535 hashes in lexing raw stringGrisha-0/+17
2020-08-30Move lexer unit tests to rustc_lexerAleksey Kladov-20/+140
StringReader is an intornal abstraction which at the moment changes a lot, so these unit tests cause quite a bit of friction. Moving them to rustc_lexer and more ingerated-testing style should make them much less annoying, hopefully without decreasing their usefulness much. Note that coloncolon tests are removed (it's unclear what those are testing). \r\n tests are removed as well, as we normalize line endings even before lexing.
2020-08-30mv compiler to compiler/mark-0/+167