about summary refs log tree commit diff
path: root/compiler/rustc_parse/src/lexer
AgeCommit message (Collapse)AuthorLines
2023-12-19Add `EmitResult` associated type to `EmissionGuarantee`.Nicholas Nethercote-2/+4
This lets different error levels share the same return type from `emit_*`. - A lot of inconsistencies in the `DiagCtxt` API are removed. - `Noted` is removed. - `FatalAbort` is introduced for fatal errors (abort via `raise`), replacing the `EmissionGuarantee` impl for `!`. - `Bug` is renamed `BugAbort` (to avoid clashing with `Level::Bug` and to mirror `FatalAbort`), and modified to work in the new way with bug errors (abort via panic). - Various diagnostic creators and emitters updated to the new, better signatures. Note that `DiagCtxt::bug` no longer needs to call `panic_any`, because `emit` handles that. Also shorten the obnoxiously long `diagnostic_builder_emit_producing_guarantee` name.
2023-12-18Rename many `DiagCtxt` arguments.Nicholas Nethercote-25/+22
2023-12-18Rename `ParseSess::span_diagnostic` as `ParseSess::dcx`.Nicholas Nethercote-13/+13
2023-12-18Rename `Handler` as `DiagCtxt`.Nicholas Nethercote-2/+2
2023-12-16Auto merge of #118897 - nnethercote:more-unescaping-cleanups, r=fee1-deadbors-33/+42
More unescaping cleanups More minor improvements I found while working on #118699. r? `@fee1-dead`
2023-12-14Remove one use of `span_bug_no_panic`.Nicholas Nethercote-2/+1
It's unclear why this is used here. All entries in the third column of `UNICODE_ARRAY` are covered by `ASCII_ARRAY`, so if the lookup fails it's a genuine compiler bug. It was added way back in #29837, for no clear reason. This commit changes it to `span_bug`, which is more typical.
2023-12-13Rename the `span` args to `emit_unescape_error`.Nicholas Nethercote-33/+42
The `span` arg is described in a comment as "interior span of the literal, without quotes", which is incorrect. It's actually the span of the error part of the literal, corresponding to `range`. This commit renames `span` and `span_without_quotes` to make things clearer, and fixes the erroneous comment.
2023-12-11Add spacing information to delimiters.Nicholas Nethercote-35/+54
This is an extension of the previous commit. It means the output of something like this: ``` stringify!(let a: Vec<u32> = vec![];) ``` goes from this: ``` let a: Vec<u32> = vec![] ; ``` With this PR, it now produces this string: ``` let a: Vec<u32> = vec![]; ```
2023-12-11Improve `print_tts` by changing `tokenstream::Spacing`.Nicholas Nethercote-2/+7
`tokenstream::Spacing` appears on all `TokenTree::Token` instances, both punct and non-punct. Its current usage: - `Joint` means "can join with the next token *and* that token is a punct". - `Alone` means "cannot join with the next token *or* can join with the next token but that token is not a punct". The fact that `Alone` is used for two different cases is awkward. This commit augments `tokenstream::Spacing` with a new variant `JointHidden`, resulting in: - `Joint` means "can join with the next token *and* that token is a punct". - `JointHidden` means "can join with the next token *and* that token is a not a punct". - `Alone` means "cannot join with the next token". This *drastically* improves the output of `print_tts`. For example, this: ``` stringify!(let a: Vec<u32> = vec![];) ``` currently produces this string: ``` let a : Vec < u32 > = vec! [] ; ``` With this PR, it now produces this string: ``` let a: Vec<u32> = vec![] ; ``` (The space after the `]` is because `TokenTree::Delimited` currently doesn't have spacing information. The subsequent commit fixes this.) The new `print_tts` doesn't replicate original code perfectly. E.g. multiple space characters will be condensed into a single space character. But it's much improved. `print_tts` still produces the old, uglier output for code produced by proc macros. Because we have to translate the generated code from `proc_macro::Spacing` to the more expressive `token::Spacing`, which results in too much `proc_macro::Along` usage and no `proc_macro::JointHidden` usage. So `space_between` still exists and is used by `print_tts` in conjunction with the `Spacing` field. This change will also help with the removal of `Token::Interpolated`. Currently interpolated tokens are pretty-printed nicely via AST pretty printing. `Token::Interpolated` removal will mean they get printed with `print_tts`. Without this change, that would result in much uglier output for code produced by decl macro expansions. With this change, AST pretty printing and `print_tts` produce similar results. The commit also tweaks the comments on `proc_macro::Spacing`. In particular, it refers to "compound tokens" rather than "multi-char operators" because lifetimes aren't operators.
2023-12-01Auto merge of #117472 - jmillikin:stable-c-str-literals, r=Nilstriebbors-3/+0
Stabilize C string literals RFC: https://rust-lang.github.io/rfcs/3348-c-str-literal.html Tracking issue: https://github.com/rust-lang/rust/issues/105723 Documentation PR (reference manual): https://github.com/rust-lang/reference/pull/1423 # Stabilization report Stabilizes C string and raw C string literals (`c"..."` and `cr#"..."#`), which are expressions of type [`&CStr`](https://doc.rust-lang.org/stable/core/ffi/struct.CStr.html). Both new literals require Rust edition 2021 or later. ```rust const HELLO: &core::ffi::CStr = c"Hello, world!"; ``` C strings may contain any byte other than `NUL` (`b'\x00'`), and their in-memory representation is guaranteed to end with `NUL`. ## Implementation Originally implemented by PR https://github.com/rust-lang/rust/pull/108801, which was reverted due to unintentional changes to lexer behavior in Rust editions < 2021. The current implementation landed in PR https://github.com/rust-lang/rust/pull/113476, which restricts C string literals to Rust edition >= 2021. ## Resolutions to open questions from the RFC * Adding C character literals (`c'.'`) of type `c_char` is not part of this feature. * Support for `c"..."` literals does not prevent `c'.'` literals from being added in the future. * C string literals should not be blocked on making `&CStr` a thin pointer. * It's possible to declare constant expressions of type `&'static CStr` in stable Rust (as of v1.59), so C string literals are not adding additional coupling on the internal representation of `CStr`. * The unstable `concat_bytes!` macro should not accept `c"..."` literals. * C strings have two equally valid `&[u8]` representations (with or without terminal `NUL`), so allowing them to be used in `concat_bytes!` would be ambiguous. * Adding a type to represent C strings containing valid UTF-8 is not part of this feature. * Support for a hypothetical `&Utf8CStr` may be explored in the future, should such a type be added to Rust.
2023-11-21Fix `clippy::needless_borrow` in the compilerNilstrieb-6/+6
`x clippy compiler -Aclippy::all -Wclippy::needless_borrow --fix`. Then I had to remove a few unnecessary parens and muts that were exposed now.
2023-11-11Move unclosed delim errors to separate functionsjwang05-53/+58
2023-11-10Correctly handle while-let-chainssjwang05-1/+1
2023-11-09Catch an edge casesjwang05-1/+5
2023-11-09Catch stray { in let-chainssjwang05-1/+33
2023-11-01Stabilize C string literalsJohn Millikin-3/+0
2023-10-30When encountering unclosed delimiters during parsing, check for diff markersEsteban Küber-24/+54
Fix #116252.
2023-10-13Format all the let chains in compilerMichael Goulet-3/+4
2023-10-12Reorder an expression to improve readability.Nicholas Nethercote-12/+7
2023-10-12Rename `Token::is_op` as `Token::is_punct`.Nicholas Nethercote-2/+5
For consistency with `proc_macro::Punct`.
2023-08-16Fix suggestion for attempting to define a string with single quotesbeetrees-14/+8
2023-08-13Remove reached_eof from ParseSessbjorn3-1/+0
It was only ever set in a function which isn't called anywhere.
2023-07-30inline format!() args up to and including rustc_middleMatthias Krüger-11/+10
2023-07-25Auto merge of #113476 - fee1-dead-contrib:c-str-lit, r=petrochenkovbors-9/+32
Reimplement C-str literals This reverts #113334, cc `@fmease.` While converting lexer tokens to ast Tokens in `rustc_parse`, we check the edition of the span of the token. If the edition < 2021, we split the token into two, one being the identifier and other being the str literal.
2023-07-25extract common codeDeadbeef-11/+10
2023-07-23fix some clippy::style findingsMatthias Krüger-1/+1
comparison_to_empty iter_nth_zero for_kv_map manual_next_back redundant_pattern
2023-07-23reimplement C string literalsDeadbeef-1/+25
2023-06-10Use a better linkHankai Zhang-1/+1
2023-06-10Update links to Rust Reference page on literals in diagnosticHankai Zhang-1/+1
Instead of linking to the old Rust Reference site on static.rust-lang.org, link to the current website doc.rust-lang.org/stable/reference instead in diagnostic about incorrect literals.
2023-05-16Avoid `&format("...")` calls in error message code.Nicholas Nethercote-1/+1
Error message all end up passing into a function as an `impl Into<{D,Subd}iagnosticMessage>`. If an error message is creatd as `&format("...")` that means we allocate a string (in the `format!` call), then take a reference, and then clone (allocating again) the reference to produce the `{D,Subd}iagnosticMessage`, which is silly. This commit removes the leading `&` from a lot of these cases. This means the original `String` is moved into the `{D,Subd}iagnosticMessage`, avoiding the double allocations. This requires changing some function argument types from `&str` to `String` (when all arguments are `String`) or `impl Into<{D,Subd}iagnosticMessage>` (when some arguments are `String` and some are `&str`).
2023-05-05Rollup merge of #108801 - fee1-dead-contrib:c-str, r=compiler-errorsDylan DPC-11/+74
Implement RFC 3348, `c"foo"` literals RFC: https://github.com/rust-lang/rfcs/pull/3348 Tracking issue: #105723
2023-05-03Restrict `From<S>` for `{D,Subd}iagnosticMessage`.Nicholas Nethercote-12/+8
Currently a `{D,Subd}iagnosticMessage` can be created from any type that impls `Into<String>`. That includes `&str`, `String`, and `Cow<'static, str>`, which are reasonable. It also includes `&String`, which is pretty weird, and results in many places making unnecessary allocations for patterns like this: ``` self.fatal(&format!(...)) ``` This creates a string with `format!`, takes a reference, passes the reference to `fatal`, which does an `into()`, which clones the reference, doing a second allocation. Two allocations for a single string, bleh. This commit changes the `From` impls so that you can only create a `{D,Subd}iagnosticMessage` from `&str`, `String`, or `Cow<'static, str>`. This requires changing all the places that currently create one from a `&String`. Most of these are of the `&format!(...)` form described above; each one removes an unnecessary static `&`, plus an allocation when executed. There are also a few places where the existing use of `&String` was more reasonable; these now just use `clone()` at the call site. As well as making the code nicer and more efficient, this is a step towards possibly using `Cow<'static, str>` in `{D,Subd}iagnosticMessage::{Str,Eager}`. That would require changing the `From<&'a str>` impls to `From<&'static str>`, which is doable, but I'm not yet sure if it's worthwhile.
2023-05-02make cook genericDeadbeef-37/+27
2023-05-02make it semantic errorDeadbeef-0/+3
2023-05-02refactor unescapeDeadbeef-8/+14
2023-05-02initial step towards implementing C string literalsDeadbeef-0/+64
2023-04-25Fix static string lintsclubby789-5/+1
2023-04-17Spelling - compilerJosh Soref-1/+1
* account * achieved * advising * always * ambiguous * analysis * annotations * appropriate * build * candidates * cascading * category * character * clarification * compound * conceptually * constituent * consts * convenience * corresponds * debruijn * debug * debugable * debuggable * deterministic * discriminant * display * documentation * doesn't * ellipsis * erroneous * evaluability * evaluate * evaluation * explicitly * fallible * fulfill * getting * has * highlighting * illustrative * imported * incompatible * infringing * initialized * into * intrinsic * introduced * javascript * liveness * metadata * monomorphization * nonexistent * nontrivial * obligation * obligations * offset * opaque * opportunities * opt-in * outlive * overlapping * paragraph * parentheses * poisson * precisely * predecessors * predicates * preexisting * propagated * really * reentrant * referent * responsibility * rustonomicon * shortcircuit * simplifiable * simplifications * specify * stabilized * structurally * suggestibility * translatable * transmuting * two * unclosed * uninhabited * visibility * volatile * workaround Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-04-12Auto merge of #110252 - matthiaskrgr:rollup-ovaixra, r=matthiaskrgrbors-7/+2
Rollup of 8 pull requests Successful merges: - #109810 (Replace rustdoc-ui/{c,z}-help tests with a stable run-make test ) - #110035 (fix: ensure bad `#[test]` invocs retain correct AST) - #110089 (sync::mpsc: synchronize receiver disconnect with initialization) - #110103 (Report overflows gracefully with new solver) - #110122 (Fix x check --stage 1 when download-ci-llvm=false) - #110133 (Do not use ImplDerivedObligationCause for inherent impl method error reporting) - #110135 (Revert "Don't recover lifetimes/labels containing emojis as character literals") - #110235 (Fix `--extend-css` option) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup
2023-04-12Rollup merge of #110135 - compiler-errors:revert-108031, r=davidtwcoMatthias Krüger-7/+2
Revert "Don't recover lifetimes/labels containing emojis as character literals" Reverts PR #108031 per https://github.com/rust-lang/rust/pull/109754#issuecomment-1490452045 Fixes (doesnt close until beta backported) #109746 This reverts commit e3f9db5fc319c6d8eee5d47d216ea6a426070c41. This reverts commit 98b82aedba3f3f581e89df54352914b27f42c6f7. This reverts commit 380fa264132ad481e73cbbf0f3a0feefd99a1d78.
2023-04-10Fix typos in compilerDaniPopes-5/+5
2023-04-10Revert "Don't recover lifetimes/labels containing emojis as character literals"Michael Goulet-7/+2
Reverts PR #108031 Fixes (doesnt close until beta backported) #109746 This reverts commit e3f9db5fc319c6d8eee5d47d216ea6a426070c41. This reverts commit 98b82aedba3f3f581e89df54352914b27f42c6f7. This reverts commit 380fa264132ad481e73cbbf0f3a0feefd99a1d78.
2023-04-09Remove identity castsNilstrieb-2/+2
2023-04-09Fix some clippy::complexityNilstrieb-2/+2
2023-04-04Replace another lock with an append-only vecOli Scherer-1/+1
2023-03-07Simplify `sort_by` callsMaybe Waffle-1/+1
2023-02-28refactor parse_token_trees to not return unmatched_delimsyukang-4/+26
2023-02-28rename unmatched_braces to unmatched_delimsyukang-12/+13
2023-02-28remove duplicated diagnostic for unclosed delimiteryukang-8/+9
2023-02-14Don't recover lifetimes/labels containing emojis as character literals许杰友 Jieyou Xu (Joe)-2/+7
Note that at the time of this commit, `unic-emoji-char` seems to have data tables only up to Unicode 5.0, but Unicode is already newer than this. A newer emoji such as `🥺` will not be recognized as an emoji but older emojis such as `🐱` will.
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-02-22 23:29:58 +0000