summary refs log tree commit diff
path: root/compiler/rustc_parse/src/lexer/mod.rs
AgeCommit message (Collapse)AuthorLines
2024-01-29Stop using `String` for error codes.Nicholas Nethercote-8/+8
Error codes are integers, but `String` is used everywhere to represent them. Gross! This commit introduces `ErrCode`, an integral newtype for error codes, replacing `String`. It also introduces a constant for every error code, e.g. `E0123`, and removes the `error_code!` macro. The constants are imported wherever used with `use rustc_errors::codes::*`. With the old code, we have three different ways to specify an error code at a use point: ``` error_code!(E0123) // macro call struct_span_code_err!(dcx, span, E0123, "msg"); // bare ident arg to macro call \#[diag(name, code = "E0123")] // string struct Diag; ``` With the new code, they all use the `E0123` constant. ``` E0123 // constant struct_span_code_err!(dcx, span, E0123, "msg"); // constant \#[diag(name, code = E0123)] // constant struct Diag; ``` The commit also changes the structure of the error code definitions: - `rustc_error_codes` now just defines a higher-order macro listing the used error codes and nothing else. - Because that's now the only thing in the `rustc_error_codes` crate, I moved it into the `lib.rs` file and removed the `error_codes.rs` file. - `rustc_errors` uses that macro to define everything, e.g. the error code constants and the `DIAGNOSTIC_TABLES`. This is in its new `codes.rs` file.
2024-01-25Use `unescape_unicode` for raw C string literals.Nicholas Nethercote-1/+1
They can't contain `\x` escapes, which means they can't contain high bytes, which means we can used `unescape_unicode` instead of `unescape_mixed` to unescape them. This avoids unnecessary used of `MixedUnit`.
2024-01-25Rename the unescaping functions.Nicholas Nethercote-12/+12
`unescape_literal` becomes `unescape_unicode`, and `unescape_c_string` becomes `unescape_mixed`. Because rfc3349 will mean that C string literals will no longer be the only mixed utf8 literals.
2024-01-11Stop using `DiagnosticBuilder::buffer` in the parser.Nicholas Nethercote-4/+4
One consequence is that errors returned by `maybe_new_parser_from_source_str` now must be consumed, so a bunch of places that previously ignored those errors now cancel them. (Most of them explicitly dropped the errors before. I guess that was to indicate "we are explicitly ignoring these", though I'm not 100% sure.)
2024-01-11Fix lifetimes in `StringReader`.Nicholas Nethercote-12/+12
Two different lifetimes are conflated. This doesn't matter right now, but needs to be fixed for the next commit to work. And the more descriptive lifetime names make the code easier to read.
2024-01-10Rename consuming chaining methods on `DiagnosticBuilder`.Nicholas Nethercote-6/+6
In #119606 I added them and used a `_mv` suffix, but that wasn't great. A `with_` prefix has three different existing uses. - Constructors, e.g. `Vec::with_capacity`. - Wrappers that provide an environment to execute some code, e.g. `with_session_globals`. - Consuming chaining methods, e.g. `Span::with_{lo,hi,ctxt}`. The third case is exactly what we want, so this commit changes `DiagnosticBuilder::foo_mv` to `DiagnosticBuilder::with_foo`. Thanks to @compiler-errors for the suggestion.
2024-01-08Remove `DiagnosticBuilder::delay_as_bug_without_consuming`.Nicholas Nethercote-1/+1
The existing uses are replaced in one of three ways. - In a function that also has calls to `emit`, just rearrange the code so that exactly one of `delay_as_bug` or `emit` is called on every path. - In a function returning a `DiagnosticBuilder`, use `downgrade_to_delayed_bug`. That's good enough because it will get emitted later anyway. - In `unclosed_delim_err`, one set of errors is being replaced with another set, so just cancel the original errors.
2024-01-08Remove all eight `DiagnosticBuilder::*_with_code` methods.Nicholas Nethercote-36/+37
These all have relatively low use, and can be perfectly emulated with a simpler construction method combined with `code` or `code_mv`.
2024-01-08Use chaining in `DiagnosticBuilder` construction.Nicholas Nethercote-3/+3
To avoid the use of a mutable local variable, and because it reads more nicely.
2024-01-04Inline and remove `StringReader::struct_fatal_span_char`.Nicholas Nethercote-22/+11
It has a single call site.
2024-01-03Rename some `Diagnostic` setters.Nicholas Nethercote-1/+1
`Diagnostic` has 40 methods that return `&mut Self` and could be considered setters. Four of them have a `set_` prefix. This doesn't seem necessary for a type that implements the builder pattern. This commit removes the `set_` prefixes on those four methods.
2023-12-24Remove `Session` methods that duplicate `DiagCtxt` methods.Nicholas Nethercote-8/+8
Also add some `dcx` methods to types that wrap `TyCtxt`, for easier access.
2023-12-24Remove `ParseSess` methods that duplicate `DiagCtxt` methods.Nicholas Nethercote-11/+15
Also add missing `#[track_caller]` attributes to `DiagCtxt` methods as necessary to keep tests working.
2023-12-19Add `EmitResult` associated type to `EmissionGuarantee`.Nicholas Nethercote-2/+4
This lets different error levels share the same return type from `emit_*`. - A lot of inconsistencies in the `DiagCtxt` API are removed. - `Noted` is removed. - `FatalAbort` is introduced for fatal errors (abort via `raise`), replacing the `EmissionGuarantee` impl for `!`. - `Bug` is renamed `BugAbort` (to avoid clashing with `Level::Bug` and to mirror `FatalAbort`), and modified to work in the new way with bug errors (abort via panic). - Various diagnostic creators and emitters updated to the new, better signatures. Note that `DiagCtxt::bug` no longer needs to call `panic_any`, because `emit` handles that. Also shorten the obnoxiously long `diagnostic_builder_emit_producing_guarantee` name.
2023-12-18Rename `ParseSess::span_diagnostic` as `ParseSess::dcx`.Nicholas Nethercote-10/+10
2023-12-11Add spacing information to delimiters.Nicholas Nethercote-1/+1
This is an extension of the previous commit. It means the output of something like this: ``` stringify!(let a: Vec<u32> = vec![];) ``` goes from this: ``` let a: Vec<u32> = vec![] ; ``` With this PR, it now produces this string: ``` let a: Vec<u32> = vec![]; ```
2023-12-01Auto merge of #117472 - jmillikin:stable-c-str-literals, r=Nilstriebbors-3/+0
Stabilize C string literals RFC: https://rust-lang.github.io/rfcs/3348-c-str-literal.html Tracking issue: https://github.com/rust-lang/rust/issues/105723 Documentation PR (reference manual): https://github.com/rust-lang/reference/pull/1423 # Stabilization report Stabilizes C string and raw C string literals (`c"..."` and `cr#"..."#`), which are expressions of type [`&CStr`](https://doc.rust-lang.org/stable/core/ffi/struct.CStr.html). Both new literals require Rust edition 2021 or later. ```rust const HELLO: &core::ffi::CStr = c"Hello, world!"; ``` C strings may contain any byte other than `NUL` (`b'\x00'`), and their in-memory representation is guaranteed to end with `NUL`. ## Implementation Originally implemented by PR https://github.com/rust-lang/rust/pull/108801, which was reverted due to unintentional changes to lexer behavior in Rust editions < 2021. The current implementation landed in PR https://github.com/rust-lang/rust/pull/113476, which restricts C string literals to Rust edition >= 2021. ## Resolutions to open questions from the RFC * Adding C character literals (`c'.'`) of type `c_char` is not part of this feature. * Support for `c"..."` literals does not prevent `c'.'` literals from being added in the future. * C string literals should not be blocked on making `&CStr` a thin pointer. * It's possible to declare constant expressions of type `&'static CStr` in stable Rust (as of v1.59), so C string literals are not adding additional coupling on the internal representation of `CStr`. * The unstable `concat_bytes!` macro should not accept `c"..."` literals. * C strings have two equally valid `&[u8]` representations (with or without terminal `NUL`), so allowing them to be used in `concat_bytes!` would be ambiguous. * Adding a type to represent C strings containing valid UTF-8 is not part of this feature. * Support for a hypothetical `&Utf8CStr` may be explored in the future, should such a type be added to Rust.
2023-11-21Fix `clippy::needless_borrow` in the compilerNilstrieb-3/+3
`x clippy compiler -Aclippy::all -Wclippy::needless_borrow --fix`. Then I had to remove a few unnecessary parens and muts that were exposed now.
2023-11-01Stabilize C string literalsJohn Millikin-3/+0
2023-10-30When encountering unclosed delimiters during parsing, check for diff markersEsteban Küber-6/+8
Fix #116252.
2023-08-13Remove reached_eof from ParseSessbjorn3-1/+0
It was only ever set in a function which isn't called anywhere.
2023-07-25extract common codeDeadbeef-11/+10
2023-07-23reimplement C string literalsDeadbeef-1/+25
2023-05-16Avoid `&format("...")` calls in error message code.Nicholas Nethercote-1/+1
Error message all end up passing into a function as an `impl Into<{D,Subd}iagnosticMessage>`. If an error message is creatd as `&format("...")` that means we allocate a string (in the `format!` call), then take a reference, and then clone (allocating again) the reference to produce the `{D,Subd}iagnosticMessage`, which is silly. This commit removes the leading `&` from a lot of these cases. This means the original `String` is moved into the `{D,Subd}iagnosticMessage`, avoiding the double allocations. This requires changing some function argument types from `&str` to `String` (when all arguments are `String`) or `impl Into<{D,Subd}iagnosticMessage>` (when some arguments are `String` and some are `&str`).
2023-05-05Rollup merge of #108801 - fee1-dead-contrib:c-str, r=compiler-errorsDylan DPC-3/+60
Implement RFC 3348, `c"foo"` literals RFC: https://github.com/rust-lang/rfcs/pull/3348 Tracking issue: #105723
2023-05-03Restrict `From<S>` for `{D,Subd}iagnosticMessage`.Nicholas Nethercote-2/+2
Currently a `{D,Subd}iagnosticMessage` can be created from any type that impls `Into<String>`. That includes `&str`, `String`, and `Cow<'static, str>`, which are reasonable. It also includes `&String`, which is pretty weird, and results in many places making unnecessary allocations for patterns like this: ``` self.fatal(&format!(...)) ``` This creates a string with `format!`, takes a reference, passes the reference to `fatal`, which does an `into()`, which clones the reference, doing a second allocation. Two allocations for a single string, bleh. This commit changes the `From` impls so that you can only create a `{D,Subd}iagnosticMessage` from `&str`, `String`, or `Cow<'static, str>`. This requires changing all the places that currently create one from a `&String`. Most of these are of the `&format!(...)` form described above; each one removes an unnecessary static `&`, plus an allocation when executed. There are also a few places where the existing use of `&String` was more reasonable; these now just use `clone()` at the call site. As well as making the code nicer and more efficient, this is a step towards possibly using `Cow<'static, str>` in `{D,Subd}iagnosticMessage::{Str,Eager}`. That would require changing the `From<&'a str>` impls to `From<&'static str>`, which is doable, but I'm not yet sure if it's worthwhile.
2023-05-02make cook genericDeadbeef-37/+27
2023-05-02make it semantic errorDeadbeef-0/+3
2023-05-02initial step towards implementing C string literalsDeadbeef-0/+64
2023-04-25Fix static string lintsclubby789-5/+1
2023-04-17Spelling - compilerJosh Soref-1/+1
* account * achieved * advising * always * ambiguous * analysis * annotations * appropriate * build * candidates * cascading * category * character * clarification * compound * conceptually * constituent * consts * convenience * corresponds * debruijn * debug * debugable * debuggable * deterministic * discriminant * display * documentation * doesn't * ellipsis * erroneous * evaluability * evaluate * evaluation * explicitly * fallible * fulfill * getting * has * highlighting * illustrative * imported * incompatible * infringing * initialized * into * intrinsic * introduced * javascript * liveness * metadata * monomorphization * nonexistent * nontrivial * obligation * obligations * offset * opaque * opportunities * opt-in * outlive * overlapping * paragraph * parentheses * poisson * precisely * predecessors * predicates * preexisting * propagated * really * reentrant * referent * responsibility * rustonomicon * shortcircuit * simplifiable * simplifications * specify * stabilized * structurally * suggestibility * translatable * transmuting * two * unclosed * uninhabited * visibility * volatile * workaround Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-04-12Rollup merge of #110135 - compiler-errors:revert-108031, r=davidtwcoMatthias Krüger-7/+2
Revert "Don't recover lifetimes/labels containing emojis as character literals" Reverts PR #108031 per https://github.com/rust-lang/rust/pull/109754#issuecomment-1490452045 Fixes (doesnt close until beta backported) #109746 This reverts commit e3f9db5fc319c6d8eee5d47d216ea6a426070c41. This reverts commit 98b82aedba3f3f581e89df54352914b27f42c6f7. This reverts commit 380fa264132ad481e73cbbf0f3a0feefd99a1d78.
2023-04-10Revert "Don't recover lifetimes/labels containing emojis as character literals"Michael Goulet-7/+2
Reverts PR #108031 Fixes (doesnt close until beta backported) #109746 This reverts commit e3f9db5fc319c6d8eee5d47d216ea6a426070c41. This reverts commit 98b82aedba3f3f581e89df54352914b27f42c6f7. This reverts commit 380fa264132ad481e73cbbf0f3a0feefd99a1d78.
2023-04-09Remove identity castsNilstrieb-2/+2
2023-04-04Replace another lock with an append-only vecOli Scherer-1/+1
2023-02-28refactor parse_token_trees to not return unmatched_delimsyukang-3/+26
2023-02-28rename unmatched_braces to unmatched_delimsyukang-2/+2
2023-02-14Don't recover lifetimes/labels containing emojis as character literals许杰友 Jieyou Xu (Joe)-2/+7
Note that at the time of this commit, `unic-emoji-char` seems to have data tables only up to Unicode 5.0, but Unicode is already newer than this. A newer emoji such as `🥺` will not be recognized as an emoji but older emojis such as `🐱` will.
2023-02-06Migrate `rustc_parse` to derive diagnosticsclubby789-112/+47
2023-01-27Improve unexpected close and mismatch delimiter hint in TokenTreesReaderyukang-0/+1
2023-01-14Emit only one nbsp error per fileDavid Tolnay-5/+26
2023-01-12Emit a single error for contiguous sequences of Unicode homoglyphsclubby789-4/+24
2022-11-16Use `token::Lit` in `ast::ExprKind::Lit`.Nicholas Nethercote-1/+12
Instead of `ast::Lit`. Literal lowering now happens at two different times. Expression literals are lowered when HIR is crated. Attribute literals are lowered during parsing. This commit changes the language very slightly. Some programs that used to not compile now will compile. This is because some invalid literals that are removed by `cfg` or attribute macros will no longer trigger errors. See this comment for more details: https://github.com/rust-lang/rust/pull/102944#issuecomment-1277476773
2022-11-09Rollup merge of #103919 - nnethercote:unescaping-cleanups, r=matkladDylan DPC-59/+48
Unescaping cleanups Some code improvements, and some error message improvements. Best reviewed one commit at a time. r? ````@matklad````
2022-11-07Make underscore_literal_suffix a hard error.Nicholas Nethercote-11/+1
It's been a warning for 5.5 years. Time to make it a hard error. Closes #42326.
2022-11-04Inline and remove `validate_int_literal`.Nicholas Nethercote-18/+13
It has a single callsite, and is fairly small. The `Float` match arm already has base-specific checking inline, so this makes things more consistent.
2022-11-04Refactor `cook_lexer_literal`.Nicholas Nethercote-42/+36
It deals with eight cases: ints, floats, and the six quoted types (char/byte/strings). For ints and floats we have an early return, and the other six types fall through to the code at the end, which makes the function hard to read. This commit rearranges things to avoid the early returns.
2022-10-22Recover unclosed char literal being parsed as lifetimeMichael Goulet-2/+7
2022-10-03Merge `parse_token_trees_until_close_delim` and `parse_all_token_trees`.Nicholas Nethercote-1/+1
Because they're very similar, and this will allow some follow-up changes.
2022-09-28Address review comments.Nicholas Nethercote-1/+1
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-07-07 09:13:56 +0000