about summary refs log tree commit diff
path: root/compiler/rustc_parse/src/parser/mod.rs
AgeCommit message (Collapse)AuthorLines
2023-12-30Update to bitflags 2 in the compilerNilstrieb-0/+1
This involves lots of breaking changes. There are two big changes that force changes. The first is that the bitflag types now don't automatically implement normal derive traits, so we need to derive them manually. Additionally, bitflags now have a hidden inner type by default, which breaks our custom derives. The bitflags docs recommend using the impl form in these cases, which I did.
2023-12-28Rollup merge of #119379 - ShE3py:parse-seq-doc, r=compiler-errorsMatthias Krüger-10/+28
Update `parse_seq` doc Some doc changes I made while working on an issue.
2023-12-28Update `parse_seq` docLieselotte-10/+28
2023-12-27Simplify Parser::ident_or_errorDaniPopes-12/+4
Avoid a nested `Result<T, PResult<T>>`.
2023-12-24Remove `ParseSess` methods that duplicate `DiagCtxt` methods.Nicholas Nethercote-3/+4
Also add missing `#[track_caller]` attributes to `DiagCtxt` methods as necessary to keep tests working.
2023-12-23Give `DiagnosticBuilder` a default type.Nicholas Nethercote-3/+3
`IntoDiagnostic` defaults to `ErrorGuaranteed`, because errors are the most common diagnostic level. It makes sense to do likewise for the closely-related (and much more widely used) `DiagnosticBuilder` type, letting us write `DiagnosticBuilder<'a, ErrorGuaranteed>` as just `DiagnosticBuilder<'a>`. This cuts over 200 lines of code due to many multi-line things becoming single line things.
2023-12-18Use `.into_diagnostic()` less.Nicholas Nethercote-6/+3
This commit replaces this pattern: ``` err.into_diagnostic(dcx) ``` with this pattern: ``` dcx.create_err(err) ``` in a lot of places. It's a little shorter, makes the error level explicit, avoids some `IntoDiagnostic` imports, and is a necessary prerequisite for the next commit which will add a `level` arg to `into_diagnostic`. This requires adding `track_caller` on `create_err` to avoid mucking up the output of `tests/ui/track-diagnostics/track4.rs`. It probably should have been there already.
2023-12-18Rename `ParseSess::span_diagnostic` as `ParseSess::dcx`.Nicholas Nethercote-1/+1
2023-12-11Add spacing information to delimiters.Nicholas Nethercote-17/+18
This is an extension of the previous commit. It means the output of something like this: ``` stringify!(let a: Vec<u32> = vec![];) ``` goes from this: ``` let a: Vec<u32> = vec![] ; ``` With this PR, it now produces this string: ``` let a: Vec<u32> = vec![]; ```
2023-12-11Improve `print_tts` by changing `tokenstream::Spacing`.Nicholas Nethercote-5/+12
`tokenstream::Spacing` appears on all `TokenTree::Token` instances, both punct and non-punct. Its current usage: - `Joint` means "can join with the next token *and* that token is a punct". - `Alone` means "cannot join with the next token *or* can join with the next token but that token is not a punct". The fact that `Alone` is used for two different cases is awkward. This commit augments `tokenstream::Spacing` with a new variant `JointHidden`, resulting in: - `Joint` means "can join with the next token *and* that token is a punct". - `JointHidden` means "can join with the next token *and* that token is a not a punct". - `Alone` means "cannot join with the next token". This *drastically* improves the output of `print_tts`. For example, this: ``` stringify!(let a: Vec<u32> = vec![];) ``` currently produces this string: ``` let a : Vec < u32 > = vec! [] ; ``` With this PR, it now produces this string: ``` let a: Vec<u32> = vec![] ; ``` (The space after the `]` is because `TokenTree::Delimited` currently doesn't have spacing information. The subsequent commit fixes this.) The new `print_tts` doesn't replicate original code perfectly. E.g. multiple space characters will be condensed into a single space character. But it's much improved. `print_tts` still produces the old, uglier output for code produced by proc macros. Because we have to translate the generated code from `proc_macro::Spacing` to the more expressive `token::Spacing`, which results in too much `proc_macro::Along` usage and no `proc_macro::JointHidden` usage. So `space_between` still exists and is used by `print_tts` in conjunction with the `Spacing` field. This change will also help with the removal of `Token::Interpolated`. Currently interpolated tokens are pretty-printed nicely via AST pretty printing. `Token::Interpolated` removal will mean they get printed with `print_tts`. Without this change, that would result in much uglier output for code produced by decl macro expansions. With this change, AST pretty printing and `print_tts` produce similar results. The commit also tweaks the comments on `proc_macro::Spacing`. In particular, it refers to "compound tokens" rather than "multi-char operators" because lifetimes aren't operators.
2023-12-08Support async gen fnMichael Goulet-16/+23
2023-12-04Option<CoroutineKind>Eric Holk-8/+8
2023-12-04Merge Async and Gen into CoroutineKindEric Holk-8/+16
2023-11-29Bubble parse error when expecting `)`Esteban Küber-0/+3
2023-11-17Auto merge of #114292 - estebank:issue-71039, r=b-naberbors-9/+11
More detail when expecting expression but encountering bad macro argument On nested macro invocations where the same macro fragment changes fragment type from one to the next, point at the chain of invocations and at the macro fragment definition place, explaining that the change has occurred. Fix #71039. ``` error: expected expression, found pattern `1 + 1` --> $DIR/trace_faulty_macros.rs:49:37 | LL | (let $p:pat = $e:expr) => {test!(($p,$e))}; | ------- -- this is interpreted as expression, but it is expected to be pattern | | | this macro fragment matcher is expression ... LL | (($p:pat, $e:pat)) => {let $p = $e;}; | ------ ^^ expected expression | | | this macro fragment matcher is pattern ... LL | test!(let x = 1+1); | ------------------ | | | | | this is expected to be expression | in this macro invocation | = note: when forwarding a matched fragment to another macro-by-example, matchers in the second macro will see an opaque AST of the fragment type, not the underlying tokens = note: this error originates in the macro `test` (in Nightly builds, run with -Z macro-backtrace for more info) ```
2023-11-16Fix code indentationEsteban Küber-2/+2
2023-11-16More detail when expecting expression but encountering bad macro argumentEsteban Küber-9/+11
Partially address #71039.
2023-11-15Re-format code with new rustfmtMark Rousskov-9/+9
2023-11-12Fix `is_keyword_ahead` visibilitysjwang05-1/+1
Co-authored-by: Takayuki Maeda <takoyaki0316@gmail.com>
2023-11-10Correctly handle while-let-chainssjwang05-1/+1
2023-11-02Minimize `pub` usage in `source_map.rs`.Nicholas Nethercote-1/+1
Most notably, this commit changes the `pub use crate::*;` in that file to `use crate::*;`. This requires a lot of `use` items in other crates to be adjusted, because everything defined within `rustc_span::*` was also available via `rustc_span::source_map::*`, which is bizarre. The commit also removes `SourceMap::span_to_relative_line_string`, which is unused.
2023-10-29Auto merge of #116447 - oli-obk:gen_fn, r=compiler-errorsbors-0/+11
Implement `gen` blocks in the 2024 edition Coroutines tracking issue https://github.com/rust-lang/rust/issues/43122 `gen` block tracking issue https://github.com/rust-lang/rust/issues/117078 This PR implements `gen` blocks that implement `Iterator`. Most of the logic with `async` blocks is shared, and thus I renamed various types that were referring to `async` specifically. An example usage of `gen` blocks is ```rust fn foo() -> impl Iterator<Item = i32> { gen { yield 42; for i in 5..18 { if i.is_even() { continue } yield i * 2; } } } ``` The limitations (to be resolved) of the implementation are listed in the tracking issue
2023-10-26Reserve `gen` keyword for `gen {}` blocks and `gen fn` in 2024 editionOli Scherer-0/+11
2023-10-25Avoid unbounded O(n^2) when parsing nested type argsEsteban Küber-2/+4
When encountering code like `f::<f::<f::<f::<f::<f::<f::<f::<...` with unmatched closing angle brackets, add a linear check that avoids the exponential behavior of the parse recovery mechanism. Fix #117080.
2023-10-13Format all the let chains in compilerMichael Goulet-8/+7
2023-10-03Detect missing `=>` after match guard during parsingEsteban Küber-0/+1
``` error: expected one of `,`, `:`, or `}`, found `.` --> $DIR/missing-fat-arrow.rs:25:14 | LL | Some(a) if a.value == b { | - while parsing this struct LL | a.value = 1; | -^ expected one of `,`, `:`, or `}` | | | while parsing this struct field | help: try naming a field | LL | a: a.value = 1; | ++ help: you might have meant to start a match arm after the match guard | LL | Some(a) if a.value == b => { | ++ ``` Fix #78585.
2023-09-11Move let expression checking to parsingMatthew Jasper-0/+1
There was an incomplete version of the check in parsing and a second version in AST validation. This meant that some, but not all, invalid uses were allowed inside macros/disabled cfgs. It also means that later passes have a hard time knowing when the let expression is in a valid location, sometimes causing ICEs. - Add a field to ExprKind::Let in AST/HIR to mark whether it's in a valid location. - Suppress later errors and MIR construction for invalid let expressions.
2023-08-18Rename `NtOrTt` as `ParseNtResult`.Nicholas Nethercote-1/+1
It's more descriptive, and future-proofs it if/when additional variants get added.
2023-08-17Add some useful comments to `Parser::look_ahead`.Nicholas Nethercote-8/+23
2023-08-13Remove reached_eof from ParseSessbjorn3-13/+0
It was only ever set in a function which isn't called anywhere.
2023-08-03Remove `MacDelimiter`.Nicholas Nethercote-4/+2
It's the same as `Delimiter`, minus the `Invisible` variant. I'm generally in favour of using types to make impossible states unrepresentable, but this one feels very low-value, and the conversions between the two types are annoying and confusing. Look at the change in `src/tools/rustfmt/src/expr.rs` for an example: the old code converted from `MacDelimiter` to `Delimiter` and back again, for no good reason. This suggests the author was confused about the types.
2023-08-02Avoid an unnecessary local variable.Nicholas Nethercote-2/+1
2023-08-02Move `TokenCursor::break_last_token` into `Parser`.Nicholas Nethercote-31/+20
Similar to the last commit, it's more of a `Parser`-level concern than a `TokenCursor`-level concern. And the struct size reductions are nice. After this change, `TokenCursor` is as minimal as possible (two fields and two methods) which is nice.
2023-08-02Move `TokenCursor::num_next_calls` into `Parser` and rename it.Nicholas Nethercote-8/+5
It's more of a `Parser`-level concern than a `TokenCursor`-level concern. Also, `num_bump_calls` is a more accurate name, because it's incremented in `Parser::bump`.
2023-08-02Inline and remove `parse_all_token_trees`.Nicholas Nethercote-10/+1
It has a single call site.
2023-08-02`parse_all_token_trees` cannot fail.Nicholas Nethercote-2/+2
2023-08-01Auto merge of #114273 - nnethercote:move-doc-comment-desugaring, r=petrochenkovbors-78/+13
Move doc comment desugaring out of `TokenCursor`. It's awkward that `TokenCursor` sometimes desugars doc comments on the fly, but usually doesn't. r? `@petrochenkov`
2023-07-31Remove `desugar_doc_comments` arg from `Parser::new()`.Nicholas Nethercote-6/+1
It's only true at one call site; do the desugaring there instead.
2023-07-31Move doc comment desugaring out of `TokenCursor`.Nicholas Nethercote-73/+13
`TokenCursor` currently does doc comment desugaring on the fly, if the `desugar_doc_comment` field is set. This requires also modifying the token stream on the fly with `replace_prev_and_rewind`. This commit moves the doc comment desugaring out of `TokenCursor`, by introducing a new `TokenStream::desugar_doc_comment` method. This separation of desugaring and iterating makes the code nicer.
2023-07-31Fix a typo in a comment.Nicholas Nethercote-3/+3
2023-07-31Remove an unnecessary `return` keyword.Nicholas Nethercote-1/+1
2023-07-30inline format!() args up to and including rustc_middleMatthias Krüger-3/+3
2023-07-26Add a comment to `TokenCursor::desugar_doc_comments`.Nicholas Nethercote-0/+3
Useful information that took me some time to discern.
2023-07-26Remove `desugar_doc_comments` arguments from `TokenCursor::{inlined_,}next`.Nicholas Nethercote-18/+23
Because it's now always `self.desugar_doc_comments`.
2023-07-26Tweak `Parser::look_ahead`.Nicholas Nethercote-1/+1
It doesn't really matter what the `desugar_doc_comments` argument is here, because in practice we never look ahead through doc comments. Changing it to `cursor.desugar_doc_comments` will allow some follow-up simplifications.
2023-07-26Remove `Parser::desugar_doc_comments`.Nicholas Nethercote-3/+1
It's currently stored twice: once in `Parser`, once in the `TokenStream` within `Parser`. We only need the latter.
2023-07-19Fix inline_const with interpolated blockMichael Goulet-1/+2
2023-07-12Re-format let-else per rustfmt updateMark Rousskov-1/+3
2023-05-13fix(parse): return unpected when current token is EOFbohan-1/+3
2023-05-03Restrict `From<S>` for `{D,Subd}iagnosticMessage`.Nicholas Nethercote-2/+2
Currently a `{D,Subd}iagnosticMessage` can be created from any type that impls `Into<String>`. That includes `&str`, `String`, and `Cow<'static, str>`, which are reasonable. It also includes `&String`, which is pretty weird, and results in many places making unnecessary allocations for patterns like this: ``` self.fatal(&format!(...)) ``` This creates a string with `format!`, takes a reference, passes the reference to `fatal`, which does an `into()`, which clones the reference, doing a second allocation. Two allocations for a single string, bleh. This commit changes the `From` impls so that you can only create a `{D,Subd}iagnosticMessage` from `&str`, `String`, or `Cow<'static, str>`. This requires changing all the places that currently create one from a `&String`. Most of these are of the `&format!(...)` form described above; each one removes an unnecessary static `&`, plus an allocation when executed. There are also a few places where the existing use of `&String` was more reasonable; these now just use `clone()` at the call site. As well as making the code nicer and more efficient, this is a step towards possibly using `Cow<'static, str>` in `{D,Subd}iagnosticMessage::{Str,Eager}`. That would require changing the `From<&'a str>` impls to `From<&'static str>`, which is doable, but I'm not yet sure if it's worthwhile.