about summary refs log tree commit diff
path: root/compiler/rustc_parse/src/parser
AgeCommit message (Collapse)AuthorLines
2020-11-28Auto merge of #78296 - Aaron1011:fix/stmt-tokens, r=petrochenkovbors-44/+103
Properly handle attributes on statements We now collect tokens for the underlying node wrapped by `StmtKind` nstead of storing tokens directly in `Stmt`. `LazyTokenStream` now supports capturing a trailing semicolon after it is initially constructed. This allows us to avoid refactoring statement parsing to wrap the parsing of the semicolon in `parse_tokens`. Attributes on item statements (e.g. `fn foo() { #[bar] struct MyStruct; }`) are now treated as item attributes, not statement attributes, which is consistent with how we handle attributes on other kinds of statements. The feature-gating code is adjusted so that proc-macro attributes are still allowed on item statements on stable. Two built-in macros (`#[global_allocator]` and `#[test]`) needed to be adjusted to support being passed `Annotatable::Stmt`.
2020-11-27Auto merge of #79433 - calebcartwright:parse-attr-vis, r=petrochenkovbors-2/+4
rustc_parse: restore public visibility on parse_attribute Make `parse_attribute` public as rustfmt is a downstream consumer. Refs https://github.com/rust-lang/rust/pull/78782#discussion_r530658904 r? `@petrochenkov`
2020-11-27Auto merge of #79266 - b-naber:gat_trait_path_parser, r=petrochenkovbors-26/+83
Generic Associated Types in Trait Paths - Ast part The Ast part of https://github.com/rust-lang/rust/pull/78978 r? `@petrochenkov`
2020-11-26Only eat semicolons for statements that need themAaron Hill-2/+1
When parsing a statement (e.g. inside a function body), we now consider `struct Foo {};` and `$stmt;` to each consist of two statements: `struct Foo {}` and `;`, and `$stmt` and `;`. As a result, an attribute macro invoke as `fn foo() { #[attr] struct Bar{}; }` will see `struct Bar{}` as its input. Additionally, the 'unused semicolon' lint now fires in more places.
2020-11-26Properly handle attributes on statementsAaron Hill-42/+102
We now collect tokens for the underlying node wrapped by `StmtKind` instead of storing tokens directly in `Stmt`. `LazyTokenStream` now supports capturing a trailing semicolon after it is initially constructed. This allows us to avoid refactoring statement parsing to wrap the parsing of the semicolon in `parse_tokens`. Attributes on item statements (e.g. `fn foo() { #[bar] struct MyStruct; }`) are now treated as item attributes, not statement attributes, which is consistent with how we handle attributes on other kinds of statements. The feature-gating code is adjusted so that proc-macro attributes are still allowed on item statements on stable. Two built-in macros (`#[global_allocator]` and `#[test]`) needed to be adjusted to support being passed `Annotatable::Stmt`.
2020-11-26rustc_parse: restore pub vis on parse_attributeCaleb Cartwright-2/+4
2020-11-25ast and parserb-naber-26/+83
2020-11-19Rollup merge of #79185 - petrochenkov:derattr2, r=Aaron1011Dylan DPC-7/+6
expand/resolve: Pre-requisites to "Turn `#[derive]` into a regular macro attribute" Miscellaneous refactorings and error reporting changes extracted from https://github.com/rust-lang/rust/pull/79078. Unlike https://github.com/rust-lang/rust/pull/79078 this PR doesn't make any observable changes to the language or library. r? ```@Aaron1011```
2020-11-19resolve/expand: Misc cleanupVadim Petrochenkov-7/+6
2020-11-18Permit standalone generic parameters as const generic arguments in macrosvarkor-3/+15
2020-11-15Rollup merge of #79016 - fanzier:underscore-expressions, r=petrochenkovJonas Schievink-0/+3
Make `_` an expression, to discard values in destructuring assignments This is the third and final step towards implementing destructuring assignment (RFC: rust-lang/rfcs#2909, tracking issue: #71126). This PR is the third and final part of #71156, which was split up to allow for easier review. With this PR, an underscore `_` is parsed as an expression but is allowed *only* on the left-hand side of a destructuring assignment. There it simply discards a value, similarly to the wildcard `_` in patterns. For instance, ```rust (a, _) = (1, 2) ``` will simply assign 1 to `a` and discard the 2. Note that for consistency, ``` _ = foo ``` is also allowed and equivalent to just `foo`. Thanks to ````@varkor```` who helped with the implementation, particularly around pre-expansion gating. r? ````@petrochenkov````
2020-11-14Add underscore expressions for destructuring assignmentsFabian Zaiser-0/+3
Co-authored-by: varkor <github@varkor.com>
2020-11-14Auto merge of #78736 - petrochenkov:lazyenum, r=Aaron1011bors-9/+5
rustc_parse: Remove optimization for 0-length streams in `collect_tokens` The optimization conflates empty token streams with unknown token stream, which is at least suspicious, and doesn't affect performance because 0-length token streams are very rare. r? `@Aaron1011`
2020-11-12rustc_parse: Remove optimization for 0-length streams in `collect_tokens`Vadim Petrochenkov-9/+5
The optimization conflates empty token streams with unknown token stream, which is at least suspicious, and doesn't affect performance because 0-length token streams are very rare.
2020-11-12Rollup merge of #78836 - fanzier:struct-and-slice-destructuring, r=petrochenkovMara Bos-2/+8
Implement destructuring assignment for structs and slices This is the second step towards implementing destructuring assignment (RFC: rust-lang/rfcs#2909, tracking issue: #71126). This PR is the second part of #71156, which was split up to allow for easier review. Note that the first PR (#78748) is not merged yet, so it is included as the first commit in this one. I thought this would allow the review to start earlier because I have some time this weekend to respond to reviews. If ``@petrochenkov`` prefers to wait until the first PR is merged, I totally understand, of course. This PR implements destructuring assignment for (tuple) structs and slices. In order to do this, the following *parser change* was necessary: struct expressions are not required to have a base expression, i.e. `Struct { a: 1, .. }` becomes legal (in order to act like a struct pattern). Unfortunately, this PR slightly regresses the diagnostics implemented in #77283. However, it is only a missing help message in `src/test/ui/issues/issue-77218.rs`. Other instances of this diagnostic are not affected. Since I don't exactly understand how this help message works and how to fix it yet, I was hoping it's OK to regress this temporarily and fix it in a follow-up PR. Thanks to ``@varkor`` who helped with the implementation, particularly around the struct rest changes. r? ``@petrochenkov``
2020-11-12Auto merge of #78782 - petrochenkov:nodoctok, r=Aaron1011bors-99/+74
Do not collect tokens for doc comments Doc comment is a single token and AST has all the information to re-create it precisely. Doc comments are also responsible for majority of calls to `collect_tokens` (with `num_calls == 1` and `num_calls == 0`, cc https://github.com/rust-lang/rust/pull/78736). (I also moved token collection into `fn parse_attribute` to deduplicate code a bit.) r? `@Aaron1011`
2020-11-11Implement destructuring assignment for structs and slicesFabian Zaiser-2/+8
Co-authored-by: varkor <github@varkor.com>
2020-11-09Rollup merge of #78710 - petrochenkov:macvisit, r=davidtwcoDylan DPC-5/+1
rustc_ast: Do not panic by default when visiting macro calls Panicking by default made sense when we didn't have HIR or MIR and everything worked on AST, but now all AST visitors run early and majority of them have to deal with macro calls, often by ignoring them. The second commit renames `visit_mac` to `visit_mac_call`, the corresponding structures were renamed earlier in https://github.com/rust-lang/rust/pull/69589.
2020-11-09Do not collect tokens for doc commentsVadim Petrochenkov-99/+74
2020-11-03rustc_ast: `visit_mac` -> `visit_mac_call`Vadim Petrochenkov-1/+1
2020-11-03rustc_ast: Do not panic by default when visiting macro callsVadim Petrochenkov-4/+0
2020-11-02Use reparsed `TokenStream` if we captured any inner attributesAaron Hill-24/+9
Fixes #78675 We now bail out of `prepend_attrs` if we ended up capturing any inner attributes (which can happen in several places, due to token capturing for `macro_rules!` arguments.
2020-10-31parser: Cleanup `LazyTokenStream` and avoid some clonesVadim Petrochenkov-26/+37
by using a named struct instead of a closure.
2020-10-30Add back missing commentsJoshua Nelson-2/+3
2020-10-30Fix even more clippy warningsJoshua Nelson-52/+38
2020-10-30Rollup merge of #78523 - estebank:fix-return-type-parse-regression, r=dtolnayYuki Okushi-13/+4
Revert invalid `fn` return type parsing change Revert one of the changes in #78379. Fix #78507.
2020-10-29Revert invalid `fn` return type parsing changeEsteban Küber-13/+4
Fix #78507.
2020-10-29Rollup merge of #78460 - varkor:turbofish-string-generic, r=lcnrYuki Okushi-4/+5
Adjust turbofish help message for const generics Types are no longer special. (This message arguably only makes sense with `min_const_generics` or more, but we'll be there soon.) r? @lcnr
2020-10-28Adjust turbofish help message for const genericsvarkor-4/+5
2020-10-28Rollup merge of #78453 - Storyyeller:patch-1, r=jonas-schievinkDylan DPC-1/+1
Fix typo in comments
2020-10-28Rollup merge of #78379 - estebank:fn-signature-parse, r=varkorDylan DPC-18/+45
Tweak invalid `fn` header and body parsing * Rely on regular "expected"/"found" parser error for `fn`, fix #77115 * Recover empty `fn` bodies when encountering `}` * Recover trailing `>` in return types * Recover from non-type in array type `[<BAD TOKEN>; LEN]`
2020-10-27Fix typo in commentsRobert Grosse-1/+1
2020-10-27Auto merge of #77502 - varkor:const-generics-suggest-enclosing-braces, ↵bors-22/+190
r=petrochenkov Suggest that expressions that look like const generic arguments should be enclosed in brackets I pulled out the changes for const expressions from https://github.com/rust-lang/rust/pull/71592 (without the trait object diagnostic changes) and made some small changes; the implementation is `@estebank's.` We're also going to want to make some changes separately to account for trait objects (they result in poor diagnostics, as is evident from one of the test cases here), such as an adaption of https://github.com/rust-lang/rust/pull/72273. Fixes https://github.com/rust-lang/rust/issues/70753. r? `@petrochenkov`
2020-10-26Suggest expressions that look like const generic arguments should be ↵varkor-22/+190
enclosed in brackets Co-Authored-By: Esteban Kuber <github@kuber.com.ar>
2020-10-26Rollup merge of #78214 - estebank:match-semicolon, r=oli-obkDylan DPC-1/+7
Tweak match arm semicolon removal suggestion to account for futures * Tweak and extend "use `.await`" suggestions * Suggest removal of semicolon on prior match arm * Account for `impl Future` when suggesting semicolon removal * Silence some errors when encountering `await foo()?` as can't be certain what the intent was *Thanks to https://twitter.com/a_hoverbear/status/1318960787105353728 for pointing this out!*
2020-10-25Tweak invalid `fn` header and body parsingEsteban Küber-9/+43
* Recover empty `fn` bodies when encountering `}` * Recover trailing `>` in return types * Recover from non-type in array type `[<BAD TOKEN>; LEN]`
2020-10-25Rely on regular "expected"/"found" parser error for `fn`Esteban Küber-12/+5
2020-10-24Auto merge of #77255 - Aaron1011:feature/collect-attr-tokens, r=petrochenkovbors-68/+107
Unconditionally capture tokens for attributes. This allows us to avoid synthesizing tokens in `prepend_attr`, since we have the original tokens available. We still need to synthesize tokens when expanding `cfg_attr`, but this is an unavoidable consequence of the syntax of `cfg_attr` - the user does not supply the `#` and `[]` tokens that a `cfg_attr` expands to. This is based on PR https://github.com/rust-lang/rust/pull/77250 - this PR exposes a bug in the current `collect_tokens` implementation, which is fixed by the rewrite.
2020-10-23Silence unnecessary `await foo?` knock-down errorEsteban Küber-1/+7
2020-10-22Only call `collect_tokens` when we have an attribute to parseAaron Hill-26/+32
2020-10-22Make inline const work for half open rangesSantiago Pastorino-8/+8
2020-10-22Make inline const work in range patternsSantiago Pastorino-6/+15
2020-10-22Rename parse_const_expr to parse_const_blockSantiago Pastorino-3/+3
2020-10-22Don't create an empty `LazyTokenStream`Aaron Hill-19/+25
2020-10-21Unconditionally capture tokens for attributes.Aaron Hill-52/+79
This allows us to avoid synthesizing tokens in `prepend_attr`, since we have the original tokens available. We still need to synthesize tokens when expanding `cfg_attr`, but this is an unavoidable consequence of the syntax of `cfg_attr` - the user does not supply the `#` and `[]` tokens that a `cfg_attr` expands to.
2020-10-21Auto merge of #77250 - Aaron1011:feature/flat-token-collection, r=petrochenkovbors-146/+158
Rewrite `collect_tokens` implementations to use a flattened buffer Instead of trying to collect tokens at each depth, we 'flatten' the stream as we go allong, pushing open/close delimiters to our buffer just like regular tokens. One capturing is complete, we reconstruct a nested `TokenTree::Delimited` structure, producing a normal `TokenStream`. The reconstructed `TokenStream` is not created immediately - instead, it is produced on-demand by a closure (wrapped in a new `LazyTokenStream` type). This closure stores a clone of the original `TokenCursor`, plus a record of the number of calls to `next()/next_desugared()`. This is sufficient to reconstruct the tokenstream seen by the callback without storing any additional state. If the tokenstream is never used (e.g. when a captured `macro_rules!` argument is never passed to a proc macro), we never actually create a `TokenStream`. This implementation has a number of advantages over the previous one: * It is significantly simpler, with no edge cases around capturing the start/end of a delimited group. * It can be easily extended to allow replacing tokens an an arbitrary 'depth' by just using `Vec::splice` at the proper position. This is important for PR #76130, which requires us to track information about attributes along with tokens. * The lazy approach to `TokenStream` construction allows us to easily parse an AST struct, and then decide after the fact whether we need a `TokenStream`. This will be useful when we start collecting tokens for `Attribute` - we can discard the `LazyTokenStream` if the parsed attribute doesn't need tokens (e.g. is a builtin attribute). The performance impact seems to be neglibile (see https://github.com/rust-lang/rust/pull/77250#issuecomment-703960604). There is a small slowdown on a few benchmarks, but it only rises above 1% for incremental builds, where it represents a larger fraction of the much smaller instruction count. There a ~1% speedup on a few other incremental benchmarks - my guess is that the speedups and slowdowns will usually cancel out in practice.
2020-10-21Rollup merge of #78118 - spastorino:inline-const-followups, r=petrochenkovYuki Okushi-1/+5
Inline const followups r? @petrochenkov Follow ups of #77124
2020-10-19Allow NtBlock to parse on check inline const next tokenSantiago Pastorino-1/+5
2020-10-19Rewrite `collect_tokens` implementations to use a flattened bufferAaron Hill-146/+158
Instead of trying to collect tokens at each depth, we 'flatten' the stream as we go allong, pushing open/close delimiters to our buffer just like regular tokens. One capturing is complete, we reconstruct a nested `TokenTree::Delimited` structure, producing a normal `TokenStream`. The reconstructed `TokenStream` is not created immediately - instead, it is produced on-demand by a closure (wrapped in a new `LazyTokenStream` type). This closure stores a clone of the original `TokenCursor`, plus a record of the number of calls to `next()/next_desugared()`. This is sufficient to reconstruct the tokenstream seen by the callback without storing any additional state. If the tokenstream is never used (e.g. when a captured `macro_rules!` argument is never passed to a proc macro), we never actually create a `TokenStream`. This implementation has a number of advantages over the previous one: * It is significantly simpler, with no edge cases around capturing the start/end of a delimited group. * It can be easily extended to allow replacing tokens an an arbitrary 'depth' by just using `Vec::splice` at the proper position. This is important for PR #76130, which requires us to track information about attributes along with tokens. * The lazy approach to `TokenStream` construction allows us to easily parse an AST struct, and then decide after the fact whether we need a `TokenStream`. This will be useful when we start collecting tokens for `Attribute` - we can discard the `LazyTokenStream` if the parsed attribute doesn't need tokens (e.g. is a builtin attribute). The performance impact seems to be neglibile (see https://github.com/rust-lang/rust/pull/77250#issuecomment-703960604). There is a small slowdown on a few benchmarks, but it only rises above 1% for incremental builds, where it represents a larger fraction of the much smaller instruction count. There a ~1% speedup on a few other incremental benchmarks - my guess is that the speedups and slowdowns will usually cancel out in practice.
2020-10-19Avoid cloning the contents of a `TokenStream` in a few placesAaron Hill-5/+5
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-02-25 23:41:15 +0000