summary refs log tree commit diff
path: root/compiler/rustc_parse/src/parser/item.rs
AgeCommit message (Collapse)AuthorLines
2021-04-23Revert "Rollup merge of #82296 - spastorino:pubrules, r=nikomatsakis"Santiago Pastorino-1/+9
This reverts commit e2561c58a41023a14e0e583113dcf55e1ecb236a, reversing changes made to 2982ba50fc4bb629b8fe4108a81cb2f9b053510b.
2021-04-19fix few typosklensy-1/+1
2021-04-11Implement token-based handling of attributes during expansionAaron Hill-25/+29
This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2021-03-18Auto merge of #76447 - pickfire:async-pub, r=estebankbors-7/+37
Detect async visibility wrong order, `async pub` Partially address #76437.
2021-03-17Add pub as optional check_front_matterIvan Tham-6/+15
async-pub check created a regression for default
2021-03-17Detect pub fn attr wrong order like `async pub`Ivan Tham-3/+24
Redirects `const? async? unsafe? pub` to `pub const? async? unsafe?`. Fix #76437
2021-03-16ast/hir: Rename field-related structuresVadim Petrochenkov-11/+9
StructField -> FieldDef ("field definition") Field -> ExprField ("expression field", not "field expression") FieldPat -> PatField ("pattern field", not "field pattern") Also rename visiting and other methods working on them.
2021-03-05use pat<no_top_alt> for patterns in let bindingsmark-2/+3
2021-02-25Rollup merge of #82321 - bugadani:ast3, r=varkorDylan DPC-1/+1
AST: Remove some unnecessary boxes
2021-02-23Rollup merge of #82296 - spastorino:pubrules, r=nikomatsakisDylan DPC-9/+1
Support `pub` on `macro_rules` This rebases and updates `since` version of #78166 from ``@petrochenkov`` r? ``@nikomatsakis``
2021-02-21parser: remove unneccessary wrapping of return value in parse_extern()Matthias Krüger-1/+1
2021-02-20Remove some P-sDániel Buga-1/+1
2021-02-19Support `pub` on `macro_rules`Vadim Petrochenkov-9/+1
2021-02-18ast: Keep expansion status for out-of-line module itemsVadim Petrochenkov-5/+7
Also remove `ast::Mod` which is mostly redundant now
2021-02-18ast: Stop using `Mod` in `Crate`Vadim Petrochenkov-23/+8
Crate root is sufficiently different from `mod` items, at least at syntactic level. Also remove customization point for "`mod` item or crate root" from AST visitors.
2021-02-13Require passing an `AttrWrapper` to `collect_tokens_trailing_token`Aaron Hill-122/+154
This is a pure refactoring split out from #80689. It represents the most invasive part of that PR, requiring changes in every caller of `parse_outer_attributes` In order to eagerly expand `#[cfg]` attributes while preserving the original `TokenStream`, we need to know the range of tokens that corresponds to every attribute target. This is accomplished by making `parse_outer_attributes` return an opaque `AttrWrapper` struct. An `AttrWrapper` must be converted to a plain `AttrVec` by passing it to `collect_tokens_trailing_token`. This makes it difficult to accidentally construct an AST node with attributes without calling `collect_tokens_trailing_token`, since AST nodes store an `AttrVec`, not an `AttrWrapper`. As a result, we now call `collect_tokens_trailing_token` for attribute targets which only support inert attributes, such as generic arguments and struct fields. Currently, the constructed `LazyTokenStream` is simply discarded. Future PRs will record the token range corresponding to the attribute target, allowing those tokens to be removed from an enclosing `collect_tokens_trailing_token` call if necessary.
2021-02-08parser: Fix panic in 'const impl' recoveryÖmer Sinan Ağacan-2/+12
The panic happens when in recovery parsing a full `impl` (`parse_item_impl`) fails and we drop the `DiagnosticBuilder` for the recovery suggestion and return the `parse_item_impl` error. We now raise the original error "expected identifier found `impl`" when parsing the `impl` fails. Note that the regression test is slightly simplified version of the original repro in #81806, to make the error output smaller and more resilient to unrelated changes in parser error messages. Fixes #81806
2021-02-01Box the biggest ast::ItemKind variantsDániel Buga-9/+11
2021-01-22Refactor token collection to capture trailing token immediatelyAaron Hill-2/+2
2021-01-20Force token collection to run when parsing nonterminalsAaron Hill-41/+46
Fixes #81007 Previously, we would fail to collect tokens in the proper place when only builtin attributes were present. As a result, we would end up with attribute tokens in the collected `TokenStream`, leading to duplication when we attempted to prepend the attributes from the AST node. We now explicitly track when token collection must be performed due to nomterminal parsing.
2021-01-13Set tokens on AST node in `collect_tokens`Aaron Hill-13/+1
A new `HasTokens` trait is introduced, which is used to move logic from the callers of `collect_tokens` into the body of `collect_tokens`. In addition to reducing duplication, this paves the way for PR #80689, which needs to perform additional logic during token collection.
2021-01-08Add a note for `*` and `{}` usage on `use`Yuki Okushi-1/+16
2020-12-31Consistently call editions "Rust 20xx" in messages.Mara Bos-1/+1
2020-12-31Add edition 2021.Mara Bos-3/+3
2020-12-30Rename kw::Invalid -> kw::EmptyJoshua Nelson-3/+3
See https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Is.20there.20a.20symbol.20for.20the.20empty.20string.3F/near/220054471 for context.
2020-12-19Auto merge of #77035 - mibac138:fn-fat-arrow-return, r=davidtwcobors-3/+4
Gracefully handle mistyping -> as => in function return type Fixes #77019
2020-12-12Recover on `const impl<> X for Y`Noah-3/+38
2020-12-03Gracefully handle confusing -> with : in function return typemibac138-3/+4
2020-12-01Gracefully handle mistyping -> as => in function return typemibac138-2/+2
2020-11-02Use reparsed `TokenStream` if we captured any inner attributesAaron Hill-24/+9
Fixes #78675 We now bail out of `prepend_attrs` if we ended up capturing any inner attributes (which can happen in several places, due to token capturing for `macro_rules!` arguments.
2020-10-30Fix even more clippy warningsJoshua Nelson-18/+13
2020-10-29Revert invalid `fn` return type parsing changeEsteban Küber-13/+4
Fix #78507.
2020-10-28Rollup merge of #78379 - estebank:fn-signature-parse, r=varkorDylan DPC-9/+32
Tweak invalid `fn` header and body parsing * Rely on regular "expected"/"found" parser error for `fn`, fix #77115 * Recover empty `fn` bodies when encountering `}` * Recover trailing `>` in return types * Recover from non-type in array type `[<BAD TOKEN>; LEN]`
2020-10-25Tweak invalid `fn` header and body parsingEsteban Küber-8/+30
* Recover empty `fn` bodies when encountering `}` * Recover trailing `>` in return types * Recover from non-type in array type `[<BAD TOKEN>; LEN]`
2020-10-25Rely on regular "expected"/"found" parser error for `fn`Esteban Küber-4/+5
2020-10-22Don't create an empty `LazyTokenStream`Aaron Hill-1/+1
2020-10-19Rewrite `collect_tokens` implementations to use a flattened bufferAaron Hill-2/+3
Instead of trying to collect tokens at each depth, we 'flatten' the stream as we go allong, pushing open/close delimiters to our buffer just like regular tokens. One capturing is complete, we reconstruct a nested `TokenTree::Delimited` structure, producing a normal `TokenStream`. The reconstructed `TokenStream` is not created immediately - instead, it is produced on-demand by a closure (wrapped in a new `LazyTokenStream` type). This closure stores a clone of the original `TokenCursor`, plus a record of the number of calls to `next()/next_desugared()`. This is sufficient to reconstruct the tokenstream seen by the callback without storing any additional state. If the tokenstream is never used (e.g. when a captured `macro_rules!` argument is never passed to a proc macro), we never actually create a `TokenStream`. This implementation has a number of advantages over the previous one: * It is significantly simpler, with no edge cases around capturing the start/end of a delimited group. * It can be easily extended to allow replacing tokens an an arbitrary 'depth' by just using `Vec::splice` at the proper position. This is important for PR #76130, which requires us to track information about attributes along with tokens. * The lazy approach to `TokenStream` construction allows us to easily parse an AST struct, and then decide after the fact whether we need a `TokenStream`. This will be useful when we start collecting tokens for `Attribute` - we can discard the `LazyTokenStream` if the parsed attribute doesn't need tokens (e.g. is a builtin attribute). The performance impact seems to be neglibile (see https://github.com/rust-lang/rust/pull/77250#issuecomment-703960604). There is a small slowdown on a few benchmarks, but it only rises above 1% for incremental builds, where it represents a larger fraction of the much smaller instruction count. There a ~1% speedup on a few other incremental benchmarks - my guess is that the speedups and slowdowns will usually cancel out in practice.
2020-10-15fix off-by-one in parameter spansAndy Russell-1/+1
2020-09-10Attach `TokenStream` to `ast::Visibility`Aaron Hill-3/+3
A `Visibility` does not have outer attributes, so we only capture tokens when parsing a `macro_rules!` matcher
2020-09-10Attach `TokenStream` to `ast::Path`Aaron Hill-1/+1
2020-09-10Attach `TokenStream` to `ast::Ty`Aaron Hill-2/+7
A `Ty` does not have outer attributes, so we only capture tokens when parsing a `macro_rules!` matcher
2020-09-10Syntactically permit unsafety on modsDavid Tolnay-16/+51
2020-08-30mv compiler to compiler/mark-0/+1843
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-02-25 18:02:01 +0000