about summary refs log tree commit diff
path: root/compiler/rustc_ast
AgeCommit message (Collapse)AuthorLines
2020-11-01Do not remove tokens before AST json serializationVadim Petrochenkov-2/+3
2020-10-31parser: Cleanup `LazyTokenStream` and avoid some clonesVadim Petrochenkov-43/+22
by using a named struct instead of a closure.
2020-10-27Auto merge of #77502 - varkor:const-generics-suggest-enclosing-braces, ↵bors-0/+16
r=petrochenkov Suggest that expressions that look like const generic arguments should be enclosed in brackets I pulled out the changes for const expressions from https://github.com/rust-lang/rust/pull/71592 (without the trait object diagnostic changes) and made some small changes; the implementation is `@estebank's.` We're also going to want to make some changes separately to account for trait objects (they result in poor diagnostics, as is evident from one of the test cases here), such as an adaption of https://github.com/rust-lang/rust/pull/72273. Fixes https://github.com/rust-lang/rust/issues/70753. r? `@petrochenkov`
2020-10-26Suggest expressions that look like const generic arguments should be ↵varkor-0/+16
enclosed in brackets Co-Authored-By: Esteban Kuber <github@kuber.com.ar>
2020-10-25Rollup merge of #78326 - Aaron1011:fix/min-stmt-lints, r=petrochenkovYuki Okushi-2/+4
Split out statement attributes changes from #78306 This is the same as PR https://github.com/rust-lang/rust/pull/78306, but `unused_doc_comments` is modified to explicitly ignore statement items (which preserves the current behavior). This shouldn't have any user-visible effects, so it can be landed without lang team discussion. --------- When the 'early' and 'late' visitors visit an attribute target, they activate any lint attributes (e.g. `#[allow]`) that apply to it. This can affect warnings emitted on sibiling attributes. For example, the following code does not produce an `unused_attributes` for `#[inline]`, since the sibiling `#[allow(unused_attributes)]` suppressed the warning. ```rust trait Foo { #[allow(unused_attributes)] #[inline] fn first(); #[inline] #[allow(unused_attributes)] fn second(); } ``` However, we do not do this for statements - instead, the lint attributes only become active when we visit the struct nested inside `StmtKind` (e.g. `Item`). Currently, this is difficult to observe due to another issue - the `HasAttrs` impl for `StmtKind` ignores attributes for `StmtKind::Item`. As a result, the `unused_doc_comments` lint will never see attributes on item statements. This commit makes two interrelated fixes to the handling of inert (non-proc-macro) attributes on statements: * The `HasAttr` impl for `StmtKind` now returns attributes for `StmtKind::Item`, treating it just like every other `StmtKind` variant. The only place relying on the old behavior was macro which has been updated to explicitly ignore attributes on item statements. This allows the `unused_doc_comments` lint to fire for item statements. * The `early` and `late` lint visitors now activate lint attributes when invoking the callback for `Stmt`. This ensures that a lint attribute (e.g. `#[allow(unused_doc_comments)]`) can be applied to sibiling attributes on an item statement. For now, the `unused_doc_comments` lint is explicitly disabled on item statements, which preserves the current behavior. The exact locatiosn where this lint should fire are being discussed in PR #78306
2020-10-24Fix inconsistencies in handling of inert attributes on statementsAaron Hill-2/+4
When the 'early' and 'late' visitors visit an attribute target, they activate any lint attributes (e.g. `#[allow]`) that apply to it. This can affect warnings emitted on sibiling attributes. For example, the following code does not produce an `unused_attributes` for `#[inline]`, since the sibiling `#[allow(unused_attributes)]` suppressed the warning. ```rust trait Foo { #[allow(unused_attributes)] #[inline] fn first(); #[inline] #[allow(unused_attributes)] fn second(); } ``` However, we do not do this for statements - instead, the lint attributes only become active when we visit the struct nested inside `StmtKind` (e.g. `Item`). Currently, this is difficult to observe due to another issue - the `HasAttrs` impl for `StmtKind` ignores attributes for `StmtKind::Item`. As a result, the `unused_doc_comments` lint will never see attributes on item statements. This commit makes two interrelated fixes to the handling of inert (non-proc-macro) attributes on statements: * The `HasAttr` impl for `StmtKind` now returns attributes for `StmtKind::Item`, treating it just like every other `StmtKind` variant. The only place relying on the old behavior was macro which has been updated to explicitly ignore attributes on item statements. This allows the `unused_doc_comments` lint to fire for item statements. * The `early` and `late` lint visitors now activate lint attributes when invoking the callback for `Stmt`. This ensures that a lint attribute (e.g. `#[allow(unused_doc_comments)]`) can be applied to sibiling attributes on an item statement. For now, the `unused_doc_comments` lint is explicitly disabled on item statements, which preserves the current behavior. The exact locatiosn where this lint should fire are being discussed in PR #78306
2020-10-21Unconditionally capture tokens for attributes.Aaron Hill-3/+10
This allows us to avoid synthesizing tokens in `prepend_attr`, since we have the original tokens available. We still need to synthesize tokens when expanding `cfg_attr`, but this is an unavoidable consequence of the syntax of `cfg_attr` - the user does not supply the `#` and `[]` tokens that a `cfg_attr` expands to.
2020-10-21Auto merge of #77250 - Aaron1011:feature/flat-token-collection, r=petrochenkovbors-11/+83
Rewrite `collect_tokens` implementations to use a flattened buffer Instead of trying to collect tokens at each depth, we 'flatten' the stream as we go allong, pushing open/close delimiters to our buffer just like regular tokens. One capturing is complete, we reconstruct a nested `TokenTree::Delimited` structure, producing a normal `TokenStream`. The reconstructed `TokenStream` is not created immediately - instead, it is produced on-demand by a closure (wrapped in a new `LazyTokenStream` type). This closure stores a clone of the original `TokenCursor`, plus a record of the number of calls to `next()/next_desugared()`. This is sufficient to reconstruct the tokenstream seen by the callback without storing any additional state. If the tokenstream is never used (e.g. when a captured `macro_rules!` argument is never passed to a proc macro), we never actually create a `TokenStream`. This implementation has a number of advantages over the previous one: * It is significantly simpler, with no edge cases around capturing the start/end of a delimited group. * It can be easily extended to allow replacing tokens an an arbitrary 'depth' by just using `Vec::splice` at the proper position. This is important for PR #76130, which requires us to track information about attributes along with tokens. * The lazy approach to `TokenStream` construction allows us to easily parse an AST struct, and then decide after the fact whether we need a `TokenStream`. This will be useful when we start collecting tokens for `Attribute` - we can discard the `LazyTokenStream` if the parsed attribute doesn't need tokens (e.g. is a builtin attribute). The performance impact seems to be neglibile (see https://github.com/rust-lang/rust/pull/77250#issuecomment-703960604). There is a small slowdown on a few benchmarks, but it only rises above 1% for incremental builds, where it represents a larger fraction of the much smaller instruction count. There a ~1% speedup on a few other incremental benchmarks - my guess is that the speedups and slowdowns will usually cancel out in practice.
2020-10-19Rewrite `collect_tokens` implementations to use a flattened bufferAaron Hill-11/+83
Instead of trying to collect tokens at each depth, we 'flatten' the stream as we go allong, pushing open/close delimiters to our buffer just like regular tokens. One capturing is complete, we reconstruct a nested `TokenTree::Delimited` structure, producing a normal `TokenStream`. The reconstructed `TokenStream` is not created immediately - instead, it is produced on-demand by a closure (wrapped in a new `LazyTokenStream` type). This closure stores a clone of the original `TokenCursor`, plus a record of the number of calls to `next()/next_desugared()`. This is sufficient to reconstruct the tokenstream seen by the callback without storing any additional state. If the tokenstream is never used (e.g. when a captured `macro_rules!` argument is never passed to a proc macro), we never actually create a `TokenStream`. This implementation has a number of advantages over the previous one: * It is significantly simpler, with no edge cases around capturing the start/end of a delimited group. * It can be easily extended to allow replacing tokens an an arbitrary 'depth' by just using `Vec::splice` at the proper position. This is important for PR #76130, which requires us to track information about attributes along with tokens. * The lazy approach to `TokenStream` construction allows us to easily parse an AST struct, and then decide after the fact whether we need a `TokenStream`. This will be useful when we start collecting tokens for `Attribute` - we can discard the `LazyTokenStream` if the parsed attribute doesn't need tokens (e.g. is a builtin attribute). The performance impact seems to be neglibile (see https://github.com/rust-lang/rust/pull/77250#issuecomment-703960604). There is a small slowdown on a few benchmarks, but it only rises above 1% for incremental builds, where it represents a larger fraction of the much smaller instruction count. There a ~1% speedup on a few other incremental benchmarks - my guess is that the speedups and slowdowns will usually cancel out in practice.
2020-10-19Avoid cloning the contents of a `TokenStream` in a few placesAaron Hill-5/+5
2020-10-18Auto merge of #78066 - bugadani:wat, r=jonas-schievinkbors-1/+1
Clean up small, surprising bits of code This PR clean up a small number of unrelated, small things I found while browsing the code base.
2020-10-18No need to map the max_distanceDániel Buga-1/+1
2020-10-16Add check_generic_arg early passSantiago Pastorino-5/+12
2020-10-16Parse inline const expressionsSantiago Pastorino-0/+10
2020-10-15Rollup merge of #77739 - est31:remove_unused_code, r=petrochenkov,varkorYuki Okushi-64/+0
Remove unused code Rustc has a builtin lint for detecting unused code inside a crate, but when an item is marked `pub`, the code, even if unused inside the entire workspace, is never marked as such. Therefore, I've built [warnalyzer](https://github.com/est31/warnalyzer) to detect unused items in a cross-crate setting. Closes https://github.com/est31/warnalyzer/issues/2
2020-10-14Remove unused code from rustc_astest31-64/+0
2020-10-11Add hack to keep `actix-web` and `actori-web` compilingAaron Hill-5/+16
This extends the existing `ident_name_compatibility_hack` to handle the `tuple_from_req` macro defined in `actix-web` (and its fork `actori-web`).
2020-10-09address review commentsEsteban Küber-0/+10
2020-10-07Auto merge of #76985 - hbina:clone_check, r=estebankbors-1/+13
Prevent stack overflow in deeply nested types. Related issue #75577 (?) Unfortunately, I am unable to test whether this actually solves the problem because apparently, 12GB RAM + 2GB swap is not enough to compile the (admittedly toy) source file.
2020-10-04Remove extra indirection in LitKind::ByteStrRobin Schoonover-4/+3
2020-09-23tidyErik Hofmayer-1/+4
2020-09-23/nightly/nightly-rustcErik Hofmayer-1/+1
2020-09-23Updated html_root_url for compiler cratesErik Hofmayer-1/+1
2020-09-22Add `#![feature(const_fn_transmute)]` to `rustc_ast`Dylan MacKenzie-0/+1
2020-09-21Prevent stackoverflowHanif Bin Ariffin-1/+13
2020-09-20Rollup merge of #76890 - matthiaskrgr:matches_simpl, r=lcnrRalf Jung-3/+3
use matches!() macro for simple if let conditions
2020-09-18use matches!() macro for simple if let conditionsMatthias Krüger-3/+3
2020-09-17Remove redundant #![feature(...)] 's from compiler/est31-4/+0
2020-09-14Auto merge of #76541 - matthiaskrgr:unstable_sort, r=davidtwcobors-1/+2
use sort_unstable to sort primitive types It's not important to retain original order if we have &[1, 1, 2, 3] for example. clippy::stable_sort_primitive
2020-09-10Fully integrate token collection for additional AST structsAaron Hill-1/+1
This commit contains miscellaneous changes that don't fit into any of the other commits in this PR
2020-09-10Attach tokens to `ast::Stmt`Aaron Hill-2/+6
We currently only attach tokens when parsing a `:stmt` matcher for a `macro_rules!` macro. Proc-macro attributes on statements are still unstable, and need additional work.
2020-09-10Attach `TokenStream` to `ast::Visibility`Aaron Hill-7/+14
A `Visibility` does not have outer attributes, so we only capture tokens when parsing a `macro_rules!` matcher
2020-09-10Attach `TokenStream` to `ast::Path`Aaron Hill-4/+5
2020-09-10Attach tokens to `NtMeta` (`ast::AttrItem`)Aaron Hill-3/+4
An `AttrItem` does not have outer attributes, so we only capture tokens when parsing a `macro_rules!` matcher
2020-09-10Attach `TokenStream` to `ast::Ty`Aaron Hill-4/+6
A `Ty` does not have outer attributes, so we only capture tokens when parsing a `macro_rules!` matcher
2020-09-10Attach `TokenStream` to `ast::Block`Aaron Hill-1/+2
A `Block` does not have outer attributes, so we only capture tokens when parsing a `macro_rules!` matcher
2020-09-10Syntactically permit unsafety on modsDavid Tolnay-5/+12
2020-09-10Auto merge of #76291 - matklad:spacing, r=petrochenkovbors-30/+27
Rename IsJoint -> Spacing Builds on #76286 and might conflict with #76285 r? `@petrochenkov`
2020-09-10Revert "Rollup merge of #76285 - matklad:censor-spacing, r=petrochenkov"Tyler Mandry-2/+2
This reverts commit 85cee57fd791d670d92dc61e0ad71594128dd45a, reversing changes made to b4d387302416c90a3f70211770292d8d8ab5e07d.
2020-09-10use sort_unstable to sort primitive typesMatthias Krüger-1/+2
It's not important to retain original order if we have &[1, 1, 2, 3] for example. clippy::stable_sort_primitive
2020-09-07Rollup merge of #76274 - scottmcm:fix-76271, r=petrochenkovDylan DPC-0/+1
Allow try blocks as the argument to return expressions Fixes #76271 I don't think this needs to be edition-aware (phew) since `return try` in 2015 is also the start of an expression, just with a struct literal instead of a block (`return try { x: 4, y: 5 }`).
2020-09-06Auto merge of #76331 - Aaron1011:fix/group-compat-hack-test, r=petrochenkovbors-3/+13
Account for version number in NtIdent hack Issue #74616 tracks a backwards-compatibility hack for certain macros. This has is implemented by hard-coding the filenames and macro names of certain code that we want to continue to compile. However, the initial implementation of the hack was based on the directory structure when building the crate from its repository (e.g. `js-sys/src/lib.rs`). When the crate is build as a dependency, it will include a version number from the clone from the cargo registry (e.g. `js-sys-0.3.17/src/lib.rs`), which would fail the check. This commit modifies the backwards-compatibility hack to check that desired crate name (`js-sys` or `time-macros-impl`) is a prefix of the proper part of the path. See https://github.com/rust-lang/rust/issues/76070#issuecomment-687215646 for more details.
2020-09-04Account for version number in NtIdent hackAaron Hill-3/+13
Issue #74616 tracks a backwards-compatibility hack for certain macros. This has is implemented by hard-coding the filenames and macro names of certain code that we want to continue to compile. However, the initial implementation of the hack was based on the directory structure when building the crate from its repository (e.g. `js-sys/src/lib.rs`). When the crate is build as a dependency, it will include a version number from the clone from the cargo registry (e.g. `js-sys-0.3.17/src/lib.rs`), which would fail the check. This commit modifies the backwards-compatibility hack to check that desired crate name (`js-sys` or `time-macros-impl`) is a prefix of the proper part of the path. See https://github.com/rust-lang/rust/issues/76070#issuecomment-687215646 for more details.
2020-09-03Optimize Cursor::look_aheadAleksey Kladov-2/+2
Cloning a tt is cheap, but not free (there's Arc inside).
2020-09-03Rename IsJoint -> SpacingAleksey Kladov-30/+27
To match better naming from proc-macro
2020-09-02Allow try blocks as the argument to return expressionsScott McMurray-0/+1
Fixes 76271
2020-09-02Auto merge of #76170 - matklad:notrivia, r=petrochenkovbors-15/+3
Remove trivia tokens r? @ghost
2020-09-01Remove trivia tokensAleksey Kladov-15/+3
2020-08-30Factor out StmtKind::MacCall fields into `MacCallStmt` structAaron Hill-13/+19
In PR #76130, I add a fourth field, which makes using a tuple variant somewhat unwieldy.
2020-08-30mv compiler to compiler/mark-0/+8724