| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Inside #123865, we are adding support for the new semantics
for expr2024, but we have noted a performance issue.
We realized there is a redundant check for each
token regarding an edition. This commit moves the edition
check to the end, avoiding some extra checks that
can slow down compilation time.
Link: https://github.com/rust-lang/rust/pull/123865
Co-Developed-by: @eholk
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
|
|
Update `expr` matcher for Edition 2024 and add `expr_2021` nonterminal
This commit adds a new nonterminal `expr_2021` in macro patterns, and `expr_fragment_specifier_2024` feature flag.
This change also updates `expr` so that on Edition 2024 it will also match `const { ... }` blocks, while `expr_2021` preserves the current behavior of `expr`, matching expressions without `const` blocks.
Joint work with `@vincenzopalazzo.`
Issue #123742
|
|
Co-authored-by: León Orell Valerian Liehr <me@fmease.dev>
|
|
The extra span is now recorded in the new `TokenKind::NtIdent` and
`TokenKind::NtLifetime`. These both consist of a single token, and so
there's no operator precedence problems with inserting them directly
into the token stream.
The other way to do this would be to wrap the ident/lifetime in invisible
delimiters, but there's a lot of code that assumes an interpolated
ident/lifetime fits in a single token, and changing all that code to work with
invisible delimiters would have been a pain. (Maybe it could be done in a
follow-up.)
This change might not seem like much of a win, but it's a first step toward the
much bigger and long-desired removal of `Nonterminal` and
`TokenKind::Interpolated`. That change is big and complex enough that it's
worth doing this piece separately. (Indeed, this commit is based on part of a
late commit in #114647, a prior attempt at that big and complex change.)
|
|
Co-authored-by: Eric Holk <eric@theincredibleholk.org>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
|
|
This commit adds a new nonterminal `expr_2021` in macro patterns, and
`expr_fragment_specifier_2024` feature flag. For now, `expr` and
`expr_2021` are treated the same, but in future PRs we will update
`expr` to match to new grammar.
Co-authored-by: Vincezo Palazzo <vincenzopalazzodev@gmail.com>
|
|
This span records the declaration of the metavariable in the LHS of the macro.
It's used in a couple of error messages. Unfortunately, it gets in the way of
the long-term goal of removing `TokenKind::Interpolated`. So this commit
removes it, which degrades a couple of (obscure) error messages but makes
things simpler and enables the next commit.
|
|
|
|
This commit combines `MatchedTokenTree` and `MatchedNonterminal`, which
are often considered together, into a single `MatchedSingle`. It shares
a representation with the newly-parameterized `ParseNtResult`.
This will also make things much simpler if/when variants from
`Interpolated` start being moved to `ParseNtResult`.
|
|
|
|
|
|
This commit replaces this pattern:
```
err.into_diagnostic(dcx)
```
with this pattern:
```
dcx.create_err(err)
```
in a lot of places.
It's a little shorter, makes the error level explicit, avoids some
`IntoDiagnostic` imports, and is a necessary prerequisite for the next
commit which will add a `level` arg to `into_diagnostic`.
This requires adding `track_caller` on `create_err` to avoid mucking up
the output of `tests/ui/track-diagnostics/track4.rs`. It probably should
have been there already.
|
|
|
|
|
|
Partially address #71039.
|
|
|
|
It's more descriptive, and future-proofs it if/when additional variants
get added.
|
|
`may_be_ident` is true for `NtPath` and `NtMeta`, so we don't need to
check for them separately.
|
|
For ones matching more than one or two variants, this is easier to think
about.
|
|
It makes the code more readable.
|
|
|
|
Adding a `ty_` makes its purpose much clearer, and consistent with other
`parse_ty_*` functions.
|
|
All nonterminals collect and store tokens now. (Unless they are very
simple, e.g. single-token, and can precisely recover them without
collecting.)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
And use them to avoid constructing some artificial `Nonterminal` tokens during expansion
|
|
|
|
|
|
|
|
By heap allocating the argument within `NtPath`, `NtVis`, and `NtStmt`.
This slightly reduces cumulative and peak allocation amounts, most
notably on `deep-vector`.
|
|
It's only needed for macro expansion, not as a general element in the
AST. This commit removes it, adds `NtOrTt` for the parser and macro
expansion cases, and renames the variants in `NamedMatch` to better
match the new type.
|
|
Follow up to #92746. Address #94510.
|
|
* Recover from invalid `'label: ` before block.
* Make suggestion to enclose statements in a block multipart.
* Point at `match`, `while`, `loop` and `unsafe` keywords when failing
to parse their expression.
* Do not suggest `{ ; }`.
* Do not suggest `|` when very unlikely to be what was wanted (in `let`
statements).
|
|
|
|
Co-authored-by: Esteban Kuber <estebank@users.noreply.github.com>
|
|
|
|
|
|
|
|
This PR modifies the macro expansion infrastructure to handle attributes
in a fully token-based manner. As a result:
* Derives macros no longer lose spans when their input is modified
by eager cfg-expansion. This is accomplished by performing eager
cfg-expansion on the token stream that we pass to the derive
proc-macro
* Inner attributes now preserve spans in all cases, including when we
have multiple inner attributes in a row.
This is accomplished through the following changes:
* New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced.
These are very similar to a normal `TokenTree`, but they also track
the position of attributes and attribute targets within the stream.
They are built when we collect tokens during parsing.
An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when
we invoke a macro.
* Token capturing and `LazyTokenStream` are modified to work with
`AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which
is created during the parsing of a nested AST node to make the 'outer'
AST node aware of the attributes and attribute target stored deeper in the token stream.
* When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`),
we tokenize and reparse our target, capturing additional information about the locations of
`#[cfg]` and `#[cfg_attr]` attributes at any depth within the target.
This is a performance optimization, allowing us to perform less work
in the typical case where captured tokens never have eager cfg-expansion run.
|
|
|
|
rename :pat2018 -> :pat2015
as requested by T-lang on zulip: https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/or.20patterns/near/231133873
No functional changes here... just renaming.
r? `@nikomatsakis`
|
|
|
|
|
|
Along the way, we also implement a handful of diagnostics improvements
and fixes, particularly with respect to the special handling of `||` in
place of `|` and when there are leading verts in function params, which
don't allow top-level or-patterns anyway.
|
|
|
|
This is a pure refactoring split out from #80689.
It represents the most invasive part of that PR, requiring changes in
every caller of `parse_outer_attributes`
In order to eagerly expand `#[cfg]` attributes while preserving the
original `TokenStream`, we need to know the range of tokens that
corresponds to every attribute target. This is accomplished by making
`parse_outer_attributes` return an opaque `AttrWrapper` struct. An
`AttrWrapper` must be converted to a plain `AttrVec` by passing it to
`collect_tokens_trailing_token`. This makes it difficult to accidentally
construct an AST node with attributes without calling `collect_tokens_trailing_token`,
since AST nodes store an `AttrVec`, not an `AttrWrapper`.
As a result, we now call `collect_tokens_trailing_token` for attribute
targets which only support inert attributes, such as generic arguments
and struct fields. Currently, the constructed `LazyTokenStream` is
simply discarded. Future PRs will record the token range corresponding
to the attribute target, allowing those tokens to be removed from an
enclosing `collect_tokens_trailing_token` call if necessary.
|
