summary refs log tree commit diff
path: root/compiler/rustc_ast/src/token.rs
AgeCommit message (Collapse)AuthorLines
2024-07-16Remove references to `maybe_whole_expr`.Nicholas Nethercote-2/+1
It was removed in #126571.
2024-06-23Rework pattern and expression nonterminal kinds.Nicholas Nethercote-29/+44
Merge `PatParam`/`PatWithOr`, and `Expr`/`Expr2021`, for a few reasons. - It's conceptually nice, because the two pattern kinds and the two expression kinds are very similar. - With expressions in particular, there are several places where both expression kinds get the same treatment. - It removes one unreachable match arm. - Most importantly, for #124141 I will need to introduce a new type `MetaVarKind` that is very similar to `NonterminalKind`, but records a couple of extra fields for expression metavars. It's nicer to have a single `MetaVarKind::Expr` expression variant to hold those extra fields instead of duplicating them across two variants `MetaVarKind::{Expr,Expr2021}`. And then it makes sense for patterns to be treated the same way, and for `NonterminalKind` to also be treated the same way. I also clarified the comments, because I have long found them a little hard to understand.
2024-06-21Rollup merge of #126700 - compiler-errors:fragment, r=fmeaseMatthias Krüger-5/+14
Make edition dependent `:expr` macro fragment act like the edition-dependent `:pat` fragment does Parse the `:expr` fragment as `:expr_2021` in editions <=2021, and as `:expr` in edition 2024. This is similar to how we parse `:pat` as `:pat_param` in edition <=2018 and `:pat_with_or` in >=2021, and means we can get rid of a span dependency from `nonterminal_may_begin_with`. Specifically, this fixes a theoretical regression since the `expr_2021` macro fragment previously would allow `const {}` if the *caller* is edition 2024. This is inconsistent with the way that the `pat` macro fragment was upgraded, and also leads to surprising behavior when a macro *caller* crate upgrades to edtion 2024, since they may have parsing changes that they never asked for (with no way of opting out of it). This PR also allows using `expr_2021` in all editions. Why was this was disallowed in the first place? It's purely additive, and also it's still feature gated? r? ```@fmease``` ```@eholk``` cc ```@vincenzopalazzo``` cc #123865 Tracking: - https://github.com/rust-lang/rust/issues/123742
2024-06-20Introduce `can_begin_string_literal`.Nicholas Nethercote-0/+15
We currently use `can_begin_literal_maybe_minus` in a couple of places where only string literals are allowed. This commit introduces a more specific function, which makes things clearer. It doesn't change behaviour because the two functions affected (`is_unsafe_foreign_mod` and `check_keyword_case`) are always followed by a call to `parse_abi`, which checks again for a string literal.
2024-06-20Inline `can_begin_literal_maybe_minus` call into two places.Nicholas Nethercote-2/+3
It's clearer this way, because the `Interpolated` cases in `can_begin_const_arg` and `is_pat_range_end_start` are more permissive than the `Interpolated` cases in `can_begin_literal_maybe_minus`.
2024-06-19Allow naming expr_2021 in all editionsMichael Goulet-5/+14
2024-06-04Handle safety keyword for extern block inner itemsSantiago Pastorino-0/+2
2024-05-17Auto merge of #123865 - eholk:expr_2021, r=fmeasebors-0/+4
Update `expr` matcher for Edition 2024 and add `expr_2021` nonterminal This commit adds a new nonterminal `expr_2021` in macro patterns, and `expr_fragment_specifier_2024` feature flag. This change also updates `expr` so that on Edition 2024 it will also match `const { ... }` blocks, while `expr_2021` preserves the current behavior of `expr`, matching expressions without `const` blocks. Joint work with `@vincenzopalazzo.` Issue #123742
2024-05-14Remove `NtIdent` and `NtLifetime`.Nicholas Nethercote-45/+32
The extra span is now recorded in the new `TokenKind::NtIdent` and `TokenKind::NtLifetime`. These both consist of a single token, and so there's no operator precedence problems with inserting them directly into the token stream. The other way to do this would be to wrap the ident/lifetime in invisible delimiters, but there's a lot of code that assumes an interpolated ident/lifetime fits in a single token, and changing all that code to work with invisible delimiters would have been a pain. (Maybe it could be done in a follow-up.) This change might not seem like much of a win, but it's a first step toward the much bigger and long-desired removal of `Nonterminal` and `TokenKind::Interpolated`. That change is big and complex enough that it's worth doing this piece separately. (Indeed, this commit is based on part of a late commit in #114647, a prior attempt at that big and complex change.)
2024-05-13Apply code review suggestionsEric Holk-1/+1
- use feature_err to report unstable expr_2021 - Update downlevel expr_2021 diagnostics Co-authored-by: León Orell Valerian Liehr <me@fmease.dev>
2024-05-13expr_2021 should be allowed on edition 2021 and laterEric Holk-1/+1
2024-05-13Add expr_2021 nonterminal and feature flagEric Holk-0/+4
This commit adds a new nonterminal `expr_2021` in macro patterns, and `expr_fragment_specifier_2024` feature flag. For now, `expr` and `expr_2021` are treated the same, but in future PRs we will update `expr` to match to new grammar. Co-authored-by: Vincezo Palazzo <vincenzopalazzodev@gmail.com>
2024-05-13Remove a `Span` from `TokenKind::Interpolated`.Nicholas Nethercote-14/+19
This span records the declaration of the metavariable in the LHS of the macro. It's used in a couple of error messages. Unfortunately, it gets in the way of the long-term goal of removing `TokenKind::Interpolated`. So this commit removes it, which degrades a couple of (obscure) error messages but makes things simpler and enables the next commit.
2024-05-06Return coherent description for boolean instead of panickingUrgau-1/+1
2024-04-29Remove `extern crate rustc_macros` from numerous crates.Nicholas Nethercote-1/+1
2024-04-18Auto merge of #124008 - nnethercote:simpler-static_assert_size, r=Nilstriebbors-1/+1
Simplify `static_assert_size`s. We want to run them on all 64-bit platforms. r? `@ghost`
2024-04-18Simplify `static_assert_size`s.Nicholas Nethercote-1/+1
We want to run them on all 64-bit platforms.
2024-04-16Fix empty-set symbol in commentsDavid Tolnay-1/+1
2024-04-16Rollup merge of #123462 - fmease:rn-mod-sep-to-path-sep, r=nnethercoteLeón Orell Valerian Liehr-9/+11
Cleanup: Rename `ModSep` to `PathSep` `::` is usually referred to as the *path separator* (citation needed). The existing name `ModSep` for *module separator* is a bit misleading since it in fact separates the segments of arbitrary path segments, not only ones resolving to modules. Let me just give a shout-out to associated items (`T::Assoc`, `<Ty as Trait>::function`) and enum variants (`Option::None`). Motivation: Reduce friction for new contributors, prevent potential confusion. cc `@petrochenkov` r? nnethercote or compiler
2024-04-04Rename ModSep to PathSepLeón Orell Valerian Liehr-9/+11
2024-04-03Check `x86_64` size assertions on `aarch64`, tooZalathar-1/+1
This makes it easier for contributors on aarch64 workstations (e.g. Macs) to notice when these assertions have been violated.
2024-03-21Fix out-of-date comment.Nicholas Nethercote-1/+1
2024-03-21Rename `Token::is_path`.Nicholas Nethercote-2/+2
This makes it consistent with `is_whole_expr` and `is_whole_block`.
2024-03-20resolve clippy errorsonur-ozkan-0/+1
Signed-off-by: onur-ozkan <work@onurozkan.dev>
2024-02-20Add newtype for raw identsclubby789-10/+28
2024-02-15Add `ErrorGuaranteed` to `ast::LitKind::Err`, `token::LitKind::Err`.Nicholas Nethercote-6/+6
This mostly works well, and eliminates a couple of delayed bugs. One annoying thing is that we should really also add an `ErrorGuaranteed` to `proc_macro::bridge::LitKind::Err`. But that's difficult because `proc_macro` doesn't have access to `ErrorGuaranteed`, so we have to fake it.
2023-12-27Introduce `const Trait` (always-const trait bounds)León Orell Valerian Liehr-9/+0
2023-12-10remove redundant importssurechen-1/+1
detects redundant imports that can be eliminated. for #117772 : In order to facilitate review and modification, split the checking code and removing redundant imports code into two PR.
2023-11-27Detect Python-like slicing and suggest how to fixHirochika Matsumoto-0/+5
Fix #108215
2023-11-17Auto merge of #114292 - estebank:issue-71039, r=b-naberbors-18/+35
More detail when expecting expression but encountering bad macro argument On nested macro invocations where the same macro fragment changes fragment type from one to the next, point at the chain of invocations and at the macro fragment definition place, explaining that the change has occurred. Fix #71039. ``` error: expected expression, found pattern `1 + 1` --> $DIR/trace_faulty_macros.rs:49:37 | LL | (let $p:pat = $e:expr) => {test!(($p,$e))}; | ------- -- this is interpreted as expression, but it is expected to be pattern | | | this macro fragment matcher is expression ... LL | (($p:pat, $e:pat)) => {let $p = $e;}; | ------ ^^ expected expression | | | this macro fragment matcher is pattern ... LL | test!(let x = 1+1); | ------------------ | | | | | this is expected to be expression | in this macro invocation | = note: when forwarding a matched fragment to another macro-by-example, matchers in the second macro will see an opaque AST of the fragment type, not the underlying tokens = note: this error originates in the macro `test` (in Nightly builds, run with -Z macro-backtrace for more info) ```
2023-11-16More detail when expecting expression but encountering bad macro argumentEsteban Küber-18/+35
Partially address #71039.
2023-11-14Detect more `=>` typosEsteban Küber-1/+2
Handle and recover `match expr { pat >= { arm } }`.
2023-11-13Fix commentSleep_AllDay-2/+2
Gt => Greater than => `>` Ge => Greater equal => `>=`
2023-10-26Reserve `gen` keyword for `gen {}` blocks and `gen fn` in 2024 editionOli Scherer-0/+1
2023-10-15Auto merge of #116688 - compiler-errors:rustfmt-up, r=WaffleLapkin,Nilstriebbors-6/+8
Format all the let-chains in compiler crates Since rust-lang/rustfmt#5910 has landed, soon we will have support for formatting let-chains (as soon as rustfmt syncs and beta gets bumped). This PR applies the changes [from master rustfmt to rust-lang/rust eagerly](https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/out.20formatting.20of.20prs/near/374997516), so that the next beta bump does not have to deal with a 200+ file diff and can remain concerned with other things like `cfg(bootstrap)` -- #113637 was a pain to land, for example, because of let-else. I will also add this commit to the ignore list after it has landed. The commands that were run -- I'm not great at bash-foo, but this applies rustfmt to every compiler crate, and then reverts the two crates that should probably be formatted out-of-tree. ``` ~/rustfmt $ ls -1d ~/rust/compiler/* | xargs -I@ cargo run --bin rustfmt -- `@/src/lib.rs` --config-path ~/rust --edition=2021 # format all of the compiler crates ~/rust $ git checkout HEAD -- compiler/rustc_codegen_{gcc,cranelift} # revert changes to cg-gcc and cg-clif ``` cc `@rust-lang/rustfmt` r? `@WaffleLapkin` or `@Nilstrieb` who said they may be able to review this purely mechanical PR :> cc `@Mark-Simulacrum` and `@petrochenkov,` who had some thoughts on the order of operations with big formatting changes in https://github.com/rust-lang/rust/pull/95262#issue-1178993801. I think the situation has changed since then, given that let-chains support exists on master rustfmt now, and I'm fairly confident that this formatting PR should land even if *bootstrap* rustfmt doesn't yet format let-chains in order to lessen the burden of the next beta bump.
2023-10-14Rollup merge of #116696 - c410-f3r:in-doc, r=petrochenkovMatthias Krüger-0/+27
Misc improvements cc https://github.com/rust-lang/rust/pull/116323#discussion_r1355282195 r? `@petrochenkov`
2023-10-13Misc improvementsCaio-0/+27
2023-10-13Format all the let chains in compilerMichael Goulet-6/+8
2023-10-12Rename `Token::is_op` as `Token::is_punct`.Nicholas Nethercote-1/+1
For consistency with `proc_macro::Punct`.
2023-10-04Fix misuses of a vs ancui fliter-1/+1
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-08-24Parse unnamed fields and anonymous structs or unionsFrank King-0/+2
Anonymous structs or unions are only allowed in struct field definitions. Co-authored-by: carbotaniuman <41451839+carbotaniuman@users.noreply.github.com>
2023-08-02Fix an erroneous comment.Nicholas Nethercote-2/+0
The `None` variant has already been renamed `Invisible`.
2023-07-27Optimize `TokenKind::clone`.Nicholas Nethercote-1/+16
`TokenKind` would impl `Copy` if it weren't for `TokenKind::Interpolated`. This commit makes `clone` reflect that.
2023-07-12Flip cfg's for bootstrap bumpMark Rousskov-1/+1
2023-05-27Add warn-by-default lint for local binding shadowing exported glob re-export ↵许杰友 Jieyou Xu (Joe)-0/+1
item
2023-05-24Use `Option::is_some_and` and `Result::is_ok_and` in the compilerMaybe Waffle-1/+1
2023-05-02initial step towards implementing C string literalsDeadbeef-0/+7
2023-01-05Fix `uninlined_format_args` for some compiler cratesnils-7/+7
Convert all the crates that have had their diagnostic migration completed (except save_analysis because that will be deleted soon and apfloat because of the licensing problem).
2022-12-16Little fixesDeep Majumder-1/+1
2022-12-14Allow .. to be parsed as let initializerDeep Majumder-0/+4
.. and ..= are valid expressions, however when used in a let statement it is not parsed.