rust/compiler/rustc_parse/src/parser/expr.rs, branch 1.86.0

Rustfmt

2025-02-08T22:12:13+00:00

Rollup merge of #135882 - hkBst:master, r=estebank

2025-01-30T11:45:27+00:00

simplify `similar_tokens` from `Option>` to `&[_]` All uses immediately invoke contains, so maybe a further simplification is possible.

Use identifiers in diagnostics more often

2025-01-27T01:23:34+00:00

simplify similar_tokens from Vec<_> to &[_]

2025-01-23T10:45:42+00:00

simplify similar_tokens from Option> to Vec<_>

2025-01-23T10:45:42+00:00

Run `clippy --fix` for `unnecessary_map_or` lint

2025-01-19T19:15:00+00:00

Change comparison operators to have Fixity::None

2024-12-21T04:12:22+00:00

Speed up `Parser::expected_token_types`.

2024-12-19T05:05:41+00:00

The parser pushes a `TokenType` to `Parser::expected_token_types` on every call to the various `check`/`eat` methods, and clears it on every call to `bump`. Some of those `TokenType` values are full tokens that require cloning and dropping. This is a *lot* of work for something that is only used in error messages and it accounts for a significant fraction of parsing execution time. This commit overhauls `TokenType` so that `Parser::expected_token_types` can be implemented as a bitset. This requires changing `TokenType` to a C-style parameterless enum, and adding `TokenTypeSet` which uses a `u128` for the bits. (The new `TokenType` has 105 variants.) The new types `ExpTokenPair` and `ExpKeywordPair` are now arguments to the `check`/`eat` methods. This is for maximum speed. The elements in the pairs are always statically known; e.g. a `token::BinOp(token::Star)` is always paired with a `TokenType::Star`. So we now compute `TokenType`s in advance and pass them in to `check`/`eat` rather than the current approach of constructing them on insertion into `expected_token_types`. Values of these pair types can be produced by the new `exp!` macro, which is used at every `check`/`eat` call site. The macro is for convenience, allowing any pair to be generated from a single identifier. The ident/keyword filtering in `expected_one_of_not_found` is no longer necessary. It was there to account for some sloppiness in `TokenKind`/`TokenType` comparisons. The existing `TokenType` is moved to a new file `token_type.rs`, and all its new infrastructure is added to that file. There is more boilerplate code than I would like, but I can't see how to make it shorter.

Rename `Parser::expected_tokens` as `Parser::expected_token_types`.

2024-12-19T05:05:41+00:00

Because the `Token` type is similar to but different to the `TokenType` type, and the difference is important, so we want to avoid confusion.

Rollup merge of #134161 - nnethercote:overhaul-token-cursors, r=spastorino

2024-12-18T14:56:53+00:00

Overhaul token cursors Some nice cleanups here. r? `````@davidtwco`````