| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
`TokenStream` is currently recursive in *two* ways:
- the `TokenTree` variant contains a `ThinTokenStream`, which can
contain a `TokenStream`;
- the `TokenStream` variant contains a `Vec<TokenStream>`.
The latter is not necessary and causes significant complexity. This
commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`.
This reduces complexity significantly. In particular, `StreamCursor` is
eliminated, and `Cursor` becomes much simpler, consisting now of just a
`TokenStream` and an index.
The commit also removes the `Extend` impl for `TokenStream`, because it
is only used in tests. (The commit also removes those tests.)
Overall, the commit reduces the number of lines of code by almost 200.
|
|
syntax: Fix regression in diagnostics for patterns in trait method parameters
Fixes https://github.com/rust-lang/rust/issues/55036
|
|
Suggest using raw identifiers in 2018 edition when using keywords
|
|
|
|
|
|
|
|
- Suggest raw ident escaping in all editions
- Keep primary label in all cases
|
|
|
|
|
|
|
|
|
|
|
|
Remove `TokenStream::JointTree`.
This is done by adding a new `IsJoint` field to `TokenStream::Tree`,
which simplifies a lot of `match` statements. And likewise for
`CursorKind`.
The commit also adds a new method `TokenTree:stream()` which can replace
a choice between `.into()` and `.joint()`.
|
|
Rework treatment of `$crate` in procedural macros
Important clarification: `$crate` below means "processed `$crate`" or "output `$crate`". In the input of a decl macro `$crate` is just two separate tokens, but in the *output of a decl macro* `$crate` is a single keyword identifier (https://github.com/rust-lang/rust/issues/55640#issuecomment-435692791).
First of all, this PR removes the `eliminate_crate_var` hack.
`$crate::foo` is no longer replaced with `::foo` or `::crate_name::foo` in the input of derive proc macros, it's passed to the macro instead with its precise span and hygiene data, and can be treated as any other path segment keyword (like `crate` or `self`) after that. (Note: `eliminate_crate_var` was never used for non-derive proc macros.)
This creates an annoying problem - derive macros still may stringify their input before processing and expect `$crate` survive that stringification and refer to the same crate (the Rust 1.15-1.29 way of doing things).
Moreover, the input of proc macro attributes and derives (but not fn-like proc macros) also effectively survives stringification before being passed to the macro (also for legacy implementation reasons).
So we kind of resurrect the `eliminate_crate_var` hack in reduced form, but apply it only to AST pretty-printing.
If an AST fragment is pretty-printed, the resulting *text* will have `$crate` replaced with `crate` or `::crate_name`. This should be enough to keep all the legacy cases working.
Closes https://github.com/rust-lang/rust/issues/55640
Closes https://github.com/rust-lang/rust/issues/56622
r? @ghost
|
|
This is done by adding a new `IsJoint` field to `TokenStream::Tree`,
which simplifies a lot of `match` statements. And likewise for
`CursorKind`.
The commit also adds a new method `TokenTree:stream()` which can replace
a choice between `.into()` and `.joint()`.
|
|
invalidation
|
|
Tweak query code for performance
Split from https://github.com/rust-lang/rust/pull/56509
r? @michaelwoerister
|
|
Do not point at delim spans for complete correct blocks
Fix #56834.
|
|
format-related tweaks
- remove an unreachable condition
- inline one-liners related to `parse_expr` (called in succession)
- refactor `report_invalid_references`
- refactor `verify_arg_type`
- minor stylistic improvements
|
|
|
|
|
|
`TokenStream` improvements
Some `TokenStream` improvements: shrinking `TokenStream` and some other types, and some other code clean-ups.
|
|
2018 edition - confusing error message when declaring unnamed parameters
Fixes #53990.
This PR adds a note providing context for the change to argument
names being required in the 2018 edition for trait methods and a
suggestion for the fix.
|
|
Use a `newtype_index!` within `Symbol`.
This shrinks `Option<Symbol>` from 8 bytes to 4 bytes, which shrinks
`Token` from 24 bytes to 16 bytes. This reduces instruction counts by up
to 1% across a range of benchmarks.
r? @oli-obk
|
|
Add non-panicking `maybe_new_parser_from_file` variant
Add (seemingly?) missing `maybe_new_parser_from_file` constructor variant.
Disclaimer: I'm not certain this is the correct approach - just found out we don't have this when working on a Rustfmt PR to catch/prevent more Rust parser panics: https://github.com/rust-lang/rustfmt/pull/3240 and tried to make it work somehow.
|
|
|
|
`TokenStream::new` is a better name for the former, and the latter is
now just equivalent to `TokenStream::Stream`.
|
|
|
|
This shrinks `Option<Symbol>` from 8 bytes to 4 bytes, which shrinks
`Token` from 24 bytes to 16 bytes. This reduces instruction counts by up
to 1% across a range of benchmarks.
|
|
Because it's an extra type layer that doesn't really help; in a couple
of places it actively gets in the way, and overall removing it makes the
code nicer. It does, however, move `tokenstream::TokenTree` further away
from the `TokenTree` in `quote.rs`.
More importantly, this change reduces the size of `TokenStream` from 48
bytes to 40 bytes on x86-64, which is enough to slightly reduce
instruction counts on numerous benchmarks, the best by 1.5%.
Note that `open_tt` and `close_tt` have gone from being methods on
`Delimited` to associated methods of `TokenTree`.
|
|
|
|
Suggest an appropriate token when encountering `pub Ident<'a>`
Fix #55403. Follow up to #45997.
|
|
|
|
This commit emits a suggestion for adding an underscore binding to
arguments in trait methods that previously did not have a argument name
specified.
|
|
This commit adds a note providing context for the change to argument
names being required in the 2018 edition for trait methods.
|
|
Delay gensym creation for "underscore items" (`use foo as _`/`const _`) until name resolution
So they cannot be cloned by macros. See https://github.com/rust-lang/rust/pull/56303 for the discussion.
Mostly fix cross-crate use of underscore items by inverting the "gensyms are lost in metadata" bug as described in https://github.com/rust-lang/rust/pull/56303#issuecomment-442464695.
Fix unused import warnings for single-segment imports (first commit) and `use crate_name as _` imports (as specified in https://github.com/rust-lang/rust/pull/56303#issuecomment-442274118).
Prohibit accidentally implemented `static _: TYPE = EXPR;` (cc https://github.com/rust-lang/rust/pull/55983).
Add more tests for `use foo as _` imports.
|
|
cleanup: remove static lifetimes from consts
A follow-up to https://github.com/rust-lang/rust/pull/56497.
|
|
libsyntax_pos: A few tweaks
|
|
r=michaelwoerister
53956 panic on include bytes of own file
fix #53956
When using `include_bytes!` on a source file in the project, compiler would panic on subsequent compilations because `expand_include_bytes` would overwrite files in the source_map with no source. This PR changes `expand_include_bytes` to check source_map and use the already existing src, if any.
|
|
syntax: Use iterator and pattern APIs instead of `char_at`
Iterating over chars with the `char_at(str, i)` `i += ch.len_utf8()` loop seems unidiomatic.
|
