| Age | Commit message (Collapse) | Author | Lines |
|---|
|
position that originates in macros
|
|
The change was "Show invisible delimiters (within comments) when pretty
printing". It's useful to show these delimiters, but is a breaking
change for some proc macros.
Fixes #97608.
|
|
|
|
|
|
|
|
then we just suggest the first legal position where you could inject a use.
To do this, I added `inject_use_span` field to `ModSpans`, and populate it in
parser (it is the span of the first token found after inner attributes, if any).
Then I rewrote the use-suggestion code to utilize it, and threw out some stuff
that is now unnecessary with this in place. (I think the result is easier to
understand.)
Then I added a test of issue 87613.
|
|
Rollup of 4 pull requests
Successful merges:
- #90529 (Skip reborrows in AbstractConstBuilder)
- #91437 (Pretty print empty blocks as {})
- #91450 (Don't suggest types whose inner type is erroneous)
- #91535 (Stabilize `-Z emit-future-incompat` as `--json future-incompat`)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
|
|
|
|
|
|
This feature is aimed at giving proc macros access to powers similar to
those used by builtin macros such as `format_args!` or `concat!`. These
macros are able to accept macros in place of string literal parameters,
such as the format string, as they perform recursive macro expansion
while being expanded.
This can be especially useful in many cases thanks to helper macros like
`concat!`, `stringify!` and `include_str!` which are often used to
construct string literals at compile-time in user code.
For now, this method only allows expanding macros which produce
literals, although more expresisons will be supported before the method
is stabilized.
|
|
|
|
|
|
|
|
Proc macro spans: make columns 1 based
This makes proc macro spans consistent with the `column!()` macro as well as `std::panic::Location`, as both are 1-based.
https://github.com/rust-lang/rust/issues/54725#issuecomment-497246753
|
|
|
|
Support negative numbers in Literal::from_str
proc_macro::Literal has allowed negative numbers in a single literal token ever since Rust 1.29, using https://doc.rust-lang.org/stable/proc_macro/struct.Literal.html#method.isize_unsuffixed and similar constructors.
```rust
let lit = proc_macro::Literal::isize_unsuffixed(-10);
```
However, the suite of constructors on Literal is not sufficient for all use cases, for example arbitrary precision floats, or custom suffixes in FFI macros.
```rust
let lit = proc_macro::Literal::f64_unsuffixed(0.101001000100001000001000000100000001); // :(
let lit = proc_macro::Literal::i???_suffixed(10ulong); // :(
```
For those, macros construct the literal using from_str instead, which preserves arbitrary precision, custom suffixes, base, and digit grouping.
```rust
let lit = "0.101001000100001000001000000100000001".parse::<Literal>().unwrap();
let lit = "10ulong".parse::<Literal>().unwrap();
let lit = "0b1000_0100_0010_0001".parse::<Literal>().unwrap();
```
However, until this PR it was not possible to construct a literal token that is **both** negative **and** preserving of arbitrary precision etc.
This PR fixes `Literal::from_str` to recognize negative integer and float literals.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
In addition to making the output look nicer for all crates, this also
aligns the pretty-printing output with what the `rental` crate expects.
This will allow us to eventually disable a backwards-compat hack in a
follow-up PR.
|
|
This PR implements span quoting, allowing proc-macros to produce spans
pointing *into their own crate*. This is used by the unstable
`proc_macro::quote!` macro, allowing us to get error messages like this:
```
error[E0412]: cannot find type `MissingType` in this scope
--> $DIR/auxiliary/span-from-proc-macro.rs:37:20
|
LL | pub fn error_from_attribute(_args: TokenStream, _input: TokenStream) -> TokenStream {
| ----------------------------------------------------------------------------------- in this expansion of procedural macro `#[error_from_attribute]`
...
LL | field: MissingType
| ^^^^^^^^^^^ not found in this scope
|
::: $DIR/span-from-proc-macro.rs:8:1
|
LL | #[error_from_attribute]
| ----------------------- in this macro invocation
```
Here, `MissingType` occurs inside the implementation of the proc-macro
`#[error_from_attribute]`. Previosuly, this would always result in a
span pointing at `#[error_from_attribute]`
This will make many proc-macro-related error message much more useful -
when a proc-macro generates code containing an error, users will get an
error message pointing directly at that code (within the macro
definition), instead of always getting a span pointing at the macro
invocation site.
This is implemented as follows:
* When a proc-macro crate is being *compiled*, it causes the `quote!`
macro to get run. This saves all of the sapns in the input to `quote!`
into the metadata of *the proc-macro-crate* (which we are currently
compiling). The `quote!` macro then expands to a call to
`proc_macro::Span::recover_proc_macro_span(id)`, where `id` is an
opaque identifier for the span in the crate metadata.
* When the same proc-macro crate is *run* (e.g. it is loaded from disk
and invoked by some consumer crate), the call to
`proc_macro::Span::recover_proc_macro_span` causes us to load the span
from the proc-macro crate's metadata. The proc-macro then produces a
`TokenStream` containing a `Span` pointing into the proc-macro crate
itself.
The recursive nature of 'quote!' can be difficult to understand at
first. The file `src/test/ui/proc-macro/quote-debug.stdout` shows
the output of the `quote!` macro, which should make this eaier to
understand.
This PR also supports custom quoting spans in custom quote macros (e.g.
the `quote` crate). All span quoting goes through the
`proc_macro::quote_span` method, which can be called by a custom quote
macro to perform span quoting. An example of this usage is provided in
`src/test/ui/proc-macro/auxiliary/custom-quote.rs`
Custom quoting currently has a few limitations:
In order to quote a span, we need to generate a call to
`proc_macro::Span::recover_proc_macro_span`. However, proc-macros
support renaming the `proc_macro` crate, so we can't simply hardcode
this path. Previously, the `quote_span` method used the path
`crate::Span` - however, this only works when it is called by the
builtin `quote!` macro in the same crate. To support being called from
arbitrary crates, we need access to the name of the `proc_macro` crate
to generate a path. This PR adds an additional argument to `quote_span`
to specify the name of the `proc_macro` crate. Howver, this feels kind
of hacky, and we may want to change this before stabilizing anything
quote-related.
Additionally, using `quote_span` currently requires enabling the
`proc_macro_internals` feature. The builtin `quote!` macro
has an `#[allow_internal_unstable]` attribute, but this won't work for
custom quote implementations. This will likely require some additional
tricks to apply `allow_internal_unstable` to the span of
`proc_macro::Span::recover_proc_macro_span`.
|
|
Expand derive invocations in left-to-right order
While derives were being collected in left-to-order order, the
corresponding `Invocation`s were being pushed in the wrong order.
|
|
While derives were being collected in left-to-order order, the
corresponding `Invocation`s were being pushed in the wrong order.
|
|
|
|
Do not emit the advanced diagnostics on macros
Fixes #83510
|
|
Add regression tests for #79825 and #81555
Closes #79825.
Closes #81555.
`@rustbot` label A-proc-macros T-compiler
|
|
|
|
|
|
Previously, we would silently remove any `None`-delimiters when
capturing a `TokenStream`, 'flattenting' them to their inner tokens.
This was not normally visible, since we usually have
`TokenKind::Interpolated` (which gets converted to a `None`-delimited
group during macro invocation) instead of an actual `None`-delimited
group.
However, there are a couple of cases where this becomes visible to
proc-macros:
1. A cross-crate `macro_rules!` macro has a `None`-delimited group
stored in its body (as a result of being produced by another
`macro_rules!` macro). The cross-crate `macro_rules!` invocation
can then expand to an attribute macro invocation, which needs
to be able to see the `None`-delimited group.
2. A proc-macro can invoke an attribute proc-macro with its re-collected
input. If there are any nonterminals present in the input, they will
get re-collected to `None`-delimited groups, which will then get
captured as part of the attribute macro invocation.
Both of these cases are incredibly obscure, so there hopefully won't be
any breakage. This change will allow more agressive 'flattenting' of
nonterminals in #82608 without losing `None`-delimited groups.
|
|
Currently, the print helper macro performs 'recollection' by doing
`token_stream.into_iter().collect()`. However, this will not affect
nonterminals that occur nested inside delimited groups, since the
wrapping delimited group will be left untouched.
This commit adds 'deep recollection', which recursively recollects every
delimited group in the token stream. As with normal recollection, we
only print out something if deep recollection results in a different
stringified token stream.
This is useful for catching bugs where we update the AST of a
nonterminal (which affects pretty-printing), but do not update the
attatched `TokenStream`
|
|
|
|
|
|
|
|
Reverts PR #80830
Fixes taiki-e/pin-project#312
We can have an arbitrary number of `None`-delimited group frames pushed
on the stack due to proc-macro invocations, which can legally be exited.
Attempting to account for this would add a lot of complexity for a tiny
performance gain, so let's just use the original strategy.
|
|
|
|
Properly handle attributes on statements
We now collect tokens for the underlying node wrapped by `StmtKind`
nstead of storing tokens directly in `Stmt`.
`LazyTokenStream` now supports capturing a trailing semicolon after it
is initially constructed. This allows us to avoid refactoring statement
parsing to wrap the parsing of the semicolon in `parse_tokens`.
Attributes on item statements
(e.g. `fn foo() { #[bar] struct MyStruct; }`) are now treated as
item attributes, not statement attributes, which is consistent with how
we handle attributes on other kinds of statements. The feature-gating
code is adjusted so that proc-macro attributes are still allowed on item
statements on stable.
Two built-in macros (`#[global_allocator]` and `#[test]`) needed to be
adjusted to support being passed `Annotatable::Stmt`.
|
|
Loading a macro from libstd causes us to load serialized
`SyntaxContext`s in a platform-dependent way, causing the printed spans
to differ between platforms.
|
|
Fixes #79242
If a `macro_rules!` recursively builds up a nested nonterminal
(passing it to a proc-macro at each step), we will end up repeatedly
pretty-printing/retokenizing the same nonterminals. Unfortunately, the
'probable equality' check we do has a non-trivial cost, which leads to a
blowup in compilation time.
As a workaround, we cache the result of the 'probable equality' check,
which eliminates the compilation time blowup for the linked issue. This
commit only touches a single file (other than adding tests), so it
should be easy to backport.
The proper solution is to remove the pretty-print/retokenize hack
entirely. However, this will almost certainly break a large number of
crates that were relying on hygiene bugs created by using the reparsed
`TokenStream`. As a result, we will definitely not want to backport
such a change.
|
|
|
|
|
|
Tracking issue: #54723
This is a continuation of PR #59002
|
|
This is a very obscure corner case, and should never be hit in practice.
|
|
Handle `macro_rules!` tokens consistently across crates
When we serialize a `macro_rules!` macro, we used a 'lowered' `TokenStream` for its body, which has all `Nonterminal`s expanded in-place via `nt_to_tokenstream`. This matters when an 'outer' `macro_rules!` macro expands to an 'inner' `macro_rules!` macro - the inner macro may use tokens captured from the 'outer' macro in its definition.
This means that invoking a foreign `macro_rules!` macro may use a different body `TokenStream` than when the same `macro_rules!` macro is invoked in the same crate. This difference is observable by proc-macros invoked by a `macro_rules!` macro - a `None`-delimited group will be seen in the same-crate case (inserted when convering `Nonterminal`s to the `proc_macro` crate's structs), but no `None`-delimited group in the cross-crate case.
To fix this inconsistency, we now insert `None`-delimited groups when 'lowering' a `Nonterminal` `macro_rules!` body, just as we do in `proc_macro_server`. Additionally, we no longer print extra spaces for `None`-delimited groups - as far as pretty-printing is concerned, they don't exist (only their contents do). This ensures that `Display` output of a `TokenStream` does not depend on which crate a `macro_rules!` macro was invoked from.
This PR is necessary in order to patch the `solana-genesis-programs` for the upcoming hygiene serialization breakage (https://github.com/rust-lang/rust/pull/72121#issuecomment-646924847). The `solana-genesis-programs` crate will need to use a proc macro to re-span certain tokens in a nested `macro_rules!`, which requires us to consistently use a `None`-delimited group.
See `src/test/ui/proc-macro/nested-macro-rules.rs` for an example of the kind of nested `macro_rules!` affected by this crate.
|
|
When a `macro_rules!` macro expands to another `macro_rules!` macro, we
may see `None`-delimited groups in odd places when another crate
deserializes the 'inner' macro. This commit 'unwraps' an outer
`None`-delimited group to avoid breaking existing code.
See https://github.com/rust-lang/rust/pull/73569#issuecomment-650860457
for more details.
The proper fix is to handle `None`-delimited groups systematically
throughout the parser, but that will require significant work. In the
meantime, this hack lets us fix important hygiene bugs in macros
|
|
|
