| Age | Commit message (Collapse) | Author | Lines |
|---|
|
proc_macro: Reorganize public API
This commit is a reorganization of the `proc_macro` crate's public user-facing
API. This is the result of a number of discussions at the recent Rust All-Hands
where we're hoping to get the `proc_macro` crate into ship shape for
stabilization of a subset of its functionality in the Rust 2018 release.
The reorganization here is motivated by experiences from the `proc-macro2`,
`quote`, and `syn` crates on crates.io (and other crates which depend on them).
The main focus is future flexibility along with making a few more operations
consistent and/or fixing bugs. A summary of the changes made from today's
`proc_macro` API is:
* The `TokenNode` enum has been removed and the public fields of `TokenTree`
have also been removed. Instead the `TokenTree` type is now a public enum
(what `TokenNode` was) and each variant is an opaque struct which internally
contains `Span` information. This makes the various tokens a bit more
consistent, require fewer wrappers, and otherwise provides good
future-compatibility as opaque structs are easy to modify later on.
* `Literal` integer constructors have been expanded to be unambiguous as to what
they're doing and also allow for more future flexibility. Previously
constructors like `Literal::float` and `Literal::integer` were used to create
unsuffixed literals and the concrete methods like `Literal::i32` would create
a suffixed token. This wasn't immediately clear to all users (the
suffixed/unsuffixed aspect) and having *one* constructor for unsuffixed
literals required us to pick a largest type which may not always be true. To
fix these issues all constructors are now of the form
`Literal::i32_unsuffixed` or `Literal::i32_suffixed` (for all integral types).
This should allow future compatibility as well as being immediately clear
what's suffixed and what isn't.
* Each variant of `TokenTree` internally contains a `Span` which can also be
configured via `set_span`. For example `Literal` and `Term` now both
internally contain a `Span` rather than having it stored in an auxiliary
location.
* Constructors of all tokens are called `new` now (aka `Term::intern` is gone)
and most do not take spans. Manufactured tokens typically don't have a fresh
span to go with them and the span is purely used for error-reporting
**except** the span for `Term`, which currently affects hygiene. The default
spans for all these constructed tokens is `Span::call_site()` for now.
The `Term` type's constructor explicitly requires passing in a `Span` to
provide future-proofing against possible hygiene changes. It's intended that a
first pass of stabilization will likely only stabilize `Span::call_site()`
which is an explicit opt-in for "I would like no hygiene here please". The
intention here is to make this explicit in procedural macros to be
forwards-compatible with a hygiene-specifying solution.
* Some of the conversions for `TokenStream` have been simplified a little.
* The `TokenTreeIter` iterator was renamed to `token_stream::IntoIter`.
Overall the hope is that this is the "final pass" at the API of `TokenStream`
and most of `TokenTree` before stabilization. Explicitly left out here is any
changes to `Span`'s API which will likely need to be re-evaluated before
stabilization.
All changes in this PR have already been reflected to the [`proc-macro2`],
`quote`, and `syn` crates. New versions of all these crates have also been
published to crates.io.
Once this lands in nightly I plan on making an internals post again summarizing
the changes made here and also calling on all macro authors to give the APIs a
spin and see how they work. Hopefully pending no major issues we can then have
an FCP to stabilize later this cycle!
[`proc-macro2`]: https://docs.rs/proc-macro2/0.3.1/proc_macro2/
Closes #49596
|
|
|
|
This commit is a reorganization of the `proc_macro` crate's public user-facing
API. This is the result of a number of discussions at the recent Rust All-Hands
where we're hoping to get the `proc_macro` crate into ship shape for
stabilization of a subset of its functionality in the Rust 2018 release.
The reorganization here is motivated by experiences from the `proc-macro2`,
`quote`, and `syn` crates on crates.io (and other crates which depend on them).
The main focus is future flexibility along with making a few more operations
consistent and/or fixing bugs. A summary of the changes made from today's
`proc_macro` API is:
* The `TokenNode` enum has been removed and the public fields of `TokenTree`
have also been removed. Instead the `TokenTree` type is now a public enum
(what `TokenNode` was) and each variant is an opaque struct which internally
contains `Span` information. This makes the various tokens a bit more
consistent, require fewer wrappers, and otherwise provides good
future-compatibility as opaque structs are easy to modify later on.
* `Literal` integer constructors have been expanded to be unambiguous as to what
they're doing and also allow for more future flexibility. Previously
constructors like `Literal::float` and `Literal::integer` were used to create
unsuffixed literals and the concrete methods like `Literal::i32` would create
a suffixed token. This wasn't immediately clear to all users (the
suffixed/unsuffixed aspect) and having *one* constructor for unsuffixed
literals required us to pick a largest type which may not always be true. To
fix these issues all constructors are now of the form
`Literal::i32_unsuffixed` or `Literal::i32_suffixed` (for all integral types).
This should allow future compatibility as well as being immediately clear
what's suffixed and what isn't.
* Each variant of `TokenTree` internally contains a `Span` which can also be
configured via `set_span`. For example `Literal` and `Term` now both
internally contain a `Span` rather than having it stored in an auxiliary
location.
* Constructors of all tokens are called `new` now (aka `Term::intern` is gone)
and most do not take spans. Manufactured tokens typically don't have a fresh
span to go with them and the span is purely used for error-reporting
**except** the span for `Term`, which currently affects hygiene. The default
spans for all these constructed tokens is `Span::call_site()` for now.
The `Term` type's constructor explicitly requires passing in a `Span` to
provide future-proofing against possible hygiene changes. It's intended that a
first pass of stabilization will likely only stabilize `Span::call_site()`
which is an explicit opt-in for "I would like no hygiene here please". The
intention here is to make this explicit in procedural macros to be
forwards-compatible with a hygiene-specifying solution.
* Some of the conversions for `TokenStream` have been simplified a little.
* The `TokenTreeIter` iterator was renamed to `token_stream::IntoIter`.
Overall the hope is that this is the "final pass" at the API of `TokenStream`
and most of `TokenTree` before stabilization. Explicitly left out here is any
changes to `Span`'s API which will likely need to be re-evaluated before
stabilization.
All changes in this PR have already been reflected to the [`proc-macro2`],
`quote`, and `syn` crates. New versions of all these crates have also been
published to crates.io.
Once this lands in nightly I plan on making an internals post again summarizing
the changes made here and also calling on all macro authors to give the APIs a
spin and see how they work. Hopefully pending no major issues we can then have
an FCP to stabilize later this cycle!
[`proc-macro2`]: https://docs.rs/proc-macro2/0.3.1/proc_macro2/
|
|
Expand Attributes on Statements and Expressions
This enables attribute-macro expansion on statements and expressions while retaining the `stmt_expr_attributes` feature requirement for attributes on expressions.
closes #41475
cc #38356 @petrochenkov @jseyfried
r? @nrc
|
|
Retains the `stmt_expr_attributes` feature requirement for attributes on expressions.
closes #41475
cc #38356
|
|
This commit tweaks the tokenization of a doc comment to use `#[doc = "..."]`
like `macro_rules!` does (instead of treating it as a `Literal` token).
Additionally it fixes treatment of negative literals in the compiler, for
exapmle `Literal::i32(-1)`. The current fix is a bit of a hack around the
current compiler implementation, providing a fix at the proc-macro layer rather
than the libsyntax layer.
|
|
A broken test to be fixed in the next commit.
|
|
|
|
|
|
|
|
|
|
|
|
Fix hygiene bug.
Fixes #42708.
r? @nrc
|
|
Account for missing keyword in fn/struct definition
Fix #38911.
|
|
|
|
Fixes #46038
|
|
|
|
Remove a couple of obsolete lints
|
|
|
|
There were several feature-gated attributes for which the
feature-not-available error spans would point to the item annotated with
the gated attribute, when it would make more sense for the span to point
to the attribute itself: if the attribute is removed, the
function/struct/&c. likely still makes sense and the program will
compile. (Note that we decline to make the analogous change for the
`main`, `start`, and `plugin_registrar` features, for in those cases it
makes sense for the span to implicate the entire function, of which
there is little hope of using without the gated attribute.)
|
|
|
|
Ignore tests that fail on stage1
That makes `./x.py test --stage 1` work on `x86_64-unknown-linux-gnu`.
|
|
In preparation for incremental compilation this commit refactors the lint
handling infrastructure in the compiler to be more "eager" and overall more
incremental-friendly. Many passes of the compiler can emit lints at various
points but before this commit all lints were buffered in a table to be emitted
at the very end of compilation. This commit changes these lints to be emitted
immediately during compilation using pre-calculated lint level-related data
structures.
Linting today is split into two phases, one set of "early" lints run on the
`syntax::ast` and a "late" set of lints run on the HIR. This commit moves the
"early" lints to running as late as possible in compilation, just before HIR
lowering. This notably means that we're catching resolve-related lints just
before HIR lowering. The early linting remains a pass very similar to how it was
before, maintaining context of the current lint level as it walks the tree.
Post-HIR, however, linting is structured as a method on the `TyCtxt` which
transitively executes a query to calculate lint levels. Each request to lint on
a `TyCtxt` will query the entire crate's 'lint level data structure' and then go
from there about whether the lint should be emitted or not.
The query depends on the entire HIR crate but should be very quick to calculate
(just a quick walk of the HIR) and the red-green system should notice that the
lint level data structure rarely changes, and should hopefully preserve
incrementality.
Overall this resulted in a pretty big change to the test suite now that lints
are emitted much earlier in compilation (on-demand vs only at the end). This in
turn necessitated the addition of many `#![allow(warnings)]` directives
throughout the compile-fail test suite and a number of updates to the UI test
suite.
|
|
That makes ./x.py test --stage 1 work on x86_64-unknown-linux-gnu.
|
|
This is then later used by `proc_macro` to generate a new
`proc_macro::TokenTree` which preserves span information. Unfortunately this
isn't a bullet-proof approach as it doesn't handle the case when there's still
other attributes on the item, especially inner attributes.
Despite this the intention here is to solve the primary use case for procedural
attributes, attached to functions as outer attributes, likely bare. In this
situation we should be able to now yield a lossless stream of tokens to preserve
span information.
|
|
This test currently fails because the tokenization of an AST item during the
expansion of a procedural macro attribute rounds-trips through strings, losing
span information.
|
|
Fix checking for missing stability annotations
This was a regression from https://github.com/rust-lang/rust/pull/37676 causing "unmarked API" ICEs like https://github.com/rust-lang/rust/issues/43027.
r? @alexcrichton
|
|
|
|
Remove couple of unnecessary `#![feature(staged_api)]`.
|
|
|
|
|
|
This commit integrates the `jobserver` crate into the compiler. The crate was
previously integrated in to Cargo as part of rust-lang/cargo#4110. The purpose
here is to two-fold:
* Primarily the compiler can cooperate with Cargo on parallelism. When you run
`cargo build -j4` then this'll make sure that the entire build process between
Cargo/rustc won't use more than 4 cores, whereas today you'd get 4 rustc
instances which may all try to spawn lots of threads.
* Secondarily rustc/Cargo can now integrate with a foreign GNU `make` jobserver.
This means that if you call cargo/rustc from `make` or another
jobserver-compatible implementation it'll use foreign parallelism settings
instead of creating new ones locally.
As the number of parallel codegen instances in the compiler continues to grow
over time with the advent of incremental compilation it's expected that this'll
become more of a problem, so this is intended to nip concurrent concerns in the
bud by having all the tools to cooperate!
Note that while rustc has support for itself creating a jobserver it's far more
likely that rustc will always use the jobserver configured by Cargo. Cargo today
will now set a jobserver unconditionally for rustc to use.
|
|
Clearer use of the error message and span labels to communicate
duplicaiton defitions/imports.
New error format:
```
error[E0428]: the name `Foo` is defined twice
--> example.rs:2:1
|
1 | trait Foo { }
| ------------- previous definition of the trait `Foo` here
2 | struct Foo { }
| ^^^^^^^^^^^^^^ `Foo` redefined here
= note: `Foo` must be defined only once in the type namespace of this module
error: aborting due to previous error
```
|
|
Also, add detection to treat such comments as tidy errors.
We also remove the found_lib_feature code because it
was just repeating the found_feature code. Originally it
was intended to allow for gate-test lines for
lib features, but apparently nobody missed it.
|
|
|
|
reproducible builds.
|
|
Adds temporary regression test; this ideally should work as-is (#41430)
Closes #41211
|
|
|
|
Stabilize pub(restricted)
Fix https://github.com/rust-lang/rust/issues/32409
|
|
`TokenStream`-based attributes, paths in attribute and derive macro invocations
This PR
- refactors `Attribute` to use `Path` and `TokenStream` instead of `MetaItem`.
- supports macro invocation paths for attribute procedural macros.
- e.g. `#[::foo::attr_macro] struct S;`, `#[cfg_attr(all(), foo::attr_macro)] struct S;`
- supports macro invocation paths for derive procedural macros.
- e.g. `#[derive(foo::Bar, super::Baz)] struct S;`
- supports arbitrary tokens as arguments to attribute procedural macros.
- e.g. `#[foo::attr_macro arbitrary + tokens] struct S;`
- supports using arbitrary tokens in "inert attributes" with derive procedural macros.
- e.g. `#[derive(Foo)] struct S(#[inert arbitrary + tokens] i32);`
where `#[proc_macro_derive(Foo, attributes(inert))]`
r? @nrc
|
|
|
|
|
|
|
|
Implement function-like procedural macros ( `#[proc_macro]`)
Adds the `#[proc_macro]` attribute, which expects bare functions of the kind `fn(TokenStream) -> TokenStream`, which can be invoked like `my_macro!()`.
cc rust-lang/rfcs#1913, #38356
r? @jseyfried
cc @nrc
|
|
|
|
`tokenstream::TokenTree::Sequence`.
|
|
This commit searchs modules for macro suggestions.
It also removes imported macro_rules from macro_names,
and adds more corner case checks for which macros
should be suggested in specific contexts.
|
|
|
|
`#![feature(proc_macro)]`.
|
|
This removes the expand_derives function, and sprinkles
the functionality throughout the Invocation Collector,
Expander and Resolver.
|
