| Age | Commit message (Collapse) | Author | Lines |
|---|
|
This commit updates the `Mac_` AST structure to keep track of the delimiters
that it originally had for its invocation. This allows us to faithfully
pretty-print macro invocations not using parentheses (e.g. `vec![...]`). This in
turn helps procedural macros due to #43081.
Closes #50840
|
|
rustc: Fix procedural macros generating lifetime tokens
This commit fixes an accidental regression from #50473 where lifetime tokens
produced by procedural macros ended up getting lost in translation in the
compiler and not actually producing parseable code. The issue lies in the fact
that a lifetime's `Ident` is prefixed with `'`. The `glue` implementation for
gluing joint tokens together forgot to take this into account so the lifetime
inside of `Ident` was missing the leading tick!
The `glue` implementation here is updated to create a new `Symbol` in these
situations to manufacture a new `Ident` with a leading tick to ensure it parses
correctly.
Closes #50942
|
|
Issue #50636: Improve error diagnostic with missing commas after struct fields.
Fixes #50636
|
|
|
|
rustc: Fix joint-ness of stringified token-streams
This commit fixes `StringReader`'s parsing of tokens which have been stringified
through procedural macros. Whether or not a token tree is joint is defined by
span information, but when working with procedural macros these spans are often
dummy and/or overridden which means that they end up considering all operators
joint if they can!
The fix here is to track the raw source span as opposed to the overridden span.
With this information we can more accurately classify `Punct` structs as either
joint or not.
Closes #50700
|
|
|
|
|
|
|
|
This commit fixes an accidental regression from #50473 where lifetime tokens
produced by procedural macros ended up getting lost in translation in the
compiler and not actually producing parseable code. The issue lies in the fact
that a lifetime's `Ident` is prefixed with `'`. The `glue` implementation for
gluing joint tokens together forgot to take this into account so the lifetime
inside of `Ident` was missing the leading tick!
The `glue` implementation here is updated to create a new `Symbol` in these
situations to manufacture a new `Ident` with a leading tick to ensure it parses
correctly.
Closes #50942
|
|
|
|
lexer: Fix span override for the first token in a string
Previously due to peculiarities of `StringReader` construction something like `"a b c d".parse::<TokenStream>()` gave you one non-overridden span for `a` and then three correctly overridden spans for `b`, `c` and `d`.
Now all the spans are overridden.
|
|
rustc: introduce {ast,hir}::AnonConst to consolidate so-called "embedded constants".
Previously, constants in array lengths and enum variant discriminants were "merely an expression", and had no separate ID for, e.g. type-checking or const-eval, instead reusing the expression's.
That complicated code working with bodies, because such constants were the only special case where the "owner" of the body wasn't the HIR parent, but rather the same node as the body itself.
Also, if the body happened to be a closure, we had no way to allocate a `DefId` for both the constant *and* the closure, leading to *several* bugs (mostly ICEs where type errors were expected).
This PR rectifies the situation by adding another (`{ast,hir}::AnonConst`) node around every such constant. Also, const generics are expected to rely on the new `AnonConst` nodes, as well (cc @varkor).
* fixes #48838
* fixes #50600
* fixes #50688
* fixes #50689
* obsoletes #50623
r? @nikomatsakis
|
|
Consider this a down payment on #50723. To recap, an `Applicability`
enum was recently (#50204) added, to convey to Rustfix and other tools
whether we think it's OK for them to blindly apply the suggestion, or
whether to prompt a human for guidance (because the suggestion might
contain placeholders that we can't infer, or because we think it has a
sufficiently high probability of being wrong even though it's—
presumably—right often enough to be worth emitting in the first place).
When a suggestion is marked as `MaybeIncorrect`, we try to use comments
to indicate precisely why (although there are a few places where we just
say `// speculative` because the present author's subjective judgement
balked at the idea that the suggestion has no false positives).
The `run-rustfix` directive is opporunistically set on some relevant UI
tests (and a couple tests that were in the `test/ui/suggestions`
directory, even if the suggestions didn't originate in librustc or
libsyntax). This is less trivial than it sounds, because a surprising
number of test files aren't equipped to be tested as fixed even when
they contain successfully fixable errors, because, e.g., there are more,
not-directly-related errors after fixing. Some test files need an
attribute or underscore to avoid unused warnings tripping up the "fixed
code is still producing diagnostics" check despite the fixes being
correct; this is an interesting contrast-to/inconsistency-with the
behavior of UI tests (which secretly pass `-A unused`), a behavior which
we probably ought to resolve one way or the other (filed issue #50926).
A few suggestion labels are reworded (e.g., to avoid phrasing it as a
question, which which is discouraged by the style guidelines listed in
`.span_suggestion`'s doc-comment).
|
|
|
|
|
|
Speed up the macro parser
These three commits reduce the number of allocations done by the macro parser, in some cases dramatically. For example, for a clean check builds of html5ever, the number of allocations is reduced by 40%.
Here are the rustc-benchmarks that are sped up by at least 1%.
```
html5ever-check
avg: -6.6% min: -10.3% max: -4.1%
html5ever
avg: -5.2% min: -9.5% max: -2.8%
html5ever-opt
avg: -4.3% min: -9.3% max: -1.6%
crates.io-check
avg: -1.8% min: -2.9% max: -0.6%
crates.io-opt
avg: -1.0% min: -2.2% max: -0.1%
crates.io
avg: -1.1% min: -2.2% max: -0.2%
```
|
|
rustc: Disallow modules and macros in expansions
This commit feature gates generating modules and macro definitions in procedural
macro expansions. Custom derive is exempt from this check as it would be a large
retroactive breaking change (#50587). It's hoped that we can hopefully stem the
bleeding to figure out a better solution here before opening up the floodgates.
The restriction here is specifically targeted at surprising hygiene results [1]
that result in non-"copy/paste" behavior. Hygiene and procedural macros is
intended to be avoided as much as possible for Macros 1.2 by saying everything
is "as if you copy/pasted the code", but modules and macros are sort of weird
exceptions to this rule that aren't fully fleshed out.
[1]: https://github.com/rust-lang/rust/issues/50504#issuecomment-387734625
cc #50504
|
|
constants".
|
|
This commit feature gates generating modules and macro definitions in procedural
macro expansions. Custom derive is exempt from this check as it would be a large
retroactive breaking change (#50587). It's hoped that we can hopefully stem the
bleeding to figure out a better solution here before opening up the floodgates.
The restriction here is specifically targeted at surprising hygiene results [1]
that result in non-"copy/paste" behavior. Hygiene and procedural macros is
intended to be avoided as much as possible for Macros 1.2 by saying everything
is "as if you copy/pasted the code", but modules and macros are sort of weird
exceptions to this rule that aren't fully fleshed out.
[1]: https://github.com/rust-lang/rust/issues/50504#issuecomment-387734625
cc #50504
|
|
This commit fixes `StringReader`'s parsing of tokens which have been stringified
through procedural macros. Whether or not a token tree is joint is defined by
span information, but when working with procedural macros these spans are often
dummy and/or overridden which means that they end up considering all operators
joint if they can!
The fix here is to track the raw source span as opposed to the overridden span.
With this information we can more accurately classify `Punct` structs as either
joint or not.
Closes #50700
|
|
Because we create a lot of these in the macro parser, but only very
rarely modify them.
This speeds up some html5ever runs by 2--3%.
|
|
This lets us store most `MatcherPos` instances on the stack. This speeds
up various runs of html5ever, the best by 3%.
|
|
Implement edition hygiene for keywords
Determine "keywordness" of an identifier in its hygienic context.
cc https://github.com/rust-lang/rust/pull/49611
I've resurrected `proc` as an Edition-2015-only keyword for testing purposes, but it should probably be buried again. EDIT: `proc` is removed again.
|
|
Streamline `StringReader::bump`
These patches make `bump` smaller and nicer. They speed up most runs for coercions and tuple-stress by 1--3%.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This avoids a `to_owned` call that can be hot, speeding up the various
runs of html5ever by 1--5%, and some runs of crates.io by 2--3%.
|
|
Implement label break value (RFC 2046)
Implement label-break-value (#48594).
|
|
|
|
|
|
Review proc macro API 1.2
cc https://github.com/rust-lang/rust/issues/38356
Summary of applied changes:
- Documentation for proc macro API 1.2 is expanded.
- Renamed APIs: `Term` -> `Ident`, `TokenTree::Term` -> `TokenTree::Ident`, `Op` -> `Punct`, `TokenTree::Op` -> `TokenTree::Punct`, `Op::op` -> `Punct::as_char`.
- Removed APIs: `Ident::as_str`, use `Display` impl for `Ident` instead.
- New APIs (not stabilized in 1.2): `Ident::new_raw` for creating a raw identifier (I'm not sure `new_x` it's a very idiomatic name though).
- Runtime changes:
- `Punct::new` now ensures that the input `char` is a valid punctuation character in Rust.
- `Ident::new` ensures that the input `str` is a valid identifier in Rust.
- Lifetimes in proc macros are now represented as two joint tokens - `Punct('\'', Spacing::Joint)` and `Ident("lifetime_name_without_quote")` similarly to multi-character operators.
- Stabilized APIs: None yet.
A bit of motivation for renaming (although it was already stated in the review comments):
- With my compiler frontend glasses on `Ident` is the single most appropriate name for this thing, *especially* if we are doing input validation on construction. `TokenTree::Ident` effectively wraps `token::Ident` or `ast::Ident + is_raw`, its meaning is "identifier" and it's already named `ident` in declarative macros.
- Regarding `Punct`, the motivation is that `Op` is actively misleading. The thing doesn't mean an operator, it's neither a subset of operators (there is non-operator punctuation in the language), nor superset (operators can be multicharacter while this thing is always a single character). So I named it `Punct` (first proposed in [the original RFC](https://github.com/rust-lang/rfcs/pull/1566), then [by @SimonSapin](https://github.com/rust-lang/rust/issues/38356#issuecomment-276676526)) , together with input validation it's now a subset of ASCII punctuation character category (`u8::is_ascii_punctuation`).
|
|
|
|
|
|
|
|
It only has a single use, within code handling indented block comments.
We can replace that with the new `FileMap::col_pos()`, which computes
the col position (BytePos instead of CharPos) based on the record of the
last newline char (which we already record).
This is actually an improvement, because
`trim_whitespace_prefix_and_push_line()` was using `col`, which is a
`CharPos`, as a slice index, which is a byte/char confusion.
|
|
|
|
|
|
Implements RFC 1576.
See: https://github.com/rust-lang/rfcs/blob/master/text/1576-macros-literal-matcher.md
Changes are mostly in libsyntax, docs, and tests. Feature gate is
enabled for 1.27.0.
Many thanks to Vadim Petrochenkov for following through code reviews
and suggestions.
Example:
````rust
macro_rules! test_literal {
($l:literal) => {
println!("literal: {}", $l);
};
($e:expr) => {
println!("expr: {}", $e);
};
}
fn main() {
let a = 1;
test_literal!(a);
test_literal!(2);
test_literal!(-3);
}
```
Output:
```
expr: 1
literal: 2
literal: -3
```
|
|
It's silly for a hot function like `bump()` to have such an expensive
bounds check. This patch replaces terminator with `end_src_index`.
Note that the `self.terminator` check in `is_eof()` wasn't necessary
because of the way `StringReader` is initialized.
|
|
- `source_text` becomes `src`, matching `FileMap::src`.
- `byte_offset()` becomes `src_index()`, which makes it clearer that
it's an index into `src`. (Likewise for variables containing
`byte_offset` in their name.) This function also now returns a `usize`
instead of a `BytePos`, because every callsite immediately converted
the `BytePos` to a `usize`.
|
|
This patch removes the "old"/"new" names in favour of "foo"/"next_foo",
which matches the field names.
It also moves the setting of `self.{ch,pos,next_pos}` in the common case
to the end, so that the meaning of "foo"/"next_foo" is consistent until
the end.
|
|
Because `bump()` is hot.
|
|
|
|
This commit fixes a hard error where the `#![feature(rust_2018_preview)]`
feature was forbidden to be mentioned when the `--edition 2018` flag was passed.
This instead silently accepts that feature gate despite it not being necessary.
It's intended that this will help ease the transition into the 2018 edition as
users will, for the time being, start off with the `rust_2018_preview` feature
and no longer immediately need to remove it.
Closes #50662
|
|
Rename the 2018 edition lint names
* `rust_2018_breakage` -> `rust_2018_compatibility` - the lint for ensuring
that your code, in the 2015 edition, is compatible with the 2018 edition's
semantics. This is required to pass *before* you enable the 2018 edition.
* `rust_2018_migration` -> `rust_2018_idioms` - the lint for writing idiomatic
code after you've already enabled the 2018 edition
|
|
Remove `nll` from `rust_2018_preview`
NLL isn't quite ready yet so gonna hold off on inserting it into the preview.
|
