about summary refs log tree commit diff
path: root/src/libsyntax/parse/token.rs
AgeCommit message (Collapse)AuthorLines
2017-12-09Use Try syntax for Option in place of macros or matchMatt Brubeck-4/+1
2017-11-21Support `::crate` in pathsVadim Petrochenkov-0/+1
2017-11-06Using `...` in expressions is now an errorBadel2-1/+1
2017-10-25Improve recovery after unexpected tokens parsing sequenceThomas Karpiniec-0/+10
2017-10-14Implement `dyn Trait` syntaxVadim Petrochenkov-0/+6
2017-09-22Add support for `..=` syntaxAlex Burka-15/+20
Add ..= to the parser Add ..= to libproc_macro Add ..= to ICH Highlight ..= in rustdoc Update impl Debug for RangeInclusive to ..= Replace `...` to `..=` in range docs Make the dotdoteq warning point to the ... Add warning for ... in expressions Updated more tests to the ..= syntax Updated even more tests to the ..= syntax Updated the inclusive_range entry in unstable book
2017-09-18rustc: Forbid interpolated tokens in the HIRAlex Crichton-0/+85
Right now the HIR contains raw `syntax::ast::Attribute` structure but nowadays these can contain arbitrary tokens. One variant of the `Token` enum is an "interpolated" token which basically means to shove all the tokens for a nonterminal in this position. A "nonterminal" in this case is roughly analagous to a macro argument: macro_rules! foo { ($a:expr) => { // $a is a nonterminal as an expression } } Currently nonterminals contain namely items and expressions, and this poses a problem for incremental compilation! With incremental we want a stable hash of all HIR items, but this means we may transitively need a stable hash *of the entire AST*, which is certainly not stable w/ node ids and whatnot. Hence today there's a "bug" where the "stable hash" of an AST is just the raw hash value of the AST, and this only arises with interpolated nonterminals. The downside of this approach, however, is that a bunch of errors get spewed out during compilation about how this isn't a great idea. This PR is focused at fixing these warnings, basically deleting them from the compiler. The implementation here is to alter attributes as they're lowered from the AST to HIR, expanding all nonterminals in-place as we see them. This code for expanding a nonterminal to a token stream already exists for the `proc_macro` crate, so we basically just reuse the same implementation there. After this PR it's considered a bug to have an `Interpolated` token and hence the stable hash implementation simply uses `bug!` in this location. Closes #40946
2017-07-28Generator literal supportJohn Kåre Alsaker-0/+1
2017-07-05Merge remote-tracking branch 'origin/master' into proc_macro_apiAlex Crichton-10/+20
2017-06-29Change some terminology around keywords and reserved identifierspetrochenkov-9/+18
2017-06-29Make `$crate` a keywordVadim Petrochenkov-1/+2
2017-06-26Address review comments.Jeffrey Seyfried-12/+32
2017-06-26Add `LazyTokenStream`.Jeffrey Seyfried-6/+47
2017-06-26Implement `quote!` and other `proc_macro` API.Jeffrey Seyfried-0/+63
2017-06-26Simplify `hygiene::Mark` application, andJeffrey Seyfried-3/+0
remove variant `Token::SubstNt` in favor of `quoted::TokenTree::MetaVar`.
2017-05-12Fix some clippy warnings in libsyntaxAndre Bogus-21/+25
This is mostly removing stray ampersands, needless returns and lifetimes.
2017-04-15move NtVis enum variant to stave off comment rotAlex Burka-1/+1
2017-04-15Implementation of the `vis` macro matcher.Daniel Keep-0/+2
2017-03-30Improve `Path` spans.Jeffrey Seyfried-24/+24
2017-03-21Refactor parsing of trait object typesVadim Petrochenkov-15/+29
2017-03-19Auto merge of #40346 - jseyfried:path_and_tokenstream_attr, r=nrcbors-2/+2
`TokenStream`-based attributes, paths in attribute and derive macro invocations This PR - refactors `Attribute` to use `Path` and `TokenStream` instead of `MetaItem`. - supports macro invocation paths for attribute procedural macros. - e.g. `#[::foo::attr_macro] struct S;`, `#[cfg_attr(all(), foo::attr_macro)] struct S;` - supports macro invocation paths for derive procedural macros. - e.g. `#[derive(foo::Bar, super::Baz)] struct S;` - supports arbitrary tokens as arguments to attribute procedural macros. - e.g. `#[foo::attr_macro arbitrary + tokens] struct S;` - supports using arbitrary tokens in "inert attributes" with derive procedural macros. - e.g. `#[derive(Foo)] struct S(#[inert arbitrary + tokens] i32);` where `#[proc_macro_derive(Foo, attributes(inert))]` r? @nrc
2017-03-14Refactor `Attribute` to use `Path` and `TokenStream` instead of `MetaItem`.Jeffrey Seyfried-2/+2
2017-03-11Temporarily prefix catch block with do keywordTaylor Cramer-0/+1
2017-02-28Remove `Token::MatchNt`.Jeffrey Seyfried-3/+0
2017-02-28Clean up `ext::tt::transcribe::TtFrame`, rename to `Frame`.Jeffrey Seyfried-2/+2
2017-01-27Rollup merge of #39335 - cramertj:cramertj/can_begin_expr_fix, r=petrochenkovAlex Crichton-1/+23
Fix can_begin_expr keyword behavior Partial fix for #28784.
2017-01-26Fix can_begin_expr keyword behaviorTaylor Cramer-1/+23
2017-01-24Refactor parsing of generic arguments/parameters and where clausesVadim Petrochenkov-0/+23
2017-01-23Remove `open_span` and `close_span` from `Delimited`.Jeffrey Seyfried-0/+6
2016-11-20Move `syntax::util::interner` -> `syntax::symbol`, cleanup.Jeffrey Seyfried-267/+1
2016-11-20Refactor `P<ast::MetaItem>` -> `ast::MetaItem`.Jeffrey Seyfried-1/+1
2016-11-20Avoid clearing the string interner.Jeffrey Seyfried-4/+0
2016-11-03Reduce the size of `Token` and make it cheaper to clone by refactoringJeffrey Seyfried-15/+20
`Token::Interpolated(Nonterminal)` -> `Token::Interpolated(Rc<Nonterminal>)`.
2016-10-20Tweak path parsing logicVadim Petrochenkov-4/+7
2016-10-19Improve `$crate`.Jeffrey Seyfried-17/+0
2016-10-15Auto merge of #37132 - petrochenkov:intern, r=alexcrichtonbors-10/+3
Get rid of double indirection in string interner
2016-10-13Get rid of double indirection in string interner by using `Rc<str>`Vadim Petrochenkov-10/+3
2016-10-10Avoid allocations in `Decoder::read_str`.Nicholas Nethercote-1/+1
`opaque::Decoder::read_str` is very hot within `rustc` due to its use in the reading of crate metadata, and it currently returns a `String`. This commit changes it to instead return a `Cow<str>`, which avoids a heap allocation. This change reduces the number of calls to `malloc` by almost 10% in some benchmarks. This is a [breaking-change] to libserialize.
2016-09-14Remove unused Token::to_binop function.Nicholas Nethercote-26/+1
2016-09-04Replace `_, _` with `..`Vadim Petrochenkov-3/+3
2016-08-19Auto merge of #33922 - estebank:doc-comment, r=alexcrichtonbors-1/+9
Specific error message for missplaced doc comments Identify when documetation comments have been missplaced in the following places: * After a struct element: ```rust // file.rs: struct X { a: u8 /** document a */, } ``` ```bash $ rustc file.rs file.rs:2:11: 2:28 error: found documentation comment that doesn't document anything file.rs:2 a: u8 /** document a */, ^~~~~~~~~~~~~~~~~ file.rs:2:11: 2:28 help: doc comments must come before what they document, maybe a comment was intended with `//`? ``` * As the last line of a struct: ```rust // file.rs: struct X { a: u8, /// incorrect documentation } ``` ```bash $ rustc file.rs file.rs:3:5: 3:27 error: found a documentation comment that doesn't document anything file.rs:3 /// incorrect documentation ^~~~~~~~~~~~~~~~~~~~~~ file.rs:3:5: 3:27 help: doc comments must come before what they document, maybe a comment was intended with `//`? ``` * As the last line of a `fn`: ```rust // file.rs: fn main() { let x = 1; /// incorrect documentation } ``` ```bash $ rustc file.rs file.rs:3:5: 3:27 error: found a documentation comment that doesn't document anything file.rs:3 /// incorrect documentation ^~~~~~~~~~~~~~~~~~~~~~ file.rs:3:5: 3:27 help: doc comments must come before what they document, maybe a comment was intended with `//`? ``` Fix #27429, #30322
2016-07-19Introduced `NoDelim` and modified the compiler to support it.cgswords-0/+2
2016-07-17Clean up and encapsulate `syntax::ext::mtwt`Jeffrey Seyfried-5/+0
2016-07-15Auto merge of #34570 - jseyfried:no_rename, r=nrcbors-30/+0
Simplify the macro hygiene algorithm This PR removes renaming from the hygiene algorithm and treats differently marked identifiers as unequal. This change makes the scope of identifiers in `macro_rules!` items empty. That is, identifiers in `macro_rules!` definitions do not inherit any semantics from the `macro_rules!`'s scope. Since `macro_rules!` macros are items, the scope of their identifiers "should" be the same as that of other items; in particular, the scope should contain only items. Since all items are unhygienic today, this would mean the scope should be empty. However, the scope of an identifier in a `macro_rules!` statement today is the scope that the identifier would have if it replaced the `macro_rules!` (excluding anything unhygienic, i.e. locals only). To continue to support this, this PR tracks the scope of each `macro_rules!` and uses it in `resolve` to ensure that an identifier expanded from a `macro_rules!` gets a chance to resolve to the locals in the `macro_rules!`'s scope. This PR is a pure refactoring. After this PR, - `syntax::ext::expand` is much simpler. - We can expand macros in any order without causing problems for hygiene (needed for macro modularization). - We can deprecate or remove today's `macro_rules!` scope easily. - Expansion performance improves by 25%, post-expansion memory usage decreases by ~5%. - Expanding a block is no longer quadratic in the number of `let` statements (fixes #10607). r? @nrc
2016-07-14Remove irrelevant testsJeffrey Seyfried-18/+0
2016-07-14Instead of renaming, treat differently marked identifiers as unequalJeffrey Seyfried-12/+0
2016-07-11Factor the `RefCell` out of the `Interner`.Jeffrey Seyfried-6/+7
2016-07-11Refactor `get_ident_interner` -> `with_ident_interner`.Jeffrey Seyfried-12/+13
2016-07-11Remove `Interner<T>` and rename `StrInterner` to `Interner`.Jeffrey Seyfried-3/+3
2016-07-11Encapsulate `RcStr` in `syntax::util::interner`.Jeffrey Seyfried-6/+5