about summary refs log tree commit diff
path: root/src/libsyntax/parse/token.rs
AgeCommit message (Collapse)AuthorLines
2016-10-20Tweak path parsing logicVadim Petrochenkov-4/+7
2016-10-19Improve `$crate`.Jeffrey Seyfried-17/+0
2016-10-15Auto merge of #37132 - petrochenkov:intern, r=alexcrichtonbors-10/+3
Get rid of double indirection in string interner
2016-10-13Get rid of double indirection in string interner by using `Rc<str>`Vadim Petrochenkov-10/+3
2016-10-10Avoid allocations in `Decoder::read_str`.Nicholas Nethercote-1/+1
`opaque::Decoder::read_str` is very hot within `rustc` due to its use in the reading of crate metadata, and it currently returns a `String`. This commit changes it to instead return a `Cow<str>`, which avoids a heap allocation. This change reduces the number of calls to `malloc` by almost 10% in some benchmarks. This is a [breaking-change] to libserialize.
2016-09-14Remove unused Token::to_binop function.Nicholas Nethercote-26/+1
2016-09-04Replace `_, _` with `..`Vadim Petrochenkov-3/+3
2016-08-19Auto merge of #33922 - estebank:doc-comment, r=alexcrichtonbors-1/+9
Specific error message for missplaced doc comments Identify when documetation comments have been missplaced in the following places: * After a struct element: ```rust // file.rs: struct X { a: u8 /** document a */, } ``` ```bash $ rustc file.rs file.rs:2:11: 2:28 error: found documentation comment that doesn't document anything file.rs:2 a: u8 /** document a */, ^~~~~~~~~~~~~~~~~ file.rs:2:11: 2:28 help: doc comments must come before what they document, maybe a comment was intended with `//`? ``` * As the last line of a struct: ```rust // file.rs: struct X { a: u8, /// incorrect documentation } ``` ```bash $ rustc file.rs file.rs:3:5: 3:27 error: found a documentation comment that doesn't document anything file.rs:3 /// incorrect documentation ^~~~~~~~~~~~~~~~~~~~~~ file.rs:3:5: 3:27 help: doc comments must come before what they document, maybe a comment was intended with `//`? ``` * As the last line of a `fn`: ```rust // file.rs: fn main() { let x = 1; /// incorrect documentation } ``` ```bash $ rustc file.rs file.rs:3:5: 3:27 error: found a documentation comment that doesn't document anything file.rs:3 /// incorrect documentation ^~~~~~~~~~~~~~~~~~~~~~ file.rs:3:5: 3:27 help: doc comments must come before what they document, maybe a comment was intended with `//`? ``` Fix #27429, #30322
2016-07-19Introduced `NoDelim` and modified the compiler to support it.cgswords-0/+2
2016-07-17Clean up and encapsulate `syntax::ext::mtwt`Jeffrey Seyfried-5/+0
2016-07-15Auto merge of #34570 - jseyfried:no_rename, r=nrcbors-30/+0
Simplify the macro hygiene algorithm This PR removes renaming from the hygiene algorithm and treats differently marked identifiers as unequal. This change makes the scope of identifiers in `macro_rules!` items empty. That is, identifiers in `macro_rules!` definitions do not inherit any semantics from the `macro_rules!`'s scope. Since `macro_rules!` macros are items, the scope of their identifiers "should" be the same as that of other items; in particular, the scope should contain only items. Since all items are unhygienic today, this would mean the scope should be empty. However, the scope of an identifier in a `macro_rules!` statement today is the scope that the identifier would have if it replaced the `macro_rules!` (excluding anything unhygienic, i.e. locals only). To continue to support this, this PR tracks the scope of each `macro_rules!` and uses it in `resolve` to ensure that an identifier expanded from a `macro_rules!` gets a chance to resolve to the locals in the `macro_rules!`'s scope. This PR is a pure refactoring. After this PR, - `syntax::ext::expand` is much simpler. - We can expand macros in any order without causing problems for hygiene (needed for macro modularization). - We can deprecate or remove today's `macro_rules!` scope easily. - Expansion performance improves by 25%, post-expansion memory usage decreases by ~5%. - Expanding a block is no longer quadratic in the number of `let` statements (fixes #10607). r? @nrc
2016-07-14Remove irrelevant testsJeffrey Seyfried-18/+0
2016-07-14Instead of renaming, treat differently marked identifiers as unequalJeffrey Seyfried-12/+0
2016-07-11Factor the `RefCell` out of the `Interner`.Jeffrey Seyfried-6/+7
2016-07-11Refactor `get_ident_interner` -> `with_ident_interner`.Jeffrey Seyfried-12/+13
2016-07-11Remove `Interner<T>` and rename `StrInterner` to `Interner`.Jeffrey Seyfried-3/+3
2016-07-11Encapsulate `RcStr` in `syntax::util::interner`.Jeffrey Seyfried-6/+5
2016-07-05Specific error message for missplaced doc commentsEsteban Küber-1/+9
Identify when documetation comments have been missplaced in the following places: * After a struct element: ```rust // file.rs: struct X { a: u8 /** document a */, } ``` ```bash $ rustc file.rs file.rs:2:11: 2:28 error: found documentation comment that doesn't document anything file.rs:2 a: u8 /** document a */, ^~~~~~~~~~~~~~~~~ file.rs:2:11: 2:28 help: doc comments must come before what they document, maybe a comment was intended with `//`? ``` * As the last line of a struct: ```rust // file.rs: struct X { a: u8, /// incorrect documentation } ``` ```bash $ rustc file.rs file.rs:3:5: 3:27 error: found a documentation comment that doesn't document anything file.rs:3 /// incorrect documentation ^~~~~~~~~~~~~~~~~~~~~~ file.rs:3:5: 3:27 help: doc comments must come before what they document, maybe a comment was intended with `//`? ``` * As the last line of a `fn`: ```rust // file.rs: fn main() { let x = 1; /// incorrect documentation } ``` ```bash $ rustc file.rs file.rs:3:5: 3:27 error: found a documentation comment that doesn't document anything file.rs:3 /// incorrect documentation ^~~~~~~~~~~~~~~~~~~~~~ file.rs:3:5: 3:27 help: doc comments must come before what they document, maybe a comment was intended with `//`? ``` Fix #27429, #30322
2016-06-21Refactored tokentrees into their own files in preparation for tokenstreams. ↵cgswords-1/+2
Modified tests to point to the new file now.
2016-04-26allow InternedString to be compared to &str directlyOliver Schneider-0/+22
2016-04-24syntax: Make `is_path_start` precise and improve some error messages about ↵Vadim Petrochenkov-0/+5
unexpected tokens
2016-04-24syntax: Merge keywords and remaining special idents in one listVadim Petrochenkov-137/+89
Simplify the macro used for generation of keywords Make `Keyword::ident` private
2016-04-24syntax: Make static/super/self/Self keywords + special ident cleanupVadim Petrochenkov-105/+54
2016-04-24syntax: Get rid of token::IdentStyleVadim Petrochenkov-32/+24
2016-04-24syntax: Don't rely on token::IdentStyle in the parserVadim Petrochenkov-62/+26
2016-04-05Fixes bug which accepting using `super` in use statemet.vlastachu-1/+1
Issue: #32225
2016-03-22try! -> ?Jorge Aparicio-1/+1
Automated conversion using the untry tool [1] and the following command: ``` $ find -name '*.rs' -type f | xargs untry ``` at the root of the Rust repo. [1]: https://github.com/japaric/untry
2016-03-14Add `default` as contextual keyword, and parse it for impl items.Aaron Turon-48/+49
2016-03-06Auto merge of #30884 - durka:inclusive-ranges, r=aturonbors-1/+1
This PR implements [RFC 1192](https://github.com/rust-lang/rfcs/blob/master/text/1192-inclusive-ranges.md), which is triple-dot syntax for inclusive range expressions. The new stuff is behind two feature gates (one for the syntax and one for the std::ops types). This replaces the deprecated functionality in std::iter. Along the way I simplified the desugaring for all ranges. This is my first contribution to rust which changes more than one character outside of a test or comment, so please review carefully! Some of the individual commit messages have more of my notes. Also thanks for putting up with my dumb questions in #rust-internals. - For implementing `std::ops::RangeInclusive`, I took @Stebalien's suggestion from https://github.com/rust-lang/rfcs/pull/1192#issuecomment-137864421. It seemed to me to make the implementation easier and increase type safety. If that stands, the RFC should be amended to avoid confusion. - I also kind of like @glaebhoerl's [idea](https://github.com/rust-lang/rfcs/pull/1254#issuecomment-147815299), which is unified inclusive/exclusive range syntax something like `x>..=y`. We can experiment with this while everything is behind a feature gate. - There are a couple of FIXMEs left (see the last commit). I didn't know what to do about `RangeArgument` and I haven't added `Index` impls yet. Those should be discussed/finished before merging. cc @Gankro since you [complained](https://www.reddit.com/r/rust/comments/3xkfro/what_happened_to_inclusive_ranges/cy5j0yq) cc #27777 #30877 rust-lang/rust#1192 rust-lang/rfcs#1254 relevant to #28237 (tracking issue)
2016-03-02Use numeric field `Name`s ("0", "1" etc) for positional fieldsVadim Petrochenkov-1/+1
2016-02-27libsyntax: parse inclusive rangesAlex Burka-1/+1
2016-02-12Use more autoderef in libsyntaxJonas Schievink-1/+1
2016-02-11[breaking-change] don't glob export ast::BinOp_Oliver Schneider-20/+20
2016-01-26Use interpolated token span when building spans for bigger expressionsFlorian Hahn-0/+8
2016-01-06Remove irrelevant commentHuon Wilson-3/+0
The fundamental problem of duplication was fixed in https://github.com/rust-lang/rust/pull/10891, but the comment was preserved. Closes https://github.com/rust-lang/rust/issues/9762.
2015-12-15Generated code spans now point to callsite parameters (where applicable)Daniel Campbell-1/+1
2015-11-26Add syntax support for attributes on expressions and all syntaxMarvin Löbel-0/+1
nodes in statement position. Extended #[cfg] folder to allow removal of statements, and of expressions in optional positions like expression lists and trailing block expressions. Extended lint checker to recognize lint levels on expressions and locals.
2015-11-11libsyntax: Add more quasiquoting macrosKyle Mayes-1/+3
2015-09-24Cleanup interfaces of Name, SyntaxContext and IdentVadim Petrochenkov-7/+7
Make sure Name, SyntaxContext and Ident are passed by value Make sure Idents don't serve as keys (or parts of keys) in maps, Ident comparison is not well defined
2015-09-11Remove some remains of virtual structs from the parserVadim Petrochenkov-6/+6
2015-09-03Use consistent terminology for byte string literalsVadim Petrochenkov-4/+4
Avoid confusion with binary integer literals and binary operator expressions in libsyntax
2015-07-28remove `get_ident` and `get_name`, make `as_str` soundOliver Schneider-17/+9
2015-05-15syntax: Add unquoting ast::{Generics,WhereClause}Erick Tryzelaar-0/+4
2015-05-09Squeeze the last bits of `task`s in documentation in favor of `thread`Barosl Lee-8/+8
An automated script was run against the `.rs` and `.md` files, subsituting every occurrence of `task` with `thread`. In the `.rs` files, only the texts in the comment blocks were affected.
2015-04-25Interpolate AST nodes in quasiquote.Geoffry Song-0/+7
This changes the `ToTokens` implementations for expressions, statements, etc. with almost-trivial ones that produce `Interpolated(*Nt(...))` pseudo-tokens. In this way, quasiquote now works the same way as macros do: already-parsed AST fragments are used as-is, not reparsed. The `ToSource` trait is removed. Quasiquote no longer involves pretty-printing at all, which removes the need for the `encode_with_hygiene` hack. All associated machinery is removed. A new `Nonterminal` is added, NtArm, which the parser now interpolates. This is just for quasiquote, not macros (although it could be in the future). `ToTokens` is no longer implemented for `Arg` (although this could be added again) and `Generics` (which I don't think makes sense). This breaks any compiler extensions that relied on the ability of `ToTokens` to turn AST fragments back into inspectable token trees. For this reason, this closes #16987. As such, this is a [breaking-change]. Fixes #16472. Fixes #15962. Fixes #17397. Fixes #16617.
2015-04-25Auto merge of #24547 - bombless:comma, r=pnkfelixbors-0/+8
Closes #20616 It breaks code such as <https://github.com/rust-lang/rust/blob/c64feb63418fd05bd6e5adc6f9ad763aa6a594b1/src/librustc_typeck/check/method/suggest.rs#L367>, so this is a [breaking-change], you have to add missing comma after the last lifetime arguement now.
2015-04-25Fix #20616York Xiang-0/+8
2015-04-24Change name of unit test sub-module to "tests".Johannes Oertel-1/+1
Changes the style guidelines regarding unit tests to recommend using a sub-module named "tests" instead of "test" for unit tests as "test" might clash with imports of libtest.
2015-04-14std: Remove old_io/old_path/rand modulesAlex Crichton-16/+0
This commit entirely removes the old I/O, path, and rand modules. All functionality has been deprecated and unstable for quite some time now!
2015-03-13Auto merge of #23229 - aturon:stab-path, r=alexcrichtonbors-0/+2
This commit stabilizes essentially all of the new `std::path` API. The API itself is changed in a couple of ways (which brings it in closer alignment with the RFC): * `.` components are now normalized away, unless they appear at the start of a path. This in turn effects the semantics of e.g. asking for the file name of `foo/` or `foo/.`, both of which yield `Some("foo")` now. This semantics is what the original RFC specified, and is also desirable given early experience rolling out the new API. * The `parent` method is now `without_file` and succeeds if, and only if, `file_name` is `Some(_)`. That means, in particular, that it fails for a path like `foo/../`. This change affects `pop` as well. In addition, the `old_path` module is now deprecated. [breaking-change] r? @alexcrichton