summary refs log tree commit diff
path: root/src/libsyntax/parse/lexer
AgeCommit message (Collapse)AuthorLines
2017-05-14Disallow underscore suffix for string-like literals.Masaki Hara-5/+1
2017-05-12Fix some clippy warnings in libsyntaxAndre Bogus-62/+62
This is mostly removing stray ampersands, needless returns and lifetimes.
2017-05-12Disallow ._ in float literal.Masaki Hara-3/+1
2017-05-08Allow bare CR in ////-style comment.Masaki Hara-1/+1
2017-05-04Remove use of `Self: Sized` from libsyntaxTommy Ip-1/+1
The bound is not required for compiling but it prevents using `next_token()` from a trait object. Fixes #33506.
2017-04-26Implement a file-path remapping feature in support of debuginfo and ↵Michael Woerister-18/+18
reproducible builds.
2017-03-29Refactor how spans are combined in the parser.Jeffrey Seyfried-21/+25
2017-03-18Parse 0e+10 as a valid floating-point literaltopecongiro-1/+1
Fixes issue #40408.
2017-03-03Integrate `TokenStream`.Jeffrey Seyfried-13/+11
2017-02-28Add warning cycle.Jeffrey Seyfried-0/+2
2017-02-28rustc_save_analysis: don't pollute the codemap with fake files.Eduard Burtescu-8/+38
2017-01-23Remove `open_span` and `close_span` from `Delimited`.Jeffrey Seyfried-5/+1
2017-01-17Fix fallout in `rustdoc`.Jeffrey Seyfried-3/+3
2017-01-17Refactor the parser to consume token trees.Jeffrey Seyfried-74/+0
2017-01-17Introduce `string_reader.parse_all_token_trees()`.Jeffrey Seyfried-8/+180
2017-01-17Give the `StringReader` a `sess: &ParseSess`.Jeffrey Seyfried-57/+50
2016-11-30Rename 'librustc_unicode' crate to 'libstd_unicode'.Corey Farwell-1/+1
Fixes #26554.
2016-11-21Fix fallout in `rustdoc` and tests.Jeffrey Seyfried-4/+5
2016-11-20Move `syntax::util::interner` -> `syntax::symbol`, cleanup.Jeffrey Seyfried-33/+34
2016-11-04Remove field `TtReader::next_tok`.Jeffrey Seyfried-2/+2
2016-11-03Move doc comment desugaring into the parser.Jeffrey Seyfried-1/+1
2016-11-03Avoid recontructing the `Parser` in `macro_parser.rs`.Jeffrey Seyfried-21/+0
2016-10-20Refactor parser lookahead buffer and increase its sizeVadim Petrochenkov-0/+6
2016-10-19Rollup merge of #37208 - jseyfried:fix_partially_consumed_tokens_in_macros, ↵Eduard-Mihai Burtescu-3/+24
r=nrc macros: fix partially consumed tokens in macro matchers Fixes #37175. This PR also avoids re-transcribing the tokens consumed by a matcher (and cloning the `TtReader` once per matcher), which improves expansion performance of the test case from #34630 by ~8%. r? @nrc
2016-10-18Fix some pretty printing testsVadim Petrochenkov-16/+30
2016-10-17Fix partially consumed tokens in macro matchers.Jeffrey Seyfried-3/+24
2016-10-05Clarify StringReader::bump.Nicholas Nethercote-13/+16
This commit renames the variables to make it clearer which char each one refers to. It also slightly reorders and rearranges some statements.
2016-10-05Rename StringReader::curr as ch.Nicholas Nethercote-99/+99
Likewise, rename StringReader::curr_is as ch_is. This is a [breaking-change] for libsyntax.
2016-10-05Rename StringReader::last_pos as pos.Nicholas Nethercote-100/+100
This is a [breaking-change] for libsyntax.
2016-10-05Rename StringReader::pos as next_pos.Nicholas Nethercote-18/+18
This is a [breaking-change] for libsyntax.
2016-10-03Simplify `start_bpos` calculation in scan_comment().Nicholas Nethercote-5/+1
The two branches of this `if` compute the same value. This commit gets rid of the first branch, which makes this calculation identical to the one in scan_block_comment().
2016-10-03Streamline StringReader::bump.Nicholas Nethercote-3/+1
First, assert! is redundant w.r.t. the unwrap() immediately afterwards. Second, `byte_offset_diff` is effectively computed as `current_byte_offset + ch.len_utf8() - current_byte_offset` (with `next` as an intermediate) which is silly and can be simplified.
2016-09-23reviewer comments and rebasingNick Cameron-23/+0
2016-09-22Adds a `ProcMacro` form of syntax extensionNick Cameron-6/+52
This commit adds syntax extension forms matching the types for procedural macros 2.0 (RFC #1566), these still require the usual syntax extension boiler plate, but this is a first step towards proper implementation and should be useful for macros 1.1 stuff too. Supports both attribute-like and function-like macros.
2016-08-07Fix old call in lexer testsJonathan Turner-3/+1
2016-07-14Remove BasicEmitterJonathan Turner-1/+1
2016-07-03prefer `if let` to match with `None => {}` arm in some placesZack M. Davis-8/+5
This is a spiritual succesor to #34268/8531d581, in which we replaced a number of matches of None to the unit value with `if let` conditionals where it was judged that this made for clearer/simpler code (as would be recommended by Manishearth/rust-clippy's `single_match` lint). The same rationale applies to matches of None to the empty block.
2016-06-23make old school mode a bit more configurableJonathan Turner-1/+4
2016-06-23Address more travis errorsJonathan Turner-1/+1
2016-06-23Move errors from libsyntax to its own crateJonathan Turner-25/+27
2016-06-16Add an abs_path member to FileMap, use it when writing debug info.Ted Mielczarek-2/+2
When items are inlined from extern crates, the filename in the debug info is taken from the FileMap that's serialized in the rlib metadata. Currently this is just FileMap.name, which is whatever path is passed to rustc. Since libcore and libstd are built by invoking rustc with relative paths, they wind up with relative paths in the rlib, and when linked into a binary the debug info uses relative paths for the names, but since the compilation directory for the final binary, tools trying to read source filenames will wind up with bad paths. We noticed this in Firefox with source filenames from libcore/libstd having bad paths. This change stores an absolute path in FileMap.abs_path, and uses that if available for writing debug info. This is not going to magically make debuggers able to find the source, but it will at least provide sensible paths.
2016-05-05Auto merge of #33128 - xen0n:more-confusing-unicode-chars, r=nagisabors-6/+53
Add more aliases for Unicode confusable chars Building upon #29837, this PR: * added aliases for space characters, * distinguished square brackets from parens, and * added common CJK punctuation characters as aliases. This will especially help CJK users who may have forgotten to switch off IME when coding.
2016-05-02lexer: do not display char confusingly in error messageGeorg Brandl-5/+4
Current code leads to messages like "... use a \xHH escape: \u{e4}" which is confusing. The printed span already points to the offending character, which should be enough to identify the non-ASCII problem. Fixes: #29088
2016-04-27Make some fatal lexer errors recoverablemitaa-66/+103
2016-04-24syntax: Check paths in visibilities for type parametersVadim Petrochenkov-10/+4
syntax: Merge PathParsingMode::NoTypesAllowed and PathParsingMode::ImportPrefix syntax: Rename PathParsingMode and its variants to better express their purpose syntax: Remove obsolete error message about 'self lifetime syntax: Remove ALLOW_MODULE_PATHS workaround syntax/resolve: Adjust some error messages resolve: Compare unhygienic (not renamed) names with keywords::Invalid, invalid identifiers may appear to be valid after renaming
2016-04-24syntax: Get rid of token::IdentStyleVadim Petrochenkov-25/+10
2016-04-21add more confusable CJK square bracket aliasesWang Xuerui-0/+12
2016-04-21correct aliases for square bracketsWang Xuerui-6/+8
2016-04-21add confusable space charactersWang Xuerui-0/+17
2016-04-21add more characters easily inputtable with CJK IMEsWang Xuerui-0/+16
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-07-13 02:47:18 +0000