summary refs log tree commit diff
path: root/src/libsyntax/parse/lexer/mod.rs
AgeCommit message (Collapse)AuthorLines
2016-08-07Fix old call in lexer testsJonathan Turner-3/+1
2016-07-14Remove BasicEmitterJonathan Turner-1/+1
2016-07-03prefer `if let` to match with `None => {}` arm in some placesZack M. Davis-8/+5
This is a spiritual succesor to #34268/8531d581, in which we replaced a number of matches of None to the unit value with `if let` conditionals where it was judged that this made for clearer/simpler code (as would be recommended by Manishearth/rust-clippy's `single_match` lint). The same rationale applies to matches of None to the empty block.
2016-06-23make old school mode a bit more configurableJonathan Turner-1/+4
2016-06-23Address more travis errorsJonathan Turner-1/+1
2016-06-23Move errors from libsyntax to its own crateJonathan Turner-23/+24
2016-06-16Add an abs_path member to FileMap, use it when writing debug info.Ted Mielczarek-1/+1
When items are inlined from extern crates, the filename in the debug info is taken from the FileMap that's serialized in the rlib metadata. Currently this is just FileMap.name, which is whatever path is passed to rustc. Since libcore and libstd are built by invoking rustc with relative paths, they wind up with relative paths in the rlib, and when linked into a binary the debug info uses relative paths for the names, but since the compilation directory for the final binary, tools trying to read source filenames will wind up with bad paths. We noticed this in Firefox with source filenames from libcore/libstd having bad paths. This change stores an absolute path in FileMap.abs_path, and uses that if available for writing debug info. This is not going to magically make debuggers able to find the source, but it will at least provide sensible paths.
2016-05-02lexer: do not display char confusingly in error messageGeorg Brandl-5/+4
Current code leads to messages like "... use a \xHH escape: \u{e4}" which is confusing. The printed span already points to the offending character, which should be enough to identify the non-ASCII problem. Fixes: #29088
2016-04-27Make some fatal lexer errors recoverablemitaa-66/+103
2016-04-24syntax: Check paths in visibilities for type parametersVadim Petrochenkov-10/+4
syntax: Merge PathParsingMode::NoTypesAllowed and PathParsingMode::ImportPrefix syntax: Rename PathParsingMode and its variants to better express their purpose syntax: Remove obsolete error message about 'self lifetime syntax: Remove ALLOW_MODULE_PATHS workaround syntax/resolve: Adjust some error messages resolve: Compare unhygienic (not renamed) names with keywords::Invalid, invalid identifiers may appear to be valid after renaming
2016-04-24syntax: Get rid of token::IdentStyleVadim Petrochenkov-25/+10
2016-03-07Auto merge of #29734 - Ryman:whitespace_consistency, r=Aatchbors-10/+10
libsyntax: be more accepting of whitespace in lexer Fixes #29590. Perhaps this may need more thorough testing? r? @Aatch
2016-01-16libsyntax: accept only whitespace with the PATTERN_WHITE_SPACE propertyKevin Butler-7/+10
This aligns with unicode recommendations and should be stable for all future unicode releases. See http://unicode.org/reports/tr31/#R3. This renames `libsyntax::lexer::is_whitespace` to `is_pattern_whitespace` so potentially breaks users of libsyntax.
2016-01-14libsyntax: use char::is_whitespace instead of custom implementationsKevin Butler-4/+1
Fixes #29590.
2016-01-14Display better snippet for invalid char literalGreg Chapple-10/+15
Given this code: fn main() { let _ = 'abcd'; } The compiler would give a message like: error: character literal may only contain one codepoint: '; let _ = 'abcd'; ^~ With this change, the message now displays: error: character literal may only contain one codepoint: 'abcd' let _ = 'abcd' ^~~~~~ Fixes #30033
2016-01-12re-instate comment that was mysteriously disappearedTshepang Lekhonkhobe-0/+1
2016-01-03fix "make tidy" failureTshepang Lekhonkhobe-1/+2
2016-01-03run rustfmt on syntax::parse::lexerTshepang Lekhonkhobe-462/+693
2015-12-30use structured errorsNick Cameron-21/+42
2015-12-17test errorsNick Cameron-27/+51
2015-12-17move error handling from libsyntax/diagnostics.rs to libsyntax/errors/*Nick Cameron-9/+8
Also split out emitters into their own module.
2015-11-17Detect confusing unicode characters and show the alternativeRavi Shankar-1/+3
2015-11-05Improve error message for char literalsSteve Klabnik-5/+6
If you try to put something that's bigger than a char into a char literal, you get an error: fn main() { let c = 'ஶ்ரீ'; } error: unterminated character constant: This is a very compiler-centric message. Yes, it's technically 'unterminated', but that's not what you, the user did wrong. Instead, this commit changes it to error: character literal may only contain one codepoint As this actually tells you what went wrong. Fixes #28851
2015-10-27Start pushing panics outward in lexer.Eli Friedman-33/+37
2015-10-10Prevent `/**/` from being parsed as a doc commentBarosl Lee-2/+3
Previously, `/**/` was incorrectly regarded as a doc comment because it starts with `/**` and ends with `*/`. However, this caused an ICE because some code assumed that the length of a doc comment is at least 5. This commit adds an additional check to `is_block_doc_comment` that tests the length of the input. Fixes #28844.
2015-09-03Use consistent terminology for byte string literalsVadim Petrochenkov-4/+4
Avoid confusion with binary integer literals and binary operator expressions in libsyntax
2015-07-29Replace illegal with invalid in most diagnosticsSimonas Kazlauskas-8/+8
2015-07-21Use a span from the correct file for the inner span of a moduleNick Cameron-0/+1
This basically only affects modules which are empty (or only contain comments). Closes #26755
2015-07-13Tell unicode escapes can’t be used as bytes earlier/moreSimonas Kazlauskas-16/+14
2015-07-10Improve some of the string escape diagnostic spansSimonas Kazlauskas-6/+4
2015-07-10Improve incomplete unicode escape reportingSimonas Kazlauskas-5/+19
This improves diagnostic messages when \u escape is used incorrectly and { is missing. Instead of saying “unknown character escape: u”, it will now report that unicode escape sequence is incomplete and suggest what the correct syntax is.
2015-07-01Make the unused_mut lint smarter with respect to locals.Ariel Ben-Yehuda-1/+1
Fixes #26332
2015-06-22fix minor indentation issuesYongqian Li-13/+13
2015-05-17Auto merge of #25387 - eddyb:syn-file-loader, r=nikomatsakisbors-2/+2
This allows compiling entire crates from memory or preprocessing source files before they are tokenized. Minor API refactoring included, which is a [breaking-change] for libsyntax users: * `ParseSess::{next_node_id, reserve_node_ids}` moved to rustc's `Session` * `new_parse_sess` -> `ParseSess::new` * `new_parse_sess_special_handler` -> `ParseSess::with_span_handler` * `mk_span_handler` -> `SpanHandler::new` * `default_handler` -> `Handler::new` * `mk_handler` -> `Handler::with_emitter` * `string_to_filemap(sess source, path)` -> `sess.codemap().new_filemap(path, source)`
2015-05-14Fix stupid mistake from previous commitLee Jeffery-2/+5
2015-05-14Make BytePos calculation same as originalLee Jeffery-1/+2
2015-05-14syntax: refactor (Span)Handler and ParseSess constructors to be methods.Eduard Burtescu-2/+2
2015-05-13Added test to check that newlines are stripped from commentsLee Jeffery-0/+9
2015-05-13Fix byte offset and error message inconsistenciesLee Jeffery-3/+3
2015-05-08Fix CRLF line-ending parsing for comments.Lee Jeffery-24/+26
2015-04-25Interpolate AST nodes in quasiquote.Geoffry Song-107/+0
This changes the `ToTokens` implementations for expressions, statements, etc. with almost-trivial ones that produce `Interpolated(*Nt(...))` pseudo-tokens. In this way, quasiquote now works the same way as macros do: already-parsed AST fragments are used as-is, not reparsed. The `ToSource` trait is removed. Quasiquote no longer involves pretty-printing at all, which removes the need for the `encode_with_hygiene` hack. All associated machinery is removed. A new `Nonterminal` is added, NtArm, which the parser now interpolates. This is just for quasiquote, not macros (although it could be in the future). `ToTokens` is no longer implemented for `Arg` (although this could be added again) and `Generics` (which I don't think makes sense). This breaks any compiler extensions that relied on the ability of `ToTokens` to turn AST fragments back into inspectable token trees. For this reason, this closes #16987. As such, this is a [breaking-change]. Fixes #16472. Fixes #15962. Fixes #17397. Fixes #16617.
2015-04-24Change name of unit test sub-module to "tests".Johannes Oertel-1/+1
Changes the style guidelines regarding unit tests to recommend using a sub-module named "tests" instead of "test" for unit tests as "test" might clash with imports of libtest.
2015-04-21syntax: Copy unstable str::char_at into libsyntaxErick Tryzelaar-9/+10
2015-04-21syntax: remove uses of `.into_cow()`Erick Tryzelaar-4/+4
2015-04-13Added a help span which informs the user about the escaping of curly braces ↵Manuel Hoffmann-1/+7
in a format string if a wrongly escaped one is detected in a string.
2015-04-05Work towards a non-panicing parser (libsyntax)Phil Dawes-2/+2
- Functions in parser.rs return PResult<> rather than panicing - Other functions in libsyntax call panic! explicitly for now if they rely on panicing behaviour. - 'panictry!' macro added as scaffolding while converting panicing functions. (This does the same as 'unwrap()' but is easier to grep for and turn into try!()) - Leaves panicing wrappers for the following functions so that the quote_* macros behave the same: - parse_expr, parse_item, parse_pat, parse_arm, parse_ty, parse_stmt
2015-03-31rollup merge of #23872: huonw/eager-lexingAlex Crichton-14/+26
Conflicts: src/libsyntax/parse/lexer/mod.rs
2015-03-31Stabilize std::numAaron Turon-4/+3
This commit stabilizes the `std::num` module: * The `Int` and `Float` traits are deprecated in favor of (1) the newly-added inherent methods and (2) the generic traits available in rust-lang/num. * The `Zero` and `One` traits are reintroduced in `std::num`, which together with various other traits allow you to recover the most common forms of generic programming. * The `FromStrRadix` trait, and associated free function, is deprecated in favor of inherent implementations. * A wide range of methods and constants for both integers and floating point numbers are now `#[stable]`, having been adjusted for integer guidelines. * `is_positive` and `is_negative` are renamed to `is_sign_positive` and `is_sign_negative`, in order to address #22985 * The `Wrapping` type is moved to `std::num` and stabilized; `WrappingOps` is deprecated in favor of inherent methods on the integer types, and direct implementation of operations on `Wrapping<X>` for each concrete integer type `X`. Closes #22985 Closes #21069 [breaking-change]
2015-03-31Lex binary and octal literals more eagerly.Huon Wilson-13/+25
Previously 0b12 was considered two tokens, 0b1 and 2, as 2 isn't a valid base 2 digit. This patch changes that to collapse them into one (and makes `0b12` etc. an error: 2 isn't a valid base 2 digit). This may break some macro invocations of macros with `tt` (or syntax extensions) that rely on adjacent digits being separate tokens and hence is a [breaking-change] The fix is to separate the tokens, e.g. `0b12` -> `0b1 2`. cc https://github.com/rust-lang/rfcs/pull/879
2015-03-28Rollup merge of #23803 - richo:unused-braces, r=ManishearthManish Goregaokar-1/+1
Pretty much what it says on the tin.