about summary refs log tree commit diff
path: root/src/libsyntax/parse/lexer/mod.rs
AgeCommit message (Collapse)AuthorLines
2016-01-14Display better snippet for invalid char literalGreg Chapple-10/+15
Given this code: fn main() { let _ = 'abcd'; } The compiler would give a message like: error: character literal may only contain one codepoint: '; let _ = 'abcd'; ^~ With this change, the message now displays: error: character literal may only contain one codepoint: 'abcd' let _ = 'abcd' ^~~~~~ Fixes #30033
2016-01-12re-instate comment that was mysteriously disappearedTshepang Lekhonkhobe-0/+1
2016-01-03fix "make tidy" failureTshepang Lekhonkhobe-1/+2
2016-01-03run rustfmt on syntax::parse::lexerTshepang Lekhonkhobe-462/+693
2015-12-30use structured errorsNick Cameron-21/+42
2015-12-17test errorsNick Cameron-27/+51
2015-12-17move error handling from libsyntax/diagnostics.rs to libsyntax/errors/*Nick Cameron-9/+8
Also split out emitters into their own module.
2015-11-17Detect confusing unicode characters and show the alternativeRavi Shankar-1/+3
2015-11-05Improve error message for char literalsSteve Klabnik-5/+6
If you try to put something that's bigger than a char into a char literal, you get an error: fn main() { let c = 'ஶ்ரீ'; } error: unterminated character constant: This is a very compiler-centric message. Yes, it's technically 'unterminated', but that's not what you, the user did wrong. Instead, this commit changes it to error: character literal may only contain one codepoint As this actually tells you what went wrong. Fixes #28851
2015-10-27Start pushing panics outward in lexer.Eli Friedman-33/+37
2015-10-10Prevent `/**/` from being parsed as a doc commentBarosl Lee-2/+3
Previously, `/**/` was incorrectly regarded as a doc comment because it starts with `/**` and ends with `*/`. However, this caused an ICE because some code assumed that the length of a doc comment is at least 5. This commit adds an additional check to `is_block_doc_comment` that tests the length of the input. Fixes #28844.
2015-09-03Use consistent terminology for byte string literalsVadim Petrochenkov-4/+4
Avoid confusion with binary integer literals and binary operator expressions in libsyntax
2015-07-29Replace illegal with invalid in most diagnosticsSimonas Kazlauskas-8/+8
2015-07-21Use a span from the correct file for the inner span of a moduleNick Cameron-0/+1
This basically only affects modules which are empty (or only contain comments). Closes #26755
2015-07-13Tell unicode escapes can’t be used as bytes earlier/moreSimonas Kazlauskas-16/+14
2015-07-10Improve some of the string escape diagnostic spansSimonas Kazlauskas-6/+4
2015-07-10Improve incomplete unicode escape reportingSimonas Kazlauskas-5/+19
This improves diagnostic messages when \u escape is used incorrectly and { is missing. Instead of saying “unknown character escape: u”, it will now report that unicode escape sequence is incomplete and suggest what the correct syntax is.
2015-07-01Make the unused_mut lint smarter with respect to locals.Ariel Ben-Yehuda-1/+1
Fixes #26332
2015-06-22fix minor indentation issuesYongqian Li-13/+13
2015-05-17Auto merge of #25387 - eddyb:syn-file-loader, r=nikomatsakisbors-2/+2
This allows compiling entire crates from memory or preprocessing source files before they are tokenized. Minor API refactoring included, which is a [breaking-change] for libsyntax users: * `ParseSess::{next_node_id, reserve_node_ids}` moved to rustc's `Session` * `new_parse_sess` -> `ParseSess::new` * `new_parse_sess_special_handler` -> `ParseSess::with_span_handler` * `mk_span_handler` -> `SpanHandler::new` * `default_handler` -> `Handler::new` * `mk_handler` -> `Handler::with_emitter` * `string_to_filemap(sess source, path)` -> `sess.codemap().new_filemap(path, source)`
2015-05-14Fix stupid mistake from previous commitLee Jeffery-2/+5
2015-05-14Make BytePos calculation same as originalLee Jeffery-1/+2
2015-05-14syntax: refactor (Span)Handler and ParseSess constructors to be methods.Eduard Burtescu-2/+2
2015-05-13Added test to check that newlines are stripped from commentsLee Jeffery-0/+9
2015-05-13Fix byte offset and error message inconsistenciesLee Jeffery-3/+3
2015-05-08Fix CRLF line-ending parsing for comments.Lee Jeffery-24/+26
2015-04-25Interpolate AST nodes in quasiquote.Geoffry Song-107/+0
This changes the `ToTokens` implementations for expressions, statements, etc. with almost-trivial ones that produce `Interpolated(*Nt(...))` pseudo-tokens. In this way, quasiquote now works the same way as macros do: already-parsed AST fragments are used as-is, not reparsed. The `ToSource` trait is removed. Quasiquote no longer involves pretty-printing at all, which removes the need for the `encode_with_hygiene` hack. All associated machinery is removed. A new `Nonterminal` is added, NtArm, which the parser now interpolates. This is just for quasiquote, not macros (although it could be in the future). `ToTokens` is no longer implemented for `Arg` (although this could be added again) and `Generics` (which I don't think makes sense). This breaks any compiler extensions that relied on the ability of `ToTokens` to turn AST fragments back into inspectable token trees. For this reason, this closes #16987. As such, this is a [breaking-change]. Fixes #16472. Fixes #15962. Fixes #17397. Fixes #16617.
2015-04-24Change name of unit test sub-module to "tests".Johannes Oertel-1/+1
Changes the style guidelines regarding unit tests to recommend using a sub-module named "tests" instead of "test" for unit tests as "test" might clash with imports of libtest.
2015-04-21syntax: Copy unstable str::char_at into libsyntaxErick Tryzelaar-9/+10
2015-04-21syntax: remove uses of `.into_cow()`Erick Tryzelaar-4/+4
2015-04-13Added a help span which informs the user about the escaping of curly braces ↵Manuel Hoffmann-1/+7
in a format string if a wrongly escaped one is detected in a string.
2015-04-05Work towards a non-panicing parser (libsyntax)Phil Dawes-2/+2
- Functions in parser.rs return PResult<> rather than panicing - Other functions in libsyntax call panic! explicitly for now if they rely on panicing behaviour. - 'panictry!' macro added as scaffolding while converting panicing functions. (This does the same as 'unwrap()' but is easier to grep for and turn into try!()) - Leaves panicing wrappers for the following functions so that the quote_* macros behave the same: - parse_expr, parse_item, parse_pat, parse_arm, parse_ty, parse_stmt
2015-03-31rollup merge of #23872: huonw/eager-lexingAlex Crichton-14/+26
Conflicts: src/libsyntax/parse/lexer/mod.rs
2015-03-31Stabilize std::numAaron Turon-4/+3
This commit stabilizes the `std::num` module: * The `Int` and `Float` traits are deprecated in favor of (1) the newly-added inherent methods and (2) the generic traits available in rust-lang/num. * The `Zero` and `One` traits are reintroduced in `std::num`, which together with various other traits allow you to recover the most common forms of generic programming. * The `FromStrRadix` trait, and associated free function, is deprecated in favor of inherent implementations. * A wide range of methods and constants for both integers and floating point numbers are now `#[stable]`, having been adjusted for integer guidelines. * `is_positive` and `is_negative` are renamed to `is_sign_positive` and `is_sign_negative`, in order to address #22985 * The `Wrapping` type is moved to `std::num` and stabilized; `WrappingOps` is deprecated in favor of inherent methods on the integer types, and direct implementation of operations on `Wrapping<X>` for each concrete integer type `X`. Closes #22985 Closes #21069 [breaking-change]
2015-03-31Lex binary and octal literals more eagerly.Huon Wilson-13/+25
Previously 0b12 was considered two tokens, 0b1 and 2, as 2 isn't a valid base 2 digit. This patch changes that to collapse them into one (and makes `0b12` etc. an error: 2 isn't a valid base 2 digit). This may break some macro invocations of macros with `tt` (or syntax extensions) that rely on adjacent digits being separate tokens and hence is a [breaking-change] The fix is to separate the tokens, e.g. `0b12` -> `0b1 2`. cc https://github.com/rust-lang/rfcs/pull/879
2015-03-28Rollup merge of #23803 - richo:unused-braces, r=ManishearthManish Goregaokar-1/+1
Pretty much what it says on the tin.
2015-03-28cleanup: Remove unused braces in use statementsRicho Healey-1/+1
2015-03-27Prevent ICEs when parsing invalid escapes, closes #23620Florian Hahn-11/+29
2015-03-25Add trivial cast lints.Nick Cameron-2/+2
This permits all coercions to be performed in casts, but adds lints to warn in those cases. Part of this patch moves cast checking to a later stage of type checking. We acquire obligations to check casts as part of type checking where we previously checked them. Once we have type checked a function or module, then we check any cast obligations which have been acquired. That means we have more type information available to check casts (this was crucial to making coercions work properly in place of some casts), but it means that casts cannot feed input into type inference. [breaking change] * Adds two new lints for trivial casts and trivial numeric casts, these are warn by default, but can cause errors if you build with warnings as errors. Previously, trivial numeric casts and casts to trait objects were allowed. * The unused casts lint has gone. * Interactions between casting and type inference have changed in subtle ways. Two ways this might manifest are: - You may need to 'direct' casts more with extra type information, for example, in some cases where `foo as _ as T` succeeded, you may now need to specify the type for `_` - Casts do not influence inference of integer types. E.g., the following used to type check: ``` let x = 42; let y = &x as *const u32; ``` Because the cast would inform inference that `x` must have type `u32`. This no longer applies and the compiler will fallback to `i32` for `x` and thus there will be a type error in the cast. The solution is to add more type information: ``` let x: u32 = 42; let y = &x as *const u32; ```
2015-03-20don't use Result::ok just to be able to use unwrap/unwrap_orOliver Schneider-2/+2
2015-03-17std: Tweak some unstable features of `str`Alex Crichton-7/+9
This commit clarifies some of the unstable features in the `str` module by moving them out of the blanket `core` and `collections` features. The following methods were moved to the `str_char` feature which generally encompasses decoding specific characters from a `str` and dealing with the result. It is unclear if any of these methods need to be stabilized for 1.0 and the most conservative route for now is to continue providing them but to leave them as unstable under a more specific name. * `is_char_boundary` * `char_at` * `char_range_at` * `char_at_reverse` * `char_range_at_reverse` * `slice_shift_char` The following methods were moved into the generic `unicode` feature as they are specifically enabled by the `unicode` crate itself. * `nfd_chars` * `nfkd_chars` * `nfc_chars` * `graphemes` * `grapheme_indices` * `width`
2015-03-13Fallout of std::old_io deprecationAlex Crichton-2/+2
2015-03-06syntax: Remove deprecated unicode escapesAlex Crichton-22/+2
These have been deprecated for quite some time, so we should be good to remove them now.
2015-03-05Rollup merge of #22764 - ivanradanov:fileline_help, r=huonwManish Goregaokar-1/+1
When warnings and errors occur, the associated help message should not print the same code snippet. https://github.com/rust-lang/rust/issues/21938
2015-03-04Encode codemap and span information in crate metadata.Michael Woerister-12/+21
This allows to create proper debuginfo line information for items inlined from other crates (e.g. instantiations of generics). Only the codemap's 'metadata' is stored in a crate's metadata. That is, just filename, line-beginnings, etc. but not the actual source code itself. We are thus missing the opportunity of making Rust the first "open-source-only" programming language out there. Pity.
2015-03-03Switched to Box::new in many places.Felix S. Klock II-2/+3
Many of the modifications putting in `Box::new` calls also include a pointer to Issue 22405, which tracks going back to `box <expr>` if possible in the future. (Still tried to use `Box<_>` where it sufficed; thus some tests still have `box_syntax` enabled, as they use a mix of `box` and `Box::new`.) Precursor for overloaded-`box` and placement-`in`; see Issue 22181.
2015-03-03Change span_help calls to fileline_help where appropriateIvan Radanov Ivanov-1/+1
2015-02-20Remove remaining uses of `[]`. This time I tried to use deref coercions ↵Niko Matsakis-1/+1
where possible.
2015-02-18Round 3 test fixes and conflictsAlex Crichton-3/+2
2015-02-18Replace all uses of `&foo[]` with `&foo[..]` en masse.Niko Matsakis-4/+4