Lex binary and octal literals more eagerly.

Previously 0b12 was considered two tokens, 0b1 and 2, as 2 isn't a valid
base 2 digit. This patch changes that to collapse them into one (and
makes `0b12` etc. an error: 2 isn't a valid base 2 digit).

This may break some macro invocations of macros with `tt` (or syntax
extensions) that rely on adjacent digits being separate tokens and hence
is a

[breaking-change]

The fix is to separate the tokens, e.g. `0b12` -> `0b1 2`.

cc https://github.com/rust-lang/rfcs/pull/879

0 files changed, 0 insertions, 0 deletions