https://github.com/rust-lang/rust http://git.dreamy.place/mirrors/rust/atom?h=1.60.0 2021-04-11T05:31:36+00:00 2021-04-11T05:31:36+00:00 Aaron Hill aa1ronham@gmail.com 2020-11-28T23:33:17+00:00 urn:sha1:a93c4f05de5c5cdd8158e23de7c3cf86a548447f This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run. 2021-03-01T03:41:21+00:00 Aaron Hill aa1ronham@gmail.com 2021-02-28T19:51:44+00:00 urn:sha1:8aed5b43342e25f6a9baa9cb472bb729fd150b79