Auto merge of #65324 - Centril:organize-syntax, r=petrochenkov

Split libsyntax apart

In this PR the general idea is to separate the AST, parser, and friends by a more data / logic structure (tho not fully realized!) by separating out the parser and macro expansion code from libsyntax. Specifically have now three crates instead of one (libsyntax):

- libsyntax:

   - concrete syntax tree (`syntax::ast`)

   - definition of tokens and token-streams (`syntax::{token, tokenstream}`) -- used by `syntax::ast`

   - visitors (`syntax::visit`, `syntax::mut_visit`)

   - shared definitions between `libsyntax_expand`

   - feature gating (`syntax::feature_gate`) -- we could possibly move this out to its own crater later.

   - attribute and meta item utilities, including used-marking (`syntax::attr`)

   - pretty printer (`syntax::print`) -- this should possibly be moved out later. For now I've reduced down the dependencies to a single essential one which could be broken via `ParseSess`. This entails that e.g. `Debug` impls for `Path` cannot reference the pretty printer.

   - definition of `ParseSess` (`syntax::sess`) -- this is used by `syntax::{attr, print, feature_gate}` and is a common definition used by the parser and other things like librustc.

   - the `syntax::source_map` -- this includes definitions used by `syntax::ast` and other things but could ostensibly be moved `syntax_pos` since that is more related to this module.

   - a smattering of misc utilities not sufficiently important to itemize -- some of these could be moved to where they are used (often a single place) but I wanted to limit the scope of this PR.

- librustc_parse:

   - parser (`rustc_parse::parser`) -- reading a file and such are defined in the crate root tho.

   - lexer (`rustc_parse::lexer`)

   - validation of meta grammar (post-expansion) in (`rustc_parse::validate_attr`)

- libsyntax_expand -- this defines the infra for macro expansion and conditional compilation but this is not libsyntax_ext; we might want to merge them later but currently libsyntax_expand is depended on by librustc_metadata which libsyntax_ext is not.

   - conditional compilation (`syntax_expand::config`) -- moved from `syntax::config` to here

   - the bulk of this crate is made up of the old `syntax::ext`

r? @estebank

1 files changed, 3 insertions, 20 deletions

diff --git a/src/libsyntax/feature_gate/check.rs b/src/libsyntax/feature_gate/check.rs
index ecff89ad59b..4742e01d7f4 100644
--- a/src/libsyntax/feature_gate/check.rs
+++ b/src/libsyntax/feature_gate/check.rs
@@ -3,18 +3,14 @@ use super::accepted::ACCEPTED_FEATURES;
 use super::removed::{REMOVED_FEATURES, STABLE_REMOVED_FEATURES};
 use super::builtin_attrs::{AttributeGate, BUILTIN_ATTRIBUTE_MAP};
 
-use crate::ast::{
-    self, AssocTyConstraint, AssocTyConstraintKind, NodeId, GenericParam, GenericParamKind,
-    PatKind, RangeEnd, VariantData,
-};
-use crate::attr::{self, check_builtin_attribute};
+use crate::ast::{self, AssocTyConstraint, AssocTyConstraintKind, NodeId};
+use crate::ast::{GenericParam, GenericParamKind, PatKind, RangeEnd, VariantData};
+use crate::attr;
 use crate::source_map::Spanned;
 use crate::edition::{ALL_EDITIONS, Edition};
 use crate::visit::{self, FnKind, Visitor};
-use crate::token;
 use crate::sess::ParseSess;
 use crate::symbol::{Symbol, sym};
-use crate::tokenstream::TokenTree;
 
 use errors::{Applicability, DiagnosticBuilder, Handler};
 use rustc_data_structures::fx::FxHashMap;
@@ -331,19 +327,6 @@ impl<'a> Visitor<'a> for PostExpansionVisitor<'a> {
         if let Some((.., AttributeGate::Gated(_, name, descr, has_feature))) = attr_info {
             gate_feature_fn!(self, has_feature, attr.span, name, descr, GateStrength::Hard);
         }
-        // Check input tokens for built-in and key-value attributes.
-        match attr_info {
-            // `rustc_dummy` doesn't have any restrictions specific to built-in attributes.
-            Some((name, _, template, _)) if name != sym::rustc_dummy =>
-                check_builtin_attribute(self.parse_sess, attr, name, template),
-            _ => if let Some(TokenTree::Token(token)) =
-                    attr.get_normal_item().tokens.trees().next() {
-                if token == token::Eq {
-                    // All key-value attributes are restricted to meta-item syntax.
-                    attr.parse_meta(self.parse_sess).map_err(|mut err| err.emit()).ok();
-                }
-            }
-        }
         // Check unstable flavors of the `#[doc]` attribute.
         if attr.check_name(sym::doc) {
             for nested_meta in attr.meta_item_list().unwrap_or_default() {