about summary refs log tree commit diff
path: root/src/doc/rustc-dev-guide
diff options
context:
space:
mode:
authorMark Mansi <markm@cs.wisc.edu>2018-03-26 10:46:39 -0500
committerWho? Me?! <mark-i-m@users.noreply.github.com>2018-03-26 20:28:00 -0500
commitae62bb6cf46386e5c593ecf51557d154c22761c6 (patch)
treed910c5495f0a6f0272e51d3db63fd51b475f6433 /src/doc/rustc-dev-guide
parent8901200fab817a0c8ae97df55801123c33dea41e (diff)
downloadrust-ae62bb6cf46386e5c593ecf51557d154c22761c6.tar.gz
rust-ae62bb6cf46386e5c593ecf51557d154c22761c6.zip
reformat basically the whole rustdoc.md
Diffstat (limited to 'src/doc/rustc-dev-guide')
-rw-r--r--src/doc/rustc-dev-guide/src/rustdoc.md293
1 files changed, 162 insertions, 131 deletions
diff --git a/src/doc/rustc-dev-guide/src/rustdoc.md b/src/doc/rustc-dev-guide/src/rustdoc.md
index ae640d11070..7a9905d3b11 100644
--- a/src/doc/rustc-dev-guide/src/rustdoc.md
+++ b/src/doc/rustc-dev-guide/src/rustdoc.md
@@ -1,7 +1,8 @@
 # The walking tour of rustdoc
 
-Rustdoc actually uses the rustc internals directly. It lives in-tree with the compiler and standard
-library. This chapter is about how it works. (A new implementation is also [under way], though).
+Rustdoc actually uses the rustc internals directly. It lives in-tree with the
+compiler and standard library. This chapter is about how it works. (A new
+implementation is also [under way], though).
 
 [under way]: https://github.com/steveklabnik/rustdoc
 
@@ -17,50 +18,57 @@ and [queries] are discussed in the linked chapters.
 `librustdoc` performs two major steps after that to render a set of
 documentation:
 
-* "Clean" the AST into a form that's more suited to creating documentation (and slightly more
-  resistant to churn in the compiler).
+* "Clean" the AST into a form that's more suited to creating documentation (and
+  slightly more resistant to churn in the compiler).
 * Use this cleaned AST to render a crate's documentation, one page at a time.
 
-Naturally, there's more than just this, and those descriptions simplify out lots of details, but
-that's the high-level overview.
+Naturally, there's more than just this, and those descriptions simplify out
+lots of details, but that's the high-level overview.
 
-(Side note: `librustdoc` is a library crate! The `rustdoc` binary is crated using the project in
-[`src/tools/rustdoc`][bin]. Note that literally all that does is call the `main()` that's in this crate's
-`lib.rs`, though.)
+(Side note: `librustdoc` is a library crate! The `rustdoc` binary is crated
+using the project in [`src/tools/rustdoc`][bin]. Note that literally all that
+does is call the `main()` that's in this crate's `lib.rs`, though.)
 
 [bin]: https://github.com/rust-lang/rust/tree/master/src/tools/rustdoc
 
 ## Cheat sheet
 
-* Use `x.py build --stage 1 src/libstd src/tools/rustdoc` to make a useable rustdoc you can run on
-  other projects.
+* Use `x.py build --stage 1 src/libstd src/tools/rustdoc` to make a useable
+  rustdoc you can run on other projects.
   * Add `src/libtest` to be able to use `rustdoc --test`.
-  * If you've used `rustup toolchain link local /path/to/build/$TARGET/stage1` previously, then
-    after the previous build command, `cargo +local doc` will Just Work.
-* Use `x.py doc --stage 1 src/libstd` to use this rustdoc to generate the standard library docs.
-  * The completed docs will be available in `build/$TARGET/doc/std`, though the bundle is meant to
-    be used as though you would copy out the `doc` folder to a web server, since that's where the
-    CSS/JS and landing page are.
-* Most of the HTML printing code is in `html/format.rs` and `html/render.rs`. It's in a bunch of
-  `fmt::Display` implementations and supplementary functions.
-* The types that got `Display` impls above are defined in `clean/mod.rs`, right next to the custom
-  `Clean` trait used to process them out of the rustc HIR.
+  * If you've used `rustup toolchain link local /path/to/build/$TARGET/stage1`
+    previously, then after the previous build command, `cargo +local doc` will
+    Just Work.
+* Use `x.py doc --stage 1 src/libstd` to use this rustdoc to generate the
+  standard library docs.
+  * The completed docs will be available in `build/$TARGET/doc/std`, though the
+    bundle is meant to be used as though you would copy out the `doc` folder to
+    a web server, since that's where the CSS/JS and landing page are.
+* Most of the HTML printing code is in `html/format.rs` and `html/render.rs`.
+  It's in a bunch of `fmt::Display` implementations and supplementary
+  functions.
+* The types that got `Display` impls above are defined in `clean/mod.rs`, right
+  next to the custom `Clean` trait used to process them out of the rustc HIR.
 * The bits specific to using rustdoc as a test harness are in `test.rs`.
-* The Markdown renderer is loaded up in `html/markdown.rs`, including functions for extracting
-  doctests from a given block of Markdown.
-* The tests on rustdoc *output* are located in `src/test/rustdoc`, where they're handled by the test
-  runner of rustbuild and the supplementary script `src/etc/htmldocck.py`.
-* Tests on search index generation are located in `src/test/rustdoc-js`, as a series of JavaScript
-  files that encode queries on the standard library search index and expected results.
+* The Markdown renderer is loaded up in `html/markdown.rs`, including functions
+  for extracting doctests from a given block of Markdown.
+* The tests on rustdoc *output* are located in `src/test/rustdoc`, where
+  they're handled by the test runner of rustbuild and the supplementary script
+  `src/etc/htmldocck.py`.
+* Tests on search index generation are located in `src/test/rustdoc-js`, as a
+  series of JavaScript files that encode queries on the standard library search
+  index and expected results.
 
 ## From crate to clean
 
-In `core.rs` are two central items: the `DocContext` struct, and the `run_core` function. The latter
-is where rustdoc calls out to rustc to compile a crate to the point where rustdoc can take over. The
-former is a state container used when crawling through a crate to gather its documentation.
+In `core.rs` are two central items: the `DocContext` struct, and the `run_core`
+function. The latter is where rustdoc calls out to rustc to compile a crate to
+the point where rustdoc can take over. The former is a state container used
+when crawling through a crate to gather its documentation.
 
-The main process of crate crawling is done in `clean/mod.rs` through several implementations of the
-`Clean` trait defined within. This is a conversion trait, which defines one method:
+The main process of crate crawling is done in `clean/mod.rs` through several
+implementations of the `Clean` trait defined within. This is a conversion
+trait, which defines one method:
 
 ```rust
 pub trait Clean<T> {
@@ -68,40 +76,47 @@ pub trait Clean<T> {
 }
 ```
 
-`clean/mod.rs` also defines the types for the "cleaned" AST used later on to render documentation
-pages. Each usually accompanies an implementation of `Clean` that takes some AST or HIR type from
-rustc and converts it into the appropriate "cleaned" type. "Big" items like modules or associated
-items may have some extra processing in its `Clean` implementation, but for the most part these
-impls are straightforward conversions. The "entry point" to this module is the `impl Clean<Crate>
-for visit_ast::RustdocVisitor`, which is called by `run_core` above.
-
-You see, I actually lied a little earlier: There's another AST transformation that happens before
-the events in `clean/mod.rs`.  In `visit_ast.rs` is the type `RustdocVisitor`, which *actually*
-crawls a `hir::Crate` to get the first intermediate representation, defined in `doctree.rs`. This
-pass is mainly to get a few intermediate wrappers around the HIR types and to process visibility
-and inlining. This is where `#[doc(inline)]`, `#[doc(no_inline)]`, and `#[doc(hidden)]` are
-processed, as well as the logic for whether a `pub use` should get the full page or a "Reexport"
-line in the module page.
-
-The other major thing that happens in `clean/mod.rs` is the collection of doc comments and
-`#[doc=""]` attributes into a separate field of the Attributes struct, present on anything that gets
-hand-written documentation. This makes it easier to collect this documentation later in the process.
-
-The primary output of this process is a `clean::Crate` with a tree of Items which describe the
-publicly-documentable items in the target crate.
+`clean/mod.rs` also defines the types for the "cleaned" AST used later on to
+render documentation pages. Each usually accompanies an implementation of
+`Clean` that takes some AST or HIR type from rustc and converts it into the
+appropriate "cleaned" type. "Big" items like modules or associated items may
+have some extra processing in its `Clean` implementation, but for the most part
+these impls are straightforward conversions. The "entry point" to this module
+is the `impl Clean<Crate> for visit_ast::RustdocVisitor`, which is called by
+`run_core` above.
+
+You see, I actually lied a little earlier: There's another AST transformation
+that happens before the events in `clean/mod.rs`.  In `visit_ast.rs` is the
+type `RustdocVisitor`, which *actually* crawls a `hir::Crate` to get the first
+intermediate representation, defined in `doctree.rs`. This pass is mainly to
+get a few intermediate wrappers around the HIR types and to process visibility
+and inlining. This is where `#[doc(inline)]`, `#[doc(no_inline)]`, and
+`#[doc(hidden)]` are processed, as well as the logic for whether a `pub use`
+should get the full page or a "Reexport" line in the module page.
+
+The other major thing that happens in `clean/mod.rs` is the collection of doc
+comments and `#[doc=""]` attributes into a separate field of the Attributes
+struct, present on anything that gets hand-written documentation. This makes it
+easier to collect this documentation later in the process.
+
+The primary output of this process is a `clean::Crate` with a tree of Items
+which describe the publicly-documentable items in the target crate.
 
 ### Hot potato
 
-Before moving on to the next major step, a few important "passes" occur over the documentation.
-These do things like combine the separate "attributes" into a single string and strip leading
-whitespace to make the document easier on the markdown parser, or drop items that are not public or
-deliberately hidden with `#[doc(hidden)]`. These are all implemented in the `passes/` directory, one
-file per pass. By default, all of these passes are run on a crate, but the ones regarding dropping
-private/hidden items can be bypassed by passing `--document-private-items` to rustdoc. Note that
-unlike the previous set of AST transformations, the passes happen on the _cleaned_ crate.
+Before moving on to the next major step, a few important "passes" occur over
+the documentation.  These do things like combine the separate "attributes" into
+a single string and strip leading whitespace to make the document easier on the
+markdown parser, or drop items that are not public or deliberately hidden with
+`#[doc(hidden)]`. These are all implemented in the `passes/` directory, one
+file per pass. By default, all of these passes are run on a crate, but the ones
+regarding dropping private/hidden items can be bypassed by passing
+`--document-private-items` to rustdoc. Note that unlike the previous set of AST
+transformations, the passes happen on the _cleaned_ crate.
 
-(Strictly speaking, you can fine-tune the passes run and even add your own, but [we're trying to
-deprecate that][44136]. If you need finer-grain control over these passes, please let us know!)
+(Strictly speaking, you can fine-tune the passes run and even add your own, but
+[we're trying to deprecate that][44136]. If you need finer-grain control over
+these passes, please let us know!)
 
 [44136]: https://github.com/rust-lang/rust/issues/44136
 
@@ -116,9 +131,9 @@ Here is current (as of this writing) list of passes:
   markdown to like it. This is necessary because the convention for writing
   documentation is to provide a space between the `///` or `//!` marker and the
   text, and stripping that leading space will make the text easier to parse by
-  the Markdown parser. (In the past, the markdown parser used was not Commonmark-
-  compliant, which caused annoyances with extra whitespace but this seems to be 
-  less of an issue today.)
+  the Markdown parser. (In the past, the markdown parser used was not
+  Commonmark- compliant, which caused annoyances with extra whitespace but this
+  seems to be less of an issue today.)
 - `strip-priv-imports` strips all private import statements (`use`, `extern
   crate`) from a crate. This is necessary because rustdoc will handle *public*
   imports by either inlining the item's documentation to the module or creating
@@ -130,85 +145,101 @@ Here is current (as of this writing) list of passes:
 
 ## From clean to crate
 
-This is where the "second phase" in rustdoc begins. This phase primarily lives in the `html/`
-folder, and it all starts with `run()` in `html/render.rs`. This code is responsible for setting up
-the `Context`, `SharedContext`, and `Cache` which are used during rendering, copying out the static
-files which live in every rendered set of documentation (things like the fonts, CSS, and JavaScript
-that live in `html/static/`), creating the search index, and printing out the source code rendering,
-before beginning the process of rendering all the documentation for the crate.
-
-Several functions implemented directly on `Context` take the `clean::Crate` and set up some state
-between rendering items or recursing on a module's child items. From here the "page rendering"
-begins, via an enormous `write!()` call in `html/layout.rs`. The parts that actually generate HTML
-from the items and documentation occurs within a series of `std::fmt::Display` implementations and
-functions that pass around a `&mut std::fmt::Formatter`. The top-level implementation that writes
-out the page body is the `impl<'a> fmt::Display for Item<'a>` in `html/render.rs`, which switches
-out to one of several `item_*` functions based on the kind of `Item` being rendered.
-
-Depending on what kind of rendering code you're looking for, you'll probably find it either in
-`html/render.rs` for major items like "what sections should I print for a struct page" or
-`html/format.rs` for smaller component pieces like "how should I print a where clause as part of
-some other item".
-
-Whenever rustdoc comes across an item that should print hand-written documentation alongside, it
-calls out to `html/markdown.rs` which interfaces with the Markdown parser. This is exposed as a
-series of types that wrap a string of Markdown, and implement `fmt::Display` to emit HTML text. It
-takes special care to enable certain features like footnotes and tables and add syntax highlighting
-to Rust code blocks (via `html/highlight.rs`) before running the Markdown parser. There's also a
-function in here (`find_testable_code`) that specifically scans for Rust code blocks so the
+This is where the "second phase" in rustdoc begins. This phase primarily lives
+in the `html/` folder, and it all starts with `run()` in `html/render.rs`. This
+code is responsible for setting up the `Context`, `SharedContext`, and `Cache`
+which are used during rendering, copying out the static files which live in
+every rendered set of documentation (things like the fonts, CSS, and JavaScript
+that live in `html/static/`), creating the search index, and printing out the
+source code rendering, before beginning the process of rendering all the
+documentation for the crate.
+
+Several functions implemented directly on `Context` take the `clean::Crate` and
+set up some state between rendering items or recursing on a module's child
+items. From here the "page rendering" begins, via an enormous `write!()` call
+in `html/layout.rs`. The parts that actually generate HTML from the items and
+documentation occurs within a series of `std::fmt::Display` implementations and
+functions that pass around a `&mut std::fmt::Formatter`. The top-level
+implementation that writes out the page body is the `impl<'a> fmt::Display for
+Item<'a>` in `html/render.rs`, which switches out to one of several `item_*`
+functions based on the kind of `Item` being rendered.
+
+Depending on what kind of rendering code you're looking for, you'll probably
+find it either in `html/render.rs` for major items like "what sections should I
+print for a struct page" or `html/format.rs` for smaller component pieces like
+"how should I print a where clause as part of some other item".
+
+Whenever rustdoc comes across an item that should print hand-written
+documentation alongside, it calls out to `html/markdown.rs` which interfaces
+with the Markdown parser. This is exposed as a series of types that wrap a
+string of Markdown, and implement `fmt::Display` to emit HTML text. It takes
+special care to enable certain features like footnotes and tables and add
+syntax highlighting to Rust code blocks (via `html/highlight.rs`) before
+running the Markdown parser. There's also a function in here
+(`find_testable_code`) that specifically scans for Rust code blocks so the
 test-runner code can find all the doctests in the crate.
 
 ### From soup to nuts
 
-(alternate title: ["An unbroken thread that stretches from those first `Cell`s to us"][video])
+(alternate title: ["An unbroken thread that stretches from those first `Cell`s
+to us"][video])
 
 [video]: https://www.youtube.com/watch?v=hOLAGYmUQV0
 
-It's important to note that the AST cleaning can ask the compiler for information (crucially,
-`DocContext` contains a `TyCtxt`), but page rendering cannot. The `clean::Crate` created within
-`run_core` is passed outside the compiler context before being handed to `html::render::run`. This
-means that a lot of the "supplementary data" that isn't immediately available inside an item's
-definition, like which trait is the `Deref` trait used by the language, needs to be collected during
-cleaning, stored in the `DocContext`, and passed along to the `SharedContext` during HTML rendering.
-This manifests as a bunch of shared state, context variables, and `RefCell`s.
-
-Also of note is that some items that come from "asking the compiler" don't go directly into the
-`DocContext` - for example, when loading items from a foreign crate, rustdoc will ask about trait
-implementations and generate new `Item`s for the impls based on that information. This goes directly
-into the returned `Crate` rather than roundabout through the `DocContext`. This way, these
-implementations can be collected alongside the others, right before rendering the HTML.
+It's important to note that the AST cleaning can ask the compiler for
+information (crucially, `DocContext` contains a `TyCtxt`), but page rendering
+cannot. The `clean::Crate` created within `run_core` is passed outside the
+compiler context before being handed to `html::render::run`. This means that a
+lot of the "supplementary data" that isn't immediately available inside an
+item's definition, like which trait is the `Deref` trait used by the language,
+needs to be collected during cleaning, stored in the `DocContext`, and passed
+along to the `SharedContext` during HTML rendering.  This manifests as a bunch
+of shared state, context variables, and `RefCell`s.
+
+Also of note is that some items that come from "asking the compiler" don't go
+directly into the `DocContext` - for example, when loading items from a foreign
+crate, rustdoc will ask about trait implementations and generate new `Item`s
+for the impls based on that information. This goes directly into the returned
+`Crate` rather than roundabout through the `DocContext`. This way, these
+implementations can be collected alongside the others, right before rendering
+the HTML.
 
 ## Other tricks up its sleeve
 
-All this describes the process for generating HTML documentation from a Rust crate, but there are
-couple other major modes that rustdoc runs in. It can also be run on a standalone Markdown file, or
-it can run doctests on Rust code or standalone Markdown files. For the former, it shortcuts straight
-to `html/markdown.rs`, optionally including a mode which inserts a Table of Contents to the output
-HTML.
-
-For the latter, rustdoc runs a similar partial-compilation to get relevant documentation in
-`test.rs`, but instead of going through the full clean and render process, it runs a much simpler
-crate walk to grab *just* the hand-written documentation. Combined with the aforementioned
-"`find_testable_code`" in `html/markdown.rs`, it builds up a collection of tests to run before
-handing them off to the libtest test runner. One notable location in `test.rs` is the function
-`make_test`, which is where hand-written doctests get transformed into something that can be
-executed.
+All this describes the process for generating HTML documentation from a Rust
+crate, but there are couple other major modes that rustdoc runs in. It can also
+be run on a standalone Markdown file, or it can run doctests on Rust code or
+standalone Markdown files. For the former, it shortcuts straight to
+`html/markdown.rs`, optionally including a mode which inserts a Table of
+Contents to the output HTML.
+
+For the latter, rustdoc runs a similar partial-compilation to get relevant
+documentation in `test.rs`, but instead of going through the full clean and
+render process, it runs a much simpler crate walk to grab *just* the
+hand-written documentation. Combined with the aforementioned
+"`find_testable_code`" in `html/markdown.rs`, it builds up a collection of
+tests to run before handing them off to the libtest test runner. One notable
+location in `test.rs` is the function `make_test`, which is where hand-written
+doctests get transformed into something that can be executed.
 
 Some extra reading about `make_test` can be found
 [here](https://quietmisdreavus.net/code/2018/02/23/how-the-doctests-get-made/).
 
 ## Dotting i's and crossing t's
 
-So that's rustdoc's code in a nutshell, but there's more things in the repo that deal with it. Since
-we have the full `compiletest` suite at hand, there's a set of tests in `src/test/rustdoc` that make
-sure the final HTML is what we expect in various situations. These tests also use a supplementary
-script, `src/etc/htmldocck.py`, that allows it to look through the final HTML using XPath notation
-to get a precise look at the output. The full description of all the commands available to rustdoc
-tests is in `htmldocck.py`.
-
-In addition, there are separate tests for the search index and rustdoc's ability to query it. The
-files in `src/test/rustdoc-js` each contain a different search query and the expected results,
-broken out by search tab. These files are processed by a script in `src/tools/rustdoc-js` and the
-Node.js runtime. These tests don't have as thorough of a writeup, but a broad example that features
-results in all tabs can be found in `basic.js`. The basic idea is that you match a given `QUERY`
-with a set of `EXPECTED` results, complete with the full item path of each item.
+So that's rustdoc's code in a nutshell, but there's more things in the repo
+that deal with it. Since we have the full `compiletest` suite at hand, there's
+a set of tests in `src/test/rustdoc` that make sure the final HTML is what we
+expect in various situations. These tests also use a supplementary script,
+`src/etc/htmldocck.py`, that allows it to look through the final HTML using
+XPath notation to get a precise look at the output. The full description of all
+the commands available to rustdoc tests is in `htmldocck.py`.
+
+In addition, there are separate tests for the search index and rustdoc's
+ability to query it. The files in `src/test/rustdoc-js` each contain a
+different search query and the expected results, broken out by search tab.
+These files are processed by a script in `src/tools/rustdoc-js` and the Node.js
+runtime. These tests don't have as thorough of a writeup, but a broad example
+that features results in all tabs can be found in `basic.js`. The basic idea is
+that you match a given `QUERY` with a set of `EXPECTED` results, complete with
+the full item path of each item.