Auto merge of #22549 - steveklabnik:doc_documentation, r=huonw

This chapter covers writing documentation in depth.

Fixes #4361
Fixes #12862
Fixes #14070
Fixes #14967

1 files changed, 419 insertions, 214 deletions

diff --git a/src/doc/trpl/documentation.md b/src/doc/trpl/documentation.md
index ded30063eba..0553fe3280c 100644
--- a/src/doc/trpl/documentation.md
+++ b/src/doc/trpl/documentation.md
@@ -1,224 +1,223 @@
 % Documentation
 
-`rustdoc` is the built-in tool for generating documentation. It integrates
-with the compiler to provide accurate hyperlinking between usage of types and
-their documentation. Furthermore, by not using a separate parser, it will
-never reject your valid Rust code.
+Documentation is an important part of any software project, and it's
+first-class in Rust. Let's talk about the tooling Rust gives you to
+document your project.
 
-# Creating Documentation
+## About `rustdoc`
 
-Documenting Rust APIs is quite simple. To document a given item, we have "doc
-comments":
+The Rust distribution includes a tool, `rustdoc`, that generates documentation.
+`rustdoc` is also used by Cargo through `cargo doc`.
 
-~~~
-# #![allow(unused_attribute)]
-// the "link" crate attribute is currently required for rustdoc, but normally
-// isn't needed.
-#![crate_id = "universe"]
-#![crate_type= "lib"]
-
-//! Tools for dealing with universes (this is a doc comment, and is shown on
-//! the crate index page. The ! makes it apply to the parent of the comment,
-//! rather than what follows).
-
-# mod workaround_the_outer_function_rustdoc_inserts {
-/// Widgets are very common (this is a doc comment, and will show up on
-/// Widget's documentation).
-pub struct Widget {
-	/// All widgets have a purpose (this is a doc comment, and will show up
-	/// the field's documentation).
-	purpose: String,
-	/// Humans are not allowed to understand some widgets
-	understandable: bool
-}
+Documentation can be generated in two ways: from source code, and from
+standalone Markdown files.
 
-pub fn recalibrate() {
-	//! Recalibrate a pesky universe (this is also a doc comment, like above,
-	//! the documentation will be applied to the *parent* item, so
-	//! `recalibrate`).
-	/* ... */
-}
-# }
-~~~
+## Documenting source code
 
-Documentation can also be controlled via the `doc` attribute on items. This is
-implicitly done by the compiler when using the above form of doc comments
-(converting the slash-based comments to `#[doc]` attributes).
+The primary way of documenting a Rust project is through annotating the source
+code. You can use documentation comments for this purpose:
 
-~~~
-#[doc = "
-Calculates the factorial of a number.
+```rust,ignore
+/// Constructs a new `Rc<T>`.
+///
+/// # Examples
+///
+/// ```
+/// use std::rc::Rc;
+///
+/// let five = Rc::new(5);
+/// ```
+pub fn new(value: T) -> Rc<T> {
+    // implementation goes here
+}
+```
 
-Given the input integer `n`, this function will calculate `n!` and return it.
-"]
-pub fn factorial(n: int) -> int { if n < 2 {1} else {n * factorial(n - 1)} }
-# fn main() {}
-~~~
+This code generates documentation that looks [like this][rc-new]. I've left the
+implementation out, with a regular comment in its place. That's the first thing
+to notice about this annotation: it uses `///`, instead of `//`. The triple slash
+indicates a documentation comment.
 
-The `doc` attribute can also be used to control how rustdoc emits documentation
-in some cases.
+Documentation comments are written in Markdown.
+
+Rust keeps track of these comments, and uses them when generating
+documentation. This is important when documenting things like enums:
 
 ```
-// Rustdoc will inline documentation of a `pub use` into this crate when the
-// `pub use` reaches across crates, but this behavior can also be disabled.
-#[doc(no_inline)]
-pub use std::option::Option;
-# fn main() {}
+/// The `Option` type. See [the module level documentation](../) for more.
+enum Option<T> {
+    /// No value
+    None,
+    /// Some value `T`
+    Some(T),
+}
 ```
 
-Doc comments are markdown, and are currently parsed with the
-[hoedown][hoedown] library. rustdoc does not yet do any fanciness such as
-referencing other items inline, like javadoc's `@see`. One exception to this
-is that the first paragraph will be used as the "summary" of an item in the
-generated documentation:
+The above works, but this does not:
 
-~~~
-/// A whizbang. Does stuff. (this line is the summary)
-///
-/// Whizbangs are ...
-struct Whizbang;
-~~~
-
-To generate the docs, run `rustdoc universe.rs`. By default, it generates a
-directory called `doc`, with the documentation for `universe` being in
-`doc/universe/index.html`. If you are using other crates with `extern crate`,
-rustdoc will even link to them when you use their types, as long as their
-documentation has already been generated by a previous run of rustdoc, or the
-crate advertises that its documentation is hosted at a given URL.
+```rust,ignore
+/// The `Option` type. See [the module level documentation](../) for more.
+enum Option<T> {
+    None, /// No value
+    Some(T), /// Some value `T`
+}
+```
 
-The generated output can be controlled with the `doc` crate attribute, which
-is how the above advertisement works. An example from the `libstd`
-documentation:
+You'll get an error:
 
-~~~
-#[doc(html_logo_url = "http://www.rust-lang.org/logos/rust-logo-128x128-blk-v2.png",
-      html_favicon_url = "http://www.rust-lang.org/favicon.ico",
-      html_root_url = "http://doc.rust-lang.org/")];
-~~~
+```text
+hello.rs:4:1: 4:2 error: expected ident, found `}`
+hello.rs:4 }
+           ^
+```
 
-The `html_root_url` is the prefix that rustdoc will apply to any references to
-that crate's types etc.
+This [unfortunate error](https://github.com/rust-lang/rust/issues/22547) is
+correct: documentation comments apply to the thing after them, and there's no
+thing after that last comment.
 
-rustdoc can also generate JSON, for consumption by other tools, with
-`rustdoc --output-format json`, and also consume already-generated JSON with
-`rustdoc --input-format json`.
+[rc-new]: http://doc.rust-lang.org/nightly/std/rc/struct.Rc.html#method.new
 
-rustdoc also supports personalizing the output from crates' documentation,
-similar to markdown options.
+### Writing documentation comments
 
-- `--html-in-header FILE`: includes the contents of `FILE` at the
-  end of the `<head>...</head>` section.
-- `--html-before-content FILE`: includes the contents of `FILE`
-  directly after `<body>`, before the rendered content (including the
-  search bar).
-- `--html-after-content FILE`: includes the contents of `FILE`
-  after all the rendered content.
+Anyway, let's cover each part of this comment in detail:
 
-# Using the Documentation
+```
+/// Constructs a new `Rc<T>`.
+# fn foo() {}
+```
 
-The web pages generated by rustdoc present the same logical hierarchy that one
-writes a library with. Every kind of item (function, struct, etc) has its own
-color, and one can always click on a colored type to jump to its
-documentation. There is a search bar at the top, which is powered by some
-JavaScript and a statically-generated search index. No special web server is
-required for the search.
+The first line of a documentation comment should be a short summary of its
+functionality. One sentence. Just the basics. High level.
 
-[hoedown]: https://github.com/hoedown/hoedown
+```
+///
+/// Other details about constructing `Rc<T>`s, maybe describing complicated
+/// semantics, maybe additional options, all kinds of stuff.
+///
+# fn foo() {}
+```
 
-# Testing the Documentation
+Our original example had just a summary line, but if we had more things to say,
+we could have added more explanation in a new paragraph.
 
-`rustdoc` has support for testing code examples which appear in the
-documentation. This is helpful for keeping code examples up to date with the
-source code.
+#### Special sections
 
-To test documentation, the `--test` argument is passed to rustdoc:
+```
+/// # Examples
+# fn foo() {}
+```
 
-~~~ {.sh}
-rustdoc --test crate.rs
-~~~
+Next, are special sections. These are indicated with a header, `#`. There
+are three kinds of headers that are commonly used. They aren't special syntax,
+just convention, for now.
 
-## Defining tests
+```
+/// # Panics
+# fn foo() {}
+```
 
-Rust documentation currently uses the markdown format, and rustdoc treats all
-code blocks as testable-by-default unless they carry a language tag of another
-language. In order to not run a test over a block of code, the `ignore` string
-can be added to the three-backtick form of markdown code block.
+Unrecoverable misuses of a function (i.e. programming errors) in Rust are
+usually indicated by panics, which kill the whole current thread at the very
+least. If your function has a non-trivial contract like this, that is
+detected/enforced by panics, documenting it is very important.
 
-~~~md
 ```
-// This is a testable code block
+/// # Failures
+# fn foo() {}
 ```
 
-```rust{.example}
-// This is rust and also testable
-```
+If your function or method returns a `Result<T, E>`, then describing the
+conditions under which it returns `Err(E)` is a nice thing to do. This is
+slightly less important than `Panics`, because failure is encoded into the type
+system, but it's still a good thing to do.
 
-```ignore
-// This is not a testable code block
+```
+/// # Safety
+# fn foo() {}
 ```
 
-    // This is a testable code block (4-space indent)
+If your function is `unsafe`, you should explain which invariants the caller is
+responsible for upholding.
 
-```sh
-# this is shell code and not tested
 ```
-~~~
+/// # Examples
+///
+/// ```
+/// use std::rc::Rc;
+///
+/// let five = Rc::new(5);
+/// ```
+# fn foo() {}
+```
 
-You can specify that the test's execution should fail with the `should_fail`
-directive.
+Third, `Examples`. Include one or more examples of using your function or
+method, and your users will love you for it. These examples go inside of
+code block annotations, which we'll talk about in a moment, and can have
+more than one section:
 
-~~~md
-```should_fail
-// This code block is expected to generate a panic when run
 ```
-~~~
+/// # Examples
+///
+/// Simple `&str` patterns:
+///
+/// ```
+/// let v: Vec<&str> = "Mary had a little lamb".split(' ').collect();
+/// assert_eq!(v, vec!["Mary", "had", "a", "little", "lamb"]);
+/// ```
+///
+/// More complex patterns with a lambda:
+///
+/// ```
+/// let v: Vec<&str> = "abc1def2ghi".split(|c: char| c.is_numeric()).collect();
+/// assert_eq!(v, vec!["abc", "def", "ghi"]);
+/// ```
+# fn foo() {}
+```
 
-You can specify that the code block should be compiled but not run with the
-`no_run` directive.
+Let's discuss the details of these code blocks.
 
-~~~md
-```no_run
-// This code will be compiled but not executed
-```
-~~~
+#### Code block annotations
 
-Lastly, you can specify that a code block be compiled as if `--test`
-were passed to the compiler using the `test_harness` directive.
+To write some Rust code in a comment, use the triple graves:
 
-~~~md
-```test_harness
-#[test]
-fn foo() {
-    panic!("oops! (will run & register as a failed test)")
-}
 ```
-~~~
+/// ```
+/// println!("Hello, world");
+/// ```
+# fn foo() {}
+```
 
-Rustdoc also supplies some extra sugar for helping with some tedious
-documentation examples. If a line is prefixed with `# `, then the line
-will not show up in the HTML documentation, but it will be used when
-testing the code block (NB. the space after the `#` is required, so
-that one can still write things like `#[derive(Eq)]`).
+If you want something that's not Rust code, you can add an annotation:
 
-~~~md
 ```
-# /!\ The three following lines are comments, which are usually stripped off by
-# the doc-generating tool.  In order to display them anyway in this particular
-# case, the character following the leading '#' is not a usual space like in
-# these first five lines but a non breakable one.
-# // showing 'fib' in this documentation would just be tedious and detracts from
-# // what's actually being documented.
-# fn fib(n: int) { n + 2 }
-
-spawn(move || { fib(200); })
+/// ```c
+/// printf("Hello, world\n");
+/// ```
+# fn foo() {}
 ```
-~~~
 
-The documentation online would look like `spawn(move || { fib(200); })`, but when
-testing this code, the `fib` function will be included (so it can compile).
+This will highlight according to whatever language you're showing off.
+If you're just showing plain text, choose `text`.
+
+It's important to choose the correct annotation here, because `rustdoc` uses it
+in an interesting way: It can be used to actually test your examples, so that
+they don't get out of date. If you have some C code but `rustdoc` thinks it's
+Rust because you left off the annotation, `rustdoc` will complain when trying to
+generate the documentation.
 
-Rustdoc will automatically add a `main()` wrapper around your code, and in the right
-place. For example:
+## Documentation as tests
+
+Let's discuss our sample example documentation:
+
+```
+/// ```
+/// println!("Hello, world");
+/// ```
+# fn foo() {}
+```
+
+You'll notice that you don't need a `fn main()` or anything here. `rustdoc` will
+automatically add a main() wrapper around your code, and in the right place.
+For example:
 
 ```
 /// ```
@@ -240,80 +239,286 @@ fn main() {
 
 Here's the full algorithm:
 
-1. Given a code block, if it does not contain `fn main`, it is wrapped in `fn main() { your_code }`
+1. Given a code block, if it does not contain `fn main()`, it is wrapped in
+   `fn main() { your_code }`
 2. Given that result, if it contains no `extern crate` directives but it also
    contains the name of the crate being tested, then `extern crate <name>` is
    injected at the top.
-3. Some common `allow` attributes are added for documentation examples at the top.
+3. Some common allow attributes are added for documentation examples at the top.
+
+Sometimes, this isn't enough, though. For example, all of these code samples
+with `///` we've been talking about? The raw text:
+
+```text
+/// Some documentation.
+# fn foo() {}
+```
+
+looks different than the output:
+
+```
+/// Some documentation.
+# fn foo() {}
+```
+
+Yes, that's right: you can add lines that start with `# `, and they will
+be hidden from the output, but will be used when compiling your code. You
+can use this to your advantage. In this case, documentation comments need
+to apply to some kind of function, so if I want to show you just a
+documentation comment, I need to add a little function definition below
+it. At the same time, it's just there to satisfy the compiler, so hiding
+it makes the example more clear. You can use this technique to explain
+longer examples in detail, while still preserving the testability of your
+documentation. For example, this code:
+
+```
+let x = 5;
+let y = 6;
+println!("{}", x + y);
+```
+
+Here's an explanation, rendered:
+
+First, we set `x` to five:
+
+```
+let x = 5;
+# let y = 6;
+# println!("{}", x + y);
+```
+
+Next, we set `y` to six:
+
+```
+# let x = 5;
+let y = 6;
+# println!("{}", x + y);
+```
+
+Finally, we print the sum of `x` and `y`:
+
+```
+# let x = 5;
+# let y = 6;
+println!("{}", x + y);
+```
+
+Here's the same explanation, in raw text:
+
+> First, we set `x` to five:
+> 
+> ```text
+> let x = 5;
+> # let y = 6;
+> # println!("{}", x + y);
+> ```
+> 
+> Next, we set `y` to six:
+> 
+> ```text
+> # let x = 5;
+> let y = 6;
+> # println!("{}", x + y);
+> ```
+> 
+> Finally, we print the sum of `x` and `y`:
+> 
+> ```text
+> # let x = 5;
+> # let y = 6;
+> println!("{}", x + y);
+> ```
+
+By repeating all parts of the example, you can ensure that your example still
+compiles, while only showing the parts that are relevant to that part of your
+explanation.
+
+To run the tests, either
+
+```bash
+$ rustdoc --test path/to/my/crate/root.rs
+# or
+$ cargo test
+```
+
+That's right, `cargo test` tests embedded documentation too.
+
+There are a few more annotations that are useful to help `rustdoc` do the right
+thing when testing your code:
+
+```
+/// ```ignore
+/// fn foo() {
+/// ```
+# fn foo() {}
+```
+
+The `ignore` directive tells Rust to ignore your code. This is almost never
+what you want, as it's the most generic. Instead, consider annotating it
+with `text` if it's not code, or using `#`s to get a working example that
+only shows the part you care about.
+
+```
+/// ```should_panic
+/// assert!(false);
+/// ```
+# fn foo() {}
+```
+
+`should_panic` tells `rustdoc` that the code should compile correctly, but
+not actually pass as a test.
+
+```
+/// ```no_run
+/// loop {
+///     println!("Hello, world");
+/// }
+/// ```
+# fn foo() {}
+```
+
+The `no_run` attribute will compile your code, but not run it. This is
+important for examples such as "Here's how to start up a network service,"
+which you would want to make sure compile, but might run in an infinite loop!
+
+### Documenting modules
+
+Rust has another kind of doc comment, `//!`. This comment doesn't document the next item, but the enclosing item. In other words:
+
+```
+mod foo {
+    //! This is documentation for the `foo` module.
+    //!
+    //! # Examples
+
+    // ...
+}
+```
+
+This is where you'll see `//!` used most often: for module documentation. If
+you have a module in `foo.rs`, you'll often open its code and see this:
+
+```
+//! A module for using `foo`s.
+//!
+//! The `foo` module contains a lot of useful functionality blah blah blah
+```
+
+### Documentation comment style
+
+Check out [RFC 505][rfc505] for full conventions around the style and format of
+documentation.
+
+[rfc505]: https://github.com/rust-lang/rfcs/blob/master/text/0505-api-comment-conventions.md
+
+## Other documentation
+
+All of this behavior works in non-Rust source files too. Because comments
+are written in Markdown, they're often `.md` files.
+
+When you write documentation in Markdown files, you don't need to prefix
+the documentation with comments. For example:
 
-## Running tests (advanced)
+```
+/// # Examples
+///
+/// ```
+/// use std::rc::Rc;
+///
+/// let five = Rc::new(5);
+/// ```
+# fn foo() {}
+```
 
-Running tests often requires some special configuration to filter tests, find
-libraries, or try running ignored examples. The testing framework that rustdoc
-uses is built on crate `test`, which is also used when you compile crates with
-rustc's `--test` flag. Extra arguments can be passed to rustdoc's test harness
-with the `--test-args` flag.
+is just
 
-~~~console
-# Only run tests containing 'foo' in their name
-$ rustdoc --test lib.rs --test-args 'foo'
+~~~markdown
+# Examples
 
-# See what's possible when running tests
-$ rustdoc --test lib.rs --test-args '--help'
+```
+use std::rc::Rc;
+
+let five = Rc::new(5);
+```
 ~~~
 
-When testing a library, code examples will often show how functions are used,
-and this code often requires `use`-ing paths from the crate. To accommodate this,
-rustdoc will implicitly add `extern crate <crate>;` where `<crate>` is the name of
-the crate being tested to the top of each code example. This means that rustdoc
-must be able to find a compiled version of the library crate being tested. Extra
-search paths may be added via the `-L` flag to `rustdoc`.
+when it's in a Markdown file. There is one wrinkle though: Markdown files need
+to have a title like this:
 
-# Standalone Markdown files
+```markdown
+% The title
 
-As well as Rust crates, rustdoc supports rendering pure Markdown files
-into HTML and testing the code snippets from them. A Markdown file is
-detected by a `.md` or `.markdown` extension.
+This is the example documentation.
+```
+
+This `%` line needs to be the very first line of the file.
+
+## `doc` attributes
+
+At a deeper level, documentation comments are sugar for documentation attributes:
+
+```
+/// this
+# fn foo() {}
+
+#[doc="this"]
+# fn bar() {}
+```
 
-There are 4 options to modify the output that Rustdoc creates.
+are the same, as are these:
 
-- `--markdown-css PATH`: adds a `<link rel="stylesheet">` tag pointing to `PATH`.
-- `--html-in-header FILE`: includes the contents of `FILE` at the
-  end of the `<head>...</head>` section.
-- `--html-before-content FILE`: includes the contents of `FILE`
-  directly after `<body>`, before the rendered content (including the
-  title).
-- `--html-after-content FILE`: includes the contents of `FILE`
-  directly before `</body>`, after all the rendered content.
+```
+//! this
 
-All of these can be specified multiple times, and they are output in
-the order in which they are specified. The first line of the file being rendered must
-be the title, prefixed with `%` (e.g. this page has `% Rust
-Documentation` on the first line).
+#![doc="/// this"]
+```
 
-Like with a Rust crate, the `--test` argument will run the code
-examples to check they compile, and obeys any `--test-args` flags. The
-tests are named after the last `#` heading.
+You won't often see this attribute used for writing documentation, but it
+can be useful when changing some options, or when writing a macro.
 
-# Re-exports
+### Re-exports
 
-Rustdoc will show the documentation for a publc re-export in both places:
+`rustdoc` will show the documentation for a publc re-export in both places:
 
-```{rust,ignore}
+```ignore
 extern crate foo;
 
 pub use foo::bar;
 ```
 
-This will create documentation for `bar` both inside the documentation for
-the crate `foo`, as well as the documentation for your crate. It will use
-the same documentation in both places.
+This will create documentation for bar both inside the documentation for the
+crate `foo`, as well as the documentation for your crate. It will use the same
+documentation in both places.
 
 This behavior can be supressed with `no_inline`:
 
-```{rust,ignore}
+```ignore
 extern crate foo;
 
 #[doc(no_inline)]
 pub use foo::bar;
 ```
+
+### Controlling HTML
+
+You can control a few aspects of the HTML that `rustdoc` generates through the
+`#![doc]` version of the attribute:
+
+```
+#![doc(html_logo_url = "http://www.rust-lang.org/logos/rust-logo-128x128-blk-v2.png",
+       html_favicon_url = "http://www.rust-lang.org/favicon.ico",
+       html_root_url = "http://doc.rust-lang.org/")];
+```
+
+This sets a few different options, with a logo, favicon, and a root URL.
+
+## Generation options
+
+`rustdoc` also contains a few other options on the command line, for further customiziation:
+
+- `--html-in-header FILE`: includes the contents of FILE at the end of the
+  `<head>...</head>` section.
+- `--html-before-content FILE`: includes the contents of FILE directly after
+  `<body>`, before the rendered content (including the search bar).
+- `--html-after-content FILE`: includes the contents of FILE after all the rendered content.
+