rollup merge of #23568: steveklabnik/closure_docs

[rendered](https://github.com/steveklabnik/rust/blob/closure_docs/src/doc/trpl/closures.md)

r? @nikomatsakis

1 files changed, 401 insertions, 137 deletions

diff --git a/src/doc/trpl/closures.md b/src/doc/trpl/closures.md
index bf4c2d90357..01b8163ffd3 100644
--- a/src/doc/trpl/closures.md
+++ b/src/doc/trpl/closures.md
@@ -1,214 +1,478 @@
 % Closures
 
-So far, we've made lots of functions in Rust, but we've given them all names.
-Rust also allows us to create anonymous functions. Rust's anonymous
-functions are called *closures*. By themselves, closures aren't all that
-interesting, but when you combine them with functions that take closures as
-arguments, really powerful things are possible.
+Rust not only has named functions, but anonymous functions as well. Anonymous
+functions that have an associated environment are called 'closures', because they
+close over an environment. Rust has a really great implementation of them, as
+we'll see.
 
-Let's make a closure:
+# Syntax
 
-```{rust}
-let add_one = |x| { 1 + x };
+Closures look like this:
 
-println!("The sum of 5 plus 1 is {}.", add_one(5));
+```rust
+let plus_one = |x: i32| x + 1;
+
+assert_eq!(2, plus_one(1));
+```
+
+We create a binding, `plus_one`, and assign it to a closure. The closure's
+arguments go between the pipes (`|`), and the body is an expression, in this
+case, `x + 1`. Remember that `{ }` is an expression, so we can have multi-line
+closures too:
+
+```rust
+let plus_two = |x| {
+    let mut result: i32 = x;
+
+    result += 1;
+    result += 1;
+
+    result
+};
+
+assert_eq!(4, plus_two(2));
+```
+
+You'll notice a few things about closures that are a bit different than regular
+functions defined with `fn`. The first of which is that we did not need to
+annotate the types of arguments the closure takes or the values it returns. We
+can:
+
+```rust
+let plus_one = |x: i32| -> i32 { x + 1 };
+
+assert_eq!(2, plus_one(1));
+```
+
+But we don't have to. Why is this? Basically, it was chosen for ergonomic reasons.
+While specifying the full type for named functions is helpful with things like
+documentation and type inference, the types of closures are rarely documented
+since they’re anonymous, and they don’t cause the kinds of error-at-a-distance
+that inferring named function types can.
+
+The second is that the syntax is similar, but a bit different. I've added spaces
+here to make them look a little closer:
+
+```rust
+fn  plus_one_v1 (  x: i32 ) -> i32 { x + 1 }
+let plus_one_v2 = |x: i32 | -> i32 { x + 1 };
+let plus_one_v3 = |x: i32 |          x + 1  ;
 ```
 
-We create a closure using the `|...| { ... }` syntax, and then we create a
-binding so we can use it later. Note that we call the function using the
-binding name and two parentheses, just like we would for a named function.
+Small differences, but they're similar in ways.
 
-Let's compare syntax. The two are pretty close:
+# Closures and their environment
 
-```{rust}
-let add_one = |x: i32| -> i32 { 1 + x };
-fn  add_one   (x: i32) -> i32 { 1 + x }
+Closures are called such because they 'close over their environment.' It
+looks like this:
+
+```rust
+let num = 5;
+let plus_num = |x: i32| x + num;
+
+assert_eq!(10, plus_num(5));
 ```
 
-As you may have noticed, closures infer their argument and return types, so you
-don't need to declare one. This is different from named functions, which
-default to returning unit (`()`).
+This closure, `plus_num`, refers to a `let` binding in its scope: `num`. More
+specifically, it borrows the binding. If we do something that would conflict
+with that binding, we get an error. Like this one:
+
+```rust,ignore
+let mut num = 5;
+let plus_num = |x: i32| x + num;
 
-There's one big difference between a closure and named functions, and it's in
-the name: a closure "closes over its environment." What does that mean? It means
-this:
+let y = &mut num;
+```
 
-```{rust}
+Which errors with:
+
+```text
+error: cannot borrow `num` as mutable because it is also borrowed as immutable
+    let y = &mut num;
+                 ^~~
+note: previous borrow of `num` occurs here due to use in closure; the immutable
+  borrow prevents subsequent moves or mutable borrows of `num` until the borrow
+  ends
+    let plus_num = |x| x + num;
+                   ^~~~~~~~~~~
+note: previous borrow ends here
 fn main() {
-    let x: i32 = 5;
+    let mut num = 5;
+    let plus_num = |x| x + num;
+    
+    let y = &mut num;
+}
+^
+```
+
+A verbose yet helpful error message! As it says, we can't take a mutable borrow
+on `num` because the closure is already borrowing it. If we let the closure go
+out of scope, we can:
+
+```rust
+let mut num = 5;
+{
+    let plus_num = |x: i32| x + num;
+
+} // plus_num goes out of scope, borrow of num ends
 
-    let printer = || { println!("x is: {}", x); };
+let y = &mut num;
+```
+
+If your closure requires it, however, Rust will take ownership and move
+the environment instead:
+
+```rust,ignore
+let nums = vec![1, 2, 3];
+
+let takes_nums = || nums;
+
+println!("{:?}", nums);
+```
+
+This gives us:
+
+```text
+note: `nums` moved into closure environment here because it has type
+  `[closure(()) -> collections::vec::Vec<i32>]`, which is non-copyable
+let takes_nums = || nums;
+                    ^~~~~~~
+```
+
+`Vec<T>` has ownership over its contents, and therefore, when we refer to it
+in our closure, we have to take ownership of `nums`. It's the same as if we'd
+passed `nums` to a function that took ownership of it.
+
+## `move` closures
+
+We can force our closure to take ownership of its environment with the `move`
+keyword:
 
-    printer(); // prints "x is: 5"
+```rust
+let num = 5;
+
+let owns_num = move |x: i32| x + num;
+```
+
+Now, even though the keyword is `move`, the variables follow normal move semantics.
+In this case, `5` implements `Copy`, and so `owns_num` takes ownership of a copy
+of `num`. So what's the difference?
+
+```rust
+let mut num = 5;
+
+{ 
+    let mut add_num = |x: i32| num += x;
+
+    add_num(5);
 }
+
+assert_eq!(10, num);
 ```
 
-The `||` syntax means this is an anonymous closure that takes no arguments.
-Without it, we'd just have a block of code in `{}`s.
+So in this case, our closure took a mutable reference to `num`, and then when
+we called `add_num`, it mutated the underlying value, as we'd expect. We also
+needed to declare `add_num` as `mut` too, because we’re mutating its
+environment.
 
-In other words, a closure has access to variables in the scope where it's
-defined. The closure borrows any variables it uses, so this will error:
+We also had to declare `add_num` as mut, since we will be modifying its
+environment.
 
-```{rust,ignore}
-fn main() {
-    let mut x: i32 = 5;
+If we change to a `move` closure, it's different:
+
+```rust
+let mut num = 5;
 
-    let printer = || { println!("x is: {}", x); };
+{ 
+    let mut add_num = move |x: i32| num += x;
 
-    x = 6; // error: cannot assign to `x` because it is borrowed
+    add_num(5);
 }
+
+assert_eq!(5, num);
 ```
 
-## Moving closures
+We only get `5`. Rather than taking a mutable borrow out on our `num`, we took
+ownership of a copy.
+
+Another way to think about `move` closures: they give a closure its own stack
+frame.  Without `move`, a closure may be tied to the stack frame that created
+it, while a `move` closure is self-contained. This means that you cannot
+generally return a non-`move` closure from a function, for example.
+
+But before we talk about taking and returning closures, we should talk some more
+about the way that closures are implemented. As a systems language, Rust gives
+you tons of control over what your code does, and closures are no different.
 
-Rust has a second type of closure, called a *moving closure*. Moving
-closures are indicated using the `move` keyword (e.g., `move || x *
-x`). The difference between a moving closure and an ordinary closure
-is that a moving closure always takes ownership of all variables that
-it uses. Ordinary closures, in contrast, just create a reference into
-the enclosing stack frame. Moving closures are most useful with Rust's
-concurrency features, and so we'll just leave it at this for
-now. We'll talk about them more in the "Concurrency" chapter of the book.
+# Closure implementation
 
-## Accepting closures as arguments
+Rust's implementation of closures is a bit different than other languages. They
+are effectively syntax sugar for traits. You'll want to make sure to have read
+the [traits chapter][traits] before this one, as well as the chapter on [static
+and dynamic dispatch][dispatch], which talks about trait objects.
 
-Closures are most useful as an argument to another function. Here's an example:
+[traits]: traits.html
+[dispatch]: static-and-dynamic-dispatch.html
 
-```{rust}
-fn twice<F: Fn(i32) -> i32>(x: i32, f: F) -> i32 {
-    f(x) + f(x)
+Got all that? Good.
+
+The key to understanding how closures work under the hood is something a bit
+strange: Using `()` to call a function, like `foo()`, is an overloadable
+operator. From this, everything else clicks into place. In Rust, we use the
+trait system to overload operators. Calling functions is no different. We have
+three separate traits to overload with:
+
+```rust
+# mod foo {
+pub trait Fn<Args> : FnMut<Args> {
+    extern "rust-call" fn call(&self, args: Args) -> Self::Output;
 }
 
-fn main() {
-    let square = |x: i32| { x * x };
+pub trait FnMut<Args> : FnOnce<Args> {
+    extern "rust-call" fn call_mut(&mut self, args: Args) -> Self::Output;
+}
+
+pub trait FnOnce<Args> {
+    type Output;
 
-    twice(5, square); // evaluates to 50
+    extern "rust-call" fn call_once(self, args: Args) -> Self::Output;
 }
+# }
 ```
 
-Let's break the example down, starting with `main`:
+You'll notice a few differences between these traits, but a big one is `self`:
+`Fn` takes `&self`, `FnMut` takes `&mut self`, and `FnOnce` takes `self`. This
+covers all three kinds of `self` via the usual method call syntax. But we've
+split them up into three traits, rather than having a single one. This gives us
+a large amount of control over what kind of closures we can take.
 
-```{rust}
-let square = |x: i32| { x * x };
-```
+The `|| {}` syntax for closures is sugar for these three traits. Rust will
+generate a struct for the environment, `impl` the appropriate trait, and then
+use it.
+
+# Taking closures as arguments
+
+Now that we know that closures are traits, we already know how to accept and
+return closures: just like any other trait!
+
+This also means that we can choose static vs dynamic dispatch as well. First,
+let's write a function which takes something callable, calls it, and returns
+the result:
+
+```rust
+fn call_with_one<F>(some_closure: F) -> i32
+    where F : Fn(i32) -> i32 {
+
+    some_closure(1)
+}
 
-We've seen this before. We make a closure that takes an integer, and returns
-its square.
+let answer = call_with_one(|x| x + 2);
 
-```{rust}
-# fn twice<F: Fn(i32) -> i32>(x: i32, f: F) -> i32 { f(x) + f(x) }
-# let square = |x: i32| { x * x };
-twice(5, square); // evaluates to 50
+assert_eq!(3, answer);
 ```
 
-This line is more interesting. Here, we call our function, `twice`, and we pass
-it two arguments: an integer, `5`, and our closure, `square`. This is just like
-passing any other two variable bindings to a function, but if you've never
-worked with closures before, it can seem a little complex. Just think: "I'm
-passing two variables: one is an i32, and one is a function."
+We pass our closure, `|x| x + 2`, to `call_with_one`. It just does what it
+suggests: it calls the closure, giving it `1` as an argument.
 
-Next, let's look at how `twice` is defined:
+Let's examine the signature of `call_with_one` in more depth:
 
-```{rust,ignore}
-fn twice<F: Fn(i32) -> i32>(x: i32, f: F) -> i32 {
+```rust
+fn call_with_one<F>(some_closure: F) -> i32
+#    where F : Fn(i32) -> i32 {
+#    some_closure(1) }
 ```
 
-`twice` takes two arguments, `x` and `f`. That's why we called it with two
-arguments. `x` is an `i32`, we've done that a ton of times. `f` is a function,
-though, and that function takes an `i32` and returns an `i32`. This is
-what the requirement `Fn(i32) -> i32` for the type parameter `F` says.
-Now `F` represents *any* function that takes an `i32` and returns an `i32`.
+We take one parameter, and it has the type `F`. We also return a `i32`. This part
+isn't interesting. The next part is:
 
-This is the most complicated function signature we've seen yet! Give it a read
-a few times until you can see how it works. It takes a teeny bit of practice, and
-then it's easy. The good news is that this kind of passing a closure around
-can be very efficient. With all the type information available at compile-time
-the compiler can do wonders.
+```rust
+# fn call_with_one<F>(some_closure: F) -> i32
+    where F : Fn(i32) -> i32 {
+#   some_closure(1) }
+```
+
+Because `Fn` is a trait, we can bound our generic with it. In this case, our closure
+takes a `i32` as an argument and returns an `i32`, and so the generic bound we use
+is `Fn(i32) -> i32`.
 
-Finally, `twice` returns an `i32` as well.
+There's one other key point here: because we're bounding a generic with a
+trait, this will get monomorphized, and therefore, we'll be doing static
+dispatch into the closure. That's pretty neat. In many langauges, closures are
+inherently heap allocated, and will always involve dynamic dispatch. In Rust,
+we can stack allocate our closure environment, and statically dispatch the
+call. This happens quite often with iterators and their adapters, which often
+take closures as arguments.
 
-Okay, let's look at the body of `twice`:
+Of course, if we want dynamic dispatch, we can get that too. A trait object
+handles this case, as usual:
 
-```{rust}
-fn twice<F: Fn(i32) -> i32>(x: i32, f: F) -> i32 {
-  f(x) + f(x)
+```rust
+fn call_with_one(some_closure: &Fn(i32) -> i32) -> i32 {
+    some_closure(1)
 }
+
+let answer = call_with_one(&|x| x + 2);
+
+assert_eq!(3, answer);
 ```
 
-Since our closure is named `f`, we can call it just like we called our closures
-before, and we pass in our `x` argument to each one, hence the name `twice`.
+Now we take a trait object, a `&Fn`. And we have to make a reference
+to our closure when we pass it to `call_with_one`, so we use `&||`.
 
-If you do the math, `(5 * 5) + (5 * 5) == 50`, so that's the output we get.
+# Returning closures
 
-Play around with this concept until you're comfortable with it. Rust's standard
-library uses lots of closures where appropriate, so you'll be using
-this technique a lot.
+It’s very common for functional-style code to return closures in various
+situations. If you try to return a closure, you may run into an error. At
+first, it may seem strange, but we'll figure it out. Here's how you'd probably
+try to return a closure from a function:
 
-If we didn't want to give `square` a name, we could just define it inline.
-This example is the same as the previous one:
+```rust,ignore
+fn factory() -> (Fn(i32) -> Vec<i32>) {
+    let vec = vec![1, 2, 3];
 
-```{rust}
-fn twice<F: Fn(i32) -> i32>(x: i32, f: F) -> i32 {
-    f(x) + f(x)
+    |n| vec.push(n)
 }
 
-fn main() {
-    twice(5, |x: i32| { x * x }); // evaluates to 50
-}
+let f = factory();
+
+let answer = f(4);
+assert_eq!(vec![1, 2, 3, 4], answer);
 ```
 
-A named function's name can be used wherever you'd use a closure. Another
-way of writing the previous example:
+This gives us these long, related errors:
+
+```text
+error: the trait `core::marker::Sized` is not implemented for the type
+`core::ops::Fn(i32) -> collections::vec::Vec<i32>` [E0277]
+f = factory();
+^
+note: `core::ops::Fn(i32) -> collections::vec::Vec<i32>` does not have a
+constant size known at compile-time
+f = factory();
+^
+error: the trait `core::marker::Sized` is not implemented for the type
+`core::ops::Fn(i32) -> collections::vec::Vec<i32>` [E0277]
+factory() -> (Fn(i32) -> Vec<i32>) {
+             ^~~~~~~~~~~~~~~~~~~~~
+note: `core::ops::Fn(i32) -> collections::vec::Vec<i32>` does not have a constant size known at compile-time
+fa ctory() -> (Fn(i32) -> Vec<i32>) {
+              ^~~~~~~~~~~~~~~~~~~~~
 
-```{rust}
-fn twice<F: Fn(i32) -> i32>(x: i32, f: F) -> i32 {
-    f(x) + f(x)
-}
+```
 
-fn square(x: i32) -> i32 { x * x }
+In order to return something from a function, Rust needs to know what
+size the return type is. But since `Fn` is a trait, it could be various
+things of various sizes: many different types can implement `Fn`. An easy
+way to give something a size is to take a reference to it, as references
+have a known size. So we'd write this:
 
-fn main() {
-    twice(5, square); // evaluates to 50
+```rust,ignore
+fn factory() -> &(Fn(i32) -> Vec<i32>) {
+    let vec = vec![1, 2, 3];
+
+    |n| vec.push(n)
 }
+
+let f = factory();
+
+let answer = f(4);
+assert_eq!(vec![1, 2, 3, 4], answer);
+```
+
+But we get another error:
+
+```text
+error: missing lifetime specifier [E0106]
+fn factory() -> &(Fn(i32) -> i32) {
+                ^~~~~~~~~~~~~~~~~
 ```
 
-Doing this is not particularly common, but it's useful every once in a while.
+Right. Because we have a reference, we need to give it a lifetime. But
+our `factory()` function takes no arguments, so elision doesn't kick in
+here. What lifetime can we choose? `'static`:
 
-Before we move on, let us look at a function that accepts two closures.
+```rust,ignore
+fn factory() -> &'static (Fn(i32) -> i32) {
+    let num = 5;
 
-```{rust}
-fn compose<F, G>(x: i32, f: F, g: G) -> i32
-    where F: Fn(i32) -> i32, G: Fn(i32) -> i32 {
-    g(f(x))
+    |x| x + num
 }
 
-fn main() {
-    compose(5,
-            |n: i32| { n + 42 },
-            |n: i32| { n * 2 }); // evaluates to 94
+let f = factory();
+
+let answer = f(1);
+assert_eq!(6, answer);
+```
+
+But we get another error:
+
+```text
+error: mismatched types:
+ expected `&'static core::ops::Fn(i32) -> i32`,
+    found `[closure <anon>:7:9: 7:20]`
+(expected &-ptr,
+    found closure) [E0308]
+         |x| x + num
+         ^~~~~~~~~~~
+
+```
+
+This error is letting us know that we don't have a `&'static Fn(i32) -> i32`,
+we have a `[closure <anon>:7:9: 7:20]`. Wait, what?
+
+Because each closure generates its own environment `struct` and implementation
+of `Fn` and friends, these types are anonymous. They exist just solely for
+this closure. So Rust shows them as `closure <anon>`, rather than some
+autogenerated name.
+
+But why doesn't our closure implement `&'static Fn`? Well, as we discussed before,
+closures borrow their environment. And in this case, our environment is based
+on a stack-allocated `5`, the `num` variable binding. So the borrow has a lifetime
+of the stack frame. So if we returned this closure, the function call would be
+over, the stack frame would go away, and our closure is capturing an environment
+of garbage memory!
+
+So what to do? This _almost_ works:
+
+```rust,ignore
+fn factory() -> Box<Fn(i32) -> i32> {
+    let num = 5;
+
+    Box::new(|x| x + num)
 }
+# fn main() {
+let f = factory();
+
+let answer = f(1);
+assert_eq!(6, answer);
+# }
 ```
 
-You might ask yourself: why do we need to introduce two type
-parameters `F` and `G` here?  Evidently, both `f` and `g` have the
-same signature: `Fn(i32) -> i32`.
+We use a trait object, by `Box`ing up the `Fn`. There's just one last problem:
 
-That is because in Rust each closure has its own unique type.
-So, not only do closures with different signatures have different types,
-but different closures with the *same* signature have *different*
-types, as well!
+```text
+error: `num` does not live long enough
+Box::new(|x| x + num)
+         ^~~~~~~~~~~
+```
+
+We still have a reference to the parent stack frame. With one last fix, we can
+make this work:
 
-You can think of it this way: the behavior of a closure is part of its
-type.  Therefore, using a single type parameter for both closures
-will accept the first of them, rejecting the second. The distinct
-type of the second closure does not allow it to be represented by the
-same type parameter as that of the first.  We acknowledge this, and
-use two different type parameters `F` and `G`.
+```rust
+fn factory() -> Box<Fn(i32) -> i32> {
+    let num = 5;
 
-This also introduces the `where` clause, which lets us describe type
-parameters in a more flexible manner.
+    Box::new(move |x| x + num)
+}
+# fn main() {
+let f = factory();
+
+let answer = f(1);
+assert_eq!(6, answer);
+# }
+```
 
-That's all you need to get the hang of closures! Closures are a little bit
-strange at first, but once you're used to them, you'll miss them
-in other languages. Passing functions to other functions is
-incredibly powerful, as you will see in the following chapter about iterators.
+By making the inner closure a `move Fn`, we create a new stack frame for our
+closure. By `Box`ing it up, we've given it a known size, and allowing it to
+escape our stack frame.