diff options
| author | Kornel <kornel@geekhood.net> | 2020-03-14 02:29:10 +0000 |
|---|---|---|
| committer | Kornel <kornel@geekhood.net> | 2020-03-14 02:29:35 +0000 |
| commit | a5a786d00c1490aaca066df88f8fe2b1cfd8ac75 (patch) | |
| tree | e9181718211bcc2d7812d4b15e51c1c420679d75 | |
| parent | be055d96c4c223a5ad49a0181f0b43bc46781708 (diff) | |
| download | rust-a5a786d00c1490aaca066df88f8fe2b1cfd8ac75.tar.gz rust-a5a786d00c1490aaca066df88f8fe2b1cfd8ac75.zip | |
Block version-specific docs from search engines
Stable, beta and nightly URLs remain
| -rw-r--r-- | src/doc/robots.txt | 19 |
1 files changed, 2 insertions, 17 deletions
diff --git a/src/doc/robots.txt b/src/doc/robots.txt index d119cc46473..3a2552e9a15 100644 --- a/src/doc/robots.txt +++ b/src/doc/robots.txt @@ -1,21 +1,6 @@ User-agent: * -Disallow: /0.3/ -Disallow: /0.4/ -Disallow: /0.5/ -Disallow: /0.6/ -Disallow: /0.7/ -Disallow: /0.8/ -Disallow: /0.9/ -Disallow: /0.10/ -Disallow: /0.11.0/ -Disallow: /0.12.0/ -Disallow: /1.0.0-alpha/ -Disallow: /1.0.0-alpha.2/ -Disallow: /1.0.0-beta/ -Disallow: /1.0.0-beta.2/ -Disallow: /1.0.0-beta.3/ -Disallow: /1.0.0-beta.4/ -Disallow: /1.0.0-beta.5/ +Disallow: /1. +Disallow: /0. Disallow: /book/first-edition/ Disallow: /book/second-edition/ Disallow: /stable/book/first-edition/ |