diff options
Diffstat (limited to 'src/vendor/libc/ci')
24 files changed, 828 insertions, 0 deletions
diff --git a/src/vendor/libc/ci/README.md b/src/vendor/libc/ci/README.md new file mode 100644 index 00000000000..13c7c8da52f --- /dev/null +++ b/src/vendor/libc/ci/README.md @@ -0,0 +1,203 @@ +The goal of the libc crate is to have CI running everywhere to have the +strongest guarantees about the definitions that this library contains, and as a +result the CI is pretty complicated and also pretty large! Hopefully this can +serve as a guide through the sea of scripts in this directory and elsewhere in +this project. + +# Files + +First up, let's talk about the files in this directory: + +* `run-travis.sh` - a shell script run by all Travis builders, this is + responsible for setting up the rest of the environment such as installing new + packages, downloading Rust target libraries, etc. + +* `run.sh` - the actual script which runs tests for a particular architecture. + Called from the `run-travis.sh` script this will run all tests for the target + specified. + +* `cargo-config` - Cargo configuration of linkers to use copied into place by + the `run-travis.sh` script before builds are run. + +* `dox.sh` - script called from `run-travis.sh` on only the linux 64-bit nightly + Travis bots to build documentation for this crate. + +* `landing-page-*.html` - used by `dox.sh` to generate a landing page for all + architectures' documentation. + +* `run-qemu.sh` - see discussion about QEMU below + +* `mips`, `rumprun` - instructions to build the docker image for each respective + CI target + +# CI Systems + +Currently this repository leverages a combination of Travis CI and AppVeyor for +running tests. The triples tested are: + +* AppVeyor + * `{i686,x86_64}-pc-windows-{msvc,gnu}` +* Travis + * `{i686,x86_64,mips,aarch64}-unknown-linux-gnu` + * `x86_64-unknown-linux-musl` + * `arm-unknown-linux-gnueabihf` + * `arm-linux-androideabi` + * `{i686,x86_64}-apple-{darwin,ios}` + * `x86_64-rumprun-netbsd` + * `x86_64-unknown-freebsd` + * `x86_64-unknown-openbsd` + +The Windows triples are all pretty standard, they just set up their environment +then run tests, no need for downloading any extra target libs (we just download +the right installer). The Intel Linux/OSX builds are similar in that we just +download the right target libs and run tests. Note that the Intel Linux/OSX +builds are run on stable/beta/nightly, but are the only ones that do so. + +The remaining architectures look like: + +* Android runs in a [docker image][android-docker] with an emulator, the NDK, + and the SDK already set up. The entire build happens within the docker image. +* The MIPS, ARM, and AArch64 builds all use the QEMU userspace emulator to run + the generated binary to actually verify the tests pass. +* The MUSL build just has to download a MUSL compiler and target libraries and + then otherwise runs tests normally. +* iOS builds need an extra linker flag currently, but beyond that they're built + as standard as everything else. +* The rumprun target builds an entire kernel from the test suite and then runs + it inside QEMU using the serial console to test whether it succeeded or + failed. +* The BSD builds, currently OpenBSD and FreeBSD, use QEMU to boot up a system + and compile/run tests. More information on that below. + +[android-docker]: https://github.com/rust-lang/rust-buildbot/blob/master/slaves/android/Dockerfile + +## QEMU + +Lots of the architectures tested here use QEMU in the tests, so it's worth going +over all the crazy capabilities QEMU has and the various flavors in which we use +it! + +First up, QEMU has userspace emulation where it doesn't boot a full kernel, it +just runs a binary from another architecture (using the `qemu-<arch>` wrappers). +We provide it the runtime path for the dynamically loaded system libraries, +however. This strategy is used for all Linux architectures that aren't intel. +Note that one downside of this QEMU system is that threads are barely +implemented, so we're careful to not spawn many threads. + +For the rumprun target the only output is a kernel image, so we just use that +plus the `rumpbake` command to create a full kernel image which is then run from +within QEMU. + +Finally, the fun part, the BSDs. Quite a few hoops are jumped through to get CI +working for these platforms, but the gist of it looks like: + +* Cross compiling from Linux to any of the BSDs seems to be quite non-standard. + We may be able to get it working but it might be difficult at that point to + ensure that the libc definitions align with what you'd get on the BSD itself. + As a result, we try to do compiles within the BSD distro. +* On Travis we can't run a VM-in-a-VM, so we resort to userspace emulation + (QEMU). +* Unfortunately on Travis we also can't use KVM, so the emulation is super slow. + +With all that in mind, the way BSD is tested looks like: + +1. Download a pre-prepared image for the OS being tested. +2. Generate the tests for the OS being tested. This involves running the `ctest` + library over libc to generate a Rust file and a C file which will then be + compiled into the final test. +3. Generate a disk image which will later be mounted by the OS being tested. + This image is mostly just the libc directory, but some modifications are made + to compile the generated files from step 2. +4. The kernel is booted in QEMU, and it is configured to detect the libc-test + image being available, run the test script, and then shut down afterwards. +5. Look for whether the tests passed in the serial console output of the kernel. + +There's some pretty specific instructions for setting up each image (detailed +below), but the main gist of this is that we must avoid a vanilla `cargo run` +inside of the `libc-test` directory (which is what it's intended for) because +that would compile `syntex_syntax`, a large library, with userspace emulation. +This invariably times out on Travis, so we can't do that. + +Once all those hoops are jumped through, however, we can be happy that we're +testing almost everything! + +Below are some details of how to set up the initial OS images which are +downloaded. Each image must be enabled have input/output over the serial +console, log in automatically at the serial console, detect if a second drive in +QEMU is available, and if so mount it, run a script (it'll specifically be +`run-qemu.sh` in this folder which is copied into the generated image talked +about above), and then shut down. + +### QEMU setup - FreeBSD + +1. Download CD installer (most minimal is fine) +2. `qemu-img create -f qcow2 foo.qcow2 2G` +3. `qemu -cdrom foo.iso -drive if=virtio,file=foo.qcow2 -net nic,model=virtio -net user` +4. run installer +5. `echo 'console="comconsole"' >> /boot/loader.conf` +6. `echo 'autoboot_delay="0"' >> /boot/loader.conf` +7. look at /etc/ttys, see what getty argument is for ttyu0 +8. edit /etc/gettytab, look for ttyu0 argument, prepend `:al=root` to line + beneath + +(note that the current image has a `freebsd` user, but this isn't really +necessary) + +Once that's done, arrange for this script to run at login: + +``` +#!/bin/sh + +sudo kldload ext2fs +[ -e /dev/vtbd1 ] || exit 0 +sudo mount -t ext2fs /dev/vtbd1 /mnt +sh /mnt/run.sh /mnt +sudo poweroff +``` + +Helpful links + +* https://en.wikibooks.org/wiki/QEMU/Images +* https://blog.nekoconeko.nl/blog/2015/06/04/creating-an-openstack-freebsd-image.html +* https://www.freebsd.org/doc/handbook/serialconsole-setup.html + + +### QEMU setup - OpenBSD + +1. Download CD installer +2. `qemu-img create -f qcow2 foo.qcow2 2G` +3. `qemu -cdrom foo.iso -drive if=virtio,file=foo.qcow2 -net nic,model=virtio -net user` +4. run installer +5. `echo 'set tty com0' >> /etc/boot.conf` +6. `echo 'boot' >> /etc/boot.conf` +7. Modify /etc/ttys, change the `tty00` at the end from 'unknown off' to + 'vt220 on secure' +8. Modify same line in /etc/ttys to have `"/root/foo.sh"` as the shell +9. Add this script to `/root/foo.sh` + +``` +#!/bin/sh +exec 1>/dev/tty00 +exec 2>&1 + +if mount -t ext2fs /dev/sd1c /mnt; then + sh /mnt/run.sh /mnt + shutdown -ph now +fi + +# limited shell... +exec /bin/sh < /dev/tty00 +``` + +10. `chmod +x /root/foo.sh` + +Helpful links: + +* https://en.wikibooks.org/wiki/QEMU/Images +* http://www.openbsd.org/faq/faq7.html#SerCon + +# Questions? + +Hopefully that's at least somewhat of an introduction to everything going on +here, and feel free to ping @alexcrichton with questions! + diff --git a/src/vendor/libc/ci/docker/aarch64-unknown-linux-gnu/Dockerfile b/src/vendor/libc/ci/docker/aarch64-unknown-linux-gnu/Dockerfile new file mode 100644 index 00000000000..2ba69e15442 --- /dev/null +++ b/src/vendor/libc/ci/docker/aarch64-unknown-linux-gnu/Dockerfile @@ -0,0 +1,7 @@ +FROM ubuntu:16.10 +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc libc6-dev ca-certificates \ + gcc-aarch64-linux-gnu libc6-dev-arm64-cross qemu-user +ENV CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc \ + PATH=$PATH:/rust/bin diff --git a/src/vendor/libc/ci/docker/arm-linux-androideabi/Dockerfile b/src/vendor/libc/ci/docker/arm-linux-androideabi/Dockerfile new file mode 100644 index 00000000000..0e41ba6dbee --- /dev/null +++ b/src/vendor/libc/ci/docker/arm-linux-androideabi/Dockerfile @@ -0,0 +1,4 @@ +FROM alexcrichton/rust-slave-android:2015-11-22 +ENV CARGO_TARGET_ARM_LINUX_ANDROIDEABI_LINKER=arm-linux-androideabi-gcc \ + PATH=$PATH:/rust/bin +ENTRYPOINT ["sh"] diff --git a/src/vendor/libc/ci/docker/arm-unknown-linux-gnueabihf/Dockerfile b/src/vendor/libc/ci/docker/arm-unknown-linux-gnueabihf/Dockerfile new file mode 100644 index 00000000000..3824c046640 --- /dev/null +++ b/src/vendor/libc/ci/docker/arm-unknown-linux-gnueabihf/Dockerfile @@ -0,0 +1,7 @@ +FROM ubuntu:16.10 +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc libc6-dev ca-certificates \ + gcc-arm-linux-gnueabihf libc6-dev-armhf-cross qemu-user +ENV CARGO_TARGET_ARM_UNKNOWN_LINUX_GNUEABIHF_LINKER=arm-linux-gnueabihf-gcc \ + PATH=$PATH:/rust/bin diff --git a/src/vendor/libc/ci/docker/i686-unknown-linux-gnu/Dockerfile b/src/vendor/libc/ci/docker/i686-unknown-linux-gnu/Dockerfile new file mode 100644 index 00000000000..c149d840729 --- /dev/null +++ b/src/vendor/libc/ci/docker/i686-unknown-linux-gnu/Dockerfile @@ -0,0 +1,5 @@ +FROM ubuntu:16.10 +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc-multilib libc6-dev ca-certificates +ENV PATH=$PATH:/rust/bin diff --git a/src/vendor/libc/ci/docker/i686-unknown-linux-musl/Dockerfile b/src/vendor/libc/ci/docker/i686-unknown-linux-musl/Dockerfile new file mode 100644 index 00000000000..87459a1672b --- /dev/null +++ b/src/vendor/libc/ci/docker/i686-unknown-linux-musl/Dockerfile @@ -0,0 +1,22 @@ +FROM ubuntu:16.10 + +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc make libc6-dev git curl ca-certificates +# Below we're cross-compiling musl for i686 using the system compiler on an +# x86_64 system. This is an awkward thing to be doing and so we have to jump +# through a couple hoops to get musl to be happy. In particular: +# +# * We specifically pass -m32 in CFLAGS and override CC when running ./configure, +# since otherwise the script will fail to find a compiler. +# * We manually unset CROSS_COMPILE when running make; otherwise the makefile +# will call the non-existent binary 'i686-ar'. +RUN curl https://www.musl-libc.org/releases/musl-1.1.15.tar.gz | \ + tar xzf - && \ + cd musl-1.1.15 && \ + CC=gcc CFLAGS=-m32 ./configure --prefix=/musl-i686 --disable-shared --target=i686 && \ + make CROSS_COMPILE= install -j4 && \ + cd .. && \ + rm -rf musl-1.1.15 +ENV PATH=$PATH:/musl-i686/bin:/rust/bin \ + CC_i686_unknown_linux_musl=musl-gcc diff --git a/src/vendor/libc/ci/docker/mips-unknown-linux-gnu/Dockerfile b/src/vendor/libc/ci/docker/mips-unknown-linux-gnu/Dockerfile new file mode 100644 index 00000000000..eea1f652c3c --- /dev/null +++ b/src/vendor/libc/ci/docker/mips-unknown-linux-gnu/Dockerfile @@ -0,0 +1,10 @@ +FROM ubuntu:16.10 + +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc libc6-dev qemu-user ca-certificates \ + gcc-mips-linux-gnu libc6-dev-mips-cross \ + qemu-system-mips + +ENV CARGO_TARGET_MIPS_UNKNOWN_LINUX_GNU_LINKER=mips-linux-gnu-gcc \ + PATH=$PATH:/rust/bin diff --git a/src/vendor/libc/ci/docker/mips-unknown-linux-musl/Dockerfile b/src/vendor/libc/ci/docker/mips-unknown-linux-musl/Dockerfile new file mode 100644 index 00000000000..77c6adb435f --- /dev/null +++ b/src/vendor/libc/ci/docker/mips-unknown-linux-musl/Dockerfile @@ -0,0 +1,14 @@ +FROM ubuntu:16.10 + +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc libc6-dev qemu-user ca-certificates qemu-system-mips curl \ + bzip2 + +RUN mkdir /toolchain +RUN curl -L https://downloads.openwrt.org/snapshots/trunk/ar71xx/generic/OpenWrt-SDK-ar71xx-generic_gcc-5.3.0_musl-1.1.15.Linux-x86_64.tar.bz2 | \ + tar xjf - -C /toolchain --strip-components=1 + +ENV PATH=$PATH:/rust/bin:/toolchain/staging_dir/toolchain-mips_34kc_gcc-5.3.0_musl-1.1.15/bin \ + CC_mips_unknown_linux_musl=mips-openwrt-linux-gcc \ + CARGO_TARGET_MIPS_UNKNOWN_LINUX_MUSL_LINKER=mips-openwrt-linux-gcc diff --git a/src/vendor/libc/ci/docker/mips64-unknown-linux-gnuabi64/Dockerfile b/src/vendor/libc/ci/docker/mips64-unknown-linux-gnuabi64/Dockerfile new file mode 100644 index 00000000000..2eb5de24538 --- /dev/null +++ b/src/vendor/libc/ci/docker/mips64-unknown-linux-gnuabi64/Dockerfile @@ -0,0 +1,11 @@ +FROM ubuntu:16.10 + +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc libc6-dev qemu-user ca-certificates \ + gcc-mips64-linux-gnuabi64 libc6-dev-mips64-cross \ + qemu-system-mips64 + +ENV CARGO_TARGET_MIPS64_UNKNOWN_LINUX_GNUABI64_LINKER=mips64-linux-gnuabi64-gcc \ + CC_mips64_unknown_linux_gnuabi64=mips64-linux-gnuabi64-gcc \ + PATH=$PATH:/rust/bin diff --git a/src/vendor/libc/ci/docker/mipsel-unknown-linux-musl/Dockerfile b/src/vendor/libc/ci/docker/mipsel-unknown-linux-musl/Dockerfile new file mode 100644 index 00000000000..36c4d90ef68 --- /dev/null +++ b/src/vendor/libc/ci/docker/mipsel-unknown-linux-musl/Dockerfile @@ -0,0 +1,14 @@ +FROM ubuntu:16.10 + +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc libc6-dev qemu-user ca-certificates qemu-system-mips curl \ + bzip2 + +RUN mkdir /toolchain +RUN curl -L https://downloads.openwrt.org/snapshots/trunk/malta/generic/OpenWrt-Toolchain-malta-le_gcc-5.3.0_musl-1.1.15.Linux-x86_64.tar.bz2 | \ + tar xjf - -C /toolchain --strip-components=2 + +ENV PATH=$PATH:/rust/bin:/toolchain/bin \ + CC_mipsel_unknown_linux_musl=mipsel-openwrt-linux-gcc \ + CARGO_TARGET_MIPSEL_UNKNOWN_LINUX_MUSL_LINKER=mipsel-openwrt-linux-gcc diff --git a/src/vendor/libc/ci/docker/powerpc-unknown-linux-gnu/Dockerfile b/src/vendor/libc/ci/docker/powerpc-unknown-linux-gnu/Dockerfile new file mode 100644 index 00000000000..d9d7db0f41d --- /dev/null +++ b/src/vendor/libc/ci/docker/powerpc-unknown-linux-gnu/Dockerfile @@ -0,0 +1,10 @@ +FROM ubuntu:16.10 + +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc libc6-dev qemu-user ca-certificates \ + gcc-powerpc-linux-gnu libc6-dev-powerpc-cross \ + qemu-system-ppc + +ENV CARGO_TARGET_POWERPC_UNKNOWN_LINUX_GNU_LINKER=powerpc-linux-gnu-gcc \ + PATH=$PATH:/rust/bin diff --git a/src/vendor/libc/ci/docker/powerpc64-unknown-linux-gnu/Dockerfile b/src/vendor/libc/ci/docker/powerpc64-unknown-linux-gnu/Dockerfile new file mode 100644 index 00000000000..df0e6057b4f --- /dev/null +++ b/src/vendor/libc/ci/docker/powerpc64-unknown-linux-gnu/Dockerfile @@ -0,0 +1,11 @@ +FROM ubuntu:16.10 + +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc libc6-dev qemu-user ca-certificates \ + gcc-powerpc64-linux-gnu libc6-dev-ppc64-cross \ + qemu-system-ppc + +ENV CARGO_TARGET_POWERPC64_UNKNOWN_LINUX_GNU_LINKER=powerpc64-linux-gnu-gcc \ + CC=powerpc64-linux-gnu-gcc \ + PATH=$PATH:/rust/bin diff --git a/src/vendor/libc/ci/docker/x86_64-rumprun-netbsd/Dockerfile b/src/vendor/libc/ci/docker/x86_64-rumprun-netbsd/Dockerfile new file mode 100644 index 00000000000..129771e76b7 --- /dev/null +++ b/src/vendor/libc/ci/docker/x86_64-rumprun-netbsd/Dockerfile @@ -0,0 +1,6 @@ +FROM mato/rumprun-toolchain-hw-x86_64 +USER root +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + qemu +ENV PATH=$PATH:/rust/bin diff --git a/src/vendor/libc/ci/docker/x86_64-unknown-freebsd/Dockerfile b/src/vendor/libc/ci/docker/x86_64-unknown-freebsd/Dockerfile new file mode 100644 index 00000000000..b1273382223 --- /dev/null +++ b/src/vendor/libc/ci/docker/x86_64-unknown-freebsd/Dockerfile @@ -0,0 +1,13 @@ +FROM alexcrichton/rust-slave-linux-cross:2016-04-15 +USER root + +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + qemu genext2fs + +ENTRYPOINT ["sh"] + +ENV PATH=$PATH:/rust/bin \ + QEMU=freebsd.qcow2.gz \ + CAN_CROSS=1 \ + CARGO_TARGET_X86_64_UNKNOWN_FREEBSD_LINKER=x86_64-unknown-freebsd10-gcc diff --git a/src/vendor/libc/ci/docker/x86_64-unknown-linux-gnu/Dockerfile b/src/vendor/libc/ci/docker/x86_64-unknown-linux-gnu/Dockerfile new file mode 100644 index 00000000000..4af3f834cbe --- /dev/null +++ b/src/vendor/libc/ci/docker/x86_64-unknown-linux-gnu/Dockerfile @@ -0,0 +1,5 @@ +FROM ubuntu:16.10 +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc libc6-dev ca-certificates +ENV PATH=$PATH:/rust/bin diff --git a/src/vendor/libc/ci/docker/x86_64-unknown-linux-musl/Dockerfile b/src/vendor/libc/ci/docker/x86_64-unknown-linux-musl/Dockerfile new file mode 100644 index 00000000000..9c2499948a2 --- /dev/null +++ b/src/vendor/libc/ci/docker/x86_64-unknown-linux-musl/Dockerfile @@ -0,0 +1,13 @@ +FROM ubuntu:16.10 + +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc make libc6-dev git curl ca-certificates +RUN curl https://www.musl-libc.org/releases/musl-1.1.15.tar.gz | \ + tar xzf - && \ + cd musl-1.1.15 && \ + ./configure --prefix=/musl-x86_64 && \ + make install -j4 && \ + cd .. && \ + rm -rf musl-1.1.15 +ENV PATH=$PATH:/musl-x86_64/bin:/rust/bin diff --git a/src/vendor/libc/ci/docker/x86_64-unknown-openbsd/Dockerfile b/src/vendor/libc/ci/docker/x86_64-unknown-openbsd/Dockerfile new file mode 100644 index 00000000000..26340a5ed1e --- /dev/null +++ b/src/vendor/libc/ci/docker/x86_64-unknown-openbsd/Dockerfile @@ -0,0 +1,8 @@ +FROM ubuntu:16.10 + +RUN apt-get update +RUN apt-get install -y --no-install-recommends \ + gcc libc6-dev qemu curl ca-certificates \ + genext2fs +ENV PATH=$PATH:/rust/bin \ + QEMU=2016-09-07/openbsd-6.0-without-pkgs.qcow2 diff --git a/src/vendor/libc/ci/dox.sh b/src/vendor/libc/ci/dox.sh new file mode 100644 index 00000000000..88d882dcacd --- /dev/null +++ b/src/vendor/libc/ci/dox.sh @@ -0,0 +1,33 @@ +#!/bin/sh + +# Builds documentation for all target triples that we have a registered URL for +# in liblibc. This scrapes the list of triples to document from `src/lib.rs` +# which has a bunch of `html_root_url` directives we pick up. + +set -e + +TARGETS=`grep html_root_url src/lib.rs | sed 's/.*".*\/\(.*\)"/\1/'` + +rm -rf target/doc +mkdir -p target/doc + +cp ci/landing-page-head.html target/doc/index.html + +for target in $TARGETS; do + echo documenting $target + + rustdoc -o target/doc/$target --target $target src/lib.rs --cfg dox \ + --crate-name libc + + echo "<li><a href="/libc/$target/libc/index.html">$target</a></li>" \ + >> target/doc/index.html +done + +cat ci/landing-page-footer.html >> target/doc/index.html + +# If we're on travis, not a PR, and on the right branch, publish! +if [ "$TRAVIS_PULL_REQUEST" = "false" ] && [ "$TRAVIS_BRANCH" = "master" ]; then + pip install ghp-import --user $USER + $HOME/.local/bin/ghp-import -n target/doc + git push -qf https://${GH_TOKEN}@github.com/${TRAVIS_REPO_SLUG}.git gh-pages +fi diff --git a/src/vendor/libc/ci/landing-page-footer.html b/src/vendor/libc/ci/landing-page-footer.html new file mode 100644 index 00000000000..941cc8d2b40 --- /dev/null +++ b/src/vendor/libc/ci/landing-page-footer.html @@ -0,0 +1,3 @@ + </ul> + </body> +</html> diff --git a/src/vendor/libc/ci/landing-page-head.html b/src/vendor/libc/ci/landing-page-head.html new file mode 100644 index 00000000000..fc69fa88eb5 --- /dev/null +++ b/src/vendor/libc/ci/landing-page-head.html @@ -0,0 +1,7 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8"> + </head> + <body> + <ul> diff --git a/src/vendor/libc/ci/run-docker.sh b/src/vendor/libc/ci/run-docker.sh new file mode 100644 index 00000000000..e34e65ffcdc --- /dev/null +++ b/src/vendor/libc/ci/run-docker.sh @@ -0,0 +1,26 @@ +# Small script to run tests for a target (or all targets) inside all the +# respective docker images. + +set -ex + +run() { + echo $1 + docker build -t libc ci/docker/$1 + docker run \ + --rm \ + -v `rustc --print sysroot`:/rust:ro \ + -v `pwd`:/checkout:ro \ + -e CARGO_TARGET_DIR=/tmp/target \ + -w /checkout \ + --privileged \ + -it libc \ + ci/run.sh $1 +} + +if [ -z "$1" ]; then + for d in `ls ci/docker/`; do + run $d + done +else + run $1 +fi diff --git a/src/vendor/libc/ci/run-qemu.sh b/src/vendor/libc/ci/run-qemu.sh new file mode 100644 index 00000000000..b2f457df916 --- /dev/null +++ b/src/vendor/libc/ci/run-qemu.sh @@ -0,0 +1,32 @@ +# Initial script which is run inside of all qemu images. The first argument to +# this script (as arranged by the qemu image itself) is the path to where the +# libc crate is mounted. +# +# For qemu images we currently need to install Rust manually as this wasn't done +# by the initial run-travis.sh script +# +# FIXME: feels like run-travis.sh should be responsible for downloading the +# compiler. + +set -ex + +ROOT=$1 +cp -r $ROOT/libc /tmp/libc +cd /tmp/libc + +TARGET=$(cat $ROOT/TARGET) +export CARGO_TARGET_DIR=/tmp + +case $TARGET in + *-openbsd) + pkg_add cargo gcc%4.9 rust + export CC=egcc + ;; + + *) + echo "Unknown target: $TARGET" + exit 1 + ;; +esac + +exec sh ci/run.sh $TARGET diff --git a/src/vendor/libc/ci/run.sh b/src/vendor/libc/ci/run.sh new file mode 100755 index 00000000000..15721ab965d --- /dev/null +++ b/src/vendor/libc/ci/run.sh @@ -0,0 +1,160 @@ +#!/bin/sh + +# Builds and runs tests for a particular target passed as an argument to this +# script. + +set -ex + +TARGET=$1 + +# If we're going to run tests inside of a qemu image, then we don't need any of +# the scripts below. Instead, download the image, prepare a filesystem which has +# the current state of this repository, and then run the image. +# +# It's assume that all images, when run with two disks, will run the `run.sh` +# script from the second which we place inside. +if [ "$QEMU" != "" ]; then + tmpdir=/tmp/qemu-img-creation + mkdir -p $tmpdir + + if [ -z "${QEMU#*.gz}" ]; then + # image is .gz : download and uncompress it + qemufile=$(echo ${QEMU%.gz} | sed 's/\//__/g') + if [ ! -f $tmpdir/$qemufile ]; then + curl https://people.mozilla.org/~acrichton/libc-test/qemu/$QEMU | \ + gunzip -d > $tmpdir/$qemufile + fi + else + # plain qcow2 image: just download it + qemufile=$(echo ${QEMU} | sed 's/\//__/g') + if [ ! -f $tmpdir/$qemufile ]; then + curl https://people.mozilla.org/~acrichton/libc-test/qemu/$QEMU \ + > $tmpdir/$qemufile + fi + fi + + # Create a mount a fresh new filesystem image that we'll later pass to QEMU. + # This will have a `run.sh` script will which use the artifacts inside to run + # on the host. + rm -f $tmpdir/libc-test.img + mkdir $tmpdir/mount + + # If we have a cross compiler, then we just do the standard rigamarole of + # cross-compiling an executable and then the script to run just executes the + # binary. + # + # If we don't have a cross-compiler, however, then we need to do some crazy + # acrobatics to get this to work. Generate all.{c,rs} on the host which will + # be compiled inside QEMU. Do this here because compiling syntex_syntax in + # QEMU would time out basically everywhere. + if [ "$CAN_CROSS" = "1" ]; then + cargo build --manifest-path libc-test/Cargo.toml --target $TARGET + cp $CARGO_TARGET_DIR/$TARGET/debug/libc-test $tmpdir/mount/ + echo 'exec $1/libc-test' > $tmpdir/mount/run.sh + else + rm -rf $tmpdir/generated + mkdir -p $tmpdir/generated + cargo build --manifest-path libc-test/generate-files/Cargo.toml + (cd libc-test && TARGET=$TARGET OUT_DIR=$tmpdir/generated SKIP_COMPILE=1 \ + $CARGO_TARGET_DIR/debug/generate-files) + + # Copy this folder into the mounted image, the `run.sh` entry point, and + # overwrite the standard libc-test Cargo.toml with the overlay one which will + # assume the all.{c,rs} test files have already been generated + mkdir $tmpdir/mount/libc + cp -r Cargo.* libc-test src ci $tmpdir/mount/libc/ + ln -s libc-test/target $tmpdir/mount/libc/target + cp ci/run-qemu.sh $tmpdir/mount/run.sh + echo $TARGET | tee -a $tmpdir/mount/TARGET + cp $tmpdir/generated/* $tmpdir/mount/libc/libc-test + cp libc-test/run-generated-Cargo.toml $tmpdir/mount/libc/libc-test/Cargo.toml + fi + + du -sh $tmpdir/mount + genext2fs \ + --root $tmpdir/mount \ + --size-in-blocks 100000 \ + $tmpdir/libc-test.img + + # Pass -snapshot to prevent tampering with the disk images, this helps when + # running this script in development. The two drives are then passed next, + # first is the OS and second is the one we just made. Next the network is + # configured to work (I'm not entirely sure how), and then finally we turn off + # graphics and redirect the serial console output to out.log. + qemu-system-x86_64 \ + -m 1024 \ + -snapshot \ + -drive if=virtio,file=$tmpdir/$qemufile \ + -drive if=virtio,file=$tmpdir/libc-test.img \ + -net nic,model=virtio \ + -net user \ + -nographic \ + -vga none 2>&1 | tee $CARGO_TARGET_DIR/out.log + exec grep "^PASSED .* tests" $CARGO_TARGET_DIR/out.log +fi + +case "$TARGET" in + *-apple-ios) + cargo rustc --manifest-path libc-test/Cargo.toml --target $TARGET -- \ + -C link-args=-mios-simulator-version-min=7.0 + ;; + + *) + cargo build --manifest-path libc-test/Cargo.toml --target $TARGET + ;; +esac + +case "$TARGET" in + arm-linux-androideabi) + emulator @arm-21 -no-window & + adb wait-for-device + adb push $CARGO_TARGET_DIR/$TARGET/debug/libc-test /data/libc-test + adb shell /data/libc-test 2>&1 | tee /tmp/out + grep "^PASSED .* tests" /tmp/out + ;; + + arm-unknown-linux-gnueabihf) + qemu-arm -L /usr/arm-linux-gnueabihf $CARGO_TARGET_DIR/$TARGET/debug/libc-test + ;; + + mips-unknown-linux-gnu) + qemu-mips -L /usr/mips-linux-gnu $CARGO_TARGET_DIR/$TARGET/debug/libc-test + ;; + + mips64-unknown-linux-gnuabi64) + qemu-mips64 -L /usr/mips64-linux-gnuabi64 $CARGO_TARGET_DIR/$TARGET/debug/libc-test + ;; + + mips-unknown-linux-musl) + qemu-mips -L /toolchain/staging_dir/toolchain-mips_34kc_gcc-5.3.0_musl-1.1.15 \ + $CARGO_TARGET_DIR/$TARGET/debug/libc-test + ;; + + mipsel-unknown-linux-musl) + qemu-mipsel -L /toolchain $CARGO_TARGET_DIR/$TARGET/debug/libc-test + ;; + + powerpc-unknown-linux-gnu) + qemu-ppc -L /usr/powerpc-linux-gnu $CARGO_TARGET_DIR/$TARGET/debug/libc-test + ;; + + powerpc64-unknown-linux-gnu) + qemu-ppc64 -L /usr/powerpc64-linux-gnu $CARGO_TARGET_DIR/$TARGET/debug/libc-test + ;; + + aarch64-unknown-linux-gnu) + qemu-aarch64 -L /usr/aarch64-linux-gnu/ $CARGO_TARGET_DIR/$TARGET/debug/libc-test + ;; + + *-rumprun-netbsd) + rumprun-bake hw_virtio /tmp/libc-test.img $CARGO_TARGET_DIR/$TARGET/debug/libc-test + qemu-system-x86_64 -nographic -vga none -m 64 \ + -kernel /tmp/libc-test.img 2>&1 | tee /tmp/out & + sleep 5 + grep "^PASSED .* tests" /tmp/out + ;; + + *) + $CARGO_TARGET_DIR/$TARGET/debug/libc-test + ;; +esac diff --git a/src/vendor/libc/ci/style.rs b/src/vendor/libc/ci/style.rs new file mode 100644 index 00000000000..32e4ba772c5 --- /dev/null +++ b/src/vendor/libc/ci/style.rs @@ -0,0 +1,204 @@ +//! Simple script to verify the coding style of this library +//! +//! ## How to run +//! +//! The first argument to this script is the directory to run on, so running +//! this script should be as simple as: +//! +//! ```notrust +//! rustc ci/style.rs +//! ./style src +//! ``` +//! +//! ## Guidelines +//! +//! The current style is: +//! +//! * No trailing whitespace +//! * No tabs +//! * 80-character lines +//! * `extern` instead of `extern "C"` +//! * Specific module layout: +//! 1. use directives +//! 2. typedefs +//! 3. structs +//! 4. constants +//! 5. f! { ... } functions +//! 6. extern functions +//! 7. modules + pub use +//! +//! Things not verified: +//! +//! * alignment +//! * 4-space tabs +//! * leading colons on paths + +use std::env; +use std::fs; +use std::io::prelude::*; +use std::path::Path; + +macro_rules! t { + ($e:expr) => (match $e { + Ok(e) => e, + Err(e) => panic!("{} failed with {}", stringify!($e), e), + }) +} + +fn main() { + let arg = env::args().skip(1).next().unwrap_or(".".to_string()); + + let mut errors = Errors { errs: false }; + walk(Path::new(&arg), &mut errors); + + if errors.errs { + panic!("found some lint errors"); + } else { + println!("good style!"); + } +} + +fn walk(path: &Path, err: &mut Errors) { + for entry in t!(path.read_dir()).map(|e| t!(e)) { + let path = entry.path(); + if t!(entry.file_type()).is_dir() { + walk(&path, err); + continue + } + + let name = entry.file_name().into_string().unwrap(); + match &name[..] { + n if !n.ends_with(".rs") => continue, + + "dox.rs" | + "lib.rs" | + "macros.rs" => continue, + + _ => {} + } + + let mut contents = String::new(); + t!(t!(fs::File::open(&path)).read_to_string(&mut contents)); + + check_style(&contents, &path, err); + } +} + +struct Errors { + errs: bool, +} + +#[derive(Clone, Copy, PartialEq)] +enum State { + Start, + Imports, + Typedefs, + Structs, + Constants, + FunctionDefinitions, + Functions, + Modules, +} + +fn check_style(file: &str, path: &Path, err: &mut Errors) { + let mut state = State::Start; + let mut s_macros = 0; + let mut f_macros = 0; + let mut prev_blank = false; + + for (i, line) in file.lines().enumerate() { + if line == "" { + if prev_blank { + err.error(path, i, "double blank line"); + } + prev_blank = true; + } else { + prev_blank = false; + } + if line != line.trim_right() { + err.error(path, i, "trailing whitespace"); + } + if line.contains("\t") { + err.error(path, i, "tab character"); + } + if line.len() > 80 { + err.error(path, i, "line longer than 80 chars"); + } + if line.contains("extern \"C\"") { + err.error(path, i, "use `extern` instead of `extern \"C\""); + } + if line.contains("#[cfg(") && !line.contains(" if ") { + if state != State::Structs { + err.error(path, i, "use cfg_if! and submodules \ + instead of #[cfg]"); + } + } + + let line = line.trim_left(); + let is_pub = line.starts_with("pub "); + let line = if is_pub {&line[4..]} else {line}; + + let line_state = if line.starts_with("use ") { + if is_pub { + State::Modules + } else { + State::Imports + } + } else if line.starts_with("const ") { + State::Constants + } else if line.starts_with("type ") { + State::Typedefs + } else if line.starts_with("s! {") { + s_macros += 1; + State::Structs + } else if line.starts_with("f! {") { + f_macros += 1; + State::FunctionDefinitions + } else if line.starts_with("extern ") { + State::Functions + } else if line.starts_with("mod ") { + State::Modules + } else { + continue + }; + + if state as usize > line_state as usize { + err.error(path, i, &format!("{} found after {} when \ + it belongs before", + line_state.desc(), state.desc())); + } + + if f_macros == 2 { + f_macros += 1; + err.error(path, i, "multiple f! macros in one module"); + } + if s_macros == 2 { + s_macros += 1; + err.error(path, i, "multiple s! macros in one module"); + } + + state = line_state; + } +} + +impl State { + fn desc(&self) -> &str { + match *self { + State::Start => "start", + State::Imports => "import", + State::Typedefs => "typedef", + State::Structs => "struct", + State::Constants => "constant", + State::FunctionDefinitions => "function definition", + State::Functions => "extern function", + State::Modules => "module", + } + } +} + +impl Errors { + fn error(&mut self, path: &Path, line: usize, msg: &str) { + self.errs = true; + println!("{}:{} - {}", path.display(), line + 1, msg); + } +} |