summary refs log tree commit diff
path: root/library/std/src/sys/unix
AgeCommit message (Collapse)AuthorLines
2020-12-21Auto merge of #80088 - operutka:fix-cmsg-len-uclibc, r=dtolnaybors-1/+5
Fix failing build of std on armv5te-unknown-linux-uclibceabi due to missing cmsg_len_zero I'm getting the following error when trying to build `std` on `armv5te-unknown-linux-uclibceabi`: ``` error[E0425]: cannot find value `cmsg_len_zero` in this scope --> /home/operutka/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/unix/ext/net/ancillary.rs:376:47 | 376 | let data_len = (*cmsg).cmsg_len - cmsg_len_zero; | ^^^^^^^^^^^^^ not found in this scope ``` Obviously, this branch: ```rust cfg_if::cfg_if! { if #[cfg(any(target_os = "android", all(target_os = "linux", target_env = "gnu")))] { let cmsg_len_zero = libc::CMSG_LEN(0) as libc::size_t; } else if #[cfg(any( target_os = "dragonfly", target_os = "emscripten", target_os = "freebsd", all(target_os = "linux", target_env = "musl",), target_os = "netbsd", target_os = "openbsd", ))] { let cmsg_len_zero = libc::CMSG_LEN(0) as libc::socklen_t; } } ``` does not cover the case `all(target_os = "linux", target_env = "uclibc")`.
2020-12-20Auto merge of #74699 - notriddle:fd-non-negative, r=m-ou-sebors-2/+9
Mark `-1` as an available niche for file descriptors Based on discussion from <https://internals.rust-lang.org/t/can-the-standard-library-shrink-option-file/12768>, the file descriptor `-1` is chosen based on the POSIX API designs that use it as a sentinel to report errors. A bigger niche could've been chosen, particularly on Linux, but would not necessarily be portable. This PR also adds a test case to ensure that the -1 niche (which is kind of hacky and has no obvious test case) works correctly. It requires the "upper" bound, which is actually -1, to be expressed in two's complement.
2020-12-20Check that c_int is i32 in FileDesc::new.Mara Bos-1/+1
2020-12-16Fix failing build of std on armv5te-unknown-linux-uclibceabi due to missing ā†µOndrej Perutka-1/+5
cmsg_len_zero
2020-12-10Rollup merge of #79375 - vext01:kernel-copy-temps, r=bjorn3Tyler Mandry-8/+11
Make the kernel_copy tests more robust/concurrent. These tests write to the same filenames in /tmp and in some cases these files don't get cleaned up properly. This caused issues for us when different users run the tests on the same system, e.g.: ``` ---- sys::unix::kernel_copy::tests::bench_file_to_file_copy stdout ---- thread 'sys::unix::kernel_copy::tests::bench_file_to_file_copy' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', library/std/src/sys/unix/kernel_copy/tests.rs:71:10 ---- sys::unix::kernel_copy::tests::bench_file_to_socket_copy stdout ---- thread 'sys::unix::kernel_copy::tests::bench_file_to_socket_copy' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', library/std/src/sys/unix/kernel_copy/tests.rsšŸ’Æ10 ``` Use `std::sys_common::io__test::tmpdir()` to solve this. CC ``@the8472.``
2020-12-10Fix fd test caseMichael Howell-1/+1
2020-12-10Add safety note to library/std/src/sys/unix/fd.rsMichael Howell-0/+1
Co-authored-by: Elichai Turkel <elichai.turkel@gmail.com>
2020-12-10Mark `-1` as an available niche for file descriptorsMichael Howell-1/+7
Based on discussion from https://internals.rust-lang.org/t/can-the-standard-library-shrink-option-file/12768, the file descriptor -1 is chosen based on the POSIX API designs that use it as a sentinel to report errors. A bigger niche could've been chosen, particularly on Linux, but would not necessarily be portable. This PR also adds a test case to ensure that the -1 niche (which is kind of hacky and has no obvious test case) works correctly. It requires the "upper" bound, which is actually -1, to be expressed in two's complement.
2020-12-09Improve comment grammarThe8472-2/+2
2020-12-09implement better availability probing for copy_file_rangeThe8472-34/+48
previously any attempt to copy to an immutable file (EPERM) would disable copy_file_range support for the whole process.
2020-12-09Auto merge of #79387 - woodruffw-forks:ww/peer-cred-pid-macos, r=Amanieubors-15/+57
ext/ucred: Support PID in peer creds on macOS This is a follow-up to https://github.com/rust-lang/rust/pull/75148 (RFC: https://github.com/rust-lang/rust/issues/42839). The original PR used `getpeereid` on macOS and the BSDs, since they don't (generally) support the `SO_PEERCRED` mechanism that Linux supplies. This PR splits the macOS/iOS implementation of `peer_cred()` from that of the BSDs, since macOS supplies the `LOCAL_PEERPID` sockopt as a source of the missing PID. It also adds a `cfg`-gated tests that ensures that platforms with support for PIDs in `UCred` have the expected data.
2020-12-03Make the kernel_copy tests more robust/concurrent.Edd Barrett-8/+11
These tests write to the same filenames in /tmp and in some cases these files don't get cleaned up properly. This caused issues for us when different users run the tests on the same system, e.g.: ``` ---- sys::unix::kernel_copy::tests::bench_file_to_file_copy stdout ---- thread 'sys::unix::kernel_copy::tests::bench_file_to_file_copy' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', library/std/src/sys/unix/kernel_copy/tests.rs:71:10 ---- sys::unix::kernel_copy::tests::bench_file_to_socket_copy stdout ---- thread 'sys::unix::kernel_copy::tests::bench_file_to_socket_copy' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', library/std/src/sys/unix/kernel_copy/tests.rs:100:10 ``` Use `std::sys_common::io__test::tmpdir()` to solve this.
2020-12-03fix copy specialization not updating Take wrappersThe8472-14/+45
2020-12-02update test to check Take limits after copyingThe8472-2/+9
2020-12-02Auto merge of #69864 - LinkTed:master, r=Amanieubors-1771/+3097
unix: Extend UnixStream and UnixDatagram to send and receive file descriptors Add the functions `recv_vectored_fds` and `send_vectored_fds` to `UnixDatagram` and `UnixStream`. With this functions `UnixDatagram` and `UnixStream` can send and receive file descriptors, by using `recvmsg` and `sendmsg` system call.
2020-12-01Leverage kernel copy for UnixStreamNicolas Koch-0/+29
UDS can be a sendfile destination, just like TCP sockets.
2020-12-01Add benchmark for File to UnixStream copyNicolas Koch-0/+29
2020-11-26Add comment for the previous android bug fixLinkTed-0/+4
2020-11-24Bug fix for android platform, because of the wrong behavior of CMSG_NXTHDRLinkTed-0/+19
2020-11-24ext/ucred: fmt checkWilliam Woodruff-26/+8
2020-11-24ext/ucred: Support PID in peer creds on macOSWilliam Woodruff-5/+65
2020-11-20Auto merge of #79196 - RalfJung:syscall, r=m-ou-sebors-1/+1
unix/weak: pass arguments to syscall at the given type Given that we know the type the argument should have, it seems a bit strange not to use that information. r? `@m-ou-se` `@cuviper`
2020-11-20unix/weak: pass arguments to syscall at the given typeRalf Jung-1/+1
2020-11-19Auto merge of #79060 - dtolnay:symlinkarg, r=Mark-Simulacrumbors-12/+12
Disambiguate symlink argument names The current argument naming in the following standard library functions is horribly ambiguous. - std::os::unix::fs::symlink: https://doc.rust-lang.org/1.47.0/std/os/unix/fs/fn.symlink.html - std::os::windows::fs::symlink_file: https://doc.rust-lang.org/1.47.0/std/os/windows/fs/fn.symlink_file.html - std::os::windows::fs::symlink_dir: https://doc.rust-lang.org/1.47.0/std/os/windows/fs/fn.symlink_dir.html **Notice that Swift uses one of the same names we do (`dst`) to refer to the opposite thing.** <br> | | the&nbsp;one&nbsp;that&nbsp;exists | the&nbsp;one&nbsp;that&nbsp;is<br>being&nbsp;created | reference | | --- | --- | --- | --- | | Rust | `src` | `dst` | | | Swift | `withDestinationPath`<br>`destPath` | `atPath`<br>`path` | <sub>https://developer.apple.com/documentation/foundation/filemanager/1411007-createsymboliclink</sub> | | D | `original` | `link` | <sub>https://dlang.org/library/std/file/symlink.html</sub> | | Go | `oldname` | `newname` | <sub>https://golang.org/pkg/os/#Symlink</sub> | | C++| `target` | `link` | <sub>https://en.cppreference.com/w/cpp/filesystem/create_symlink</sub> | | POSIX | `path1` | `path2` | <sub>https://pubs.opengroup.org/onlinepubs/9699919799/functions/symlink.html</sub> | | Linux | `target` | `linkpath` | <sub>https://man7.org/linux/man-pages/man2/symlink.2.html</sub> | Out of these I happen to like D's argument names and am proposing that we adopt them.
2020-11-18Rollup merge of #79039 - thomcc:weakly-relaxing, r=AmanieuMara Bos-6/+40
Tighten the bounds on atomic Ordering in std::sys::unix::weak::Weak This moves reading this from multiple SeqCst reads to Relaxed read + Acquire fence if we are actually going to use the data. Would love to avoid the Acquire fence, but doing so would need Ordering::Consume, which neither Rust, nor LLVM supports (a shame, since this fence is hardly free on ARM, which is what I was hoping to improve). r? ``@Amanieu`` (Sorry for always picking you, but I know a lot of people wouldn't feel comfortable reviewing atomic ordering changes)
2020-11-18Rollup merge of #78785 - cuviper:weak-getrandom, r=m-ou-seMara Bos-18/+35
linux: try to use libc getrandom to allow interposition We'll try to use a weak `getrandom` symbol first, because that allows things like `LD_PRELOAD` interposition. For example, perf measurements might want to disable randomness to get reproducible results. If the weak symbol is not found, we fall back to a raw `SYS_getrandom` call.
2020-11-17Auto merge of #78924 - bjorn3:less_sysroot_build_scripts, r=Mark-Simulacrumbors-0/+52
Make the libstd build script smaller Of all sysroot crates currently only compiler_builtins, miniz_oxide and std require a build script. compiler_builtins uses to conditionally enable certain features and possibly compile a C version ([source](https://github.com/rust-lang/compiler-builtins/blob/63ccaf11f08fb5d0b39cc33884c5a1a63f547ace/build.rs)), miniz_oxide only uses it to detect if liballoc is supported as the MSRV is 1.34.0 instead of the 1.36.0 which stabilized liballoc ([source](https://github.com/Frommi/miniz_oxide/blob/28514ec09f0b1ce74bfb2d561de52a6652ce377a/miniz_oxide/build.rs)). std now only uses it to enable `freebsd12` when the `RUST_STD_FREEBSD_12_ABI` env var is set, to determine if `restricted-std` should be set, to set the `STD_ENV_ARCH` env var identical to `CARGO_CFG_TARGET_ARCH`, and to unconditionally enable `backtrace_in_libstd`. If all build scripts were to be removed, it would be possible for rustc to completely compile it's own sysroot. It currently requires a rustc version that already has an available libstd to compile the build scripts. If rustc can completely compile it's own sysroot, rustbuild could be simplified to not forcefully use the bootstrap compiler for build scripts. `@rustbot` modify labels: +T-compiler +libs-impl
2020-11-16Use syscall! for copy_file_range tooJosh Stone-9/+9
2020-11-16Try weak symbols for all linux syscall! wrappersJosh Stone-17/+8
2020-11-16linux: try to use libc getrandom to allow interpositionJosh Stone-5/+31
We'll try to use a weak `getrandom` symbol first, because that allows things like `LD_PRELOAD` interposition. For example, perf measurements might want to disable randomness to get reproducible results. If the weak symbol is not found, we fall back to a raw `SYS_getrandom` call.
2020-11-15Make the libstd build script smallerbjorn3-0/+52
Remove all rustc-link-lib from the std build script. Also remove use of feature = "restricted-std" where not necessary.
2020-11-14Disambiguate symlink argument namesDavid Tolnay-12/+12
2020-11-14Auto merge of #75272 - the8472:spec-copy, r=KodrAusbors-77/+826
specialize io::copy to use copy_file_range, splice or sendfile Fixes #74426. Also covers #60689 but only as an optimization instead of an official API. The specialization only covers std-owned structs so it should avoid the problems with #71091 Currently linux-only but it should be generalizable to other unix systems that have sendfile/sosplice and similar. There is a bit of optimization potential around the syscall count. Right now it may end up doing more syscalls than the naive copy loop when doing short (<8KiB) copies between file descriptors. The test case executes the following: ``` [pid 103776] statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=17, ...}) = 0 [pid 103776] write(4, "wxyz", 4) = 4 [pid 103776] write(4, "iklmn", 5) = 5 [pid 103776] copy_file_range(3, NULL, 4, NULL, 5, 0) = 5 ``` 0-1 `stat` calls to identify the source file type. 0 if the type can be inferred from the struct from which the FD was extracted š–¬ `write` to drain the `BufReader`/`BufWriter` wrappers. only happen when buffers are present. š–¬ ā‰¾ number of wrappers present. If there is a write buffer it may absorb the read buffer contents first so only result in a single write. Vectored writes would also be an option but that would require more invasive changes to `BufWriter`. š–­ `copy_file_range`/`splice`/`sendfile` until file size, EOF or the byte limit from `Take` is reached. This should generally be *much* more efficient than the read-write loop and also have other benefits such as DMA offload or extent sharing. ## Benchmarks ``` OLD test io::tests::bench_file_to_file_copy ... bench: 21,002 ns/iter (+/- 750) = 6240 MB/s [ext4] test io::tests::bench_file_to_file_copy ... bench: 35,704 ns/iter (+/- 1,108) = 3671 MB/s [btrfs] test io::tests::bench_file_to_socket_copy ... bench: 57,002 ns/iter (+/- 4,205) = 2299 MB/s test io::tests::bench_socket_pipe_socket_copy ... bench: 142,640 ns/iter (+/- 77,851) = 918 MB/s NEW test io::tests::bench_file_to_file_copy ... bench: 14,745 ns/iter (+/- 519) = 8889 MB/s [ext4] test io::tests::bench_file_to_file_copy ... bench: 6,128 ns/iter (+/- 227) = 21389 MB/s [btrfs] test io::tests::bench_file_to_socket_copy ... bench: 13,767 ns/iter (+/- 3,767) = 9520 MB/s test io::tests::bench_socket_pipe_socket_copy ... bench: 26,471 ns/iter (+/- 6,412) = 4951 MB/s ```
2020-11-13Tighten the bounds on atomic Ordering in std::sys::unix::weakThom Chiovoloni-6/+40
2020-11-13Always handle EOVERFLOW by falling back to the generic copy loopThe8472-11/+9
Previously EOVERFLOW handling was only applied for io::copy specialization but not for fs::copy sharing the same code. Additionally we lower the chunk size to 1GB since we have a user report that older kernels may return EINVAL when passing 0x8000_0000 but smaller values succeed.
2020-11-13do direct splice syscall and probe availability to get android builds to workThe8472-4/+72
Android builds use feature level 14, the libc wrapper for splice is gated on feature level 21+ so we have to invoke the syscall directly. Additionally the emulator doesn't seem to support it so we also have to add ENOSYS checks.
2020-11-13move sendfile/splice/copy_file_range into kernel_copy moduleThe8472-153/+150
2020-11-13limit visibility of copy offload helpers to sys::unix moduleThe8472-4/+186
2020-11-13move copy specialization into sys::unix moduleThe8472-15/+427
2020-11-13reduce syscalls by inferring FD types based on source struct instead of ā†µThe8472-27/+56
calling stat() also adds handling for edge-cases involving large sparse files where sendfile could fail with EOVERFLOW
2020-11-13hide unused exports on other platformsThe8472-2/+2
2020-11-13specialize io::copy to use copy_file_range, splice or sendfileThe8472-6/+69
Currently it only applies to linux systems. It can be extended to make use of similar syscalls on other unix systems.
2020-11-12Auto merge of #78965 - jryans:emscripten-threads-libc, r=kennytmbors-20/+42
Update thread and futex APIs to work with Emscripten This updates the thread and futex APIs in `std` to match the APIs exposed by Emscripten. This allows threads to run on `wasm32-unknown-emscripten` and the thread parker to compile without errors related to the missing `futex` module. To make use of this, Rust code must be compiled with `-C target-feature=atomics` and Emscripten must link with `-pthread`. I have confirmed this works well locally when building multithreaded crates. Attempting to enable `std` thread tests currently fails for seemingly obscure reasons and Emscripten is currently disabled in CI, so further work is needed to have proper test coverage here.
2020-11-12Fix timeout conversionJ. Ryan Stinnett-2/+1
2020-11-12Update thread and futex APIs to work with EmscriptenJ. Ryan Stinnett-20/+43
This updates the thread and futex APIs in `std` to match the APIs exposed by Emscripten. This allows threads to run on `wasm32-unknown-emscripten` and the thread parker to compile without errors related to the missing `futex` module. To make use of this, Rust code must be compiled with `-C target-feature=atomics` and Emscripten must link with `-pthread`. I have confirmed this works well locally when building multithreaded crates. Attempting to enable `std` thread tests currently fails for seemingly obscure reasons and Emscripten is currently disabled in CI, so further work is needed to have proper test coverage here.
2020-11-09Rollup merge of #78878 - shepmaster:intersecting-ignores, r=Mark-SimulacrumDylan DPC-10/+15
Avoid overlapping cfg attributes when both macOS and aarch64 r? ``@Mark-Simulacrum``
2020-11-09Rollup merge of #78026 - sunfishcode:symlink-hard-link, r=dtolnayDylan DPC-1/+14
Define `fs::hard_link` to not follow symlinks. POSIX leaves it [implementation-defined] whether `link` follows symlinks. In practice, for example, on Linux it does not and on FreeBSD it does. So, switch to `linkat`, so that we can pick a behavior rather than depending on OS defaults. Pick the option to not follow symlinks. This is somewhat arbitrary, but seems the less surprising choice because hard linking is a very low-level feature which requires the source and destination to be on the same mounted filesystem, and following a symbolic link could end up in a different mounted filesystem. [implementation-defined]: https://pubs.opengroup.org/onlinepubs/9699919799/functions/link.html
2020-11-08Avoid overlapping cfg attributes when both macOS and aarch64Jake Goulding-10/+15
2020-11-08Rollup merge of #78572 - de-vri-es:bsd-cloexec, r=m-ou-seMara Bos-6/+38
Use SOCK_CLOEXEC and accept4() on more platforms. This PR enables the use of `SOCK_CLOEXEC` and `accept4` on more platforms. ----- Android uses the linux kernel, so it should also support it. DragonflyBSD introduced them in 4.4 (December 2015): https://www.dragonflybsd.org/release44/ FreeBSD introduced them in 10.0 (January 2014): https://wiki.freebsd.org/AtomicCloseOnExec Illumos introduced them in a commit in April 2013, not sure when it was released. It is quite possible that is has always been in Illumos: https://github.com/illumos/illumos-gate/commit/5dbfd19ad5fcc2b779f40f80fa05c1bd28fd0b4e https://illumos.org/man/3socket/socket https://illumos.org/man/3socket/accept4 NetBSD introduced them in 6.0 (Oktober 2012) and 8.0 (July 2018): https://man.netbsd.org/NetBSD-6.0/socket.2 https://man.netbsd.org/NetBSD-8.0/accept.2 OpenBSD introduced them in 5.7 (May 2015): https://man.openbsd.org/socket https://man.openbsd.org/accept
2020-11-06Disable accept4 on Android.Maarten de Vries-1/+7
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-07-13 02:19:56 +0000