Auto merge of #73265 - mark-i-m:mv-std, r=Mark-Simulacrum,mark-i-m

mv std libs to library/

This is the first step in refactoring the directory layout of this repository, with further followup steps planned (but not done yet).

Background: currently, all crates are under src/, without nested src directories and with the unconventional `lib*` prefixes (e.g., `src/libcore/lib.rs`). This directory structures is not idiomatic and makes the `src/` directory rather overwhelming. To improve contributor experience and make things a bit more approachable, we are reorganizing the repo a bit.

In this PR, we move the standard libs (basically anything that is "runtime", as opposed to part of the compiler, build system, or one of the tools, etc). The new layout moves these libraries to a new `library/` directory in the root of the repo. Additionally, we remove the `lib*` prefixes and add nested `src/` directories.  The other crates/tools in this repo are not touched. So in summary:

```
library/<crate>/src/*.rs
src/<all the rest>     // unchanged
```

where `<crate>` is:
- core
- alloc
- std
- test
- proc_macro
- panic_abort
- panic_unwind
- profiler_builtins
- term
- unwind
- rtstartup
- backtrace
- rustc-std-workspace-*

There was a lot of discussion about this and a few rounds of compiler team approvals, FCPs, MCPs, and nominations. The original MCP is https://github.com/rust-lang/compiler-team/issues/298. The final approval of the compiler team was given here: https://github.com/rust-lang/rust/pull/73265#issuecomment-659498446.

The name `library` was chosen to complement a later move of the compiler crates to a `compiler/` directory. There was a lot of discussion around adding the nested `src/` directories. Note that this does increase the nesting depth (plausibly important for manual traversal of the tree, e.g., through GitHub's UI or `cd`), but this is deemed to be better as it fits the standard layout of Rust crates throughout most of the ecosystem, though there is some debate about how much this should apply to multi-crate projects. Overall, there seem to be more people in favor of nested `src/` than against.

After this PR, there are no dependencies out of the `library/` directory except on the `build_helper` (or crates.io crates).

1 files changed, 382 insertions, 0 deletions

diff --git a/library/std/src/sys/unix/net.rs b/library/std/src/sys/unix/net.rs
new file mode 100644
index 00000000000..011325fddc5
--- /dev/null
+++ b/library/std/src/sys/unix/net.rs
@@ -0,0 +1,382 @@
+use crate::cmp;
+use crate::ffi::CStr;
+use crate::io::{self, IoSlice, IoSliceMut};
+use crate::mem;
+use crate::net::{Shutdown, SocketAddr};
+use crate::str;
+use crate::sys::fd::FileDesc;
+use crate::sys_common::net::{getsockopt, setsockopt, sockaddr_to_addr};
+use crate::sys_common::{AsInner, FromInner, IntoInner};
+use crate::time::{Duration, Instant};
+
+use libc::{c_int, c_void, size_t, sockaddr, socklen_t, EAI_SYSTEM, MSG_PEEK};
+
+pub use crate::sys::{cvt, cvt_r};
+
+#[allow(unused_extern_crates)]
+pub extern crate libc as netc;
+
+pub type wrlen_t = size_t;
+
+pub struct Socket(FileDesc);
+
+pub fn init() {}
+
+pub fn cvt_gai(err: c_int) -> io::Result<()> {
+    if err == 0 {
+        return Ok(());
+    }
+
+    // We may need to trigger a glibc workaround. See on_resolver_failure() for details.
+    on_resolver_failure();
+
+    if err == EAI_SYSTEM {
+        return Err(io::Error::last_os_error());
+    }
+
+    let detail = unsafe {
+        str::from_utf8(CStr::from_ptr(libc::gai_strerror(err)).to_bytes()).unwrap().to_owned()
+    };
+    Err(io::Error::new(
+        io::ErrorKind::Other,
+        &format!("failed to lookup address information: {}", detail)[..],
+    ))
+}
+
+impl Socket {
+    pub fn new(addr: &SocketAddr, ty: c_int) -> io::Result<Socket> {
+        let fam = match *addr {
+            SocketAddr::V4(..) => libc::AF_INET,
+            SocketAddr::V6(..) => libc::AF_INET6,
+        };
+        Socket::new_raw(fam, ty)
+    }
+
+    pub fn new_raw(fam: c_int, ty: c_int) -> io::Result<Socket> {
+        unsafe {
+            cfg_if::cfg_if! {
+                if #[cfg(target_os = "linux")] {
+                    // On Linux we pass the SOCK_CLOEXEC flag to atomically create
+                    // the socket and set it as CLOEXEC, added in 2.6.27.
+                    let fd = cvt(libc::socket(fam, ty | libc::SOCK_CLOEXEC, 0))?;
+                    Ok(Socket(FileDesc::new(fd)))
+                } else {
+                    let fd = cvt(libc::socket(fam, ty, 0))?;
+                    let fd = FileDesc::new(fd);
+                    fd.set_cloexec()?;
+                    let socket = Socket(fd);
+
+                    // macOS and iOS use `SO_NOSIGPIPE` as a `setsockopt`
+                    // flag to disable `SIGPIPE` emission on socket.
+                    #[cfg(target_vendor = "apple")]
+                    setsockopt(&socket, libc::SOL_SOCKET, libc::SO_NOSIGPIPE, 1)?;
+
+                    Ok(socket)
+                }
+            }
+        }
+    }
+
+    pub fn new_pair(fam: c_int, ty: c_int) -> io::Result<(Socket, Socket)> {
+        unsafe {
+            let mut fds = [0, 0];
+
+            cfg_if::cfg_if! {
+                if #[cfg(target_os = "linux")] {
+                    // Like above, set cloexec atomically
+                    cvt(libc::socketpair(fam, ty | libc::SOCK_CLOEXEC, 0, fds.as_mut_ptr()))?;
+                    Ok((Socket(FileDesc::new(fds[0])), Socket(FileDesc::new(fds[1]))))
+                } else {
+                    cvt(libc::socketpair(fam, ty, 0, fds.as_mut_ptr()))?;
+                    let a = FileDesc::new(fds[0]);
+                    let b = FileDesc::new(fds[1]);
+                    a.set_cloexec()?;
+                    b.set_cloexec()?;
+                    Ok((Socket(a), Socket(b)))
+                }
+            }
+        }
+    }
+
+    pub fn connect_timeout(&self, addr: &SocketAddr, timeout: Duration) -> io::Result<()> {
+        self.set_nonblocking(true)?;
+        let r = unsafe {
+            let (addrp, len) = addr.into_inner();
+            cvt(libc::connect(self.0.raw(), addrp, len))
+        };
+        self.set_nonblocking(false)?;
+
+        match r {
+            Ok(_) => return Ok(()),
+            // there's no ErrorKind for EINPROGRESS :(
+            Err(ref e) if e.raw_os_error() == Some(libc::EINPROGRESS) => {}
+            Err(e) => return Err(e),
+        }
+
+        let mut pollfd = libc::pollfd { fd: self.0.raw(), events: libc::POLLOUT, revents: 0 };
+
+        if timeout.as_secs() == 0 && timeout.subsec_nanos() == 0 {
+            return Err(io::Error::new(
+                io::ErrorKind::InvalidInput,
+                "cannot set a 0 duration timeout",
+            ));
+        }
+
+        let start = Instant::now();
+
+        loop {
+            let elapsed = start.elapsed();
+            if elapsed >= timeout {
+                return Err(io::Error::new(io::ErrorKind::TimedOut, "connection timed out"));
+            }
+
+            let timeout = timeout - elapsed;
+            let mut timeout = timeout
+                .as_secs()
+                .saturating_mul(1_000)
+                .saturating_add(timeout.subsec_nanos() as u64 / 1_000_000);
+            if timeout == 0 {
+                timeout = 1;
+            }
+
+            let timeout = cmp::min(timeout, c_int::MAX as u64) as c_int;
+
+            match unsafe { libc::poll(&mut pollfd, 1, timeout) } {
+                -1 => {
+                    let err = io::Error::last_os_error();
+                    if err.kind() != io::ErrorKind::Interrupted {
+                        return Err(err);
+                    }
+                }
+                0 => {}
+                _ => {
+                    // linux returns POLLOUT|POLLERR|POLLHUP for refused connections (!), so look
+                    // for POLLHUP rather than read readiness
+                    if pollfd.revents & libc::POLLHUP != 0 {
+                        let e = self.take_error()?.unwrap_or_else(|| {
+                            io::Error::new(io::ErrorKind::Other, "no error set after POLLHUP")
+                        });
+                        return Err(e);
+                    }
+
+                    return Ok(());
+                }
+            }
+        }
+    }
+
+    pub fn accept(&self, storage: *mut sockaddr, len: *mut socklen_t) -> io::Result<Socket> {
+        // Unfortunately the only known way right now to accept a socket and
+        // atomically set the CLOEXEC flag is to use the `accept4` syscall on
+        // Linux. This was added in 2.6.28, glibc 2.10 and musl 0.9.5.
+        cfg_if::cfg_if! {
+            if #[cfg(target_os = "linux")] {
+                let fd = cvt_r(|| unsafe {
+                    libc::accept4(self.0.raw(), storage, len, libc::SOCK_CLOEXEC)
+                })?;
+                Ok(Socket(FileDesc::new(fd)))
+            } else {
+                let fd = cvt_r(|| unsafe { libc::accept(self.0.raw(), storage, len) })?;
+                let fd = FileDesc::new(fd);
+                fd.set_cloexec()?;
+                Ok(Socket(fd))
+            }
+        }
+    }
+
+    pub fn duplicate(&self) -> io::Result<Socket> {
+        self.0.duplicate().map(Socket)
+    }
+
+    fn recv_with_flags(&self, buf: &mut [u8], flags: c_int) -> io::Result<usize> {
+        let ret = cvt(unsafe {
+            libc::recv(self.0.raw(), buf.as_mut_ptr() as *mut c_void, buf.len(), flags)
+        })?;
+        Ok(ret as usize)
+    }
+
+    pub fn read(&self, buf: &mut [u8]) -> io::Result<usize> {
+        self.recv_with_flags(buf, 0)
+    }
+
+    pub fn peek(&self, buf: &mut [u8]) -> io::Result<usize> {
+        self.recv_with_flags(buf, MSG_PEEK)
+    }
+
+    pub fn read_vectored(&self, bufs: &mut [IoSliceMut<'_>]) -> io::Result<usize> {
+        self.0.read_vectored(bufs)
+    }
+
+    #[inline]
+    pub fn is_read_vectored(&self) -> bool {
+        self.0.is_read_vectored()
+    }
+
+    fn recv_from_with_flags(
+        &self,
+        buf: &mut [u8],
+        flags: c_int,
+    ) -> io::Result<(usize, SocketAddr)> {
+        let mut storage: libc::sockaddr_storage = unsafe { mem::zeroed() };
+        let mut addrlen = mem::size_of_val(&storage) as libc::socklen_t;
+
+        let n = cvt(unsafe {
+            libc::recvfrom(
+                self.0.raw(),
+                buf.as_mut_ptr() as *mut c_void,
+                buf.len(),
+                flags,
+                &mut storage as *mut _ as *mut _,
+                &mut addrlen,
+            )
+        })?;
+        Ok((n as usize, sockaddr_to_addr(&storage, addrlen as usize)?))
+    }
+
+    pub fn recv_from(&self, buf: &mut [u8]) -> io::Result<(usize, SocketAddr)> {
+        self.recv_from_with_flags(buf, 0)
+    }
+
+    pub fn peek_from(&self, buf: &mut [u8]) -> io::Result<(usize, SocketAddr)> {
+        self.recv_from_with_flags(buf, MSG_PEEK)
+    }
+
+    pub fn write(&self, buf: &[u8]) -> io::Result<usize> {
+        self.0.write(buf)
+    }
+
+    pub fn write_vectored(&self, bufs: &[IoSlice<'_>]) -> io::Result<usize> {
+        self.0.write_vectored(bufs)
+    }
+
+    #[inline]
+    pub fn is_write_vectored(&self) -> bool {
+        self.0.is_write_vectored()
+    }
+
+    pub fn set_timeout(&self, dur: Option<Duration>, kind: libc::c_int) -> io::Result<()> {
+        let timeout = match dur {
+            Some(dur) => {
+                if dur.as_secs() == 0 && dur.subsec_nanos() == 0 {
+                    return Err(io::Error::new(
+                        io::ErrorKind::InvalidInput,
+                        "cannot set a 0 duration timeout",
+                    ));
+                }
+
+                let secs = if dur.as_secs() > libc::time_t::MAX as u64 {
+                    libc::time_t::MAX
+                } else {
+                    dur.as_secs() as libc::time_t
+                };
+                let mut timeout = libc::timeval {
+                    tv_sec: secs,
+                    tv_usec: dur.subsec_micros() as libc::suseconds_t,
+                };
+                if timeout.tv_sec == 0 && timeout.tv_usec == 0 {
+                    timeout.tv_usec = 1;
+                }
+                timeout
+            }
+            None => libc::timeval { tv_sec: 0, tv_usec: 0 },
+        };
+        setsockopt(self, libc::SOL_SOCKET, kind, timeout)
+    }
+
+    pub fn timeout(&self, kind: libc::c_int) -> io::Result<Option<Duration>> {
+        let raw: libc::timeval = getsockopt(self, libc::SOL_SOCKET, kind)?;
+        if raw.tv_sec == 0 && raw.tv_usec == 0 {
+            Ok(None)
+        } else {
+            let sec = raw.tv_sec as u64;
+            let nsec = (raw.tv_usec as u32) * 1000;
+            Ok(Some(Duration::new(sec, nsec)))
+        }
+    }
+
+    pub fn shutdown(&self, how: Shutdown) -> io::Result<()> {
+        let how = match how {
+            Shutdown::Write => libc::SHUT_WR,
+            Shutdown::Read => libc::SHUT_RD,
+            Shutdown::Both => libc::SHUT_RDWR,
+        };
+        cvt(unsafe { libc::shutdown(self.0.raw(), how) })?;
+        Ok(())
+    }
+
+    pub fn set_nodelay(&self, nodelay: bool) -> io::Result<()> {
+        setsockopt(self, libc::IPPROTO_TCP, libc::TCP_NODELAY, nodelay as c_int)
+    }
+
+    pub fn nodelay(&self) -> io::Result<bool> {
+        let raw: c_int = getsockopt(self, libc::IPPROTO_TCP, libc::TCP_NODELAY)?;
+        Ok(raw != 0)
+    }
+
+    #[cfg(not(any(target_os = "solaris", target_os = "illumos")))]
+    pub fn set_nonblocking(&self, nonblocking: bool) -> io::Result<()> {
+        let mut nonblocking = nonblocking as libc::c_int;
+        cvt(unsafe { libc::ioctl(*self.as_inner(), libc::FIONBIO, &mut nonblocking) }).map(drop)
+    }
+
+    #[cfg(any(target_os = "solaris", target_os = "illumos"))]
+    pub fn set_nonblocking(&self, nonblocking: bool) -> io::Result<()> {
+        // FIONBIO is inadequate for sockets on illumos/Solaris, so use the
+        // fcntl(F_[GS]ETFL)-based method provided by FileDesc instead.
+        self.0.set_nonblocking(nonblocking)
+    }
+
+    pub fn take_error(&self) -> io::Result<Option<io::Error>> {
+        let raw: c_int = getsockopt(self, libc::SOL_SOCKET, libc::SO_ERROR)?;
+        if raw == 0 { Ok(None) } else { Ok(Some(io::Error::from_raw_os_error(raw as i32))) }
+    }
+}
+
+impl AsInner<c_int> for Socket {
+    fn as_inner(&self) -> &c_int {
+        self.0.as_inner()
+    }
+}
+
+impl FromInner<c_int> for Socket {
+    fn from_inner(fd: c_int) -> Socket {
+        Socket(FileDesc::new(fd))
+    }
+}
+
+impl IntoInner<c_int> for Socket {
+    fn into_inner(self) -> c_int {
+        self.0.into_raw()
+    }
+}
+
+// In versions of glibc prior to 2.26, there's a bug where the DNS resolver
+// will cache the contents of /etc/resolv.conf, so changes to that file on disk
+// can be ignored by a long-running program. That can break DNS lookups on e.g.
+// laptops where the network comes and goes. See
+// https://sourceware.org/bugzilla/show_bug.cgi?id=984. Note however that some
+// distros including Debian have patched glibc to fix this for a long time.
+//
+// A workaround for this bug is to call the res_init libc function, to clear
+// the cached configs. Unfortunately, while we believe glibc's implementation
+// of res_init is thread-safe, we know that other implementations are not
+// (https://github.com/rust-lang/rust/issues/43592). Code here in libstd could
+// try to synchronize its res_init calls with a Mutex, but that wouldn't
+// protect programs that call into libc in other ways. So instead of calling
+// res_init unconditionally, we call it only when we detect we're linking
+// against glibc version < 2.26. (That is, when we both know its needed and
+// believe it's thread-safe).
+#[cfg(target_env = "gnu")]
+fn on_resolver_failure() {
+    use crate::sys;
+
+    // If the version fails to parse, we treat it the same as "not glibc".
+    if let Some(version) = sys::os::glibc_version() {
+        if version < (2, 26) {
+            unsafe { libc::res_init() };
+        }
+    }
+}
+
+#[cfg(not(target_env = "gnu"))]
+fn on_resolver_failure() {}