| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
instead of through the Drop impl for Handler
|
|
Reading a file into an empty vector or string buffer can incur
unnecessary `read` syscalls and memory re-allocations as the buffer
"warms up" and grows to its final size. This is perhaps a necessary evil
with generic readers, but files can be read in smarter by checking the
file size and reserving that much capacity.
`std::fs::read` and `read_to_string` already perform this optimization:
they open the file, reads its metadata, and call `with_capacity` with
the file size. This ensures that the buffer does not need to be resized
and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls
`file.read_to_end` or `file.read_to_string` they do not get this
optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of
examples of this. They're found all over in unit tests, which isn't a
big deal, but there are also several real instances in the compiler and
in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the `Read` trait and the `read_to_end` method both show
this exact pattern as examples of how to use readers. What this says to
me is that this shouldn't be solved by simply fixing the instances of it
in this codebase. If it's here it's certain to be prevalent in the wider
Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and
`read_to_string` directly on `File`. This way it's no longer a minor
footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as
a bare `Read` constraint or via a `dyn Read` trait object will benefit.
For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to
`self.inner`'s methods. That way it can call `File`'s optimized
implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked
`unsafe`.
- `File::read_to_string` being more efficient means that the performance
note for `io::read_to_string` can be softened. I've added @camelid's
suggested wording from:
https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
|
|
|
|
Make cfg imply doc(cfg)
This is a reopening of #79341, rebased and modified a bit (we made a lot of refactoring in rustdoc's types so they needed to be reflected in this PR as well):
* `hidden_cfg` is now in the `Cache` instead of `DocContext` because `cfg` information isn't stored anymore on `clean::Attributes` type but instead computed on-demand, so we need this information in later parts of rustdoc.
* I removed the `bool_to_options` feature (which makes the code a bit simpler to read for `SingleExt` trait implementation.
* I updated the version for the feature.
There is only one thing I couldn't figure out: [this comment](https://github.com/rust-lang/rust/pull/79341#discussion_r561855624)
> I think I'll likely scrap the whole `SingleExt` extension trait as the diagnostics for 0 and >1 items should be different.
How/why should they differ?
EDIT: this part has been solved, the current code was fine, just needed a little simplification.
cc `@Nemo157`
r? `@jyn514`
Original PR description:
This is only active when the `doc_cfg` feature is active.
The implicit cfg can be overridden via `#[doc(cfg(...))]`, so e.g. to hide a `#[cfg]` you can use something like:
```rust
#[cfg(unix)]
#[doc(cfg(all()))]
pub struct Unix;
```
By adding `#![doc(cfg_hide(foobar))]` to the crate attributes the cfg `#[cfg(foobar)]` (and _only_ that _exact_ cfg) will not be implicitly treated as a `doc(cfg)` to render a message in the documentation.
|
|
Rename `std::thread::available_conccurrency` to `std::thread::available_parallelism`
_Tracking issue: https://github.com/rust-lang/rust/issues/74479_
This PR renames `std::thread::available_conccurrency` to `std::thread::available_parallelism`.
## Rationale
The API was initially named `std::thread::hardware_concurrency`, mirroring the [C++ API of the same name](https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency). We eventually decided to omit any reference to the word "hardware" after [this comment](https://github.com/rust-lang/rust/pull/74480#issuecomment-662045841). And so we ended up with `available_concurrency` instead.
---
For a talk I was preparing this week I was reading through ["Understanding and expressing scalable concurrency" (A. Turon, 2013)](http://aturon.github.io/academic/turon-thesis.pdf), and the following passage stood out to me (emphasis mine):
> __Concurrency is a system-structuring mechanism.__ An interactive system that deals with disparate asynchronous events is naturally structured by division into concurrent threads with disparate responsibilities. Doing so creates a better fit between problem and solution, and can also decrease the average latency of the system by preventing long-running computations from obstructing quicker ones.
> __Parallelism is a resource.__ A given machine provides a certain capacity for parallelism, i.e., a bound on the number of computations it can perform simultaneously. The goal is to maximize throughput by intelligently using this resource. For interactive systems, parallelism can decrease latency as well.
_Chapter 2.1: Concurrency is not Parallelism. Page 30._
---
_"Concurrency is a system-structuring mechanism. Parallelism is a resource."_ — It feels like this accurately captures the way we should be thinking about these APIs. What this API returns is not "the amount of concurrency available to the program" which is a property of the program, and thus even with just a single thread is effectively unbounded. But instead it returns "the amount of _parallelism_ available to the program", which is a resource hard-constrained by the machine's capacity (and can be further restricted by e.g. operating systems).
That's why I'd like to propose we rename this API from `available_concurrency` to `available_parallelism`. This still meets the criteria we previously established of not attempting to define what exactly we mean by "hardware", "threads", and other such words. Instead we only talk about "concurrency" as an abstract resource available to our program.
r? `@joshtriplett`
|
|
Add functions to add unsigned and signed integers
This PR adds methods to unsigned integers to add signed integers with good overflow semantics under `#![feature(mixed_integer_ops)]`.
The added API is:
```rust
// `uX` is `u8`, `u16`, `u32`, `u64`,`u128`, `usize`
impl uX {
pub const fn checked_add_signed(self, iX) -> Option<Self>;
pub const fn overflowing_add_signed(self, iX) -> (Self, bool);
pub const fn saturating_add_signed(self, iX) -> Self;
pub const fn wrapping_add_signed(self, iX) -> Self;
}
impl iX {
pub const fn checked_add_unsigned(self, uX) -> Option<Self>;
pub const fn overflowing_add_unsigned(self, uX) -> (Self, bool);
pub const fn saturating_add_unsigned(self, uX) -> Self;
pub const fn wrapping_add_unsigned(self, uX) -> Self;
pub const fn checked_sub_unsigned(self, uX) -> Option<Self>;
pub const fn overflowing_sub_unsigned(self, uX) -> (Self, bool);
pub const fn saturating_sub_unsigned(self, uX) -> Self;
pub const fn wrapping_sub_unsigned(self, uX) -> Self;
}
```
Maybe it would be interesting to also have `add_signed` that panics in debug and wraps in release ?
|
|
* Remove "bool_to_options" feature
* Update version for compiler feature
* rustfmt
|
|
|
|
Co-authored-by: Jonah Petri <jonah@petri.us>
|
|
|
|
|
|
|
|
|
|
Use `libc::sigaction()` instead of `sys::signal()` to prevent a deadlock
Fixes #88585. POSIX [specifies](https://man7.org/linux/man-pages/man3/fork.3p.html) that after forking,
> to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called.
Rust's standard library does not currently adhere to this, as evidenced by #88585. The child process calls [`sys::signal()`](https://github.com/rust-lang/rust/blob/7bf0736e130e2203c58654f7353dbf9575e49d5c/library/std/src/sys/unix/android.rs#L76), which on Android calls [`libc::dlsym()`](https://github.com/rust-lang/rust/blob/7bf0736e130e2203c58654f7353dbf9575e49d5c/library/std/src/sys/unix/weak.rs#L101), which is [**not**](https://man7.org/linux/man-pages/man7/signal-safety.7.html) async-signal-safe, and in fact causes a deadlock in the example in #88585.
I think the easiest solution here would be to just call `libc::sigaction()` instead, which [is](https://man7.org/linux/man-pages/man7/signal-safety.7.html) async-signal-safe, provides the functionality we need, and is apparently available on all Android versions because it is also used e.g. [here](https://github.com/rust-lang/rust/blob/7bf0736e130e2203c58654f7353dbf9575e49d5c/library/std/src/sys/unix/stack_overflow.rs#L112-L114).
|
|
|
|
haiku thread affinity build fix
|
|
Use the 64b inner:monotonize() implementation not the 128b one for aarch64
aarch64 prior to v8.4 (FEAT_LSE2) doesn't have an instruction that guarantees
untorn 128b reads except for completing a 128b load/store exclusive pair
(ldxp/stxp) or compare-and-swap (casp) successfully. The requirement to
complete a 128b read+write atomic is actually more expensive and more unfair
than the previous implementation of monotonize() which used a Mutex on aarch64,
especially at large core counts. For aarch64 switch to the 64b atomic
implementation which is about 13x faster for a benchmark that involves many
calls to Instant::now().
|
|
os current_exe using same approach as linux to get always the full ab…
…solute path
|
|
path.push() should work as expected on windows verbatim paths
On Windows, std::fs::canonicalize() returns an so-called UNC path. UNC paths differ with regular paths because:
- This type of path can much longer than a non-UNC path (32k vs 260 characters).
- The prefix for a UNC path is ``Component::Prefix(Prefix::DiskVerbatim(..)))``
- No `/` is allowed
- No `.` is allowed
- No `..` is allowed
Rust has poor handling of such paths. If you join a UNC path with a path with any of the above, then this will not work.
I've implemented a new method `fn join_fold()` which joins paths and also removes any `.` and `..` from it, and replaces `/` with `\` on Windows. Using this function it is possible to use UNC paths without issue. In addition, this function is useful on Linux too; paths can be appended without having to call `canonicalize()` to remove the `.` and `..`.
This PR needs test cases, which can I add. I hope this will a start of a discussion.
|
|
Stabilize try_reserve
Stabilization PR for the [`try_reserve` feature](https://github.com/rust-lang/rust/issues/48043#issuecomment-898040475).
|
|
Practice diagnostic message convention
Detected by #89455.
r? ```@estebank```
|
|
|
|
|
|
Rollup of 14 pull requests
Successful merges:
- #86434 (Add `Ipv6Addr::is_benchmarking`)
- #86828 (const fn for option copied, take & replace)
- #87679 (BTree: refine some comments)
- #87910 (Mark unsafe methods NonZero*::unchecked_(add|mul) as const.)
- #88286 (Remove unnecessary unsafe block in `process_unix`)
- #88305 (Manual Debug for Unix ExitCode ExitStatus ExitStatusError)
- #88353 (Partially stabilize `array_methods`)
- #88370 (Add missing `# Panics` section to `Vec` method)
- #88481 (Remove some feature gates)
- #89138 (Fix link in Ipv6Addr::to_ipv4 docs)
- #89401 (Add truncate note to Vec::resize)
- #89467 (Fix typos in rustdoc/lints)
- #89472 (Only register `WSACleanup` if `WSAStartup` is actually ever called)
- #89505 (Add regression test for spurious const error with NLL)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
|
|
Only register `WSACleanup` if `WSAStartup` is actually ever called
See https://github.com/rust-lang/rust/pull/85595
Fixes #85441
|
|
Fix link in Ipv6Addr::to_ipv4 docs
|
|
Manual Debug for Unix ExitCode ExitStatus ExitStatusError
These structs have misleading names. An ExitStatus[Error] is actually a Unix wait status; an ExitCode is actually an exit status. These misleading names appear in the `Debug` output.
The `Display` impls on Unix have been improved, but the `Debug` impls are still misleading, as reported in #74832.
Fix this by pretending that these internal structs are called `unix_exit_status` and `unix_wait_status` as applicable. (We can't actually rename the structs because of the way that the cross-platform machinery works: the names are cross-platform.)
After this change, this program
```
#![feature(exit_status_error)]
fn main(){
let x = std::process::Command::new("false").status().unwrap();
dbg!(x.exit_ok());
eprintln!("x={:?}",x);
}
```
produces this output
```
[src/main.rs:4] x.exit_ok() = Err(
ExitStatusError(
unix_wait_status(
256,
),
),
)
x=ExitStatus(unix_wait_status(256))
```
Closes #74832
|
|
Remove unnecessary unsafe block in `process_unix`
Because it's nested under this unsafe fn!
This block isn't detected as unnecessary because of a bug in the compiler: #88260.
|
|
Add `Ipv6Addr::is_benchmarking`
This PR adds the unstable method `Ipv6Addr::is_benchmarking`. This method is added for parity with `Ipv4Addr::is_benchmarking`, and I intend to use it in a future rework of `Ipv6Addr::is_global` (edit: #86634) to more accurately follow the [IANA Special Address Registry](https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml) (like is done in `Ipv4Addr::is_global`).
With `Ipv6Addr::is_benchmarking` and `Ipv4Addr::is_benchmarking` now both existing, `IpAddr::is_benchmarking` is also added.
|
|
Fix read_to_end to not grow an exact size buffer
If you know how much data to expect and use `Vec::with_capacity` to pre-allocate a buffer of that capacity, `Read::read_to_end` will still double its capacity. It needs some space to perform a read, even though that read ends up returning `0`.
It's a bummer to carefully pre-allocate 1GB to read a 1GB file into memory and end up using 2GB.
This fixes that behavior by special casing a full buffer and reading into a small "probe" buffer instead. If that read returns `0` then it's confirmed that the buffer was the perfect size. If it doesn't, the probe buffer is appended to the normal buffer and the read loop continues.
Fixing this allows several workarounds in the standard library to be removed:
- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in `initial_buffer_size`.
Curiously, there was a unit test that specifically checked that `Read::read_to_end` *does* over-allocate. I removed that test, too.
|
|
This allows using longer paths for filesystem operations without the user needing to `canonicalize` or manually prefix paths.
If the path is already verbatim than this has no effect.
|
|
|
|
|
|
|
|
|
|
On MinGW toolchains the various features (such as function sections)
necessary to eliminate dead function references are disabled due to
various bugs. This means that the windows sockets library will most
likely remain linked to any mingw toolchain built program that also
utilizes libstd.
That said, I made an attempt to also enable `function-sections` and
`--gc-sections` during my experiments, but the symbol references
remained, sadly.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Fixes #89177
|
|
thread: implements available_concurrency on haiku
|
|
Add `#[must_not_suspend]` to some types in std
I am not sure what else should have it? `Ref`?
|
|
Clarify that `CString::from_vec_unchecked` appends 0 byte.
|
|
Restructure std::rt
These changes should reduce binary size slightly while at the same slightly improving performance of startup, thread spawning and `std::thread::current()`. I haven't verified if the compiler is able to optimize some of these cases already, but at least for some others the compiler is unable to do these optimizations as they slightly change behavior in cases where program startup would crash anyway by omitting a backtrace and panic location.
I can remove 6f6bb16 if preferred.
|
|
The reference automatically coerces to a pointer. Writing an explicit
cast here is slightly misleading because that's most commonly used when
a pointer needs to be converted from one pointer type to another, e.g.
`*const c_void` to `*const sigaction` or vice versa.
|
|
|
