| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
unify read_to_end and io::copy impls for reading into a Vec
This ports over the initial probe (to avoid allocation) and the dynamic read sizing from the io::copy specialization to the `default_read_to_end` implementation which already had its own optimizations for different cases.
I think it should be a best-of-both now.
suggested by `@a1phyr` in https://github.com/rust-lang/rust/pull/117576#issuecomment-1803408492
|
|
|
|
Improve rewind documentation
The persistent use of an internal cursor for readers is expected for buffer data types that aren't read all at once, but for files it leads to the confusing situation where calling `read_to_end` on the same file handle multiple times only returns the contents of the file for the first call. This PR adds a note to the documentation clarifying that in that case, `rewind()` must first be called.
I'm unsure if this is the right location for the docs update. Maybe it should also be duplicated on `File`?
|
|
Add `BufRead::skip_until`
Alternative version of `BufRead::read_until` that simply discards data, rather than copying it into a buffer.
Useful for situations like skipping irrelevant data in a binary file format that is NUL-terminated.
<details>
<summary>Benchmark</summary>
```
running 2 tests
test bench_read_until ... bench: 123 ns/iter (+/- 6)
test bench_skip_until ... bench: 66 ns/iter (+/- 3)
```
```rs
#![feature(test)]
extern crate test;
use test::Bencher;
use std::io::{ErrorKind, BufRead};
fn skip_until<R: BufRead + ?Sized>(r: &mut R, delim: u8) -> Result<usize, std::io::Error> {
let mut read = 0;
loop {
let (done, used) = {
let available = match r.fill_buf() {
Ok(n) => n,
Err(ref e) if e.kind() == ErrorKind::Interrupted => continue,
Err(e) => return Err(e),
};
match memchr::memchr(delim, available) {
Some(i) => (true, i + 1),
None => (false, available.len()),
}
};
r.consume(used);
read += used;
if done || used == 0 {
return Ok(read);
}
}
}
const STR: &[u8] = b"Ferris\0Hello, world!\0";
#[bench]
fn bench_skip_until(b: &mut Bencher) {
b.iter(|| {
let mut io = std::io::Cursor::new(test::black_box(STR));
skip_until(&mut io, b'\0').unwrap();
let mut hello = Vec::with_capacity(b"Hello, world!\0".len());
let num_bytes = io.read_until(b'\0', &mut hello).unwrap();
assert_eq!(num_bytes, b"Hello, world!\0".len());
assert_eq!(hello, b"Hello, world!\0");
});
}
#[bench]
fn bench_read_until(b: &mut Bencher) {
b.iter(|| {
let mut io = std::io::Cursor::new(test::black_box(STR));
io.read_until(b'\0', &mut Vec::new()).unwrap();
let mut hello = Vec::with_capacity(b"Hello, world!\0".len());
let num_bytes = io.read_until(b'\0', &mut hello).unwrap();
assert_eq!(num_bytes, b"Hello, world!\0".len());
assert_eq!(hello, b"Hello, world!\0");
});
}
```
</details>
|
|
Add Seek::seek_relative
The `BufReader` struct has a `seek_relative` method because its `Seek::seek` implementation involved dumping the internal buffer (https://github.com/rust-lang/rust/issues/31100).
Unfortunately, there isn't really a good way to take advantage of that method in generic code. This PR adds the same method to the main `Seek` trait with the straightforward default method, and an override for `BufReader` that calls its implementation.
_Also discussed in [this](https://internals.rust-lang.org/t/add-seek-seek-relative/19546) internals.rust-lang.org thread._
|
|
Assigned new feature name `core_io_borrowed_buf` to distinguish from the
`Read::read_buf` functionality in `std::io`.
|
|
|
|
|
|
|
|
|
|
This reduces the runtime for a simple program using `Bytes::next` to
iterate through a file from 220ms to 70ms on my Linux box.
|
|
|
|
This greatly increases its speed.
|
|
|
|
This variable seems to serve no purpose, and it's a little confusing
when reading std source code, so remove it.
|
|
|
|
|
|
add more explicit I/O safety documentation
Fixes https://github.com/rust-lang/unsafe-code-guidelines/issues/434
Cc https://github.com/rust-lang/rust/issues/114167
Cc `@Manishearth` `@sunfishcode` `@joshtriplett`
|
|
|
|
|
|
|
|
|
|
Co-authored-by: Dan Gohman <dev@sunfishcode.online>
|
|
`Ok(0)` is indeed something the caller may interpret as an error, but
that's the *correct* thing to return if the writer can't accept any more
bytes.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
io: soften ‘at most one write attempt’ requirement in io::Write::write
At the moment, documentation of std::io::Write::write indicates that
call to it ‘represents at most one attempt to write to any wrapped
object’. It seems that such wording was put there to contrast it with
pre-1.0 interface which attempted to write all the data (it has since
been changed in [RFC 517]).
However, the requirement puts unnecessary constraints and may
complicate adaptors which perform non-trivial transformations on the
data. For example, they may maintain an internal buffer which needs
to be written out before the write method accepts more data. It might
be natural to code the method such that it flushes the buffer and then
grabs another chunk of user data. With the current wording in the
documentation, the adaptor would be forced to return Ok(0).
This commit softens the wording such that implementations can choose
code structure which makes most sense for their particular use case.
While at it, elaborate on the meaning of `Ok(0)` return pointing out
that the write_all methods interprets it as an error.
[RFC 517]: https://rust-lang.github.io/rfcs/0517-io-os-reform.html
|
|
r=Amanieu
Relax implicit `T: Sized` bounds on `BufReader<T>`, `BufWriter<T>` and `LineWriter<T>`
TL;DR:
```diff,rust
-pub struct BufReader<R> { /* ... */ }
+pub struct BufReader<R: ?Sized> { /* ... */ }
-pub struct BufWriter<W: Write> { /* ... */ }
+pub struct BufWriter<W: ?Sized + Write> { /* ... */ }
-pub struct LineWriter<W: Write> { /* ... */ }
+pub struct LineWriter<W: ?Sized + Write> { /* ... */ }
```
This allows using `&mut BufReader<dyn Read>`, for example.
**This is an insta-stable change**.
|
|
|
|
|
|
|
|
|
|
Limit read size in `File::read_to_end` loop
Fixes #110650.
Windows file reads have perf overhead that's proportional to the buffer size. When we have a reasonable expectation that we know the file size, we can set a reasonable upper bound for the size of the buffer in one read call.
|
|
This works around performance issues on Windows by limiting reads the size of reads when the expected size is known.
|
|
A bunch of places were using `replace(…, &mut [])`, but that can just be `take`.
|
|
closes: https://github.com/rust-lang/rust/issues/98070
|
|
Guarantee that when `read_buf_exact` returns, all bytes read will be
appended to the buffer. Including the case when the operations fails.
The motivating use case are operations on a non-blocking reader. When
`read_buf_exact` fails with `ErrorKind::WouldBlock` error, the operation
can be resumed at a later time.
|
|
Co-authored-by: Jacob Lifshay <programmerjake@gmail.com>
|
|
Co-authored-by: Andrew Gallant <jamslam@gmail.com>
|
|
|
|
|
|
Implement rust-lang/libs-team#173.
|
|
At the moment, documentation of std::io::Write::write indicates that
call to it ‘represents at most one attempt to write to any wrapped
object’. It seems that such wording was put there to contrast it
with pre-1.0 interface which attempted to write all the data (it has
since been changed in [RFC 517]).
However, the requirement puts unnecessary constraints and may complicate
adaptors which perform non-trivial transformations on the data. For
example, they may maintain an internal buffer which needs to be written
out before the write method accepts more data. It might be natural to
code the method such that it flushes the buffer and then grabs another
chunk of user data. With the current wording in the documentation, the
adaptor would be forced to return Ok(0).
This commit softens the wording such that implementations can choose
code structure which makes most sense for their particular use case.
While at it, elaborate on the meaning of `Ok(0)` return pointing out
that the write_all methods interprets it as an error.
[RFC 517]: https://rust-lang.github.io/rfcs/0517-io-os-reform.html
|
|
|
|
|
