about summary refs log tree commit diff
path: root/compiler/rustc_llvm/llvm-wrapper/PassWrapper.cpp
diff options
context:
space:
mode:
authorJohn Kugelman <john@kugelman.name>2021-09-21 21:14:32 -0400
committerJohn Kugelman <john@kugelman.name>2021-09-22 00:54:27 -0400
commit9b9c24ec7f7c850d0c92babeb45a5ccff940a2c1 (patch)
tree1f6f7db6e75a35d9699a131590ad6938f399adcb /compiler/rustc_llvm/llvm-wrapper/PassWrapper.cpp
parent7743c9fadd64886d537966ba224b9c20e6014a59 (diff)
downloadrust-9b9c24ec7f7c850d0c92babeb45a5ccff940a2c1.tar.gz
rust-9b9c24ec7f7c850d0c92babeb45a5ccff940a2c1.zip
Fix read_to_end to not grow an exact size buffer
If you know how much data to expect and use `Vec::with_capacity` to
pre-allocate a buffer of that capacity, `Read::read_to_end` will still
double its capacity. It needs some space to perform a read, even though
that read ends up returning `0`.

It's a bummer to carefully pre-allocate 1GB to read a 1GB file into
memory and end up using 2GB.

This fixes that behavior by special casing a full buffer and reading
into a small "probe" buffer instead. If that read returns `0` then it's
confirmed that the buffer was the perfect size. If it doesn't, the probe
buffer is appended to the normal buffer and the read loop continues.

Fixing this allows several workarounds in the standard library to be
removed:

- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the
  previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in
  `initial_buffer_size`.

Curiously, there was a unit test that specifically checked that
`Read::read_to_end` *does* over-allocate. I removed that test, too.
Diffstat (limited to 'compiler/rustc_llvm/llvm-wrapper/PassWrapper.cpp')
0 files changed, 0 insertions, 0 deletions