$ the-wire · showcase
flate2 fixes truncation, panic, and a 30x decompression regression
By RepoJournal · Filed · About Rust · Composed from the cited sources · methodology
flate2-rs landed five independent fixes that together close a silent data-integrity hole in its write decoders, remove a panic on cancelled compressors, and restore performance lost to zeroed output buffers.
Reject incomplete streams in write decoders rust-lang/flate2-rs
Write-based DEFLATE and zlib decoders now require the stream to reach StreamEnd before finish, try_finish or reset can succeed. Previously a truncated stream was accepted once the decoder stopped producing output, returning partial decompressed data with no error; incomplete streams now return UnexpectedEof when they make no further progress.
Use native zlib-rs uninitialized output rust-lang/flate2-rs
Wiring in zlib-rs 0.6.7's native uninitialized compression and decompression APIs drops the zeroing of the whole output buffer before each call. In the reporter's measurements, 2 KiB input chunks against a 2 MiB output buffer went from roughly 145 ms to 20 to 24 ms for decompression, and from roughly 190 ms to 64 to 72 ms for compression.
Avoid shifting buffered output on partial writes rust-lang/flate2-rs
Instead of draining the output buffer after every partial write, the writer now tracks an offset and compacts after 8 KiB. For 1 MiB of incompressible input a one-byte writer fell from about 109 ms to 4.5 ms, with peak allocation unchanged.
Avoid panicking when dropping unfinished compressors rust-lang/flate2-rs
Dropping a compressor before finishing it is valid cancellation, but the zlib and zlib-ng backends report the discarded pending output as MZ_DATA_ERROR, so asserting MZ_OK in Drop turned that into a panic. The inflateReset2 check is left intact because a failure there still indicates a broken internal invariant.
Preserve gzip header progress on input read errors rust-lang/flate2-rs
A gzip read encoder that advanced its header position and then hit Interrupted or WouldBlock on the input lost the header on retry, letting read_to_end succeed with an invalid stream after a single interruption. Header byte count is now reported as a successful partial read when output made progress; errors with no progress still propagate. The open rust-forge and crates.io traffic is routine: ...
