Repository navigation
Conversation
A compressed item starts with its uncompressed length, and the read path allocated that many bytes before decompressing. Reject a length larger than the compressed data can expand to (1032:1 for zlib, 255:1 for FastLZ, 32768:1 for zstd), and for zstd require it to match the frame's content size. A rejected length fails like any other decompression error: a warning and a false result.
FastLZ and zstd results kept the stored length as the string length, so a stored length larger than the decompressed output left uninitialized bytes at the end of the value. Set the length to what the decompressor returned, as the zlib path already does, and terminate the string there.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Compressed values carry their uncompressed length in a 4-byte header, and
s_decompress_value()allocated a buffer of that size before decompressing. The header was trusted as stored, so an inconsistent item could cause a very large allocation, and a fatal memory-limit error, before decompression started. Separately, the FastLZ and zstd paths kept the stored length as the value length even when the library produced fewer bytes, so a value could come back with extra bytes past the decompressed data.falsewith a warning, like any other decompression failure.Maximally compressible values up to 32 MB round-trip with all three algorithms. New tests cover items whose length header doesn't match their data.