-
Notifications
You must be signed in to change notification settings - Fork 25
Frequently Asked Questions
- How do these packages differ from those in the standard library?
- When should I use one format over another?
- How does the performance compare to other implementations?
- Why is my decompression rate much slower than those reported here?
The performance improvements in packages like flate and bzip2 are planned to be eventually merged into the standard library. However, it is planned to have these packages to provide other low-level functionality that the standard library should probably never provide control over. See the Performance Metrics page to see whether the standard library has incorporated the improvements developed here.
Furthermore, some packages like brotli are not included in the standard library. There is also planned support for other formats like XZ and others that probably do not belong in the standard library. This repository is intended to host a collection of different compression algorithms. The advantage of having a single repository is because different compression algorithms often share common techniques which can benefit from code re-use and common optimizations.
See Performance Metrics.
All of the decompressors in this library guarantee that they will never read more bytes than necessary from the underlying io.Reader if that reader also satisfies the io.ByteReader interface. This is an important property and is actually relied upon by framing formats like GZip (which wraps DEFLATE internally).
However, this property is harmful to decompression performance since it requires a ReadByte method call to obtain every single input byte. In order to get around this issue, the bit reader used by all the decompressors will check if the input reader satisfies the compress.BufferedReader interface. That interface is designed to allow the decompressor to peek forward in the input stream, and declare what bytes it had processed by discarding some number of bytes.
Thus, if you are experiencing poor performance, it may be that the input reader you are providing satisfies the io.ByteReader interface, but not the compress.BufferedReader interface. The easiest way to work around this is to simply wrap your reader with a bufio.Reader, which is guaranteed to satisfy the interface.
As a special exception, there is internal logic to wrap common readers like bytes.Buffer, bytes.Reader, and strings.Reader so that they satisfy the compress.BufferedReader interface. Those readers should perform just as well as bufio.Reader.