zlib
Template:RefimproveTemplate:Short description
Script error: No such module "Distinguish".
Page Module:Infobox/styles.css has no content.
| zlib (library) | |
|---|---|
| Lua error in package.lua at line 80: module 'Module:InfoboxImage/data' not found. | |
| [[Programmer|DeveloperTemplate:Pluralize from text]] | Page Template:Plainlist/styles.css has no content.[1] |
| Initial release | Script error: No such module "Date time". |
| Written in | C |
| Engine | Page Template:Plainlist/styles.css has no content.Template:EditAtWikidata |
| Operating system | Cross-platform |
| Type | Data compression |
| License | zlib License |
| Website | Script error: No such module "WikidataIB". |
| Repository | Page Template:Plainlist/styles.css has no content.
|
Script error: No such module "Check for conflicting parameters".
zlib (/ˈziːlɪb/ or "zeta-lib", /ˈziːtəˌlɪb/)[2][3] is a data format and a lossless data compression software library created by Jean-loup Gailly and Mark Adler. The library implements the Deflate algorithm and supports compressing and decompressing data using the zlib data format, the gzip file format, or simply as a Deflate stream.[4]
zlib is also a crucial component of many software platforms, including Linux, macOS, and iOS. It has also been used in gaming consoles such as the PlayStation 4, PlayStation 3, Wii U, Wii, Xbox One and Xbox 360.[citation needed]
The first public version of Zlib, 0.9, was released on 1 May 1995 and was originally intended for use with the libpng image library. It is free software, distributed under the zlib License.
RFC 1950 specifies the zlib data format.[5]
Data format
Page Module:Infobox/styles.css has no content.
| zlib (data format) | |
|---|---|
| Internet media type | Template:Monodiv |
| Size limitation | Unlimited |
| Developed by | Page Template:Plainlist/styles.css has no content.[7] |
| Type of format | Data compression |
| Compression | Lossless |
| Container for | Deflate stream |
| Standard | Template:IETF RFC |
The zlib data format consists of a 2-byte header, an optional extra header field, Deflate-compressed data, and a 4-byte trailer.
Endianness is big-endian.[8] Bit 0 is the least significant bit in a byte.[9]
| Offset (bytes) | Field[10] | Size (bytes) | Description |
|---|---|---|---|
| 0 | CMF | 1 |
This byte is divided into two 4-bit fields: |
| 1 | FLG | 1 |
|
| 2 | DICTID | 0 or 4 | Adler-32 checksum of the preset DEFLATE dictionary used during compression. Present if the FDICT flag is set. The decompressor can use this checksum to determine which dictionary has been used by the compressor (e.g. deflateSetDictionary in zlib.h).
|
| rowspan=2" Template:Varies | Compressed data | {Template:Varies | Deflate stream. |
| ADLER32 | 4 | Adler-32 checksum of the uncompressed data (excluding any dictionary data). |
Library
Encapsulation
Deflate compression typically uses the zlib data format or the gzip file format, which add a header and a trailer to the compressed data. This provides stream identification and error detection that are not provided by a Deflate stream.
The zlib format (at least 6 bytes) is smaller than the gzip format (at least 18 bytes) as the latter stores filesystem metadata.
Algorithm
As of September 2018[update], zlib only supports one algorithm, called DEFLATE, which uses a combination of a variation of LZ77 (Lempel–Ziv 1977) and Huffman coding.[11] This algorithm provides good compression on a wide variety of data with minimal use of system resources. This is also the algorithm used in the Zip archive format. The header makes allowance for other algorithms, but none are currently implemented.
Resource use
zlib provides facilities for control of processor and memory use. A compression level value may be supplied that trades speed for compression. There are also facilities for conserving memory, useful in restricted memory environments, such as some embedded systems.
Strategy
The compression can be optimized for specific types of data. If one is using the library to always compress specific types of data, then using a specific strategy may improve compression and performance. For example, if the data contain long lengths of repeated bytes, the run-length encoding (RLE) strategy may give good results at a higher speed. For general data, the default strategy is preferred.
Error handling
Errors in compressed data may be detected and skipped. Further, if "full-flush" points are written to the compressed stream, then corrupt data can be skipped, and the decompression will resynchronize at the next flush point—although no error recovery of the corrupt data is provided. Full-flush points are useful for large data streams on unreliable channels, where some data loss is unimportant, such as in some multimedia applications. However, creating many flush points can affect the speed as well as the amount (ratio) of compression.
Data length
There is no limit to the length of data that can be compressed or decompressed. Repeated calls to the library allow an unlimited number of blocks of data to be handled. Some ancillary code (counters) may suffer from overflow for long data streams, but this does not affect the actual compression or decompression.
When compressing a long (or infinite) data stream, it is advisable to write regular full-flush points.
Applications
Today, zlib is something of a de facto standard, to the point that zlib and DEFLATE are often used interchangeably in standards documents, with thousands of applications relying on it for compression, either directly or indirectly.[12] These include:
- The Linux kernel, where zlib is used to implement compressed network protocols, compressed file systems, and to decompress the kernel image at boot time.
- GNU Binutils and GNU Debugger (GDB)
- libpng, the reference implementation for the PNG image format, which specifies DEFLATE as the stream compression for its bitmap data.
- libwww, an API for web applications like web browsers.
- The Apache HTTP Server, which uses zlib to implement HTTP/1.1.
- Similarly, the cURL library uses zlib to decompress HTTP responses.[13]
- The OpenSSH client and server, which rely on zlib to perform the optional compression offered by the Secure Shell protocol.
- The OpenSSL and GnuTLS security libraries, which can optionally use zlib to compress TLS connections.
- The FFmpeg multimedia library, which uses zlib to read and write the DEFLATE-compressed parts of stream formats, such as Matroska.
- The rsync remote file synchronizer, which uses zlib to implement optional protocol compression.
- The dpkg and RPM package managers, which use zlib to unpack files from compressed software packages.
- The Apache Subversion and CVS version control systems, which use zlib to compress traffic to and from remote repositories.
- The Apache ORC column-oriented data storage format use ZLib as its default compression method.[14]
- The Git version control system uses zlib to store the contents of its data objects (blobs, trees, commits and tags).
- The PostgreSQL RDBMS uses zlib with custom dump format (pg_dump -Fc) for database backups.
- The class System.IO.Compression.DeflateStream of the Microsoft .NET Framework 2.0 and higher.[15]
- The "deflate" utility in TORNADO as part of VxWorks Operating System made by Wind River Systems uses zlib to compress boot ROM images.
- zlib-flate,[16] raw zlib compression program, part of qpdf[17]
- The MySQL RDBMS uses ZLib LZ77 for compression in InnoDB Tables [18]
zlib is also used in many embedded devices, such as the Apple iPhone and Sony PlayStation 3, because the code is portable, liberally licensed, and has a relatively small memory footprint.
Forks
A commonly used library built on an old codebase, zlib is also frequently forked by third-parties that claim improvements to this library:
- Intel has a high-performance fork of zlib.[19]
- Cloudflare maintains a high-performance fork with "massive" improvements.[20]
- zlib-ng is a zlib replacement fork for "next generation" systems. It removes workaround code for compilers that do not support ANSI C, integrates Cloudflare and Intel optimizations, adds hardware acceleration (SIMD and intrinsic functions), and uses code sanitizers, fuzzing, and code coverage to help find bugs.[21]
See also
Lua error in mw.title.lua at line 404: bad argument #2 to 'title.new' (unrecognized namespace name 'Portal').
References
Page Template:Reflist/styles.css has no content.
- ^ Page Module:Citation/CS1/styles.css has no content."zlib Home Site". zlib.net. Archived from the original on 2 June 2006.
zlib was written by Jean-loup Gailly (compression) and Mark Adler (decompression).
- ^ Page Module:Citation/CS1/styles.css has no content.Stutz, Michael (October 22, 2004). The Linux Cookbook, 2nd Edition: Tips and Techniques for Everyday Use. No Starch Press. ISBN 9781593270315. Archived from the original on January 25, 2023. Retrieved October 28, 2021 – via Google Books.
- ^ Page Module:Citation/CS1/styles.css has no content.Loki Software; Hall, John R. (October 22, 2001). Programming Linux Games. No Starch Press. ISBN 9781886411494. Archived from the original on July 11, 2024. Retrieved October 28, 2021 – via Google Books.
- ^ Page Module:Citation/CS1/styles.css has no content."zlib 1.3.1 Manual". zlib.net. Archived from the original on 8 February 2024.
- ^ Script error: No such module "template wrapper".
- ^ Script error: No such module "template wrapper".
- ^ Script error: No such module "template wrapper".
- ^ Script error: No such module "template wrapper".
- ^ Script error: No such module "template wrapper".
- ^ Script error: No such module "template wrapper".
- ^ Script error: No such module "template wrapper".
- ^ Page Module:Citation/CS1/styles.css has no content.Gailly, Jean-loup; Adler, Mark (2002-04-18), zlib Applications, archived from the original on 2019-06-10, retrieved 2007-12-09
- ^ Page Module:Citation/CS1/styles.css has no content."c++ - Why does cURL use zlib?". Stack Overflow.
- ^ Page Module:Citation/CS1/styles.css has no content."ORCFile in HDP 2: Better Compression, Better Performance". 6 September 2013. Archived from the original on 6 August 2020. Retrieved 16 May 2020.
- ^ System.IO.Compression.DeflateStream Script error: No such module "webarchive".. MSDN Library.
- ^ Template:Man
- ^ Page Module:Citation/CS1/styles.css has no content."qpdf". Archived from the original on 2024-07-11. Retrieved 2021-03-28.
- ^ Page Module:Citation/CS1/styles.css has no content."MySQL :: MySQL 8.0 Reference Manual :: 15.9.1.5 How Compression Works for InnoDB Tables". dev.mysql.com. Archived from the original on 2021-08-01. Retrieved 2021-07-14.
- ^ Page Module:Citation/CS1/styles.css has no content."github Intel ZLIB". github. 25 January 2024. Archived from the original on 25 January 2024. Retrieved 25 January 2024.
- ^ Page Module:Citation/CS1/styles.css has no content."Fighting Cancer: The Unexpected Benefit Of Open Sourcing Our Code". The Cloudflare Blog. 8 July 2015. Archived from the original on 21 September 2019. Retrieved 18 January 2020.
- ^ Page Module:Citation/CS1/styles.css has no content.Rosbach, HK (24 February 2023). "zlib-ng/zlib-ng: zlib replacement with optimizations for "next generation" systems". GitHub. zlib-ng. Archived from the original on 20 April 2020. Retrieved 18 January 2020.
