Lbzip2
Lbzip2 is a Pthreads-based parallel bzip2/bunzip2 filter, passable to GNU tar with the --use-compress-program option. It isn't restricted to regular files on input, nor output. Successful splitting for decompression isn't guaranteed, just very likely (failure is detected). Splitting in both modes and compression itself occur with an approximate 900k block size. On an Athlon-64 X2 6000+, lbzip2 was 92% faster than standard bzip2 when compressing, and 45% faster when decompressing (based on wall clock time). Lbzip2 strives to be portable by requiring UNIX 98 APIs only, besides an unmodified libbz2.
| Tags | Archiving Compression Stable |
|---|---|
| Licenses | GPLv3 GPLv2 GPL |
| Operating Systems | Unix POSIX Linux |
| Implementation | C |
| Translations | English |
Recent releases
- 05 Apr 2009 23:13
Changes: A bug has been fixed where the single-worker decompressor could theoretically skip decompression of the last part of the compressed input and incorrectly signify premature EOF instead. In this release, logically independent buffer sizes are detached from each other. Several buffer sizes should be more I/O-friendly now. The splitter block size of the multiple-workers decompressor was recalculated; an effort was made to deduce it formally in comments. The "malloc_trace.pl" script tracks peak memory usage. The documentation was refined.
- 15 Mar 2009 23:37
Changes: The single-worker decompressor's multiplexer, while passing back all accrued released input slots to the splitter, wrote out at most one decompressed sub-block per one iteration of its outermost loop. In case of a slowly progressing muxer, this could lead to skyrocketing memory allocation on part of the worker. Now the SWD's muxer writes out all accrued decompressed sub-blocks per iteration. The documentation has been updated. The build system has received a minuscule portability improvement.
- 28 Jan 2009 22:16
Changes: Besides some code cleanup, various portability and other workarounds were introduced to cope with bugs in C libraries, shells, and upstream bzip2.
- 26 Jan 2009 02:28
Changes: The shell script test.sh tests the installed instance of lbzip2, which can be different from the currently built one. pbzip2 is not a requirement for testing anymore, but test.sh will try to measure its performance if it's available. The same applies to the bzip2 module of 7za (from p7zip). Some GNU/Linux specific hints on enabling SUSv2 conformance (which is required for building and testing) were added to the README.
- 21 Jan 2009 06:44
Changes: This release adds support for testing by end-users in a portable way.
