Jul 20, 2023

Integer compression with SIMD based on Google’s varint

StreamVByte is an integer compression technique that applies SIMD instructions vectorization to Google’s varint approach. The net result is faster than other byte-oriented compression techniques. The approach is patent-free, the code is available under the Apache License. It includes fast differential coding. It assumes a recent Intel processor e.g., haswell or better, though we provide runtime dispatching for compatibility with legacy systems or an ARM processor with NEON instructions which is almost all of them.

Checkout these related ports:
  • Zutils - Utilities for searching in bzip2, gzip, lzip, and xz archives
  • Zstr - C++ header-only ZLib wrapper
  • Zstd - Fast real-time compression algorithm
  • Zpaqfranz - Swiss army knife for the serious backup manager
  • Zopfli - Zopfli Compression Algorithm
  • Zoo - Manipulate archives of files in compressed form
  • Zlib-ng - Fork of the zlib data compression library
  • Zipper - Tool for inspecting the contents of a compressed archive
  • Zipmix - Produce .ZIP file from two other ones with the best compressed files
  • Zip - Create/update ZIP files compatible with PKZIP
  • Zchunk - Compressed file format that splits the file into independent chunks
  • Xpk - The eXternal PacKer (XPK) library system
  • Xmill - Efficient compressor for XML
  • Xdms - Tool for decompressing Amiga DMS files
  • Xarchiver - Desktop-agnostic GTK frontend to various archiving tools