Py-ddsketch

Jul 20, 2023

Distributed quantile sketches

This is the Python implementation of the distributed quantile sketch algorithm DDSketch. DDSketch has relative-error guarantees for any quantile q in [0, 1]. That is if the true value of the qth-quantile is x then DDSketch returns a value y such that |x-y| / x < e where e is the relative error parameter. The default here is set to 0.01. DDSketch is also fully mergeable, meaning that multiple sketches from distributed systems can be combined in a central node.

Our default implementation, DDSketch, is guaranteed to not grow too large in size for any data that can be described by a distribution whose tails are sub-exponential.

We also provide implementations LogCollapsingLowestDenseDDSketch and LogCollapsingHighestDenseDDSketch where the q-quantile will be accurate up to the specified relative error for q that is not too small or large. Concretely, the q-quantile will be accurate up to the specified relative error as long as it belongs to one of the m bins kept by the sketch. If the data is time in seconds, the default of m = 2048 covers 80 microseconds to 1 year.



Checkout these related ports:
  • Zziplib - Library to provide transparent read access to zipped files
  • Zydis - Fast and lightweight x86/x86-64 disassembler library
  • Zycore-c - Support library with platform independent types, macros, etc for Zydis
  • Zthread - Platform-independent object-oriented C++ threading library
  • Zookeeper - Coordination Service for Distributed Applications
  • Zls - Zig LSP implementation + Zig Language Server
  • Zfp - High throughput library for compressed floating-point arrays
  • Zeal - Offline documentation browser
  • Zapcc - C++ caching compiler based on clang
  • Zanata-platform - Web-based translation platform
  • Zanata-cli - Zanata Java command line client
  • Z88dk - Complete Z80/Z180 development kit
  • Z80ex - ZiLOG Z80 CPU emulator library
  • Z80asm - Assembler for the Z80 microprocessor
  • Z80-asm - Z80 assembly code assembler and disassembler