Flowgger

Jul 20, 2023

Fast data collector

Flowgger is a fast, simple and lightweight data collector. It reads log entries over a given protocol, extracts them, decodes them using a given format, re-encodes them into a different format, and asynchronously pushes the result into a remote data store.

Flowgger is designed to be

  • Paranoid it carefully validates input data to prevent injection of malformed/incomplete records down the chain.
  • Safe written in Rust, without any unsafe code.
  • Fast even though messages are systematically parsed and validated, Flowgger is orders of magnitude faster than Logstash and Fluentd.
  • Standalone it comes as a single executable file, and doesn’t require a JVM.

Flowgger supports common input types stdin, UDP, TCP, TLS and Redis, as well as multiple input formats JSON GELF, LTSV, Cap’n Proto and RFC5424. Normalized messages can be sent to Kafka, Graylog, to downstream Flowgger servers, or to other log collectors for further processing.



Checkout these related ports:
  • Zxfer - Easily and reliably transfer ZFS filesystems
  • Ztop - Display ZFS dataset I/O in real time
  • Zsm - ZFS Snapshot Manager
  • Zsd - Destroys ZFS snapshots
  • Zrepl - ZFS dataset replication tool
  • Zrep - ZFS based replication and failover solution
  • Zpool-iostat-viz - ZFS pool I/O latency statistics
  • Zoxide - Fast cd alternative that learns your habits
  • Zogftw - Creates redundant backups on encrypted ZFS pools
  • Znapzend - ZFS-centric backup tool
  • Zisofs-tools - User utilities for zisofs
  • Zidrav - File corruption detection and repair program
  • Zfstools - OpenSolaris-compatible auto snapshotting for ZFS
  • Zfsnap2 - Portable performant script to make rolling ZFS snapshots easy
  • Zfsnap - Simple sh script to make zfs rolling snaphosts with cron