May 26, 2018

Lightweight pipelining using Python functions as jobs

Joblib is a set of tools to provide lightweight pipelining in Python. In particular, joblib offers

  • transparent disk-caching of the output values and lazy re-evaluation
  • easy simple parallel computing
  • logging and tracing of the execution

Joblib is optimized to be fast and robust in particular on large data and has specific optimizations for numpy arrays.

WWW https//