May 26, 2018
Lightweight pipelining using Python functions as jobs
Joblib is a set of tools to provide lightweight pipelining in Python. In particular, joblib offers
- transparent disk-caching of the output values and lazy re-evaluation
- easy simple parallel computing
- logging and tracing of the execution
Joblib is optimized to be fast and robust in particular on large data and has specific optimizations for numpy arrays.