Multithreading and -processing with joblib

from joblib import Parallel, delayed

Create a function to be executed in parallel:

def my_embarassingly_parallel_job(arg):
    # perform work...
    return

Uses the default “loky” backend for process based parallelism:

results = Parallel(n_jobs=2)(
    delayed(my_embarassingly_parallel_job)(i) for i in range(10)
)

Thread-based parallelism:

results = Parallel(n_jobs=2, prefer="threads")(
    delayed(my_embarassingly_parallel_job)(i) for i in range(10)
)