Parallelize functions and ML models in Python

  • What is Ray and how can it be used to parallelize Python functions;
  • How to use Huggingface’s pre-trained machine learning models;
  • How to parallelize model inference (usage).

Parallelizing Python function execution with Ray

pip install -U ray #if you are using terminal
!pip install -U ray #if you are using Jupyter notebook
Output:
duration = 9.000171661376953
results = [792793, 818272, 791557, 808990, 818746, 815934, 815764, 814871, 799357]
  1. Firstly, we import ray and initialize ray cluster with 3 CPUs;
  2. Secondly, we add @ray.remote decorator to our function so that we could use it in distributed setting;
  3. Finally, we execute sleeper_function in remote setting sleeper_function.remote(x) and gather the results with ray.get() .
Ray dashboard showing 3 CPUs running in parallel.

Parallelizing Machine Learning Model inference

pip install transformers #if you are using terminal
!pip install transformers #if you are using Jupyter notebook

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store