Installation#

Dependencies#

The PyLops-MPI project strives to create a library that is easy to install in any environment and has limited number of dependencies. Required dependencies are as follows:

Download and Install MPI#

Visit the official MPI website to download an appropriate MPI implementation for your system. Follow the installation instructions provided by the MPI vendor.

Verify MPI Installation#

After installing MPI, verify its installation by opening a terminal and running the following command:

>> mpiexec --version

Fork PyLops-MPI#

Fork the PyLops-MPI repository and clone it by executing the following in your terminal:

>> git clone https://github.com/YOUR-USERNAME/pylops-mpi.git

We recommend installing dependencies into a separate environment. For that end, we provide a Makefile with useful commands for setting up the environment.

Enable Nvidia Collective Communication Library#

To obtain highly-optimized performance on GPU clusters, PyLops-MPI also supports the Nvidia’s collective communication calls (NCCL). Two additional dependencies are required: CuPy and NCCL

Step-by-step installation for users#

Conda#

For a conda environment, run

>> make install_conda

This will create and activate an environment called pylops_mpi, with all required dependencies.

Pip#

If you prefer a pip installation, simply type the following command in your terminal to install the PyPI distribution:

>> pip install pylops-mpi

When installing via pip, only required dependencies are installed. Note that, differently from the conda command, the above will not create a virtual environment. Make sure you create and activate your environment previously.

Step-by-step installation for developers#

Install dependencies#

Conda#

For a conda environment, run

>> make dev-install_conda

This will create and activate an environment called pylops_mpi, with all required and optional dependencies.

If you want to enable NCCL in PyLops-MPI, run this instead

>> make dev-install_conda_nccl

Pip#

If you prefer a pip installation, we provide the following command

>> make dev-install

Note that, differently from the conda command, the above will not create a virtual environment. Make sure you create and activate your environment previously.

Simlarly, if you want to enable NCCL but prefer using pip, you must first check the CUDA version of your system:

>> nvidia-smi

The Makefile is pre-configured with CUDA 12.x. If you use this version, run

>> make dev-install_nccl

Otherwise, you can change the command in Makefile to an appropriate CUDA version i.e., If you use CUDA 11.x, change cupy-cuda12x and nvidia-nccl-cu12 to cupy-cuda11x and nvidia-nccl-cu11 and run the command.

Run tests#

To ensure that everything has been setup correctly, run tests:

>> make tests

Make sure no tests fail, this guarantees that the installation has been successful.

If PyLops-MPI is installed with NCCL, also run tests:

>> make tests_nccl

Run examples and tutorials#

Since the sphinx-gallery creates examples/tutorials using only a single process, it is highly recommended to test the examples/tutorials using n processes.

run examples:

>> make run_examples

run tutorials:

>> make run_tutorials

Make sure all the examples and tutorials python scripts are executed without any errors.

Add remote (optional)#

To keep up-to-date on the latest changes while you are developing, you may optionally add the PyLops-MPI repository as a remote. Run the following command to add the PyLops-MPI repo as a remote named upstream:

>> git remote add upstream https://github.com/PyLops/pylops-mpi

From then on, you can pull changes (for example, in the main branch) with:

>> git pull upstream main

Final steps#

PyLops-MPI does not enforce the use of a linter as a pre-commit hook, but we do highly encourage using one before submitting a Pull Request. A properly configured linter (flake8) can be run with:

>> make lint

In addition, it is highly encouraged to build the docs prior to submitting a Pull Request. Apart from ensuring that docstrings are properly formatted, they can aid in catching bugs during development.

Build the docs with:

>> make doc

or Update the docs with:

>> make docupdate