Installation#
Dependencies#
The PyLops-MPI project strives to create a library that is easy to install in any environment and has limited number of dependencies. Required dependencies are as follows:
MPI(Message Passing Interface)
Python 3.8 or greater
Download and Install MPI#
Visit the official MPI website to download an appropriate MPI implementation for your system. Follow the installation instructions provided by the MPI vendor.
Verify MPI Installation#
After installing MPI, verify its installation by opening a terminal and running the following command:
>> mpiexec --version
Fork PyLops-MPI#
Fork the PyLops-MPI repository and clone it by executing the following in your terminal:
>> git clone https://github.com/YOUR-USERNAME/pylops-mpi.git
We recommend installing dependencies into a separate environment. For that end, we provide a Makefile with useful commands for setting up the environment.
Enable Nvidia Collective Communication Library#
To obtain highly-optimized performance on GPU clusters, PyLops-MPI also supports the Nvidia’s collective communication calls (NCCL). Two additional dependencies are required: CuPy and NCCL
Step-by-step installation for users#
Conda#
For a conda
environment, run
>> make install_conda
This will create and activate an environment called pylops_mpi
, with all required dependencies.
Pip#
If you prefer a pip
installation, simply type the following command in your terminal to install the
PyPI distribution:
>> pip install pylops-mpi
When installing via pip, only required dependencies are installed.
Note that, differently from the conda
command, the above will not create a virtual environment.
Make sure you create and activate your environment previously.
Step-by-step installation for developers#
Install dependencies#
Conda#
For a conda
environment, run
>> make dev-install_conda
This will create and activate an environment called pylops_mpi
, with all required and optional dependencies.
If you want to enable NCCL in PyLops-MPI, run this instead
>> make dev-install_conda_nccl
Pip#
If you prefer a pip
installation, we provide the following command
>> make dev-install
Note that, differently from the conda
command, the above will not create a virtual environment.
Make sure you create and activate your environment previously.
Simlarly, if you want to enable NCCL but prefer using pip, you must first check the CUDA version of your system:
>> nvidia-smi
The Makefile is pre-configured with CUDA 12.x. If you use this version, run
>> make dev-install_nccl
Otherwise, you can change the command in Makefile to an appropriate CUDA version
i.e., If you use CUDA 11.x, change cupy-cuda12x
and nvidia-nccl-cu12
to cupy-cuda11x
and nvidia-nccl-cu11
and run the command.
Run tests#
To ensure that everything has been setup correctly, run tests:
>> make tests
Make sure no tests fail, this guarantees that the installation has been successful.
If PyLops-MPI is installed with NCCL, also run tests:
>> make tests_nccl
Run examples and tutorials#
Since the sphinx-gallery creates examples/tutorials using only a single process, it is highly recommended to test the examples/tutorials using n processes.
run examples:
>> make run_examples
run tutorials:
>> make run_tutorials
Make sure all the examples and tutorials python scripts are executed without any errors.
Add remote (optional)#
To keep up-to-date on the latest changes while you are developing, you may optionally add the PyLops-MPI repository as a remote. Run the following command to add the PyLops-MPI repo as a remote named upstream:
>> git remote add upstream https://github.com/PyLops/pylops-mpi
From then on, you can pull changes (for example, in the main branch) with:
>> git pull upstream main
Final steps#
PyLops-MPI does not enforce the use of a linter as a pre-commit hook, but we do highly encourage using one before submitting a Pull Request.
A properly configured linter (flake8
) can be run with:
>> make lint
In addition, it is highly encouraged to build the docs prior to submitting a Pull Request. Apart from ensuring that docstrings are properly formatted, they can aid in catching bugs during development.
Build the docs with:
>> make doc
or Update the docs with:
>> make docupdate