Python’s pip+venv and alternatives

This exercise aims to give you hands-on experience with installing Python packages and contrasting some different approaches. It should take approximately 30 minutes to an hour to complete (potentially more when also conducting the benchmark exercises). To get started, choose a Python package that has at least one non-trivial dependency (meaning a dependency that is not part of the Python standard libary) and ideally includes some non-Python source code (which is then typically connected to the Python package with cython or f2py). It is recommended to choose a Python package related to your area of research, but if you are really lacking inspiration, you can complete this exercise using the astropy package.

You can find the relevant documentation in the “Python’s pip+venv” section of the GSSI training material and in the Python package management section on VSCDocs.

Here are the different methods you can use to install astropy on a VSC cluster:

• using pip in a venv with the system Python
• using pip in a venv, satisfying some dependencies with centrally installed modules
• using pip in a venv, but without downloading prebuilt binaries (only relevant if the package includes non-Python source code that requires compilation)
• using vsc-venv (see vsc-venv on VSCDocs)
• using a Conda-based environment manager as discussed in another part of the training (see conda-based environment managers on VSCDocs)
• using uv (not discussed in the training, see the uv pages)

NOTE: Make sure to keep the different installations separated, mixing approaches will likely cause problems

After trying the different installation approaches, compare them with respect to the following characteristics:

• ease of use
• time it takes to install the package
• disk space and number of files consumed by the installation
• reproducibility
• portability (e.g., does the same installation work on different architectures?)
• performance: ideally, use a benchmark (check if the source of the chosen package includes one) to compare runtimes for the different installations. Try to explain possible differences from a theoretical point of view and connect to what you observe in practice.

  NOTE: In case you chose astropy as the example to work with, have a look at the astropy-benchmarks repository.