Installation

There are two things necessary for a good Python package installation. First, the setup.cfg and/or setup.py files which control the installation of the package itself. This is needed, if you want to create your package and use it like a pip installed package. The other thing is the environment.yml which is used to set up the environment for the installation. For this small project, the environment.yml is using a sledgehammer to crack a nut. We will still want to keep it here for completeness.

Actual installation

Having all these corner stones in place, we simply need to install it. This is simply done by the following line of code.

# Install your package
pip install -e .

The -e simply let’s you modify the package without having to reinstall it all the time. So, testing of the package can be done with every change of the source files even though there is no re-installation. Note that this is only necessary for packages that you install from the source code and plan on modifying!

setup.cfg

In newer installations, the setup.cfg replaces parts of the setup.py or, in many cases, even the whole file. Here, we define all the options that we want setuptools.setup() to use when executing:

pip install -e .

First, let’s have a look at the file in our project:

[metadata]
# replace with your username:
name = htmapp
version = 0.0.4
author = Lucas Sawade, Peter Makus
author_email = lsawade@princeton.edu
description = How to make a Python package.
long_description = file: README.md
long_description_content_type = text/markdown
url = https://github.com/lsawade/how_to_make_a_python_package
project_urls =
    Documentation = https://how-to-make-a-python-package.readthedocs.io/en/latest/
    TravisCI = https://travis-ci.com/github/lsawade/how_to_make_a_python_package
classifiers =
    Programming Language :: Python :: 3
    License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
    Operating System :: OS Independent
keywords = Fun, Education, Learning, Programming

[options]
package_dir =
    = src
packages = find:
python_requires = >=3.6
install_requires = numpy
tests_require = pytest
zip_safe = False

[options.extras_require]
docs = 
    sphinx 
    sphinx-rtd-theme
tests = pytest; py

[options.entry_points]
console_scripts =
    sample-bin = matpy.bins.sample_bin:main

[options.packages.find]
where = src

Most of the defined options are quite self-explanatory.

[options.packages.find]
where = src

defines where the package is located. Typically, we have a src folder that contains the whole package. This line changes the import structure so that we can later run import matpy instead of import src.matpy. For a list of all available parameters and their correct syntax check out the setuptools documentation.

Scripts argument

The scripts argument lets you create executables that are installed upon running the installation. For example, in the directory bins (binaries, which are not technically binaries in this case, but used in the same way as traditional binary files), there is the executable sample-bin:

#!/usr/bin/env python

from matpy import MatrixMultiplication
import numpy as np
import logging

# Use this to set the logger to different levels.
logger = logging.getLogger("matpy")


def main():
    # Set up tests arrays
    a = np.array([[1, 2],
                  [3, 4]])
    b = np.array([[2, 3, 5],
                  [4, 5, 6]])

    # Use class and function call
    M = MatrixMultiplication(a, b, method='matmul')
    c = M()

    print("C:")
    print(c)

if __name__ == "__main__":
    main()

It is normal Python code, but after installation

pip install -e .

you can simply run

sample-bin

from anywhere on your computer, which will run the python command sample_bin(). Do make sure that this filenames do not conflict with your traditional command line tools such as ls, cd, etc. Otherwise you won’t be able to use your created executables.

setup.py

In some cases, you might still want to include a setup.py which used to be the old standard way. One of those cases is if you would like to be able to use the -e (editable) pip install command. Then, you would just include a minimal setup.py containing the following:

from setuptools import setup

# configuration is done in setup.cfg
setup()

A second use-case of the setup.py is illustrated in our sample file:

""" :noindex:
Setup.py file that governs the installatino process of
`how_to_make_a_python_package` it is used by
`conda install -f environment.yml` which will install the package in an
environment specified in that file.

"""
from setuptools import setup
from setuptools.command.test import test as testcommand

# This installs the pytest command. Meaning that you can simply type pytest
# anywhere and "pytest" will look for all available tests in the current
# directory and subdirectories recursively (not yet supported in the setup.cfg)


class PyTest(testcommand):
    user_options = [('pytest-args=', 'a', "Arguments to pass to py.tests")]

    def initialize_options(self):
        testcommand.initialize_options(self)
        self.pytest_args = []

    def run_tests(self):
        import pytest
        import sys
        errno = pytest.main(self.pytest_args)
        sys.exit(errno)


setup(cmdclass={'tests': PyTest})

Here, we add the cmdclass keyword, which is not supported by the setup.cfg (at least, at the time of writing). After package installation, when you in the repository, a simple

pytest

will check all subdirectories for tests and run them subsequently. It is amazing for debugging your code. A third case is for more advanced dynamic installs, in which we wish to compile our module differenly depending upon which machine it is installed on (e.g., different OS or architecture).

environment.yml

This one is mainly to set up an environment and install dependencies there, so that you main Python installation stays untouched and cannot be broken. The general structure looks as follows:

dependencies:
- python=3.8
- numpy
- pip
- pip:
  - recommonmark
  - readthedocs-sphinx-ext
  - sphinx
  - sphinx-rtd-theme
  - pytest
  - -e .
- mock
- pillow
- sphinx
- sphinx_rtd_theme
name: htmapp

name specifies the name of the environment to be created, and pip defines extra packages that possibly aren’t available from the conda package manager. And additional setting that is often used, but we are skipping here are channels, which are online locations from where to grab the dependencies when they are installed with conda (a typical example is conda-forge). The file is pretty straightforward, so I’m gonna stop talking about it now.

To create an environment from the environment.yml file simply execute the following:

conda env create -f environment.yml