Unit Testing

Since Github offerd a builtin unittesting/CI (continuous integration) service it is relatively easy make your code pull-request-safe. But first a little bit about testing itself.

Setting up your tests

The first thing you need to do is to set up some tests. As you may have seen in The module, matpy contains a subdirectory, which contains a testing file, specifically the test file test_matmul_and_dot.py:

  1#!/usr/bin/env python
  2
  3"""
  4This is a small script that shows how to simply create a tests class. The
  5reason why a tests class is the superior choice over a function is that it can
  6set up a testing environment, e.g. a tests directory structure needed to check
  7existing files. The unittest testclass also contains an easy way to check
  8whether your function throws an error, when it should.
  9
 10
 11:author:
 12    Lucas Sawade (lsawade@princeton.edu, 2019)
 13
 14:license:
 15    GNU Lesser General Public License, Version 3
 16    (http://www.gnu.org/copyleft/lgpl.html)
 17
 18"""
 19
 20import unittest
 21import numpy as np
 22from matpy.matrixmultiplication import matmul
 23from matpy.matrixmultiplication import dotprod
 24from matpy.matrixmultiplication import MatrixMultiplication
 25
 26
 27class TestMatMul(unittest.TestCase):
 28    """"A sample tests class to check if your modules' functions ar
 29    functioning."""
 30
 31    def setUp(self):
 32        """
 33        The setUp command is used to reduce the need for large amounts of
 34        redudandant code. This will be executed and setup once before every
 35        of your test-class' method.
 36        Very useful if you want to load and setup a certain object.
 37        """
 38        # This might seem a little fabricated and unecessary for our example
 39        # here setUp is actually an overkill
 40        self.a1 = [1, 0]
 41        self.a2 = [0, 1]
 42        self.b1 = [4, 1]
 43        self.b2 = [2, 2]
 44
 45    def test_raise_shape_error(self):
 46        """Tests if error is raised when either A or B does not have 2
 47        dimensions"""
 48
 49        # A does not have 2 dimensions
 50        a = np.array([[self.a1, self.a2], [self.a2, self.a2]])
 51        b = np.array([self.b1, self.b2])
 52
 53        with self.assertRaises(ValueError):
 54            matmul(a, b)
 55
 56        # B does not have 2 dimensions
 57        a = np.array([self.a1, self.a2])
 58        b = np.array([[self.a1, self.a2], [self.a2, self.a2]])
 59
 60        with self.assertRaises(ValueError):
 61            matmul(a, b)
 62
 63    def test_raise_shape_match_error(self):
 64        """Tests whether an error is thrown when b doesn't match a."""
 65
 66        # B has more rows than a has columns!
 67        a = np.array([self.a1, self.a2])
 68        b = np.array([self.b1, self.b2, self.b2])
 69
 70        # Check if error is raised
 71        with self.assertRaises(ValueError):
 72            matmul(a, b)
 73
 74    def test_multiplication(self):
 75        """Test the multiplication itself."""
 76
 77        # Define matrix content.
 78        a = np.array([self.a1, self.a2])
 79        b = np.array([self.b1, self.b2])
 80
 81        # Check result
 82        self.assertTrue(np.all(np.array([self.b1, self.b2] == matmul(a, b))))
 83
 84
 85class TestDot(unittest.TestCase):
 86    """"A sample tests class to check if your modules' functions ar
 87    functioning."""
 88
 89    def test_raise_shape_error(self):
 90        """Tests if error is raised when either A or B does not have 2
 91        dimensions"""
 92
 93        # A does not have 2 dimensions
 94        a = np.array([[[1, 0], [0, 1]], [[0, 1], [0, 1]]])
 95        b = np.array([[4, 1], [2, 2]])
 96
 97        with self.assertRaises(ValueError):
 98            dotprod(a, b)
 99
100        # B does not have 2 dimensions
101        a = np.array([[1, 0], [0, 1]])
102        b = np.array([[[1, 0], [0, 1]], [[0, 1], [0, 1]]])
103
104        with self.assertRaises(ValueError):
105            dotprod(a, b)
106
107    def test_raise_shape_match_error(self):
108        """Tests whether an error is thrown when b doesn't match a."""
109
110        # B has more rows than a has columns!
111        a = np.array([[1, 0], [0, 1]])
112        b = np.array([[4, 1], [2, 2], [2, 2]])
113
114        # Check if error is raised
115        with self.assertRaises(ValueError):
116            dotprod(a, b)
117
118    def test_multiplication(self):
119        """Test the multiplication itself."""
120
121        # Define matrix content.
122        a = np.array([[1, 0], [0, 1]])
123        b = np.array([[4, 1], [2, 2]])
124
125        # Check result
126        self.assertTrue(np.all(np.array([[4, 1], [2, 2]] == matmul(a, b))))
127
128
129class TestMM(unittest.TestCase):
130    """"A sample tests class to check if your modules' functions are
131    functioning."""
132
133    def test_raise_method_error(self):
134        """Tests if error is raised when either A or B does not have 2
135        dimensions, but here mainly for class initiation. As the methods
136        themselves are already proven to work."""
137
138        # A does not have 2 dimensions
139        a = np.array([[[1, 0], [0, 1]], [[0, 1], [0, 1]]])
140        b = np.array([[4, 1], [2, 2]])
141
142        # Assign wrong method to raise error
143        method = "blub"
144
145        with self.assertRaises(ValueError):
146            MM = MatrixMultiplication(a, b, method=method)
147
148        # Assign right method to check for size error
149        method = "matmul"
150        with self.assertRaises(ValueError):
151            MM = MatrixMultiplication(a, b, method=method)
152            MM()
153
154        # B does not have 2 dimensions
155        a = np.array([[1, 0], [0, 1]])
156        b = np.array([[[1, 0], [0, 1]], [[0, 1], [0, 1]]])
157
158        with self.assertRaises(ValueError):
159            MM = MatrixMultiplication(a, b, method=method)
160            print(MM())
161
162
163if __name__ == "__main__":
164    unittest.main()

When inspecting the file you will, see three different unittest classes – for each function and one for the class. These are not perfect tests, but they do illustrate the basic usage of the unittest.TestCase, which is very useful especially if you want to set up a test environment. Note that in the last test we are not testing the output of the class’ run since it is already tested with the previous two tests. This is the great thing about unit testing. The TestMatMul class’ first method is SetUp(). This method is executed when the class is initialised, so variables and some computations do not have to run for each testing method. Instead those variables will be saved as attributes of the test class. This is great if you want to avoid having to write the same piece of code several times or have to initialise an object that takes a lot of computation time and, thereby, reducing your total test computation time.

Setting up a .travis.yml

After having at least one file that contains unit tests, we can start setting up the continuous integration. And as you might suspect, it’s also not as hard as it sounds. You simply need to open an account on travis-ci.com preferably using your github account ( or linking the both works, too I presume) and then add the .travis.yml to your repository. After adding the .travis.yml, travis-ci.com will automatically detect the file and start running a test. This will start after every commit pull request etc.

So now, let’s look at one of these .travis.yml files (the one from matpy)

dist: xenial   # required for Python >= 3.7
language: python
python:
  - "3.7"

# command to install dependencies
install:
  - wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O miniconda.sh;
  - bash miniconda.sh -b -p $HOME/miniconda
  - export PATH="$HOME/miniconda/bin:$PATH"
  - hash -r
  - conda config --set always_yes yes --set changeps1 no
  - conda config --add channels conda-forge
  - conda update -q conda
  - conda info -a
  - conda env create -f environment.yml
  - source activate htmapp
  - pip install -e .

# command to run tests
script: pytest

The first few things are just system settings. The install keyword however introduces a sequence of bash commands that are executed. Let’s go through this thing line by line.

  1. Gets the anaconda installation file from the server.

  2. Starts installation

  3. Add miniconda path to PATH

  4. hash -r (idk)

  5. In installation always say yes and turn off command line environment indicator

  6. Add package channels

  7. Update conda

  8. conda info -a

  9. Create environment from environment file

  10. Activate environment

  11. Install package

The script is the test script to be executed. If everything is executed without errors, the continuous integration test will show up as passed.

That means you’re all set for continuous integration! Wasn’t all that hard was it?