Skip to main content

Automated testing

Writing tests is an essential part of application development. Whether the application is intended for self-use only or for distribution, it is important to test that all functionalities work as expected. The more thorough your application is tested, the more robust it will be and code changes can be made with more confidence.

An application can be tested by manually clicking through the different entities and checking all functionalities for different scenarios. This, however, takes a lot of time and can easily be replaced by writing a few automated tests. Automated testing will prevent you from spending a lot of time trying to find and fix bugs in the application later on.

There is a lot of documentation available on writing tests. The following webpages provide info on good practices for writing automated tests in Python:

Testing frameworks​

Python has its own builtin library available for writing automated tests: unittest. The CLI test and ci-test commands works with the unittest library out-of-the-box.

Other testing frameworks

There are other third-party Python libraries for testing, such as pytest, nose2, or robotframework, but currently only unittest is directly supported by the CLI.

Testing as part of the development lifecycle​

It is good practice to not only write a few tests for your application here and there, but make testing an integral part of your Software Development Life Cycle (SDLC). This allows you to worry less about you or your colleague introducing new bugs, so that you can focus more on developing interesting and impactful additions to your app.

A good way to make testing an integral part of your development lifecycle, is through Continuous Integration (CI). Please refer to our CI/CD guide to set this up.

Writing tests​

When writing automated tests, two main types of tests can be considered: unit and integration tests. In short, a unit test is the test of a single specific module or function and shows that the individual parts are working correctly. An integration test is a test of an overall system of multiple modules or functions, and shows that the interfaces between the modules are working correctly.

A unit test verifies that a certain input results in the expected output (for all cases of interest), but does not detect integration errors or other application-wide issues. On the other hand, an integration test may detect errors when functions or modules are incorrectly integrated but usually doesn't check intermediate results and specific cases.

Integration tests tend to be much more complex than unit tests and often requires more mocking of certain parts of your application.

A generic example of a unittest is shown below. In this example, a particular function is imported from the app code and tested using the unittest framework. It tests whether the function provides the correct output for a valid input, and it tests whether a correct error is raised when the input is incorrect.

import unittest

from app.function_library.weight_functions import calculate_weight

class TestWeightFunctions(unittest.TestCase):
    def test_calculate_weight_returns_correct_value(self):
        # Arrange
        volume = 10
        density = 100
        expected_weight = 1000
        # Act
        weight = calculate_weight(volume, density)
        # Assert
        self.assertEqual(weight, expected_weight)

    def test_calculate_weight_raises_TypeError_with_incorrect_input(self):
        # Arrange
        volume = 10
        density = 'one hundred'

        # Act & Assert
        with self.assertRaises(TypeError):
          weight = calculate_weight(volume, density)

Folder structure​

Whether you are writing unit tests or integration tests, it is good-practice to separate the test code from the application code. We advise using the general folder structure described below.

note

the CLI test and ci-test commands require all tests to reside in a folder named "tests".

my-app
├── app.py
├── tests
│   ├── test_app.py  <- you are free to split this file up into separate files within the tests folder
│   └── __init__.py
└── ...

Or for larger apps:

my-app
├── app
│   ├── entity_type_1
│   │   ├── controller.py
│   │   └── parametrization.py
│   ├── entity_type_2
│   │   ├── controller.py
│   │   └── parametrization.py
│   ├── function_library
│   │   └── weight_functions.py
│   └── __init__.py
├── tests
│   ├── entity_type_1
│   │   ├── test_controller.py
│   │   └── __init__.py
│   ├── entity_type_2
│   │   ├── test_controller.py
│   │   └── __init__.py
│   ├── function_library
│   │   ├── test_weight_functions.py
│   │   └── __init__.py
│   └── __init__.py
└── ...
caution

Always make sure that each folder in the 'tests' directory contains an __init__.py and that the Python files start with test_<module_name_here>.py!