Skip to main content

Writing automated tests

Writing tests is an essential part of application development. Whether the application is intended for self-use only or for distribution, it is important to test that all functionalities work as expected. The more thorough your application is tested, the more robust it will be and code changes can be made with more confidence.

An application can be tested by manually clicking through the different entities and checking all functionalities for different scenarios. This, however, takes a lot of time and can easily be replaced by writing a few automated tests. Automated testing will prevent you from spending a lot of time trying to find and fix bugs in the application later on.

There is a lot of documentation available on writing tests. The following webpages provide info on good practices for writing automated tests in Python:

Testing frameworks​

Python has its own builtin library available for writing automated tests: unittest. The CLI test and ci-test commands works with the unittest library out-of-the-box.

Other testing frameworks

There are other third-party Python libraries for testing, such as pytest, nose2, or robotframework, but currently only unittest is directly supported by the CLI.

Unit tests vs. integration tests​

When writing automated tests, two main types of tests can be considered: unit and integration tests. In short, a unit test is the test of a single specific module or function and shows that the individual parts are working correctly. An integration test is a test of an overall system of multiple modules or functions, and shows that the interfaces between the modules are working correctly.

A unit test verifies that a certain input results in the expected output (for all cases of interest), but does not detect integration errors or other application-wide issues. On the other hand, an integration test may detect errors when functions or modules are incorrectly integrated but usually doesn't check intermediate results and specific cases.

Integration tests tend to be much more complex than unit tests and often requires the mocking of certain parts of your application.

Folder structure​

Whether you are writing unit tests or integration tests, it is good-practice to separate the test code from the application code. We advise using the general folder structure described below.

note

the CLI test and ci-test commands require all tests to reside in a folder named "tests".

my-app
├── app.py
├── tests
│   ├── test_app.py  <- you are free to split this file up into separate files within the tests folder
│   └── __init__.py
└── ...

Or for larger apps:

my-app
├── app
│   ├── entity_type_1
│   │   ├── controller.py
│   │   └── parametrization.py
│   ├── entity_type_2
│   │   ├── controller.py
│   │   └── parametrization.py
│   ├── function_library
│   │   └── weight_functions.py
│   └── __init__.py
├── tests
│   ├── entity_type_1
│   │   ├── test_controller.py
│   │   └── __init__.py
│   ├── entity_type_2
│   │   ├── test_controller.py
│   │   └── __init__.py
│   ├── function_library
│   │   ├── test_weight_functions.py
│   │   └── __init__.py
│   └── __init__.py
└── ...
caution

Always make sure that each folder in the 'tests' directory contains an __init__.py and that the Python files start with test_<module_name_here>.py!

Writing a (unit) test​

A generic example of a unittest is shown below. In this example, a particular function is imported from the app code and tested using the unittest framework. It tests whether the function provides the correct output for a valid input, and it tests whether a correct error is raised when the input is incorrect.

import unittest

from app.function_library.weight_functions import calculate_weight

class TestWeightFunctions(unittest.TestCase):
    def test_calculate_weight_returns_correct_value(self):
        # Arrange
        volume = 10
        density = 100
        expected_weight = 1000
        # Act
        weight = calculate_weight(volume, density)
        # Assert
        self.assertEqual(weight, expected_weight)

    def test_calculate_weight_raises_TypeError_with_incorrect_input(self):
        # Arrange
        volume = 10
        density = 'one hundred'

        # Act & Assert
        with self.assertRaises(TypeError):
          weight = calculate_weight(volume, density)

Mocking​

Mocking is (temporary) replacing parts of your to-be-tested application with 'fake' objects within the scope of the test, so that you have full control over input and output within a certain test function. Python’s unittest library has built-in functionalities for mocking: https://docs.python.org/3/library/unittest.mock.html

Learn more?

The topic of 'mocking in Python' is extensively covered on the website: https://realpython.com/python-mock-library

When writing tests for your VIKTOR application, it is likely that you will need to mock certain VIKTOR functions. For example, API calls in your app encountered within a test will return a "OSError: Job token is not set" error, as the token is not available outside the app context.

The viktor.testing module provides various mock-objects to simplify the process of mocking certain VIKTOR objects.

VIKTOR components that require mocking:

Mocking ParamsFromFile​

New in v12.10.0

mock_ParamsFromFile decorator for easier testing of ParamsFromFile decorated method

A file-type entity implements the @ParamsFromFile decorator on one of its controller methods. In order to test this method, the ParamsFromFile decorator (object) needs to be mocked (ParamsFromFile performs API calls internally). The viktor.testing module provides the mock_ParamsFromFile decorator to simplify this mocking:

import unittest

from viktor.testing import mock_ParamsFromFile

from app.my_entity_type.controller import MyEntityTypeController

class TestMyEntityTypeController(unittest.TestCase):

    @mock_ParamsFromFile(MyEntityTypeController)
    def test_process_file(self):
        file = File.from_data("abc")
        returned_dict = MyEntityTypeController().process_file(file)
        self.assertDictEqual(returned_dict, {...})

Mocking View functions​

New in v13.3.0

mock_View decorator for easier testing of view methods

View functions can be tested by mocking the decorator using mock_View:

import unittest

from viktor.testing import mock_View

from app.my_entity_type.controller import MyEntityTypeController

class TestMyEntityTypeController(unittest.TestCase):

    @mock_View(MyEntityTypeController)
    def test_my_view(self):
        params = ...
        result = MyEntityTypeController().my_view(params=params)
        self.assertEqual(result, ...)

Please see each individual view guide for a specific example.

Mocking functions that call external worker​

When calling the execute() method on a class that inherits from ExternalProgram, a job is sent to an external worker. These function calls need to be mocked within the context of (automated) testing.

Currently, the viktor.testing module provides the following decorators that facilitate mocking of workers:

Mocking functions that call external servers​

There are various functions within the VIKTOR library that make use of (internal) services that are being processed on external servers. These external server calls cannot be performed outside the app-context and therefore these function calls need to be mocked within the context of (automated) testing.

The complete list of all functions that call external servers (hence require mocking):

  • viktor.external.dfoundations.BearingPilesModel.generate_input_file
  • viktor.external.dfoundations.TensionPilesModel.generate_input_file
  • viktor.external.dsettlement.Model1D.generate_input_file
  • viktor.external.dsettlement.Model2D.generate_input_file
  • viktor.external.idea_rcs.Model.generate_xml_input
  • viktor.external.idea_rcs.OpenModel.generate_xml_input
  • viktor.external.scia.Model.generate_xml_input
  • viktor.external.spreadsheet.SpreadsheetCalculation.evaluate
  • viktor.external.spreadsheet.SpreadsheetTemplate.render
  • viktor.external.spreadsheet.render_spreadsheet
  • viktor.external.word.WordFileTemplate.render
  • viktor.external.word.render_word_file
  • viktor.geo.GEFData.classify
  • viktor.geo.GEFFile.parse
  • viktor.utils.convert_excel_to_pdf
  • viktor.utils.convert_svg_to_pdf
  • viktor.utils.convert_word_to_pdf
  • viktor.utils.merge_pdf_files
  • viktor.utils.render_jinja_template