Uploading files

There are two ways for the user to upload a file in the VIKTOR interface:

1. By defining a FileField in the parametrization, a user can upload a file through an upload modal and directly attach it to a specific field. This way, the developer can easily use the file (content) which will be present in the params of a job. This method is encouraged due to its simplicity and the possibility for users to stay within the current entity in which the FileField resides.
2. By creating a file-like entity type. When a user creates a new entity of this type, he or she will be prompted with an upload modal instead of just the entity name. This method is a bit more cumbersome for users, since it requires more actions (clicks) to be performed and navigation between entities. This method can be used to isolate a file with, for example, attached views.

Upload using FileField

Adding a FileField (or MultiFileField) is as easy as adding any other field:

from viktor.parametrization import ViktorParametrization, FileField, MultiFileField


class Parametrization(ViktorParametrization):
    cpt_file = FileField('CPT file')

These fields will be generated in the VIKTOR interface as dropdown fields, with the possibility to upload one or multiple files. When the user has made a selection, the file resource will be available in the params of every job as FileResource object:

In this example we use the .file property to get the file contents of the uploaded file as a VIKTOR File object. You can also use the .filename property to get the file name, these two properties make up a VIKTOR File object.

...

class Controller(ViktorController):
    ...
    def get_cpt_file(self, params, **kwargs):
        cpt_file = params.cpt_file.file
        return cpt_file

tip

FileResource objects originating from a FileField or MultiFileField are memoizable

Temporary files

Although not recommended, it is possible to use temporary files. This module creates temporary files that get deleted after closing. You can use this as a temporary storage solution as the file is created securely and on completion is removed. For a NamedTemporaryFile the returned object is always a file-like object that can be used in a with statement, just like a normal file.

Upload using file-like entity type

In short, the following steps should be implemented. A more elaborate example is shown in the following sections.

1. Add a new entity type by defining a controller.

2. Implement a process method (can be named arbitrarily) on the controller class, decorated by ParamsFromFile.

   from viktor import ViktorController, File, ParamsFromFile
   
   class Controller(ViktorController):
       ...
   
       @ParamsFromFile()
       def process_file(self, file: File, **kwargs):
           return {}  # nothing to store

In the example above, the process_file method does not contain any logic. However, this method can also be used to parse certain information from the file, which can be stored in the parametrization of the entity. See the example below if this is desired.

File processing

caution

Prevent storing the complete file content on the properties if the file is large, as this may cause speed and/or stability issues. The file content can be retrieved at all times using the API whenever necessary.

The processed content can only be stored on fields that are defined in the entity's parametrization. These can be input fields (e.g. NumberField), but also read-only (OutputField) or even a HiddenField. Let's assume the following parametrization and text file to be uploaded:

class Parametrization(ViktorParametrization):
    number_of_entries = NumberField('Number of entries')
    project_name = TextField('Project')

Project=Super nice project
Entry1
Entry2
Entry3

The decorated process method should return the data in a dictionary format. The structure of this dictionary should match with the structure of the parametrization fields:

class Controller(ViktorController):
    ...

    @ParamsFromFile()
    def process_file(self, file: File, **kwargs):  # viktor.core.File
        # app specific parse logic
        number_of_entries = 0
        project_name = ''

        with file.open(encoding='utf-8') as f:
            for lineno, line in enumerate(f):
                if lineno == 0:
                    _, project_name = line.split('=')
                elif line.startswith('Entry'):
                    number_of_entries += 1

        # linking the parsed output to the parametrization fields (names in database)
        return {
            'number_of_entries': number_of_entries,
            'project_name': project_name
        }

In case of a nested parametrization structure, the return value is a nested dictionary:

class Controller(ViktorController):
    ...

    @ParamsFromFile()
    def process_file(self, file: File, **kwargs):
        ...

        return {
            'tab': {
                'section' : {
                    'number_of_entries': number_of_entries,
                    'project_name': project_name
                }
            }
        }

tip

The file entity can be retrieved and the file (content) can be extracted (from either the current entity or another) using the API.

File restrictions

A limit is set on the file size of an upload to protect the developer for potential memory issues and/or instability of the application when handled incorrectly. Sometimes however, it is required to deal with large files. In this case the limit can be increased using the max_size argument:

file = FileField('CPT file', max_size=100_000_000)  # 100 MB

@ParamsFromFile(max_size=100_000_000)  # 100 MB

Keep in mind that reading the complete content of a large file in memory will lead to memory issues. We therefore recommend reading the file in chunks (see File).

Similarly, you are able to restrict the file type(s) that may be uploaded, by means of file_types:

file = FileField('CPT file', file_types=['.png', '.jpg', '.jpeg'])

@ParamsFromFile(file_types=['.png', '.jpg', '.jpeg'])

Excel vs CSV

If you want to upload an Excel file, the preferred method is to convert it to a plain-text CSV (comma-separated values) format (extension ".csv"). Excel files (".xls", ".xlsx") are not plain-text and special characters are not always imported correctly. An additional advantage of a CSV file is that, being plain-text, they can be compared using version control software (e.g. Git).

info

A CSV file does not support 'sheets' as an Excel file does, meaning that a CSV file must be created for each sheet in a multi-sheet Excel file.