Developer's guide Software architecture

28 mai 2024 à 05:35

Software architecture

Table of Contents

The Petit Rapporteur app is a basic Web app built around a REST API.

Basic app architecture schema. The link to the Graf software is not yet implemented.

Front-end

The front-end is built with VueJS 3 and few dependencies:

The following are development dependencies and are not required to run the application in production:

The philosophy of the front-end is to be as dumb as possible and to do as less computation as possible. All processing should be done on the backend side. The frontend mostly only displays data transmitted by the backend and only sometimes has some logic in the way it displays the fields according to their field_type.

Pages

The front-end is composed of three pages:

  • HomeView.vue: page listing all the reports
  • ReportCompletionView.vue: page allowing to explore a report and to complete data through tables
  • ReportEditionView.vue: page allowing to edit a report (only accessible with the report:edit permission)

Components

There are quite a number of components wich are quite diverse. Some are very basic and only contain HTML like some buttons.

One is a bit more complex: ReportFormItem.vue which is used to display reports contents in ReportCompletionView and in ReportEditionView. It is a component which is displayed in recursion and can be display as imbricated <details><summary></summary></details> and also as <table></table> once a container_promo field has been completely deployed.

Tests

Front-end tests are working but not much used. Cypress’ end-to-end tests are not at all used besides the example test of checking the homepage. Whereas the component unit tests only check the specific FieldsTableCell.vue component for a few behaviours. Until now it was assumed as the components are quite simple, we would see them break instantly while developing.

Back-end

The back-end is built with Python FastAPI and some dependencies:

The following are development dependencies and are not required to run the application in production:

Tests

There are quite a number of unit tests. 168 units tests which need about 4 minutes to be run at the time of writing. Here unit tests are important because they help to catch regressions and to test multiple cases.

Models

The API handles these type of models:

  • Field: one field of a report.
  • Report: a report. Technically it is simply stored in the database as a Field with parent_id: null.
  • Policy: a list of allowed actions on a report.
  • Share: a list of policies. It is what is used to be encoded in the access token to allow access to a report.
  • Token: an access token containing a share.
  • User: a “user” even if it doesn’t mean much today. Currently there is one hard-coded root user which you can setup the password through the ROOT_PASSWORD_HASH and ROOT_PASSWORD_SALT environment variables. It is used to access the API Web interface (/api/docs) to manually use routes.

Handling report’s fields

All report’s fields are stored in the fields database table. Fields are linked to each other through their id and parent_id attributes in order to form a tree of fields. To fetch report with id 1 and all its fields, you need to use a recursive CTE in the SQL query to get the field with id 1 and retrieve all linked fields through their parent_id.

For example:

WITH RECURSIVE report_1 as (
        SELECT id,
            name,
            parent_id,
            data_id,
            field_type,
            template_id,
            0 AS generation_number
        FROM fields
        WHERE id = 1

    UNION ALL

        SELECT child.id,
            child.name,
            child.parent_id,
            child.data_id,
            child.field_type,
            child.template_id,
            generation_number+1 AS generation_number
        FROM fields child
        JOIN report_1 v
          ON v.id = child.parent_id)

SELECT * FROM report_1;

To facilitate retrieving reports, views and materialized views are used in PostgreSQL. For instance, report 1 can be instead be fetched the following way:

SELECT * FROM view_report_1;

Users can filter and search through reports. This processing can be quite heavy. Therefore materialized views with these filters are used to optimize performance.

As described above, fields are structured as a nested tree. In order to filter through columns, we first need to transform this tree into a table. PostgreSQL’s crosstab function is used for this purpose.

  • For the CSV export we do the following processing: Recursive CTE -> Crosstab
  • For the JSON export we do the following processing: Recursive CTE -> Crosstab -> Recursive CTE on the filtered fields

Therefore, the CSV export is faster than the JSON one (which is almost the same format used by the Web interface).

If you look in the code, you will see additional SQL views named view_field_tree_$field_id. They behave like a report view but can start at any field. The reason they exist it because in the permission system design it was expected that you could also precise in which sub-fields a user can access. Currently this feature doesn’t really work and only allows access to this specific field, but not to its children. For the moment the discussion is still open wether this feature should be removed or not.

Following is a schema with the views currently in use:

All views and materialized views currently used in the app