Review application A.4.1
Backend tests
On this page
The application's Python tests live in tests/. They cover the Flask blueprints, queries, bundle import and publication, scheduling, the documentation portal and the export services. Keep their environment separate from the engine's: the application consumes a versioned bundle and does not need the engine package installed.
Setup and execution
make quality is the supported complete run. It supplies Python 3.13, native PDF dependencies, an isolated engine interpreter and disposable PostGIS. For native focused development, use Python 3.13 and install the app dependencies in their own virtual environment:
python3.13 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements.txt -r requirements-dev.txt
DATABASE_URI=sqlite:// python -m pytest tests/test_result_bundle.py tests/test_bundle_properties.py -q
Native PDF tests also require WeasyPrint's platform libraries. Use the quality container if these are unavailable; an import or missing-library failure is an environment failure, not proof of a product regression. A4.4 Running the quality checks covers complete native setup, supported versions and Docker troubleshooting.
Run an individual test with python -m pytest tests/test_file.py::test_name -q, or use -k expression to select a behaviour. Use DATABASE_URI=sqlite:// for app-only runs. ENGINE_DIR selects the engine source/documentation checkout when it is not in the default sibling or nested location (A4.4); it does not install the engine into the app environment.
The full quality run collects branch coverage of backend/ and the test results; A4.4 lists the evidence files and explains coverage sanity.
Behaviour and failure-path coverage
| Area | Representative tests | What the assertions establish |
|---|---|---|
| Result contract | test_result_bundle.py, test_bundle_properties.py, test_import_projection.py |
Engine-free consumption, valid projection, invalid container/row/identifier/coordinate rejection and cross-section references |
| Publication safety | test_snapshot_publication.py |
Private staging, active snapshot preservation, rollback and advisory lock contention |
| Schema | test_migrations.py |
Upgrading an empty PostGIS database gives exactly the schema of backend/models.py |
| Engine import contract | test_engine_import_contract.py |
Real engine output from the three examples: progress events resolve in the consumer, each bundle publishes into PostGIS, stored match types come from the bundle and are all known to matching_rules.py, and together the examples fill every snapshot table |
| Database connections | test_db_connection_options.py |
Web connection timeouts and timeout error detection |
| Filters and API | test_filter_semantics.py, test_validators.py, test_routes_blueprint.py |
Shared filter meaning, parameter validation, pagination and maintenance behaviour |
| Scheduling and progress | test_scheduler_service.py, test_job_runner.py, test_pipeline_status.py, test_pipeline_progress.py |
Execution/state ownership, recovery and event reduction |
| Statistics and exports | test_global_stats_service.py, test_stats_export.py, test_async_export_payloads.py |
Aggregation, filter-preserving exports, asynchronous request validation and the shared /api/exports/<task_id>/… endpoints |
| Documentation and PDFs | test_docs_links.py, test_docs_diagrams.py, test_docs_rendering.py, test_docs_pdf_layout.py |
Cross-repository links, resolvable portal and PDF anchors, literal code, generated diagrams that match their source, document selection and SVG/layout/resource policy |
| Quality infrastructure | test_quality_system.py, test_quality_runner.py, test_quality_report.py, test_quality_dashboard.py |
Catalog ownership, forbidden imports, coverage completeness, fingerprints, report normalisation, clone multiplicity, the code map with its unused functions and blind spots, runner failure handling, and the dashboard's evidence: the running commit, the download, its cache and missing results |
Hypothesis properties vary data within the public contract instead of restating implementation steps. A minimised failing example should become a useful regression case. Fixed examples remain appropriate for precise endpoint and transactional failure behaviour.
Database and producer isolation
Tests never reach a real database by accident: tests/conftest.py sets DATABASE_URI=sqlite:// before anything is imported, including the importer's connection.
Flask, page and query tests use SQLite. Create the model tables a test needs with the sqlite_tables fixture, for example session = sqlite_tables(StopsMatched, OsmNode); problems_db creates the stop, problem and snapshot tables. Geometry columns become text. Do not write CREATE TABLE statements by hand: model tables keep the tests in step with backend/models.py. SQLite tests cannot establish PostgreSQL transaction or locking behaviour, so do not test the import or publication on SQLite.
PostGIS tests use the postgres fixture. Set TEST_POSTGRES_URI to a disposable PostGIS database whose name ends in _test: the fixture resets its public schema for each test. The full quality command creates this database automatically; never point the variable at a deployment database. test_engine_import_contract.py additionally requires ENGINE_PYTHON (the isolated engine interpreter passed by make quality) so the app suite can invoke the engine CLI as a subprocess without importing the engine package. Focused native runs without TEST_POSTGRES_URI or ENGINE_PYTHON skip these tests (pytest -rs lists them).
tests/fixtures/result-v1 is a small schema-version-1 bundle that the contract tests read without the engine; one of them blocks engine imports outright. To test current engine output, extend the engine examples rather than adding hand-written rows to this fixture.
Adding or changing backend behaviour
Add meaningful acceptance and failure examples, and update the relevant documentation. Assign each new production file to a module of the catalog, as Maintain the module catalog describes. Prefer exercising public behaviour through the existing fixtures. Keep the producer and consumer validators independent: cross-repository compatibility is tested through versioned data, not shared Python imports.
Run focused tests while developing, then make quality-fast or its container equivalent. Finish with make quality for changes affecting contracts, publication, scheduling, dependencies or the quality infrastructure. Inspect failures and raw evidence even if a report was generated: generating a report is not itself a passing gate.
Code
| File | Role |
|---|---|
tests/conftest.py |
Sets DATABASE_URI=sqlite:// before any import; the sqlite_tables, problems_db and postgres fixtures |
tests/fixtures/result-v1/ |
The small schema-version-1 bundle the contract tests read without the engine |
tests/test_engine_import_contract.py |
Runs the engine examples through ENGINE_PYTHON and publishes each bundle into PostGIS |
requirements-dev.txt |
pytest, pytest-cov, Hypothesis, the other test dependencies and the quality analysers |
scripts/run_quality.py |
Runs the app suite with branch coverage of backend/ and passes ENGINE_PYTHON to it |
Related pages
- A4.4 Running the quality checks: the full run, native setup, evidence files and troubleshooting.
- A4.2 JavaScript tests: the browser suite.
- A1.1 Bundle import and publication: the import and publication the PostGIS tests exercise.
- E6 Engine tests: the engine's own suite, run in its own interpreter.