Review application A.3.1
Dependencies and builds
On this page
The engine and the review application are installed separately and never import each other. The engine's pyproject.toml declares NumPy and SciPy plus optional extras for its input adapters. Flask, SQLAlchemy, the PostGIS driver, report rendering and scheduling belong to the review application's requirements.txt. This page lists the images and the requirement files, and explains how releases of both projects are identified and pinned.
Images
The application's dockerfile builds three stages; the engine repository has its own Dockerfile.
| Build | Contents |
|---|---|
Sibling engine Dockerfile |
Engine package, selected adapters and CLI. No backend code. Build from the engine repository root. |
docs-diagram-stage |
Build-only Node environment with local Chromium. Uses the pinned Mermaid package plus puppeteer-core to pre-render content-addressed SVGs; only those SVGs cross into the app image. |
app-stage |
Review app with requirements.txt, templates, static assets, migrations, review configurations and docs, including the engine's Markdown pages. No engine source or package. The migrator service runs it too. |
scheduler-stage |
Deployment composition: app importer/jobs plus a separately installed engine executable. Communicates through files, with no cross-project Python imports. |
A development environment can hold both projects:
python -m pip install -e "../engine[test]"
python -m pip install -r requirements.txt -r requirements-dev.txt
The application tests do not need the engine; only the producer-to-consumer contract test runs both. The tests run in the quality image, not in an application image (A4.4).
Application release identity and Git tag
The review application has its own release identity, independent of the engine:
| Identifier | Example | Meaning |
|---|---|---|
| Application version | 0.6.0 |
Canonical version declared in backend/version.py, shown in the navbar and beside the review application's collection in the documentation sidebar. |
| Application Git tag | v0.6.0 |
Annotated tag attached to the exact application release commit. It exists in the application repository and must never be moved or reused. |
| Application commit SHA | 0123abcd... |
Immutable identity of the application source. Deployments should record the SHA or an image digest derived from it. |
The application and engine are separate repositories, so each may have a tag such as v0.6.0 without conflict. A tag belongs only to the repository in which it was created. An application release tag says which importer, database migrations, API and UI were deployed; it does not select an engine release.
Application release checklist:
- Update
APP_VERSIONinbackend/version.pyand add the release todocumentation/Changelog.md. - Run the application, JavaScript and producer-to-consumer contract tests.
- Commit the complete release, including migrations and documentation.
- Create an annotated
v<application-version>tag on that commit and push it. Never tag an earlier commit while release changes are uncommitted, and never force-move a published tag. - Deploy that exact commit or an image built from it, and record the commit SHA or immutable image digest.
git tag -a v0.6.0 -m "Review application 0.6.0"
git push origin v0.6.0
git rev-list -n 1 v0.6.0
Engine release identity and deployment pin
Do not use the words “version” and “tag” interchangeably. The integration uses four separate identifiers:
| Identifier | Example | Meaning |
|---|---|---|
| Engine package version | 0.1.0 |
Release version declared by the engine and written to bundle manifests as engine_version. |
| Git tag | v0.1.0 |
Human-readable Git/GitHub name attached to the release commit. Published release tags must never be moved or reused. |
| Commit SHA | 0123abcd... |
Immutable identity of the exact engine source used for a build. This is what engine.ref pins. |
| Result schema version | 1 |
Version of the file contract consumed by the app. It changes independently of the engine package version. |
The committed file engine.ref pins the engine: one line with the full commit SHA. CI checks out that commit from the public engine repository. Docker builds install the local checkout in ENGINE_DIR (default ../engine), so before every build, run:
./scripts/verify_engine_checkout.sh
The verifier rejects a checkout that is not exactly at the pinned commit or has uncommitted files. Docker receives the verified directory as a named build context.
Upgrade procedure
- Fetch the engine repository and inspect the intended annotated release tag.
- Resolve it with
git -C ../engine rev-list -n 1 v0.1.0. - Check out that commit and run the engine suite plus the producer-to-consumer contract test.
- Put the full resolved SHA—not a branch name or tag—in
engine.refand commit it. CI then tests the app against that engine. - Run
./scripts/verify_engine_checkout.sh. - Rebuild the app/documentation and scheduler images, then verify that the engine version in the documentation sidebar and the installed
osm-public-transport-matcherpackage report the expected package version.
The package version may change without a schema change when algorithms or documentation change. A breaking result-format change requires a schema-version change and coordinated consumer support.
Python requirements
| File | Contents | Installed in |
|---|---|---|
requirements.txt |
Everything the review application runs: the web app, the importer, the PDF rendering and the scheduler | The app-stage image, and therefore the migrator and the scheduler |
requirements-dev.txt |
The test tools and the quality analysers. Analysers and the property/schema test tools have exact versions, so a gate gives the same answer everywhere | The quality image (Dockerfile.quality) and native development environments, together with requirements.txt |
Neither file lists the engine's dependencies: the scheduler image installs the engine package with the extras of its own pyproject.toml. The quality runner stops before any check when an installed package does not satisfy either file, so a stale image cannot pass. A4.4 describes the CI jobs.
Frontend and documentation-build JavaScript dependencies are pinned in package.json and package-lock.json. The Mermaid package is shared by the browser documentation view and the local PDF pre-renderer so both paths interpret the same diagram syntax. Chromium exists only in the discarded docs-diagram-stage; production PDF generation consumes the resulting SVG cache and has no Node, browser or outbound-network requirement.
The Swiss/acquisition engine extras include duckdb>=1.4,<2 for native GTFS processing. The generic GTFS extra only adds pandas. DuckDB is absent from the independent web image; PostgreSQL COPY uses the app's psycopg dependency. After changing engine or scheduler dependencies, rebuild and recreate the scheduler service so its installed osm-public-transport-matcher executable matches the checkout. See E5.2 Pipeline performance for memory and thread settings and benchmark commands.
Code
| File | Role |
|---|---|
dockerfile |
The docs-diagram-stage, app-stage and scheduler-stage images |
docker-compose.yml |
The services, built from the dockerfile stages with the engine checkout (ENGINE_DIR) as a named build context |
requirements.txt, requirements-dev.txt |
The Python dependencies of the review application, and its test and quality tools |
package.json, package-lock.json |
The pinned frontend and documentation-build JavaScript dependencies |
backend/version.py |
APP_VERSION, the engine source links built from engine.ref, and engine_dir(), which finds the engine checkout |
engine.ref |
The full commit SHA of the pinned engine |
scripts/verify_engine_checkout.sh |
Rejects an engine checkout that is not exactly at the pinned commit or has uncommitted files |
Related pages
- A3 Architecture: the services each image runs as.
- A3.4 Frontend asset management: how the npm packages become the vendored browser files.
- A4.4 Running the quality checks: the CI jobs that test the app against the pinned engine.
- E5.2 Pipeline performance: memory and thread settings of the engine in the scheduler image.