Review application A.2.8
Documentation portal
On this page
The /docs section of the web app presents the Markdown of both projects as one manual and exports it as one PDF. This page explains where the pages come from, how they are numbered and named, how Markdown becomes safe HTML, how links to pages and source files are resolved, and how the PDF stays in step with the portal.
Where the pages come from
Each project keeps its Markdown next to the code it describes. backend/services/docs_sources.py reads both trees for the portal and the PDF:
| Tree | Location | Covers |
|---|---|---|
| Application | documentation/ in this repository |
Bundle import, database, API and UI, deployment, cross-project guidance |
| Engine | documentation/ in the engine checkout, presented as engine/documentation/ |
Input integrations, acquisition, matching, route comparison, problem detection, engine performance |
The engine checkout is ENGINE_DIR if it is set, else engine/ inside the application if it contains a pyproject.toml, else the sibling ../engine (engine_dir() in backend/version.py). The web image contains no engine code: its build copies only the engine's documentation/ and pyproject.toml (for the version shown in the sidebar) to engine/.
Numbers, titles and addresses
Section numbers belong to the project that owns the page, so each repository can renumber its pages without touching the other:
| Prefix | Pages |
|---|---|
0. |
User guide: the start page, the pages for users of the site and the glossary |
E1–E6 |
Matching engine |
A1–A4 |
Review application |
| none | Changelogs and contributing guides |
The sidebar arranges the pages in three collections: User guide (application section 0) under a For users heading, then, under a For developers heading, Matching engine (every engine page) and Review application (the other application pages). Unnumbered pages close the collection of their owner. The collections are listed in _DOC_COLLECTIONS in docs_sources.py.
| Item | Taken from |
|---|---|
Section titles (E2, A3, …) |
A curated label in docs_sources.py, so the sidebar stays short |
| Other page titles | The page's # heading; the number is shown separately |
| Address | The file name without its number: 2.1 Reference matching.md is /docs/reference_matching. When both trees have a page of the same name, the application page keeps the short address and the engine page gets a prefix: /docs/changelog and /docs/engine_changelog |
| Section anchors | The heading text, lower-cased and hyphenated as on GitHub: ## Top N distances is #top-n-distances; a repeated heading gets -1, -2, … |
Any other address, including the file name itself, returns 404, so renaming a file changes its address: update the links that point to it. An unknown anchor opens the page at its top without an error, so renaming a heading breaks links to its section silently; tests/test_docs_rendering.py checks that every /docs/<address>#<anchor> written in a template or script of the application names an existing section.
From Markdown to HTML
backend/services/docs_render.py prepares each page in this order before Mistune renders it:
- Code mentions become links. Inline code that is exactly a file path, such as
backend/app.pyorbackend/app.py::create_app, and bare file names in prose link to the file. A Mermaid node named after a file gets aclickcommand instead, so the diagram source stays valid. - Code blocks are set aside, so the passes below never change an example.
- Links to pages in either tree become
/docs/<address>routes and keep their#anchor. - Other relative links point to source files: application files become GitHub links; engine files are handled as described under Links to source code.
- Alerts written as
> [!NOTE],> [!IMPORTANT]and so on become styled blocks. - Placeholders are filled in prose, never inside code: dataset statistics (
{{stat:summary.match_rate_percent}}) and the colour palette ([[canonical_palette]], used in A2.7). - Images with a relative path such as
images/route-model.svgare served from the owner's tree at/docs/assets/<owner>/images/…. An image written as<img src="images/…" alt="…" width="648">keeps itswidth, in pixels, so a screenshot of a small part of a page is not stretched to the column.
After rendering, a list whose every item reads - <i class="fas fa-route"></i> **Routes** · [Comparing routes](0.2%20Comparing%20routes.md) — One sentence. becomes a grid of cards (_render_card_lists); the start page uses them. GitHub and the PDF show the same list as written, without the icon. Then Bleach removes every tag and attribute outside an allowlist and turns URLs written with http:// or https:// into links, except inside code; then kit diagrams are inlined (see Diagrams) and headings receive their anchors. Only written-out URLs become links: a file name is never mistaken for a web address, although .py and .name are top-level domains.
Links to source code
repo_scanner.py indexes the code files of the application checkout the first time a page needs it: Python, SQL, shell, configuration, JavaScript, TypeScript, CSS and HTML files. It skips hidden directories and every directory named node_modules, venv, __pycache__, data, vendor, build, dist, coverage, htmlcov or generated. The engine checkout that engine_dir() finds is indexed too, below engine/, whether it sits inside the application or beside it, so a written-out engine path such as src/osm_public_transport_matcher/routes.py links to the pinned engine commit. The web image holds only the engine's documentation, so there engine source paths stay unlinked. A file name links only when it identifies one file; when two files share a name, write enough of the path to tell them apart, such as backend/blueprints/docs.py.
Application files link below DOCS_GITHUB_BLOB_BASE, by default https://github.com/openTdataCH/stop_sync_osm_atlas/blob/main/. Engine files link to the public engine repository at the commit pinned in engine.ref, so the links show the code that produced the published data.
Statistics placeholders
docs_stats.py fills the statistics placeholders from data/stats.json (A1.2). Numbers get thousands separators, and keys ending in _percent get their % sign, so do not type one after the placeholder. The portal wraps each value in a <span> for styling, a Mermaid block receives the raw value so its syntax stays valid, and the PDF receives escaped text.
Code quality dashboard
The code quality dashboard at /quality (backend/blueprints/quality.py) is a page of its own, not a documentation page; the sidebar and the header of A4.3 link to it. /api/quality returns the evidence of the commit the site runs as JSON: the report and the code map, with source saying whether it is the CI run of that commit or a local run. When there is none it returns {"available": false} with the commit and the reason, no_results or unknown_commit. backend/services/quality_report.py loads the evidence (A4.4). The dashboard draws the code map with the vendored Cytoscape.js (A3.4); A4.3 explains how to read it.
Neither the renderer nor the dashboard runs tests or measures anything while serving a page.
Diagrams
| Kind | In the portal | In the PDF |
|---|---|---|
| Mermaid block | Rendered in the browser by the vendored Mermaid | Pre-rendered to SVG with npm run docs:render-mermaid, by the Docker build, or downloaded from CI for the site's commit; a note pointing to the online page if none of them has it (A2.5) |
Kit diagram (images/*.svg) |
Inlined into the page, so links in it work | The same file, embedded as a static image |
| Other SVG or bitmap image | A static image | The same file, embedded |
Kit diagrams are generated from documentation/diagrams/catalog.py; npm run docs:diagrams regenerates them.
A browser draws an SVG loaded through <img> as a static picture and ignores its links. The portal therefore replaces the <img> of a kit diagram with the file's markup, after Bleach, because Bleach would strip the SVG. A diagram is inlined only when it lies below an owner's images/ directory and carries the kit's "Generated by documentation/diagrams/build.py" comment. The kit writes no styles or scripts and prefixes its element IDs with the diagram name, so several diagrams can share a page. Other SVGs, such as hand-drawn illustrations with their own stylesheet, stay images, so their CSS cannot affect the page. GitHub and the PDF always show kit diagrams as images, where their links cannot be clicked.
Full pages and partial updates
docs.html is the shell of the portal: the sidebar, the page header, the article, "On this page", previous and next links, and the PDF export dialog.
| Part | Shows |
|---|---|
| Sidebar | A filter box (/ focuses it, Escape clears it), then the collections. Each collection heading is in small capitals with its project's version; each page shows its number in grey before its title, and long titles wrap. Only the page being read is highlighted; the section that holds it is in bold. At the bottom, a link opens the code quality dashboard and another the PDF export for the whole documentation. |
| Page header | The collection and the page number (Matching engine · E.2.1), the page's # heading without its number, the page's links to other pages of the site, and a link that opens the PDF export for this page. The links are listed in _PAGE_HEADER_LINKS in docs.py; A4.3 links to the dashboard. docs.py takes the heading out of the rendered article for it. |
| On this page | The article's ## and ### headings, when there are at least two. On screens 1400 pixels wide or more it is a column beside the article that marks the section being read; on narrower screens it is a collapsed list above the article. |
Colour says which project owns a page, faintly: a dot before each collection heading and the page header's collection, and the tint of the highlighted page. The engine is green, the review application navy and the shared overview purple, as in the system diagrams. The text always names the collection as well, so colour is never the only signal.
Moving between pages does not reload the shell. static/js/docs/portal.js intercepts internal /docs/... links and requests the page with the header X-Docs-Partial: 1. The route then returns JSON with the rendered article, the page header, the address and section, and the previous and next pages; the script swaps the article and the header, rebuilds "On this page" and runs Mermaid again. The portal draws the Mermaid diagrams itself (startOnLoad is off) and scrolls to the section named in the address only once they are drawn, since a drawn diagram is hundreds of pixels taller than its source. A normal request to the same route returns the full page. The sidebar remembers which sections are expanded in localStorage; filtering does not change what it remembers.
PDF export
The export dialog offers this page, the whole documentation or chosen sections; the link that opens it picks the first choice, and chosen sections start from the section being read. It starts a background task at /api/docs/generate_pdf_async and downloads the file when it is ready (A2.5). build_docs_pdf.py takes the pages, their order, the section keys and the links between pages from docs_sources.py, and the same placeholder and alert helpers as the portal, adapted to one document:
| Portal | |
|---|---|
/docs/<address> |
#doc- followed by the address, such as #doc-reference_matching |
/docs/<address>#<section> |
The page anchor followed by --<section>, such as #doc-reference_matching--platform-codes |
| A page left out of a partial PDF | The page's source on GitHub |
A page of the web app, such as /quality |
The link text only |
Sections are selected by owner-qualified identifiers such as engine:2, app:2 and engine:extra-changelog, so equal numbers and file names in the two trees never collide. A single page is selected by its address and downloads under it, such as reference_matching.pdf. The complete PDF is cached and rebuilt when any page, image, statistic (data/stats.json), print style or renderer changes.
Code
| File | Role |
|---|---|
backend/blueprints/docs.py |
Routes, page header, PDF tasks and the cached whole-documentation PDF |
backend/services/docs_sources.py |
Pages, addresses, titles, sidebar collections, section keys and links between pages, for the portal and the PDF |
backend/services/docs_render.py |
Markdown enrichment and sanitising |
backend/services/repo_scanner.py |
Index of code files for automatic links |
backend/services/docs_stats.py |
Statistics placeholders, alerts, palette, heading anchors |
backend/services/quality_report.py |
Loads the code quality evidence of the running commit |
backend/blueprints/quality.py |
Code quality dashboard page and its /api/quality evidence |
templates/pages/docs.html |
Portal shell |
templates/components/docs_page_header.html |
Page header, for full pages and partial updates |
static/js/docs/portal.js |
Partial navigation, sidebar sections and filter, "On this page", PDF export dialog |
documentation/pdf_generator/build_docs_pdf.py |
PDF generator |
documentation/diagrams/build.py |
Kit diagram generator |
Related pages
- Contributing: how to write the pages the portal serves
- A2.5 Reports and PDFs: the background task behind the PDF download
- A1.2 Statistics and timestamps: where the statistics placeholders get their values
- A4.3 Code quality: how to read the code quality dashboard
- A2.7 Colours and styles: the colours of the diagrams and the project marks