Matching engine E.4
Problems
On this page
After matching, the engine looks for data-quality problems in both datasets, such as matched stops that are far apart or stops without a counterpart. Each problem concerns one stop, OSM element or match, and has a type and a priority, so reviewers can start with what matters most. E4.1 Stop problems lists every type and its priority rules. This chapter explains how problems are detected; A2.4 shows how reviewers browse them.
The engine also compares routes (E3) and exports the result, but derives no problems from that comparison.
Priorities
| Priority | Meaning |
|---|---|
| P1 | Critical: a large discrepancy, such as a match more than 80 m apart, or a missing counterpart |
| P2 | Significant: a moderate discrepancy that needs attention |
| P3 | Minor: a small discrepancy, worth checking |
Priorities follow fixed rules: distance thresholds from the dataset profile, exemptions for operators with large stations (SBB in the Swiss profile), the importance of the attribute that differs, and whether a counterpart exists nearby. E4.1 lists them.
How detection works
Problem detection uses the same records as matching; no database or object mapping is involved. ProblemContext.build() first indexes the complete MatchingOutput once:
| Index | Holds | Used by |
|---|---|---|
osm_kdtree, source_kdtree |
Positions of all OSM elements except disused ones and those kept only for their identifier, and of all source stops | Nearest-counterpart distances |
source_count_by_uic, osm_count_by_uic |
Occurrences per station reference, over matches and unmatched entities | Missing-counterpart checks |
osm_platform_count_by_uic |
Platform-like OSM occurrences per station reference | Platform count comparison |
unmatched_source_by_station_ref |
Unmatched source stops per station reference | Disused problems |
duplicate_key_map |
Source duplicate groups | Duplicate problems |
duplicate_osm_group_map, duplicate_osm_node_ids |
OSM elements with the same UIC number and local_ref |
Duplicate problems |
duplicate_report_by_source, duplicate_report_by_osm |
For each matched duplicate stop, the one match that reports it | Duplicate problems |
source_route_evidence_by_key, osm_node_routes, osm_name_dirs |
The route evidence of matching | Route contradictions; disused problems use osm_node_routes |
The counts count occurrences, not distinct entities: an element that takes part in several matches counts once per match. The context also carries the dataset profile and the run's RouteState, which the rules read for their thresholds and route evidence.
Every problem rule is then a plain function:
def rule(ctx: ProblemContext, record: MatchRecord | SourceStop | OsmNode) -> list[ProblemResult]
Each rule decides from the record's type whether it applies, and returns an empty list otherwise:
| Rule | Applies to |
|---|---|
distance_problem, attributes_problem, contradicts_route_matching_problem |
Matches (MatchRecord) |
unmatched_problem |
Unmatched source stops and OSM elements |
duplicates_problem |
All three |
disused_problem |
Disused OSM elements |
The list of rules, STOP_PROBLEM_PIPELINE, therefore serves matched and unmatched records alike. disused_problem and the identifier rule are not in it:
| Records | Evaluated by |
|---|---|
| Matches | MatchRecord.evaluate_matched_problems(ctx, STOP_PROBLEM_PIPELINE) in api.py |
| Unmatched entities | A loop over STOP_PROBLEM_PIPELINE in api.py, which also records the distance to the nearest counterpart as evidence |
| Disused OSM elements | A loop in api.py that calls only disused_problem and records the lifecycle prefix and conflicting source stops as evidence |
| Active OSM elements with an identifier tag | _identifier_problems() in api.py, which reads each value through IdentifierIndex in core/identifiers.py (identifier problems) |
Effectively matched two-sided stop middles and disused elements are not evaluated as unmatched.
Every rule returns ProblemResult values:
@dataclass(frozen=True)
class ProblemResult:
problem_type: str # distance, attributes, contradicts_route_matching, unmatched, duplicates, disused, identifier
priority: int # 1 = P1, 2 = P2, 3 = P3
has_source_duplicate: bool = False
has_osm_duplicate: bool = False
The engine exports each problem with a stable problem_id, derived from its type and the keys of the affected entities, plus its priority and evidence (result bundle format). The review application stores the exported type and priority; it never re-evaluates the rules.
Code
| Component | File |
|---|---|
| Result type | src/osm_public_transport_matcher/core/problem_detection/result.py |
| Shared indexes | src/osm_public_transport_matcher/core/problem_detection/context.py |
| Rule list | src/osm_public_transport_matcher/core/problem_detection/pipeline.py |
| Rules | src/osm_public_transport_matcher/core/problem_detection/stop_predicates/, one module per rule |
| Invocation and export | src/osm_public_transport_matcher/api.py |
| Import into the review database | backend/importing/importer.py |
| Problem API of the review application | backend/blueprints/problems.py |
Related pages
- E4.1 Stop problems: every problem type and its priorities
- E2 Matching process: the matches and unmatched entities that are checked
- Result bundle format: how problems are written
- A2.4 Problems page: how reviewers browse them