Matching engine E.4

Problems

After matching, the engine looks for data-quality problems in both datasets, such as matched stops that are far apart or stops without a counterpart. Each problem concerns one stop, OSM element or match, and has a type and a priority, so reviewers can start with what matters most. E4.1 Stop problems lists every type and its priority rules. This chapter explains how problems are detected; A2.4 shows how reviewers browse them.

The engine also compares routes (E3) and exports the result, but derives no problems from that comparison.

Priorities

Priority Meaning
P1 Critical: a large discrepancy, such as a match more than 80 m apart, or a missing counterpart
P2 Significant: a moderate discrepancy that needs attention
P3 Minor: a small discrepancy, worth checking

Priorities follow fixed rules: distance thresholds from the dataset profile, exemptions for operators with large stations (SBB in the Swiss profile), the importance of the attribute that differs, and whether a counterpart exists nearby. E4.1 lists them.

How detection works

Problem detection uses the same records as matching; no database or object mapping is involved. ProblemContext.build() first indexes the complete MatchingOutput once:

Index Holds Used by
osm_kdtree, source_kdtree Positions of all OSM elements except disused ones and those kept only for their identifier, and of all source stops Nearest-counterpart distances
source_count_by_uic, osm_count_by_uic Occurrences per station reference, over matches and unmatched entities Missing-counterpart checks
osm_platform_count_by_uic Platform-like OSM occurrences per station reference Platform count comparison
unmatched_source_by_station_ref Unmatched source stops per station reference Disused problems
duplicate_key_map Source duplicate groups Duplicate problems
duplicate_osm_group_map, duplicate_osm_node_ids OSM elements with the same UIC number and local_ref Duplicate problems
duplicate_report_by_source, duplicate_report_by_osm For each matched duplicate stop, the one match that reports it Duplicate problems
source_route_evidence_by_key, osm_node_routes, osm_name_dirs The route evidence of matching Route contradictions; disused problems use osm_node_routes

The counts count occurrences, not distinct entities: an element that takes part in several matches counts once per match. The context also carries the dataset profile and the run's RouteState, which the rules read for their thresholds and route evidence.

Every problem rule is then a plain function:

def rule(ctx: ProblemContext, record: MatchRecord | SourceStop | OsmNode) -> list[ProblemResult]

Each rule decides from the record's type whether it applies, and returns an empty list otherwise:

Rule Applies to
distance_problem, attributes_problem, contradicts_route_matching_problem Matches (MatchRecord)
unmatched_problem Unmatched source stops and OSM elements
duplicates_problem All three
disused_problem Disused OSM elements

The list of rules, STOP_PROBLEM_PIPELINE, therefore serves matched and unmatched records alike. disused_problem and the identifier rule are not in it:

Records Evaluated by
Matches MatchRecord.evaluate_matched_problems(ctx, STOP_PROBLEM_PIPELINE) in api.py
Unmatched entities A loop over STOP_PROBLEM_PIPELINE in api.py, which also records the distance to the nearest counterpart as evidence
Disused OSM elements A loop in api.py that calls only disused_problem and records the lifecycle prefix and conflicting source stops as evidence
Active OSM elements with an identifier tag _identifier_problems() in api.py, which reads each value through IdentifierIndex in core/identifiers.py (identifier problems)

Effectively matched two-sided stop middles and disused elements are not evaluated as unmatched.

Every rule returns ProblemResult values:

@dataclass(frozen=True)
class ProblemResult:
    problem_type: str   # distance, attributes, contradicts_route_matching, unmatched, duplicates, disused, identifier
    priority: int       # 1 = P1, 2 = P2, 3 = P3
    has_source_duplicate: bool = False
    has_osm_duplicate: bool = False

The engine exports each problem with a stable problem_id, derived from its type and the keys of the affected entities, plus its priority and evidence (result bundle format). The review application stores the exported type and priority; it never re-evaluates the rules.

Code

Component File
Result type src/osm_public_transport_matcher/core/problem_detection/result.py
Shared indexes src/osm_public_transport_matcher/core/problem_detection/context.py
Rule list src/osm_public_transport_matcher/core/problem_detection/pipeline.py
Rules src/osm_public_transport_matcher/core/problem_detection/stop_predicates/, one module per rule
Invocation and export src/osm_public_transport_matcher/api.py
Import into the review database backend/importing/importer.py
Problem API of the review application backend/blueprints/problems.py