osm merge, optmisation

README.md

@@ -12,7 +12,7 @@ Tauri 2 desktop app for reviewing, correcting and exporting fixed-asset / range-
    - **Range assets (JSON)** — linear assets with start / mid / end (e.g., crash barriers).
    - **KML scope polygon** — defines which assets are "in scope". Points outside the polygon are hidden by default.
    - **OSM roads (GeoJSON)** — imported road network for snapping / direction lookups.
-   - **Metadata polyline (KML)** — vehicle GPS track from the source video; takes priority over OSM for direction inference.
+   - **Per-video metadata (JSON)** — vehicle GPS track per video; takes priority over OSM for direction inference. **Persists in the SQLite database**, so you only pick the file once — every subsequent app start hydrates it automatically. Cleared only by the **Clear** button on the Per-video metadata row, or by **Reset DB**.
 3. The map auto-centers on the first imported asset. Pan / zoom triggers viewport-bounded fetches (debounced) to keep the deck.gl rendering snappy.
 
 ---
@@ -51,6 +51,23 @@ Tauri 2 desktop app for reviewing, correcting and exporting fixed-asset / range-
 
 ---
 
+## Single-video focus mode
+
+When you want to inspect / clean one video's worth of data in isolation:
+
+- **Tap that video's metadata polyline on the map**, or
+- **Leave exactly one video checked in the Filters panel**.
+
+Either path triggers focus mode:
+
+- Other videos' metadata polylines are hidden — only the focused track renders.
+- OSM roads are spatially filtered to those within ~75 m of the focused track (cheap spatial-hash buffer; recomputed on each switch).
+- Clicking the metadata line again, or unchecking the lone video filter, restores the full view.
+
+Focus mode also unlocks the bulk OSM road tools (multi-select / hide / merge — see **OSM road editing**). Selection and hidden state are session-scoped: they reset whenever you exit OSM edit mode or change the focused video.
+
+---
+
 ## Cross-side / cross-video pair linking
 
 Common situation: one physical asset (e.g., a median streetlight) is captured in two videos (LHS and RHS) and shows up as two near-duplicate points. Pair linking marks them as the same physical thing without deleting either.
@@ -97,45 +114,70 @@ Range assets (start / mid / end) translate as a rigid body — all three vertice
 ## OSM road editing
 
 ### Modes
-- **OSM tools…** modal — generate roads for the visible bbox (Overpass), import existing GeoJSON, export current roads, prune small road classes.
+- **OSM tools…** modal — generate roads for the visible bbox or for a buffer around imported assets (Overpass), import existing GeoJSON, export current roads, prune small road classes.
 - **Edit OSM roads** button — toggles OSM edit mode. While on, every other layer is dimmed and only OSM roads are interactive.
+- **Magnifier** button — cursor-following circular lens that zooms +3 levels around the cursor; renders satellite + OSM cyan + dashed-green metadata + red asset dots. Toggle on, hover anywhere, then ESC or click again to exit.
 - **Lane offset (m)** — controls the parallel offset used by snap-to-road. Range 0.1–30 m.
 
-### Picking a road to edit
-1. Turn on **Edit OSM roads** (right panel → Data section).
-2. Zoom in to **z ≥ 14** — vertex handles only render at that zoom or deeper, to keep the map responsive.
-3. Click any road. Its yellow `•` vertex pins appear; other roads stay un-pinned. The selected road's id is shown in the OSM-edit panel.
+### Two click models
 
-### Modifying a road
-- **Drag a yellow `•` pin** — moves that vertex to wherever you drop it. The change is saved immediately.
-- **Click a yellow `+` ghost pin** between two vertices — inserts a new vertex at that midpoint. Useful when an OSM segment is too sparse for the curve.
-- **Flip direction** — cycles `oneway` through forward (`1`) → reverse (`-1`) → undirected (`0`). Use when OSM has the heading wrong on a divided road.
+OSM edit mode behaves differently depending on whether a single video is focused (see **Single-video focus mode**):
+
+- **Non-focus** — click a road to enter the legacy single-road vertex editor (the only road remains visible; vertex pins appear). One road at a time.
+- **Focus mode** — click a road to **toggle multi-select** (yellow highlight). All roads stay visible. When the selection narrows to **exactly one** road, the vertex editor automatically engages on it (Flip / Simplify / Delete this road / Ignore for snap appear inline). Click the road again to drop back to multi-select.
+
+### Modifying a road's vertices (single road active)
+- **Drag a yellow `•` pin** — moves that vertex. Saved immediately.
+- **Click a yellow `+` ghost pin** between two vertices — inserts a new vertex at that midpoint.
+- **Flip direction** — cycles `oneway` through forward (`1`) → reverse (`-1`) → undirected (`0`).
+- Vertex handles render at z ≥ 14; zoom in if you don't see them.
 
 ### Deleting vertices
-Two tools, depending on scope:
-
 **A. Surgical — Shift-click + Del**
-1. Hold **Shift** and click a vertex pin → it turns red with an `✕`.
-2. Repeat to mark more.
-3. Press **Del** (or Backspace) → all marked vertices are removed in one shot.
-4. The "Currently marked: N" label in the panel shows how many you've flagged. **Clear marks** unselects without deleting.
-5. Shift-click a marked vertex again to unmark it individually.
-6. The road must keep ≥ 2 vertices; the backend rejects deletes that would break that.
+1. **Shift-click** a vertex pin → turns red with `✕`. Repeat to mark more.
+2. Press **Del** → all marked vertices removed in one shot.
+3. **Clear marks** unselects without deleting. Shift-click a marked vertex to unmark.
+4. The road must keep ≥ 2 vertices.
 
 **B. Sweeping — Simplify by tolerance**
-1. Use the **Simplify (m)** slider in the panel (0.5–20 m, default 2 m). Higher = more aggressive.
-2. Click **Simplify this road**. Runs Douglas-Peucker on the polyline: any vertex within tolerance of the simplified line is dropped.
+1. **Simplify (m)** slider (0.5–20 m, default 2 m). Higher = more aggressive.
+2. Click **Simplify this road** → Douglas-Peucker on the polyline.
 3. Status bar reports `before → after` vertex counts.
-4. Try 1–2 m for cleanup, 5+ m for heavy decimation. Re-running with the same tolerance is idempotent.
 
-### Deleting / flipping the whole road
-- **Delete this road** — removes the road outright (with confirmation).
-- **Deselect road** — exits the per-road editing tool but keeps OSM edit mode on.
+### Bulk road tools (focus mode only)
 
-### Notes
-- OSM road edits are **not** undoable via the Recent actions panel — they're treated like reference-data tweaks. Be deliberate, especially with delete and simplify.
-- Dragging a vertex triggers a single `update_osm_road` write per drag. Bulk delete and simplify are also single writes.
-- After a vertex change, the road line refreshes in real time but the surrounding draggable pins re-render — slight visual flicker is normal.
+A "Bulk road tools — focus: <video>" panel appears when OSM edit mode is on inside single-video focus. Workflow:
+
+- **Click** roads to build a selection (yellow + thicker stroke).
+- **Right-click** any road → instantly hides it (no selection needed).
+- **Del** → moves all currently-selected roads into the hidden bucket and clears the selection.
+- **Hide selected / Clear selection / Restore hidden** buttons in the panel mirror those shortcuts.
+- **Merge selected** (≥ 2 picked) — concatenates connected ways into one. Endpoints within ~8 m of each other are clustered first, so OSM ways with near-but-not-touching endpoints still merge — both ends are snapped to the cluster centroid so the merged geometry has no visible gap. **Disconnected clusters become independent merged roads** (no false bridging).
+- **Merge all visible** — same logic over every non-hidden road in the focused area.
+- Branched clusters (any node with > 2 incident ways) are skipped rather than corrupted; the toast tells you how many.
+
+Both **hidden** and **selection** sets are session-scoped — cleared on **Exit OSM edit mode** or focused-video change, never persisted.
+
+### Ignore-for-snap (persistent)
+
+Per-road toggle that keeps a road on the map for visual reference but excludes it from snap candidates. Useful for service roads / driveways that the snap should never pick.
+
+- With **one road selected** in the vertex editor, the **Ignore for snap** button appears next to **Flip direction**.
+- Ignored roads render in **grey** (vs. the default cyan) and are skipped by both single-asset and bulk snap.
+- Stored as a `snap_ignored` column on the `roads` table — survives app restarts. Toggle again to re-enable.
+
+### Whole-road actions
+- **Delete this road** — removes the road (with confirmation).
+- **Deselect road** — exits the per-road editor (non-focus only; in focus mode, click the road again).
+
+### Per-session OSM edit undo
+
+A 3-deep ring buffer scoped to the current OSM edit session captures the last edits so you can revert them without affecting the global asset audit log.
+
+- **↶ Undo last edit (N)** button right under "Exit OSM edit mode" — visible in both focused and non-focused mode. Tooltip names the action it will undo.
+- Covers: **flip direction, simplify road, delete vertices, drag vertex, ignore-for-snap toggle, hide road (right-click + Del), restore hidden**.
+- **Not** undoable via this stack: merge selected / merge all visible (they consume source rows; this stack doesn't recreate them), creating new roads, deleting whole roads.
+- Cleared on **Exit OSM edit mode**.
 
 ---
 
@@ -149,6 +191,31 @@ Two tools, depending on scope:
 
 ---
 
+## Quality dashboard
+
+A `<details>` panel in the right sidebar that scans the live asset set for likely-bad rows. Click **Run scan** the first time, **Refresh** thereafter.
+
+Three checks (cap 500 hits each; totals shown even when capped):
+
+- **Moved far from import** — modified rows whose current position is > 50 m from the original (import-time) lat/lng. Flags accidental drags.
+- **Off track (far from metadata polyline)** — perpendicular distance to the asset's video metadata polyline exceeds the configured threshold (default **30 m**, tunable in Settings). Flags assets that were assigned to a video / side they don't actually belong to. Skipped silently for videos with no metadata loaded.
+- **Missing side** — `side IS NULL`. Flags assets the labeller didn't tag.
+
+For any non-zero count, **View** in the row launches a step-through:
+- Zooms the map to z=18 on the first issue.
+- **Prev / Next** in the same panel walks through the rest, one at a time at z=18.
+
+### Post-snap quality check
+
+Every snap entrypoint (single asset, bulk visible, snap by video, snap to road) runs a quality scan immediately after and surfaces a toast:
+
+- **Green** — "No quality issues."
+- **Red** — "N quality issues found (M moved · O off track · P no side)."
+
+The scan reuses your configured off-track threshold. Toast auto-dismisses after 8 s; click `×` to dismiss sooner.
+
+---
+
 ## Centerline override
 
 Some asset classes (expansion joint, vms gantry) belong on the road centerline, not the LHS/RHS offset. Settings → **Snap to centerline (instead of offset lane)** lets you tick those class names.
@@ -161,6 +228,10 @@ Some asset classes (expansion joint, vms gantry) belong on the road centerline,
   - On / off (when off, marker pins still appear but no image opens on click).
   - Width: S / M / L / XL.
   - Aspect ratio: 16:9 / 4:3 / 3:2 / 1:1.
+- **Quality thresholds**
+  - **Off-track distance (m)** — perpendicular distance from metadata polyline beyond which an asset is flagged "off track" by the Quality dashboard and post-snap scan. Default 30 m. **Reset to 30 m** button restores the default. Persisted in localStorage. Lowering below ~25 m starts producing noise on curved / sparse polylines (chord cuts a corner the asset is on).
+- **Diagnostics**
+  - **Performance mode** — while panning / zooming, render only basemap + asset markers + range paths (hide roads / scope / metadata / links / labels). Diagnostic; leave off for normal use.
 - **Snap to centerline names** — list of asset_names that should snap to centerline.
 - **Image folder (offline)** — a local folder used to resolve `image_path` URLs to disk (HTTPS URLs always pass through unchanged).
 
@@ -181,17 +252,27 @@ OSM roads can be exported separately via **OSM tools… → Export current roads
 
 ## Keyboard shortcuts
 
-| Key | Action |
-|---|---|
-| `[` / `]` | Step prev / next asset in the filtered list |
-| `ESC` | Cancel: lasso / link-pick / draw-road / road-edit / image popup |
-| `Delete` / `Backspace` | Unlink the currently-selected link line |
+| Key | Context | Action |
+|---|---|---|
+| `[` / `]` | filtered list | Step prev / next asset |
+| `ESC` | various | Cancel: lasso / link-pick / draw-road / road-edit / image popup / metadata focus / magnifier |
+| `Delete` / `Backspace` | link selected | Unlink the currently-selected link line |
+| `Shift` + click | OSM vertex | Mark / unmark a road vertex (red ✕) |
+| `Delete` / `Backspace` | OSM edit, marked vertices | Delete all marked vertices on the active road |
+| `Delete` / `Backspace` | OSM edit, focus + selection | Hide all currently-selected roads (session-only) |
+| Right-click | OSM edit, focus | Hide that road instantly |
 
 ---
 
 ## Undo / Redo
 
-Every state-changing action writes an audit row. The **Recent actions** panel shows the last N. Click **Undo** on any row to revert; **Redo** on a previously undone row replays it. Multi-asset operations (bulk delete, distribute, auto-link) revert as a single atomic step. Road edits (OSM vertex drag, create / delete / flip) are not audited.
+Two independent stacks:
+
+**Asset audit log** — every state-changing asset action writes an audit row. The **Recent actions** panel shows the last N. **Undo** on any row reverts; **Redo** replays. Multi-asset operations (bulk delete, distribute, auto-link, snap) revert atomically as a single step.
+
+**OSM edit ring buffer** — separate 3-deep, in-session undo for road edits, with its own **↶ Undo last edit** button in the OSM edit panel. Covers flip / simplify / delete vertices / drag vertex / ignore-for-snap toggle / hide / restore-hidden. Cleared on Exit OSM edit mode. Merge and whole-road creation / deletion are **not** captured here — be deliberate with those.
+
+Road edits do not appear in the asset audit log; the asset Undo and the OSM Undo never collide.
 
 ---
 
@@ -210,3 +291,8 @@ Every state-changing action writes an audit row. The **Recent actions** panel sh
 - **Streetlights not visible on satellite** — switch to **Google Satellite (deep zoom)** basemap and zoom to z≥22.
 - **Snap moves assets to the wrong side** — load the metadata polyline (gives true vehicle heading), or fix the OSM road's `oneway` direction.
 - **Re-importing wipes my edits** — re-import preserves rows where `modified=1` (manual moves) and rows the user deleted; only `modified=0` geometry is re-stamped from source. If you need a fresh start, use **Reset DB** in the Data section first.
+- **Per-video metadata didn't auto-load on startup** — confirm the **Per-video metadata** row in the Loaded data section shows a non-zero count. If it does and the polylines aren't drawing, toggle **Metadata** in the Layers picker. If the count is zero after a previous import, the DB was wiped (Reset DB or manual `markers.db` delete) — re-pick the JSON.
+- **Quality "Off track" flags too many / too few rows** — adjust **Settings → Quality thresholds → Off-track distance**. Default 30 m. Going below ~25 m starts producing noise on curved / sparse polylines (chord cuts a corner the asset is on).
+- **Off-track flagged assets that look fine on the map** — check that the asset's `video_name` has metadata loaded. If the metadata is for a different video the chord-distance calc compares against the wrong track.
+- **Bulk road tools panel doesn't show** — needs OSM edit mode AND a focused video. Tap a metadata polyline on the map (or check exactly one video in Filters) to focus, then press **Edit OSM roads**.
+- **Magnifier shows a black circle** — basemap tile fetch hadn't completed when the lens map mounted; move the cursor a couple of pixels to trigger a re-render, or close and re-open the magnifier.