performance improvement

sam-tool-tauri/docs/BUILD_RELEASE.md

@@ -0,0 +1,226 @@
+# Building the Tauri app for release (instead of `dev` mode)
+
+Day-to-day you've been running the desktop app via `npm run tauri dev` —
+that's the dev server: hot-reload, attached browser devtools, slower
+binary, and it stays bound to your terminal. For real use (or shipping to
+another machine), you build a standalone executable + installer.
+
+---
+
+## TL;DR
+
+```bash
+cd sam-tool-tauri
+npm install                  # only on first run / after pulling deps
+npm run tauri build          # produces a release binary + installer
+```
+
+Output lands under
+`sam-tool-tauri/src-tauri/target/release/bundle/`. On Linux you'll find:
+
+- `bundle/deb/sam-tool-tauri_<version>_amd64.deb` — installable on
+  Debian/Ubuntu hosts (`sudo dpkg -i …`).
+- `bundle/appimage/sam-tool-tauri_<version>_amd64.AppImage` — single
+  portable file, `chmod +x` and run.
+- `bundle/rpm/…` if rpm tooling is present.
+
+Plus the raw binary at
+`src-tauri/target/release/sam-tool-tauri` (no installer, just `./run` it).
+
+---
+
+## Prerequisites (Linux)
+
+If you've only ever run `tauri dev`, you might be missing the system
+libraries the bundler needs. On Ubuntu / Debian / Pop!_OS:
+
+```bash
+sudo apt update
+sudo apt install -y \
+    libwebkit2gtk-4.1-dev \
+    libgtk-3-dev \
+    libayatana-appindicator3-dev \
+    librsvg2-dev \
+    patchelf \
+    build-essential \
+    curl wget file \
+    libssl-dev pkg-config
+```
+
+You also need a recent Rust toolchain (Tauri 2 requires `rustc >= 1.77`):
+
+```bash
+rustup update stable
+rustc --version    # confirm >= 1.77
+```
+
+For AppImage bundling specifically, the Tauri CLI auto-fetches the
+AppImage tools on first build. The first run will be slower; subsequent
+builds reuse the cache.
+
+---
+
+## Build profiles
+
+Two ways to produce a release binary, with different tradeoffs:
+
+### A. `npm run tauri build` — full installer build
+
+```bash
+npm run tauri build
+```
+
+What it does:
+1. Runs `vite build` → produces optimized JS/CSS in `dist/`.
+2. Compiles `src-tauri/` Rust crate in `--release` mode.
+3. Bundles everything into platform-native installers under
+   `bundle/`.
+
+Pros: production-grade, optimised, signable, distributable.
+Cons: slow first time (Rust release builds + bundler downloads).
+
+### B. `cargo build --release` — binary only, skip installers
+
+If you only want the executable (e.g. for quick testing on a sibling
+machine on the same OS), drop the bundling step:
+
+```bash
+cd sam-tool-tauri
+npm run build                # bundles the frontend → dist/
+cd src-tauri
+cargo build --release        # binary at target/release/sam-tool-tauri
+```
+
+The resulting binary expects `dist/` from the same checkout to be
+present (Tauri 2 embeds frontend assets, so once compiled it's
+self-contained). Faster than the full bundle, but no `.deb`/`.AppImage`.
+
+### C. Debug build — fast compile, big binary
+
+For iterating on Rust changes without a full release re-compile:
+
+```bash
+cd src-tauri
+cargo build           # debug profile — fast compile, big slow binary
+./target/debug/sam-tool-tauri
+```
+
+This is essentially what `tauri dev` runs under the hood, minus the dev
+server.
+
+---
+
+## Where the artefacts go
+
+```
+sam-tool-tauri/
+├── dist/                                 ← bundled frontend (vite build)
+└── src-tauri/target/
+    ├── debug/sam-tool-tauri              ← debug binary
+    └── release/
+        ├── sam-tool-tauri                ← optimised binary
+        └── bundle/
+            ├── deb/sam-tool-tauri_<v>_amd64.deb
+            ├── appimage/sam-tool-tauri_<v>_amd64.AppImage
+            └── rpm/…    (if `rpm` tools are installed)
+```
+
+`bundle/deb/` and `bundle/appimage/` are the things you'd ship to other
+machines.
+
+---
+
+## Versioning
+
+The version string comes from
+[`src-tauri/tauri.conf.json`](../sam-tool-tauri/src-tauri/tauri.conf.json) →
+`"version": "..."`. Bump it before each release; the value is baked
+into both the binary and the installer filenames.
+
+---
+
+## Smoke-test the release build locally
+
+```bash
+# Run the binary directly:
+./src-tauri/target/release/sam-tool-tauri
+
+# Or install the .deb and launch from your app menu:
+sudo dpkg -i src-tauri/target/release/bundle/deb/sam-tool-tauri_*_amd64.deb
+sam-tool-tauri          # binary on $PATH after dpkg
+
+# Or run the AppImage portably:
+chmod +x src-tauri/target/release/bundle/appimage/sam-tool-tauri_*.AppImage
+./src-tauri/target/release/bundle/appimage/sam-tool-tauri_*.AppImage
+```
+
+The release build behaves exactly like `dev` (same UI, same SAM
+integration), with these differences:
+
+- No devtools by default. To re-enable in a release build, edit
+  `src-tauri/tauri.conf.json` → `"app.windows[0].devtools": true`,
+  rebuild.
+- Faster cold start, smaller memory footprint, optimised JS bundle.
+- React `StrictMode` still doubles state-updaters in dev — but that's a
+  React-side flag, not Tauri's. The stripped-down behaviour comes from
+  Vite's production mode (`process.env.NODE_ENV === 'production'`).
+
+---
+
+## Distributing to another PC
+
+The user only needs **one** of the bundles:
+
+**Debian/Ubuntu:**
+```bash
+scp src-tauri/target/release/bundle/deb/sam-tool-tauri_*_amd64.deb \
+    target-host:/tmp/
+ssh target-host "sudo dpkg -i /tmp/sam-tool-tauri_*_amd64.deb"
+```
+
+**Anywhere with glibc:**
+```bash
+scp src-tauri/target/release/bundle/appimage/sam-tool-tauri_*.AppImage \
+    target-host:/home/user/
+ssh target-host "chmod +x ~/sam-tool-tauri_*.AppImage && ~/sam-tool-tauri_*.AppImage"
+```
+
+The target host needs `webkit2gtk` runtime (most desktops already have
+it) and `ffmpeg` if they want the Extract mode to decode video. The
+sibling SAM2 backend and weights are *not* bundled into the desktop app —
+those live separately and are reached over HTTP, configurable from
+**Settings → SAM backend URL**.
+
+---
+
+## Common build errors
+
+| Error                                                        | Fix                                                                                       |
+|--------------------------------------------------------------|-------------------------------------------------------------------------------------------|
+| `error: linker 'cc' not found`                               | `sudo apt install build-essential`                                                        |
+| `failed to find webkit2gtk-4.1` / `gobject-introspection`    | `sudo apt install libwebkit2gtk-4.1-dev libgtk-3-dev` (see prerequisites)                 |
+| `failed to bundle .deb`                                      | Missing `dpkg-deb`. `sudo apt install dpkg`                                               |
+| AppImage step hangs on first build                           | Tauri downloading appimagetool; wait or pre-fetch under `~/.cache/tauri/`                  |
+| `Could not resolve "@tauri-apps/api"`                         | `npm install` again in `sam-tool-tauri/`                                                  |
+| Build OK on dev machine, fails on Ubuntu 22.04                | webkit2gtk-4.0 vs 4.1 mismatch — Tauri 2 needs 4.1; older Ubuntus may need a backport PPA |
+
+---
+
+## Cross-compilation (Windows / macOS targets)
+
+Out of scope for this repo (Linux-only per CLAUDE.md), but for
+completeness — Tauri supports it via extra rust targets and a Windows or
+macOS host machine. See the upstream Tauri docs:
+https://tauri.app/distribute/
+
+---
+
+## Recap — when to run what
+
+| You want to…                                | Command                                         |
+|---------------------------------------------|-------------------------------------------------|
+| Iterate on UI (hot reload)                  | `npm run tauri dev`                             |
+| Iterate on Rust quickly                     | `cd src-tauri && cargo build && ./target/debug/sam-tool-tauri` |
+| Make a release executable, no installer     | `npm run build && cd src-tauri && cargo build --release`        |
+| Make installers (.deb / AppImage)           | `npm run tauri build`                           |
+| Inspect production frontend without Tauri   | `npm run preview` (browser-only, no native APIs)|

sam-tool-tauri/docs/HF_DEPLOY_OFFLINE.md

@@ -0,0 +1,226 @@
+# Deploying SAM3 via HuggingFace, then going fully offline
+
+The SAM3 backend uses Meta's `facebook/sam3` weights via HuggingFace. The
+flow is **online once** (to download + cache the weights) → **then
+air-gapped** (server runs without ever talking to HuggingFace again).
+This doc walks through both phases and how to verify your box is actually
+offline at the end.
+
+---
+
+## Phase 0 — prerequisites
+
+On the target PC:
+
+- The `sam2-backend/` folder, copied or cloned, with its `.venv`
+  containing `torch`, `transformers>=4.57`, `huggingface_hub`, and the
+  rest of `requirements.txt`. See [`sam2-backend/README.md`](../sam2-backend/README.md)
+  for the full install.
+- A HuggingFace account that has been **granted access to
+  `facebook/sam3`** — gated. Request access at
+  https://huggingface.co/facebook/sam3 (usually approved within minutes).
+- A read token from https://huggingface.co/settings/tokens.
+
+---
+
+## Phase 1 — first-time online download
+
+The first call to `Sam3TrackerModel.from_pretrained("facebook/sam3")`
+streams the weights from HF and writes them to
+`~/.cache/huggingface/hub/`. Everything afterward can be served from that
+cache with no network.
+
+### 1.1 Authenticate
+
+```bash
+cd /home/<you>/sam2-backend
+source .venv/bin/activate
+hf auth login                 # paste your read token
+hf auth whoami                # confirm the right account
+```
+
+Token is stored at `~/.cache/huggingface/token`. It's a plain file with
+`chmod 600`.
+
+### 1.2 Set offline flag to OFF for the first start
+
+In `run.sh`, the relevant lines are:
+```bash
+export HF_HUB_DISABLE_TELEMETRY="${HF_HUB_DISABLE_TELEMETRY:-1}"
+export DO_NOT_TRACK="${DO_NOT_TRACK:-1}"
+export HF_HUB_OFFLINE="${HF_HUB_OFFLINE:-0}"        # ← MUST be 0 first time
+export TRANSFORMERS_OFFLINE="${TRANSFORMERS_OFFLINE:-0}"   # ← MUST be 0 first time
+```
+
+For a one-off override, just don't pass the env vars:
+```bash
+./run.sh
+```
+
+### 1.3 Make ONE successful predict call
+
+Either from the Tauri app (Settings → pick `SAM 3 · tracker (HuggingFace)` →
+draw a bbox), or directly:
+
+```bash
+curl -s -X POST "http://localhost:9090/predict?model=sam3_tracker" \
+  -H "Content-Type: application/json" \
+  --data '@some_test_payload.json'
+```
+
+A successful response means the weights are now fully cached. Confirm:
+
+```bash
+ls -la ~/.cache/huggingface/hub/models--facebook--sam3/
+# expect a snapshots/<hash>/ subdir with model.safetensors and configs
+du -sh ~/.cache/huggingface/hub/models--facebook--sam3/
+# ~3-4 GB for SAM3
+```
+
+### 1.4 Optional — pre-warm without a real predict
+
+If you want to download but not yet wire up Tauri, do it from a Python
+shell:
+
+```bash
+python3 - <<'EOF'
+from transformers import Sam3TrackerModel, Sam3TrackerProcessor
+Sam3TrackerProcessor.from_pretrained("facebook/sam3")
+Sam3TrackerModel.from_pretrained("facebook/sam3")
+print("cached.")
+EOF
+```
+
+Same effect — fills `~/.cache/huggingface/hub/`.
+
+---
+
+## Phase 2 — flip to fully offline
+
+Once the cache is populated, set the offline flags so the server never
+tries to revalidate metadata or check for updates.
+
+### 2.1 Edit `run.sh`
+
+Change the defaults to `1`:
+
+```bash
+export HF_HUB_DISABLE_TELEMETRY="${HF_HUB_DISABLE_TELEMETRY:-1}"
+export DO_NOT_TRACK="${DO_NOT_TRACK:-1}"
+export HF_HUB_OFFLINE="${HF_HUB_OFFLINE:-1}"           # ← 1
+export TRANSFORMERS_OFFLINE="${TRANSFORMERS_OFFLINE:-1}"   # ← 1
+```
+
+Or, if you're running via systemd, add to your unit file:
+
+```ini
+[Service]
+Environment=HF_HUB_DISABLE_TELEMETRY=1
+Environment=DO_NOT_TRACK=1
+Environment=HF_HUB_OFFLINE=1
+Environment=TRANSFORMERS_OFFLINE=1
+```
+
+Then `sudo systemctl daemon-reload && sudo systemctl restart sam-backend`.
+
+### 2.2 Restart and verify
+
+```bash
+./run.sh
+```
+
+You should see clean startup logs **without** any `httpx HTTP Request`
+lines hitting `huggingface.co`. The `Loading weights: 685/685 [00:00<00:00]`
+line confirms a cache hit (instant load, no download).
+
+### 2.3 Pin a custom cache location (optional)
+
+If you want the cache somewhere predictable (for backup, rsync to other
+boxes, or to keep `$HOME` clean), set `HF_HOME` in `run.sh`:
+
+```bash
+export HF_HOME=/var/cache/sam-backend/huggingface
+```
+
+Move the existing cache once:
+```bash
+mv ~/.cache/huggingface /var/cache/sam-backend/
+```
+
+Subsequent reads come from there.
+
+---
+
+## Verifying you're really offline
+
+Two-stage check:
+
+### Step 1 — disconnect & restart
+
+Easiest sanity test: pull the network cable (or `sudo ip link set eth0 down`),
+restart the server, hit `/predict` from the Tauri app on the same box.
+If it returns a polygon, you're truly offline.
+
+### Step 2 — packet-level proof
+
+If you can't disconnect (e.g., other services need the network), watch
+for any HF traffic during normal use:
+
+```bash
+sudo tcpdump -nn -i any \
+  'host huggingface.co or host cdn-lfs.huggingface.co' \
+  2>/dev/null
+```
+
+Restart the backend, hit `/predict` ten times from the Tauri app. With
+the OFFLINE flags set you should see **zero packets**.
+
+---
+
+## Moving the cache to a different machine
+
+Useful when you want to deploy to a new box without the first-run
+download (e.g., setting up a second annotation station behind the same
+firewall).
+
+```bash
+# On the source machine:
+tar czf hf-sam3-cache.tar.gz -C ~/.cache huggingface
+scp hf-sam3-cache.tar.gz target-box:/tmp/
+
+# On the target machine:
+mkdir -p ~/.cache
+tar xzf /tmp/hf-sam3-cache.tar.gz -C ~/.cache
+```
+
+Then start the new backend with `HF_HUB_OFFLINE=1` from day one — it'll
+load from the imported cache without ever calling out.
+
+---
+
+## Common gotchas
+
+| Symptom                                                                  | Cause / fix                                                                 |
+|--------------------------------------------------------------------------|-----------------------------------------------------------------------------|
+| `Cannot find the requested files in the cached path` while OFFLINE=1     | Cache is incomplete or in a different `HF_HOME`. Re-run online once.        |
+| `gated repo, access denied`                                               | Token doesn't have `facebook/sam3` access. Accept gate on the model page, then `hf auth login` again. |
+| Many `HEAD https://huggingface.co/...` lines on startup despite OFFLINE=1 | One of the two env flags is missing. Both `HF_HUB_OFFLINE=1` AND `TRANSFORMERS_OFFLINE=1` are required. |
+| `Loading weights: ...` takes minutes even with cache                     | The cache file is on a slow disk (NFS, HDD). Move it to local SSD via `HF_HOME`. |
+| Token leaked into logs                                                   | `hf auth login` only echoes your username, never the token. The token at `~/.cache/huggingface/token` is the only place it's stored. |
+
+---
+
+## What HF still sees in offline mode
+
+**Nothing.** With both `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`:
+
+- `from_pretrained` reads only from `~/.cache/huggingface/hub/` — no API
+  calls, no HEAD revalidation pings.
+- Telemetry is independently off via `HF_HUB_DISABLE_TELEMETRY=1` /
+  `DO_NOT_TRACK=1`.
+- Inference itself never touches the network — your images, bboxes, and
+  resulting masks stay on the box regardless of online/offline mode.
+
+The only thing HF "knows" about you, ever, is "this account downloaded
+`facebook/sam3` once on date X" — recorded server-side at the moment
+`from_pretrained` first ran in Phase 1.

sam-tool-tauri/docs/LINUX_VIDEO_PLAYBACK_CHECKLIST.md

@@ -0,0 +1,137 @@
+# Linux Video Playback Checklist
+
+Use this checklist on Linux machines where video playback or frame stepping feels
+slow or sluggish.
+
+## Baseline Packages
+
+Install or update the common runtime and diagnostic tools:
+
+```bash
+sudo apt update
+sudo apt full-upgrade
+sudo apt install ffmpeg ffprobe vainfo vdpauinfo mesa-utils vulkan-tools
+```
+
+Make sure the user has access to GPU render devices:
+
+```bash
+sudo usermod -aG render,video $USER
+```
+
+Log out and back in after changing groups.
+
+## Tauri / WebKit Runtime
+
+For Debian or Ubuntu systems:
+
+```bash
+sudo apt install libwebkit2gtk-4.1-0 libgtk-3-0 libayatana-appindicator3-1
+```
+
+## Intel GPU
+
+For newer Intel integrated GPUs:
+
+```bash
+sudo apt install intel-media-va-driver intel-media-va-driver-non-free va-driver-all
+```
+
+If VAAPI fails, try launching with:
+
+```bash
+LIBVA_DRIVER_NAME=iHD ./sam-tool
+```
+
+For older Intel GPUs:
+
+```bash
+sudo apt install i965-va-driver
+LIBVA_DRIVER_NAME=i965 ./sam-tool
+```
+
+## AMD GPU
+
+Use Mesa VAAPI / VDPAU drivers:
+
+```bash
+sudo apt install mesa-va-drivers mesa-vdpau-drivers va-driver-all
+```
+
+In the app, prefer `VAAPI`.
+
+## NVIDIA GPU
+
+Use the proprietary NVIDIA driver, not Nouveau, for best decode support:
+
+```bash
+sudo ubuntu-drivers autoinstall
+sudo reboot
+```
+
+After reboot:
+
+```bash
+nvidia-smi
+ffmpeg -hide_banner -hwaccels
+```
+
+In the app, prefer `CUDA`. Try `VDPAU` only as a fallback.
+
+## GStreamer / WebKit Video Stack
+
+These packages are useful for WebKit video support, especially if native
+`<video>` playback is used:
+
+```bash
+sudo apt install \
+  gstreamer1.0-libav \
+  gstreamer1.0-plugins-base \
+  gstreamer1.0-plugins-good \
+  gstreamer1.0-plugins-bad \
+  gstreamer1.0-plugins-ugly \
+  gstreamer1.0-vaapi
+```
+
+## Verification Commands
+
+Check available acceleration support:
+
+```bash
+ffmpeg -hide_banner -hwaccels
+vainfo
+vdpauinfo
+ls -l /dev/dri/renderD*
+```
+
+Test VAAPI:
+
+```bash
+ffmpeg -v verbose -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i input.mp4 -f null -
+```
+
+Test NVIDIA CUDA:
+
+```bash
+ffmpeg -v verbose -hwaccel cuda -i input.mp4 -f null -
+```
+
+## App Settings
+
+Recommended hardware decode setting:
+
+- Intel / AMD: `VAAPI`
+- NVIDIA: `CUDA`
+- Unknown or mixed systems: `auto`
+- If unstable: `software`
+
+## WebKit Workaround
+
+On machines where WebKit or the Intel iGPU compositor is the bottleneck, try:
+
+```bash
+WEBKIT_DISABLE_COMPOSITING_MODE=1 ./sam-tool
+```
+
+This disables WebKit accelerated compositing and can improve playback on some
+Linux systems.

sam-tool-tauri/docs/POLYGON_SMOOTHING.md

@@ -0,0 +1,164 @@
+# Tuning polygon output (point count + edge smoothness)
+
+When SAM returns a mask, the backend converts it to a polygon before sending
+it back to the Tauri annotation tool. Two things you can tune:
+
+1. **Point count** — how many vertices the polygon has. Each vertex is
+   draggable in the UI, so too many makes editing painful.
+2. **Edge smoothness** — how the polygon traces curved edges (smooth curves
+   vs. pixel staircases).
+
+Both are controlled in one file:
+[`sam2-backend/adapters/utils.py`](../sam2-backend/adapters/utils.py).
+
+---
+
+## Where it's called
+
+Both adapters do the same thing in their predict path:
+
+```python
+# sam2-backend/adapters/sam2_adapter.py
+return PredictResult(
+    polygon=_mask_to_polygon(masks_np[best]),
+    score=best_score,
+)
+
+# sam2-backend/adapters/sam3_adapter.py — same call site
+```
+
+The defaults are `min_points=4`, `max_points=10`, `smooth=True`. You can
+override per-call:
+
+```python
+polygon=_mask_to_polygon(masks_np[best], min_points=3, max_points=15)
+```
+
+…or change the defaults in `utils.py::mask_to_polygon` and they'll apply
+to every model.
+
+---
+
+## Knobs
+
+### 1. Point count band — `min_points` / `max_points`
+
+The simplifier uses Douglas–Peucker (`cv2.approxPolyDP`) with a binary
+search over `epsilon` to land in `[min_points, max_points]`. Within that
+band it prefers **more** detail (i.e. closer to `max_points`).
+
+| Band       | Use it when…                                                                |
+|------------|-----------------------------------------------------------------------------|
+| `4..10`    | Default. Comfortable to drag, captures common shapes.                       |
+| `4..15`    | Curvy / organic objects (animals, foliage). 15 vertices follow a curve well. |
+| `3..6`     | Very simple shapes (rectangles, traffic signs). Fewer drag handles.         |
+| `6..20`    | High-fidelity work, you don't mind dragging more.                           |
+
+**Avoid `max_points > 25`** — past that, the binary search needs many
+iterations and the marginal benefit per vertex drops sharply. If you need
+that many points, the bottleneck is mask quality, not simplification.
+
+### 2. Mask smoothing — `smooth=True | False`
+
+Before extracting the contour, the helper optionally runs:
+
+1. **Morphological close + open** (ellipse kernel) — fills single-pixel
+   notches and removes pepper noise.
+2. **Light Gaussian blur + re-threshold** — sub-pixel edge smoothing so
+   curves don't trace as staircases.
+
+Pass `smooth=False` to skip both steps and trace the raw mask exactly. Use
+this when:
+- The mask has fine features that smoothing erases (thin antennae, sharp
+  90° corners that get rounded).
+- You're A/B testing whether smoothing is helping or hurting.
+
+```python
+polygon=_mask_to_polygon(masks_np[best], smooth=False)
+```
+
+### 3. Smoothing kernel size — `_smooth_mask`
+
+Inside `_smooth_mask` (called by `mask_to_polygon` when `smooth=True`),
+kernel size scales with the **blob's bounding box diagonal**:
+
+```python
+k = int(round(blob_diag * 0.015))   # ← morphology kernel multiplier
+k = max(3, min(k | 1, 11))          # odd, clamped 3..11
+
+blur_k = max(3, min(k | 1, 9))      # ← Gaussian blur kernel
+```
+
+**The `0.015` multiplier is the main dial.**
+
+| Multiplier | Effect                                                          |
+|------------|-----------------------------------------------------------------|
+| `0.008`    | Very gentle. Preserves thin features. Use if smoothing dilates the polygon. |
+| `0.015`    | Default. Good balance for typical objects.                       |
+| `0.025`    | Aggressive. Use only on visibly noisy masks; will round corners. |
+
+The 11-pixel hard cap means giant masks don't get insanely smoothed.
+
+### 4. Drop the Gaussian, keep morphology
+
+If hard 90° corners get rounded but you still want morphology to clean up
+notches, delete just the Gaussian step in `_smooth_mask`:
+
+```python
+# remove these two lines
+m = cv2.GaussianBlur(m, (blur_k, blur_k), 0)
+_, m = cv2.threshold(m, 127, 255, cv2.THRESH_BINARY)
+```
+
+This keeps close+open (which is mostly noise removal) and skips the
+sub-pixel curve smoothing.
+
+---
+
+## Diagnostic recipe — find the right knob in 3 minutes
+
+1. **Draw a representative bbox in the Tauri app.** Screenshot the result.
+2. **Set `smooth=False`** in the adapter, restart, draw the same bbox.
+   - If polygon is *better* (less bloat, hugs edges) → smoothing is over-aggressive. Lower the kernel multiplier or delete the Gaussian step.
+   - If polygon is *worse* (jagged stairs) → smoothing was helping. Move on to step 3.
+3. **Bump `max_points` to 15 or 20**, restart.
+   - If polygon is *better* → simplification was too aggressive. Land on `max_points=12..15` as your new default.
+   - If polygon is the same → the mask itself is the bottleneck (see "If the raw mask is bad" below).
+4. **Restore `smooth=True`** and try a model swap (SAM 2.1 large ↔ SAM 3 tracker) — the underlying mask differs noticeably between models.
+
+---
+
+## If the raw mask is bad
+
+If after all the above the polygon still looks wrong, the input to the
+simplifier is bad — fix that, not the polygon code. Common moves:
+
+- **Pad the bbox before sending** to SAM (Meta's reference scripts use
+  `pad=10` pixels). SAM gives crisper masks with a small box margin.
+- **Try a larger SAM model** (e.g. switch the dropdown from `SAM 2.1
+  Hiera Small` to `SAM 2.1 Hiera Large`).
+- **Tighten the bbox** around the actual object — SAM tends to fill the
+  box, so a generous box gets a bloated mask.
+
+---
+
+## Quick reference — every dial in one block
+
+```python
+# sam2-backend/adapters/utils.py
+
+mask_to_polygon(
+    mask,
+    min_points=4,        # never below this
+    max_points=10,       # never above this; fewer = easier to drag
+    smooth=True,         # turn off to trace raw mask
+)
+
+# Inside _smooth_mask:
+k = int(round(blob_diag * 0.015))    # ← MAIN DIAL
+                                      #   smaller = preserve detail
+                                      #   bigger  = more rounded
+```
+
+If you change defaults, restart the backend (`./run.sh`). No frontend
+restart needed — the polygon arrives over the existing `/predict` call.

sam-tool-tauri/src-tauri/src/decoder.rs

@@ -2,7 +2,7 @@ use crate::video::VideoInfo;
 use std::collections::{HashMap, VecDeque};
 use std::io::{BufRead, BufReader};
 use std::path::PathBuf;
-use std::process::{Child, ChildStdout, Command, Stdio};
+use std::process::{Child, ChildStderr, ChildStdout, Command, Stdio};
 use std::sync::{Arc, Condvar, Mutex};
 use std::thread::{self, JoinHandle};
 use std::time::{Duration, Instant};
@@ -14,6 +14,10 @@ const READ_BUF_BYTES: usize = 256 * 1024;
 /// If the requested frame is this far ahead of the worker, kill + reseek.
 const SEEK_AHEAD_THRESHOLD: u64 = 60;
 const DECODE_TIMEOUT: Duration = Duration::from_secs(15);
+/// Keep the last N stderr lines from ffmpeg so error messages can include
+/// the real reason (e.g. "vaapi: failed to load driver") instead of just
+/// "pipe closed before frame completed".
+const STDERR_TAIL_LINES: usize = 32;
 
 struct FrameCache {
     map: HashMap<u64, Vec<u8>>,
@@ -89,20 +93,32 @@ struct SharedState {
 struct Shared {
     state: Mutex<SharedState>,
     cond: Condvar,
+    /// Ring buffer of recent ffmpeg stderr lines, filled by a sibling drain
+    /// thread. Cleared at each new seek.
+    stderr_tail: Mutex<VecDeque<String>>,
 }
 
 pub struct DecoderSession {
     path: PathBuf,
     info: VideoInfo,
+    /// ffmpeg `-hwaccel` argument. None / empty = software decode. Cleared
+    /// on any decode failure so the next attempt uses software — we'd
+    /// rather degrade than keep retrying a flaky GPU path.
+    hwaccel: Option<String>,
     shared: Arc<Shared>,
     worker: Option<JoinHandle<()>>,
+    /// Sibling thread draining ffmpeg's stderr into `shared.stderr_tail`.
+    /// Joined in `stop_worker` so the tail doesn't get clobbered by a
+    /// straggling drain across seeks.
+    stderr_drain: Option<JoinHandle<()>>,
 }
 
 impl DecoderSession {
-    pub fn open(path: PathBuf, info: VideoInfo) -> Self {
+    pub fn open(path: PathBuf, info: VideoInfo, hwaccel: Option<String>) -> Self {
         Self {
             path,
             info,
+            hwaccel: hwaccel.filter(|s| !s.is_empty()),
             shared: Arc::new(Shared {
                 state: Mutex::new(SharedState {
                     child: None,
@@ -115,8 +131,10 @@ impl DecoderSession {
                     shutdown: false,
                 }),
                 cond: Condvar::new(),
+                stderr_tail: Mutex::new(VecDeque::new()),
             }),
             worker: None,
+            stderr_drain: None,
         }
     }
 
@@ -132,6 +150,28 @@ impl DecoderSession {
     }
 
     pub fn decode_at(&mut self, idx: u64) -> Result<Vec<u8>, String> {
+        match self.decode_at_inner(idx) {
+            Ok(bytes) => Ok(bytes),
+            Err(e) => {
+                // Auto-fallback: any decode failure while hwaccel is on
+                // retries once with software. Covers both first-frame init
+                // failures and later seek/codec-edge failures (e.g. a GPU
+                // path that worked on h264 but chokes on an HEVC segment).
+                // Once cleared, subsequent failures propagate.
+                if let Some(prev) = self.hwaccel.take() {
+                    eprintln!(
+                        "[decoder] hwaccel '{prev}' failed ({e}); falling back to software"
+                    );
+                    self.stop_worker();
+                    self.decode_at_inner(idx)
+                } else {
+                    Err(e)
+                }
+            }
+        }
+    }
+
+    fn decode_at_inner(&mut self, idx: u64) -> Result<Vec<u8>, String> {
         if idx >= self.info.total_frames {
             return Err(format!(
                 "frame {} out of range (total {})",
@@ -194,7 +234,8 @@ impl DecoderSession {
     fn seek_and_start_worker(&mut self, idx: u64) -> Result<(), String> {
         self.stop_worker();
 
-        let (child, stdout) = spawn_ffmpeg(&self.path, &self.info, idx)?;
+        let (child, stdout, stderr) =
+            spawn_ffmpeg(&self.path, &self.info, idx, self.hwaccel.as_deref())?;
 
         {
             let mut st = self.shared.state.lock().unwrap();
@@ -205,8 +246,24 @@ impl DecoderSession {
             st.error = None;
             st.shutdown = false;
         }
+        self.shared.stderr_tail.lock().unwrap().clear();
         self.shared.cond.notify_all();
 
+        // Drain ffmpeg's stderr into a ring buffer so error messages can
+        // include the real reason. We must drain even on success — leaving
+        // stderr unread would let the pipe buffer fill and stall ffmpeg.
+        let drain_shared = self.shared.clone();
+        self.stderr_drain = Some(thread::spawn(move || {
+            let reader = BufReader::new(stderr);
+            for line in reader.lines().map_while(Result::ok) {
+                let mut tail = drain_shared.stderr_tail.lock().unwrap();
+                if tail.len() >= STDERR_TAIL_LINES {
+                    tail.pop_front();
+                }
+                tail.push_back(line);
+            }
+        }));
+
         let shared = self.shared.clone();
         let handle = thread::spawn(move || {
             worker_loop(stdout, shared);
@@ -232,6 +289,12 @@ impl DecoderSession {
         if let Some(handle) = self.worker.take() {
             let _ = handle.join();
         }
+        // Killing the child closed stderr, so the drain thread sees EOF and
+        // exits — joining it here prevents a straggling write into the new
+        // session's stderr_tail after the next seek clears it.
+        if let Some(handle) = self.stderr_drain.take() {
+            let _ = handle.join();
+        }
 
         let mut st = self.shared.state.lock().unwrap();
         st.shutdown = false;
@@ -246,15 +309,55 @@ impl Drop for DecoderSession {
     }
 }
 
+/// Pick a VAAPI render node. `SAM_TOOL_VAAPI_DEVICE` overrides; otherwise we
+/// probe `/dev/dri/renderD128..199` for the first that exists. Multi-GPU
+/// systems can end up with the integrated card on D128 and a discrete card
+/// on D129+ — the env var is the escape hatch when the auto pick is wrong.
+fn vaapi_device() -> String {
+    if let Ok(p) = std::env::var("SAM_TOOL_VAAPI_DEVICE") {
+        if !p.is_empty() {
+            return p;
+        }
+    }
+    for n in 128..200u32 {
+        let p = format!("/dev/dri/renderD{n}");
+        if std::path::Path::new(&p).exists() {
+            return p;
+        }
+    }
+    "/dev/dri/renderD128".to_string()
+}
+
 fn spawn_ffmpeg(
     path: &PathBuf,
     info: &VideoInfo,
     idx: u64,
-) -> Result<(Child, BufReader<ChildStdout>), String> {
+    hwaccel: Option<&str>,
+) -> Result<(Child, BufReader<ChildStdout>, ChildStderr), String> {
     let path_str = path.to_str().ok_or_else(|| "non-utf8 path".to_string())?;
     let t = idx as f64 / info.fps;
     let mut cmd = Command::new("ffmpeg");
     cmd.args(["-v", "error", "-threads", "0"]);
+    // -hwaccel must come BEFORE -i. Output goes to CPU memory by default
+    // (no -hwaccel_output_format), so the downstream mjpeg encoder works
+    // unchanged. If the GPU init fails, ffmpeg will exit nonzero and the
+    // worker thread surfaces the error via shared.error.
+    match hwaccel {
+        Some("auto") => {
+            cmd.args(["-hwaccel", "auto"]);
+        }
+        Some("vaapi") => {
+            let dev = vaapi_device();
+            cmd.args(["-hwaccel", "vaapi", "-vaapi_device", &dev]);
+        }
+        Some("cuda") => {
+            cmd.args(["-hwaccel", "cuda"]);
+        }
+        Some("vdpau") => {
+            cmd.args(["-hwaccel", "vdpau"]);
+        }
+        _ => {}
+    }
     if t > 1.0 {
         cmd.args(["-ss", &format!("{:.3}", t - 1.0)]);
         cmd.args(["-i", path_str]);
@@ -273,13 +376,21 @@ fn spawn_ffmpeg(
         "mjpeg",
         "pipe:1",
     ]);
-    cmd.stdout(Stdio::piped()).stderr(Stdio::null());
+    cmd.stdout(Stdio::piped()).stderr(Stdio::piped());
     let mut child = cmd.spawn().map_err(|e| format!("ffmpeg spawn: {e}"))?;
     let stdout = child
         .stdout
         .take()
         .ok_or_else(|| "ffmpeg stdout unavailable".to_string())?;
-    Ok((child, BufReader::with_capacity(READ_BUF_BYTES, stdout)))
+    let stderr = child
+        .stderr
+        .take()
+        .ok_or_else(|| "ffmpeg stderr unavailable".to_string())?;
+    Ok((
+        child,
+        BufReader::with_capacity(READ_BUF_BYTES, stdout),
+        stderr,
+    ))
 }
 
 fn worker_loop(mut stdout: BufReader<ChildStdout>, shared: Arc<Shared>) {
@@ -315,8 +426,19 @@ fn worker_loop(mut stdout: BufReader<ChildStdout>, shared: Arc<Shared>) {
                 shared.cond.notify_all();
             }
             Err(e) => {
+                // ffmpeg closes stderr right alongside stdout; on real
+                // failures the drain thread may not have flushed yet, so
+                // we wait briefly. On a clean exit there's nothing to
+                // flush — collect_stderr_tail short-circuits via the
+                // child's exit code so end-of-stream EOF isn't penalised.
+                let tail = collect_stderr_tail(&shared, Duration::from_millis(150));
                 let mut st = shared.state.lock().unwrap();
-                if e.contains("pipe closed") || e.contains("EOF") {
+                if !tail.is_empty() {
+                    // With `-v error`, anything on stderr means ffmpeg
+                    // actually failed (vs a clean exit) — surface it as a
+                    // real error so decode_at can fall back / report.
+                    st.error = Some(format!("{e}: ffmpeg: {tail}"));
+                } else if e.contains("pipe closed") || e.contains("EOF") {
                     st.eof = true;
                 } else {
                     st.error = Some(e);
@@ -328,6 +450,50 @@ fn worker_loop(mut stdout: BufReader<ChildStdout>, shared: Arc<Shared>) {
     }
 }
 
+fn read_tail_now(shared: &Arc<Shared>) -> Option<String> {
+    let tail = shared.stderr_tail.lock().unwrap();
+    if tail.is_empty() {
+        None
+    } else {
+        Some(tail.iter().cloned().collect::<Vec<_>>().join(" | "))
+    }
+}
+
+fn collect_stderr_tail(shared: &Arc<Shared>, max_wait: Duration) -> String {
+    // Fast path: drain already flushed — common when ffmpeg writes its
+    // error before exiting and stdout EOF arrives after the drain.
+    if let Some(tail) = read_tail_now(shared) {
+        return tail;
+    }
+    // Skip the poll on clean ffmpeg exit (exit code 0): natural end-of-
+    // stream has nothing on stderr (we run with `-v error`), and 150ms ×
+    // every EOF seek near the final frames adds up.
+    let exited_cleanly = {
+        let mut st = shared.state.lock().unwrap();
+        st.child
+            .as_mut()
+            .and_then(|c| c.try_wait().ok().flatten())
+            .map(|status| status.success())
+            .unwrap_or(false)
+    };
+    if exited_cleanly {
+        return String::new();
+    }
+    // Slow path: ffmpeg exited non-zero (or hasn't exited yet) and the
+    // drain thread hasn't flushed. Poll briefly so the error message
+    // carries the real reason.
+    let deadline = Instant::now() + max_wait;
+    loop {
+        if let Some(tail) = read_tail_now(shared) {
+            return tail;
+        }
+        if Instant::now() >= deadline {
+            return String::new();
+        }
+        thread::sleep(Duration::from_millis(10));
+    }
+}
+
 fn read_jpeg_bytes(reader: &mut BufReader<ChildStdout>) -> Result<Vec<u8>, String> {
     let mut out = Vec::with_capacity(64 * 1024);
     let mut last = 0u8;

sam-tool-tauri/src-tauri/src/lib.rs

@@ -33,9 +33,14 @@ fn open_video(
         return Err(format!("file not found: {}", p.display()));
     }
     let info = video::probe(&p)?;
-    let decoder = DecoderSession::open(p, info.clone());
+    let settings = settings::load(&app).ok();
-    if let Ok(settings) = settings::load(&app) {
+    let hwaccel = settings
-        decoder.set_cache_capacity(settings.cache_capacity);
+        .as_ref()
+        .map(|s| s.hwaccel.clone())
+        .filter(|s| !s.is_empty());
+    let decoder = DecoderSession::open(p, info.clone(), hwaccel);
+    if let Some(s) = settings.as_ref() {
+        decoder.set_cache_capacity(s.cache_capacity);
     }
     let mut guard = state.decoder.lock().map_err(|e| format!("lock: {e}"))?;
     *guard = Some(decoder);
@@ -117,23 +122,27 @@ fn bulk_extract(
     out_dir: String,
     shapes_by_frame: HashMap<String, Vec<coco::Shape>>,
     author: String,
+    frame_skip: Option<u64>,
 ) -> Result<u64, String> {
     if end < start {
         return Err("end must be >= start".into());
     }
+    let step = frame_skip.unwrap_or(1).max(1);
     let mut guard = state.decoder.lock().map_err(|e| format!("lock: {e}"))?;
     let decoder = guard.as_mut().ok_or("no video loaded")?;
     let dir = PathBuf::from(&out_dir);
-    let total = end - start + 1;
+    let total = ((end - start) / step) + 1;
     let empty: Vec<coco::Shape> = Vec::new();
     let mut done: u64 = 0;
-    for idx in start..=end {
+    let mut idx = start;
+    while idx <= end {
         let shapes = shapes_by_frame.get(&idx.to_string()).unwrap_or(&empty);
         extract::extract_one(decoder, idx, &dir, shapes, &author)?;
         done += 1;
         if done == 1 || done % 5 == 0 || done == total {
             let _ = app.emit("bulk-extract-progress", BulkProgress { done, total });
         }
+        idx += step;
     }
     Ok(done)
 }

sam-tool-tauri/src-tauri/src/settings.rs

@@ -15,6 +15,15 @@ pub struct Settings {
     pub sam_model: Option<String>,
     #[serde(default = "default_cache_capacity")]
     pub cache_capacity: usize,
+    /// ffmpeg `-hwaccel` argument. Empty string = software decode (default).
+    /// Recognized: "auto", "vaapi", "cuda", "vdpau". Applied on next
+    /// `open_video` or worker respawn.
+    #[serde(default)]
+    pub hwaccel: String,
+    /// Decimation factor used by playback (advance by N frames per tick) and
+    /// bulk extract (take every Nth frame in range). 1 = no decimation.
+    #[serde(default = "default_frame_skip")]
+    pub frame_skip: u64,
 }
 
 fn default_sam_url() -> String {
@@ -25,11 +34,18 @@ fn default_cache_capacity() -> usize {
     crate::decoder::DEFAULT_CACHE_CAPACITY
 }
 
+fn default_frame_skip() -> u64 {
+    1
+}
+
 fn normalize(mut s: Settings) -> Settings {
     s.cache_capacity = s.cache_capacity.clamp(
         crate::decoder::MIN_CACHE_CAPACITY,
         crate::decoder::MAX_CACHE_CAPACITY,
     );
+    if s.frame_skip < 1 {
+        s.frame_skip = 1;
+    }
     s
 }
 
@@ -40,6 +56,8 @@ impl Default for Settings {
             sam_enabled: false,
             sam_model: None,
             cache_capacity: default_cache_capacity(),
+            hwaccel: String::new(),
+            frame_skip: default_frame_skip(),
         }
     }
 }

sam-tool-tauri/src/App.tsx

@@ -75,7 +75,7 @@ export default function App() {
       <main>
         {mode === "extract" && (
           <ErrorBoundary label="Extract mode">
-            <ExtractMode username={username} />
+            <ExtractMode username={username} settings={settings} />
           </ErrorBoundary>
         )}
         {mode === "annotate" && (

sam-tool-tauri/src/components/extract/FrameCanvas.tsx

@@ -16,7 +16,10 @@ export const FrameCanvas = forwardRef<HTMLCanvasElement, Props>(function FrameCa
         className="canvas-frame"
         style={{ aspectRatio: `${video.width} / ${video.height}` }}
       >
-        <canvas ref={ref} width={video.width} height={video.height} />
+        {/* Backing-store width/height are set imperatively in drawBitmap so we
+            can cap it at the displayed (CSS) size × DPR — drawing 4K every frame
+            into a 1080p window is the main playback cost on slow systems. */}
+        <canvas ref={ref} />
         {extracted && (
           <div className="frame-extracted-banner">EXTRACTED</div>
         )}

sam-tool-tauri/src/components/shell/SettingsModal.tsx

@@ -257,6 +257,50 @@ export function SettingsModal({ open, onClose, onSaved }: Props) {
               </div>
             </label>
           </div>
+
+          <div className="settings-row">
+            <label>
+              <div className="settings-label">
+                ffmpeg hardware decode
+                <span className="dim"> · applies on next video open</span>
+              </div>
+              <select
+                value={settings.hwaccel}
+                onChange={(e) =>
+                  setSettings({ ...settings, hwaccel: e.target.value })
+                }
+              >
+                <option value="">software (safe default)</option>
+                <option value="auto">auto (let ffmpeg pick)</option>
+                <option value="vaapi">vaapi — Intel / AMD Linux</option>
+                <option value="cuda">cuda — NVIDIA, modern path</option>
+                <option value="vdpau">vdpau — NVIDIA / Linux, legacy</option>
+              </select>
+            </label>
+          </div>
+
+          <div className="settings-row">
+            <label>
+              <div className="settings-label">
+                Frame skip
+                <span className="dim">
+                  {" "}
+                  · 1 = no skip; N = every Nth frame in playback &amp; bulk extract
+                </span>
+              </div>
+              <input
+                type="number"
+                min={1}
+                max={60}
+                step={1}
+                value={settings.frame_skip}
+                onChange={(e) => {
+                  const n = Math.max(1, Math.floor(Number(e.target.value) || 1));
+                  setSettings({ ...settings, frame_skip: n });
+                }}
+              />
+            </label>
+          </div>
         </section>
 
         {error && <div className="picker-error">{error}</div>}

sam-tool-tauri/src/ipc.ts

@@ -37,7 +37,8 @@ export const api = {
     end: number,
     outDir: string,
     shapesByFrame: Record<string, CocoShape[]>,
-    author: string
+    author: string,
+    frameSkip?: number
   ) =>
     invoke<number>("bulk_extract", {
       start,
@@ -45,6 +46,7 @@ export const api = {
       outDir,
       shapesByFrame,
       author,
+      frameSkip,
     }),
   bulkUndo: (start: number, end: number, outDir: string) =>
     invoke<number>("bulk_undo", { start, end, outDir }),

sam-tool-tauri/src/modes/ExtractMode.tsx

@@ -2,7 +2,7 @@ import { useCallback, useEffect, useMemo, useRef, useState } from "react";
 import { open as openDialog } from "@tauri-apps/plugin-dialog";
 import { listen } from "@tauri-apps/api/event";
 import { api } from "../ipc";
-import type { CocoShape, ImportedCoco, VideoInfo } from "../types";
+import type { CocoShape, ImportedCoco, Settings, VideoInfo } from "../types";
 import { FrameCanvas } from "../components/extract/FrameCanvas";
 import { drawOverlay } from "../components/extract/overlay";
 
@@ -11,6 +11,7 @@ const FRAME_BITMAP_CACHE_LIMIT = 24;
 
 interface Props {
   username: string;
+  settings: Settings | null;
 }
 
 interface BulkProgress {
@@ -19,7 +20,14 @@ interface BulkProgress {
   phase: "extract" | "undo";
 }
 
-export function ExtractMode({ username }: Props) {
+export function ExtractMode({ username, settings }: Props) {
+  // frame_skip ≥ 1 means: during playback advance by N frames per tick, and
+  // bulk-extract takes every Nth frame across [start, end]. Kept in a ref so
+  // the long-lived playback loop sees changes without re-running its effect.
+  const frameSkipRef = useRef<number>(Math.max(1, settings?.frame_skip ?? 1));
+  useEffect(() => {
+    frameSkipRef.current = Math.max(1, settings?.frame_skip ?? 1);
+  }, [settings?.frame_skip]);
   const [video, setVideo] = useState<VideoInfo | null>(null);
   const [frameIdx, setFrameIdx] = useState(0);
   const [playing, setPlaying] = useState(false);
@@ -53,9 +61,6 @@ export function ExtractMode({ username }: Props) {
   const frameMapRef = useRef<Map<number, CocoShape[]>>(new Map());
   const overlayVisibleRef = useRef(overlayVisible);
   const annotatedFrames = useRef<Set<number>>(new Set());
-  // Playback clock anchor — set when playback starts, rebased on seeks so a
-  // slider drag during play doesn't get overwritten by the tick loop.
-  const playAnchorRef = useRef<{ time: number; idx: number } | null>(null);
 
   useEffect(() => { frameIdxRef.current = frameIdx; }, [frameIdx]);
   useEffect(() => {
@@ -69,6 +74,12 @@ export function ExtractMode({ username }: Props) {
   const renderedIdx = useRef(-1);
   const decodingInFlight = useRef(false);
   const bitmapCacheRef = useRef<Map<number, ImageBitmap>>(new Map());
+  // Last decode/render error message. Set in `pump` on failure, cleared on
+  // any successful render, and observed by the playback loop so a mid-play
+  // failure stops playback with a visible banner instead of leaving the UI
+  // stuck in "playing" while the canvas freezes.
+  const decodeErrorRef = useRef<string | null>(null);
+  const [playbackError, setPlaybackError] = useState<string | null>(null);
 
   const clearBitmapCache = useCallback(() => {
     for (const bitmap of bitmapCacheRef.current.values()) {
@@ -105,31 +116,82 @@ export function ExtractMode({ username }: Props) {
   const drawBitmap = useCallback(
     (bitmap: ImageBitmap, atFrameIdx: number) => {
       const canvas = canvasRef.current;
-      if (!canvas) return;
+      if (!canvas || !video) return;
       const ctx = canvas.getContext("2d");
       if (!ctx) return;
-      ctx.clearRect(0, 0, canvas.width, canvas.height);
+
-      ctx.drawImage(bitmap, 0, 0, canvas.width, canvas.height);
+      // Backing store = displayed CSS rect × DPR, capped at video native
+      // resolution (anything above is upsampling for no benefit). Halves
+      // drawImage cost on 4K-into-1080p, which is the common slow-system
+      // case. Coords below are in image-pixel space — the transform handles
+      // the mapping to backing pixels, so overlay.ts is unchanged.
+      const rect = canvas.getBoundingClientRect();
+      const dpr = window.devicePixelRatio || 1;
+      const targetW = Math.max(
+        1,
+        Math.min(video.width, Math.round(rect.width * dpr))
+      );
+      const targetH = Math.max(
+        1,
+        Math.min(video.height, Math.round(rect.height * dpr))
+      );
+      if (canvas.width !== targetW) canvas.width = targetW;
+      if (canvas.height !== targetH) canvas.height = targetH;
+
+      ctx.setTransform(
+        targetW / video.width,
+        0,
+        0,
+        targetH / video.height,
+        0,
+        0
+      );
+      ctx.clearRect(0, 0, video.width, video.height);
+      ctx.drawImage(bitmap, 0, 0, video.width, video.height);
 
       if (overlayVisibleRef.current) {
         const shapes = frameMapRef.current.get(atFrameIdx);
         if (shapes && shapes.length > 0) {
-          drawOverlay(ctx, shapes, canvas.width, canvas.height);
+          drawOverlay(ctx, shapes, video.width, video.height);
         }
       }
+      ctx.setTransform(1, 0, 0, 1, 0, 0);
     },
-    []
+    [video]
   );
 
+  // Returns true if we drew, false if we were preempted by a newer request,
+  // and throws if bitmap creation failed — pump uses this to decide whether
+  // to mark the frame rendered or surface the error to the playback loop.
   const renderFrame = useCallback(
-    async (bytes: ArrayBuffer, atFrameIdx: number) => {
+    async (bytes: ArrayBuffer, atFrameIdx: number): Promise<boolean> => {
+      // Diagnostic timing log: set `window.SAM_FRAME_DEBUG = true` in devtools
+      // on a slow machine to see per-frame bitmap and draw timings — confirms
+      // where the bottleneck lives (decode vs createImageBitmap vs drawImage).
+      const debug = (window as unknown as { SAM_FRAME_DEBUG?: boolean })
+        .SAM_FRAME_DEBUG === true;
+      const t0 = debug ? performance.now() : 0;
       const blob = new Blob([bytes], { type: "image/jpeg" });
       let bitmap: ImageBitmap;
-      try { bitmap = await createImageBitmap(blob); }
+      try {
-      catch (e) { console.error("createImageBitmap failed", e); return; }
+        bitmap = await createImageBitmap(blob);
+      } catch (e) {
+        throw new Error(
+          `createImageBitmap failed: ${e instanceof Error ? e.message : String(e)}`
+        );
+      }
+      const t1 = debug ? performance.now() : 0;
       rememberBitmap(atFrameIdx, bitmap);
-      if (requestedIdx.current !== atFrameIdx) return;
+      if (requestedIdx.current !== atFrameIdx) return false;
       drawBitmap(bitmap, atFrameIdx);
+      if (debug) {
+        const t2 = performance.now();
+        // eslint-disable-next-line no-console
+        console.debug(
+          `[SAM] f${atFrameIdx} bytes=${bytes.byteLength} bitmap=${(t1 - t0).toFixed(0)}ms draw=${(t2 - t1).toFixed(0)}ms`
+        );
+      }
+      return true;
     },
     [drawBitmap, rememberBitmap]
   );
@@ -149,11 +211,17 @@ export function ExtractMode({ username }: Props) {
         try {
           const bytes = await api.decodeFrame(want);
           if (requestedIdx.current !== want) continue;
-          await renderFrame(bytes, want);
+          const drew = await renderFrame(bytes, want);
+          if (drew) {
             renderedIdx.current = want;
+            decodeErrorRef.current = null;
+          }
+          // !drew means renderFrame was preempted by a newer request; loop
+          // continues and the new target gets a fresh decode.
         } catch (e) {
           if (requestedIdx.current !== want) continue;
-          console.error("decode_frame failed", e);
+          console.error("decode/render failed", e);
+          decodeErrorRef.current = e instanceof Error ? e.message : String(e);
           break;
         }
       }
@@ -181,47 +249,114 @@ export function ExtractMode({ username }: Props) {
 
   useEffect(() => clearBitmapCache, [clearBitmapCache]);
 
+  // Redraw on canvas resize: backing-store is sized from CSS rect × DPR, so
+  // a window resize/maximize would otherwise leave the old backing dimensions
+  // until the next decode landed. Cheap — just re-runs drawBitmap on the
+  // already-cached frame.
+  useEffect(() => {
+    const canvas = canvasRef.current;
+    if (!canvas || !video) return;
+    let scheduled = false;
+    const ro = new ResizeObserver(() => {
+      if (scheduled) return;
+      scheduled = true;
+      requestAnimationFrame(() => {
+        scheduled = false;
+        const idx = frameIdxRef.current;
+        const cached = getCachedBitmap(idx);
+        if (cached) drawBitmap(cached, idx);
+        else { renderedIdx.current = -1; pump(); }
+      });
+    });
+    ro.observe(canvas);
+    return () => ro.disconnect();
+  }, [video, drawBitmap, getCachedBitmap, pump]);
+
   // --- playback loop ---------------------------------------------------
+  // Render-paced: advance ONLY after the previous frame actually drew. On a
+  // fast box this hits target fps; on a slow box it plays smoothly at the
+  // achievable decode/render rate instead of dropping 9-of-10 frames at the
+  // canvas while the wall-clock UI counter races ahead.
   useEffect(() => {
     if (!playing || !video) return;
-    playAnchorRef.current = {
+    let cancelled = false;
-      time: performance.now(),
+    let nextDueAt = performance.now();
-      idx: frameIdxRef.current,
-    };
 
-    let raf = 0;
+    const waitForFrame = (target: number) =>
-    const tick = () => {
+      new Promise<void>((resolve) => {
-      const anchor = playAnchorRef.current;
+        const check = () => {
-      if (!anchor) return;
-      const elapsed = (performance.now() - anchor.time) / 1000;
-      const want = Math.min(
-        video.total_frames - 1,
-        anchor.idx + Math.floor(elapsed * video.fps * speed)
-      );
-      // Only trigger the review-mode auto-pause once we've actually advanced
-      // past the resume point. Otherwise pressing play while parked on an
-      // annotated frame would re-pause on the very first tick, making the
-      // play button look broken — the user would have to step a frame
-      // forward manually before play could "stick".
           if (
-        reviewMode &&
+            cancelled ||
-        want > anchor.idx &&
+            renderedIdx.current === target ||
-        annotatedFrames.current.has(want)
+            decodeErrorRef.current !== null
           ) {
-        setFrameIdx(want);
+            resolve();
+            return;
+          }
+          requestAnimationFrame(check);
+        };
+        check();
+      });
+
+    const loop = async () => {
+      while (!cancelled && playing) {
+        const skip = Math.max(1, frameSkipRef.current | 0);
+        // Real-time pacing regardless of skip: skip controls density,
+        // `speed` controls speed. Without the skip factor here, skip=3 would
+        // play at 3× wall-clock — which is what the speed slider is for.
+        const interval = (skip * 1000) / (video.fps * speed);
+        const current = frameIdxRef.current;
+        let target = Math.min(video.total_frames - 1, current + skip);
+        if (target <= current) {
           setPlaying(false);
           return;
         }
-      setFrameIdx(want);
+        // Review-mode scan: with skip > 1 a naive +skip jump can leap past
-      if (want >= video.total_frames - 1) { setPlaying(false); return; }
+        // annotations between current+1 and current+skip. Walk that window
-      raf = requestAnimationFrame(tick);
+        // and clamp `target` down to the first annotated frame.
+        if (reviewMode) {
+          for (let i = current + 1; i <= target; i++) {
+            if (annotatedFrames.current.has(i)) {
+              target = i;
+              break;
+            }
+          }
+        }
+        requestedIdx.current = target;
+        pump();
+        await waitForFrame(target);
+        if (cancelled) return;
+        // Decode failed mid-play: stop and surface so the UI doesn't claim to
+        // be playing while the canvas is frozen on the last good frame.
+        if (decodeErrorRef.current) {
+          setPlaybackError(decodeErrorRef.current);
+          setPlaying(false);
+          return;
+        }
+        setFrameIdx(target);
+        // Pause AFTER rendering so the user actually sees the annotated frame.
+        if (reviewMode && annotatedFrames.current.has(target)) {
+          setPlaying(false);
+          return;
+        }
+
+        const now = performance.now();
+        const due = nextDueAt + interval;
+        if (due > now) {
+          await new Promise((r) => setTimeout(r, due - now));
+          nextDueAt = due;
+        } else {
+          // Behind real-time — don't accumulate debt and then "catch up" by
+          // skipping frames; just reset the clock to now.
+          nextDueAt = now;
+        }
+      }
     };
-    raf = requestAnimationFrame(tick);
+    loop();
     return () => {
-      cancelAnimationFrame(raf);
+      cancelled = true;
-      playAnchorRef.current = null;
     };
-  }, [playing, video, speed, reviewMode]);
+  }, [playing, video, speed, reviewMode, pump]);
 
   // --- navigation helpers ----------------------------------------------
   const seekToFrame = useCallback((idx: number) => {
@@ -237,11 +372,6 @@ export function ExtractMode({ username }: Props) {
       pump();
     }
     setFrameIdx((prev) => (prev === clamped ? prev : clamped));
-    // If a playback tick is active, rebase its anchor so the loop continues
-    // from the new position instead of snapping back.
-    if (playAnchorRef.current) {
-      playAnchorRef.current = { time: performance.now(), idx: clamped };
-    }
   }, [drawBitmap, getCachedBitmap, pump, video]);
 
   const stepFrames = useCallback((delta: number) => {
@@ -252,7 +382,15 @@ export function ExtractMode({ username }: Props) {
 
   const togglePlay = useCallback(() => {
     if (!video) return;
-    setPlaying((p) => !p);
+    setPlaying((p) => {
+      if (!p) {
+        // Fresh play attempt — clear any stale decode error so the loop can
+        // re-arm. If it fails again, the banner will reappear.
+        decodeErrorRef.current = null;
+        setPlaybackError(null);
+      }
+      return !p;
+    });
   }, [video]);
 
   const bumpSpeed = useCallback((dir: 1 | -1) => {
@@ -326,22 +464,31 @@ export function ExtractMode({ username }: Props) {
     };
   }, [bulkStart, bulkEnd]);
 
+  const bulkCount = useMemo(() => {
+    if (!bulkRange) return 0;
+    const skip = Math.max(1, settings?.frame_skip ?? 1);
+    return Math.floor((bulkRange.end - bulkRange.start) / skip) + 1;
+  }, [bulkRange, settings?.frame_skip]);
+
   async function runBulkExtract() {
     if (!outputFolder || !bulkRange) return;
     setPlaying(false);
     setBulkError(null);
-    setBulkProgress({ done: 0, total: bulkRange.end - bulkRange.start + 1, phase: "extract" });
+    const skip = Math.max(1, settings?.frame_skip ?? 1);
+    const total = Math.floor((bulkRange.end - bulkRange.start) / skip) + 1;
+    setBulkProgress({ done: 0, total, phase: "extract" });
     const unlisten = await listen<{ done: number; total: number }>(
       "bulk-extract-progress",
       (e) => setBulkProgress({ ...e.payload, phase: "extract" })
     );
+    // Only include annotations for frames we'll actually emit; backend will
+    // step start..end by `skip` and shapesByFrame is keyed by absolute idx.
     const shapesByFrame: Record<string, CocoShape[]> = {};
     if (coco) {
-      for (const [key, shapes] of Object.entries(coco.frames)) {
+      for (let idx = bulkRange.start; idx <= bulkRange.end; idx += skip) {
-        const idx = Number(key);
+        const key = String(idx);
-        if (idx >= bulkRange.start && idx <= bulkRange.end) {
+        const shapes = coco.frames[key];
-          shapesByFrame[key] = shapes;
+        if (shapes) shapesByFrame[key] = shapes;
-        }
       }
     }
     try {
@@ -350,7 +497,8 @@ export function ExtractMode({ username }: Props) {
         bulkRange.end,
         outputFolder,
         shapesByFrame,
-        username
+        username,
+        skip
       );
       const existing = await api.listExtractedFrames(outputFolder);
       setExtractedFrames(new Set(existing));
@@ -401,6 +549,16 @@ export function ExtractMode({ username }: Props) {
       const tag = (e.target as HTMLElement | null)?.tagName;
       if (tag === "INPUT" || tag === "TEXTAREA") return;
       const step = e.ctrlKey || e.metaKey ? 10 : 1;
+      // Adaptive throttle: drop OS key-repeats when the previous step is
+      // still decoding. On fast systems this is a no-op; on slow systems it
+      // prevents a held arrow from queueing 30 seeks/sec the pump can't keep
+      // up with, which is what causes the "huge gap between keypress and
+      // frame" feel.
+      const isHeldRepeat =
+        e.repeat &&
+        (e.key === "ArrowRight" || e.key === "ArrowLeft") &&
+        requestedIdx.current !== renderedIdx.current;
+      if (isHeldRepeat) { e.preventDefault(); return; }
       switch (e.key) {
         case " ":
           e.preventDefault(); togglePlay(); break;
@@ -463,6 +621,8 @@ export function ExtractMode({ username }: Props) {
       setPlaying(false);
       setSpeed(1);
       renderedIdx.current = -1;
+      decodeErrorRef.current = null;
+      setPlaybackError(null);
       // Reset per-video extraction context, then auto-adopt the per-video
       // default folder. If the user already extracted frames into it in a
       // prior session, listExtractedFrames rehydrates them immediately.
@@ -590,7 +750,7 @@ export function ExtractMode({ username }: Props) {
             onClick={runBulkExtract}
             disabled={!outputFolder || !bulkRange || !!bulkProgress}
           >
-            Bulk Extract{bulkRange && ` (${bulkRange.end - bulkRange.start + 1})`}
+            Bulk Extract{bulkRange && ` (${bulkCount})`}
           </button>
           <button
             onClick={runBulkUndo}
@@ -683,6 +843,22 @@ export function ExtractMode({ username }: Props) {
         </button>
         <span className="dim shortcuts">space · ← → · ctrl+← → · ↑ ↓ · H · E · U · [ ]</span>
       </div>
+
+      {playbackError && (
+        <p className="error" role="alert">
+          Playback stopped: {playbackError}
+          {" "}
+          <button
+            className="link"
+            onClick={() => {
+              decodeErrorRef.current = null;
+              setPlaybackError(null);
+            }}
+          >
+            dismiss
+          </button>
+        </p>
+      )}
     </div>
   );
 }

sam-tool-tauri/src/types.ts

@@ -93,6 +93,10 @@ export interface Settings {
   sam_enabled: boolean;
   sam_model?: string | null;
   cache_capacity: number;
+  /** ffmpeg `-hwaccel` argument. "" = software. Recognized: auto, vaapi, cuda, vdpau. */
+  hwaccel: string;
+  /** Decimation factor for playback advancement + bulk extract. 1 = no decimation. */
+  frame_skip: number;
 }
 
 export interface SamModelInfo {