Ravi/SR-Label-SamTool
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first version

Browse Source
This commit is contained in:
ravi
2026-04-25 16:23:09 +05:30
commit 7d3c36050b
73 changed files with 19033 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

226
docs/BUILD_RELEASE.md Normal file
View File

@@ -0,0 +1,226 @@
# Building the Tauri app for release (instead of `dev` mode)
Day-to-day you've been running the desktop app via `npm run tauri dev` —
that's the dev server: hot-reload, attached browser devtools, slower
binary, and it stays bound to your terminal. For real use (or shipping to
another machine), you build a standalone executable + installer.
---
## TL;DR
```bash
cd sam-tool-tauri
npm install # only on first run / after pulling deps
npm run tauri build # produces a release binary + installer
```
Output lands under
`sam-tool-tauri/src-tauri/target/release/bundle/`. On Linux you'll find:
- `bundle/deb/sam-tool-tauri_<version>_amd64.deb` — installable on
Debian/Ubuntu hosts (`sudo dpkg -i …`).
- `bundle/appimage/sam-tool-tauri_<version>_amd64.AppImage` — single
portable file, `chmod +x` and run.
- `bundle/rpm/…` if rpm tooling is present.
Plus the raw binary at
`src-tauri/target/release/sam-tool-tauri` (no installer, just `./run` it).
---
## Prerequisites (Linux)
If you've only ever run `tauri dev`, you might be missing the system
libraries the bundler needs. On Ubuntu / Debian / Pop!_OS:
```bash
sudo apt update
sudo apt install -y \
libwebkit2gtk-4.1-dev \
libgtk-3-dev \
libayatana-appindicator3-dev \
librsvg2-dev \
patchelf \
build-essential \
curl wget file \
libssl-dev pkg-config
```
You also need a recent Rust toolchain (Tauri 2 requires `rustc >= 1.77`):
```bash
rustup update stable
rustc --version # confirm >= 1.77
```
For AppImage bundling specifically, the Tauri CLI auto-fetches the
AppImage tools on first build. The first run will be slower; subsequent
builds reuse the cache.
---
## Build profiles
Two ways to produce a release binary, with different tradeoffs:
### A. `npm run tauri build` — full installer build
```bash
npm run tauri build
```
What it does:
1. Runs `vite build` → produces optimized JS/CSS in `dist/`.
2. Compiles `src-tauri/` Rust crate in `--release` mode.
3. Bundles everything into platform-native installers under
`bundle/`.
Pros: production-grade, optimised, signable, distributable.
Cons: slow first time (Rust release builds + bundler downloads).
### B. `cargo build --release` — binary only, skip installers
If you only want the executable (e.g. for quick testing on a sibling
machine on the same OS), drop the bundling step:
```bash
cd sam-tool-tauri
npm run build # bundles the frontend → dist/
cd src-tauri
cargo build --release # binary at target/release/sam-tool-tauri
```
The resulting binary expects `dist/` from the same checkout to be
present (Tauri 2 embeds frontend assets, so once compiled it's
self-contained). Faster than the full bundle, but no `.deb`/`.AppImage`.
### C. Debug build — fast compile, big binary
For iterating on Rust changes without a full release re-compile:
```bash
cd src-tauri
cargo build # debug profile — fast compile, big slow binary
./target/debug/sam-tool-tauri
```
This is essentially what `tauri dev` runs under the hood, minus the dev
server.
---
## Where the artefacts go
```
sam-tool-tauri/
├── dist/ ← bundled frontend (vite build)
└── src-tauri/target/
├── debug/sam-tool-tauri ← debug binary
└── release/
├── sam-tool-tauri ← optimised binary
└── bundle/
├── deb/sam-tool-tauri_<v>_amd64.deb
├── appimage/sam-tool-tauri_<v>_amd64.AppImage
└── rpm/… (if `rpm` tools are installed)
```
`bundle/deb/` and `bundle/appimage/` are the things you'd ship to other
machines.
---
## Versioning
The version string comes from
[`src-tauri/tauri.conf.json`](../sam-tool-tauri/src-tauri/tauri.conf.json) →
`"version": "..."`. Bump it before each release; the value is baked
into both the binary and the installer filenames.
---
## Smoke-test the release build locally
```bash
# Run the binary directly:
./src-tauri/target/release/sam-tool-tauri
# Or install the .deb and launch from your app menu:
sudo dpkg -i src-tauri/target/release/bundle/deb/sam-tool-tauri_*_amd64.deb
sam-tool-tauri # binary on $PATH after dpkg
# Or run the AppImage portably:
chmod +x src-tauri/target/release/bundle/appimage/sam-tool-tauri_*.AppImage
./src-tauri/target/release/bundle/appimage/sam-tool-tauri_*.AppImage
```
The release build behaves exactly like `dev` (same UI, same SAM
integration), with these differences:
- No devtools by default. To re-enable in a release build, edit
`src-tauri/tauri.conf.json` → `"app.windows[0].devtools": true`,
rebuild.
- Faster cold start, smaller memory footprint, optimised JS bundle.
- React `StrictMode` still doubles state-updaters in dev — but that's a
React-side flag, not Tauri's. The stripped-down behaviour comes from
Vite's production mode (`process.env.NODE_ENV === 'production'`).
---
## Distributing to another PC
The user only needs **one** of the bundles:
**Debian/Ubuntu:**
```bash
scp src-tauri/target/release/bundle/deb/sam-tool-tauri_*_amd64.deb \
target-host:/tmp/
ssh target-host "sudo dpkg -i /tmp/sam-tool-tauri_*_amd64.deb"
```
**Anywhere with glibc:**
```bash
scp src-tauri/target/release/bundle/appimage/sam-tool-tauri_*.AppImage \
target-host:/home/user/
ssh target-host "chmod +x ~/sam-tool-tauri_*.AppImage && ~/sam-tool-tauri_*.AppImage"
```
The target host needs `webkit2gtk` runtime (most desktops already have
it) and `ffmpeg` if they want the Extract mode to decode video. The
sibling SAM2 backend and weights are *not* bundled into the desktop app —
those live separately and are reached over HTTP, configurable from
**Settings → SAM backend URL**.
---
## Common build errors
| Error | Fix |
|--------------------------------------------------------------|-------------------------------------------------------------------------------------------|
| `error: linker 'cc' not found` | `sudo apt install build-essential` |
| `failed to find webkit2gtk-4.1` / `gobject-introspection` | `sudo apt install libwebkit2gtk-4.1-dev libgtk-3-dev` (see prerequisites) |
| `failed to bundle .deb` | Missing `dpkg-deb`. `sudo apt install dpkg` |
| AppImage step hangs on first build | Tauri downloading appimagetool; wait or pre-fetch under `~/.cache/tauri/` |
| `Could not resolve "@tauri-apps/api"` | `npm install` again in `sam-tool-tauri/` |
| Build OK on dev machine, fails on Ubuntu 22.04 | webkit2gtk-4.0 vs 4.1 mismatch — Tauri 2 needs 4.1; older Ubuntus may need a backport PPA |
---
## Cross-compilation (Windows / macOS targets)
Out of scope for this repo (Linux-only per CLAUDE.md), but for
completeness — Tauri supports it via extra rust targets and a Windows or
macOS host machine. See the upstream Tauri docs:
https://tauri.app/distribute/
---
## Recap — when to run what
| You want to… | Command |
|---------------------------------------------|-------------------------------------------------|
| Iterate on UI (hot reload) | `npm run tauri dev` |
| Iterate on Rust quickly | `cd src-tauri && cargo build && ./target/debug/sam-tool-tauri` |
| Make a release executable, no installer | `npm run build && cd src-tauri && cargo build --release` |
| Make installers (.deb / AppImage) | `npm run tauri build` |
| Inspect production frontend without Tauri | `npm run preview` (browser-only, no native APIs)|

226
docs/HF_DEPLOY_OFFLINE.md Normal file
View File

@@ -0,0 +1,226 @@
# Deploying SAM3 via HuggingFace, then going fully offline
The SAM3 backend uses Meta's `facebook/sam3` weights via HuggingFace. The
flow is **online once** (to download + cache the weights) → **then
air-gapped** (server runs without ever talking to HuggingFace again).
This doc walks through both phases and how to verify your box is actually
offline at the end.
---
## Phase 0 — prerequisites
On the target PC:
- The `sam2-backend/` folder, copied or cloned, with its `.venv`
containing `torch`, `transformers>=4.57`, `huggingface_hub`, and the
rest of `requirements.txt`. See [`sam2-backend/README.md`](../sam2-backend/README.md)
for the full install.
- A HuggingFace account that has been **granted access to
`facebook/sam3`** — gated. Request access at
https://huggingface.co/facebook/sam3 (usually approved within minutes).
- A read token from https://huggingface.co/settings/tokens.
---
## Phase 1 — first-time online download
The first call to `Sam3TrackerModel.from_pretrained("facebook/sam3")`
streams the weights from HF and writes them to
`~/.cache/huggingface/hub/`. Everything afterward can be served from that
cache with no network.
### 1.1 Authenticate
```bash
cd /home/<you>/sam2-backend
source .venv/bin/activate
hf auth login # paste your read token
hf auth whoami # confirm the right account
```
Token is stored at `~/.cache/huggingface/token`. It's a plain file with
`chmod 600`.
### 1.2 Set offline flag to OFF for the first start
In `run.sh`, the relevant lines are:
```bash
export HF_HUB_DISABLE_TELEMETRY="${HF_HUB_DISABLE_TELEMETRY:-1}"
export DO_NOT_TRACK="${DO_NOT_TRACK:-1}"
export HF_HUB_OFFLINE="${HF_HUB_OFFLINE:-0}" # ← MUST be 0 first time
export TRANSFORMERS_OFFLINE="${TRANSFORMERS_OFFLINE:-0}" # ← MUST be 0 first time
```
For a one-off override, just don't pass the env vars:
```bash
./run.sh
```
### 1.3 Make ONE successful predict call
Either from the Tauri app (Settings → pick `SAM 3 · tracker (HuggingFace)` →
draw a bbox), or directly:
```bash
curl -s -X POST "http://localhost:9090/predict?model=sam3_tracker" \
-H "Content-Type: application/json" \
--data '@some_test_payload.json'
```
A successful response means the weights are now fully cached. Confirm:
```bash
ls -la ~/.cache/huggingface/hub/models--facebook--sam3/
# expect a snapshots/<hash>/ subdir with model.safetensors and configs
du -sh ~/.cache/huggingface/hub/models--facebook--sam3/
# ~3-4 GB for SAM3
```
### 1.4 Optional — pre-warm without a real predict
If you want to download but not yet wire up Tauri, do it from a Python
shell:
```bash
python3 - <<'EOF'
from transformers import Sam3TrackerModel, Sam3TrackerProcessor
Sam3TrackerProcessor.from_pretrained("facebook/sam3")
Sam3TrackerModel.from_pretrained("facebook/sam3")
print("cached.")
EOF
```
Same effect — fills `~/.cache/huggingface/hub/`.
---
## Phase 2 — flip to fully offline
Once the cache is populated, set the offline flags so the server never
tries to revalidate metadata or check for updates.
### 2.1 Edit `run.sh`
Change the defaults to `1`:
```bash
export HF_HUB_DISABLE_TELEMETRY="${HF_HUB_DISABLE_TELEMETRY:-1}"
export DO_NOT_TRACK="${DO_NOT_TRACK:-1}"
export HF_HUB_OFFLINE="${HF_HUB_OFFLINE:-1}" # ← 1
export TRANSFORMERS_OFFLINE="${TRANSFORMERS_OFFLINE:-1}" # ← 1
```
Or, if you're running via systemd, add to your unit file:
```ini
[Service]
Environment=HF_HUB_DISABLE_TELEMETRY=1
Environment=DO_NOT_TRACK=1
Environment=HF_HUB_OFFLINE=1
Environment=TRANSFORMERS_OFFLINE=1
```
Then `sudo systemctl daemon-reload && sudo systemctl restart sam-backend`.
### 2.2 Restart and verify
```bash
./run.sh
```
You should see clean startup logs **without** any `httpx HTTP Request`
lines hitting `huggingface.co`. The `Loading weights: 685/685 [00:00<00:00]`
line confirms a cache hit (instant load, no download).
### 2.3 Pin a custom cache location (optional)
If you want the cache somewhere predictable (for backup, rsync to other
boxes, or to keep `$HOME` clean), set `HF_HOME` in `run.sh`:
```bash
export HF_HOME=/var/cache/sam-backend/huggingface
```
Move the existing cache once:
```bash
mv ~/.cache/huggingface /var/cache/sam-backend/
```
Subsequent reads come from there.
---
## Verifying you're really offline
Two-stage check:
### Step 1 — disconnect & restart
Easiest sanity test: pull the network cable (or `sudo ip link set eth0 down`),
restart the server, hit `/predict` from the Tauri app on the same box.
If it returns a polygon, you're truly offline.
### Step 2 — packet-level proof
If you can't disconnect (e.g., other services need the network), watch
for any HF traffic during normal use:
```bash
sudo tcpdump -nn -i any \
'host huggingface.co or host cdn-lfs.huggingface.co' \
2>/dev/null
```
Restart the backend, hit `/predict` ten times from the Tauri app. With
the OFFLINE flags set you should see **zero packets**.
---
## Moving the cache to a different machine
Useful when you want to deploy to a new box without the first-run
download (e.g., setting up a second annotation station behind the same
firewall).
```bash
# On the source machine:
tar czf hf-sam3-cache.tar.gz -C ~/.cache huggingface
scp hf-sam3-cache.tar.gz target-box:/tmp/
# On the target machine:
mkdir -p ~/.cache
tar xzf /tmp/hf-sam3-cache.tar.gz -C ~/.cache
```
Then start the new backend with `HF_HUB_OFFLINE=1` from day one — it'll
load from the imported cache without ever calling out.
---
## Common gotchas
| Symptom | Cause / fix |
|--------------------------------------------------------------------------|-----------------------------------------------------------------------------|
| `Cannot find the requested files in the cached path` while OFFLINE=1 | Cache is incomplete or in a different `HF_HOME`. Re-run online once. |
| `gated repo, access denied` | Token doesn't have `facebook/sam3` access. Accept gate on the model page, then `hf auth login` again. |
| Many `HEAD https://huggingface.co/...` lines on startup despite OFFLINE=1 | One of the two env flags is missing. Both `HF_HUB_OFFLINE=1` AND `TRANSFORMERS_OFFLINE=1` are required. |
| `Loading weights: ...` takes minutes even with cache | The cache file is on a slow disk (NFS, HDD). Move it to local SSD via `HF_HOME`. |
| Token leaked into logs | `hf auth login` only echoes your username, never the token. The token at `~/.cache/huggingface/token` is the only place it's stored. |
---
## What HF still sees in offline mode
**Nothing.** With both `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`:
- `from_pretrained` reads only from `~/.cache/huggingface/hub/` — no API
calls, no HEAD revalidation pings.
- Telemetry is independently off via `HF_HUB_DISABLE_TELEMETRY=1` /
`DO_NOT_TRACK=1`.
- Inference itself never touches the network — your images, bboxes, and
resulting masks stay on the box regardless of online/offline mode.
The only thing HF "knows" about you, ever, is "this account downloaded
`facebook/sam3` once on date X" — recorded server-side at the moment
`from_pretrained` first ran in Phase 1.

164
docs/POLYGON_SMOOTHING.md Normal file
View File

@@ -0,0 +1,164 @@
# Tuning polygon output (point count + edge smoothness)
When SAM returns a mask, the backend converts it to a polygon before sending
it back to the Tauri annotation tool. Two things you can tune:
1. **Point count** — how many vertices the polygon has. Each vertex is
draggable in the UI, so too many makes editing painful.
2. **Edge smoothness** — how the polygon traces curved edges (smooth curves
vs. pixel staircases).
Both are controlled in one file:
[`sam2-backend/adapters/utils.py`](../sam2-backend/adapters/utils.py).
---
## Where it's called
Both adapters do the same thing in their predict path:
```python
# sam2-backend/adapters/sam2_adapter.py
return PredictResult(
polygon=_mask_to_polygon(masks_np[best]),
score=best_score,
)
# sam2-backend/adapters/sam3_adapter.py — same call site
```
The defaults are `min_points=4`, `max_points=10`, `smooth=True`. You can
override per-call:
```python
polygon=_mask_to_polygon(masks_np[best], min_points=3, max_points=15)
```
…or change the defaults in `utils.py::mask_to_polygon` and they'll apply
to every model.
---
## Knobs
### 1. Point count band — `min_points` / `max_points`
The simplifier uses Douglas–Peucker (`cv2.approxPolyDP`) with a binary
search over `epsilon` to land in `[min_points, max_points]`. Within that
band it prefers **more** detail (i.e. closer to `max_points`).
| Band | Use it when… |
|------------|-----------------------------------------------------------------------------|
| `4..10` | Default. Comfortable to drag, captures common shapes. |
| `4..15` | Curvy / organic objects (animals, foliage). 15 vertices follow a curve well. |
| `3..6` | Very simple shapes (rectangles, traffic signs). Fewer drag handles. |
| `6..20` | High-fidelity work, you don't mind dragging more. |
**Avoid `max_points > 25`** — past that, the binary search needs many
iterations and the marginal benefit per vertex drops sharply. If you need
that many points, the bottleneck is mask quality, not simplification.
### 2. Mask smoothing — `smooth=True | False`
Before extracting the contour, the helper optionally runs:
1. **Morphological close + open** (ellipse kernel) — fills single-pixel
notches and removes pepper noise.
2. **Light Gaussian blur + re-threshold** — sub-pixel edge smoothing so
curves don't trace as staircases.
Pass `smooth=False` to skip both steps and trace the raw mask exactly. Use
this when:
- The mask has fine features that smoothing erases (thin antennae, sharp
90° corners that get rounded).
- You're A/B testing whether smoothing is helping or hurting.
```python
polygon=_mask_to_polygon(masks_np[best], smooth=False)
```
### 3. Smoothing kernel size — `_smooth_mask`
Inside `_smooth_mask` (called by `mask_to_polygon` when `smooth=True`),
kernel size scales with the **blob's bounding box diagonal**:
```python
k = int(round(blob_diag * 0.015)) # ← morphology kernel multiplier
k = max(3, min(k | 1, 11)) # odd, clamped 3..11
blur_k = max(3, min(k | 1, 9)) # ← Gaussian blur kernel
```
**The `0.015` multiplier is the main dial.**
| Multiplier | Effect |
|------------|-----------------------------------------------------------------|
| `0.008` | Very gentle. Preserves thin features. Use if smoothing dilates the polygon. |
| `0.015` | Default. Good balance for typical objects. |
| `0.025` | Aggressive. Use only on visibly noisy masks; will round corners. |
The 11-pixel hard cap means giant masks don't get insanely smoothed.
### 4. Drop the Gaussian, keep morphology
If hard 90° corners get rounded but you still want morphology to clean up
notches, delete just the Gaussian step in `_smooth_mask`:
```python
# remove these two lines
m = cv2.GaussianBlur(m, (blur_k, blur_k), 0)
_, m = cv2.threshold(m, 127, 255, cv2.THRESH_BINARY)
```
This keeps close+open (which is mostly noise removal) and skips the
sub-pixel curve smoothing.
---
## Diagnostic recipe — find the right knob in 3 minutes
1. **Draw a representative bbox in the Tauri app.** Screenshot the result.
2. **Set `smooth=False`** in the adapter, restart, draw the same bbox.
- If polygon is *better* (less bloat, hugs edges) → smoothing is over-aggressive. Lower the kernel multiplier or delete the Gaussian step.
- If polygon is *worse* (jagged stairs) → smoothing was helping. Move on to step 3.
3. **Bump `max_points` to 15 or 20**, restart.
- If polygon is *better* → simplification was too aggressive. Land on `max_points=12..15` as your new default.
- If polygon is the same → the mask itself is the bottleneck (see "If the raw mask is bad" below).
4. **Restore `smooth=True`** and try a model swap (SAM 2.1 large ↔ SAM 3 tracker) — the underlying mask differs noticeably between models.
---
## If the raw mask is bad
If after all the above the polygon still looks wrong, the input to the
simplifier is bad — fix that, not the polygon code. Common moves:
- **Pad the bbox before sending** to SAM (Meta's reference scripts use
`pad=10` pixels). SAM gives crisper masks with a small box margin.
- **Try a larger SAM model** (e.g. switch the dropdown from `SAM 2.1
Hiera Small` to `SAM 2.1 Hiera Large`).
- **Tighten the bbox** around the actual object — SAM tends to fill the
box, so a generous box gets a bloated mask.
---
## Quick reference — every dial in one block
```python
# sam2-backend/adapters/utils.py
mask_to_polygon(
mask,
min_points=4, # never below this
max_points=10, # never above this; fewer = easier to drag
smooth=True, # turn off to trace raw mask
)
# Inside _smooth_mask:
k = int(round(blob_diag * 0.015)) # ← MAIN DIAL
# smaller = preserve detail
# bigger = more rounded
```
If you change defaults, restart the backend (`./run.sh`). No frontend
restart needed — the polygon arrives over the existing `/predict` call.