Tuning polygon output (point count + edge smoothness)

When SAM returns a mask, the backend converts it to a polygon before sending it back to the Tauri annotation tool. Two things you can tune:

Point count — how many vertices the polygon has. Each vertex is draggable in the UI, so too many makes editing painful.
Edge smoothness — how the polygon traces curved edges (smooth curves vs. pixel staircases).

Both are controlled in one file: sam2-backend/adapters/utils.py.

Where it's called

Both adapters do the same thing in their predict path:

# sam2-backend/adapters/sam2_adapter.py
return PredictResult(
    polygon=_mask_to_polygon(masks_np[best]),
    score=best_score,
)

# sam2-backend/adapters/sam3_adapter.py — same call site

The defaults are min_points=4, max_points=10, smooth=True. You can override per-call:

polygon=_mask_to_polygon(masks_np[best], min_points=3, max_points=15)

…or change the defaults in utils.py::mask_to_polygon and they'll apply to every model.

Knobs

1. Point count band — `min_points` / `max_points`

The simplifier uses Douglas–Peucker (cv2.approxPolyDP) with a binary search over epsilon to land in [min_points, max_points]. Within that band it prefers more detail (i.e. closer to max_points).

Band	Use it when…
`4..10`	Default. Comfortable to drag, captures common shapes.
`4..15`	Curvy / organic objects (animals, foliage). 15 vertices follow a curve well.
`3..6`	Very simple shapes (rectangles, traffic signs). Fewer drag handles.
`6..20`	High-fidelity work, you don't mind dragging more.

Avoid max_points > 25 — past that, the binary search needs many iterations and the marginal benefit per vertex drops sharply. If you need that many points, the bottleneck is mask quality, not simplification.

2. Mask smoothing — `smooth=True | False`

Before extracting the contour, the helper optionally runs:

Morphological close + open (ellipse kernel) — fills single-pixel notches and removes pepper noise.
Light Gaussian blur + re-threshold — sub-pixel edge smoothing so curves don't trace as staircases.

Pass smooth=False to skip both steps and trace the raw mask exactly. Use this when:

The mask has fine features that smoothing erases (thin antennae, sharp 90° corners that get rounded).
You're A/B testing whether smoothing is helping or hurting.

polygon=_mask_to_polygon(masks_np[best], smooth=False)

3. Smoothing kernel size — `_smooth_mask`

Inside _smooth_mask (called by mask_to_polygon when smooth=True), kernel size scales with the blob's bounding box diagonal:

k = int(round(blob_diag * 0.015))   # ← morphology kernel multiplier
k = max(3, min(k | 1, 11))          # odd, clamped 3..11

blur_k = max(3, min(k | 1, 9))      # ← Gaussian blur kernel

The 0.015 multiplier is the main dial.

Multiplier	Effect
`0.008`	Very gentle. Preserves thin features. Use if smoothing dilates the polygon.
`0.015`	Default. Good balance for typical objects.
`0.025`	Aggressive. Use only on visibly noisy masks; will round corners.

The 11-pixel hard cap means giant masks don't get insanely smoothed.

4. Drop the Gaussian, keep morphology

If hard 90° corners get rounded but you still want morphology to clean up notches, delete just the Gaussian step in _smooth_mask:

# remove these two lines
m = cv2.GaussianBlur(m, (blur_k, blur_k), 0)
_, m = cv2.threshold(m, 127, 255, cv2.THRESH_BINARY)

This keeps close+open (which is mostly noise removal) and skips the sub-pixel curve smoothing.

Diagnostic recipe — find the right knob in 3 minutes

Draw a representative bbox in the Tauri app. Screenshot the result.
Set smooth=False in the adapter, restart, draw the same bbox.
- If polygon is better (less bloat, hugs edges) → smoothing is over-aggressive. Lower the kernel multiplier or delete the Gaussian step.
- If polygon is worse (jagged stairs) → smoothing was helping. Move on to step 3.
Bump max_points to 15 or 20, restart.
- If polygon is better → simplification was too aggressive. Land on max_points=12..15 as your new default.
- If polygon is the same → the mask itself is the bottleneck (see "If the raw mask is bad" below).
Restore smooth=True and try a model swap (SAM 2.1 large ↔ SAM 3 tracker) — the underlying mask differs noticeably between models.

If the raw mask is bad

If after all the above the polygon still looks wrong, the input to the simplifier is bad — fix that, not the polygon code. Common moves:

Pad the bbox before sending to SAM (Meta's reference scripts use pad=10 pixels). SAM gives crisper masks with a small box margin.
Try a larger SAM model (e.g. switch the dropdown from SAM 2.1 Hiera Small to SAM 2.1 Hiera Large).
Tighten the bbox around the actual object — SAM tends to fill the box, so a generous box gets a bloated mask.

Quick reference — every dial in one block

# sam2-backend/adapters/utils.py

mask_to_polygon(
    mask,
    min_points=4,        # never below this
    max_points=10,       # never above this; fewer = easier to drag
    smooth=True,         # turn off to trace raw mask
)

# Inside _smooth_mask:
k = int(round(blob_diag * 0.015))    # ← MAIN DIAL
                                      #   smaller = preserve detail
                                      #   bigger  = more rounded

If you change defaults, restart the backend (./run.sh). No frontend restart needed — the polygon arrives over the existing /predict call.

6.1 KiB Raw Permalink Blame History Unescape Escape