first version
This commit is contained in:
226
docs/HF_DEPLOY_OFFLINE.md
Normal file
226
docs/HF_DEPLOY_OFFLINE.md
Normal file
@@ -0,0 +1,226 @@
|
||||
# Deploying SAM3 via HuggingFace, then going fully offline
|
||||
|
||||
The SAM3 backend uses Meta's `facebook/sam3` weights via HuggingFace. The
|
||||
flow is **online once** (to download + cache the weights) → **then
|
||||
air-gapped** (server runs without ever talking to HuggingFace again).
|
||||
This doc walks through both phases and how to verify your box is actually
|
||||
offline at the end.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — prerequisites
|
||||
|
||||
On the target PC:
|
||||
|
||||
- The `sam2-backend/` folder, copied or cloned, with its `.venv`
|
||||
containing `torch`, `transformers>=4.57`, `huggingface_hub`, and the
|
||||
rest of `requirements.txt`. See [`sam2-backend/README.md`](../sam2-backend/README.md)
|
||||
for the full install.
|
||||
- A HuggingFace account that has been **granted access to
|
||||
`facebook/sam3`** — gated. Request access at
|
||||
https://huggingface.co/facebook/sam3 (usually approved within minutes).
|
||||
- A read token from https://huggingface.co/settings/tokens.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — first-time online download
|
||||
|
||||
The first call to `Sam3TrackerModel.from_pretrained("facebook/sam3")`
|
||||
streams the weights from HF and writes them to
|
||||
`~/.cache/huggingface/hub/`. Everything afterward can be served from that
|
||||
cache with no network.
|
||||
|
||||
### 1.1 Authenticate
|
||||
|
||||
```bash
|
||||
cd /home/<you>/sam2-backend
|
||||
source .venv/bin/activate
|
||||
hf auth login # paste your read token
|
||||
hf auth whoami # confirm the right account
|
||||
```
|
||||
|
||||
Token is stored at `~/.cache/huggingface/token`. It's a plain file with
|
||||
`chmod 600`.
|
||||
|
||||
### 1.2 Set offline flag to OFF for the first start
|
||||
|
||||
In `run.sh`, the relevant lines are:
|
||||
```bash
|
||||
export HF_HUB_DISABLE_TELEMETRY="${HF_HUB_DISABLE_TELEMETRY:-1}"
|
||||
export DO_NOT_TRACK="${DO_NOT_TRACK:-1}"
|
||||
export HF_HUB_OFFLINE="${HF_HUB_OFFLINE:-0}" # ← MUST be 0 first time
|
||||
export TRANSFORMERS_OFFLINE="${TRANSFORMERS_OFFLINE:-0}" # ← MUST be 0 first time
|
||||
```
|
||||
|
||||
For a one-off override, just don't pass the env vars:
|
||||
```bash
|
||||
./run.sh
|
||||
```
|
||||
|
||||
### 1.3 Make ONE successful predict call
|
||||
|
||||
Either from the Tauri app (Settings → pick `SAM 3 · tracker (HuggingFace)` →
|
||||
draw a bbox), or directly:
|
||||
|
||||
```bash
|
||||
curl -s -X POST "http://localhost:9090/predict?model=sam3_tracker" \
|
||||
-H "Content-Type: application/json" \
|
||||
--data '@some_test_payload.json'
|
||||
```
|
||||
|
||||
A successful response means the weights are now fully cached. Confirm:
|
||||
|
||||
```bash
|
||||
ls -la ~/.cache/huggingface/hub/models--facebook--sam3/
|
||||
# expect a snapshots/<hash>/ subdir with model.safetensors and configs
|
||||
du -sh ~/.cache/huggingface/hub/models--facebook--sam3/
|
||||
# ~3-4 GB for SAM3
|
||||
```
|
||||
|
||||
### 1.4 Optional — pre-warm without a real predict
|
||||
|
||||
If you want to download but not yet wire up Tauri, do it from a Python
|
||||
shell:
|
||||
|
||||
```bash
|
||||
python3 - <<'EOF'
|
||||
from transformers import Sam3TrackerModel, Sam3TrackerProcessor
|
||||
Sam3TrackerProcessor.from_pretrained("facebook/sam3")
|
||||
Sam3TrackerModel.from_pretrained("facebook/sam3")
|
||||
print("cached.")
|
||||
EOF
|
||||
```
|
||||
|
||||
Same effect — fills `~/.cache/huggingface/hub/`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — flip to fully offline
|
||||
|
||||
Once the cache is populated, set the offline flags so the server never
|
||||
tries to revalidate metadata or check for updates.
|
||||
|
||||
### 2.1 Edit `run.sh`
|
||||
|
||||
Change the defaults to `1`:
|
||||
|
||||
```bash
|
||||
export HF_HUB_DISABLE_TELEMETRY="${HF_HUB_DISABLE_TELEMETRY:-1}"
|
||||
export DO_NOT_TRACK="${DO_NOT_TRACK:-1}"
|
||||
export HF_HUB_OFFLINE="${HF_HUB_OFFLINE:-1}" # ← 1
|
||||
export TRANSFORMERS_OFFLINE="${TRANSFORMERS_OFFLINE:-1}" # ← 1
|
||||
```
|
||||
|
||||
Or, if you're running via systemd, add to your unit file:
|
||||
|
||||
```ini
|
||||
[Service]
|
||||
Environment=HF_HUB_DISABLE_TELEMETRY=1
|
||||
Environment=DO_NOT_TRACK=1
|
||||
Environment=HF_HUB_OFFLINE=1
|
||||
Environment=TRANSFORMERS_OFFLINE=1
|
||||
```
|
||||
|
||||
Then `sudo systemctl daemon-reload && sudo systemctl restart sam-backend`.
|
||||
|
||||
### 2.2 Restart and verify
|
||||
|
||||
```bash
|
||||
./run.sh
|
||||
```
|
||||
|
||||
You should see clean startup logs **without** any `httpx HTTP Request`
|
||||
lines hitting `huggingface.co`. The `Loading weights: 685/685 [00:00<00:00]`
|
||||
line confirms a cache hit (instant load, no download).
|
||||
|
||||
### 2.3 Pin a custom cache location (optional)
|
||||
|
||||
If you want the cache somewhere predictable (for backup, rsync to other
|
||||
boxes, or to keep `$HOME` clean), set `HF_HOME` in `run.sh`:
|
||||
|
||||
```bash
|
||||
export HF_HOME=/var/cache/sam-backend/huggingface
|
||||
```
|
||||
|
||||
Move the existing cache once:
|
||||
```bash
|
||||
mv ~/.cache/huggingface /var/cache/sam-backend/
|
||||
```
|
||||
|
||||
Subsequent reads come from there.
|
||||
|
||||
---
|
||||
|
||||
## Verifying you're really offline
|
||||
|
||||
Two-stage check:
|
||||
|
||||
### Step 1 — disconnect & restart
|
||||
|
||||
Easiest sanity test: pull the network cable (or `sudo ip link set eth0 down`),
|
||||
restart the server, hit `/predict` from the Tauri app on the same box.
|
||||
If it returns a polygon, you're truly offline.
|
||||
|
||||
### Step 2 — packet-level proof
|
||||
|
||||
If you can't disconnect (e.g., other services need the network), watch
|
||||
for any HF traffic during normal use:
|
||||
|
||||
```bash
|
||||
sudo tcpdump -nn -i any \
|
||||
'host huggingface.co or host cdn-lfs.huggingface.co' \
|
||||
2>/dev/null
|
||||
```
|
||||
|
||||
Restart the backend, hit `/predict` ten times from the Tauri app. With
|
||||
the OFFLINE flags set you should see **zero packets**.
|
||||
|
||||
---
|
||||
|
||||
## Moving the cache to a different machine
|
||||
|
||||
Useful when you want to deploy to a new box without the first-run
|
||||
download (e.g., setting up a second annotation station behind the same
|
||||
firewall).
|
||||
|
||||
```bash
|
||||
# On the source machine:
|
||||
tar czf hf-sam3-cache.tar.gz -C ~/.cache huggingface
|
||||
scp hf-sam3-cache.tar.gz target-box:/tmp/
|
||||
|
||||
# On the target machine:
|
||||
mkdir -p ~/.cache
|
||||
tar xzf /tmp/hf-sam3-cache.tar.gz -C ~/.cache
|
||||
```
|
||||
|
||||
Then start the new backend with `HF_HUB_OFFLINE=1` from day one — it'll
|
||||
load from the imported cache without ever calling out.
|
||||
|
||||
---
|
||||
|
||||
## Common gotchas
|
||||
|
||||
| Symptom | Cause / fix |
|
||||
|--------------------------------------------------------------------------|-----------------------------------------------------------------------------|
|
||||
| `Cannot find the requested files in the cached path` while OFFLINE=1 | Cache is incomplete or in a different `HF_HOME`. Re-run online once. |
|
||||
| `gated repo, access denied` | Token doesn't have `facebook/sam3` access. Accept gate on the model page, then `hf auth login` again. |
|
||||
| Many `HEAD https://huggingface.co/...` lines on startup despite OFFLINE=1 | One of the two env flags is missing. Both `HF_HUB_OFFLINE=1` AND `TRANSFORMERS_OFFLINE=1` are required. |
|
||||
| `Loading weights: ...` takes minutes even with cache | The cache file is on a slow disk (NFS, HDD). Move it to local SSD via `HF_HOME`. |
|
||||
| Token leaked into logs | `hf auth login` only echoes your username, never the token. The token at `~/.cache/huggingface/token` is the only place it's stored. |
|
||||
|
||||
---
|
||||
|
||||
## What HF still sees in offline mode
|
||||
|
||||
**Nothing.** With both `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1`:
|
||||
|
||||
- `from_pretrained` reads only from `~/.cache/huggingface/hub/` — no API
|
||||
calls, no HEAD revalidation pings.
|
||||
- Telemetry is independently off via `HF_HUB_DISABLE_TELEMETRY=1` /
|
||||
`DO_NOT_TRACK=1`.
|
||||
- Inference itself never touches the network — your images, bboxes, and
|
||||
resulting masks stay on the box regardless of online/offline mode.
|
||||
|
||||
The only thing HF "knows" about you, ever, is "this account downloaded
|
||||
`facebook/sam3` once on date X" — recorded server-side at the moment
|
||||
`from_pretrained` first ran in Phase 1.
|
||||
Reference in New Issue
Block a user