# Central Verification Dashboard (Phase 1)

Dockerized pipeline that ingests a folder of videos + their sibling export-JSONs into Postgres and
serves a manager dashboard: completion overview, per-user leaderboard, throughput + ETA, and a
per-video table. Standalone — it does **not** touch the desktop video-annotator app.

> Phase 1 = the dashboard pipeline. Worker pull/push (claim a video, verify, push) + token auth are
> **Phase 3** and not built yet. So in this phase the dashboard shows the **annotation snapshot**
> ingested from the NAS JSONs: `pending` (no JSON) vs `annotated` (has a JSON) vs `verified`
> (a pushed pass — 0 until Phase 3).

## Run it

```bash
cd central
docker compose up --build
```

- Dashboard → http://localhost:8081
- API       → http://localhost:8080  (e.g. `curl localhost:8080/api/projects`)
- Postgres  → localhost:5433  (user/pass/db = `central`)

On startup the server creates a `default` project pointing at `INGEST_DIR` (`/data`, mounted from
`./sample-data`) and ingests it once. Click **Re-ingest** in the dashboard to re-scan after files
change. Re-ingest is idempotent and never downgrades a `verified` video.

## Point it at the NAS (on the VM)

The ingest folder is just a path/volume. On the VM, mount the Synology share and map it to `/data`:

```yaml
# docker-compose.override.yml on the VM
services:
  server:
    volumes:
      - /mnt/synology/<client-folder>:/data:ro
```

Then `POST /api/projects/<id>/ingest` (or restart the server). No code changes.

## Database backups (→ NAS)

A `db-backup` sidecar (`prodrigestivill/postgres-backup-local`) runs `pg_dump` on a schedule
and writes gzipped dumps straight to the NAS, pruning old ones automatically.

- **Schedule:** daily at 02:00 IST (`SCHEDULE` + `TZ: Asia/Kolkata` in `docker-compose.yml`).
- **Destination:** the `nasbackups` NFS volume → `:/volume4/Saudi_Video_Sync/Verification_Dashboard_Backup`
  on the NAS (`NFS_HOST`, default `192.168.1.199`). **Create that folder on the NAS** and allow
  rw NFS from the Docker host; the host needs an NFS client (`nfs-common`).
- **Retention:** `BACKUP_KEEP_DAYS=30`, `BACKUP_KEEP_WEEKS=8`, `BACKUP_KEEP_MONTHS=6` (tune to taste).
  Dumps land under `daily/`, `weekly/`, `monthly/` as `central-YYYYMMDD-HHMMSS.sql.gz`.

```bash
docker compose up -d db-backup          # start it
docker compose exec db-backup /backup.sh # run an immediate backup (verify it writes to the NAS)
docker compose logs db-backup            # check the schedule/last run
```

**Restore** a dump into the running DB:

```bash
gunzip -c /path/on/nas/daily/central-YYYYMMDD-HHMMSS.sql.gz \
  | docker compose exec -T db psql -U central -d central
```

(For a clean restore, drop+recreate the DB first, or restore into a scratch DB and compare.)
Test a restore periodically — an untested backup isn't a backup.

## Layout

```
central/
  db/init.sql            # Postgres schema (projects, videos, annotations)
  server/                # Rust/Axum API + folder ingest  (embeds db/init.sql)
  dashboard/             # React + Vite, served by nginx (proxies /api → server)
  sample-data/           # demo: one annotated video JSON + two empty .mp4 placeholders
  docker-compose.yml
```

## Ingest details

For each video file (`.mp4/.mkv/.avi/.mov/.m4v/.webm`) it looks for a sibling JSON named
`<file>_annotations.json`, `<file>.json`, or `<stem>.json` (video-annotator's export format —
`{ video, fixed_annotations[], range_annotations[] }`). From it the server derives per-video
annotation count, `annotated_by` (primary), `annotation_time_ms`, and `annotated_at` (latest
annotation timestamp, used for the throughput chart), and stores the full doc in `videos.raw_json`
for Phase 3 pull. Videos without a JSON are recorded as `pending`.

## API

### Clients → projects

A **client** (e.g. *IRB*) owns one or more **projects**; each project points at a folder via
`source_path` + `source_kind`. Project names are unique **per client**. Use the dashboard's
**Admin** tab: sign in (token) → create client (admin) → create project → **Sync**. A `default`
client/project is seeded **only on a fresh DB** (local dev); once real clients exist it never reappears.

**Sync from NFS (`source_kind='nfs'`).** A project's `source_path` may be a **bare export path**
(`/volume4/Saudi_Video_Sync` — the host comes from the `NFS_HOST` env, so the **IP stays server-side and
is never shown/returned**), or include the host explicitly as `host:/volume4/Share` or
`nfs://host/volume4/Share`. On sync the server mounts the share on demand (a fast TCP probe to port 2049
fails an unreachable NAS in ~3 s; the mount is bounded so a stuck NAS can't hang the request).
`source_kind='local'` keeps the old behaviour (a path inside the container, incl. a compose-managed NFS
volume). Point a project at a **subfolder** (e.g. `/volume4/Saudi_Video_Sync/IRB_Master_VIdeos`), not a
huge share root.

**Subfolders.** Ingest walks the folder **recursively**, so videos nested in subfolders are included;
each video's `rel_path` carries its subfolder, which the dashboard shows in a **Folder** column.

> In-app NFS mounting needs `nfs-common` (in the image) and the server container's `cap_add: SYS_ADMIN`
> + `security_opt: apparmor:unconfined` (in `docker-compose.yml`; use `privileged: true` if your host's
> `mount.nfs` still refuses). `local`-only deployments need none of this.

### Roles

- **admin** (role `admin`, or the `ADMIN_TOKEN`): everything — create clients, manage collaborators, etc.
- **collaborator** (role `worker`): can **add a project** and **change a project's sync directory**, but
  **not** create clients or manage users.

### Dashboard read APIs (open — the dashboard is manager-only on the VM)

- `GET  /api/clients`                   — clients + project counts
- `GET  /api/projects`                  — projects + counts (each carries `client_id`, `client_name`, `source_kind`)
- `GET  /api/projects/:id/videos`       — per-video rows (incl. claim/verify columns)
- `GET  /api/projects/:id/stats`        — overview, annotator leaderboard, `verifiers`, throughput, eta_days
- `GET  /api/projects/:id/activity`     — live claims + recent `video_events`
- `GET  /api/audit?limit=N`             — org audit log (client/project created, sync dir changed, synced, user created)
- `POST /api/projects/:id/ingest`       — re-scan the project's folder ("Sync"); attributes the actor if a token is sent

## Phase 3 — worker pull/push + token auth

Workers claim an annotated video, verify it in the desktop app, and push the verified
pass back. All Phase 3 routes require `Authorization: Bearer <token>`.

- **Auth model:** tokens are random 256-bit hex strings; only their SHA-256 hash is
  stored (`users.token_hash`). An admin provisions users and hands out the token (shown
  **once**). A master `ADMIN_TOKEN` env var grants admin (bootstrap / break-glass).
- **Status vs. claim:** the annotation `status` (`pending`/`annotated`/`verified`) is
  unchanged; claiming is a separate dimension (`claimed_by` + `lease_expires_at`). A video
  is claimable when `annotated` and unclaimed-or-lease-expired. A push flips it to
  `verified`, records `completed_by`/`verify_time_ms`, and clears the claim. Re-ingest then
  **skips** verified videos (the pushed pass is authoritative).
- **Leases auto-release:** a 60 s background task returns claims whose lease lapsed
  (`LEASE_SECS`, default 900) to the pool and logs an `auto_release` event.

### Admin (require admin / `ADMIN_TOKEN`)

- `POST /api/users`    `{username, display_name?, role?}` → `{username, role, token}` (token shown once)
- `GET  /api/users`    — list users (no tokens)
- `POST /api/clients`  `{name}` → the new client (unique name; 409 on dup)

### Create / manage projects (require any valid token — workers included)

- `POST /api/projects`          `{client_id, name, source_path, source_kind}` — create/repoint a project
                                under a client (idempotent on `(client_id, name)`); `source_kind` = `local`|`nfs`
- `POST /api/projects/:id/source` `{source_path, source_kind}` — change a project's sync directory

### Worker (require any valid token)

- `GET  /api/auth/whoami`            — validate token → `{username, role}`
- `POST /api/videos/:id/claim`       — atomic claim (annotated + unclaimed/expired); returns
                                       metadata + `download_url` + preloaded `annotations`
- `GET  /api/videos/:id/download`    — stream the source bytes (claimant/admin only)
- `POST /api/videos/:id/heartbeat`   — extend the lease while verifying
- `POST /api/videos/:id/release`     — abandon the claim, back to the pool
- `POST /api/videos/:id/push`        — body = export doc + `verify_time_ms`; replace annotations,
                                       mark `verified`, record verifier + time, clear claim

### Env

| var           | default                                  | meaning                                            |
|---------------|------------------------------------------|----------------------------------------------------|
| `ADMIN_TOKEN` | `dev-admin-token`                        | master admin token — **change in production**      |
| `LEASE_SECS`  | `900`                                     | claim lease length; auto-released after this        |
| `NFS_HOST`    | _(empty)_                                 | NAS host/IP for `nfs`-kind projects (server-side only, never returned) |
| `NFS_OPTS`    | `nfsvers=4,ro,soft,timeo=30,retrans=2,retry=0` | mount options for `nfs`-kind projects        |