domverse/photoanalyzer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects 1 Releases Wiki Activity

Bootstrap project and safe work-item workflow

Browse Source
This commit is contained in:
domverse
2026-07-13 13:51:41 +02:00
commit 80447092e9
90 changed files with 10562 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

282
nsfwtag/README.md Normal file
View File

@@ -0,0 +1,282 @@
# nsfwtag
Local, on-device NSFW image review and tagging for large photo libraries — built to feed [Immich](https://immich.app) search and to complement the `photo_analyzer.py` captioning pipeline.
It scores a photo library with a **local** vision model (Apple GPU via MPS), opens a fast **browser review app**, and writes an `nsfw` (or `sfw`) keyword into each photo's EXIF for the ones you confirm — **preserving every other tag**. Nothing ever leaves the machine: no API calls, no uploads, no telemetry.
```
python -m nsfwtag "pictures/" -r
```
---
## Table of contents
- [The idea](#the-idea)
- [Features](#features)
- [How it works](#how-it-works)
- [Model choice](#model-choice)
- [Requirements & install](#requirements--install)
- [CLI usage](#cli-usage)
- [The review app](#the-review-app)
- [EXIF / review-state model](#exif--review-state-model)
- [HTTP API](#http-api)
- [Data & caching](#data--caching)
- [Project layout](#project-layout)
- [Design decisions](#design-decisions)
- [Immich integration](#immich-integration)
---
## The idea
A large personal photo library has a handful of sensitive images scattered through
thousands of holiday snaps. To filter them in [Immich](https://immich.app) you need a
searchable marker on each one — but going through every photo by hand is infeasible, and
sending a private library to a cloud moderation API is a non-starter.
**The model does recall; you do precision.** A local classifier is good at narrowing
thousands of photos down to a few dozen candidates, but it is *not* trustworthy as the
final word — it over-flags swimwear, art, medical shots. So this is a **human-in-the-loop**
tool: the model proposes, you confirm in a fast visual review app, and only your confirmed
decisions are written to disk. Nothing ever leaves the machine — no API, no upload, no
telemetry.
The decision lives **in the file's own EXIF** as an `nsfw`/`sfw` keyword, not in a
side database, so it survives across tools, backups and restarts, and Immich indexes it
for free at ingest.
---
## Features
- **On-device scoring** with `AdamCodd/vit-base-nsfw-detector` (ViT, ~350 MB, downloaded once then fully offline). Chosen over Falconsai after benchmarking against the library — better recall, fewer false positives (see `bench.py`).
- **Resume-safe**: scores are cached to a CSV; a rerun reuses them and only scores newly-added files — skipping the model load entirely when nothing is new.
- **Browser review app** (dark OLED UI, zero web dependencies — served by a stdlib HTTP server):
- Per-image **checkmark** to mark NSFW / SFW; frame colors show state at a glance.
- **Folder browser** sidebar, live search, a single **Show** filter (All / To-handle / NSFW / SFW), sort, NSFW-score threshold, and a contextual action group that appears only when images are selected.
- **Lightbox** with prev/next, keyboard control, and an **EXIF panel** (incl. a clickable **Google Maps** link for geotagged photos).
- **Activity log** tab with a per-file audit trail of every EXIF write.
- Lazy, per-folder tag scan with a **progress ring**.
- **Reversible & non-destructive**: tag, untag, or toggle at any time; only `Keywords`/`Subject` are modified, so Caption, dates, GPS, camera info, etc. are always preserved.
- **Persistent decisions**: marking an image safe writes an `sfw` keyword, so your review survives restarts.
---
## How it works
```
discover images score (cached) review in browser write EXIF
folder ─────────────────▶ [*.jpg …] ───────────────▶ nsfw_scores.csv ─────────────────▶ exiftool: Keywords/Subject += nsfw|sfw
(recursive, de-duped) supported local ViT on MPS (path,score) local HTTP server + review.html (all other tags preserved)
```
1. **Discover** — walk the folder (optionally recursive), keep supported image types, de-duplicate by resolved path.
2. **Score** — load cached scores; run the model only on files not in the cache; each image is resized to 224×224 and scored 0..1 for the `nsfw` class. Merge and persist.
3. **Review** — candidates (score ≥ `--review-min`) are served as a single-page app. A lazy per-folder scan reads existing `nsfw`/`sfw` keywords and colors each card.
4. **Tag** — your actions POST to the local server, which calls `exiftool` to add/remove the keyword. EXIF is written **before** the file is later uploaded to Immich.
---
## Model choice
The tool ships with **`AdamCodd/vit-base-nsfw-detector`** — a ~350 MB ViT binary
classifier that runs offline on Apple MPS. It was picked by benchmarking candidates
against **your own library**, using the photos you'd already hand-tagged `nsfw`/`sfw`
as ground truth (`bench.py` reads those EXIF marks). A generic public benchmark wouldn't
predict how a model behaves on *these* photos, so the comparison is run on the real data.
Candidates compared (`nsfwtag/bench.py`):
| Model | Notes |
|-------|-------|
| **AdamCodd/vit-base-nsfw-detector** ✅ | Chosen — best recall with the fewest false positives on this library |
| Falconsai/nsfw_image_detection | Previous default; missed more true positives and needed a high `0.85` cutoff |
| Marqo/nsfw-image-detection-384 | Tiny/fast, but noisier on borderline shots |
| OwenElliott/image-safety-classifier-xs | 3-class (SFW/NSFW/NSFL); useful signal, weaker overall here |
**Selection criterion — recall first.** Because a human confirms every candidate, a false
*positive* costs one extra glance, but a false *negative* means a sensitive photo silently
slips through untagged. So the model is judged on catching everything at a threshold low
enough to be safe without burying you in noise. AdamCodd's sweet spot is **`0.6`** (vs
Falconsai's `0.85`) — that is the shipped default.
**Reproduce it** on any sample folder:
```bash
python -m nsfwtag.bench "pictures/SomeFolder" -r
# writes bench_scores.csv + bench_report.html (thumbnails, per-model scores,
# disagreements highlighted) and prints precision/recall vs your manual tags.
```
To switch models, change `MODEL_ID` in `nsfwtag/__init__.py` (any HF image-classification
id) and re-tune `DEFAULT_THRESHOLD`.
---
## Requirements & install
- **Python 3.10+**
- **exiftool** on `PATH` — `brew install exiftool` (macOS) / `apt install libimage-exiftool-perl` (Debian/Ubuntu)
- Python packages: `pip install torch transformers pillow numpy`
- A GPU is optional — uses Apple **MPS** if available, else CPU.
The model downloads automatically on first run (~350 MB) and is cached in `~/.cache/huggingface`; every run after that is offline (`local_files_only=True`, which also skips the slow HF Hub update check).
---
## CLI usage
Run from the directory that contains the `nsfwtag/` package:
```bash
# Score + open the review app
python -m nsfwtag "pictures/" # this folder only
python -m nsfwtag "pictures/" -r # recurse into subfolders
python nsfw_tag.py "pictures/" -r # identical (backwards-compat shim)
# Tag directly by score, no review UI
python -m nsfwtag "pictures/" -r --apply
# Tag every path listed in a text file (one path per line)
python -m nsfwtag confirmed.txt [more.txt ...]
```
| Flag | Default | Description |
|------|---------|-------------|
| `-r, --recursive` | off | Descend into every subfolder |
| `--threshold` | `0.6` | Score ≥ this is treated as "NSFW to review" (and tagged directly in `--apply`) |
| `--review-min` | `0.20` | Lowest score shown in the review app |
| `--limit N` | `0` | Only the first N images (calibration) |
| `--apply` | off | Tag score ≥ threshold directly, skip the review UI |
| `--rescore` | off | Ignore the score cache and re-run the model on everything |
The argument is auto-detected: a **folder** opens the review app; one or more **`.txt`** files tag the listed paths directly.
---
## The review app
Served on `http://127.0.0.1:<random-free-port>/` and opened in your browser automatically. Press `Ctrl-C` in the terminal when done.
There are **two independent channels** per card: the **decision** (the EXIF tag,
shown as the frame + tick color) and the **selection** (a blue ring, for bulk
actions). A card can be both selected *and* decided.
### Frame colors (the decision)
| Color | Meaning | EXIF |
|-------|---------|------|
| 🩷 **pink** | tagged NSFW | `nsfw` keyword |
| 🟢 **green** | tagged safe | `sfw` keyword |
| 🔴 **red** | undecided **and** selected — a bulk candidate, nothing written yet | none |
| ⚪ neutral | undecided, not selected | none |
| 🔵 blue ring | selected for a bulk action (independent of the frame color) | — |
Each card has a **tick** (bottom-right circle) that makes a per-image decision and
**writes EXIF on every click**, cycling: undecided → **NSFW** (pink) → **SFW**
(green) → undecided. One click is enough — it no longer depends on the selection
state. Clicking elsewhere on a card does nothing (no accidental toggles).
### Controls
The toolbar separates **viewing** (always visible) from **acting** (contextual). It reads left-to-right:
```
🔍 Search… Show ▾ Sort ▾ NSFW ≥ 0.6 ●───── Select flagged Select all ⟨ N selected · Deselect · Remove · Mark safe · Mark NSFW ⟩
```
- **Folder browser** (left sidebar, toggleable): filter to a folder + its subfolders; counts per folder.
- **Search**: filter by filename or folder.
- **Show** (single filter): **All** / **To-handle** (no keyword yet — what's left to decide) / **NSFW** (tagged nsfw) / **SFW** (tagged sfw).
- **Sort**: score / name / folder.
- **NSFW ≥ slider**: score threshold used by *Select flagged* and the red "identified" coloring.
- **Select flagged**: selects visible, untagged images at/above the threshold.
- **Select all**: selects every currently-visible image (respects the folder/filter). Pair with **Deselect** to clear. Use this to build a selection after a filter where *Select flagged* matches nothing.
- **Per-image tick**: one click writes EXIF and cycles undecided → NSFW → SFW → undecided; the tick and frame colors show the current decision.
- **Action group** (appears only when ≥1 image is selected): **Mark NSFW** / **Mark safe** / **Remove** (clears either keyword) / **Deselect**. Counts and actions are scoped to the current view (folder/filter), so the count matches what you see.
- **Lightbox** (⤢ on a thumbnail): full-res image, EXIF side panel (incl. Google Maps link), `←`/`→` navigate, `Space` toggle selection, `Esc`/backdrop close.
- **Activity log** (header button): timestamped, per-file record of every EXIF write.
---
## EXIF / review-state model
The review decision is stored **in the file's EXIF** as one of two mutually-exclusive keywords, written to both `Keywords` (IPTC) and `Subject` (XMP):
| Keyword | Meaning |
|---------|---------|
| `nsfw` | confirmed NSFW |
| `sfw` | confirmed safe |
| *(neither)* | not yet reviewed |
Rules:
- Writing `nsfw` removes any `sfw`, and vice-versa — a file can hold **at most one**.
- On read, **`nsfw` takes precedence**: a stray file with both is reported as NSFW, never as safe.
- **Only `Keywords` and `Subject` are ever modified.** All other metadata — `ImageDescription`/Caption, GPS, capture dates, camera make/model, ratings — is left byte-for-byte intact in both directions (add and remove).
- `exiftool` runs with `-overwrite_original` (no sidecar `_original` files) and `-m` (ignore minor warnings so writes still apply to slightly-nonconforming files).
Immich indexes `Keywords`, so tagged photos become searchable there (search `nsfw`).
---
## HTTP API
Stdlib `ThreadingHTTPServer` bound to `127.0.0.1` on an OS-assigned free port. **Every path argument is validated against the scanned candidate set** — the server can't be pointed at arbitrary files.
| Method | Route | Purpose |
|--------|-------|---------|
| GET | `/` | The review page |
| GET | `/img?path=` | Thumbnail bytes for a candidate image |
| GET | `/exif?path=` | Filtered EXIF for the lightbox panel (decimal GPS + Google Maps URL) |
| GET | `/log` | Activity-log entries `[{t, m}, …]` |
| POST | `/tagged` | Body: `[path,…]` → `{nsfw:[…], sfw:[…]}` (batched keyword read) |
| POST | `/apply` | Body: `[path,…]` → add `nsfw` (strips `sfw`) → `{tagged:n}` — "Mark NSFW" |
| POST | `/mark?state=sfw` | Body: `[path,…]` → add `sfw` (strips `nsfw`) — "Mark safe" / checkmark→green |
| POST | `/untag` | Body: `[path,…]` → remove both keywords (un-review) → `{untagged:n}` — "Remove" |
| POST | `/mark?state=clear` | Body: `[path,…]` → remove `sfw` only (checkmark→red) |
| POST | `/toggle` | Body: `[path,…]` → flip `nsfw` per file → `{tagged:[…]}` (endpoint retained; not used by the current UI) |
---
## Data & caching
- **`nsfw_scores.csv`** (written in the scanned folder): `path,nsfw_score`. This is the resume cache — reruns reuse it and only score new files. Delete it or pass `--rescore` to force a full re-score.
- **Review state** is **not** in the CSV; it lives in each file's EXIF (see above), so it persists independently of the score cache and survives restarts.
- The activity log is in-memory (last 2000 events) and resets when the server stops; messages are also echoed to stderr.
---
## Project layout
```
nsfwtag/
├── __init__.py # constants (model id, extensions, defaults), warning suppression
├── exif.py # read/write/remove nsfw|sfw via exiftool; read_marks, read_exif (+GPS)
├── scoring.py # discover images, cached model scoring
├── webapp.py # render the page: fill review.html tokens (cards, folder tree, totals)
├── review.html # the frontend — HTML/CSS/JS (no build step, no dependencies)
├── server.py # local HTTP server + routes + activity log
├── __main__.py # CLI entry point (python -m nsfwtag)
└── README.md
nsfw_tag.py # thin shim: `python nsfw_tag.py` == `python -m nsfwtag`
```
The frontend lives in a real `review.html` (dynamic parts injected via `%%TOKEN%%` replacement) rather than a Python string, so it edits with normal HTML/CSS/JS tooling.
---
## Design decisions
- **No pre-selection at render.** Cards render unselected; the **lazy per-folder scan** decides each card's state from its EXIF keywords (pink/green) or score (red). This is deliberate: it guarantees a photo you marked safe never reappears pre-selected-red after a restart, and keeps startup instant (the browser opens before any `exiftool` scan runs).
- **Lazy, chunked, per-folder scanning** with a progress ring, instead of one blocking scan at startup — so switching into a folder shows real progress and huge libraries stay responsive.
- **Score cache first, model second** — the model (and its load) is skipped entirely on reruns with no new files.
- **View-scoped counters/actions** — Apply/Toggle/Remove and their counts act on the folder/filter you're viewing, so the number always matches what's on screen.
- **Stdlib-only server + dependency-free frontend** — nothing to build or install for the UI; the whole app is one HTML file plus a ~120-line Python server.
---
## Immich integration
Order matters: **write EXIF first, then upload.** immich-go deduplicates on the file's SHA-1 content hash, so writing EXIF after upload would create duplicates. This tool only writes metadata in place; ingest into Immich is a separate, deliberate `immich-go upload` step. Once uploaded, `Keywords` (including `nsfw`) are searchable in Immich alongside its own ML pipeline. **Do not** also point Immich at the folder as an External Library.