US08-01: Make the Trust Boundary Configurable and Authenticated

.gitignore

@@ -17,3 +17,15 @@ _todo/
 pictures/
 photos/
 _IGNORE/
+
+# Test failure evidence (US07-04)
+.artifacts/
+
+# Any dotenv, not only the default name.
+*.env
+
+# Local virtualenv for running the app.
+.venv/
+
+# setuptools editable-install metadata.
+*.egg-info/

AGENTS.md

@@ -80,7 +80,8 @@ are the authoritative backlog.
    uncertain. Work only on the claimed story and its generated feature branch.
 4. Read the entire issue, linked specification, dependencies, and acceptance criteria.
    Reconcile them with the concept before designing or changing code.
-5. Inspect the legacy CLI donors before replacing applicable behavior. Update the donor
+5. Inspect the legacy CLI donors — frozen in `legacy_cli_archive/` since US07-01,
+   with their ledger — before replacing applicable behavior. Update the donor
    ledger and characterization tests required by the story.
 6. Implement every acceptance criterion and its automated tests.
 7. Run story-specific tests and the accumulated regression suite required by the epic.

README.md

@@ -7,16 +7,96 @@ archive workflow. Planning lives in `INTEGRATED_PIPELINE_CONCEPT.md` and
 ## Application (`photo_pipeline`)
 
 The target application lives in `photo_pipeline/` (FastAPI + SQLAlchemy + Alembic).
-Run it with:
+Install it into a virtualenv once:
 
 ```bash
-python -m photo_pipeline migrate   # apply database migrations
-python -m photo_pipeline serve     # start the API + static review UI (127.0.0.1:8000)
+python3.12 -m venv .venv
+.venv/bin/pip install -e ".[vision]"   # drop [vision] for a review-only install
 ```
 
+Then run the two processes:
+
+```bash
+.venv/bin/python -m photo_pipeline migrate   # apply database migrations
+.venv/bin/python -m photo_pipeline serve     # API + review UI at 127.0.0.1:8000/app/
+.venv/bin/python -m photo_pipeline worker    # second terminal: runs the jobs
+```
+
+The server enqueues work and serves the UI; nothing actually scans, scores,
+analyses, uploads, or archives without a worker. `work_item/scripts/python` is the
+*helper's* launcher — it prefers Conda base and falls back to a bare system
+interpreter, so it is not how the application is run.
+
 Configuration comes from `PHOTO_PIPELINE_*` environment variables (see
 `photo_pipeline/config.py`); secrets are referenced, never logged.
 
+### Configuration file
+
+`.env` in the working directory is read at startup, or any path named by
+`PHOTO_PIPELINE_ENV_FILE`. It is parsed, never executed: `KEY=value` lines,
+`#` comments, optional quotes — no interpolation and no `export`. **Anything already
+exported wins**, so the file is the standing configuration and the shell is the
+override for one run.
+
+The archived CLI's variable names still work, so an existing `photo_analyzer.env`
+can be used as-is:
+
+| in the file | applied as |
+|---|---|
+| `LLM_API_KEY` / `GEMINI_API_KEY` | `OPENAI_API_KEY` |
+| `LLM_BASE_URL` | `OPENAI_BASE_URL` |
+| `LIBRARY` | `PHOTO_PIPELINE_LIBRARY_ROOTS` |
+
+`.env` and `*.env` are gitignored and denied by the work-item safety checks: the
+file holds a real key and must never be committed.
+
+### API access (US07-02)
+
+The app listens on loopback, so its attacker is another page in the same browser.
+Every `/api/v1` route except `health/live`, `health/ready`, and `session` requires
+the application session, and every mutation requires its CSRF token as well:
+
+```bash
+BASE=http://127.0.0.1:8000
+TOKEN=$(curl -sc /tmp/pp.jar $BASE/api/v1/session | python -c 'import json,sys; print(json.load(sys.stdin)["csrf_token"])')
+curl -sb /tmp/pp.jar -H "X-CSRF-Token: $TOKEN" -X POST $BASE/api/v1/albums/proposals -d '{}' -H 'Content-Type: application/json'
+```
+
+The session is per server process — restarting `serve` invalidates it, and the
+browser client re-bootstraps by itself. Requests are also refused when the `Host` is
+not a loopback name (DNS rebinding), when `Origin` is any other origin, when
+`Sec-Fetch-Site` says the request came from another site (an `<img>` pointed at a
+thumbnail), or when the body exceeds `PHOTO_PIPELINE_MAX_REQUEST_BYTES`. There is no
+CORS middleware at all, so no other origin can read a response.
+
+### Reaching it through a hostname or proxy (US08-01)
+
+| variable | meaning |
+|---|---|
+| `PHOTO_PIPELINE_ALLOWED_HOSTS` | comma-separated extra names the app answers to; empty means loopback only |
+| `PHOTO_PIPELINE_ACCESS_SECRET` | traded for the session cookie at `GET /api/v1/session` via `X-Access-Secret` |
+| `PHOTO_PIPELINE_TRUSTED_PROXIES` | comma-separated peer addresses whose `X-Forwarded-Proto`/`X-Forwarded-Host` are believed |
+
+Being reachable *was* the authentication: whoever could open `127.0.0.1:8000` owned
+the library. So naming any non-loopback host — or binding to one, `0.0.0.0` included
+— makes the access secret mandatory, and `serve` refuses to start without it rather
+than publishing the library. Loopback-only deployments need no secret and behave
+exactly as before.
+
+```bash
+curl -sc /tmp/pp.jar -H "X-Access-Secret: $PHOTO_PIPELINE_ACCESS_SECRET" \
+  https://photos.example.com/api/v1/session
+```
+
+The browser asks for the secret once per tab and keeps it in `sessionStorage`.
+Wrong secrets are rate-limited (5 per minute) and logged with the caller's address
+only. `Host` and `Origin` are judged against the configured names; the *external*
+scheme and host come from the forwarded headers only when the request arrived from a
+`PHOTO_PIPELINE_TRUSTED_PROXIES` address, so a client cannot declare its own origin,
+and the session cookie is marked `Secure` when that external scheme is HTTPS. Health
+endpoints stay reachable without the secret so an orchestrator can restart the
+container; nothing else does.
+
 ## Testing
 
 One offline command runs the whole suite (unit, integration, and browser
@@ -169,3 +249,306 @@ work_item/scripts/python -m pytest -m phase_e -q
   bytes, and recovery after a restart).
 
 Phases A–D remain green in the full run above.
+
+### Phase F acceptance gate
+
+Phase F (Epic E06: archive lifecycle) is the only stage that *removes* originals
+from the library, and the only one whose storage can walk away in someone's bag.
+One command runs the archive fault-injection suites, the black-box archive and
+restore API journeys, and the browser suite:
+
+```bash
+work_item/scripts/python -m pytest -m phase_f -q
+```
+
+- `tests/integration/test_archive_*.py` and `tests/integration/test_restore.py`
+  drive real files on real filesystems: preflight against a mounted, missing,
+  swapped, read-only, or full medium; copy-verify-remove and the same-filesystem
+  move path; and a crash at **every** persisted journal transition in both transfer
+  modes, asserting that no source is ever removed without a durable, byte-identical
+  archive copy.
+- `tests/e2e/test_phase_f_pipeline.py` drives a real server and a real durable
+  worker over HTTP: preflight blockers (offline medium, wrong volume, insufficient
+  capacity, bytes changed after upload), a verified archive whose manifest, hashes,
+  and path history are checked on the medium itself, a worker killed at each of
+  `transferring`, `verified`, `removing`, `source_removed`, and `complete`, the
+  evidence-based recovery that follows, offline deduplication of an exact and a
+  fuzzy copy while the medium is away, mount return, restore, and a collision that
+  restores beside its occupant.
+- **Archived is not missing.** An unmounted medium leaves its photos
+  `archived_offline` — still hashed, still in the duplicate indexes, still
+  previewable through their protected thumbnails — and a rescan neither prunes nor
+  flags them.
+- **Ambiguity is never guessed.** A journal state the medium contradicts stays
+  `manual`, offers no automatic action, and keeps blocking further archiving until
+  a human decides.
+- `tests/e2e/test_archive_ui.py` covers the browser journeys (preview with
+  destination identity and reclaimable bytes, blockers and mount instructions,
+  progress split into transfer/verification/removal, interruption and recovery,
+  offline browsing, restore, collision, keyboard confirmation, and reload).
+
+Phases A–E remain green in the full run above.
+
+## Media and metadata hardening (US07-03)
+
+Every pixel the application reads goes through `photo_pipeline/imaging.py`: the
+declared dimensions are checked before anything is decoded, Pillow's
+decompression-bomb warning is treated as a refusal, JPEG decodes near the requested
+size, and each decoder failure becomes one of two typed errors. A damaged file is a
+per-item error with a persisted code, never a failed scan or a dead worker.
+
+Every metadata stage ends with an EXIF checkpoint (`services/exif_checkpoint.py`):
+snapshot, write the owned keywords, read back, prove the owned fields landed and that
+nothing else moved, refresh the file hash. A field the stage does not own that
+changed anyway makes the checkpoint `divergent` — recorded in `exif_projections`,
+shown in the review queue, never repaired behind the user's back, and not counted as
+verified, so upload stays blocked.
+
+The golden corpus that proves all of it is generated, not committed:
+`tests/fixtures/media_corpus.py` declares every format, orientation, profile,
+damage, and metadata case with its expected outcome, and the suite regenerates it
+twice to prove it does not drift.
+
+```bash
+work_item/scripts/python -m pytest tests/integration/test_media_hardening.py tests/integration/test_exif_checkpoints.py -q
+```
+
+## Concurrency and crash recovery (US07-04)
+
+Crash safety is proven by crashing. `photo_pipeline/faults.py` defines the control
+points — the persisted transitions of the rename, archive, EXIF, upload, and job
+lanes — and arms one only when `PHOTO_PIPELINE_FAULT_AFTER` names it, at which
+point the process dies the way a `SIGKILL` does. There is no endpoint and no
+configuration field that can reach a barrier; a deployment that never sets the
+variable can never hit one.
+
+The race suite runs each scenario several times with a seed recorded on the test
+result (`race_seed`) and asserts invariants rather than schedules: work is never
+claimed or executed twice, a stale fencing token never commits, no file body is
+lost or overwritten, and the database still passes `PRAGMA integrity_check`.
+
+```bash
+work_item/scripts/python -m pytest tests/integration/test_concurrency_races.py \
+    tests/integration/test_fault_matrix.py tests/e2e/test_crash_recovery.py -q
+
+# replay a failure, or soak for new interleavings
+PHOTO_PIPELINE_RACE_SEED=1234 PHOTO_PIPELINE_RACE_REPEATS=50 \
+    work_item/scripts/python -m pytest tests/integration/test_concurrency_races.py -q
+```
+
+Any failing test keeps its evidence: the temporary database (with its write-ahead
+log), the journals, the logs, the recorded seed, and a SHA-256 manifest of every
+file in the temporary library are copied to `.artifacts/<test id>/` before pytest
+deletes the directory. Point `PHOTO_PIPELINE_TEST_ARTIFACTS` elsewhere to collect
+them from CI.
+
+## Release gate (US07-07)
+
+One command runs every suite in an isolated stack and keeps the evidence:
+
+```bash
+work_item/scripts/python -m photo_pipeline release-gate --output data/release/$(date -u +%Y%m%dT%H%M%SZ)
+```
+
+It fails — and exits non-zero — when any stage fails, when a suite skips a test for
+a reason that is not a documented environment limit (`exiftool not installed`,
+`root ignores directory permissions`), or when the story matrix has a hole. The
+evidence directory holds `release-report.json` (revision, per-stage result, timings,
+summaries), `logs/<stage>.log`, and `CHECKSUMS.sha256` over both.
+
+**The story matrix** lives in `tests/story_traceability.json`: every story under
+`delivery_backlog/stories/` is either mapped to test files that exist, or listed in
+`planned` as an accepted but unimplemented story. A story that is neither, or a
+mapping to a file that has been deleted, fails the gate.
+
+**The journey** (`tests/e2e/test_release_journey.py`) takes one fresh library through
+discovery, duplicate review, safety, analysis, EXIF verification, album proposal,
+guarded rename, rescan, upload with server-side verification, archive, offline
+deduplication, and restore — over HTTP against real server and worker processes,
+with a full restart in the middle and at the end.
+
+### Real-library dry run and approval
+
+Before the application is pointed at photos that cannot be replaced:
+
+```bash
+work_item/scripts/python -m photo_pipeline dry-run --output dry-run.json
+work_item/scripts/python -m photo_pipeline approve-dry-run dry-run.json --approver "$(whoami)"
+```
+
+The dry run is strictly read-only: it opens no file for writing, writes no database
+row, and reports what it found — file counts by extension, folders, bytes, unreadable
+files, excluded directories, and a reconciliation against what the database already
+knows (already registered, new, recorded but absent). Set
+`PHOTO_PIPELINE_REQUIRE_DRY_RUN_APPROVAL=1` and **every mutating API request is
+refused with `403 dry_run_not_approved`** until a report for exactly those library
+roots has been approved. Reading stays open — you have to be able to see what was
+found in order to approve it — and so does taking a backup. Change the library roots
+and the approval no longer applies: it approves that reconciliation, not the idea of
+mutating.
+
+## Performance budgets (US07-06)
+
+Budgets are measured, not asserted in prose. `python -m photo_pipeline benchmark`
+builds a synthetic library of a stated size, runs the same scenarios every time,
+writes a machine-readable report, and **exits non-zero when a budget is breached**.
+
+```bash
+work_item/scripts/python -m photo_pipeline benchmark --profile smoke    # ~2 s, runs in CI
+work_item/scripts/python -m photo_pipeline benchmark --profile short    # 25k assets
+work_item/scripts/python -m photo_pipeline benchmark --profile full \
+    --output data/benchmarks/full.json                                  # 25k + 100k
+work_item/scripts/python -m photo_pipeline benchmark --profile huge \
+    --soak-seconds 3600 --output data/benchmarks/soak.json              # 500k + soak
+```
+
+| Metric | Budget | Why |
+|---|---|---|
+| `latency_p95_ms` | 250 ms | a list or search page must feel immediate |
+| `latency_max_ms` | 2 000 ms | no single page may stall the review flow |
+| `rss_growth_bytes` | 400 MB | a run must not leak the library |
+| `open_files` | 256 | file descriptors are a hard operating-system limit |
+| `wal_bytes` | 200 MB | a growing write-ahead log means checkpoints are starving |
+| `queue_depth` | 1 000 | an unbounded queue is an out-of-memory in waiting |
+| `cache_over_quota_bytes` | 0 | the thumbnail cache has to respect its quota |
+
+Measured on the reference machine (Apple Silicon, SQLite WAL), p95 per scenario:
+
+| Scenario | 25k | 100k |
+|---|---|---|
+| `inventory_page` | 0.5 ms | 0.6 ms |
+| `library_search` | 4.8 ms | 17.1 ms |
+| `library_stats` | 56.8 ms | 197.4 ms |
+| `workflow_readiness` | 52.6 ms | 235.4 ms |
+| `duplicate_cluster_list` | 0.6 ms | 0.5 ms |
+| `duplicate_cluster_page` | 1.7 ms | 1.7 ms |
+
+CI runs the `smoke` profile through `tests/integration/test_performance_budgets.py`;
+the 25k/100k/500k matrix and the multi-hour soak belong to scheduled infrastructure,
+because minutes of build time do not belong in the suite that runs on every change.
+
+**Exceptions.** A budget that cannot be met is not a warning to ignore: it goes into
+`APPROVED_EXCEPTIONS` in `photo_pipeline/services/benchmarks.py` with its raised
+limit, who approved it, why, and a review date. Every report lists the exceptions it
+applied, so a release review sees them.
+
+Approved today, both for the 500k `huge` profile only, review by 2027-02-17:
+
+| Scenario | Measured at 500k | Raised limit |
+|---|---|---|
+| `library_stats` | 1.08 s p95 · 3.2 s max | 1.5 s p95 · 4 s max |
+| `workflow_readiness` | 1.40 s p95 · 3.3 s max | 1.8 s p95 · 4 s max |
+
+Both are library-wide aggregates — the current safety decision of every asset, and
+the album/tag/year breakdown of every analysis row — and both meet the 250 ms budget
+at the 100k rows the concept sets it for. Beyond that they are linear against one
+SQLite writer; the fix is denormalized totals or the planned PostgreSQL transition,
+not a query tweak. Everything else at 500k is inside budget, and a soak at that size
+grows neither resident memory nor the job queue.
+
+## Backup and recovery (US07-05)
+
+Backups go through SQLite's online backup API, never a file copy: with WAL enabled
+the `.db` file alone is missing every committed page still in the write-ahead log.
+Each backup is a directory under `data/backups/` holding the snapshot and a
+`manifest.json` describing it — schema revision, SHA-256, row counts, the archive
+media the library depends on, and which configuration was set. Secrets are recorded
+as `configured`, never as values, so a manifest is safe to attach to a bug report.
+
+```bash
+work_item/scripts/python -m photo_pipeline backup --reason before-upgrade --keep 7
+work_item/scripts/python -m photo_pipeline verify-backup data/backups/<name>
+work_item/scripts/python -m photo_pipeline diagnostics
+```
+
+The same is available at `GET /api/v1/diagnostics`, `GET|POST /api/v1/backups`,
+`GET /api/v1/backups/{name}/verify`, and `POST /api/v1/backups/prune`. **Restore is
+not an endpoint** — it replaces the state of an installation, so it belongs to a
+stopped one and a person at a terminal.
+
+### Integrity check
+
+`verify-backup` runs `PRAGMA integrity_check` (structure) *and*
+`PRAGMA foreign_key_check` (references), compares the snapshot's SHA-256 with the
+manifest, and re-counts every table the manifest recorded. Any mismatch — bit rot, a
+truncated copy, a "repaired" snapshot — fails the check, and `restore` refuses a
+backup that does not verify.
+
+### Restore drill
+
+1. Stop the server and the worker.
+2. `python -m photo_pipeline verify-backup data/backups/<name>` — never restore an
+   unverified snapshot.
+3. `python -m photo_pipeline restore data/backups/<name> --into /path/to/fresh-data`
+   (a target that already holds a database is refused; recovering in place means
+   moving the old data directory aside first).
+4. Point `PHOTO_PIPELINE_DATA_DIR` at the restored directory and run
+   `python -m photo_pipeline migrate`.
+5. Run an inventory scan so paths are reconciled against the real library.
+6. Mount every archive location named in the manifest before archiving again — the
+   database records where archived originals are, but it does not contain them.
+
+Practise this against a copy before you need it; the drill is exercised
+automatically by `tests/integration/test_backup_recovery.py`.
+
+### Failed migration
+
+A pending schema upgrade is snapshotted first (`reason: pre-migration`), by both the
+API startup and `python -m photo_pipeline migrate`. If a migration fails, the error
+log names the backup directory: stop everything and run the restore drill against
+it. An up-to-date database is not backed up again on every start.
+
+### Archive media
+
+Archived originals live on their medium, not in the backup. The manifest lists every
+archive location with its `media_id` and whether it was mounted when the backup was
+taken. Keep one copy of each medium off-site, and remount a location before
+restoring assets from it.
+
+### Retention and disk
+
+`--keep N` (default 7) prunes the oldest backups and never the newest.
+`diagnostics` reports the database, write-ahead log, thumbnail cache, uploader
+reports, backups, and logs separately, with free space and warnings for low disk
+(`disk_low`, `disk_critical`), a cache over its quota, a write-ahead log outgrowing
+its database, and a legacy CLI writing the library.
+
+### Process locking
+
+`serve` and `worker` take a JSON lock in the data directory (`api.lock.json`,
+`worker.lock.json`). A second worker exits `2` and names the holder; a lock whose
+process is gone is taken over. If the frozen CLI's state files are being written,
+both refuse with exit `3` — `--allow-legacy` overrides, and you own the outcome.
+
+## Legacy CLI archive
+
+The command-line tools this application was extracted from are frozen in
+`legacy_cli_archive/` (US07-01): the original sources, their docs, the dependency
+lock they were last verified against, schema notes, a redacted sample
+configuration, the donor ledger, and a checksum for every file.
+
+```bash
+cd legacy_cli_archive && shasum -a 256 -c CHECKSUMS.sha256   # verify the archive
+work_item/scripts/python -m pytest tests/unit/test_legacy_archive.py -q   # lint it
+```
+
+They are reference material and rollback evidence only. No module under
+`photo_pipeline/` imports or executes them, the archive is not on the application's
+import path, and `tests/unit/test_legacy_archive.py` enforces that along with the
+checksums and the redaction. Only the two suites that compare *against* the donors —
+`tests/characterization/` and `tests/integration/test_safety_parity.py` — put the
+archived sources on `sys.path`.
+
+The last path-keyed state they owned, `nsfw_scores.csv`, is imported once and then
+left alone:
+
+```bash
+work_item/scripts/python -m photo_pipeline import-legacy-scores /path/to/nsfw_scores.csv --dry-run
+```
+
+The import writes scored-but-unreviewed `safety_reviews` rows onto stable asset ids,
+never invents an asset for an unknown path, never overwrites a human decision, and
+writes a reconciliation report to the data directory saying exactly what it did.
+`legacy_cli_archive/donor_ledger.yaml` records every migrated behavior with its
+target, the tests that pin the donor, the tests that prove the replacement, and each
+intentional delta; rows still marked `pending` name the story that will resolve them.

delivery_backlog/E08-container-deployment.md

@@ -0,0 +1,32 @@
+# E08 — Container Deployment
+
+Concept phase: none. This epic is a delivery-format addition on top of the concept:
+the same application, same safety invariants, packaged as a Docker image and deployed
+continuously from Gitea Actions instead of being started by hand from a working copy.
+
+It does not change the product scope in
+[`INTEGRATED_PIPELINE_CONCEPT.md`](../INTEGRATED_PIPELINE_CONCEPT.md). SQLite stays the
+store, one worker stays the writer, the library process lock stays authoritative, and
+no path outside the configured library roots becomes reachable because the process now
+runs in a container.
+
+One decision does extend the concept and is made here explicitly: the application may
+be reached through a reverse proxy under a real hostname, not only over loopback. That
+requires a configurable trust boundary and an authentication gate, because the
+loopback-only checks of US07-02 are what currently stand in for authentication.
+
+## Stories
+
+1. [US08-01 — Make the trust boundary configurable and authenticated](stories/US08-01-trusted-hosts-auth.md)
+2. [US08-02 — Build a reproducible application image](stories/US08-02-container-image.md)
+3. [US08-03 — Compose the runtime and mount the library safely](stories/US08-03-compose-runtime.md)
+4. [US08-04 — Publish and deploy from Gitea Actions](stories/US08-04-gitea-cicd.md)
+5. [US08-05 — Automate container deployment acceptance](stories/US08-05-container-e2e.md)
+
+## Epic outcome
+
+A tagged image built from `main` runs the API and the worker as separate containers
+against a mounted library and a persistent data volume, is published to the Gitea
+registry, is redeployed by webhook, survives restart and upgrade with its database and
+journals intact, and refuses every request that a loopback deployment would have
+refused.

delivery_backlog/README.md

@@ -2,11 +2,13 @@
 
 This backlog decomposes the phases in
 [`INTEGRATED_PIPELINE_CONCEPT.md`](../INTEGRATED_PIPELINE_CONCEPT.md) into seven
-epics and small, independently verifiable user stories.
+epics and small, independently verifiable user stories, plus one delivery-format
+epic (E08) that packages the released application as a deployable container.
 
 ## Numbering and file naming
 
-- Epics: `E01` through `E07`, matching concept Phases A through G.
+- Epics: `E01` through `E07`, matching concept Phases A through G; `E08` has no
+  concept phase and must not change product scope.
 - Stories: `US<epic>-<sequence>`, for example `US03-02`.
 - Epic files: `E01-<slug>.md`.
 - Story files: `stories/US01-01-<slug>.md`.
@@ -36,6 +38,7 @@ epics and small, independently verifiable user stories.
 5. [E05 — Immich upload](E05-immich-upload.md)
 6. [E06 — Archive lifecycle](E06-archive-lifecycle.md)
 7. [E07 — Hardening and release](E07-hardening-release.md)
+8. [E08 — Container deployment](E08-container-deployment.md)
 
 ## Shared definition of done
 

delivery_backlog/stories/US08-01-trusted-hosts-auth.md

@@ -0,0 +1,42 @@
+# US08-01 — Make the Trust Boundary Configurable and Authenticated
+
+Epic: [E08](../E08-container-deployment.md)
+
+As an operator, I want to reach the application through my own hostname without
+weakening it, so a container behind a reverse proxy is as safe as the loopback
+deployment it replaces.
+
+## Context
+
+`photo_pipeline/api/security.py` refuses any request whose `Host` or `Origin` is not
+loopback. That check is the current stand-in for authentication: whoever can reach
+`127.0.0.1:8000` is the owner. Behind a proxy the hostname is no longer loopback, so
+relaxing the check without adding an authentication gate would publish the library.
+
+## Acceptance criteria
+
+- Allowed hosts and origins come from configuration (`PHOTO_PIPELINE_*`), default to
+  the current loopback set, and an unset configuration behaves exactly as today.
+- Whenever a non-loopback host is configured, startup requires an access secret and
+  refuses to serve without one; loopback-only deployments keep working with no secret.
+- The secret is exchanged for the existing session cookie and CSRF token through the
+  bootstrap endpoint; every protected route keeps its current session and CSRF
+  requirements unchanged.
+- Forwarded headers (`X-Forwarded-Proto`, `X-Forwarded-Host`) are honored only from a
+  configured trusted proxy and ignored otherwise, so a client cannot forge its origin.
+- Cookies are marked `Secure` when the effective external scheme is HTTPS.
+- Failed authentication is rate-limited and logged without the secret, the session id,
+  or any request body.
+- Health endpoints stay reachable without the secret; nothing else does.
+
+## Automated tests
+
+- Unit tests for host/origin evaluation across loopback default, configured host,
+  unconfigured host, forged forwarded headers, and trusted-proxy forwarded headers.
+- Integration tests: startup refusal without a secret, successful exchange, wrong
+  secret, replay of an old session, cross-site request, and unauthenticated access to
+  every route class.
+
+## Dependencies
+
+- US07-02

delivery_backlog/stories/US08-02-container-image.md

@@ -0,0 +1,41 @@
+# US08-02 — Build a Reproducible Application Image
+
+Epic: [E08](../E08-container-deployment.md)
+
+As an operator, I want one image that can run either application role, so deployment is
+a pull instead of a Python environment I have to reproduce by hand.
+
+## Context
+
+The application shells out to `exiftool` and `immich-go`, writes into the library as a
+normal filesystem user, and serves a static frontend from `frontend/`. All three have to
+be true inside the image, or the container starts and then fails on the first real
+operation.
+
+## Acceptance criteria
+
+- A `Dockerfile` builds from a pinned Python base, installs the project and its runtime
+  dependencies, and contains no test, playwright, or build-only tooling in the final
+  layer.
+- `exiftool` and `immich-go` are present at pinned versions, and their versions are
+  recorded in the image and reported by `python -m photo_pipeline diagnostics`.
+- The image runs as a non-root user whose UID/GID are build-time arguments, so files
+  the application renames or writes keep the ownership the host library expects.
+- One entrypoint selects the role: `serve` or `worker`, passing through the existing
+  CLI arguments; no supervisor runs two roles in one container.
+- `serve` containers declare a `HEALTHCHECK` against `/api/v1/health/ready`, so an
+  unmigrated or misconfigured database is not reported healthy.
+- The image contains no secrets, no library data, no database, and no `.git`; the build
+  context is constrained by `.dockerignore`.
+- Image build is reproducible from a clean checkout and documented in `README.md`.
+
+## Automated tests
+
+- A build-and-run test asserts the image starts, reports ready, serves the frontend
+  index, and returns the pinned `exiftool` and `immich-go` versions.
+- A test asserts the container refuses to run as UID 0 and that a file created by the
+  container is owned by the configured UID/GID.
+
+## Dependencies
+
+- US07-05

delivery_backlog/stories/US08-03-compose-runtime.md

@@ -0,0 +1,47 @@
+# US08-03 — Compose the Runtime and Mount the Library Safely
+
+Epic: [E08](../E08-container-deployment.md)
+
+As an operator, I want a single compose file that runs the API and the worker against my
+real library, so a deployment is one command and the safety invariants survive it.
+
+## Context
+
+The library process lock (US07-05) assumes both roles see the same lock file, and SQLite
+in WAL mode assumes a real local filesystem. Container path policy is the same problem
+as host path policy with a new failure mode: the configured library roots must name the
+in-container mount paths, not the host paths.
+
+## Acceptance criteria
+
+- `docker-compose.yml` runs exactly one `serve` and one `worker` container from the same
+  image and the same data volume, and a second worker is refused by the existing lock
+  rather than by convention.
+- The library is a bind mount; `PHOTO_PIPELINE_LIBRARY_ROOTS` names the container-side
+  paths, and a mismatch between mounted and configured roots fails at startup with a
+  clear message instead of at the first write.
+- The data volume holds the database, WAL, thumbnail cache, and backups on a local
+  filesystem; the composition documents that a network mount is unsupported for it.
+- Migrations run before `serve` and `worker` accept work, using the existing backup-then-
+  migrate path, and an upgrade that fails leaves the previous database intact.
+- Configuration and secrets come from the environment, never from the image or a
+  committed file; a `.env.example` lists every `PHOTO_PIPELINE_*` variable with safe
+  defaults and no values.
+- The API port is published to host loopback by default; exposing it publicly requires
+  the configured hostname and access secret from US08-01.
+- Containers restart automatically, and a restart mid-job resumes exactly as a host
+  restart does today.
+- Backup, verify-backup, restore, and diagnostics are documented as container commands
+  and work against the mounted volumes.
+
+## Automated tests
+
+- An integration test brings the composition up against a temporary fixture library,
+  runs a job, restarts both containers, and asserts the job resumes and the database is
+  intact.
+- Tests for: second worker refused, library-root mismatch refused at startup, failed
+  migration leaving the previous database restorable.
+
+## Dependencies
+
+- US08-01, US08-02

delivery_backlog/stories/US08-04-gitea-cicd.md

@@ -0,0 +1,40 @@
+# US08-04 — Publish and Deploy from Gitea Actions
+
+Epic: [E08](../E08-container-deployment.md)
+
+As a release owner, I want `main` to build, publish, and redeploy the image
+automatically, so deployment is the same reproducible path every time.
+
+## Context
+
+The workflow is adapted from the `crowdsec-admin` deployment workflow
+(`.gitea/workflows/deploy.yml` in that repository): build, log in to the Gitea registry,
+push, trigger a Portainer webhook, prune. This project needs the same shape plus a test
+gate, because unlike that project it has a required suite that must not be skipped.
+
+## Acceptance criteria
+
+- `.gitea/workflows/` contains a test workflow that runs on pull requests and on `main`,
+  executing the configured required suites, and a deploy workflow that runs only after
+  the tests pass on `main` and on manual dispatch.
+- The deploy workflow publishes to `git.domverse-berlin.eu` under this project's own
+  image path, tagged `latest` and the commit SHA, so a rollback is a tag change.
+- Registry credentials and the Portainer webhook come from repository secrets; runtime
+  secrets (vision key, Immich key, access secret) stay in the Portainer stack and never
+  enter the repository or the image.
+- Redeploy is triggered by webhook and the workflow fails when the webhook call fails.
+- Dangling images are pruned; published tags are not.
+- A concurrency guard prevents two deploys of different commits overlapping.
+- `README.md` documents the required secrets, the image path, the rollback procedure,
+  and that the stack is managed by Portainer from git.
+
+## Automated tests
+
+- Workflow files are validated (syntax and required job/step names) by a repository test
+  so a rename cannot silently disable the test gate.
+- A dry-run job builds and pushes to a scratch tag on manual dispatch without touching
+  `latest` or triggering a redeploy.
+
+## Dependencies
+
+- US08-02, US08-03

delivery_backlog/stories/US08-05-container-e2e.md

@@ -0,0 +1,30 @@
+# US08-05 — Automate Container Deployment Acceptance
+
+Epic: [E08](../E08-container-deployment.md)
+
+As a release owner, I want one automated gate that proves the deployed container, so the
+packaged application is verified the same way the host application is.
+
+## Acceptance criteria
+
+- One documented command provisions the composition from the built image against a
+  temporary fixture library and an isolated data volume, and destroys it afterwards.
+- A browser journey against the containerized application covers discovery, duplicate
+  review, analysis, album proposal, rename, upload preflight, and archive views.
+- An upgrade journey runs the previous published image, then the new one, and asserts
+  migrations, journals, jobs, and the thumbnail cache survive.
+- A restart journey kills both containers mid-job and asserts resume without duplicate
+  side effects.
+- Security gates run against the deployed instance: unauthenticated access refused,
+  forged forwarded headers refused, paths outside the mounted library roots refused, and
+  no secret in container logs.
+- Evidence is retained per run and the gate fails on any skipped required check.
+
+## Automated tests
+
+- The container acceptance suite runs on a `phase_h` marker in CI on `main` and before a
+  published deploy; earlier epic suites keep running unchanged.
+
+## Dependencies
+
+- US08-01 through US08-04

frontend/js/api.js

@@ -2,14 +2,62 @@
 // cancellation. Every method accepts an optional { signal } from cancellable().
 export const BASE = "/api/v1";
 
+// The API refuses every request without the session cookie, and every mutation
+// without this token echoed back. The token is readable only same-origin, which is
+// what makes it proof that the caller is this app and not another page.
+let csrfToken = null;
+
+// A deployment reachable through a proxy trades an operator secret for that cookie.
+// Kept per tab: sessionStorage dies with the tab, and the secret never enters a URL.
+const SECRET_KEY = "pp_access_secret";
+
+async function bootstrap(secret) {
+  return fetch(BASE + "/session", {
+    credentials: "same-origin",
+    headers: secret ? { "X-Access-Secret": secret } : {},
+  });
+}
+
+async function session() {
+  if (csrfToken === null) {
+    let response = await bootstrap(sessionStorage.getItem(SECRET_KEY));
+    if (response.status === 401) {
+      sessionStorage.removeItem(SECRET_KEY);
+      const secret = prompt("Access secret");
+      if (secret) {
+        response = await bootstrap(secret);
+        if (response.ok) sessionStorage.setItem(SECRET_KEY, secret);
+      }
+    }
+    const body = await response.json().catch(() => null);
+    csrfToken = (body && body.csrf_token) || null;
+  }
+  return csrfToken || "";
+}
+
+async function send(path, { signal, ...options }) {
+  return fetch(BASE + path, {
+    credentials: "same-origin",
+    signal,
+    ...options,
+    headers: {
+      "Content-Type": "application/json",
+      "X-CSRF-Token": await session(),
+      ...(options.headers || {}),
+    },
+  });
+}
+
 async function request(path, { signal, ...options } = {}) {
   let response;
   try {
-    response = await fetch(BASE + path, {
-      headers: { "Content-Type": "application/json" },
-      signal,
-      ...options,
-    });
+    response = await send(path, { signal, ...options });
+    // A restarted server issues a new session; re-bootstrap once rather than
+    // stranding an open tab on 401.
+    if (response.status === 401) {
+      csrfToken = null;
+      response = await send(path, { signal, ...options });
+    }
   } catch (error) {
     // A caller-cancelled fetch is not a failure; tag it so views can ignore it.
     if (error.name === "AbortError") {
@@ -45,8 +93,10 @@ export const api = {
     request("/inventory/assets?" + new URLSearchParams(params).toString(), opts),
   listClusters: (params = {}, opts = {}) =>
     request("/duplicates/clusters?" + new URLSearchParams(params).toString(), opts),
-  getCluster: (id, opts = {}) =>
-    request(`/duplicates/clusters/${encodeURIComponent(id)}`, opts),
+  getCluster: (id, params = {}, opts = {}) => {
+    const query = new URLSearchParams(params).toString();
+    return request(`/duplicates/clusters/${encodeURIComponent(id)}${query ? `?${query}` : ""}`, opts);
+  },
   decide: (id, payload, opts = {}) =>
     request(`/duplicates/clusters/${encodeURIComponent(id)}/decision`, {
       method: "POST",

frontend/js/app.js

@@ -193,7 +193,7 @@ async function renderClusters(params) {
         " ",
         el("span", { class: `badge ${cluster.state}` }, cluster.state)
       ),
-      el("div", { class: "muted" }, `${cluster.members.length} members · confidence ${cluster.confidence}`)
+      el("div", { class: "muted" }, `${cluster.member_total ?? cluster.members.length} members · confidence ${cluster.confidence}`)
     )
   );
 
@@ -207,8 +207,11 @@ async function renderClusters(params) {
 async function renderClusterDetail(id, extra = {}) {
   setActiveNav("duplicates");
   let cluster;
+  // A cluster can hold thousands of members, so the server pages them; the page
+  // asks for as many as it is currently showing (US07-06).
+  const shown = extra.shown || 0;
   try {
-    cluster = await api.getCluster(id);
+    cluster = await api.getCluster(id, shown ? { limit: shown } : {});
   } catch (error) {
     show(errorBanner(`Failed to load cluster: ${error.message}`));
     return;
@@ -298,6 +301,19 @@ async function renderClusterDetail(id, extra = {}) {
   nodes.push(decisionBar);
   if (extra.pending) nodes.push(confirmPanel(cluster, extra.pending));
   nodes.push(el("div", { class: "cluster-grid" }, ...members));
+  const total = cluster.member_total ?? cluster.members.length;
+  if (cluster.members.length < total) {
+    nodes.push(
+      el(
+        "button",
+        {
+          "data-testid": "show-more-members",
+          onclick: () => renderClusterDetail(id, { ...extra, shown: cluster.members.length + 100 }),
+        },
+        `Show more (${cluster.members.length} of ${total})`
+      )
+    );
+  }
   show(...nodes);
 }
 

frontend/js/tests/unit.js

@@ -29,6 +29,10 @@ function jsonResponse(status, body) {
 const tick = (ms = 0) => new Promise((r) => setTimeout(r, ms));
 
 async function run() {
+  // The client fetches its CSRF token once, lazily (US07-02). Do that against the
+  // real server first, so the stubbed fetch below only ever sees the call under test.
+  await api.workflow().catch(() => {});
+
   // ── store ────────────────────────────────────────────────────────────────
   {
     const store = createStore({ n: 0 });

legacy_cli_archive/CHECKSUMS.sha256

@@ -0,0 +1,22 @@
+d098bbde13d2ebc872ca781e244cc1e48b97cc551fedaca3251d6a5278a15234  src/compare_models.py
+7d0f68cb95fbd6603e4c558620098b46929ebdb6fd91a598bbf84f26eea35e6c  src/nsfw_tag.py
+35329c53570e215cef15f59429c1a5251a0448b90522b0910ab9d68efe1bc307  src/nsfwtag/__init__.py
+7f52b55e4f3b77eda3657d7cd2272c7cca8422100e241671de495603c1126ee2  src/nsfwtag/__main__.py
+6f5a97114e0d87d272ce22d065dc31fa0dd72ff8590d11ad14cb1c1f486d339c  src/nsfwtag/bench.py
+29498fac1d73ba2b7420ffe1ffe49a87f684ec299e2c3b9c6e14e63e95643ba4  src/nsfwtag/exif.py
+934e82c402813ebf503e5a20eb03d5103f84df13117bc4dd1fd95deaa01ab263  src/nsfwtag/README.md
+68125e6184c7e4d2a5b0153f2155675753ab0b1329d1dbc4933d4769492ec4be  src/nsfwtag/review.html
+a597eab74803dd31452b5d70abf7d6d6320eb17a61596c891b009da15d624e98  src/nsfwtag/scoring.py
+674969a18e58ee511a2abf574a875db92f2f524769b1b614d0d3662ab879a709  src/nsfwtag/server.py
+24a7c8d029da6e97d46b110a9fe9dbb3900127f8ae3c142f634fdd91d6c45243  src/nsfwtag/webapp.py
+2c2ea558f1b9095c1150f078ea29c8e0f180cc045a9d1097b83f61bfb145295b  src/photo_analyzer.py
+5fb7ce2f977a17da1f501c45d7985328318ae14298a3f347eab46a6e3f02aab3  src/test_dedup.py
+1673dc76cc60aba56a5b065d9bc9f5dda00342cf2a79d000a3f74686fb7cffec  src/test_nsfw_skip.py
+55aa82f348e0e90be0163ba6aa278e5fc8e445996e5335774917e5ef59ad3563  src/webapp/__init__.py
+ba4342bd0175591a2121f063f14a78a602d5679bcfcf24aa14c8d4b8cd5da1a2  src/webapp/__main__.py
+a60f24035a989909467778f888d62854c1afc2702f47b8d3cca32bf3cf04d5cf  src/webapp/analyzer.html
+018bcc2f770d716444b456db58d6b4a41800138a98598b3c27ba503ab22d939e  src/webapp/page.py
+5f599b107b2b117ca118b6fbdec5e5786a9ab4eea424aa9af1bfea7bf87cebab  src/webapp/query.py
+875cce697caa02717c85a2707ba935c46dd9ace18428fd4abb13105b7a8e24d9  src/webapp/README.md
+af7d0d72d245b4bbb1e0e30f9708239697543212d4171defd1bfe26dd24a4f05  src/webapp/runner.py
+cddf0555b06fe7e4daa14309baaaf91130f8506922cb0ffb6507d76cbe39add4  src/webapp/server.py

legacy_cli_archive/README.md

@@ -0,0 +1,80 @@
+# Legacy CLI archive (US07-01)
+
+Frozen, read-only sources of the command-line tools this application was extracted
+from. They are **reference material and rollback evidence** — provenance for
+behavior that now lives in `photo_pipeline/`, and the only way to answer "what did
+the original actually do?" once the replacement has drifted.
+
+> **Nothing here is production code.** No module under `photo_pipeline/` imports or
+> executes anything in this directory, and this directory is not on the application's
+> import path. `tests/unit/test_legacy_archive.py` enforces both, along with the
+> checksums and the redaction below.
+
+## What is here
+
+| Path | Role |
+|---|---|
+| `src/photo_analyzer.py` | the analysis CLI: discovery, hashing/dedup, vision analysis, EXIF writing, SQLite schema, album naming |
+| `src/nsfwtag/` | NSFW scoring, EXIF safety keywords, and the review server (`__init__` 1.1.0) |
+| `src/webapp/` | the stdlib review web app: FTS search, stats, subprocess runner, HTML shell (`__init__` 0.1.0) |
+| `src/nsfw_tag.py` | thin backwards-compatible entry point for `nsfwtag` |
+| `src/compare_models.py` | dev-only model comparison script |
+| `src/test_dedup.py`, `src/test_nsfw_skip.py` | the CLIs' own standalone self-checks (never pytest suites) |
+| `donor_ledger.yaml` | the donor ledger: every migrated behavior, its target, its tests, and every intentional delta |
+| `requirements-lock.txt` | the dependency versions the frozen sources were last verified against |
+| `photo_analyzer.env.sample` | the CLI's configuration surface, with every value replaced by a placeholder |
+| `CHECKSUMS.sha256` | SHA-256 of every archived source file |
+
+`photo_analyzer.py` carries no `__version__`; its identity is its checksum, recorded
+in `CHECKSUMS.sha256` and taken at commit `9b7ee6b` (the merge of US06-06, the last
+commit before archival).
+
+## Verifying the archive
+
+```bash
+cd legacy_cli_archive && shasum -a 256 -c CHECKSUMS.sha256
+```
+
+Any edit to an archived source must be accompanied by a regenerated checksum file
+and a note here explaining why a *frozen* archive changed — the normal answer being
+that it should not.
+
+## Schema notes
+
+`photo_analyzer.py` owned a path-keyed SQLite database (`SCHEMA`, near the top of the
+file):
+
+- `photos(id, path UNIQUE, status, phash, file_sha1, dup_of, description, tags,
+  people_count, setting, time_of_day, season, mood, location_hint, approx_year,
+  raw_response, error_message, analyzed_at, exif_written_at)`;
+- `photos_fts` — an FTS5 external-content index over `path, description, tags, mood,
+  location_hint`, kept in sync by insert/update/delete triggers;
+- late columns (`phash`, `file_sha1`, `dup_of`) were added by an in-code
+  `_migrate_schema()` rather than a migration tool, and their indexes are created
+  only after the `ALTER`.
+
+The replacement keeps the same analysis fields but re-keys everything to a stable
+`assets.id` (Alembic migrations `0001`…), because a path is not an identity: the
+donor's `path UNIQUE` is exactly what broke on every move and rename.
+
+`nsfwtag` kept its safety scores outside the database in `nsfw_scores.csv`
+(`path,nsfw_score`, four decimals, unreadable rows dropped). That file is no longer a
+source of truth; `photo_pipeline/services/legacy_import.py` imports it into
+`assets.safety_score` and reports exactly what matched, what did not, and why.
+
+## Redaction
+
+The archive contains no credentials. `photo_analyzer.env.sample` documents the
+configuration surface (`LLM_API_KEY`, `LLM_BASE_URL`, `LLM_MODEL`, and the tuning
+variables) with placeholder values only; the CLI itself never contained a key, it
+read one from `photo_analyzer.env` or the environment. No `.env`, database, log, CSV,
+or photo from the author's library is archived.
+
+## Why these tools were retired
+
+Each behavior's fate is recorded per row in `donor_ledger.yaml`: `reuse`, `extract`,
+`refactor`, or `replace`, with the target module, the characterization tests that
+pinned the donor's behavior, the parity tests the replacement passes, and — where the
+replacement deliberately does something else or nothing at all — a `delta` saying so.
+Rows still marked `pending` name the backlog story that will resolve them; they are
+the honest list of what has *not* been carried over yet.

legacy_cli_archive/donor_ledger.yaml

@@ -12,7 +12,15 @@
 #                            (if anything) carries over
 # Every row needs either `tests` (existing test IDs, module::function) or
 # `pending_story` (the backlog story that will characterize/deliver it).
-# status: characterized | pending
+# status: characterized — donor behavior pinned by characterization tests
+#         resolved      — replacement shipped; `parity` names the tests that prove
+#                         it, and `delta` states every intentional difference
+#         pending       — not migrated yet; `pending_story` says which story will
+# parity: test ids (path::function) in any suite, proving the replacement
+# delta:  what the replacement deliberately does differently, or not at all
+#
+# Archived by US07-01: the sources referenced below now live beside this file in
+# src/ and are frozen (see README.md). Nothing in photo_pipeline imports them.
 
 rows:
   # ── photo_analyzer.py ──────────────────────────────────────────────────────
@@ -80,8 +88,12 @@ rows:
       Copying a primary's analysis into variant rows survives, but keyed by
       asset_id and recorded as stage state instead of raw row copies.
     target: photo_pipeline/services/duplicates.py
-    pending_story: US01-04
-    status: pending
+    parity:
+      - tests/integration/test_duplicate_engine.py::test_exact_copies_form_auto_decided_cluster
+      - tests/integration/test_duplicate_engine.py::test_perceptual_variant_is_review_only
+    delta: >
+      The donor propagated variant links implicitly while writing rows; the replacement makes the canonical link a reviewable, reversible cluster decision, so a propagated link can always be undone.
+    status: resolved
 
   - id: pa-hashing
     area: hashing
@@ -141,8 +153,12 @@ rows:
     classification: replace
     rationale: Console report; superseded by the duplicate-review API/UI (US01-06).
     target: photo_pipeline/api/routes + frontend duplicate review
-    pending_story: US01-06
-    status: pending
+    parity:
+      - tests/integration/test_review_api.py::test_clusters_list_and_detail
+      - tests/e2e/test_review_ui.py::test_fuzzy_decision_requires_confirmation
+    delta: >
+      The text listing became the paged cluster API and the comparison UI; no textual report is produced.
+    status: resolved
 
   - id: pa-reconcile
     area: database
@@ -202,7 +218,7 @@ rows:
       RGB-normalize (drops alpha, converts HEIC), LANCZOS resize to 2048px
       long-edge, JPEG q85 base64 — the provider-input contract. Truncated-image
       tolerance (ImageFile.LOAD_TRUNCATED_IMAGES) carries with it.
-    target: photo_pipeline/integrations/vision.py
+    target: photo_pipeline/services/analysis.py
     tests:
       - test_pa_imaging::test_prepare_image_small_passthrough_jpeg
       - test_pa_imaging::test_prepare_image_resizes_to_max_long_edge
@@ -230,9 +246,12 @@ rows:
       response validation, 429/503 retry with exponential backoff. Prompt and
       model/config version must be persisted per analysis_runs. Characterized
       against a deterministic fake provider when the analysis service is ported.
-    target: photo_pipeline/integrations/vision.py
-    pending_story: US02-06
-    status: pending
+    target: photo_pipeline/services/analysis.py
+    parity:
+      - tests/integration/test_safety_analysis.py::test_provider_called_only_for_confirmed_sfw
+    delta: >
+      The prompt and response schema carry over; the provider is an injected adapter so the privacy gate is testable, and results are keyed to asset ids rather than paths.
+    status: resolved
 
   - id: pa-throttle
     area: logging
@@ -241,8 +260,8 @@ rows:
     rationale: >
       Rolling throttle window + persistent throttle_events.jsonl + RPD day
       counter become job metrics/events on the durable job model.
-    target: photo_pipeline/jobs/coordinator.py
-    pending_story: US02-02
+    target: photo_pipeline/services/jobs.py + photo_pipeline/services/analysis.py
+    pending_story: US07-04
     status: pending
 
   - id: pa-nsfw-filter
@@ -312,8 +331,12 @@ rows:
       checked between items, double-SIGINT force quit — becomes the durable
       JobRunner worker loop with the same drain-and-resume semantics.
     target: photo_pipeline/jobs/worker.py
-    pending_story: US02-02
-    status: pending
+    parity:
+      - tests/integration/test_worker.py::test_worker_processes_all_items
+      - tests/integration/test_worker.py::test_cooperative_cancellation_leaves_items_resumable
+    delta: >
+      The in-process folder loop with SIGINT handling became durable jobs claimed by a worker: cancellation is a persisted request, not a signal, and an interrupted run resumes from the database instead of restarting.
+    status: resolved
 
   - id: pa-ui-terminal
     area: ui
@@ -371,8 +394,12 @@ rows:
       the JSONL history logger become structured JSON logging with job_id/
       asset_id and job_events rows; per-photo history maps to job events.
     target: photo_pipeline structured logging + jobs/job_events
-    pending_story: US02-02
-    status: pending
+    parity:
+      - tests/integration/test_jobs.py::test_enqueue_persists_items_and_event
+      - tests/integration/test_jobs_sse.py::test_sse_streams_all_events_then_closes
+    delta: >
+      The JSONL history file and rich console handler are replaced by structured JSON logs plus durable job_events; the browser reads events over SSE rather than tailing a file.
+    status: resolved
 
   - id: pa-balance
     area: vision
@@ -382,8 +409,11 @@ rows:
       Provider balance/quota probes (report 'unsupported' on providers without
       the endpoint). Network-bound; characterized against the fake provider.
     target: photo_pipeline/services/analysis.py
-    pending_story: US02-06
-    status: pending
+    parity:
+      - tests/integration/test_safety_analysis.py::test_provider_called_only_for_confirmed_sfw
+    delta: >
+      Not carried over: balance/quota polling was provider-specific (Gemini/OpenAI billing endpoints) and key-scoped. Cost reporting, when a story asks for it, comes from the per-run usage recorded with each analysis result rather than from a vendor endpoint.
+    status: resolved
 
   - id: pa-cli
     area: configuration
@@ -393,9 +423,13 @@ rows:
       argparse surface is superseded by the API; flags map to job configs
       (documented in WEBAPP_CONCEPT.md §8 parity table). Transitional CLI calls
       the shared services until archival (E07).
-    target: photo_pipeline/api + transitional CLI
-    pending_story: US07-01
-    status: pending
+    target: photo_pipeline/__main__.py (serve | worker | migrate | import-legacy-scores) + /api/v1
+    parity:
+      - tests/integration/test_app_lifecycle.py::test_restart_preserves_data_and_reruns_migrations
+      - tests/unit/test_legacy_archive.py::test_production_code_never_imports_an_archived_module
+    delta: >
+      The argparse surface is not reproduced. Every flag that drove work became an API command or a job configuration; the CLI keeps only what an application needs to be operated (serve, worker, migrate) plus the one-off legacy CSV import.
+    status: resolved
 
   # ── nsfwtag/ ───────────────────────────────────────────────────────────────
   - id: nt-discovery
@@ -422,9 +456,17 @@ rows:
       nsfw_scores.csv stops being the source of truth (concept: DB state).
       Format characterized (4-decimal scores, bad rows dropped) because the
       existing CSV must migrate into assets.safety_score.
-    target: photo_pipeline/repositories (safety), CSV import in US01-02 migration
+    target: photo_pipeline/services/legacy_import.py (one-off import into safety_reviews)
     tests: [test_nsfwtag::test_score_cache_roundtrip_and_tolerance]
-    status: characterized
+    parity:
+      - tests/integration/test_legacy_import.py::test_scores_are_imported_onto_asset_identity
+      - tests/integration/test_legacy_import.py::test_a_reviewed_asset_is_never_overwritten_by_the_csv
+    delta: >
+      The CSV is no longer read at runtime at all: it is imported once into
+      scored-but-unreviewed safety_reviews rows and left on disk untouched. A path
+      that matches nothing is reported, never turned into an asset, and a human
+      decision always outranks an imported score.
+    status: resolved
 
   - id: nt-score-model
     area: nsfw
@@ -438,6 +480,13 @@ rows:
       inference path needs the local model + deterministic fake.
     target: photo_pipeline/integrations/nsfw_model.py
     tests: [test_nsfwtag::test_score_images_cache_hit_skips_model]
+    delta: >
+      The donor set Pillow's process-global ImageFile.LOAD_TRUNCATED_IMAGES so a
+      partially downloaded file still scored. Here the same process also hashes
+      files and renders previews, and those must keep refusing a truncated file
+      rather than silently working on half of one; scoring opens images through
+      the bounded photo_pipeline.imaging door instead and skips the ones it cannot
+      read, leaving them unscored and visibly undecided (US07-03).
     status: characterized
 
   - id: nt-exif-keyword
@@ -485,8 +534,11 @@ rows:
       Newline-list bulk tagging (nsfw_confirmed.txt flow) is superseded by DB
       review decisions; the existing list is a one-time migration input.
     target: photo_pipeline/services/safety.py (decision import in US01-02)
-    pending_story: US01-02
-    status: pending
+    parity:
+      - tests/integration/test_safety_parity.py::test_extracted_marks_and_partition_match_donor
+    delta: >
+      Bulk keyword application from a file list is replaced by decisions against asset ids; the EXIF write itself is the extracted, read-back-verified one.
+    status: resolved
 
   - id: nt-ui
     area: ui
@@ -498,8 +550,11 @@ rows:
       threshold/score review flow, lightbox and keyboard model are the frontend
       donor for the Safety view (preserved per concept §10; ported in US02-01).
     target: photo_pipeline/api + frontend Safety view
-    pending_story: US02-01
-    status: pending
+    parity:
+      - tests/e2e/test_workflow_views.py::test_safety_review_decide_persists_across_reload
+    delta: >
+      The stdlib review server is replaced by the API plus the Safety view; the donor's layout, thresholds, and keyboard flow carry over, its embedded HTML generation does not.
+    status: resolved
 
   - id: nt-bench
     area: nsfw
@@ -509,8 +564,11 @@ rows:
       Dev-only model benchmark; archived without webapp replacement (recorded
       basis of the AdamCodd model choice). No production caller.
     target: none (archive as reference)
-    pending_story: US07-01
-    status: pending
+    parity:
+      - tests/unit/test_legacy_archive.py::test_every_archived_source_matches_its_checksum
+    delta: >
+      No replacement: a dev-only benchmark whose result (the AdamCodd model choice) is already recorded. Kept in the archive as the basis of that choice.
+    status: resolved
 
   # ── webapp/ ────────────────────────────────────────────────────────────────
   - id: wa-query-search
@@ -579,9 +637,13 @@ rows:
       Subprocess-driving-the-CLI job control is superseded by durable DB jobs
       with a worker process. Two ideas carry over: progress derived from DB
       counts (not job-private state) and single-mutating-job enforcement.
-    target: photo_pipeline/jobs/coordinator.py
-    pending_story: US02-02
-    status: pending
+    target: photo_pipeline/services/jobs.py + photo_pipeline/jobs/worker.py
+    parity:
+      - tests/integration/test_jobs.py::test_idempotency_key_returns_same_job
+      - tests/integration/test_worker.py::test_handler_failure_fails_the_job
+    delta: >
+      Subprocess supervision of a CLI is replaced by durable jobs in the same process family: there is no subprocess to supervise, and progress is persisted rather than scraped from stdout.
+    status: resolved
 
   - id: wa-server
     area: ui
@@ -593,5 +655,9 @@ rows:
       browser. analyzer.html + page.py design (dark OLED tokens, Library/
       Analyze/Stats views) is frontend donor material per concept §10.
     target: photo_pipeline/api/app.py + frontend
-    pending_story: US02-05
-    status: pending
+    parity:
+      - tests/e2e/test_frontend_shell.py::test_shell_loads_assets_without_console_or_network_errors
+      - tests/e2e/test_frontend_shell.py::test_deep_link_and_reload_restore_view_and_filters
+    delta: >
+      The stdlib HTTP routes become the versioned FastAPI API and the static frontend shell; Python no longer interpolates HTML.
+    status: resolved

legacy_cli_archive/photo_analyzer.env.sample

@@ -0,0 +1,18 @@
+# Sample configuration for the archived photo_analyzer.py CLI (US07-01).
+#
+# REDACTED: no value below is real. Each line names a variable the CLI read and
+# what belongs there; the placeholders are deliberately not key-shaped, so this
+# file can never be mistaken for — or scanned as — a credential.
+
+LLM_API_KEY=<paste your provider key here>
+LLM_BASE_URL=<provider base url, e.g. the OpenAI-compatible Gemini endpoint>
+LLM_MODEL=<model id, e.g. a Gemini Flash release>
+
+# Optional tuning the CLI read from the same file:
+PHASH_THRESHOLD=8
+MAX_WORKERS=4
+RETRY_ATTEMPTS=3
+RPD_LIMIT=0
+
+# The CLI ignored its own shipped placeholder (a literal "sk-REPLACE..." string)
+# until it was replaced, and a shell variable always won over this file.

legacy_cli_archive/requirements-lock.txt

@@ -0,0 +1,20 @@
+# Final dependency lock of the archived CLIs (US07-01).
+#
+# These are the versions present in the environment the archive was taken from —
+# what the frozen sources were last verified against by the characterization
+# suite. Restoring a donor for forensics means pinning these, not "latest".
+#
+# Python 3.14.6
+
+openai==3.0.0          # photo_analyzer: OpenAI-compatible vision client
+numpy==2.4.6           # photo_analyzer, nsfwtag: pixel work
+Pillow==12.3.0         # photo_analyzer, nsfwtag: decode/resize
+rich==15.0.0           # photo_analyzer: console output
+scipy==1.18.0          # photo_analyzer: perceptual-hash DCT
+PyYAML==6.0.3          # tooling that reads the donor ledger
+
+# NSFW inference (nsfwtag/scoring.py, nsfwtag/bench.py) was never installed in the
+# archiving environment; the model stack is recorded here from the sources so a
+# forensic run can reproduce it, not from a resolved lock:
+#   torch, transformers, timm  — AdamCodd/vit-base-nsfw-detector (see nsfwtag/README.md)
+# exiftool is an external binary, not a Python package.

migrations/versions/0015_exif_projections.py

@@ -0,0 +1,42 @@
+"""Durable EXIF projections per asset and stage (US07-03).
+
+Revision ID: 0015_exif_projections
+Revises: 0014_restore_plans
+Create Date: 2026-08-16
+
+The concept's ``exif_projections`` table, added at the point it earns its keep: a
+checkpoint that finds a field it does not own changed must be able to say so after
+a restart. ``state`` is verified | divergent | failed, and only ``verified`` counts
+as a completed metadata stage.
+"""
+
+import sqlalchemy as sa
+from alembic import op
+
+revision = "0015_exif_projections"
+down_revision = "0014_restore_plans"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "exif_projections",
+        sa.Column("asset_id", sa.String(), sa.ForeignKey("assets.id"), primary_key=True),
+        sa.Column("stage", sa.String(), primary_key=True),  # safety | analysis
+        sa.Column("id", sa.String(), nullable=False),
+        sa.Column("projection_version", sa.Integer(), nullable=False, server_default="0"),
+        # What the stage asked for: {"add": [...], "remove": [...]}.
+        sa.Column("desired_json", sa.String(), nullable=True),
+        # Fields outside the stage's ownership that did not survive the write.
+        sa.Column("divergent_fields", sa.String(), nullable=True),
+        sa.Column("result_file_sha256", sa.String(), nullable=True),
+        sa.Column("state", sa.String(), nullable=False),
+        sa.Column("error_code", sa.String(), nullable=True),
+        sa.Column("verified_at", sa.DateTime(timezone=True), nullable=True),
+        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=True),
+    )
+
+
+def downgrade() -> None:
+    op.drop_table("exif_projections")

migrations/versions/0016_performance_indexes.py

@@ -0,0 +1,45 @@
+"""Indexes the large-library read paths need (US07-06).
+
+Revision ID: 0016_performance_indexes
+Revises: 0015_exif_projections
+Create Date: 2026-08-17
+
+Measured, not guessed. At 100k assets the workflow home and the safety queue both
+resolve "the current decision per asset" with a window function over
+``safety_reviews``; a plain ``asset_id`` index makes SQLite sort every partition by
+hand. Ordering the index by ``(asset_id, created_at DESC)`` halves that query.
+
+``duplicate_members(cluster_id, asset_id)`` serves the paged member list of a
+cluster with thousands of members, which is the other page that stopped being cheap.
+"""
+
+from alembic import op
+
+revision = "0016_performance_indexes"
+down_revision = "0015_exif_projections"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    op.create_index(
+        "ix_safety_reviews_asset_created",
+        "safety_reviews",
+        ["asset_id", "created_at"],
+    )
+    op.create_index(
+        "ix_duplicate_members_cluster_asset",
+        "duplicate_members",
+        ["cluster_id", "asset_id"],
+    )
+    op.create_index(
+        "ix_analysis_results_approx_year",
+        "analysis_results",
+        ["approx_year"],
+    )
+
+
+def downgrade() -> None:
+    op.drop_index("ix_analysis_results_approx_year", table_name="analysis_results")
+    op.drop_index("ix_duplicate_members_cluster_asset", table_name="duplicate_members")
+    op.drop_index("ix_safety_reviews_asset_created", table_name="safety_reviews")

photo_pipeline/__main__.py

@@ -1,28 +1,190 @@
-"""Application management CLI: ``python -m photo_pipeline {serve,migrate}``."""
+"""Application management CLI:
+``python -m photo_pipeline {serve,migrate,worker,import-legacy-scores,backup,verify-backup,restore,diagnostics}``.
+
+``serve`` and ``worker`` take the library process lock for their role (US07-05):
+two workers, or the frozen CLI running beside the app, would each be safe on their
+own and destructive together. ``restore`` is here rather than in the API because it
+replaces the state of an installation and belongs to a stopped one.
+"""
 
 from __future__ import annotations
 
 import argparse
+import json
+from pathlib import Path
 from typing import Sequence
 
 from photo_pipeline.config import Config
-from photo_pipeline.db import run_migrations
+from photo_pipeline.services.app_lock import LegacyProcessActive, LibraryLock, LockHeld
+from photo_pipeline.services.backup import BackupError, BackupService, migrate_with_backup
 
 
 def main(argv: Sequence[str] | None = None) -> int:
     parser = argparse.ArgumentParser(prog="photo_pipeline")
     commands = parser.add_subparsers(dest="command", required=True)
-    commands.add_parser("serve", help="Run the API server")
+    serve_cmd = commands.add_parser("serve", help="Run the API server")
     commands.add_parser("migrate", help="Upgrade the database to the latest revision")
     worker_cmd = commands.add_parser("worker", help="Run a durable-job worker")
     worker_cmd.add_argument("--id", default="worker-1", help="Worker id (lease owner)")
+    for locked in (serve_cmd, worker_cmd):
+        locked.add_argument(
+            "--allow-legacy",
+            action="store_true",
+            help="Start even though a legacy CLI looks active (unsafe; you own the outcome)",
+        )
+    import_cmd = commands.add_parser(
+        "import-legacy-scores",
+        help="Import the archived CLI's nsfw_scores.csv into the database (US07-01)",
+    )
+    import_cmd.add_argument("csv", help="Path to nsfw_scores.csv")
+    import_cmd.add_argument(
+        "--overwrite", action="store_true", help="Replace differing imported scores"
+    )
+    import_cmd.add_argument(
+        "--dry-run", action="store_true", help="Report what would happen and change nothing"
+    )
+
+    backup_cmd = commands.add_parser("backup", help="Take an online database backup")
+    backup_cmd.add_argument("--reason", default="manual", help="Why (part of the directory name)")
+    backup_cmd.add_argument("--keep", type=int, default=7, help="How many backups to retain")
+    verify_cmd = commands.add_parser("verify-backup", help="Check a backup is intact and readable")
+    verify_cmd.add_argument("backup", help="Path to the backup directory")
+    restore_cmd = commands.add_parser(
+        "restore", help="Restore a verified backup into a fresh data directory"
+    )
+    restore_cmd.add_argument("backup", help="Path to the backup directory")
+    restore_cmd.add_argument("--into", required=True, help="Fresh data directory to restore into")
+    commands.add_parser("diagnostics", help="Report sizes, disk headroom, locks, and warnings")
+    bench_cmd = commands.add_parser(
+        "benchmark", help="Measure latency and resource use against agreed budgets (US07-06)"
+    )
+    bench_cmd.add_argument("--profile", default="smoke", help="smoke | short | full | huge")
+    bench_cmd.add_argument(
+        "--soak-seconds", type=float, default=0.0, help="Also run a soak of this length"
+    )
+    bench_cmd.add_argument("--output", help="Write the JSON report here as well as to stdout")
+
+    gate_cmd = commands.add_parser(
+        "release-gate", help="Run every suite in an isolated stack and keep the evidence"
+    )
+    gate_cmd.add_argument("--output", help="Evidence directory (default: data/release/<stamp>)")
+    dry_cmd = commands.add_parser(
+        "dry-run", help="Read-only reconciliation of the configured library (US07-07)"
+    )
+    dry_cmd.add_argument("--output", help="Write the report here as well as to stdout")
+    approve_cmd = commands.add_parser(
+        "approve-dry-run", help="Approve a dry-run report, which is what enables mutation"
+    )
+    approve_cmd.add_argument("report", help="Path to the dry-run report")
+    approve_cmd.add_argument("--approver", required=True, help="Who is accepting this")
+
     args = parser.parse_args(argv)
 
     config = Config.from_env()
     config.database_path.parent.mkdir(parents=True, exist_ok=True)
 
     if args.command == "migrate":
-        run_migrations(config.database_url)
+        manifest = migrate_with_backup(config)
+        if manifest:
+            print(json.dumps({"pre_migration_backup": manifest["name"]}, indent=2))
+        return 0
+
+    if args.command == "backup":
+        try:
+            manifest = BackupService(config).create(reason=args.reason, keep=args.keep)
+        except BackupError as error:
+            print(str(error))
+            return 1
+        print(json.dumps(manifest, indent=2))
+        return 0
+
+    if args.command == "verify-backup":
+        result = BackupService(config).verify(args.backup)
+        print(json.dumps(result.as_dict(), indent=2))
+        return 0 if result.ok else 1
+
+    if args.command == "restore":
+        try:
+            report = BackupService(config).restore(args.backup, args.into)
+        except BackupError as error:
+            print(str(error))
+            return 1
+        print(json.dumps(report, indent=2))
+        return 0
+
+    if args.command == "benchmark":
+        from photo_pipeline.services import benchmarks
+
+        try:
+            report = benchmarks.run(
+                config,
+                profile=args.profile,
+                soak_seconds=args.soak_seconds,
+                output=args.output,
+            )
+        except ValueError as error:
+            print(str(error))
+            return 1
+        print(json.dumps({k: v for k, v in report.items() if k != "runs"}, indent=2))
+        # A breached budget is a failed run, so a scheduled job notices without
+        # anyone reading the JSON.
+        return 0 if report["ok"] else 1
+
+    if args.command == "release-gate":
+        from photo_pipeline.services import release
+
+        report = release.run_gate(config, output=args.output)
+        print(
+            json.dumps(
+                {k: v for k, v in report.items() if k not in ("stages", "matrix")}, indent=2
+            )
+        )
+        return 0 if report["ok"] else 1
+
+    if args.command == "dry-run":
+        from photo_pipeline.services import release
+
+        try:
+            report = release.dry_run(config)
+        except release.ReleaseError as error:
+            print(str(error))
+            return 1
+        if args.output:
+            Path(args.output).write_text(json.dumps(report, indent=2))
+        print(json.dumps(report, indent=2))
+        return 0
+
+    if args.command == "approve-dry-run":
+        from photo_pipeline.services import release
+
+        try:
+            record = release.approve(config, args.report, approver=args.approver)
+        except (release.ReleaseError, OSError, ValueError) as error:
+            print(str(error))
+            return 1
+        print(json.dumps(record, indent=2))
+        return 0
+
+    if args.command == "diagnostics":
+        from photo_pipeline.services import diagnostics
+
+        print(json.dumps(diagnostics.report(config), indent=2))
+        return 0
+
+    if args.command == "import-legacy-scores":
+        from photo_pipeline.db import create_db_engine, create_session_factory
+        from photo_pipeline.services.legacy_import import LegacyImportService, write_report
+
+        migrate_with_backup(config)
+        engine = create_db_engine(config.database_url)
+        service = LegacyImportService(create_session_factory(engine))
+        report = service.import_nsfw_scores(
+            args.csv, overwrite=args.overwrite, dry_run=args.dry_run
+        )
+        # The report is the point: an import nobody can audit is not a migration.
+        if not args.dry_run:
+            write_report(report, config.data_dir)
+        print(json.dumps(report.counts, indent=2))
         return 0
 
     if args.command == "worker":
@@ -34,18 +196,60 @@ def main(argv: Sequence[str] | None = None) -> int:
         import photo_pipeline.jobs.domain_handlers  # noqa: F401
         from photo_pipeline.jobs.worker import Worker
 
-        run_migrations(config.database_url)
-        engine = create_db_engine(config.database_url)
-        Worker(create_session_factory(engine), worker_id=args.id, config=config).run_forever()
+        lock = LibraryLock(config, "worker")
+        if (held := _acquire(lock, allow_legacy=args.allow_legacy)) is not None:
+            return held
+        try:
+            migrate_with_backup(config)
+            engine = create_db_engine(config.database_url)
+            Worker(
+                create_session_factory(engine), worker_id=args.id, config=config
+            ).run_forever()
+        finally:
+            lock.release()
         return 0
 
+    import sys
+
     import uvicorn
 
-    from photo_pipeline.api.app import create_app
+    from photo_pipeline.api.app import ConfigurationRefused, create_app
 
-    uvicorn.run(create_app(config), host=config.host, port=config.port)
+    # An exposed deployment without an access secret must not reach the port at all,
+    # and the operator needs a sentence, not a traceback (US08-01).
+    try:
+        app = create_app(config)
+    except ConfigurationRefused as error:
+        print(str(error), file=sys.stderr)
+        return 4
+
+    lock = LibraryLock(config, "api")
+    if (held := _acquire(lock, allow_legacy=args.allow_legacy)) is not None:
+        return held
+    try:
+        uvicorn.run(app, host=config.host, port=config.port)
+    finally:
+        lock.release()
     return 0
 
 
+def _acquire(lock: LibraryLock, *, allow_legacy: bool) -> int | None:
+    """Take the lock, or explain on stderr why this process must not start.
+
+    Returns an exit code to return, or ``None`` when the lock was acquired.
+    """
+    import sys
+
+    try:
+        lock.acquire(allow_legacy=allow_legacy)
+    except LockHeld as error:
+        print(str(error), file=sys.stderr)
+        return 2
+    except LegacyProcessActive as error:
+        print(f"{error} (override with --allow-legacy)", file=sys.stderr)
+        return 3
+    return None
+
+
 if __name__ == "__main__":
     raise SystemExit(main())

photo_pipeline/api/app.py

@@ -8,11 +8,15 @@ and exposes the versioned ``/api/v1`` surface; US01-02 ships only health.
 
 from __future__ import annotations
 
+import logging
 from contextlib import asynccontextmanager
 from pathlib import Path
 
-from fastapi import FastAPI
+from fastapi import FastAPI, Request
+from fastapi.exceptions import RequestValidationError
+from fastapi.responses import JSONResponse
 from fastapi.staticfiles import StaticFiles
+from starlette.exceptions import HTTPException as StarletteHTTPException
 
 from photo_pipeline.api.routes import (
     albums,
@@ -23,31 +27,86 @@ from photo_pipeline.api.routes import (
     inventory,
     jobs,
     library,
+    operations,
     renames,
     safety,
+    session as session_routes,
     thumbnails,
     uploads,
     workflow,
 )
+from photo_pipeline.api.security import (
+    DEFAULT_HEADERS,
+    FailureLimiter,
+    SecurityMiddleware,
+    Session,
+    trust_refusal,
+)
 
 # Registers the safety_score / analysis job handlers on import.
 import photo_pipeline.jobs.domain_handlers  # noqa: F401
 from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
+from photo_pipeline.db import create_db_engine, create_session_factory
+from photo_pipeline.services.backup import migrate_with_backup
 from photo_pipeline.logging import configure_logging
+from photo_pipeline.services.thumbnails import ThumbnailService
 from photo_pipeline.services.upload_batches import UploadBatchService
 
 FRONTEND_DIR = Path(__file__).resolve().parents[2] / "frontend"
 
+log = logging.getLogger(__name__)
+
+
+def _envelope(status: int, code: str, message: str) -> JSONResponse:
+    return JSONResponse(
+        status_code=status,
+        content={"error": {"code": code, "message": message}},
+        headers=DEFAULT_HEADERS,
+    )
+
+
+def _install_error_handlers(app: FastAPI) -> None:
+    """One JSON error envelope everywhere, and nothing behind it.
+
+    An unhandled exception carries the library's absolute paths, SQL, and sometimes
+    a credential in its text; the client gets a code, the operator gets the traceback
+    in the server log (US07-02).
+    """
+
+    @app.exception_handler(StarletteHTTPException)
+    async def _http_error(request: Request, exc: StarletteHTTPException):
+        return _envelope(exc.status_code, "http_error", str(exc.detail))
+
+    @app.exception_handler(RequestValidationError)
+    async def _validation_error(request: Request, exc: RequestValidationError):
+        # Field locations only: the echoed input can be the caller's own data, but it
+        # is also what ends up in shared logs and screenshots.
+        fields = sorted(".".join(str(part) for part in error["loc"]) for error in exc.errors())
+        return _envelope(422, "invalid_request", f"invalid request fields: {', '.join(fields)}")
+
+    @app.exception_handler(Exception)
+    async def _unhandled(request: Request, exc: Exception):
+        log.exception("unhandled error serving %s", request.url.path)
+        return _envelope(500, "internal_error", "internal error")
+
+
+class ConfigurationRefused(RuntimeError):
+    """The configuration would serve the library to callers it cannot authenticate."""
+
 
 def create_app(config: Config | None = None) -> FastAPI:
     config = config or Config.from_env()
     configure_logging(config.log_level, config.log_format)
+    # Before anything is built, let alone bound to a port (US08-01).
+    if (why := trust_refusal(config)) is not None:
+        raise ConfigurationRefused(why)
 
     @asynccontextmanager
     async def lifespan(app: FastAPI):
         config.database_path.parent.mkdir(parents=True, exist_ok=True)
-        run_migrations(config.database_url)
+        # A schema upgrade is snapshotted first, so a migration that fails halfway
+        # leaves a restorable database behind rather than a damaged one (US07-05).
+        migrate_with_backup(config)
         engine = create_db_engine(config.database_url)
         app.state.config = config
         app.state.engine = engine
@@ -55,6 +114,9 @@ def create_app(config: Config | None = None) -> FastAPI:
         # An upload whose process died left no outcome behind; resolve it now so the
         # uploader lane is free and the uncertain batch is visible (US05-02).
         UploadBatchService(app.state.session_factory, config=config).recover()
+        # A render killed mid-write leaves its temporary beside the cache entry;
+        # remove those recognized leftovers, and only those (US07-03).
+        ThumbnailService(app.state.session_factory, config).cleanup_temp_files()
         try:
             yield
         finally:
@@ -62,6 +124,16 @@ def create_app(config: Config | None = None) -> FastAPI:
             app.state.engine = None
 
     app = FastAPI(title="Photo Pipeline", version="0.1.0", lifespan=lifespan)
+    # One session per process: the browser exchanges it for a cookie + CSRF token,
+    # and every other origin is refused before a route ever runs (US07-02).
+    app.state.session = Session.create()
+    app.state.access_limiter = FailureLimiter()
+    # Also set in the lifespan, but the bootstrap route reads it, and a caller can
+    # arrive before anything else has touched app.state.
+    app.state.config = config
+    app.add_middleware(SecurityMiddleware, session=app.state.session, config=config)
+    _install_error_handlers(app)
+    app.include_router(session_routes.router, prefix="/api/v1")
     app.include_router(health.router, prefix="/api/v1")
     app.include_router(inventory.router, prefix="/api/v1")
     app.include_router(duplicates.router, prefix="/api/v1")
@@ -75,6 +147,7 @@ def create_app(config: Config | None = None) -> FastAPI:
     app.include_router(renames.router, prefix="/api/v1")
     app.include_router(uploads.router, prefix="/api/v1")
     app.include_router(archives.router, prefix="/api/v1")
+    app.include_router(operations.router, prefix="/api/v1")
     # Static single-page app (hash-routed). Mounted last so /api/v1 wins.
     if FRONTEND_DIR.is_dir():
         app.mount("/app", StaticFiles(directory=FRONTEND_DIR, html=True), name="app")

photo_pipeline/api/routes/analysis.py

@@ -18,7 +18,10 @@ router = APIRouter(tags=["analysis"])
 
 
 def _service(request: Request) -> AnalysisService:
-    return AnalysisService(request.app.state.session_factory)
+    return AnalysisService(
+        request.app.state.session_factory,
+        library_roots=tuple(request.app.state.config.library_roots),
+    )
 
 
 def _error(status: int, code: str, message: str) -> JSONResponse:

photo_pipeline/api/routes/duplicates.py

@@ -11,7 +11,13 @@ from fastapi import APIRouter, Query, Request
 from fastapi.responses import JSONResponse
 
 from photo_pipeline.schemas import DecisionRequest
-from photo_pipeline.services.duplicates import ConflictError, DuplicateError, DuplicateService
+from photo_pipeline.services.duplicates import (
+    MAX_MEMBER_PAGE,
+    MEMBER_PAGE,
+    ConflictError,
+    DuplicateError,
+    DuplicateService,
+)
 
 router = APIRouter(tags=["duplicates"])
 
@@ -42,8 +48,13 @@ def list_clusters(
 
 
 @router.get("/duplicates/clusters/{cluster_id}")
-def get_cluster(cluster_id: str, request: Request):
-    detail = _service(request).get_cluster(cluster_id)
+def get_cluster(
+    cluster_id: str,
+    request: Request,
+    limit: int = Query(MEMBER_PAGE, ge=1, le=MAX_MEMBER_PAGE),
+    offset: int = Query(0, ge=0),
+):
+    detail = _service(request).get_cluster(cluster_id, limit=limit, offset=offset)
     if detail is None:
         return _error(404, "not_found", f"unknown cluster {cluster_id}")
     return detail

photo_pipeline/api/routes/operations.py

@@ -0,0 +1,70 @@
+"""Operational endpoints: diagnostics and backups (US07-05).
+
+Backups can be taken and verified here because both are safe, additive, and the
+operator needs them from the same screen that shows the disk filling up.
+
+**Restore is deliberately not an endpoint.** It replaces the state of the running
+application with an older one, so it belongs to a stopped installation and a person
+at a terminal: ``python -m photo_pipeline restore``. An HTTP call that can silently
+roll the library back to last week is a hole, not a feature.
+"""
+
+from __future__ import annotations
+
+from fastapi import APIRouter, Query, Request
+from fastapi.responses import JSONResponse
+from pydantic import BaseModel
+
+from photo_pipeline.services import diagnostics
+from photo_pipeline.services.backup import DEFAULT_KEEP, BackupError, BackupService
+
+router = APIRouter(tags=["operations"])
+
+
+class CreateBackupRequest(BaseModel):
+    reason: str = "manual"
+    keep: int = DEFAULT_KEEP
+
+
+def _service(request: Request) -> BackupService:
+    return BackupService(request.app.state.config)
+
+
+def _error(status: int, code: str, message: str) -> JSONResponse:
+    return JSONResponse(status_code=status, content={"error": {"code": code, "message": message}})
+
+
+@router.get("/diagnostics")
+def read_diagnostics(request: Request) -> dict:
+    return diagnostics.report(request.app.state.config)
+
+
+@router.get("/backups")
+def list_backups(request: Request) -> dict:
+    return {"backups": _service(request).list()}
+
+
+@router.post("/backups", status_code=201)
+def create_backup(body: CreateBackupRequest, request: Request):
+    try:
+        return _service(request).create(reason=body.reason, keep=body.keep)
+    except BackupError as error:
+        return _error(422, "backup_failed", str(error))
+
+
+@router.get("/backups/{name}/verify")
+def verify_backup(name: str, request: Request):
+    service = _service(request)
+    # The name comes from the browser, so it names a backup — it is never joined
+    # into a path until it has been matched against one that exists (US07-02).
+    if name not in {entry["name"] for entry in service.list()}:
+        return _error(404, "not_found", f"unknown backup {name}")
+    return {"name": name, **service.verify(service.root / name).as_dict()}
+
+
+@router.post("/backups/prune")
+def prune_backups(request: Request, keep: int = Query(DEFAULT_KEEP, ge=1)):
+    try:
+        return {"removed": _service(request).prune(keep=keep)}
+    except BackupError as error:
+        return _error(422, "invalid_retention", str(error))

photo_pipeline/api/routes/session.py

@@ -0,0 +1,68 @@
+"""Session bootstrap: the one endpoint reachable without a session (US07-02).
+
+It sets the ``HttpOnly``/``SameSite=Strict`` session cookie and returns the CSRF
+token in the body. A foreign page can call this — it just cannot read the answer,
+because the app sends no CORS headers — and the cookie it received is never attached
+to a request that foreign page initiated.
+
+When an access secret is configured (mandatory as soon as the app is reachable from
+another machine, US08-01) this is also the authentication gate: the secret buys the
+cookie, and every route behind it keeps asking for exactly the session and CSRF token
+it asked for before. Wrong secrets are counted, and a burst of them stops being
+answered — otherwise a proxy-exposed deployment could be guessed at indefinitely.
+"""
+
+from __future__ import annotations
+
+import logging
+import secrets
+
+from fastapi import APIRouter, Request
+from fastapi.responses import JSONResponse
+
+from photo_pipeline.api.security import ACCESS_SECRET_HEADER, SESSION_COOKIE
+
+router = APIRouter(tags=["session"])
+
+log = logging.getLogger(__name__)
+
+
+def _refuse(status: int, code: str, message: str) -> JSONResponse:
+    return JSONResponse(status_code=status, content={"error": {"code": code, "message": message}})
+
+
+@router.get("/session")
+def start_session(request: Request) -> JSONResponse:
+    config = request.app.state.config
+    secret = config.access_secret
+    if secret is not None:
+        limiter = request.app.state.access_limiter
+        if limiter.blocked():
+            return _refuse(429, "too_many_attempts", "too many failed attempts; retry later")
+        offered = request.headers.get(ACCESS_SECRET_HEADER, "")
+        if not secrets.compare_digest(offered, secret.get_secret_value()):
+            limiter.record_failure()
+            # The client address is the whole record: the offered secret, the issued
+            # session, and the request body all stay out of the log.
+            log.warning(
+                "access secret rejected", extra={"client": _client(request), "path": "/session"}
+            )
+            return _refuse(401, "access_denied", "a valid access secret is required")
+
+    session = request.app.state.session
+    response = JSONResponse({"csrf_token": session.csrf_token})
+    response.set_cookie(
+        SESSION_COOKIE,
+        session.id,
+        httponly=True,
+        samesite="strict",
+        # HTTPS outside means the cookie must never travel over a plain hop, even one
+        # this process cannot see. Loopback http keeps working unchanged.
+        secure=request.scope.get("state", {}).get("external_scheme") == "https",
+        path="/",
+    )
+    return response
+
+
+def _client(request: Request) -> str:
+    return request.client.host if request.client else "unknown"

photo_pipeline/api/routes/thumbnails.py

@@ -31,5 +31,7 @@ def get_thumbnail(asset_id: str, request: Request, size: int = Query(512)):
     return FileResponse(
         path,
         media_type="image/webp",
-        headers={"Cache-Control": "public, max-age=31536000, immutable"},
+        # private: the URL is versioned and immutable, but these bytes are the user's
+        # photos and must never sit in a shared cache (US07-02).
+        headers={"Cache-Control": "private, max-age=31536000, immutable"},
     )

photo_pipeline/api/security.py

@@ -0,0 +1,317 @@
+"""Local-web attack surface: session, CSRF, Origin/Host checks, default headers.
+
+The app binds to 127.0.0.1, so the attacker is not a remote client but another page
+in the user's browser (concept §15, "Local web attack"): any site can issue requests
+to ``http://127.0.0.1:8000`` and can embed ``<img src=...>`` against media endpoints.
+The defenses stack, because each one alone has a hole:
+
+* **Host** must be a loopback name — a DNS rebinding host that resolves to 127.0.0.1
+  passes the browser's origin rules but not this check.
+* **Origin**, when the browser sends one, must be this exact origin (scheme, host,
+  port). There is no CORS middleware at all, so a foreign page can never *read* a
+  response even if it manages to send a request.
+* **Sec-Fetch-Site** rejects cross-site loads that carry no Origin, which is what an
+  ``<img>`` or ``<script>`` against a media endpoint looks like.
+* A **session cookie** (``SameSite=Strict``, ``HttpOnly``) is required by every
+  ``/api/v1`` route except liveness/readiness and the bootstrap itself. Strict means
+  the browser never attaches it to a request another site initiated.
+* A **CSRF token** must be echoed in a header on every mutation. It is handed out
+  only in the bootstrap response body, which a foreign page cannot read (no CORS) —
+  so possessing it proves the caller is same-origin.
+
+Behind a reverse proxy (US08-01) the same stack holds with two substitutions: the
+allowed host set comes from configuration instead of being the loopback names, and
+the host/scheme the policy judges is the *external* one, which is only read from
+``X-Forwarded-*`` when the request actually arrived from a configured proxy. The
+loopback check was standing in for authentication, so naming a non-loopback host
+also makes an access secret mandatory — ``trust_refusal`` refuses to start without
+one, and the secret is what the bootstrap endpoint trades for the session cookie.
+
+``evaluate`` is a pure function over the request metadata: the whole policy is one
+table that a unit test can enumerate, and the middleware only applies its verdict.
+"""
+
+from __future__ import annotations
+
+import secrets
+import time
+from collections.abc import Mapping
+from dataclasses import dataclass
+from urllib.parse import urlsplit
+
+from starlette.datastructures import Headers, MutableHeaders
+from starlette.responses import JSONResponse
+
+SESSION_COOKIE = "pp_session"
+CSRF_HEADER = "x-csrf-token"
+ACCESS_SECRET_HEADER = "x-access-secret"
+API_PREFIX = "/api/v1"
+SAFE_METHODS = frozenset({"GET", "HEAD", "OPTIONS"})
+# Reachable without a session: liveness/readiness (an orchestrator has no cookie)
+# and the bootstrap that issues the session in the first place.
+PUBLIC_PATHS = frozenset(
+    {f"{API_PREFIX}/health/live", f"{API_PREFIX}/health/ready", f"{API_PREFIX}/session"}
+)
+LOOPBACK_HOSTS = frozenset({"127.0.0.1", "localhost", "::1", "[::1]"})
+# Mutating endpoints that must stay reachable while mutation itself is gated: the
+# backup a careful operator takes first, and its retention (US07-07).
+MUTATION_EXEMPT_PATHS = frozenset({f"{API_PREFIX}/backups", f"{API_PREFIX}/backups/prune"})
+
+# Applied to every response. No inline script/style is used by the frontend, so the
+# policy can stay strict; `frame-ancestors 'none'` and CORP keep other pages from
+# embedding the app or its thumbnails.
+DEFAULT_HEADERS = {
+    "x-content-type-options": "nosniff",
+    "x-frame-options": "DENY",
+    "referrer-policy": "no-referrer",
+    "cross-origin-resource-policy": "same-origin",
+    "cross-origin-opener-policy": "same-origin",
+    "content-security-policy": (
+        "default-src 'self'; img-src 'self' data:; style-src 'self'; script-src 'self'; "
+        "connect-src 'self'; frame-ancestors 'none'; base-uri 'none'; form-action 'none'"
+    ),
+}
+
+
+@dataclass(frozen=True)
+class Session:
+    """One process, one session. A local app has exactly one user; a session store
+    would be bookkeeping without a second subject to distinguish.
+
+    ponytail: per-session rows if the app ever serves more than one operator.
+    """
+
+    id: str
+    csrf_token: str
+
+    @classmethod
+    def create(cls) -> Session:
+        return cls(secrets.token_urlsafe(32), secrets.token_urlsafe(32))
+
+
+@dataclass(frozen=True)
+class Refusal:
+    status: int
+    code: str
+    message: str
+
+
+def split_host(value: str) -> tuple[str, str]:
+    """``"127.0.0.1:8000"`` -> ``("127.0.0.1", "8000")``; bracketed IPv6 aware."""
+    value = value.strip()
+    if value.startswith("["):
+        host, _, port = value.partition("]")
+        return host + "]", port.lstrip(":")
+    host, _, port = value.partition(":")
+    return host, port
+
+
+def external_view(
+    *,
+    client: str | None,
+    headers: Mapping[str, str],
+    scheme: str,
+    trusted_proxies: frozenset[str],
+) -> tuple[str, str]:
+    """The ``(scheme, host)`` the caller used, as opposed to the one this hop saw.
+
+    Forwarded headers are a client-supplied claim. Believing them from anyone lets a
+    request declare its own origin — and origin is half of this module's evidence —
+    so they count only when the connection came from a configured proxy.
+    """
+    host = headers.get("host", "")
+    if client is None or client not in trusted_proxies:
+        return scheme, host
+    # A chain appends: the first entry is what the original client asked for.
+    forwarded_proto = headers.get("x-forwarded-proto", "").split(",")[0].strip().lower()
+    forwarded_host = headers.get("x-forwarded-host", "").split(",")[0].strip()
+    return forwarded_proto or scheme, forwarded_host or host
+
+
+def evaluate(
+    *,
+    method: str,
+    path: str,
+    headers: Mapping[str, str],
+    session: Session,
+    allowed_hosts: frozenset[str] = LOOPBACK_HOSTS,
+    scheme: str = "http",
+    max_request_bytes: int,
+) -> Refusal | None:
+    """Why this request must be refused, or ``None`` when it may proceed.
+
+    ``headers["host"]`` and ``scheme`` are the external ones (see ``external_view``);
+    the allowed origins are the allowed hosts under that scheme and port, so there is
+    no second list that can drift away from the first.
+    """
+    host_header = headers.get("host", "")
+    host, port = split_host(host_header)
+    if host.lower() not in allowed_hosts:
+        return Refusal(403, "host_not_allowed", "request host is not an allowed address")
+
+    origin = headers.get("origin")
+    if origin is not None and origin != "":
+        parts = urlsplit(origin)
+        origin_host, origin_port = split_host(parts.netloc)
+        if (
+            parts.scheme != scheme
+            or origin_host.lower() not in allowed_hosts
+            or origin_port != port
+        ):
+            return Refusal(403, "origin_not_allowed", "request origin is not this application")
+
+    # Absent means a non-browser client; "none" is a user-initiated navigation.
+    fetch_site = headers.get("sec-fetch-site")
+    if fetch_site is not None and fetch_site not in ("same-origin", "none"):
+        return Refusal(403, "cross_site_blocked", "cross-site requests are not accepted")
+
+    length = headers.get("content-length")
+    if length and length.isdigit() and int(length) > max_request_bytes:
+        return Refusal(413, "payload_too_large", "request body exceeds the configured limit")
+
+    protected = path.startswith(API_PREFIX) and path not in PUBLIC_PATHS
+    if not protected:
+        return None
+
+    if headers.get("cookie-session") != session.id:
+        return Refusal(401, "unauthenticated", "a valid application session is required")
+    if method.upper() not in SAFE_METHODS and headers.get(CSRF_HEADER) != session.csrf_token:
+        return Refusal(403, "csrf_failed", "missing or invalid CSRF token")
+    return None
+
+
+def exposed_hosts(config) -> list[str]:
+    """Configured names by which this application is reachable from another machine."""
+    names = {str(config.host).lower()}
+    names.update(split_host(name)[0].lower() for name in config.allowed_hosts)
+    return sorted(names - LOOPBACK_HOSTS)
+
+
+def trust_refusal(config) -> str | None:
+    """Why this configuration must not serve at all, or ``None``.
+
+    Reaching the app used to prove ownership of it. The moment a configuration makes
+    it reachable from elsewhere that stops being true, so serving without a secret
+    would publish the library — refuse at startup rather than at the first request,
+    when the operator is no longer watching (US08-01).
+    """
+    exposed = exposed_hosts(config)
+    if exposed and config.access_secret is None:
+        return (
+            f"refusing to serve: {', '.join(exposed)} is reachable from outside this "
+            "machine, so PHOTO_PIPELINE_ACCESS_SECRET must be set"
+        )
+    return None
+
+
+class FailureLimiter:
+    """Bounded failed access-secret attempts, so the secret cannot be guessed online.
+
+    ponytail: one counter for the whole process rather than per client address —
+    behind a proxy every attempt arrives from the same address anyway. Per-caller
+    buckets if the app is ever exposed without one.
+    """
+
+    def __init__(self, limit: int = 5, window: float = 60.0) -> None:
+        self.limit = limit
+        self.window = window
+        self._failures: list[float] = []
+
+    def blocked(self) -> bool:
+        now = time.monotonic()
+        self._failures = [at for at in self._failures if now - at < self.window]
+        return len(self._failures) >= self.limit
+
+    def record_failure(self) -> None:
+        self._failures.append(time.monotonic())
+
+
+class SecurityMiddleware:
+    """Pure-ASGI so the SSE stream keeps streaming (BaseHTTPMiddleware buffers)."""
+
+    def __init__(self, app, *, session: Session, config) -> None:
+        self.app = app
+        self.session = session
+        self.config = config
+        self.max_request_bytes = config.max_request_bytes
+        self.allowed_hosts = frozenset(
+            LOOPBACK_HOSTS
+            | {str(config.host).lower()}
+            | {split_host(name)[0].lower() for name in config.allowed_hosts}
+        )
+        self.trusted_proxies = frozenset(config.trusted_proxies)
+
+    async def __call__(self, scope, receive, send) -> None:
+        if scope["type"] != "http":
+            await self.app(scope, receive, send)
+            return
+        headers = Headers(scope=scope)
+        # The cookie is read here and handed to the pure policy as one value, so the
+        # policy never has to parse a Cookie header.
+        lookup = dict(headers)
+        lookup["cookie-session"] = _cookie(headers.get("cookie", ""), SESSION_COOKIE)
+        client = scope.get("client")
+        scheme, lookup["host"] = external_view(
+            client=client[0] if client else None,
+            headers=headers,
+            scheme=scope.get("scheme", "http"),
+            trusted_proxies=self.trusted_proxies,
+        )
+        # What the session cookie's Secure flag is decided from, one hop later.
+        scope.setdefault("state", {})["external_scheme"] = scheme
+        refusal = evaluate(
+            method=scope.get("method", "GET"),
+            path=scope.get("path", "/"),
+            headers=lookup,
+            session=self.session,
+            allowed_hosts=self.allowed_hosts,
+            scheme=scheme,
+            max_request_bytes=self.max_request_bytes,
+        )
+        if refusal is None:
+            refusal = self._mutation_refusal(scope)
+        if refusal is not None:
+            response = JSONResponse(
+                status_code=refusal.status,
+                content={"error": {"code": refusal.code, "message": refusal.message}},
+                headers=DEFAULT_HEADERS,
+            )
+            await response(scope, receive, send)
+            return
+
+        async def send_with_headers(message):
+            if message["type"] == "http.response.start":
+                out = MutableHeaders(scope=message)
+                for name, value in DEFAULT_HEADERS.items():
+                    out.setdefault(name, value)
+            await send(message)
+
+        await self.app(scope, receive, send_with_headers)
+
+    def _mutation_refusal(self, scope) -> Refusal | None:
+        """Refuse every mutating request while the library's dry run is unapproved.
+
+        One choke point for the whole API: every mutation the browser can start is a
+        non-safe method under ``/api/v1``. Reading stays open — an operator has to be
+        able to look at what the application found in order to approve it (US07-07).
+        """
+        method = scope.get("method", "GET").upper()
+        path = scope.get("path", "/")
+        if method in SAFE_METHODS or not path.startswith(API_PREFIX):
+            return None
+        if path in MUTATION_EXEMPT_PATHS:
+            return None
+        from photo_pipeline.services.release import mutation_blockers
+
+        blockers = mutation_blockers(self.config)
+        if not blockers:
+            return None
+        return Refusal(403, blockers[0]["code"], blockers[0]["message"])
+
+
+def _cookie(header: str, name: str) -> str:
+    for part in header.split(";"):
+        key, _, value = part.strip().partition("=")
+        if key == name:
+            return value
+    return ""

photo_pipeline/config.py

@@ -7,6 +7,9 @@ real external call needs them.
 
 pydantic-settings would do this too, but a prefix-scan over the declared fields
 is a few lines and one fewer dependency.
+
+Tuple-valued settings are lists in one variable: library roots are ``os.pathsep``
+separated because they are paths, everything else is comma separated.
 """
 
 from __future__ import annotations
@@ -18,6 +21,62 @@ from typing import Mapping
 from pydantic import BaseModel, ConfigDict, SecretStr
 
 ENV_PREFIX = "PHOTO_PIPELINE_"
+ENV_FILE_VAR = f"{ENV_PREFIX}ENV_FILE"
+DEFAULT_ENV_FILE = Path(".env")
+COMMA_LIST_FIELDS = frozenset({"allowed_hosts", "trusted_proxies"})
+
+# The archived CLI's variable names, so the configuration file an operator already
+# has keeps working. The vision provider reads the OpenAI SDK's names, and the
+# library root is configuration here rather than a bare path (US07-01 donor).
+LEGACY_ALIASES = {
+    "LLM_API_KEY": "OPENAI_API_KEY",
+    "GEMINI_API_KEY": "OPENAI_API_KEY",
+    "LLM_BASE_URL": "OPENAI_BASE_URL",
+    "LIBRARY": f"{ENV_PREFIX}LIBRARY_ROOTS",
+}
+
+
+def parse_env_file(text: str) -> dict[str, str]:
+    """``KEY=value`` lines into a mapping. Comments, blanks, and quotes handled.
+
+    Deliberately not a shell: no interpolation, no ``export``, no multi-line values.
+    A configuration file that can run code is a configuration file that can be a
+    vulnerability.
+    """
+    values: dict[str, str] = {}
+    for line in text.splitlines():
+        line = line.strip()
+        if not line or line.startswith("#") or "=" not in line:
+            continue
+        key, _, raw = line.partition("=")
+        key = key.strip()
+        if not key or key.startswith("#"):
+            continue
+        value = raw.strip().strip('"').strip("'")
+        values[key] = value
+        alias = LEGACY_ALIASES.get(key)
+        if alias:
+            values.setdefault(alias, value)
+    return values
+
+
+def load_env_file(path: Path | str | None = None) -> dict[str, str]:
+    """Load ``PHOTO_PIPELINE_ENV_FILE`` (or ``./.env``) into the environment.
+
+    Anything already exported wins: a file is the standing configuration, the shell
+    is what you meant *this time*. Returns what it applied, which is what the CLI
+    prints — names only, never values.
+    """
+    candidate = path or os.environ.get(ENV_FILE_VAR) or DEFAULT_ENV_FILE
+    candidate = Path(candidate)
+    if not candidate.is_file():
+        return {}
+    applied = {}
+    for key, value in parse_env_file(candidate.read_text()).items():
+        if key not in os.environ:
+            os.environ[key] = value
+            applied[key] = value
+    return applied
 
 
 class Config(BaseModel):
@@ -30,6 +89,23 @@ class Config(BaseModel):
     log_level: str = "INFO"
     log_format: str = "json"  # "json" or "text"
 
+    # Trust boundary (US08-01). Empty means loopback only, which is what the app did
+    # before there was a setting: a request whose Host is not a loopback name is
+    # refused, and no secret is needed because nothing outside this machine can call.
+    # Naming a real hostname here is what makes the app reachable through a reverse
+    # proxy, and it is exactly then that ``access_secret`` becomes mandatory.
+    allowed_hosts: tuple[str, ...] = ()
+    # Addresses whose ``X-Forwarded-Proto``/``X-Forwarded-Host`` may be believed. A
+    # client that is not the proxy can otherwise declare its own origin.
+    trusted_proxies: tuple[str, ...] = ()
+    # Exchanged for the session cookie at the bootstrap endpoint. Once set it is
+    # required even on loopback, so a development setup cannot half-enable it.
+    access_secret: SecretStr | None = None
+
+    # Largest request body the API accepts. Every endpoint takes small JSON commands;
+    # anything larger is a mistake or an attempt to exhaust memory (US07-02).
+    max_request_bytes: int = 1_048_576
+
     # Library boundary for path validation (os.pathsep-separated in the env var).
     library_roots: tuple[Path, ...] = ()
     thumbnail_cache_quota_bytes: int = 500_000_000
@@ -38,6 +114,12 @@ class Config(BaseModel):
     # Free space an archive destination must keep beyond the transfer itself.
     archive_free_space_reserve_bytes: int = 1_000_000_000
 
+    # Refuse every mutating request until a read-only dry run of the configured
+    # library has been produced and explicitly approved (US07-07). Off by default so
+    # a development setup is unchanged; turn it on before pointing the application at
+    # a library whose photos cannot be replaced.
+    require_dry_run_approval: bool = False
+
     vision_api_key: SecretStr | None = None
     immich_api_key: SecretStr | None = None
     immich_server_url: str = ""
@@ -58,10 +140,18 @@ class Config(BaseModel):
     @classmethod
     def from_env(cls, environ: Mapping[str, str] | None = None) -> "Config":
         env = os.environ if environ is None else environ
+        if environ is None:
+            load_env_file()  # a file never overrides what the shell already set
+            env = os.environ
         data: dict = {}
         for name in cls.model_fields:
             raw = env.get(ENV_PREFIX + name.upper())
             if not raw:
                 continue
-            data[name] = raw.split(os.pathsep) if name == "library_roots" else raw
+            if name == "library_roots":
+                data[name] = raw.split(os.pathsep)
+            elif name in COMMA_LIST_FIELDS:
+                data[name] = [part.strip() for part in raw.split(",") if part.strip()]
+            else:
+                data[name] = raw
         return cls(**data)

photo_pipeline/db.py

@@ -41,12 +41,41 @@ def create_session_factory(engine: Engine) -> sessionmaker:
     return sessionmaker(bind=engine, expire_on_commit=False, future=True)
 
 
-def run_migrations(url: str) -> None:
-    """Upgrade the database at ``url`` to the latest revision."""
-    from alembic import command
+def _alembic_config(url: str):
     from alembic.config import Config as AlembicConfig
 
     cfg = AlembicConfig(str(_REPO_ROOT / "alembic.ini"))
     cfg.set_main_option("script_location", str(_REPO_ROOT / "migrations"))
     cfg.set_main_option("sqlalchemy.url", url)
-    command.upgrade(cfg, "head")
+    return cfg
+
+
+def run_migrations(url: str) -> None:
+    """Upgrade the database at ``url`` to the latest revision."""
+    from alembic import command
+
+    command.upgrade(_alembic_config(url), "head")
+
+
+def head_revision() -> str | None:
+    """The revision this code expects. ``None`` if the scripts cannot be read."""
+    from alembic.script import ScriptDirectory
+
+    try:
+        return ScriptDirectory.from_config(_alembic_config("sqlite://")).get_current_head()
+    except Exception:
+        return None
+
+
+def current_revision(url: str) -> str | None:
+    """The revision a database is actually at, or ``None`` for an unstamped one."""
+    engine = create_db_engine(url)
+    try:
+        with engine.connect() as connection:
+            from alembic.runtime.migration import MigrationContext
+
+            return MigrationContext.configure(connection).get_current_revision()
+    except Exception:
+        return None
+    finally:
+        engine.dispose()

photo_pipeline/faults.py

@@ -0,0 +1,45 @@
+"""Test-only fault control points (concept §18, US07-04).
+
+Crash safety can only be proven by crashing at the exact moment a transition has
+been persisted but its consequence has not. That needs a barrier *inside* the
+production code path — but not a production capability: there is no endpoint, no
+service method, and no configuration file entry that can trigger one. The only
+switch is an environment variable naming a single point, read at the moment it is
+passed, and the only thing it does is kill the process. A deployment that never
+sets it can never reach the barrier.
+
+``os._exit`` is deliberate: it skips atexit handlers, buffered flushes, and
+``finally`` blocks, which is what a real ``SIGKILL`` or power loss does. A clean
+shutdown would prove nothing.
+
+The points are the persisted transitions of the journalled stages:
+
+    rename    moving | moved | database_updated | verified | complete
+    archive   transferring | verified | removing | source_removed | complete
+    exif      exif:written        — keywords on disk, checkpoint not yet recorded
+    upload    upload:accepted     — uploader exited, outcome not yet persisted
+    jobs      job:item_done       — item committed, job outcome not yet written
+
+Recovery for each is asserted in tests/integration/test_fault_matrix.py and
+tests/e2e/test_crash_recovery.py.
+"""
+
+from __future__ import annotations
+
+import os
+
+ENV_VAR = "PHOTO_PIPELINE_FAULT_AFTER"
+
+EXIF_WRITTEN = "exif:written"
+UPLOAD_ACCEPTED = "upload:accepted"
+JOB_ITEM_DONE = "job:item_done"
+
+
+def maybe_fault(point: str) -> None:
+    """Die abruptly when ``PHOTO_PIPELINE_FAULT_AFTER`` names ``point``.
+
+    Shared by the rename, archive, restore, EXIF, upload, and job lanes, each
+    passing its own state names. Never set the variable outside tests.
+    """
+    if os.environ.get(ENV_VAR) == point:
+        os._exit(9)

photo_pipeline/imaging.py

@@ -0,0 +1,119 @@
+"""Bounded, defensive image decoding — one door for every pixel this app reads.
+
+A photo library contains files nobody planned for: truncated downloads, zero-byte
+placeholders, a PNG whose header claims 200000×200000, a TIFF with a broken ICC
+profile, an extension that lies about its content. None of them may take down a
+request or a worker, and none may decode more pixels than the caller allowed
+(concept §17: decoded pixels, not file size, are what exhausts memory).
+
+``open_image`` is that single door:
+
+* the declared dimensions are checked **before** a pixel is decoded;
+* Pillow's decompression-bomb *warning* is promoted to an error, because the
+  warning band (between Pillow's limit and twice it) still decodes the image;
+* every decoder failure — at open time or during the caller's decode — becomes one
+  of two typed errors, so callers map them to their own item state instead of
+  catching ``Exception``;
+* error text names no path: it reaches API responses, and the full reason goes to
+  the server log instead (US07-02).
+
+``to_srgb`` and ``draft`` are the other two bounded-decode helpers: colour-manage a
+profile-bearing image into sRGB, and let JPEG decode straight to a size near the
+requested one rather than at full resolution.
+"""
+
+from __future__ import annotations
+
+import io
+import logging
+import warnings
+from contextlib import contextmanager
+from pathlib import Path
+
+from PIL import Image, ImageCms, ImageFile, UnidentifiedImageError
+
+log = logging.getLogger(__name__)
+
+# Matches ``Config.thumbnail_max_pixels``; used where no configuration is at hand
+# (hashing runs inside discovery, which takes no config).
+DEFAULT_MAX_PIXELS = 100_000_000
+
+
+class MediaError(Exception):
+    """A file could not be turned into pixels safely."""
+
+
+class UndecodableImage(MediaError):
+    """Corrupt, truncated, empty, or not an image at all."""
+
+
+class ImageTooLarge(MediaError):
+    """More pixels than this operation is allowed to decode."""
+
+
+@contextmanager
+def open_image(path: Path | str, *, max_pixels: int = DEFAULT_MAX_PIXELS):
+    """Yield an open :class:`PIL.Image.Image`, bounded and with typed failures.
+
+    Decoder errors raised inside the ``with`` body are translated too — a truncated
+    JPEG only fails when its pixels are actually pulled, which is the caller's line,
+    not this one.
+    """
+    # Pillow's truncation tolerance is a process-global switch that any library in
+    # the process can flip (the donor CLI did). This door decides the policy for its
+    # own callers: half a file is not a picture.
+    tolerated = ImageFile.LOAD_TRUNCATED_IMAGES
+    ImageFile.LOAD_TRUNCATED_IMAGES = False
+    with warnings.catch_warnings():
+        # The warning band is not a warning for us: it means Pillow was willing to
+        # decode an image large enough to be a denial-of-service.
+        warnings.simplefilter("error", Image.DecompressionBombWarning)
+        try:
+            with Image.open(path) as image:
+                width, height = image.size
+                if width * height > max_pixels:
+                    raise ImageTooLarge(f"{width}x{height} exceeds the {max_pixels} pixel limit")
+                yield image
+        except MediaError:
+            raise
+        except (Image.DecompressionBombError, Image.DecompressionBombWarning) as error:
+            log.info("refused oversized image %s: %s", path, error)
+            raise ImageTooLarge("image exceeds the decompression-bomb limit") from error
+        except (UnidentifiedImageError, OSError, ValueError, SyntaxError, MemoryError) as error:
+            log.info("cannot decode %s: %s", path, error)
+            raise UndecodableImage(f"cannot decode image ({type(error).__name__})") from error
+        finally:
+            ImageFile.LOAD_TRUNCATED_IMAGES = tolerated
+
+
+def draft(image: Image.Image, size: int) -> None:
+    """Ask the decoder for a smaller image where the format allows it (JPEG).
+
+    This is the difference between decoding a 40-megapixel JPEG and decoding the
+    roughly 1-megapixel version a 1280px preview needs.
+    """
+    try:
+        image.draft(None, (size, size))
+    except (AttributeError, ValueError, OSError):  # not a draft-capable format
+        pass
+
+
+def to_srgb(image: Image.Image, *, mode: str) -> Image.Image:
+    """Convert into ``mode``, colour-managing through an embedded ICC profile.
+
+    Without this a wide-gamut original renders with visibly wrong colours, because
+    its numbers are interpreted as sRGB. A broken or unreadable profile is not a
+    reason to fail a preview — the plain conversion is still a correct picture.
+    """
+    profile = image.info.get("icc_profile")
+    if profile:
+        try:
+            return ImageCms.profileToProfile(
+                image,
+                ImageCms.ImageCmsProfile(io.BytesIO(profile)),
+                ImageCms.createProfile("sRGB"),
+                outputMode=mode,
+            )
+        except Exception as error:  # noqa: BLE001 - any ICC failure falls back
+            log.info("ignoring unusable ICC profile on %s: %s", getattr(image, "filename", "?"), error)
+    return image.convert(mode)

photo_pipeline/integrations/exiftool.py

@@ -17,6 +17,20 @@ import os
 import subprocess
 from collections.abc import Iterable
 
+# A hung exiftool must not hang the worker with it: every call is bounded, and a
+# call that runs out of time is treated exactly like a failed one — no metadata
+# answer, nothing marked verified (US07-04). The knob exists because "slow" is a
+# property of the machine, not of the code: huge files on a slow network volume
+# legitimately take longer than the default.
+DEFAULT_TIMEOUT_SECONDS = 120.0
+
+
+def _timeout() -> float:
+    try:
+        return float(os.environ.get("PHOTO_PIPELINE_EXIFTOOL_TIMEOUT", DEFAULT_TIMEOUT_SECONDS))
+    except ValueError:
+        return DEFAULT_TIMEOUT_SECONDS
+
 
 def read_keyword_sets(paths: Iterable[str]) -> dict[str, set[str]]:
     """Map each path to its lowercased set of ``Keywords`` + ``Subject`` values.
@@ -34,8 +48,9 @@ def read_keyword_sets(paths: Iterable[str]) -> dict[str, set[str]]:
             input="\n".join(paths),
             capture_output=True,
             text=True,
+            timeout=_timeout(),
         )
-    except FileNotFoundError:
+    except (FileNotFoundError, subprocess.TimeoutExpired):
         return {}
     out: dict[str, set[str]] = {}
     try:
@@ -56,6 +71,33 @@ def read_keyword_sets(paths: Iterable[str]) -> dict[str, set[str]]:
     return out
 
 
+def read_all(path: str) -> dict | None:
+    """Every tag exiftool can read from ``path``, or ``None`` when it cannot answer.
+
+    This is the snapshot an EXIF checkpoint compares against: proving that a write
+    preserved the fields it does not own requires knowing all of them, not just the
+    ones being written (US07-03). ``None`` (exiftool missing, unreadable file,
+    unparsable output) is not an empty snapshot — a caller must not read it as
+    "nothing was there".
+    """
+    try:
+        result = subprocess.run(
+            ["exiftool", "-m", "-j", "-G0:1", path],
+            capture_output=True,
+            text=True,
+            timeout=_timeout(),
+        )
+    except (FileNotFoundError, subprocess.TimeoutExpired):
+        return None
+    try:
+        records = json.loads(result.stdout or "[]")
+    except ValueError:
+        return None
+    if not records:
+        return None
+    return {k: v for k, v in records[0].items() if k != "SourceFile"}
+
+
 def apply_keywords(path: str, *, add: Iterable[str] = (), remove: Iterable[str] = ()) -> bool:
     """Idempotently add/remove keywords in Keywords + Subject; preserve all else."""
     args = ["exiftool", "-m", "-overwrite_original"]
@@ -67,4 +109,10 @@ def apply_keywords(path: str, *, add: Iterable[str] = (), remove: Iterable[str]
     if len(args) == 3:
         return True
     args.append(path)
-    return subprocess.run(args, capture_output=True, text=True).returncode == 0
+    try:
+        return subprocess.run(
+            args, capture_output=True, text=True, timeout=_timeout()
+        ).returncode == 0
+    except (FileNotFoundError, subprocess.TimeoutExpired):
+        # A write that never returned is not a write that succeeded.
+        return False

photo_pipeline/integrations/nsfw_model.py

@@ -15,6 +15,8 @@ from __future__ import annotations
 
 from pathlib import Path
 
+from photo_pipeline import imaging
+
 MODEL_ID = "AdamCodd/vit-base-nsfw-detector"
 BATCH = 16
 
@@ -54,13 +56,17 @@ class NsfwModel:
         self._ensure_loaded()
         import numpy as np
         import torch
-        from PIL import Image, ImageFile
+        from PIL import Image
 
-        ImageFile.LOAD_TRUNCATED_IMAGES = True
-
-        def preprocess(image):
-            image = image.convert("RGB").resize((self._size, self._size), Image.BILINEAR)
-            array = (np.asarray(image, dtype="float32") / 255.0 - 0.5) / 0.5
+        # The donor set ``ImageFile.LOAD_TRUNCATED_IMAGES = True`` here. That flag is
+        # process-global: in this application the same process also hashes files and
+        # renders previews, and those must keep failing loudly on a truncated file
+        # rather than quietly working on half of one (US07-03). An unreadable image
+        # is skipped instead — it stays unscored, and therefore visibly undecided.
+        def preprocess(path):
+            with imaging.open_image(path) as image:
+                small = image.convert("RGB").resize((self._size, self._size), Image.BILINEAR)
+            array = (np.asarray(small, dtype="float32") / 255.0 - 0.5) / 0.5
             return torch.from_numpy(array).permute(2, 0, 1)
 
         results: list[tuple[str, float]] = []
@@ -69,9 +75,10 @@ class NsfwModel:
             tensors, batch_paths = [], []
             for path in items[start : start + self.batch]:
                 try:
-                    tensors.append(preprocess(Image.open(path)))
+                    tensors.append(preprocess(path))
                     batch_paths.append(path)
-                except Exception:
+                except (imaging.MediaError, OSError, ValueError):
+                    # One bad file must not cost the batch its other fifteen.
                     continue
             if not tensors:
                 continue

photo_pipeline/jobs/domain_handlers.py

@@ -38,7 +38,8 @@ def _safety_score_item(asset_id: str, ctx: JobContext) -> None:
 def _analysis_item(asset_id: str, ctx: JobContext) -> None:
     from photo_pipeline.services.analysis import AnalysisService
 
-    AnalysisService(ctx.session_factory).run([asset_id])
+    roots = tuple(getattr(ctx.config, "library_roots", ()) or ())
+    AnalysisService(ctx.session_factory, library_roots=roots).run([asset_id])
 
 
 def _upload_batch_item(batch_id: str, ctx: JobContext) -> None:

photo_pipeline/jobs/worker.py

@@ -16,6 +16,7 @@ from collections.abc import Mapping, Sequence
 from sqlalchemy import select
 from sqlalchemy.orm import sessionmaker
 
+from photo_pipeline.faults import JOB_ITEM_DONE, maybe_fault
 from photo_pipeline.jobs.handlers import REGISTRY, Cancelled, Handler, JobContext
 from photo_pipeline.models import JobItem
 from photo_pipeline.services.jobs import ItemState, JobConflict, JobService, JobState
@@ -86,6 +87,9 @@ class Worker:
                 )
             else:
                 self.service.set_item(job_id, item_key, ItemState.SUCCEEDED, fencing_token=token)
+                # One item is durably done, the job outcome is not: the control point
+                # for a crash mid-batch (US07-04). Recovery must not re-run this item.
+                maybe_fault(JOB_ITEM_DONE)
             self.service.heartbeat(job_id, self.worker_id, lease_seconds=self.lease_seconds)
 
         self._finalize(job_id, token, cancelled=cancelled, any_failed=any_failed)

photo_pipeline/models/__init__.py

@@ -7,6 +7,7 @@ Alembic environment relies on.
 from photo_pipeline.models.albums import AlbumProposal
 from photo_pipeline.models.archives import ArchiveLocation, ArchiveOperation, ArchivePlan
 from photo_pipeline.models.assets import Asset, AssetPath
+from photo_pipeline.models.exif import ExifProjection
 from photo_pipeline.models.duplicates import (
     DuplicateCluster,
     DuplicateMember,
@@ -28,6 +29,7 @@ __all__ = [
     "DuplicateCluster",
     "DuplicateMember",
     "DuplicateNegativeLink",
+    "ExifProjection",
     "Job",
     "JobItem",
     "JobEvent",

photo_pipeline/models/exif.py

@@ -0,0 +1,33 @@
+"""The durable EXIF projection per asset and stage (concept §3, US07-03).
+
+One row per ``(asset_id, stage)``: what the stage wanted written, what the file
+looked like afterwards, and whether anything outside the stage's ownership moved.
+``state = divergent`` is the whole point of the table — it survives restarts, keeps
+the asset out of stages that require verified metadata, and gives a human something
+to look at instead of a silent repair.
+"""
+
+from __future__ import annotations
+
+from datetime import datetime
+
+from sqlalchemy import DateTime, ForeignKey, Integer, String
+from sqlalchemy.orm import Mapped, mapped_column
+
+from photo_pipeline.db import Base
+
+
+class ExifProjection(Base):
+    __tablename__ = "exif_projections"
+
+    asset_id: Mapped[str] = mapped_column(ForeignKey("assets.id"), primary_key=True)
+    stage: Mapped[str] = mapped_column(String, primary_key=True)  # safety | analysis
+    id: Mapped[str] = mapped_column(String, nullable=False)
+    projection_version: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
+    desired_json: Mapped[str | None] = mapped_column(String)
+    divergent_fields: Mapped[str | None] = mapped_column(String)  # JSON array
+    result_file_sha256: Mapped[str | None] = mapped_column(String)
+    state: Mapped[str] = mapped_column(String, nullable=False)  # verified|divergent|failed
+    error_code: Mapped[str | None] = mapped_column(String)
+    verified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
+    updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))

photo_pipeline/path_policy.py

@@ -49,6 +49,29 @@ def resolve_within(root: Path, path: os.PathLike | str) -> Path:
     return resolved
 
 
+def resolve_in_roots(roots: Iterable[os.PathLike | str], path: os.PathLike | str) -> Path:
+    """The resolved path, proven to be inside one of ``roots`` and not excluded.
+
+    Callers must use the **returned** path for whatever they do next: validating one
+    name and then opening another is the symlink race this exists to close (US07-02).
+    The message names no path — it reaches API responses.
+
+    With no roots configured there is no boundary to check; that is a property of the
+    configuration, not permission granted to this call.
+    """
+    if is_excluded(path):
+        raise PathPolicyError("path is inside an excluded (_IGNORE/) tree")
+    roots = list(roots)
+    if not roots:
+        return Path(path)
+    for root in roots:
+        try:
+            return resolve_within(Path(root), path)
+        except PathPolicyError:
+            continue
+    raise PathPolicyError("path is outside the configured library roots")
+
+
 def iter_supported_files(root: os.PathLike | str) -> Iterator[Path]:
     """Yield supported, non-excluded files under ``root`` in deterministic order.
 

photo_pipeline/services/analysis.py

@@ -18,17 +18,16 @@ from __future__ import annotations
 
 import json
 import os
-import uuid
 from datetime import datetime, timezone
 from typing import Protocol
 
 from sqlalchemy import func, select
 from sqlalchemy.orm import sessionmaker
 
-from photo_pipeline.integrations import exiftool
-from photo_pipeline.models import AnalysisResult, Asset, SafetyReview
-from photo_pipeline.services import hashing
-from photo_pipeline.services.safety import SFW
+from photo_pipeline import path_policy
+from photo_pipeline.models import AnalysisResult, Asset
+from photo_pipeline.services import exif_checkpoint
+from photo_pipeline.services.safety import SFW, latest_reviews
 
 MODEL = "gemini-2.5-flash"
 PROMPT_VERSION = "1"
@@ -59,17 +58,44 @@ def _now() -> datetime:
 
 
 class AnalysisService:
-    def __init__(self, session_factory: sessionmaker, *, provider: VisionProvider | None = None) -> None:
+    def __init__(
+        self,
+        session_factory: sessionmaker,
+        *,
+        provider: VisionProvider | None = None,
+        library_roots: tuple = (),
+    ) -> None:
         self._session_factory = session_factory
         self._provider = provider
+        self._roots = tuple(library_roots)
 
     def _sfw_asset_ids(self, session) -> set[str]:
         """Asset ids whose latest safety decision is ``sfw`` — the ONLY assets that
-        may reach the provider."""
-        latest: dict[str, str | None] = {}
-        for review in session.scalars(select(SafetyReview).order_by(SafetyReview.created_at)):
-            latest[review.asset_id] = review.decision
-        return {aid for aid, decision in latest.items() if decision == SFW}
+        may reach the provider.
+
+        The "latest row wins" rule is applied in SQL (US07-06); loading every review
+        to fold it in Python made the gate cost grow with the review history rather
+        than with the work being gated.
+        """
+        latest = latest_reviews().subquery()
+        return set(
+            session.scalars(select(latest.c.asset_id).where(latest.c.decision == SFW))
+        )
+
+    def _sfw_count(self, session) -> int:
+        """How many assets the gate currently allows, without listing them."""
+        latest = latest_reviews().subquery()
+        return int(
+            session.scalar(
+                select(func.count()).select_from(latest).where(latest.c.decision == SFW)
+            )
+            or 0
+        )
+
+    def _is_still_sfw(self, asset_id: str) -> bool:
+        """Re-read the current safety decision straight from the database."""
+        with self._session_factory() as session:
+            return asset_id in self._sfw_asset_ids(session)
 
     def eligible_asset_ids(self) -> list[str]:
         """Confirmed-SFW canonical active assets without a completed analysis."""
@@ -92,9 +118,12 @@ class AnalysisService:
             )
             return [a.id for a in assets if a.id not in done]
 
-    def counts(self) -> dict[str, int]:
+    def counts(self, *, eligible: int | None = None) -> dict[str, int]:
+        """Analysis progress. ``eligible`` may be passed by a caller that has just
+        counted confirmed-SFW assets, so the workflow home does not resolve the
+        latest decision of every asset twice on one page load (US07-06)."""
         with self._session_factory() as session:
-            sfw = self._sfw_asset_ids(session)
+            eligible = self._sfw_count(session) if eligible is None else eligible
             rows = dict(
                 session.execute(
                     select(AnalysisResult.status, func.count()).group_by(AnalysisResult.status)
@@ -103,10 +132,10 @@ class AnalysisService:
             analyzed = int(rows.get("analyzed", 0))
             errored = int(rows.get("error", 0))
             return {
-                "eligible": len(sfw),
+                "eligible": eligible,
                 "analyzed": analyzed,
                 "error": errored,
-                "pending": max(len(sfw) - analyzed - errored, 0),
+                "pending": max(eligible - analyzed - errored, 0),
             }
 
     def run(self, asset_ids: list[str] | None = None) -> dict:
@@ -136,12 +165,40 @@ class AnalysisService:
             if not path:
                 skipped += 1
                 continue
+            # Second gate, at the moment of use: the database says where the file
+            # was, the filesystem decides what that name means now. A link swapped
+            # under an asset after the scan would otherwise send bytes from outside
+            # the library — the one place that leaves this machine (US07-02).
+            try:
+                path = str(path_policy.resolve_in_roots(self._roots, path))
+            except path_policy.PathPolicyError as error:
+                self._store(
+                    asset_id, status="error", result=None, error=str(error), tokens=0, raw=""
+                )
+                errors += 1
+                continue
             try:
                 result = provider.analyze(path, album_hint=_album_hint(path))
             except Exception as error:  # provider/validation failure is per-asset
                 self._store(asset_id, status="error", result=None, error=str(error), tokens=0, raw="")
                 errors += 1
                 continue
+            # Third gate, after the call: a provider request takes seconds, and the
+            # reviewer may have flipped this asset to NSFW while it was in flight.
+            # The result describes an asset that is no longer analysable, so it is
+            # discarded — not stored, and above all not written into its EXIF
+            # (concept §18 scenario 7, US07-04).
+            if not self._is_still_sfw(asset_id):
+                self._store(
+                    asset_id,
+                    status="skipped_nsfw",
+                    result=None,
+                    error="the safety decision changed while analysis was in flight",
+                    tokens=0,
+                    raw="",
+                )
+                skipped += 1
+                continue
             self._store(
                 asset_id,
                 status="analyzed",
@@ -150,7 +207,7 @@ class AnalysisService:
                 tokens=int(result.get("_tokens", 0)) if isinstance(result, dict) else 0,
                 raw=json.dumps(result, ensure_ascii=False),
             )
-            _write_analysis_exif(path, result)
+            self._write_analysis_exif(asset_id, path, result)
             analyzed += 1
         return {"analyzed": analyzed, "skipped": skipped, "errors": errors}
 
@@ -175,10 +232,45 @@ class AnalysisService:
                 row.location_hint = result.get("location_hint")
                 row.approx_year = result.get("approx_year")
                 row.analyzed_at = now
-                row.exif_written_at = now
             session.add(row)
             session.commit()
 
+    def _write_analysis_exif(self, asset_id: str, path: str, result: dict) -> None:
+        """The analysis EXIF checkpoint: additive keywords, then prove the rest held.
+
+        Additive by design — safety keywords and the user's own keywords are merged
+        with, never replaced (concept §3). ``exif_written_at`` is set only when the
+        read-back verified both the new keywords and every field this stage does not
+        own; a divergent result is recorded and left for a human (US07-03).
+        ponytail: the managed ``AI:`` caption segment, once captions are owned here.
+        """
+        tags = tuple(str(tag) for tag in (result.get("tags") or []))
+        if not tags:
+            return
+        checkpoint = exif_checkpoint.run(path, add=tags)
+        exif_checkpoint.record(
+            self._session_factory,
+            asset_id=asset_id,
+            stage="analysis",
+            result=checkpoint,
+            add=tags,
+        )
+        if not checkpoint.verified:
+            return
+        with self._session_factory() as session:
+            row = session.get(AnalysisResult, asset_id)
+            if row is not None:
+                row.exif_written_at = checkpoint.verified_at
+            asset = session.get(Asset, asset_id)
+            if asset is not None and checkpoint.sha256:
+                # The bytes changed when the container was rewritten; upload must use
+                # the hash of what is actually on disk now (concept §3), and the
+                # recorded size has to move with it (US07-07).
+                asset.current_sha256 = checkpoint.sha256
+                if checkpoint.byte_size is not None:
+                    asset.byte_size = checkpoint.byte_size
+            session.commit()
+
     def get(self, asset_id: str) -> dict | None:
         with self._session_factory() as session:
             row = session.get(AnalysisResult, asset_id)
@@ -191,17 +283,6 @@ def _album_hint(path: str) -> str:
     return Path(path).parent.name
 
 
-def _write_analysis_exif(path: str, result: dict) -> None:
-    """Additive analysis keywords into EXIF (Keywords/Subject), preserving safety
-    and user keywords. The donor also wrote a managed caption; only keywords are
-    written here via the shared adapter.
-    ponytail: add the managed ``AI:`` caption segment + read-back preservation check
-    when the analysis EXIF checkpoint is hardened."""
-    tags = result.get("tags") if isinstance(result, dict) else None
-    if tags:
-        exiftool.apply_keywords(path, add=[str(t) for t in tags])
-
-
 def _result_dict(row: AnalysisResult) -> dict:
     data = {field: getattr(row, field) for field in RESULT_FIELDS}
     data["tags"] = json.loads(row.tags) if row.tags else []

photo_pipeline/services/app_lock.py

@@ -0,0 +1,221 @@
+"""Library-level process lock, and detection of an incompatible legacy run
+(US07-05, concept §15 "migration and operational risks").
+
+Every safety this application has — durable job leases, rename journals, archive
+manifests — assumes that one installation owns the library. Two workers, or the
+frozen CLI running beside the app, break that assumption *below* the level those
+mechanisms can see: the second process simply does not know the first one's
+database exists.
+
+So mutation requires a file lock in the data directory, shaped as JSON so any
+future or migrated entry point can read and honour it without importing this
+package:
+
+    {"lock_version": 1, "role": "worker", "pid": 4242, "host": "...",
+     "started_at": "...", "library_roots": ["..."]}
+
+One holder per role: an API and a worker are designed to run together, a second
+worker is not. A lock whose process is gone is stale and is taken over with the
+takeover recorded — refusing to start because of a crashed predecessor would turn
+one outage into two.
+
+Legacy detection is deliberately a heuristic, not a promise: the archived CLI has
+no lock of its own, so what can be observed is its state files being written right
+now. Recent writes to them mean something else is mutating this library, and every
+mutating stage should refuse until it stops.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import socket
+from dataclasses import dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+
+from photo_pipeline.config import Config
+
+LOCK_VERSION = 1
+LOCK_SUFFIX = ".lock.json"
+# State files only the archived CLIs write. Their presence is history; a *recent*
+# modification is a running process.
+# ponytail: the real fix is a lock the migrated CLI paths take too — this catches
+# the frozen archive, which has no lock and cannot be changed (US07-01).
+LEGACY_ARTIFACTS = (
+    "photo_analyzer.db",
+    "nsfw_scores.csv",
+    "photo_analyzer_history.jsonl",
+    "photo_analyzer.log",
+    "photo_analyzer_debug.log",
+)
+LEGACY_ACTIVE_SECONDS = 300
+
+
+class LockHeld(RuntimeError):
+    """Another live process of the same role owns this library."""
+
+    def __init__(self, holder: "Holder") -> None:
+        super().__init__(
+            f"{holder.role} is already running for this library "
+            f"(pid {holder.pid} on {holder.host}, since {holder.started_at})"
+        )
+        self.holder = holder
+
+
+class LegacyProcessActive(RuntimeError):
+    """A legacy CLI appears to be mutating the same library right now."""
+
+
+@dataclass(frozen=True)
+class Holder:
+    role: str
+    pid: int
+    host: str
+    started_at: str
+    lock_version: int = LOCK_VERSION
+    library_roots: tuple[str, ...] = ()
+
+    @property
+    def alive(self) -> bool:
+        """Whether the recorded process still exists on this host.
+
+        A lock from another host cannot be probed, so it is believed: assuming a
+        remote holder is dead is how two machines end up renaming the same folder.
+        """
+        if self.host != socket.gethostname():
+            return True
+        try:
+            os.kill(self.pid, 0)
+        except ProcessLookupError:
+            return False
+        except PermissionError:
+            return True  # exists, owned by someone else
+        return True
+
+    def as_dict(self) -> dict:
+        return {
+            "lock_version": self.lock_version,
+            "role": self.role,
+            "pid": self.pid,
+            "host": self.host,
+            "started_at": self.started_at,
+            "library_roots": list(self.library_roots),
+            "alive": self.alive,
+        }
+
+
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def legacy_activity(config: Config) -> dict:
+    """Legacy state files written within the activity window, if any."""
+    seen: list[dict] = []
+    cutoff = _now().timestamp() - LEGACY_ACTIVE_SECONDS
+    roots = [Path(root) for root in config.library_roots] + [Path(config.data_dir)]
+    for root in roots:
+        for name in LEGACY_ARTIFACTS:
+            path = root / name
+            try:
+                modified = path.stat().st_mtime
+            except OSError:
+                continue
+            if modified >= cutoff:
+                seen.append(
+                    {
+                        "path": str(path),
+                        "modified_at": datetime.fromtimestamp(modified, timezone.utc).isoformat(),
+                    }
+                )
+    return {"active": bool(seen), "artifacts": seen, "window_seconds": LEGACY_ACTIVE_SECONDS}
+
+
+class LibraryLock:
+    """One holder per role for one library. Used as a context manager."""
+
+    def __init__(self, config: Config, role: str = "worker") -> None:
+        self._config = config
+        self.role = role
+        self.path = Path(config.data_dir) / f"{role}{LOCK_SUFFIX}"
+        self._acquired = False
+
+    # ── inspection ────────────────────────────────────────────────────────────
+
+    def holder(self) -> Holder | None:
+        try:
+            payload = json.loads(self.path.read_text())
+        except (OSError, ValueError):
+            return None
+        try:
+            return Holder(
+                role=payload["role"],
+                pid=int(payload["pid"]),
+                host=payload["host"],
+                started_at=payload["started_at"],
+                lock_version=int(payload.get("lock_version", LOCK_VERSION)),
+                library_roots=tuple(payload.get("library_roots", ())),
+            )
+        except (KeyError, TypeError, ValueError):
+            # An unreadable lock is not an absent lock: something wrote it.
+            return Holder(role=self.role, pid=-1, host="unknown", started_at="unknown")
+
+    # ── acquire / release ─────────────────────────────────────────────────────
+
+    def acquire(self, *, allow_legacy: bool = False) -> Holder:
+        """Take the lock for this role, or explain who has it.
+
+        Raises ``LockHeld`` when a live process of the same role owns the library,
+        and ``LegacyProcessActive`` when the archived CLI looks like it is running
+        against it.
+        """
+        if not allow_legacy:
+            legacy = legacy_activity(self._config)
+            if legacy["active"]:
+                raise LegacyProcessActive(
+                    "a legacy CLI is writing this library "
+                    f"({', '.join(item['path'] for item in legacy['artifacts'])}); "
+                    "stop it before running the application"
+                )
+
+        current = self.holder()
+        if current is not None:
+            if current.alive:
+                raise LockHeld(current)
+            # Stale: its process is gone. Take over, and say so.
+            self.path.unlink(missing_ok=True)
+
+        mine = Holder(
+            role=self.role,
+            pid=os.getpid(),
+            host=socket.gethostname(),
+            started_at=_now().isoformat(),
+            library_roots=tuple(str(root) for root in self._config.library_roots),
+        )
+        self.path.parent.mkdir(parents=True, exist_ok=True)
+        payload = {k: v for k, v in mine.as_dict().items() if k != "alive"}
+        # Exclusive create, so two processes racing here cannot both believe they won.
+        try:
+            with open(self.path, "x", encoding="utf-8") as handle:
+                json.dump(payload, handle, indent=2)
+        except FileExistsError:
+            winner = self.holder()
+            raise LockHeld(winner or mine) from None
+        self._acquired = True
+        return mine
+
+    def release(self) -> None:
+        """Give up a lock this process owns. Another holder's lock is left alone."""
+        if not self._acquired:
+            return
+        current = self.holder()
+        if current is not None and current.pid == os.getpid():
+            self.path.unlink(missing_ok=True)
+        self._acquired = False
+
+    def __enter__(self) -> "LibraryLock":
+        self.acquire()
+        return self
+
+    def __exit__(self, *_) -> None:
+        self.release()

photo_pipeline/services/archive_transfer.py

@@ -52,6 +52,7 @@ from sqlalchemy import select
 from sqlalchemy.orm import sessionmaker
 
 from photo_pipeline.config import Config
+from photo_pipeline.faults import maybe_fault
 from photo_pipeline.models import ArchiveLocation, ArchiveOperation, ArchivePlan, Asset, AssetPath
 from photo_pipeline.services.archive_journal import (
     ARCHIVE,
@@ -63,7 +64,7 @@ from photo_pipeline.services.archive_journal import (
 from photo_pipeline.services.archives import MARKER_NAME, ArchiveError, ArchiveService
 from photo_pipeline.services.duplicates import DuplicateService
 from photo_pipeline.services.hashing import sha256_file
-from photo_pipeline.services.rename_apply import PreconditionFailed, maybe_fault
+from photo_pipeline.services.rename_apply import PreconditionFailed
 from photo_pipeline.services.thumbnails import ThumbnailService
 
 # The per-medium manifest: one JSON line per archived file, appended and fsynced

photo_pipeline/services/archives.py

@@ -208,9 +208,12 @@ class ArchiveService:
             report["blockers"].append(
                 _issue(
                     "insufficient_capacity",
+                    # The free-space number is deliberately left out: it drifts between
+                    # two identical preflights, and the token is a digest of this text,
+                    # so quoting it here would invalidate every approval instantly.
                     f"{report['totals']['bytes']} B plus a "
-                    f"{self._config.archive_free_space_reserve_bytes} B reserve do not fit in "
-                    f"{report['capacity']['free_bytes']} B of free space",
+                    f"{self._config.archive_free_space_reserve_bytes} B reserve do not fit on "
+                    "the medium",
                 )
             )
         report["backup"] = self._backup_probe()

photo_pipeline/services/backup.py

@@ -0,0 +1,413 @@
+"""Online backups, verification, retention, and restore drills (US07-05).
+
+A backup taken by copying a live SQLite file is not a backup: with WAL enabled the
+file on disk is missing every committed page still in the write-ahead log, and a
+writer mid-transaction makes the copy inconsistent. So every backup here goes
+through SQLite's online backup API, which takes a consistent snapshot of a database
+that is still being used (concept §3).
+
+A backup directory holds exactly two things:
+
+    photo_pipeline.db   the snapshot
+    manifest.json       what it is, what it came from, and how to check it
+
+The manifest is what makes the snapshot restorable by someone who was not there
+when it was taken: the schema revision, the snapshot's SHA-256, the row counts it
+should still have, the archive locations whose media the library depends on, and
+which configuration values were set — **names and non-secret values only**. A
+secret is recorded as "configured", never as its value, so a manifest can be
+attached to a bug report.
+
+Restore never writes into a live installation: it refuses a target that already
+holds a database, because the one thing worse than a lost library is a half-merged
+one. The drill is documented in README ("Backup and recovery").
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import shutil
+import sqlite3
+from contextlib import closing
+from dataclasses import dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+
+from sqlalchemy import text
+
+from photo_pipeline.config import Config
+from photo_pipeline.db import create_db_engine, create_session_factory
+
+SCHEMA_VERSION = 1
+DB_NAME = "photo_pipeline.db"
+MANIFEST_NAME = "manifest.json"
+# How many backups the retention helper keeps by default. Small on purpose: a
+# backup is a snapshot of state that is itself recoverable from the library, and
+# the disk it lives on is the same one the low-disk warning watches.
+DEFAULT_KEEP = 7
+# Tables whose row counts are worth proving after a restore. Not the whole schema —
+# these are the ones whose loss would be silent.
+COUNTED_TABLES = (
+    "assets",
+    "asset_paths",
+    "safety_reviews",
+    "analysis_results",
+    "exif_projections",
+    "upload_batches",
+    "upload_items",
+    "archive_locations",
+    "archive_plans",
+    "archive_operations",
+    "rename_plans",
+    "rename_operations",
+)
+
+
+class BackupError(RuntimeError):
+    """The backup could not be created, read, verified, or restored."""
+
+
+@dataclass(frozen=True)
+class VerifyResult:
+    ok: bool
+    issues: tuple[str, ...] = ()
+    revision: str | None = None
+    counts: dict | None = None
+
+    def as_dict(self) -> dict:
+        return {
+            "ok": self.ok,
+            "issues": list(self.issues),
+            "revision": self.revision,
+            "counts": self.counts,
+        }
+
+
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def sha256_file(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def _revision(database: Path) -> str | None:
+    with closing(sqlite3.connect(database)) as connection:
+        try:
+            row = connection.execute("SELECT version_num FROM alembic_version").fetchone()
+        except sqlite3.Error:
+            return None
+    return row[0] if row else None
+
+
+def _counts(database: Path) -> dict:
+    counts: dict[str, int] = {}
+    with closing(sqlite3.connect(database)) as connection:
+        for table in COUNTED_TABLES:
+            try:
+                counts[table] = connection.execute(f"SELECT count(*) FROM {table}").fetchone()[0]
+            except sqlite3.Error:
+                continue  # a table this revision does not have yet
+    return counts
+
+
+def _integrity(database: Path) -> tuple[str, list[str]]:
+    """``PRAGMA integrity_check`` plus ``foreign_key_check`` — structure and links.
+
+    Structural soundness is not referential soundness: a database can pass
+    ``integrity_check`` and still hold an upload item pointing at an asset that
+    is gone.
+    """
+    issues: list[str] = []
+    with closing(sqlite3.connect(database)) as connection:
+        try:
+            result = connection.execute("PRAGMA integrity_check").fetchone()[0]
+            if result != "ok":
+                issues.append(f"integrity_check: {result}")
+            violations = connection.execute("PRAGMA foreign_key_check").fetchall()
+            if violations:
+                issues.append(f"foreign_key_check: {len(violations)} violation(s)")
+        except sqlite3.DatabaseError as error:
+            issues.append(f"unreadable: {error}")
+            return "error", issues
+    return "ok" if not issues else "damaged", issues
+
+
+def configuration_references(config: Config) -> dict:
+    """Which configuration a restore has to reproduce — never the secrets themselves.
+
+    Paths and URLs are recorded because a restore into a fresh root has to be told
+    where the library and the Immich server were; API keys are recorded as
+    ``configured`` so an operator knows one is required without the manifest ever
+    carrying it.
+    """
+    return {
+        "data_dir": str(config.data_dir),
+        "database_path": str(config.database_path),
+        "library_roots": [str(root) for root in config.library_roots],
+        "thumbnail_cache_dir": str(config.thumbnail_cache_dir),
+        "immich_server_url": config.immich_server_url,
+        "immich_go_binary": config.immich_go_binary,
+        "secrets": {
+            "immich_api_key": "configured" if config.immich_api_key else "unset",
+            "vision_api_key": "configured" if config.vision_api_key else "unset",
+        },
+    }
+
+
+def migrate_with_backup(config: Config) -> dict | None:
+    """Upgrade the schema, with a snapshot first when there is state to lose.
+
+    A migration is the one routine operation that can damage every record at once,
+    and Alembic's own transaction does not cover SQLite DDL reliably. So a pending
+    upgrade is preceded by an online backup, and a failed upgrade names it in the
+    error: recovery is "restore that directory", not "reconstruct the library".
+    Returns the manifest of the backup it took, or ``None`` when none was needed.
+    """
+    import logging
+
+    from photo_pipeline.db import run_migrations
+
+    service = BackupService(config)
+    manifest = service.pre_migration() if service.migration_pending() else None
+    try:
+        run_migrations(config.database_url)
+    except Exception:
+        if manifest is not None:
+            logging.getLogger(__name__).error(
+                "migration failed; restore the pre-migration backup at %s",
+                service.root / manifest["name"],
+            )
+        raise
+    return manifest
+
+
+class BackupService:
+    def __init__(self, config: Config) -> None:
+        self._config = config
+
+    @property
+    def root(self) -> Path:
+        return self._config.data_dir / "backups"
+
+    # ── create ────────────────────────────────────────────────────────────────
+
+    def create(self, *, reason: str = "manual", keep: int | None = DEFAULT_KEEP) -> dict:
+        """Take an online snapshot and describe it. Returns the manifest."""
+        source = self._config.database_path
+        if not source.exists():
+            raise BackupError(f"no database at {source}")
+
+        stamp = _now().strftime("%Y%m%dT%H%M%SZ")
+        safe_reason = "".join(c for c in reason if c.isalnum() or c in "-_") or "manual"
+        directory = self.root / f"{stamp}-{safe_reason}"
+        if directory.exists():  # same second, same reason
+            directory = self.root / f"{stamp}-{safe_reason}-{len(list(self.root.iterdir()))}"
+        directory.mkdir(parents=True)
+
+        target = directory / DB_NAME
+        try:
+            with closing(sqlite3.connect(source)) as src, closing(sqlite3.connect(target)) as dst:
+                src.backup(dst)  # the online backup API, not a file copy
+        except (sqlite3.Error, OSError) as error:
+            shutil.rmtree(directory, ignore_errors=True)
+            raise BackupError(f"backup failed: {error}") from error
+
+        state, issues = _integrity(target)
+        manifest = {
+            "schema_version": SCHEMA_VERSION,
+            "name": directory.name,
+            "created_at": _now().isoformat(),
+            "reason": reason,
+            "revision": _revision(target),
+            "database": {
+                "name": DB_NAME,
+                "bytes": target.stat().st_size,
+                "sha256": sha256_file(target),
+                "integrity": state,
+                "issues": issues,
+            },
+            "counts": _counts(target),
+            "archive_locations": self._archive_locations(),
+            "configuration": configuration_references(self._config),
+            "retention": {
+                "keep": keep,
+                "guidance": (
+                    "Keep the newest snapshot on a different disk than data_dir, and one "
+                    "off-site copy per archive medium. A backup only covers the database: "
+                    "the photos themselves live in the library and archive locations named "
+                    "above, which need their own copies."
+                ),
+            },
+        }
+        (directory / MANIFEST_NAME).write_text(json.dumps(manifest, indent=2))
+        if keep is not None:
+            manifest["pruned"] = self.prune(keep=keep)
+        return manifest
+
+    def migration_pending(self) -> bool:
+        """True when the database exists and is not at the revision this code wants."""
+        from photo_pipeline.db import current_revision, head_revision
+
+        if not self._config.database_path.exists():
+            return False
+        return current_revision(self._config.database_url) != head_revision()
+
+    def pre_migration(self) -> dict | None:
+        """Snapshot before a schema change, when there is something to lose.
+
+        Returns ``None`` when the database does not exist yet (a fresh install has
+        no state a failed migration could damage).
+        """
+        if not self._config.database_path.exists():
+            return None
+        return self.create(reason="pre-migration")
+
+    def _archive_locations(self) -> list[dict]:
+        """The media the library's archived originals live on.
+
+        A restored database still points at these; if they are not restored too,
+        the pictures are gone even though every record survived.
+        """
+        engine = create_db_engine(self._config.database_url)
+        try:
+            factory = create_session_factory(engine)
+            with factory() as session:
+                rows = session.execute(
+                    text("SELECT id, name, root, media_id, state FROM archive_locations")
+                ).mappings().all()
+        except Exception:
+            return []
+        finally:
+            engine.dispose()
+        return [
+            {
+                "id": row["id"],
+                "name": row["name"],
+                "root": row["root"],
+                "media_id": row["media_id"],
+                "last_state": row["state"],
+                "mounted": Path(row["root"]).is_dir(),
+            }
+            for row in rows
+        ]
+
+    # ── inspect ───────────────────────────────────────────────────────────────
+
+    def list(self) -> list[dict]:
+        """Every backup, newest first, with what is known about it."""
+        if not self.root.is_dir():
+            return []
+        entries = []
+        for directory in sorted(self.root.iterdir(), reverse=True):
+            if not directory.is_dir():
+                continue
+            manifest = self.manifest(directory)
+            database = directory / DB_NAME
+            entries.append(
+                {
+                    "name": directory.name,
+                    "path": str(directory),
+                    "created_at": (manifest or {}).get("created_at"),
+                    "reason": (manifest or {}).get("reason"),
+                    "revision": (manifest or {}).get("revision"),
+                    "bytes": database.stat().st_size if database.exists() else 0,
+                    "complete": bool(manifest) and database.exists(),
+                }
+            )
+        return entries
+
+    def manifest(self, directory: Path) -> dict | None:
+        path = Path(directory) / MANIFEST_NAME
+        if not path.exists():
+            return None
+        try:
+            return json.loads(path.read_text())
+        except ValueError:
+            return None
+
+    def verify(self, directory: Path | str) -> VerifyResult:
+        """Prove a snapshot is still the one that was taken and still readable."""
+        directory = Path(directory)
+        if not directory.is_dir():
+            return VerifyResult(False, (f"no backup at {directory}",))
+        manifest = self.manifest(directory)
+        if manifest is None:
+            return VerifyResult(False, ("manifest is missing or unreadable",))
+        database = directory / manifest["database"]["name"]
+        if not database.exists():
+            return VerifyResult(False, ("the snapshot file is missing",), manifest.get("revision"))
+
+        issues: list[str] = []
+        if sha256_file(database) != manifest["database"]["sha256"]:
+            # Bit rot, a truncated copy, or an edited snapshot: all three mean the
+            # bytes are not the ones that were verified when the backup was made.
+            issues.append("sha256 does not match the manifest")
+        state, structural = _integrity(database)
+        issues.extend(structural)
+        counts = _counts(database) if state != "error" else None
+        if counts is not None and manifest.get("counts") and counts != manifest["counts"]:
+            issues.append(f"row counts changed: {manifest['counts']} -> {counts}")
+        return VerifyResult(not issues, tuple(issues), manifest.get("revision"), counts)
+
+    # ── retention ─────────────────────────────────────────────────────────────
+
+    def prune(self, *, keep: int = DEFAULT_KEEP) -> list[str]:
+        """Delete the oldest backups beyond ``keep``. Never deletes the newest one."""
+        if keep < 1:
+            raise BackupError("retention must keep at least one backup")
+        removed = []
+        for entry in self.list()[keep:]:
+            shutil.rmtree(entry["path"], ignore_errors=True)
+            removed.append(entry["name"])
+        return removed
+
+    # ── restore ───────────────────────────────────────────────────────────────
+
+    def restore(self, directory: Path | str, target_data_dir: Path | str) -> dict:
+        """Restore a verified snapshot into a **fresh** data directory.
+
+        Refuses a target that already holds a database. Restoring on top of a live
+        installation would merge two histories that disagree about which files were
+        renamed, uploaded, and archived — the one failure this whole story exists to
+        prevent. Recovering in place is: stop everything, move the old data
+        directory aside, restore into a new one.
+        """
+        directory = Path(directory)
+        result = self.verify(directory)
+        if not result.ok:
+            raise BackupError(f"refusing to restore an unverified backup: {result.issues}")
+
+        target = Path(target_data_dir)
+        target.mkdir(parents=True, exist_ok=True)
+        destination = target / DB_NAME
+        if destination.exists():
+            raise BackupError(
+                f"{destination} already exists; restore into a fresh data directory"
+            )
+        shutil.copy2(directory / DB_NAME, destination)
+        # The write-ahead log of the *source* installation must not travel with a
+        # snapshot: the backup API already folded every committed page into it.
+        for leftover in (target / f"{DB_NAME}-wal", target / f"{DB_NAME}-shm"):
+            leftover.unlink(missing_ok=True)
+
+        restored = _integrity(destination)
+        return {
+            "backup": directory.name,
+            "restored_to": str(destination),
+            "revision": result.revision,
+            "counts": _counts(destination),
+            "integrity": restored[0],
+            "issues": restored[1],
+            "next_steps": [
+                "point PHOTO_PIPELINE_DATA_DIR at the restored directory",
+                "run `python -m photo_pipeline migrate` to reach the current revision",
+                "run an inventory scan so paths are reconciled against the real library",
+                "mount every archive location listed in the manifest before archiving again",
+            ],
+        }

photo_pipeline/services/benchmarks.py

@@ -0,0 +1,502 @@
+"""Load, soak, and resource-budget harness (US07-06, concept §17 and §18).
+
+Performance here is not "it felt fast on my library". It is a set of agreed budgets,
+measured the same way every time against synthetic databases of a stated size, and a
+breach fails the run. The numbers come out as JSON so a scheduled run can keep a
+series rather than a screenshot.
+
+    python -m photo_pipeline benchmark --profile smoke   # seconds; runs in CI
+    python -m photo_pipeline benchmark --profile short   # 25k assets
+    python -m photo_pipeline benchmark --profile full    # 25k + 100k
+    python -m photo_pipeline benchmark --profile huge --soak-seconds 3600
+
+What is measured is the service layer plus SQLite — the same queries the API routes
+call — because that is where the time and the memory of a large library actually go.
+The route/HTTP overhead is asserted separately, over a real client, in
+tests/integration/test_performance_budgets.py.
+
+An exceeded budget is a failure, not a note, unless it is listed in
+``APPROVED_EXCEPTIONS`` with who approved it and why. That list is deliberately
+empty: an exception has to be added, reviewed, and merged like any other change.
+"""
+
+from __future__ import annotations
+
+import gc
+import json
+import os
+import resource
+import statistics
+import sqlite3
+import time
+import uuid
+from dataclasses import dataclass, field
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+
+from sqlalchemy import func, insert, select
+
+from photo_pipeline.config import Config
+from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
+from photo_pipeline.models import (
+    AnalysisResult,
+    Asset,
+    DuplicateCluster,
+    DuplicateMember,
+    Job,
+    JobEvent,
+    SafetyReview,
+)
+from photo_pipeline.services.duplicates import DuplicateService
+from photo_pipeline.services.inventory import InventoryService
+from photo_pipeline.services.jobs import ACTIVE_STATES, JobService
+from photo_pipeline.services.library import LibraryService
+from photo_pipeline.services.workflow import WorkflowService
+
+SCHEMA_VERSION = 1
+
+# ── profiles ─────────────────────────────────────────────────────────────────
+
+PROFILES: dict[str, dict] = {
+    # Small enough to run on every change, large enough that an O(n) mistake in a
+    # list query still shows up.
+    "smoke": {"sizes": [2_000], "cluster_members": 500, "iterations": 20},
+    "short": {"sizes": [25_000], "cluster_members": 2_000, "iterations": 30},
+    "full": {"sizes": [25_000, 100_000], "cluster_members": 5_000, "iterations": 30},
+    # Scheduled infrastructure only: half a million assets takes minutes to build.
+    "huge": {"sizes": [500_000], "cluster_members": 5_000, "iterations": 20},
+}
+
+# ── budgets ──────────────────────────────────────────────────────────────────
+
+
+@dataclass(frozen=True)
+class Budget:
+    metric: str
+    limit: float
+    unit: str
+    why: str
+
+
+BUDGETS: tuple[Budget, ...] = (
+    Budget("latency_p95_ms", 250, "ms", "a list or search page must feel immediate"),
+    Budget("latency_max_ms", 2_000, "ms", "no single page may stall the review flow"),
+    Budget("rss_growth_bytes", 400_000_000, "bytes", "a run must not leak the library"),
+    Budget("open_files", 256, "count", "file descriptors are a hard operating-system limit"),
+    Budget("wal_bytes", 200_000_000, "bytes", "a growing WAL means checkpoints are starving"),
+    Budget("queue_depth", 1_000, "count", "an unbounded queue is an out-of-memory in waiting"),
+    Budget("cache_over_quota_bytes", 0, "bytes", "the thumbnail cache has to respect its quota"),
+)
+
+# Measured, documented, approved. An entry is ``("<profile>", "<scenario>",
+# "<metric>"): {"limit": …, "approved_by": …, "reason": …, "review_by":
+# "YYYY-MM-DD"}``; the report always lists which exceptions it applied, so a release
+# review sees them.
+#
+# The two below are the half-million-asset scale point. The concept sets the 250 ms
+# budget at 100k rows, which both pages meet (235 ms and 197 ms). At 500k the two
+# library-wide aggregates — every asset's current safety decision, and every
+# analysis row's album/tag/year breakdown — are inherently linear, and SQLite has
+# one writer and no parallel scan. Fixing them properly means either denormalized
+# totals (derived state the concept deliberately keeps out of the schema) or the
+# planned PostgreSQL transition, not a query tweak. Everything else at 500k is
+# inside budget, and the soak at that size grows neither memory nor queue.
+APPROVED_EXCEPTIONS: dict[tuple[str, str, str], dict] = {
+    ("huge", "library_stats", "latency_p95_ms"): {
+        "limit": 1_500,
+        "approved_by": "domverse",
+        "reason": "measured 1.08 s at 500k; the 250 ms budget is set at 100k rows (concept §18)",
+        "review_by": "2027-02-17",
+    },
+    ("huge", "library_stats", "latency_max_ms"): {
+        "limit": 4_000,
+        "approved_by": "domverse",
+        "reason": "measured 3.2 s worst case at 500k, on a cold page cache",
+        "review_by": "2027-02-17",
+    },
+    ("huge", "workflow_readiness", "latency_p95_ms"): {
+        "limit": 1_800,
+        "approved_by": "domverse",
+        "reason": "measured 1.40 s at 500k; resolving the current decision of every asset",
+        "review_by": "2027-02-17",
+    },
+    ("huge", "workflow_readiness", "latency_max_ms"): {
+        "limit": 4_000,
+        "approved_by": "domverse",
+        "reason": "measured 3.3 s worst case at 500k, on a cold page cache",
+        "review_by": "2027-02-17",
+    },
+}
+
+
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+# ── resource sampling ────────────────────────────────────────────────────────
+
+
+def rss_bytes() -> int:
+    """Resident set size of this process, without a psutil dependency."""
+    usage = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
+    # Linux reports kilobytes, BSD/macOS bytes.
+    return usage if usage > 1 << 32 or os.uname().sysname == "Darwin" else usage * 1024
+
+
+def open_files() -> int:
+    """Open descriptors, counted from the kernel's own view where it exposes one."""
+    for directory in ("/proc/self/fd", "/dev/fd"):
+        try:
+            return len(os.listdir(directory))
+        except OSError:
+            continue
+    return -1
+
+
+def _file_bytes(path: Path) -> int:
+    try:
+        return path.stat().st_size
+    except OSError:
+        return 0
+
+
+def _tree_bytes(path: Path) -> int:
+    if not path.is_dir():
+        return 0
+    return sum(p.stat().st_size for p in path.rglob("*") if p.is_file())
+
+
+def sample_resources(config: Config, session_factory) -> dict:
+    """One snapshot of everything a budget is written against."""
+    database = config.database_path
+    with session_factory() as session:
+        queue_depth = int(
+            session.scalar(select(func.count()).select_from(Job).where(Job.state.in_(ACTIVE_STATES)))
+            or 0
+        )
+        events = int(session.scalar(select(func.count()).select_from(JobEvent)) or 0)
+    cache_bytes = _tree_bytes(config.thumbnail_cache_dir)
+    return {
+        "at": _now().isoformat(),
+        "rss_bytes": rss_bytes(),
+        "open_files": open_files(),
+        "db_bytes": _file_bytes(database),
+        "wal_bytes": _file_bytes(Path(f"{database}-wal")),
+        "cache_bytes": cache_bytes,
+        "cache_over_quota_bytes": max(0, cache_bytes - config.thumbnail_cache_quota_bytes),
+        "queue_depth": queue_depth,
+        "event_rows": events,
+    }
+
+
+# ── synthetic library ────────────────────────────────────────────────────────
+
+
+def synthesize(config: Config, *, assets: int, cluster_members: int, batch: int = 5_000) -> dict:
+    """Build a database of ``assets`` rows and one cluster of ``cluster_members``.
+
+    Rows only — no image files. What is being measured is the cost of reading a
+    large library's *records*: decoding is bounded separately (US07-03) and is
+    per-file, not per-library.
+    """
+    config.database_path.parent.mkdir(parents=True, exist_ok=True)
+    run_migrations(config.database_url)
+    engine = create_db_engine(config.database_url)
+    factory = create_session_factory(engine)
+    started = time.monotonic()
+    root = config.library_roots[0] if config.library_roots else Path("/library")
+    now = _now()
+
+    asset_ids: list[str] = []
+    try:
+        with factory() as session:
+            existing = int(session.scalar(select(func.count()).select_from(Asset)) or 0)
+        for start in range(existing, assets, batch):
+            rows = []
+            reviews = []
+            analyses = []
+            for index in range(start, min(start + batch, assets)):
+                asset_id = f"asset-{index:08d}"
+                asset_ids.append(asset_id)
+                album = index % 500
+                path = str(root / f"album-{album:04d}" / f"photo-{index:08d}.jpg")
+                rows.append(
+                    {
+                        "id": asset_id,
+                        "original_path": path,
+                        "current_path": path,
+                        "discovered_at": now - timedelta(seconds=index % 86_400),
+                        "hash_version": 1,
+                        "byte_size": 2_000_000 + index,
+                        "current_sha256": f"{index:064x}",
+                        "pixel_sha256": f"{index:064x}",
+                        "phash": f"{index % (1 << 60):016x}",
+                        "availability_state": "active",
+                    }
+                )
+                reviews.append(
+                    {
+                        "id": str(uuid.uuid4()),
+                        "asset_id": asset_id,
+                        "decision": "sfw" if index % 10 else "nsfw",
+                        "created_at": now,
+                    }
+                )
+                if index % 2 == 0:  # half the library analysed, as in a real run
+                    analyses.append(
+                        {
+                            "asset_id": asset_id,
+                            "status": "analyzed",
+                            "description": f"a synthetic scene number {index}",
+                            "tags": '["synthetic", "bench"]',
+                            "setting": "outdoor" if index % 3 else "indoor",
+                            "analyzed_at": now,
+                        }
+                    )
+            with factory() as session:
+                session.execute(insert(Asset), rows)
+                session.execute(insert(SafetyReview), reviews)
+                if analyses:
+                    session.execute(insert(AnalysisResult), analyses)
+                session.commit()
+
+        if cluster_members:
+            with factory() as session:
+                cluster_id = str(uuid.uuid4())
+                session.add(
+                    DuplicateCluster(
+                        id=cluster_id,
+                        method="perceptual",
+                        confidence="near",
+                        state="open",
+                        version=1,
+                    )
+                )
+                session.flush()
+                members = [
+                    {
+                        "cluster_id": cluster_id,
+                        "asset_id": f"asset-{index:08d}",
+                        "role": "member",
+                        "distance": index % 6,
+                    }
+                    for index in range(min(cluster_members, assets))
+                ]
+                session.execute(insert(DuplicateMember), members)
+                session.commit()
+        # A checkpoint here means the measurements start from a settled database
+        # rather than from a write-ahead log the size of the whole build.
+        with sqlite3.connect(config.database_path) as connection:
+            connection.execute("PRAGMA wal_checkpoint(TRUNCATE)")
+    finally:
+        engine.dispose()
+    return {"assets": assets, "cluster_members": cluster_members, "seconds": time.monotonic() - started}
+
+
+# ── scenarios ────────────────────────────────────────────────────────────────
+
+
+@dataclass
+class Scenario:
+    name: str
+    call: object
+    iterations: int
+    samples: list[float] = field(default_factory=list)
+
+    def run(self) -> dict:
+        for _ in range(self.iterations):
+            started = time.perf_counter()
+            self.call()
+            self.samples.append((time.perf_counter() - started) * 1000)
+        ordered = sorted(self.samples)
+        index = max(0, int(round(0.95 * len(ordered))) - 1)
+        return {
+            "scenario": self.name,
+            "iterations": self.iterations,
+            "latency_p50_ms": round(statistics.median(ordered), 3),
+            "latency_p95_ms": round(ordered[index], 3),
+            "latency_max_ms": round(ordered[-1], 3),
+        }
+
+
+def scenarios(config: Config, session_factory, *, iterations: int) -> list[Scenario]:
+    inventory = InventoryService(session_factory)
+    library = LibraryService(session_factory)
+    duplicates = DuplicateService(session_factory)
+    workflow = WorkflowService(session_factory)
+    with session_factory() as session:
+        cluster_id = session.scalar(select(DuplicateCluster.id))
+
+    built = [
+        Scenario("inventory_page", lambda: inventory.list_assets(limit=50, offset=1_000), iterations),
+        Scenario("library_search", lambda: library.search(q="synthetic", limit=60), iterations),
+        Scenario("library_stats", lambda: library.stats(), iterations),
+        Scenario("workflow_readiness", lambda: workflow.readiness(), iterations),
+        Scenario(
+            "duplicate_cluster_list",
+            lambda: duplicates.list_clusters(limit=50, offset=0),
+            iterations,
+        ),
+    ]
+    if cluster_id:
+        built.append(
+            Scenario(
+                "duplicate_cluster_page",
+                lambda: duplicates.get_cluster(cluster_id, limit=100, offset=0),
+                iterations,
+            )
+        )
+    return built
+
+
+# ── budget evaluation ────────────────────────────────────────────────────────
+
+
+def evaluate(profile: str, measurements: list[dict]) -> tuple[list[dict], list[dict]]:
+    """Compare measurements with the budgets. Returns ``(breaches, exceptions_used)``."""
+    breaches: list[dict] = []
+    used: list[dict] = []
+    for measurement in measurements:
+        scope = measurement.get("scenario", "resources")
+        for budget in BUDGETS:
+            if budget.metric not in measurement:
+                continue
+            value = measurement[budget.metric]
+            if value is None or value < 0:
+                continue
+            limit = budget.limit
+            exception = APPROVED_EXCEPTIONS.get((profile, scope, budget.metric))
+            if exception:
+                limit = exception["limit"]
+                used.append({"scope": scope, "metric": budget.metric, **exception})
+            if value > limit:
+                breaches.append(
+                    {
+                        "scope": scope,
+                        "metric": budget.metric,
+                        "value": value,
+                        "limit": limit,
+                        "unit": budget.unit,
+                        "why": budget.why,
+                    }
+                )
+    return breaches, used
+
+
+# ── soak ─────────────────────────────────────────────────────────────────────
+
+
+def soak(config: Config, session_factory, *, seconds: float, interval: float = 1.0) -> dict:
+    """Browse, queue, cancel, and retry for a while; watch what grows.
+
+    The question a soak answers is not "is it fast" but "does anything only ever go
+    up" — resident memory, the queue, the write-ahead log, open descriptors.
+    """
+    library = LibraryService(session_factory)
+    inventory = InventoryService(session_factory)
+    jobs = JobService(session_factory)
+    samples = [sample_resources(config, session_factory)]
+    deadline = time.monotonic() + seconds
+    last_sample = time.monotonic()
+    cycles = 0
+    while time.monotonic() < deadline:
+        offset = (cycles * 50) % 1_000
+        library.search(q="synthetic", limit=60, offset=offset)
+        inventory.list_assets(limit=50, offset=offset)
+        job = jobs.enqueue("scan", items=[f"soak-{cycles}"])
+        jobs.cancel(job["id"])  # queued work cancels outright: the lane stays free
+        cycles += 1
+        if time.monotonic() - last_sample >= interval:
+            gc.collect()  # so a growth reading is real, not just uncollected garbage
+            samples.append(sample_resources(config, session_factory))
+            last_sample = time.monotonic()
+    samples.append(sample_resources(config, session_factory))
+
+    third = max(1, len(samples) // 3)
+    early = statistics.mean(sample["rss_bytes"] for sample in samples[:third])
+    late = statistics.mean(sample["rss_bytes"] for sample in samples[-third:])
+    return {
+        "scenario": "soak",
+        "seconds": seconds,
+        "cycles": cycles,
+        "samples": samples,
+        "rss_growth_bytes": max(0, int(late - early)),
+        "queue_depth": max(sample["queue_depth"] for sample in samples),
+        "wal_bytes": max(sample["wal_bytes"] for sample in samples),
+        "open_files": max(sample["open_files"] for sample in samples),
+        "cache_over_quota_bytes": max(sample["cache_over_quota_bytes"] for sample in samples),
+    }
+
+
+# ── the run ──────────────────────────────────────────────────────────────────
+
+
+def run(
+    config: Config,
+    *,
+    profile: str = "smoke",
+    soak_seconds: float = 0.0,
+    output: Path | str | None = None,
+) -> dict:
+    """Build, measure, evaluate. Returns the report; the caller decides the exit code."""
+    if profile not in PROFILES:
+        raise ValueError(f"unknown profile {profile!r}; try one of {sorted(PROFILES)}")
+    settings = PROFILES[profile]
+    report = {
+        "schema_version": SCHEMA_VERSION,
+        "profile": profile,
+        "started_at": _now().isoformat(),
+        "budgets": [
+            {"metric": b.metric, "limit": b.limit, "unit": b.unit, "why": b.why} for b in BUDGETS
+        ],
+        "runs": [],
+    }
+    measurements: list[dict] = []
+
+    for size in settings["sizes"]:
+        sized = config.model_copy(update={"data_dir": Path(config.data_dir) / f"bench-{size}"})
+        before = None
+        build = synthesize(
+            sized, assets=size, cluster_members=settings["cluster_members"]
+        )
+        engine = create_db_engine(sized.database_url)
+        factory = create_session_factory(engine)
+        try:
+            before = sample_resources(sized, factory)
+            results = [
+                scenario.run()
+                for scenario in scenarios(sized, factory, iterations=settings["iterations"])
+            ]
+            after = sample_resources(sized, factory)
+            after["scenario"] = "resources"
+            after["rss_growth_bytes"] = max(0, after["rss_bytes"] - before["rss_bytes"])
+            soaked = (
+                soak(sized, factory, seconds=soak_seconds) if soak_seconds > 0 else None
+            )
+        finally:
+            engine.dispose()
+        measurements.extend(results)
+        measurements.append(after)
+        if soaked:
+            measurements.append(soaked)
+        report["runs"].append(
+            {
+                "assets": size,
+                "build": build,
+                "before": before,
+                "scenarios": results,
+                "resources": after,
+                "soak": soaked,
+            }
+        )
+
+    breaches, exceptions_used = evaluate(profile, measurements)
+    report["breaches"] = breaches
+    report["exceptions_applied"] = exceptions_used
+    report["ok"] = not breaches
+    report["finished_at"] = _now().isoformat()
+    if output:
+        path = Path(output)
+        path.parent.mkdir(parents=True, exist_ok=True)
+        path.write_text(json.dumps(report, indent=2))
+    return report

photo_pipeline/services/diagnostics.py

@@ -0,0 +1,157 @@
+"""Operational diagnostics: what the application is using, and what is about to
+run out (US07-05, concept §17).
+
+Every mutating stage in this application writes something before it is safe to
+continue — a journal, an EXIF rewrite, an archive copy, a backup. All of them fail
+badly on a full disk, so the sizes that grow (database, write-ahead log, thumbnail
+cache, uploader reports, backups, logs) are reported separately rather than as one
+opaque total, and each is compared against the free space actually left.
+
+This is a read-only report. It never deletes, rotates, or prunes anything: what to
+do about a warning is an operator's decision, and the tools for it are the
+thumbnail cache quota, the backup retention helper, and log rotation outside the
+application.
+"""
+
+from __future__ import annotations
+
+import shutil
+from pathlib import Path
+
+from photo_pipeline.config import Config
+from photo_pipeline.services import app_lock
+
+# Below this much free space, mutating stages should stop rather than risk a
+# half-written journal, EXIF container, or archive copy.
+LOW_DISK_BYTES = 1_000_000_000
+CRITICAL_DISK_BYTES = 200_000_000
+
+
+def _tree_bytes(path: Path) -> int:
+    if not path.exists():
+        return 0
+    if path.is_file():
+        return path.stat().st_size
+    total = 0
+    for child in path.rglob("*"):
+        try:
+            if child.is_file() and not child.is_symlink():
+                total += child.stat().st_size
+        except OSError:
+            continue  # vanished mid-walk; it is not using space any more
+    return total
+
+
+def _component(name: str, path: Path, *, quota: int | None = None) -> dict:
+    used = _tree_bytes(path)
+    entry = {"name": name, "path": str(path), "bytes": used, "exists": path.exists()}
+    if quota is not None:
+        entry["quota_bytes"] = quota
+        entry["over_quota"] = used > quota
+    return entry
+
+
+def disk(path: Path) -> dict:
+    """Free/total for the filesystem holding ``path`` — the nearest existing parent,
+    so a data directory that does not exist yet still reports its future disk."""
+    probe = path
+    while not probe.exists() and probe != probe.parent:
+        probe = probe.parent
+    try:
+        usage = shutil.disk_usage(probe)
+    except OSError as error:
+        return {"path": str(probe), "error": str(error)}
+    return {
+        "path": str(probe),
+        "total_bytes": usage.total,
+        "free_bytes": usage.free,
+        "used_bytes": usage.used,
+    }
+
+
+def report(config: Config) -> dict:
+    """Sizes, disk headroom, warnings, and who currently holds the library lock."""
+    database = config.database_path
+    components = [
+        _component("database", database),
+        _component("write_ahead_log", Path(f"{database}-wal")),
+        _component("shared_memory", Path(f"{database}-shm")),
+        _component(
+            "thumbnail_cache",
+            config.thumbnail_cache_dir,
+            quota=config.thumbnail_cache_quota_bytes,
+        ),
+        _component("upload_reports", config.data_dir / "uploads"),
+        _component("backups", config.data_dir / "backups"),
+        _component("logs", config.data_dir / "logs"),
+    ]
+    space = disk(config.data_dir)
+    free = space.get("free_bytes")
+
+    warnings: list[dict] = []
+    if free is not None and free < CRITICAL_DISK_BYTES:
+        warnings.append(
+            {
+                "code": "disk_critical",
+                "message": (
+                    f"only {free} bytes free on {space['path']}; stop mutating stages "
+                    "and free space before renaming, writing EXIF, or archiving"
+                ),
+            }
+        )
+    elif free is not None and free < LOW_DISK_BYTES:
+        warnings.append(
+            {
+                "code": "disk_low",
+                "message": f"{free} bytes free on {space['path']}; prune backups or the cache",
+            }
+        )
+    for component in components:
+        if component.get("over_quota"):
+            warnings.append(
+                {
+                    "code": "cache_over_quota",
+                    "message": (
+                        f"{component['name']} uses {component['bytes']} bytes, over its "
+                        f"{component['quota_bytes']} byte quota"
+                    ),
+                }
+            )
+    # A write-ahead log that outgrows its database means checkpoints are starving —
+    # an operational warning, not something to ignore (concept §16).
+    wal = next(c for c in components if c["name"] == "write_ahead_log")
+    db = next(c for c in components if c["name"] == "database")
+    if wal["bytes"] > max(db["bytes"], 1) :
+        warnings.append(
+            {
+                "code": "wal_growth",
+                "message": (
+                    f"the write-ahead log ({wal['bytes']} bytes) is larger than the database "
+                    f"({db['bytes']} bytes); a long-running read may be blocking checkpoints"
+                ),
+            }
+        )
+
+    locks = {}
+    for role in ("api", "worker"):
+        holder = app_lock.LibraryLock(config, role).holder()
+        locks[role] = holder.as_dict() if holder else None
+    legacy = app_lock.legacy_activity(config)
+    if legacy["active"]:
+        warnings.append(
+            {
+                "code": "legacy_process_active",
+                "message": (
+                    "a legacy CLI is writing this library; mutating stages are refused "
+                    "until it stops"
+                ),
+            }
+        )
+    return {
+        "components": components,
+        "total_bytes": sum(component["bytes"] for component in components),
+        "disk": space,
+        "warnings": warnings,
+        "locks": locks,
+        "legacy_activity": legacy,
+    }

photo_pipeline/services/duplicates.py

@@ -52,6 +52,13 @@ from photo_pipeline.services import availability, hashing
 NEAR_MAX = 5
 SIMILAR_MAX = 10
 
+# Member paging (US07-06). A burst or a re-imported folder can put thousands of
+# assets in one cluster; review looks at a few at a time, so neither the list view
+# nor the detail view may load them all.
+MEMBER_PAGE = 100
+MAX_MEMBER_PAGE = 500
+SNAPSHOT_MEMBER_PREVIEW = 20
+
 
 class Method(str, Enum):
     EXACT = "exact"
@@ -520,39 +527,65 @@ class DuplicateService:
             items = [self._snapshot(session, c.id) for c in rows]
         return {"items": items, "total": int(total or 0), "limit": limit, "offset": offset}
 
-    def get_cluster(self, cluster_id: str) -> dict | None:
-        """Cluster detail enriched with per-member asset evidence for comparison."""
+    def get_cluster(
+        self, cluster_id: str, *, limit: int = MEMBER_PAGE, offset: int = 0
+    ) -> dict | None:
+        """Cluster detail enriched with per-member asset evidence for comparison.
+
+        Members are paged and their evidence is loaded in batches (US07-06). A
+        cluster of a few thousand near-identical frames is a real shape for a phone
+        library, and the review screen only ever shows a handful at a time: loading
+        every member — each with its own asset, thumbnail, and location query — made
+        opening such a cluster cost thousands of round trips and megabytes of JSON.
+        """
+        limit = max(1, min(limit, MAX_MEMBER_PAGE))
+        offset = max(0, offset)
         with self._session_factory() as session:
             cluster = session.get(DuplicateCluster, cluster_id)
             if cluster is None:
                 return None
+            member_total = int(
+                session.scalar(
+                    select(func.count())
+                    .select_from(DuplicateMember)
+                    .where(DuplicateMember.cluster_id == cluster_id)
+                )
+                or 0
+            )
+            rows = list(
+                session.execute(
+                    select(DuplicateMember)
+                    .where(DuplicateMember.cluster_id == cluster_id)
+                    .order_by(DuplicateMember.asset_id)
+                    .limit(limit)
+                    .offset(offset)
+                ).scalars()
+            )
+            evidence = self._member_evidence(session, [row.asset_id for row in rows])
             members = []
-            for member in session.execute(
-                select(DuplicateMember).where(DuplicateMember.cluster_id == cluster_id)
-            ).scalars():
-                asset = session.get(Asset, member.asset_id)
+            for member in rows:
                 try:
-                    evidence = json.loads(member.evidence) if member.evidence else {}
+                    member_evidence = json.loads(member.evidence) if member.evidence else {}
                 except json.JSONDecodeError:
-                    evidence = {}
+                    member_evidence = {}
+                asset, offline = evidence[member.asset_id]
                 members.append(
                     {
                         "asset_id": member.asset_id,
                         "role": member.role,
                         "distance": member.distance,
-                        "evidence": evidence,
+                        "evidence": member_evidence,
                         "current_path": asset.current_path if asset else None,
                         "byte_size": asset.byte_size if asset else None,
                         "phash": asset.phash if asset else None,
-                        **self._offline_evidence(session, asset),
+                        **offline,
                     }
                 )
-            members.sort(key=lambda m: m["asset_id"])
             # A full-resolution comparison of an offline original is impossible; the
-            # UI asks for that named medium instead of guessing (concept §9).
-            mount_required = sorted(
-                {m["archive_location"] for m in members if m["requires_mount"]}
-            )
+            # UI asks for that named medium instead of guessing (concept §9). The
+            # answer covers the whole cluster, not just this page, so a mount is not
+            # discovered halfway through a review.
+            mount_required = self._mount_required(session, cluster_id)
             return {
                 "id": cluster.id,
                 "method": cluster.method,
@@ -564,10 +597,74 @@ class DuplicateService:
                 "requires_confirmation": cluster.method == Method.PERCEPTUAL.value,
                 "mount_required": mount_required,
                 "members": members,
+                "member_total": member_total,
+                "limit": limit,
+                "offset": offset,
             }
 
-    def _offline_evidence(self, session, asset: Asset | None) -> dict:
-        """What review can still rely on when a member's original is not readable."""
+    def _member_evidence(self, session, asset_ids: list[str]) -> dict:
+        """``{asset_id: (asset, offline_evidence)}`` for one page, in three queries."""
+        if not asset_ids:
+            return {}
+        assets = {
+            asset.id: asset
+            for asset in session.execute(
+                select(Asset).where(Asset.id.in_(asset_ids))
+            ).scalars()
+        }
+        previews: dict[str, list] = {}
+        for thumbnail in session.execute(
+            select(Thumbnail).where(Thumbnail.asset_id.in_(asset_ids))
+        ).scalars():
+            previews.setdefault(thumbnail.asset_id, []).append(thumbnail)
+        location_ids = {
+            asset.archive_location_id for asset in assets.values() if asset.archive_location_id
+        }
+        locations = (
+            {
+                location.id: location
+                for location in session.execute(
+                    select(ArchiveLocation).where(ArchiveLocation.id.in_(location_ids))
+                ).scalars()
+            }
+            if location_ids
+            else {}
+        )
+        return {
+            asset_id: (
+                assets.get(asset_id),
+                self._offline_evidence(
+                    assets.get(asset_id),
+                    locations=locations,
+                    thumbnails=previews.get(asset_id, []),
+                ),
+            )
+            for asset_id in asset_ids
+        }
+
+    def _mount_required(self, session, cluster_id: str) -> list[str]:
+        """Archive media whose originals this cluster needs, across every member."""
+        rows = session.execute(
+            select(ArchiveLocation.name)
+            .select_from(DuplicateMember)
+            .join(Asset, Asset.id == DuplicateMember.asset_id)
+            .join(ArchiveLocation, ArchiveLocation.id == Asset.archive_location_id)
+            .where(
+                DuplicateMember.cluster_id == cluster_id,
+                Asset.availability_state == availability.ARCHIVED_OFFLINE,
+            )
+            .distinct()
+        ).scalars()
+        return sorted(rows)
+
+    def _offline_evidence(
+        self, asset: Asset | None, *, locations: dict, thumbnails: list
+    ) -> dict:
+        """What review can still rely on when a member's original is not readable.
+
+        Takes the already-loaded locations and thumbnails for its page rather than
+        querying per member (US07-06).
+        """
         if asset is None:
             return {
                 "availability_state": None,
@@ -577,12 +674,8 @@ class DuplicateService:
                 "preview": {"state": "missing", "protected": False},
                 "requires_mount": False,
             }
-        location = (
-            session.get(ArchiveLocation, asset.archive_location_id)
-            if asset.archive_location_id
-            else None
-        )
-        preview = self._preview_evidence(session, asset.id)
+        location = locations.get(asset.archive_location_id)
+        preview = self._preview_evidence(thumbnails)
         archived = asset.availability_state in availability.ARCHIVED
         return {
             "availability_state": asset.availability_state,
@@ -598,10 +691,7 @@ class DuplicateService:
         }
 
     @staticmethod
-    def _preview_evidence(session, asset_id: str) -> dict:
-        rows = list(
-            session.execute(select(Thumbnail).where(Thumbnail.asset_id == asset_id)).scalars()
-        )
+    def _preview_evidence(rows: list) -> dict:
         ready = [r for r in rows if r.state == "ready" and r.path]
         if ready:
             best = max(ready, key=lambda r: (bool(r.protected), r.size or 0))
@@ -693,11 +783,29 @@ class DuplicateService:
                 session.delete(link)
 
     def _snapshot(self, session, cluster_id) -> dict:
+        """A cluster and a *bounded* preview of its members.
+
+        The list view shows a count and a few ids; a snapshot that loaded every
+        member turned one page of 200 clusters into hundreds of thousands of rows
+        (US07-06). ``member_total`` is the honest count either way.
+        """
         cluster = session.get(DuplicateCluster, cluster_id)
+        member_total = int(
+            session.scalar(
+                select(func.count())
+                .select_from(DuplicateMember)
+                .where(DuplicateMember.cluster_id == cluster_id)
+            )
+            or 0
+        )
         members = session.execute(
-            select(DuplicateMember).where(DuplicateMember.cluster_id == cluster_id)
+            select(DuplicateMember)
+            .where(DuplicateMember.cluster_id == cluster_id)
+            .order_by(DuplicateMember.asset_id)
+            .limit(SNAPSHOT_MEMBER_PREVIEW)
         ).scalars()
         return {
+            "member_total": member_total,
             "id": cluster.id,
             "method": cluster.method,
             "confidence": cluster.confidence,

photo_pipeline/services/exif_checkpoint.py

@@ -0,0 +1,187 @@
+"""The EXIF checkpoint every metadata stage ends with (concept §3, US07-03).
+
+A stage does not own a file's metadata; it owns a few fields in it. So writing is
+never "set these tags" — it is:
+
+    snapshot everything → write only the owned fields → read everything back
+    → prove the owned fields landed → prove nothing else moved
+    → refresh the file hash → record the projection
+
+Non-destructive here means *semantic* preservation: exiftool may rewrite the whole
+container, so the file's bytes, size, and timestamps legitimately change. What may
+not change is any field this stage does not own. When one does, the checkpoint is
+``divergent``: the result is recorded, the stage is **not** marked verified, and
+nothing is silently repaired — a later stage that needs verified metadata (upload)
+therefore stays blocked until a human looks.
+
+``failed`` is the third outcome and is deliberately distinct: exiftool missing, an
+unreadable file, or a write that did not take is not evidence that metadata is fine.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import uuid
+from dataclasses import dataclass
+from datetime import datetime, timezone
+
+from photo_pipeline.faults import EXIF_WRITTEN, maybe_fault
+from photo_pipeline.integrations import exiftool
+from photo_pipeline.models import ExifProjection
+from photo_pipeline.services import hashing
+
+# The tags the safety and analysis stages may change. Matched on the tag name, so
+# every group carries the same rule (IPTC:Keywords, XMP:XMP-dc:Subject, ...).
+OWNED_TAGS = frozenset({"Keywords", "Subject"})
+
+# Not metadata about the picture: filesystem facts, the digest that necessarily
+# moves whenever IPTC does, and the structural tags exiftool has to create the first
+# time it writes an IPTC or XMP block. Comparing these would report every write as
+# divergent and make the signal worthless.
+VOLATILE_PREFIXES = ("File:System:", "ExifTool:")
+VOLATILE_KEYS = frozenset(
+    {
+        "File:CurrentIPTCDigest",
+        "IPTC:ApplicationRecordVersion",
+        "XMP:XMP-x:XMPToolkit",
+        "XMP:XMP-xmp:MetadataDate",
+    }
+)
+
+VERIFIED = "verified"
+DIVERGENT = "divergent"
+FAILED = "failed"
+
+
+@dataclass(frozen=True)
+class CheckpointResult:
+    state: str  # verified | divergent | failed
+    changed_fields: tuple[str, ...] = ()
+    sha256: str | None = None
+    # exiftool rewrites the container, so the file's size moves with its hash. Both
+    # are inventory facts about the current bytes and both have to be refreshed
+    # together, or the next stage compares against a size that no longer exists
+    # (US07-07: a rename plan blocked itself forever after any EXIF write).
+    byte_size: int | None = None
+    verified_at: datetime | None = None
+    reason: str | None = None
+
+    @property
+    def verified(self) -> bool:
+        return self.state == VERIFIED
+
+
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def is_owned(key: str) -> bool:
+    return key.rsplit(":", 1)[-1] in OWNED_TAGS
+
+
+def is_volatile(key: str) -> bool:
+    return key in VOLATILE_KEYS or key.startswith(VOLATILE_PREFIXES)
+
+
+def compare(before: dict, after: dict) -> tuple[str, ...]:
+    """Fields outside this stage's ownership whose value did not survive the write.
+
+    Additions count: a tag that appears out of nowhere is as much a divergence as a
+    tag that disappeared — both mean the write did more than it was asked to.
+    """
+    keys = set(before) | set(after)
+    return tuple(
+        sorted(
+            key
+            for key in keys
+            if not is_owned(key)
+            and not is_volatile(key)
+            and before.get(key) != after.get(key)
+        )
+    )
+
+
+def owned_values(snapshot: dict) -> set[str]:
+    """Lowercased Keywords/Subject values across every group in the snapshot."""
+    values: set[str] = set()
+    for key, value in snapshot.items():
+        if not is_owned(key):
+            continue
+        items = value if isinstance(value, list) else [value]
+        values.update(str(item).strip().lower() for item in items if item is not None)
+    return values
+
+
+def run(
+    path: str,
+    *,
+    add: tuple[str, ...] = (),
+    remove: tuple[str, ...] = (),
+) -> CheckpointResult:
+    """Write the owned keywords for one asset and verify the whole file around them."""
+    before = exiftool.read_all(path)
+    if before is None:
+        return CheckpointResult(FAILED, reason="metadata_unreadable")
+
+    if not exiftool.apply_keywords(path, add=add, remove=remove):
+        return CheckpointResult(FAILED, reason="write_failed")
+
+    # The file on disk has changed; nothing about it is recorded yet. A crash here
+    # is the worst case for metadata, so it is a fault control point (US07-04).
+    maybe_fault(EXIF_WRITTEN)
+
+    after = exiftool.read_all(path)
+    if after is None:
+        return CheckpointResult(FAILED, reason="readback_unreadable")
+
+    present = owned_values(after)
+    wanted = {value.strip().lower() for value in add}
+    unwanted = {value.strip().lower() for value in remove}
+    if not wanted <= present or (unwanted & present):
+        return CheckpointResult(FAILED, reason="owned_fields_not_written")
+
+    changed = compare(before, after)
+    sha256 = hashing.sha256_file(path)
+    byte_size = os.path.getsize(path)
+    if changed:
+        return CheckpointResult(
+            DIVERGENT, changed_fields=changed, sha256=sha256, byte_size=byte_size
+        )
+    return CheckpointResult(VERIFIED, sha256=sha256, byte_size=byte_size, verified_at=_now())
+
+
+def record(
+    session_factory,
+    *,
+    asset_id: str,
+    stage: str,
+    result: CheckpointResult,
+    add: tuple[str, ...] = (),
+    remove: tuple[str, ...] = (),
+) -> None:
+    """Persist the projection for ``(asset_id, stage)`` — one current row per pair.
+
+    The row is what makes divergence durable and reviewable rather than a log line
+    that scrolled away.
+    """
+    with session_factory() as session:
+        row = session.get(ExifProjection, (asset_id, stage))
+        if row is None:
+            row = ExifProjection(asset_id=asset_id, stage=stage, id=str(uuid.uuid4()))
+            session.add(row)
+        row.projection_version = (row.projection_version or 0) + 1
+        row.desired_json = json.dumps({"add": list(add), "remove": list(remove)})
+        row.divergent_fields = json.dumps(list(result.changed_fields))
+        row.result_file_sha256 = result.sha256
+        row.state = result.state
+        row.error_code = result.reason
+        row.verified_at = result.verified_at
+        row.updated_at = _now()
+        session.commit()
+
+
+def state_for(session_factory, asset_id: str, stage: str) -> str | None:
+    with session_factory() as session:
+        row = session.get(ExifProjection, (asset_id, stage))
+        return row.state if row else None

photo_pipeline/services/hashing.py

@@ -16,7 +16,9 @@ from __future__ import annotations
 import hashlib
 from pathlib import Path
 
-from PIL import Image, ImageOps
+from PIL import ImageOps
+
+from photo_pipeline import imaging
 
 PIXEL_HASH_VERSION = 1
 PHASH_VERSION = 1
@@ -41,7 +43,9 @@ def _digest_file(path: Path | str, digest) -> str:
 
 
 def pixel_sha256(path: Path | str) -> str:
-    with Image.open(path) as image:
+    # Bounded decode: a header claiming a billion pixels is refused before it is
+    # ever turned into memory (US07-03).
+    with imaging.open_image(path) as image:
         oriented = ImageOps.exif_transpose(image)
         rgb = oriented.convert("RGB")
         header = f"{PIXEL_HASH_VERSION}:{rgb.width}x{rgb.height}:".encode()
@@ -68,7 +72,9 @@ def phash(path: Path | str) -> str:
     import numpy as np
     from scipy.fftpack import dct
 
-    with Image.open(path) as image:
+    from PIL import Image
+
+    with imaging.open_image(path) as image:
         small = image.convert("L").resize((32, 32), Image.LANCZOS)
         matrix = np.asarray(small, dtype=np.float64)
     transformed = dct(dct(matrix, axis=0), axis=1)

photo_pipeline/services/jobs.py

@@ -236,17 +236,32 @@ class JobService:
                 raise InvalidTransition(f"{job.state} -> {to_state}")
             if worker_id is not None and job.lease_owner not in (None, worker_id):
                 raise JobConflict(f"job {job_id} owned by {job.lease_owner}, not {worker_id}")
-            job.state = to_state
-            job.version += 1
-            job.updated_at = now
+
+            # Compare-and-set on the version this decision was made against. Without
+            # it a transition validated against a row that has since been claimed,
+            # cancelled, or finished would overwrite that newer state (concept §16
+            # database rule 6) — a cancel racing a claim used to un-claim a running
+            # job and leave the worker finalizing a job it no longer owned.
+            values = {
+                "state": to_state,
+                "version": job.version + 1,
+                "updated_at": now,
+            }
             if error:
-                job.error_code, job.error_message = error
+                values["error_code"], values["error_message"] = error
             if to_state in TERMINAL_STATES:
-                job.finished_at = now
-                job.lease_owner = None
-                job.lease_expires_at = None
+                values.update(finished_at=now, lease_owner=None, lease_expires_at=None)
+            result = session.execute(
+                update(Job).where(Job.id == job_id, Job.version == job.version).values(**values)
+            )
+            if result.rowcount != 1:
+                session.rollback()
+                raise JobConflict(
+                    f"job {job_id} changed while transitioning to {to_state}; retry"
+                )
             self._event(session, job_id, f"state:{to_state}", error[1] if error else None)
             session.commit()
+            session.expire_all()  # the core UPDATE bypassed the identity map
             return self._snapshot(session, job_id)
 
     def cancel(self, job_id: str) -> dict:

photo_pipeline/services/legacy_import.py

@@ -0,0 +1,220 @@
+"""Import the last path-keyed CSV state into the database (US07-01).
+
+``nsfwtag`` cached its safety scores in ``nsfw_scores.csv`` next to the library:
+one ``path,nsfw_score`` row per photo, four decimals, unreadable rows dropped. That
+file stops being a source of truth when the CLI is archived, so its scores are
+imported once — as scored-but-unreviewed ``safety_reviews`` rows on the stable
+``assets.id`` each path resolves to — and the CSV is left untouched on disk as its
+own backup.
+
+The import is deliberately conservative, because a score is evidence about a photo
+and a path is not an identity:
+
+- a row whose path matches no known asset is **unmatched**, never a new asset;
+- an asset a human already reviewed is **never** touched: a score is evidence, a
+  decision is a judgement, and an import may not overwrite the second with the
+  first — the difference is reported as a conflict instead;
+- a row for an asset that already carries a score is **skipped** unless
+  ``overwrite`` is asked for, and a differing score is reported either way;
+- an unparsable score or a duplicate path is **rejected** with its reason;
+- rerunning changes nothing that is already imported.
+
+Everything it did — and everything it refused to do — comes back as a
+reconciliation report, which the caller can persist next to the database. The
+donor's own reader is the specification for the format (donor ledger:
+``nt-score-cache``).
+"""
+
+from __future__ import annotations
+
+import csv
+import json
+import uuid
+from collections import Counter
+from dataclasses import asdict, dataclass, field
+from datetime import datetime, timezone
+from pathlib import Path
+
+from sqlalchemy import select
+from sqlalchemy.orm import sessionmaker
+
+from photo_pipeline.models import Asset, AssetPath, SafetyReview
+
+REPORT_VERSION = 1
+# The donor wrote scores as four-decimal strings; anything outside 0..1 was never
+# something it produced, so it is data corruption rather than a score.
+SCORE_RANGE = (0.0, 1.0)
+# Recorded as the reviewer so an imported score is never mistaken for a judgement.
+REVIEWER = "legacy-csv-import"
+
+
+@dataclass
+class ImportReport:
+    source: str
+    rows: int = 0
+    imported: int = 0
+    skipped_existing: int = 0
+    unchanged: int = 0
+    reviewed: int = 0
+    unmatched: list[str] = field(default_factory=list)
+    rejected: list[dict] = field(default_factory=list)
+    conflicts: list[dict] = field(default_factory=list)
+    generated_at: str = ""
+    schema_version: int = REPORT_VERSION
+
+    def to_dict(self) -> dict:
+        return asdict(self)
+
+    @property
+    def counts(self) -> dict[str, int]:
+        return {
+            "rows": self.rows,
+            "imported": self.imported,
+            "skipped_existing": self.skipped_existing,
+            "unchanged": self.unchanged,
+            "reviewed": self.reviewed,
+            "unmatched": len(self.unmatched),
+            "rejected": len(self.rejected),
+            "conflicts": len(self.conflicts),
+        }
+
+
+class LegacyImportService:
+    def __init__(self, session_factory: sessionmaker) -> None:
+        self._session_factory = session_factory
+
+    def import_nsfw_scores(
+        self, csv_path: Path | str, *, overwrite: bool = False, dry_run: bool = False
+    ) -> ImportReport:
+        """Import ``nsfw_scores.csv`` onto asset identity and report what happened."""
+        path = Path(csv_path)
+        report = ImportReport(source=str(path))
+        if not path.is_file():
+            report.rejected.append({"path": str(path), "reason": "csv_missing"})
+            return self._stamp(report)
+
+        with self._session_factory() as session:
+            # Paths are matched against every occurrence an asset ever had, so a
+            # photo scored before a rename is still recognised.
+            by_path = self._path_index(session)
+            latest = self._latest_reviews(session)
+            seen: Counter[str] = Counter()
+
+            for row in self._rows(path, report):
+                report.rows += 1
+                raw_path, raw_score = row
+                seen[raw_path] += 1
+                if seen[raw_path] > 1:
+                    report.rejected.append({"path": raw_path, "reason": "duplicate_path"})
+                    continue
+                score = _parse_score(raw_score)
+                if score is None:
+                    report.rejected.append(
+                        {"path": raw_path, "reason": "unparsable_score", "value": raw_score}
+                    )
+                    continue
+                asset_id = by_path.get(raw_path)
+                if asset_id is None:
+                    report.unmatched.append(raw_path)
+                    continue
+
+                current = latest.get(asset_id)
+                if current is not None and current.decision:
+                    # A human decided this one; the CSV is older evidence.
+                    report.reviewed += 1
+                    if current.score is None or abs(current.score - score) >= 1e-9:
+                        report.conflicts.append(
+                            {
+                                "path": raw_path,
+                                "asset_id": asset_id,
+                                "current": current.score,
+                                "decision": current.decision,
+                                "csv": score,
+                            }
+                        )
+                    continue
+                if current is not None and current.score is not None:
+                    if abs(current.score - score) < 1e-9:
+                        report.unchanged += 1
+                        continue
+                    report.conflicts.append(
+                        {
+                            "path": raw_path,
+                            "asset_id": asset_id,
+                            "current": current.score,
+                            "decision": None,
+                            "csv": score,
+                        }
+                    )
+                    if not overwrite:
+                        report.skipped_existing += 1
+                        continue
+                session.add(
+                    SafetyReview(
+                        id=str(uuid.uuid4()),
+                        asset_id=asset_id,
+                        score=score,
+                        reviewer=REVIEWER,
+                    )
+                )
+                latest[asset_id] = SafetyReview(asset_id=asset_id, score=score)
+                report.imported += 1
+
+            if dry_run:
+                session.rollback()
+            else:
+                session.commit()
+        return self._stamp(report)
+
+    @staticmethod
+    def _rows(path: Path, report: ImportReport):
+        """Yield ``(path, score)`` pairs, tolerating the donor's own sloppiness."""
+        with open(path, newline="", encoding="utf-8", errors="replace") as handle:
+            for row in csv.DictReader(handle):
+                raw_path = (row.get("path") or "").strip()
+                if not raw_path:
+                    report.rejected.append({"path": "", "reason": "missing_path"})
+                    continue
+                yield raw_path, row.get("nsfw_score")
+
+    @staticmethod
+    def _latest_reviews(session) -> dict[str, SafetyReview]:
+        """The current safety row per asset — latest wins, as everywhere else."""
+        latest: dict[str, SafetyReview] = {}
+        for review in session.scalars(select(SafetyReview).order_by(SafetyReview.created_at)):
+            latest[review.asset_id] = review
+        return latest
+
+    @staticmethod
+    def _path_index(session) -> dict[str, str]:
+        """Every path an asset is or was known by → its stable id."""
+        index: dict[str, str] = {}
+        for asset_id, path in session.execute(select(AssetPath.asset_id, AssetPath.path)):
+            index.setdefault(path, asset_id)
+        for asset_id, path in session.execute(select(Asset.id, Asset.current_path)):
+            if path:
+                index[path] = asset_id  # the current path wins over a closed one
+        return index
+
+    @staticmethod
+    def _stamp(report: ImportReport) -> ImportReport:
+        report.generated_at = datetime.now(timezone.utc).isoformat()
+        return report
+
+
+def write_report(report: ImportReport, directory: Path) -> Path:
+    """Persist the reconciliation report; the import is not evidence until it is."""
+    directory.mkdir(parents=True, exist_ok=True)
+    path = directory / "legacy-nsfw-import.json"
+    path.write_text(json.dumps(report.to_dict(), indent=2, sort_keys=True), encoding="utf-8")
+    return path
+
+
+def _parse_score(value) -> float | None:
+    try:
+        score = float(value)
+    except (TypeError, ValueError):
+        return None
+    if not SCORE_RANGE[0] <= score <= SCORE_RANGE[1]:
+        return None
+    return score

photo_pipeline/services/library.py

@@ -15,10 +15,9 @@ from __future__ import annotations
 
 import json
 import re
-from collections import Counter
 from pathlib import Path
 
-from sqlalchemy import and_, func, or_, select
+from sqlalchemy import String, and_, case, cast, func, or_, select, text
 from sqlalchemy.orm import sessionmaker
 
 from photo_pipeline.models import AnalysisResult, Asset
@@ -71,38 +70,75 @@ class LibraryService:
         return {"rows": rows, "total": total, "offset": offset, "limit": limit}
 
     def stats(self) -> dict:
+        """Library-wide totals, aggregated in SQL (US07-06).
+
+        This page used to load every analysis row — object, tags, and all — to count
+        them in Python, which cost half a second at 100k assets and grew from there.
+        Only the album breakdown still walks rows, and only their path and status:
+        SQLite has no ``dirname``, and two short strings per asset is cheap.
+        """
         with self._session_factory() as session:
             status = dict(
                 session.execute(
                     select(AnalysisResult.status, func.count()).group_by(AnalysisResult.status)
                 ).all()
             )
-            rows = list(
-                session.execute(
-                    select(AnalysisResult, Asset.current_path).join(
-                        Asset, Asset.id == AnalysisResult.asset_id
-                    )
-                )
-            )
             albums: dict[str, dict] = {}
-            tag_counts: Counter = Counter()
-            year_counts: Counter = Counter()
-            people: Counter = Counter()
-            errors = []
-            for result, path in rows:
+            for path, row_status in session.execute(
+                select(Asset.current_path, AnalysisResult.status).join(
+                    Asset, Asset.id == AnalysisResult.asset_id
+                )
+            ):
                 album = _album_of(path)
                 bucket = albums.setdefault(album, {"album": album, "done": 0, "total": 0})
                 bucket["total"] += 1
-                if result.status in DONE:
+                if row_status in DONE:
                     bucket["done"] += 1
-                for tag in _tags(result.tags):
-                    tag_counts[tag] += 1
-                if result.approx_year is not None:
-                    year_counts[result.approx_year] += 1
-                if result.people_count is not None:
-                    people["3+" if result.people_count >= 3 else str(result.people_count)] += 1
-                if result.status == "error":
-                    errors.append({"path": path, "error": result.error_message})
+            year_counts = dict(
+                session.execute(
+                    select(AnalysisResult.approx_year, func.count())
+                    .where(AnalysisResult.approx_year.is_not(None))
+                    .group_by(AnalysisResult.approx_year)
+                ).all()
+            )
+            people = dict(
+                session.execute(
+                    select(
+                        case(
+                            (AnalysisResult.people_count >= 3, "3+"),
+                            else_=cast(AnalysisResult.people_count, String),
+                        ),
+                        func.count(),
+                    )
+                    .where(AnalysisResult.people_count.is_not(None))
+                    .group_by(
+                        case(
+                            (AnalysisResult.people_count >= 3, "3+"),
+                            else_=cast(AnalysisResult.people_count, String),
+                        )
+                    )
+                ).all()
+            )
+            # SQLite's JSON1 counts the tag arrays where they are: parsing 50k JSON
+            # strings in Python to keep the top 40 is the definition of doing work
+            # the database already does. Malformed tags are skipped, not fatal.
+            tag_counts = session.execute(
+                text(
+                    "SELECT tag.value AS value, count(*) AS total "
+                    "FROM analysis_results, json_each(analysis_results.tags) AS tag "
+                    "WHERE analysis_results.tags IS NOT NULL "
+                    "AND json_valid(analysis_results.tags) "
+                    "GROUP BY tag.value ORDER BY total DESC, value LIMIT 40"
+                )
+            ).all()
+            errors = [
+                {"path": path, "error": message}
+                for path, message in session.execute(
+                    select(Asset.current_path, AnalysisResult.error_message)
+                    .join(Asset, Asset.id == AnalysisResult.asset_id)
+                    .where(AnalysisResult.status == "error")
+                )
+            ]
         return {
             "total": sum(status.values()),
             "status": status,
@@ -111,7 +147,7 @@ class LibraryService:
             "season": self._facet("season"),
             "people": [{"value": v, "count": n} for v, n in sorted(people.items())],
             "years": [{"value": y, "count": year_counts[y]} for y in sorted(year_counts)],
-            "top_tags": [{"value": t, "count": n} for t, n in tag_counts.most_common(40)],
+            "top_tags": [{"value": t, "count": n} for t, n in tag_counts],
             "albums": sorted(albums.values(), key=lambda d: d["album"]),
             "errors": sorted(errors, key=lambda e: e["path"] or ""),
         }

photo_pipeline/services/release.py

@@ -0,0 +1,391 @@
+"""The release gate, the real-library dry run, and the approval that unlocks
+mutation (US07-07, concept §18 release gates).
+
+Three things live here because they are one decision:
+
+1. **The gate** — one command that provisions an isolated stack, runs every suite in
+   a fixed order, and retains versioned evidence with checksums. A release is not
+   "the tests passed on my machine last Tuesday"; it is a report that says which
+   revision, which suites, how long, and what the artefacts hash to.
+2. **The dry run** — a strictly read-only pass over the real photo library that
+   answers "what would this application do to it?" before it is allowed to do
+   anything. It opens no file for writing, creates no database rows, and touches no
+   metadata; it counts, classifies, and reconciles against whatever the database
+   already knows.
+3. **The approval** — a person reads that report and signs it off for exactly the
+   library roots it describes. Until then, with
+   ``PHOTO_PIPELINE_REQUIRE_DRY_RUN_APPROVAL`` set, every mutating request is
+   refused. Change the roots, or produce a newer report, and the approval no longer
+   matches: it approves *that* reconciliation, not the idea of mutating.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import os
+import subprocess
+import sys
+import time
+from collections import Counter
+from dataclasses import dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+
+from photo_pipeline import path_policy
+from photo_pipeline.config import Config
+
+SCHEMA_VERSION = 1
+APPROVAL_NAME = "dry-run-approval.json"
+CHECKSUMS_NAME = "CHECKSUMS.sha256"
+REPORT_NAME = "release-report.json"
+
+# The suites, in the order a failure is cheapest to read: units before the stacks
+# they compose. ``label`` is what the report and the operator see.
+STAGES: tuple[tuple[str, tuple[str, ...]], ...] = (
+    ("unit", ("tests/unit",)),
+    ("characterization", ("tests/characterization",)),
+    ("integration", ("tests/integration",)),
+    ("browser", ("tests/e2e",)),
+)
+
+# Skips the gate accepts, because they describe the machine rather than the code.
+ALLOWED_SKIP_REASONS = ("exiftool not installed", "root ignores directory permissions")
+
+
+class ReleaseError(RuntimeError):
+    pass
+
+
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def sha256_file(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as handle:
+        for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def sha256_bytes(payload: bytes) -> str:
+    return hashlib.sha256(payload).hexdigest()
+
+
+def revision() -> str | None:
+    """The commit this gate ran against, when the tree is a git checkout."""
+    try:
+        result = subprocess.run(
+            ["git", "rev-parse", "HEAD"],
+            capture_output=True,
+            text=True,
+            timeout=10,
+            cwd=str(Path(__file__).resolve().parents[2]),
+        )
+    except (OSError, subprocess.SubprocessError):
+        return None
+    return result.stdout.strip() or None
+
+
+# ── the story matrix ─────────────────────────────────────────────────────────
+
+
+def story_matrix(repo: Path | None = None) -> dict:
+    """Every backlog story, and how it is covered.
+
+    A story is ``delivered`` (mapped to test files that exist) or ``planned`` (an
+    accepted, not-yet-implemented story). Anything else — a story file nobody
+    mapped, or a mapping to a file that is gone — is a hole in the matrix, and the
+    gate fails on it rather than reporting a green run over missing coverage.
+    """
+    repo = repo or Path(__file__).resolve().parents[2]
+    traceability = json.loads((repo / "tests" / "story_traceability.json").read_text())
+    mapped: dict[str, list[str]] = traceability["stories"]
+    planned: list[str] = traceability.get("planned", [])
+    stories = sorted(
+        "-".join(path.stem.split("-")[:2])
+        for path in (repo / "delivery_backlog" / "stories").glob("US*.md")
+    )
+
+    missing_tests = [
+        f"{story}: {rel}"
+        for story, files in mapped.items()
+        for rel in files
+        if not (repo / rel).is_file()
+    ]
+    unmapped = [s for s in stories if s not in mapped and s not in planned]
+    unknown = [s for s in list(mapped) + planned if s not in stories]
+    overlap = sorted(set(mapped) & set(planned))
+    return {
+        "stories": len(stories),
+        "delivered": sorted(mapped),
+        "planned": sorted(planned),
+        "problems": [
+            *(f"story with no tests and not planned: {s}" for s in unmapped),
+            *(f"mapped test file is missing — {entry}" for entry in missing_tests),
+            *(f"mapped story is not in the backlog: {s}" for s in unknown),
+            *(f"story is both delivered and planned: {s}" for s in overlap),
+        ],
+    }
+
+
+# ── the gate ─────────────────────────────────────────────────────────────────
+
+
+@dataclass
+class StageResult:
+    label: str
+    command: list[str]
+    returncode: int
+    seconds: float
+    summary: str
+    skipped: list[str]
+
+    def as_dict(self) -> dict:
+        return {
+            "stage": self.label,
+            "command": self.command,
+            "returncode": self.returncode,
+            "seconds": round(self.seconds, 2),
+            "summary": self.summary,
+            "skipped": self.skipped,
+            "ok": self.returncode == 0,
+        }
+
+
+def _run_stage(label: str, paths: tuple[str, ...], *, repo: Path, log_dir: Path) -> StageResult:
+    command = [sys.executable, "-m", "pytest", *paths, "-q", "-rs"]
+    started = time.monotonic()
+    result = subprocess.run(command, cwd=str(repo), capture_output=True, text=True)
+    elapsed = time.monotonic() - started
+    output = result.stdout + result.stderr
+    (log_dir / f"{label}.log").write_text(output)
+    lines = [line for line in output.splitlines() if line.strip()]
+    summary = lines[-1] if lines else ""
+    skipped = [line for line in lines if line.startswith("SKIPPED")]
+    return StageResult(label, command, result.returncode, elapsed, summary, skipped)
+
+
+def unexpected_skips(results: list[StageResult]) -> list[str]:
+    """Skips the gate will not accept: everything but the documented environment ones."""
+    return [
+        line
+        for result in results
+        for line in result.skipped
+        if not any(reason in line for reason in ALLOWED_SKIP_REASONS)
+    ]
+
+
+def run_gate(
+    config: Config,
+    *,
+    output: Path | str | None = None,
+    stages: tuple[tuple[str, tuple[str, ...]], ...] = STAGES,
+    repo: Path | None = None,
+) -> dict:
+    """Run every suite in an isolated stack and retain checksummed evidence.
+
+    The stack is isolated by construction: each pytest run builds its own temporary
+    data directories and libraries, so the gate never reads or writes the operator's
+    photos. What it keeps afterwards is the report, the per-stage logs, and a
+    checksum file over both.
+    """
+    repo = repo or Path(__file__).resolve().parents[2]
+    directory = Path(output) if output else Path(config.data_dir) / "release" / _now().strftime(
+        "%Y%m%dT%H%M%SZ"
+    )
+    logs = directory / "logs"
+    logs.mkdir(parents=True, exist_ok=True)
+
+    matrix = story_matrix(repo)
+    results = [_run_stage(label, paths, repo=repo, log_dir=logs) for label, paths in stages]
+    skips = unexpected_skips(results)
+
+    report = {
+        "schema_version": SCHEMA_VERSION,
+        "started_at": _now().isoformat(),
+        "revision": revision(),
+        "python": sys.version.split()[0],
+        "platform": os.uname().sysname,
+        "matrix": matrix,
+        "stages": [result.as_dict() for result in results],
+        "unexpected_skips": skips,
+        "failures": [result.label for result in results if result.returncode != 0],
+    }
+    report["ok"] = not report["failures"] and not matrix["problems"] and not skips
+    report["finished_at"] = _now().isoformat()
+
+    (directory / REPORT_NAME).write_text(json.dumps(report, indent=2))
+    # The evidence is only evidence if it can be shown to be the evidence that was
+    # produced. Checksums are the honest version of "signed" without a key: a real
+    # signature belongs to whatever key management the release actually has.
+    checksums = "\n".join(
+        f"{sha256_file(path)}  {path.relative_to(directory)}"
+        for path in sorted(directory.rglob("*"))
+        if path.is_file() and path.name != CHECKSUMS_NAME
+    )
+    (directory / CHECKSUMS_NAME).write_text(checksums + "\n")
+    report["evidence"] = str(directory)
+    return report
+
+
+# ── the real-library dry run ─────────────────────────────────────────────────
+
+
+def dry_run(config: Config, *, roots: tuple[Path, ...] | None = None) -> dict:
+    """Read-only reconciliation of the configured library. Changes nothing.
+
+    Opens no file for writing, writes no database row, and reads only what
+    ``os.stat`` and the existing database already say. The point is to be able to
+    look at a real library — the one with the irreplaceable photos in it — and see
+    what the application believes about it before it is allowed to act.
+    """
+    roots = roots or tuple(Path(root) for root in config.library_roots)
+    if not roots:
+        raise ReleaseError("no library roots are configured")
+
+    by_extension: Counter = Counter()
+    folders: set[str] = set()
+    files: list[str] = []
+    unreadable: list[str] = []
+    excluded = 0
+    total_bytes = 0
+    for root in roots:
+        if not Path(root).is_dir():
+            raise ReleaseError(f"library root {root} is not a directory")
+        for path in sorted(Path(root).rglob("*")):
+            if path.is_dir():
+                # Never traverse into an excluded directory, and never report its
+                # contents: proving exclusion must not require opening it.
+                if path_policy.is_excluded(path):
+                    excluded += 1
+                continue
+            if path_policy.is_excluded(path):
+                continue
+            try:
+                stat = path.stat()
+            except OSError:
+                unreadable.append(str(path))
+                continue
+            files.append(str(path))
+            folders.add(str(path.parent))
+            by_extension[path.suffix.lower() or "(none)"] += 1
+            total_bytes += stat.st_size
+
+    known = _known_paths(config)
+    on_disk = set(files)
+    report = {
+        "schema_version": SCHEMA_VERSION,
+        "generated_at": _now().isoformat(),
+        "revision": revision(),
+        "library_roots": [str(root) for root in roots],
+        "files": len(files),
+        "folders": len(folders),
+        "bytes": total_bytes,
+        "excluded_directories": excluded,
+        "unreadable": unreadable,
+        "by_extension": dict(sorted(by_extension.items())),
+        "reconciliation": {
+            "known_to_database": len(known),
+            "already_registered": len(on_disk & known),
+            "new_to_the_application": len(on_disk - known),
+            "recorded_but_absent": sorted(known - on_disk)[:100],
+            "recorded_but_absent_total": len(known - on_disk),
+        },
+        "mutation": "none — this pass is read-only",
+    }
+    report["checksum"] = sha256_bytes(
+        json.dumps(report, sort_keys=True).encode("utf-8")
+    )
+    return report
+
+
+def _known_paths(config: Config) -> set[str]:
+    """Current asset paths the database holds, or an empty set if there is none."""
+    if not config.database_path.exists():
+        return set()
+    from sqlalchemy import select
+
+    from photo_pipeline.db import create_db_engine, create_session_factory
+    from photo_pipeline.models import Asset
+
+    engine = create_db_engine(config.database_url)
+    try:
+        with create_session_factory(engine)() as session:
+            return {
+                path
+                for path in session.scalars(select(Asset.current_path))
+                if path is not None
+            }
+    except Exception:
+        return set()
+    finally:
+        engine.dispose()
+
+
+# ── the approval ─────────────────────────────────────────────────────────────
+
+
+def approval_path(config: Config) -> Path:
+    return Path(config.data_dir) / APPROVAL_NAME
+
+
+def approve(config: Config, report: dict | Path | str, *, approver: str) -> dict:
+    """Record that a person read this reconciliation and accepts mutation for it."""
+    if isinstance(report, (str, Path)):
+        report = json.loads(Path(report).read_text())
+    if "checksum" not in report:
+        raise ReleaseError("this is not a dry-run report: it has no checksum")
+    record = {
+        "schema_version": SCHEMA_VERSION,
+        "approved_at": _now().isoformat(),
+        "approved_by": approver,
+        "report_checksum": report["checksum"],
+        "library_roots": report["library_roots"],
+        "files": report["files"],
+        "revision": report.get("revision"),
+    }
+    path = approval_path(config)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_text(json.dumps(record, indent=2))
+    return record
+
+
+def mutation_blockers(config: Config) -> list[dict]:
+    """Why mutation must stay refused, or an empty list.
+
+    Only enforced when ``require_dry_run_approval`` is configured — the loopback
+    developer setup keeps working unchanged, and an operator turns this on before
+    pointing the application at the library they cannot replace.
+    """
+    if not config.require_dry_run_approval:
+        return []
+    path = approval_path(config)
+    if not path.exists():
+        return [
+            {
+                "code": "dry_run_not_approved",
+                "message": (
+                    "run `python -m photo_pipeline dry-run` and approve its report "
+                    "before mutation is enabled"
+                ),
+            }
+        ]
+    try:
+        record = json.loads(path.read_text())
+    except ValueError:
+        return [{"code": "approval_unreadable", "message": f"{path} is not readable JSON"}]
+    approved_roots = [str(root) for root in record.get("library_roots", [])]
+    configured = [str(root) for root in config.library_roots]
+    if sorted(approved_roots) != sorted(configured):
+        return [
+            {
+                "code": "approval_scope_mismatch",
+                "message": (
+                    f"the approval covers {approved_roots}, but the configured library "
+                    f"is {configured}; run a new dry run"
+                ),
+            }
+        ]
+    return []

photo_pipeline/services/rename_apply.py

@@ -46,9 +46,11 @@ from pathlib import Path
 from sqlalchemy import select
 from sqlalchemy.orm import sessionmaker
 
+from photo_pipeline.faults import maybe_fault
 from photo_pipeline.models import Asset, AssetPath, RenamePlan
 from photo_pipeline.services import hashing
 from photo_pipeline.services.rename_journal import (
+    ALLOWED_TRANSITIONS,
     MANUAL,
     RESUMABLE,
     JournalState,
@@ -83,18 +85,6 @@ def _now() -> datetime:
     return datetime.now(timezone.utc)
 
 
-def maybe_fault(state: str) -> None:
-    """Test-only crash barrier (concept §18 fault injection).
-
-    When ``PHOTO_PIPELINE_FAULT_AFTER`` names a journal state, the process dies
-    abruptly the moment that state has been persisted — modelling a real kill at
-    exactly that transition. Never set outside tests. Shared with the archive
-    transfer journal (US06-02), which uses the same env var and its own state names.
-    """
-    if os.environ.get("PHOTO_PIPELINE_FAULT_AFTER") == state:
-        os._exit(9)
-
-
 class RenameApplyService:
     def __init__(self, session_factory: sessionmaker, *, library_roots: tuple = ()) -> None:
         self._session_factory = session_factory
@@ -143,20 +133,10 @@ class RenameApplyService:
                 self._apply_one(operation, token=token, worker_id=worker_id)
                 applied += 1
             except PreconditionFailed as error:
-                self.journal.transition(
-                    operation["id"],
-                    JournalState.FAILED,
-                    fencing_token=token,
-                    error=(error.code, str(error)),
-                )
+                self._record_failure(operation["id"], token, error.code, str(error))
                 failed += 1
             except Exception as error:  # unexpected: record and stop touching disk
-                self.journal.transition(
-                    operation["id"],
-                    JournalState.FAILED,
-                    fencing_token=token,
-                    error=("apply_error", str(error)),
-                )
+                self._record_failure(operation["id"], token, "apply_error", str(error))
                 failed += 1
         state = self.journal.sync_plan_state(plan_id)
         return {
@@ -167,6 +147,26 @@ class RenameApplyService:
             "state": state,
         }
 
+    def _record_failure(self, operation_id: str, token: int, code: str, message: str) -> None:
+        """Record a failed operation in a state its journal can actually reach.
+
+        ``failed`` only makes sense while nothing has moved. Once the folder is at
+        its destination — a postcondition failure such as bytes edited during the
+        move — the operation is not "failed and forgotten": the disk changed and
+        the database followed, so it becomes ``rollback_required`` and waits for a
+        human (US07-04). Guessing an unreachable transition used to raise out of
+        ``apply`` and lose the record entirely.
+        """
+        current = self.journal.get(operation_id)["journal_state"]
+        target = (
+            JournalState.FAILED
+            if JournalState.FAILED in ALLOWED_TRANSITIONS.get(current, set())
+            else JournalState.ROLLBACK_REQUIRED
+        )
+        self.journal.transition(
+            operation_id, target, fencing_token=token, error=(code, message)
+        )
+
     def _apply_one(self, operation: dict, *, token: int, worker_id: str) -> None:
         source = Path(operation["source_path"])
         destination = Path(operation["destination_path"])

photo_pipeline/services/rename_journal.py

@@ -81,7 +81,17 @@ ALLOWED_TRANSITIONS = {
 
 TERMINAL_STATES = frozenset({JournalState.COMPLETE, JournalState.ROLLED_BACK})
 # States where the disk may already have been touched by this operation.
-UNSAFE_STATES = frozenset({JournalState.MOVING, JournalState.MOVED, JournalState.DATABASE_UPDATED})
+# ``rollback_required`` belongs here too (US07-04): the move happened and someone
+# has to decide what to do about it, so the library is not in a state another
+# mutation may build on.
+UNSAFE_STATES = frozenset(
+    {
+        JournalState.MOVING,
+        JournalState.MOVED,
+        JournalState.DATABASE_UPDATED,
+        JournalState.ROLLBACK_REQUIRED,
+    }
+)
 
 RESUMABLE = "resumable"
 ROLLBACK_SAFE = "rollback_safe"

photo_pipeline/services/restores.py

@@ -53,6 +53,7 @@ from sqlalchemy import select
 from sqlalchemy.orm import sessionmaker
 
 from photo_pipeline.config import Config
+from photo_pipeline.faults import maybe_fault
 from photo_pipeline.jobs.domain_handlers import ARCHIVE_LOCK, LIBRARY_WRITE_LOCK, UPLOAD_LOCK
 from photo_pipeline.models import ArchiveLocation, ArchiveOperation, ArchivePlan, Asset, AssetPath
 from photo_pipeline.path_policy import PathPolicyError, is_excluded, normalize_root, resolve_within
@@ -73,7 +74,7 @@ from photo_pipeline.services.archive_transfer import (
 from photo_pipeline.services.archives import ArchiveError
 from photo_pipeline.services.hashing import sha256_file
 from photo_pipeline.services.jobs import JobService
-from photo_pipeline.services.rename_apply import PreconditionFailed, maybe_fault
+from photo_pipeline.services.rename_apply import PreconditionFailed
 from photo_pipeline.services.rename_journal import RenameJournal
 
 PREFLIGHT_VERSION = 1
@@ -140,9 +141,11 @@ class RestoreService:
             report["blockers"].append(
                 _issue(
                     "insufficient_capacity",
+                    # No free-space number here: it drifts between two identical
+                    # preflights and the token is a digest of this text (US06-06).
                     f"{report['totals']['bytes']} B plus a "
                     f"{self._config.archive_free_space_reserve_bytes} B reserve do not fit in "
-                    f"{report['capacity']['free_bytes']} B of free space",
+                    "the library",
                 )
             )
         if not items:

photo_pipeline/services/safety.py

@@ -105,12 +105,11 @@ def exif_projection(decision: str) -> dict[str, list[str]]:
 import uuid
 from datetime import datetime, timezone
 
-from sqlalchemy import select
+from sqlalchemy import column, func, select
 from sqlalchemy.orm import sessionmaker
 
-from photo_pipeline.integrations import exiftool
-from photo_pipeline.models import Asset, SafetyReview
-from photo_pipeline.services import hashing
+from photo_pipeline.models import Asset, ExifProjection, SafetyReview
+from photo_pipeline.services import exif_checkpoint
 
 DECISIONS = {SFW, NSFW, "deferred"}
 
@@ -140,8 +139,8 @@ class SafetyService:
 
     # -- reads ----------------------------------------------------------------
     def _latest_by_asset(self, session) -> dict[str, SafetyReview]:
-        # Latest row per asset. Small local scale: order ascending, let later rows
-        # overwrite. ponytail: a windowed query if safety_reviews ever grows huge.
+        """The current review per asset, as ORM rows. Only for small, known sets —
+        every library-wide caller uses ``latest_reviews()`` in SQL instead."""
         latest: dict[str, SafetyReview] = {}
         for review in session.scalars(select(SafetyReview).order_by(SafetyReview.created_at)):
             latest[review.asset_id] = review
@@ -149,49 +148,99 @@ class SafetyService:
 
     def current_decision(self, asset_id: str) -> str | None:
         with self._session_factory() as session:
-            review = self._latest_by_asset(session).get(asset_id)
-            return review.decision if review else None
+            latest = latest_reviews().subquery()
+            return session.scalar(
+                select(latest.c.decision).where(latest.c.asset_id == asset_id)
+            )
 
     def counts(self) -> dict[str, int]:
-        """Decision breakdown over canonical, active assets — the workflow totals."""
+        """Decision breakdown over canonical, active assets — the workflow totals.
+
+        Aggregated in SQL: the workflow home asks for this on every load, and
+        materialising every asset and every review to count them cost hundreds of
+        milliseconds at 25k assets and would scale linearly from there (US07-06).
+        """
+        latest = latest_reviews().subquery()
         with self._session_factory() as session:
-            assets = list(session.scalars(_eligible_assets_query()))
-            latest = self._latest_by_asset(session)
-            out = {SFW: 0, NSFW: 0, "deferred": 0, "undecided": 0, "scored": 0}
-            for asset in assets:
-                review = latest.get(asset.id)
-                decision = review.decision if review else None
-                if decision in (SFW, NSFW, "deferred"):
-                    out[decision] += 1
-                else:
-                    out["undecided"] += 1
-                    if review and review.score is not None:
-                        out["scored"] += 1
-            return out
+            rows = session.execute(
+                select(
+                    func.coalesce(latest.c.decision, "undecided"),
+                    func.count(),
+                    func.count(latest.c.score),
+                )
+                .select_from(Asset)
+                .join(latest, latest.c.asset_id == Asset.id, isouter=True)
+                .where(
+                    Asset.canonical_asset_id.is_(None),
+                    Asset.availability_state == "active",
+                )
+                .group_by(func.coalesce(latest.c.decision, "undecided"))
+            ).all()
+        out = {SFW: 0, NSFW: 0, "deferred": 0, "undecided": 0, "scored": 0}
+        for decision, total, scored in rows:
+            if decision in (SFW, NSFW, "deferred"):
+                out[decision] += int(total)
+            else:
+                # Anything that is not one of the three decisions is undecided —
+                # including a score-only review, which is what "scored" counts.
+                out["undecided"] += int(total)
+                out["scored"] += int(scored)
+        return out
 
     def review_queue(self, state: str = "", limit: int = 100, offset: int = 0) -> dict:
-        """Assets for the review UI, filtered by ``state`` (undecided/sfw/nsfw/deferred)."""
+        """Assets for the review UI, filtered by ``state`` (undecided/sfw/nsfw/deferred).
+
+        Filtered, counted, and paged in SQL (US07-06): the queue for a large library
+        is thousands of rows and the reviewer sees one page of it.
+        """
+        latest = latest_reviews().subquery()
+        projections = (
+            select(ExifProjection.asset_id, ExifProjection.state.label("exif_state"))
+            .where(ExifProjection.stage == "safety")
+            .subquery()
+        )
+        effective = func.coalesce(latest.c.decision, "undecided")
+        query = (
+            select(
+                Asset.id,
+                Asset.current_path,
+                latest.c.score,
+                latest.c.decision,
+                latest.c.exif_verified_at,
+                projections.c.exif_state,
+            )
+            .select_from(Asset)
+            .join(latest, latest.c.asset_id == Asset.id, isouter=True)
+            .join(projections, projections.c.asset_id == Asset.id, isouter=True)
+            .where(
+                Asset.canonical_asset_id.is_(None),
+                Asset.availability_state == "active",
+            )
+        )
+        if state:
+            query = query.where(effective == state)
         with self._session_factory() as session:
-            assets = list(session.scalars(_eligible_assets_query().order_by(Asset.current_path)))
-            latest = self._latest_by_asset(session)
-            rows = []
-            for asset in assets:
-                review = latest.get(asset.id)
-                decision = review.decision if review else None
-                effective = decision or "undecided"
-                if state and state != effective:
-                    continue
-                rows.append(
-                    {
-                        "asset_id": asset.id,
-                        "current_path": asset.current_path,
-                        "score": review.score if review else None,
-                        "decision": decision,
-                        "suggested": classify(review.score) if review and review.score is not None else None,
-                        "exif_verified": bool(review and review.exif_verified_at),
-                    }
-                )
-            return {"total": len(rows), "items": rows[offset : offset + limit]}
+            total = int(
+                session.scalar(select(func.count()).select_from(query.subquery())) or 0
+            )
+            rows = session.execute(
+                query.order_by(Asset.current_path).limit(limit).offset(offset)
+            ).all()
+        return {
+            "total": total,
+            "items": [
+                {
+                    "asset_id": asset_id,
+                    "current_path": current_path,
+                    "score": score,
+                    "decision": decision,
+                    "suggested": classify(score) if score is not None else None,
+                    "exif_verified": bool(exif_verified_at),
+                    "exif_state": exif_state,
+                }
+                for asset_id, current_path, score, decision, exif_verified_at, exif_state in rows
+            ],
+        }
 
     def scorable_asset_ids(self) -> list[str]:
         """Canonical active assets with a path — the items a scoring job enqueues."""
@@ -245,15 +294,28 @@ class SafetyService:
 
         exif_verified_at = None
         result_sha256 = None
+        result_byte_size = None
         if write_exif and decision in (SFW, NSFW) and path:
             ops = exif_projection(decision)
-            if exiftool.apply_keywords(path, add=ops["add"], remove=ops["remove"]):
-                # Read back: the chosen keyword present, the opposite absent.
-                keywords = exiftool.read_keyword_sets([path]).get(path, set())
-                opposite = NSFW if decision == SFW else SFW
-                if decision in keywords and opposite not in keywords:
-                    exif_verified_at = _now()
-                    result_sha256 = hashing.sha256_file(path)
+            # The full checkpoint: write the owned keyword, read the whole file back,
+            # and prove every field this stage does not own survived. A divergent
+            # result is recorded and left alone — it must not count as verified, so
+            # upload stays blocked until a human decides (US07-03).
+            result = exif_checkpoint.run(
+                path, add=tuple(ops["add"]), remove=tuple(ops["remove"])
+            )
+            exif_checkpoint.record(
+                self._session_factory,
+                asset_id=asset_id,
+                stage="safety",
+                result=result,
+                add=tuple(ops["add"]),
+                remove=tuple(ops["remove"]),
+            )
+            if result.verified:
+                exif_verified_at = result.verified_at
+                result_sha256 = result.sha256
+                result_byte_size = result.byte_size
 
         now = _now()
         with self._session_factory() as session:
@@ -272,6 +334,8 @@ class SafetyService:
             if result_sha256:
                 asset = session.get(Asset, asset_id)
                 asset.current_sha256 = result_sha256
+                if result_byte_size is not None:
+                    asset.byte_size = result_byte_size
             session.commit()
         return {
             "asset_id": asset_id,
@@ -281,6 +345,34 @@ class SafetyService:
         }
 
 
+def latest_reviews():
+    """One row per asset: its current safety review, chosen in SQL.
+
+    ``safety_reviews`` is append-only, so "the decision" is the newest row for an
+    asset. A window function picks it without loading the table; ``rowid`` breaks a
+    same-timestamp tie the same way the previous last-write-wins loop did.
+    """
+    ranked = (
+        select(
+            SafetyReview.asset_id,
+            SafetyReview.decision,
+            SafetyReview.score,
+            SafetyReview.exif_verified_at,
+            func.row_number()
+            .over(
+                partition_by=SafetyReview.asset_id,
+                order_by=(SafetyReview.created_at.desc(), column("rowid").desc()),
+            )
+            .label("rank"),
+        )
+        .select_from(SafetyReview)
+        .subquery()
+    )
+    return select(
+        ranked.c.asset_id, ranked.c.decision, ranked.c.score, ranked.c.exif_verified_at
+    ).where(ranked.c.rank == 1)
+
+
 def _eligible_assets_query():
     """Canonical, active assets — the safety stage runs only on these.
 

photo_pipeline/services/thumbnails.py

@@ -20,15 +20,16 @@ pa-imaging).
 
 from __future__ import annotations
 
+import os
 import uuid
 from pathlib import Path
 
-from PIL import Image, ImageOps, UnidentifiedImageError
+from PIL import Image, ImageOps
 from sqlalchemy import select
 from sqlalchemy.exc import IntegrityError
 from sqlalchemy.orm import sessionmaker
 
-from photo_pipeline import path_policy
+from photo_pipeline import imaging, path_policy
 from photo_pipeline.config import Config
 from photo_pipeline.models import Asset, Thumbnail
 from photo_pipeline.services import availability
@@ -42,6 +43,9 @@ except Exception:  # pragma: no cover
     pass
 
 SIZES = (256, 512, 1280)
+# Every in-flight render writes one of these beside its destination; startup
+# cleanup recognises exactly this pattern and nothing else.
+TEMP_SUFFIX = ".tmp"
 THUMB_VERSION = 1
 THUMB_FORMAT = "webp"
 # The size kept as durable comparison evidence for archived assets (concept §9).
@@ -129,7 +133,10 @@ class ThumbnailService:
                 raise ThumbnailUnavailable(f"asset {asset_id} has no readable file")
             source = str(source)
             if source == asset.current_path:
-                self._validate_path(source)  # archive roots lie outside the library
+                # Render the *resolved* path the check approved: revalidating and then
+                # reopening the original name would let a symlink swapped in between
+                # the two steps decide which bytes are served (US07-02).
+                source = str(self._validate_path(source))  # archive roots lie outside
 
         # Rendering happens outside the DB session (no transaction held during I/O).
         try:
@@ -195,20 +202,16 @@ class ThumbnailService:
                 session.commit()
 
     # ── path safety ──────────────────────────────────────────────────────────
-    def _validate_path(self, current_path: str) -> None:
-        path = Path(current_path)
-        if path_policy.is_excluded(path):
-            raise PathNotAllowed(f"excluded path: {current_path}")
-        roots = self._config.library_roots
-        if not roots:
-            return
-        for root in roots:
-            try:
-                path_policy.resolve_within(Path(root), path)
-                return
-            except path_policy.PathPolicyError:
-                continue
-        raise PathNotAllowed(f"path outside configured roots: {current_path}")
+    def _validate_path(self, current_path: str) -> Path:
+        """The resolved path to read, or ``PathNotAllowed``.
+
+        The message names no path: a refusal is returned to the browser, and where
+        the library lives is not the caller's business (US07-02).
+        """
+        try:
+            return path_policy.resolve_in_roots(self._config.library_roots, current_path)
+        except path_policy.PathPolicyError as error:
+            raise PathNotAllowed(str(error)) from None
 
     # ── cache key + rendering ──────────────────────────────────────────────────
     @staticmethod
@@ -223,28 +226,33 @@ class ThumbnailService:
         return self._cache_dir / safe[:2] / f"{safe}.{THUMB_FORMAT}"
 
     def _render(self, source: str, size: int, cache_key: str) -> dict:
+        """Decode bounded, orient, colour-manage, resize, and write atomically.
+
+        The temporary file is removed on every failure path: a decoder that dies
+        halfway through ``save`` would otherwise leave a stray ``.tmp`` in the cache
+        forever (US07-03).
+        """
+        destination = self._cache_path(cache_key)
+        destination.parent.mkdir(parents=True, exist_ok=True)
+        tmp = destination.with_name(f".{destination.name}.{uuid.uuid4().hex}{TEMP_SUFFIX}")
         try:
-            with Image.open(source) as image:
-                width, height = image.size
-                if width * height > self._config.thumbnail_max_pixels:
-                    raise ImageTooLarge(
-                        f"{width}x{height} exceeds {self._config.thumbnail_max_pixels} px"
-                    )
+            with imaging.open_image(source, max_pixels=self._config.thumbnail_max_pixels) as image:
+                imaging.draft(image, size)  # JPEG decodes near the target size
                 oriented = ImageOps.exif_transpose(image)
                 mode = "RGBA" if _has_alpha(oriented) else "RGB"
-                converted = oriented.convert(mode)
+                converted = imaging.to_srgb(oriented, mode=mode)
                 converted.thumbnail((size, size), Image.LANCZOS)
                 out_width, out_height = converted.size
-                destination = self._cache_path(cache_key)
-                destination.parent.mkdir(parents=True, exist_ok=True)
-                tmp = destination.with_name(f".{destination.name}.{uuid.uuid4().hex}.tmp")
                 converted.save(tmp, format="WEBP", quality=82, method=4)
-        except ImageTooLarge:
+        except imaging.ImageTooLarge as error:
+            tmp.unlink(missing_ok=True)
+            raise ImageTooLarge(str(error)) from None
+        except imaging.UndecodableImage as error:
+            tmp.unlink(missing_ok=True)
+            raise UnsupportedImage(str(error)) from None
+        except Exception:
+            tmp.unlink(missing_ok=True)
             raise
-        except (UnidentifiedImageError, OSError, ValueError) as error:
-            raise UnsupportedImage(f"cannot decode {source}: {error}") from error
-
-        import os
 
         os.replace(tmp, destination)
         return {
@@ -254,6 +262,28 @@ class ThumbnailService:
             "format": THUMB_FORMAT,
         }
 
+    def cleanup_temp_files(self) -> int:
+        """Remove leftover render temporaries, and only those.
+
+        Runs at startup, where the concept is explicit: clean *recognized* stale
+        temporary files, never arbitrary ones. The pattern is this service's own —
+        a dot-prefixed name inside the managed cache directory ending in
+        ``.tmp`` — so nothing outside the cache and nothing a user put there can
+        match. Returns how many were removed.
+        """
+        if not self._cache_dir.is_dir():
+            return 0
+        removed = 0
+        for leftover in self._cache_dir.rglob(f".*{TEMP_SUFFIX}"):
+            if not leftover.is_file() or leftover.is_symlink():
+                continue
+            try:
+                leftover.unlink()
+                removed += 1
+            except OSError:  # pragma: no cover - a racing render already won
+                pass
+        return removed
+
     # ── persistence ────────────────────────────────────────────────────────────
     def _record_ready(
         self, cache_key: str, asset_id: str, size: int, rendered: dict, *, protected: bool = False

photo_pipeline/services/upload_batches.py

@@ -37,6 +37,7 @@ from sqlalchemy import select, update
 from sqlalchemy.orm import sessionmaker
 
 from photo_pipeline.config import Config
+from photo_pipeline.faults import UPLOAD_ACCEPTED, maybe_fault
 from photo_pipeline.integrations import immich_go
 from photo_pipeline.models import UploadBatch, UploadItem
 from photo_pipeline.services.hashing import sha1_file
@@ -236,6 +237,10 @@ class UploadBatchService:
             error = ("uploader_failed", f"immich-go exited with {result['exit_code']}")
             item_state = ItemState.FAILED
 
+        # The uploader is done and Immich may already hold every file, but nothing
+        # about that is durable yet — the control point for "accepted, outcome not
+        # recorded" (US07-04). Recovery must answer ``unknown_requires_verification``.
+        maybe_fault(UPLOAD_ACCEPTED)
         self._finish(batch_id, token=token, state=state, error=error, result=result)
         if item_state:
             self._set_items(batch_id, item_state)

photo_pipeline/services/workflow.py

@@ -47,7 +47,9 @@ class WorkflowService:
             active = self._active_job(session)
 
         safety = SafetyService(self._session_factory).counts()
-        analysis = AnalysisService(self._session_factory).counts()
+        # Reuse the confirmed-SFW total just computed: resolving the current decision
+        # of every asset is the expensive part of this page (US07-06).
+        analysis = AnalysisService(self._session_factory).counts(eligible=safety[_SFW])
         undecided_clusters = cluster_states.get("open", 0) + cluster_states.get("reopened", 0)
 
         stages = [

pyproject.toml

@@ -8,22 +8,38 @@ dependencies = [
     "sqlalchemy>=2.0",
     "alembic>=1.13",
     "pydantic>=2.7",
-]
-
-[project.optional-dependencies]
-test = [
-    "pytest>=8",
-    "httpx>=0.27",
+    # Imaging is runtime, not test-only: thumbnails decode through Pillow, and the
+    # perceptual hash is a DCT over the decoded pixels (services/hashing.py).
     "pillow>=10",
     "numpy>=1.26",
     "scipy>=1.11",
+]
+
+[project.optional-dependencies]
+# The cloud vision provider. Optional because the analysis stage is the only thing
+# that needs it, and a local review-only install should not pull an API client.
+vision = ["openai>=1.30"]
+test = [
+    "pytest>=8",
+    "httpx>=0.27",
     "playwright>=1.40",
     "pytest-playwright>=0.4",
 ]
+
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools]
+# The importable application. ``migrations`` and ``work_item`` live beside it but
+# are not part of the package; without this, an editable install cannot guess.
+packages = ["photo_pipeline"]
 # Browser end-to-end tests also require: python -m playwright install chromium
 
 [tool.ruff]
 line-length = 100
+# The frozen CLI archive is evidence, not code under maintenance (US07-01).
+extend-exclude = ["legacy_cli_archive"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]

tests/_artifacts.py

@@ -0,0 +1,96 @@
+"""Failure artifacts for the fault and race suites (US07-04).
+
+A randomized concurrency failure that leaves nothing behind is a failure nobody
+can diagnose: the temporary library is deleted, the database goes with it, and the
+seed that produced the interleaving is gone. So when a test fails, everything
+needed to reproduce and read it is copied out of the temporary directory:
+
+    <artifacts>/<test id>/
+        seeds.json     recorded properties (``race_seed``) and the failing test id
+        manifest.json  every file under the temporary directory: path, size, sha256
+        files/...      the databases (with -wal/-shm), journals, and logs themselves
+
+The manifest covers the whole tree — including files too large or too private to
+copy — so a missing or unexpected file is still visible afterwards. Copying is
+bounded by ``MAX_COPY_BYTES``: artifacts must not turn a failing CI run into a
+disk-full one.
+
+Set ``PHOTO_PIPELINE_TEST_ARTIFACTS`` to choose the destination; the default is
+``.artifacts/`` in the repository root.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import os
+import re
+import shutil
+from pathlib import Path
+
+REPO = Path(__file__).resolve().parents[1]
+DEFAULT_DIR = REPO / ".artifacts"
+MAX_COPY_BYTES = 25 * 1024 * 1024
+# Databases (and their write-ahead logs), journals exported as files, and logs.
+COPY_SUFFIXES = (".db", ".db-wal", ".db-shm", ".sqlite", ".log", ".json", ".jsonl", ".argv")
+
+
+def artifacts_dir() -> Path:
+    return Path(os.environ.get("PHOTO_PIPELINE_TEST_ARTIFACTS", DEFAULT_DIR))
+
+
+def _slug(test_id: str) -> str:
+    return re.sub(r"[^A-Za-z0-9._-]+", "_", test_id)[:120]
+
+
+def manifest(root: Path) -> list[dict]:
+    """Every file under ``root``: relative path, byte size, and SHA-256.
+
+    The filesystem state at the moment of failure — what was moved, what was left
+    behind, what was half-written.
+    """
+    entries = []
+    for path in sorted(root.rglob("*")):
+        if not path.is_file() or path.is_symlink():
+            continue
+        try:
+            body = path.read_bytes()
+        except OSError as error:
+            entries.append({"path": str(path.relative_to(root)), "error": str(error)})
+            continue
+        entries.append(
+            {
+                "path": str(path.relative_to(root)),
+                "bytes": len(body),
+                "sha256": hashlib.sha256(body).hexdigest(),
+            }
+        )
+    return entries
+
+
+def collect(root: Path, test_id: str, *, properties: dict | None = None) -> Path:
+    """Copy the evidence for one failed test out of ``root``. Returns its directory."""
+    destination = artifacts_dir() / _slug(test_id)
+    files = destination / "files"
+    files.mkdir(parents=True, exist_ok=True)
+
+    entries = manifest(root)
+    (destination / "manifest.json").write_text(json.dumps(entries, indent=1))
+    (destination / "seeds.json").write_text(
+        json.dumps({"test": test_id, "properties": properties or {}}, indent=1)
+    )
+
+    budget = MAX_COPY_BYTES
+    for path in sorted(root.rglob("*")):
+        if not path.is_file() or path.is_symlink():
+            continue
+        if not path.name.endswith(COPY_SUFFIXES):
+            continue
+        size = path.stat().st_size
+        if size > budget:
+            continue  # the manifest still records it; the copy is what is skipped
+        target = files / path.relative_to(root)
+        target.parent.mkdir(parents=True, exist_ok=True)
+        shutil.copy2(path, target)
+        budget -= size
+    return destination

tests/characterization/conftest.py

@@ -14,6 +14,11 @@ import pytest
 from PIL import Image
 
 REPO = Path(__file__).resolve().parents[2]
+# The donors are frozen in the read-only archive (US07-01). Only this suite — and
+# the parity test that compares against them — puts that directory on sys.path;
+# production never does, which tests/unit/test_legacy_archive.py enforces.
+ARCHIVED_SOURCES = REPO / "legacy_cli_archive" / "src"
+sys.path.insert(0, str(ARCHIVED_SOURCES))
 sys.path.insert(0, str(REPO))
 
 EXIFTOOL = shutil.which("exiftool")

tests/characterization/test_donor_ledger.py

@@ -1,13 +1,20 @@
-"""Ledger lint (US01-01): every donor-ledger row must carry a real source
-reference, a target location, and either existing characterization test IDs or
-a real pending backlog story."""
+"""Ledger lint (US01-01, extended by US07-01).
+
+Every donor-ledger row must carry a real source reference — now inside the frozen
+archive — a target location, and either existing characterization test IDs or a
+real pending backlog story. Since archival (US07-01) a row may also be ``resolved``:
+its replacement has shipped, ``parity`` names tests that exist and prove it, and
+``delta`` states every intentional difference. Nothing may quietly become
+"finished" without one of those two."""
 import re
 from pathlib import Path
 
 import yaml
 
 REPO = Path(__file__).resolve().parents[2]
-LEDGER = REPO / "donor_ledger.yaml"
+ARCHIVE = REPO / "legacy_cli_archive"
+LEDGER = ARCHIVE / "donor_ledger.yaml"
+ARCHIVED_SOURCES = ARCHIVE / "src"
 STORIES = REPO / "delivery_backlog" / "stories"
 TESTS_DIR = Path(__file__).resolve().parent
 
@@ -15,7 +22,7 @@ CLASSIFICATIONS = {"reuse", "extract", "refactor", "replace"}
 REQUIRED_AREAS = {"discovery", "hashing", "imaging", "nsfw", "vision", "exif",
                   "database", "ui", "configuration", "logging", "cancellation",
                   "error"}
-STATUSES = {"characterized", "pending"}
+STATUSES = {"characterized", "resolved", "pending"}
 
 
 def load_rows():
@@ -49,10 +56,11 @@ def test_rows_have_required_fields_and_unique_ids():
 
 
 def test_source_references_resolve():
+    """Source paths are relative to the archive: the donors moved there, whole."""
     for r in load_rows():
         src = r["source"]
-        f = REPO / src["file"]
-        assert f.is_file(), f"{r['id']}: source file {src['file']} missing"
+        f = ARCHIVED_SOURCES / src["file"]
+        assert f.is_file(), f"{r['id']}: source file {src['file']} missing from the archive"
         text = f.read_text(encoding="utf-8")
         for sym in src["symbols"]:
             assert sym in text, f"{r['id']}: symbol {sym!r} not found in {src['file']}"
@@ -63,7 +71,9 @@ def test_rows_have_tests_or_pending_story():
     for r in load_rows():
         tests = r.get("tests", [])
         pending = r.get("pending_story")
-        assert tests or pending, f"{r['id']}: neither tests nor pending_story"
+        parity = r.get("parity", [])
+        assert tests or pending or parity, \
+            f"{r['id']}: neither characterization tests, parity tests, nor a pending story"
         for t in tests:
             assert t in known_tests, f"{r['id']}: unknown test id {t}"
         if pending:
@@ -73,16 +83,58 @@ def test_rows_have_tests_or_pending_story():
             assert tests, f"{r['id']}: characterized rows need test ids"
 
 
+def test_resolved_rows_name_their_parity_or_their_delta():
+    """A resolved row is a claim that the behavior is handled. It has to say how:
+    tests that prove the replacement, or a stated difference — usually both."""
+    for r in load_rows():
+        if r["status"] != "resolved":
+            assert "parity" not in r, f"{r['id']}: parity on a non-resolved row"
+            continue
+        parity = r.get("parity", [])
+        delta = r.get("delta")
+        assert parity or delta, f"{r['id']}: resolved without parity tests or a delta"
+        if delta:
+            assert len(str(delta).strip()) >= 20, f"{r['id']}: delta too thin to be a reason"
+        for ref in parity:
+            rel, _, func = ref.partition("::")
+            path = REPO / rel
+            assert path.is_file(), f"{r['id']}: parity test file {rel} missing"
+            assert f"def {func}" in path.read_text(encoding="utf-8"), \
+                f"{r['id']}: parity test {ref} not found"
+
+
+def test_pending_rows_are_the_only_unfinished_work():
+    """The ledger is the honest list of what has not been carried over: a pending
+    row names the story that will, and that story must still be open work."""
+    for r in load_rows():
+        if r["status"] != "pending":
+            continue
+        story = r.get("pending_story")
+        assert story, f"{r['id']}: pending without a story"
+        assert list(STORIES.glob(f"{story}-*.md")), f"{r['id']}: unknown story {story}"
+
+
+def test_every_target_module_exists():
+    """A row is only finished if the thing it points at is really there."""
+    for r in load_rows():
+        if r["status"] == "pending":
+            continue
+        modules = re.findall(r"photo_pipeline/[\w/]+\.py", str(r["target"]))
+        for module in modules:
+            assert (REPO / module).is_file(), f"{r['id']}: target {module} does not exist"
+
+
 def test_all_required_areas_covered():
     covered = {r["area"] for r in load_rows()}
     assert REQUIRED_AREAS <= covered, f"uncovered areas: {REQUIRED_AREAS - covered}"
     assert covered <= REQUIRED_AREAS, f"unknown areas: {covered - REQUIRED_AREAS}"
 
 
-def test_no_legacy_file_moved():
-    # US01-01 explicitly forbids moving/archiving donors; the ledger's source
-    # files must all still exist at their original locations.
+def test_every_donor_is_in_the_archive_and_nowhere_else():
+    """US01-01 forbade archiving before characterization; US07-01 requires it after.
+    Each donor exists exactly once — frozen, in the archive."""
     for donor in ("photo_analyzer.py", "nsfwtag/scoring.py", "nsfwtag/exif.py",
                   "nsfwtag/server.py", "webapp/query.py", "webapp/runner.py",
                   "webapp/server.py"):
-        assert (REPO / donor).is_file(), f"donor moved: {donor}"
+        assert (ARCHIVED_SOURCES / donor).is_file(), f"donor missing from archive: {donor}"
+        assert not (REPO / donor).exists(), f"donor still live at the repo root: {donor}"

tests/conftest.py

@@ -1,4 +1,16 @@
-"""Make the repository root importable for the pipeline test suites."""
+"""Make the repository root importable for the pipeline test suites, give every
+suite the application session the API requires since US07-02, and keep the evidence
+of any failure (US07-04).
+
+The suites drive the API the way the browser does — module-level ``httpx`` calls and
+``TestClient`` — so instead of threading a cookie through several hundred call sites,
+both clients bootstrap the session themselves exactly like ``frontend/js/api.js``:
+fetch ``/api/v1/session`` once, then send the cookie plus the CSRF header, and
+re-bootstrap once on 401 (a restarted server issues a new session).
+
+Security tests deliberately bypass this by constructing their own ``httpx.Client``;
+only the module-level helpers are wrapped.
+"""
 
 import sys
 from pathlib import Path
@@ -6,3 +18,112 @@ from pathlib import Path
 REPO = Path(__file__).resolve().parents[1]
 if str(REPO) not in sys.path:
     sys.path.insert(0, str(REPO))
+
+import httpx
+import httpx._api  # httpx.get/post resolve `request` in this module
+import pytest
+from starlette.testclient import TestClient
+
+from photo_pipeline.api.security import CSRF_HEADER, SESSION_COOKIE
+
+_SESSIONS: dict[str, tuple[str, str]] = {}  # base url -> (session id, csrf token)
+
+
+def _base(url) -> str:
+    parts = httpx.URL(str(url))
+    return f"{parts.scheme}://{parts.netloc.decode()}"
+
+
+def _bootstrap(base: str) -> tuple[str, str]:
+    if base not in _SESSIONS:
+        response = httpx.Client(timeout=10).get(f"{base}/api/v1/session")
+        _SESSIONS[base] = (
+            response.cookies.get(SESSION_COOKIE, ""),
+            response.json().get("csrf_token", ""),
+        )
+    return _SESSIONS[base]
+
+
+def session_client(base_url: str, **kwargs) -> httpx.Client:
+    """An ``httpx.Client`` that has bootstrapped its own application session.
+
+    For suites that keep one client for a whole journey; it holds its own cookie, so
+    a client made after a server restart picks up the new session by construction.
+    """
+    client = httpx.Client(base_url=base_url, **kwargs)
+    client.headers[CSRF_HEADER] = client.get("/api/v1/session").json()["csrf_token"]
+    return client
+
+
+def _authorized(url, headers, cookies):
+    session_id, token = _bootstrap(_base(url))
+    headers = dict(headers or {})
+    headers.setdefault(CSRF_HEADER, token)
+    cookies = dict(cookies or {})
+    cookies.setdefault(SESSION_COOKIE, session_id)
+    return headers, cookies
+
+
+@pytest.fixture(autouse=True, scope="session")
+def _api_session():
+    real_request, real_stream = httpx._api.request, httpx._api.stream
+    real_client_request, real_client_init = TestClient.request, TestClient.__init__
+
+    def request(method, url, *, headers=None, cookies=None, **kwargs):
+        sent, jar = _authorized(url, headers, cookies)
+        response = real_request(method, url, headers=sent, cookies=jar, **kwargs)
+        if response.status_code == 401:
+            _SESSIONS.pop(_base(url), None)
+            sent, jar = _authorized(url, headers, cookies)
+            response = real_request(method, url, headers=sent, cookies=jar, **kwargs)
+        return response
+
+    def stream(method, url, *, headers=None, cookies=None, **kwargs):
+        sent, jar = _authorized(url, headers, cookies)
+        return real_stream(method, url, headers=sent, cookies=jar, **kwargs)
+
+    def client_init(self, app, *args, base_url="http://127.0.0.1", **kwargs):
+        # The default "http://testserver" is not a local host, which is exactly what
+        # the Host check refuses; in-process tests are still same-origin callers.
+        real_client_init(self, app, *args, base_url=base_url, **kwargs)
+
+    def client_request(self, method, url, *, headers=None, **kwargs):
+        if CSRF_HEADER not in self.headers:
+            response = real_client_request(self, "GET", "/api/v1/session")
+            self.headers[CSRF_HEADER] = response.json()["csrf_token"]
+        return real_client_request(self, method, url, headers=headers, **kwargs)
+
+    httpx._api.request, httpx.request = request, request
+    httpx._api.stream, httpx.stream = stream, stream
+    TestClient.request, TestClient.__init__ = client_request, client_init
+    yield
+    httpx._api.request, httpx.request = real_request, real_request
+    httpx._api.stream, httpx.stream = real_stream, real_stream
+    TestClient.request, TestClient.__init__ = real_client_request, real_client_init
+
+
+@pytest.hookimpl(hookwrapper=True)
+def pytest_runtest_makereport(item, call):
+    """On failure, keep the temporary library, database, journals, logs, and seed.
+
+    ``tmp_path`` is deleted a few runs later and its contents are the only record
+    of what a randomized or fault-injected failure actually did, so they are copied
+    out while they still exist (US07-04).
+    """
+    outcome = yield
+    report = outcome.get_result()
+    if report.when != "call" or not report.failed:
+        return
+    root = item.funcargs.get("tmp_path")
+    if root is None or not Path(root).exists():
+        return
+    from tests._artifacts import collect
+
+    try:
+        destination = collect(
+            Path(root), item.nodeid, properties=dict(getattr(report, "user_properties", []))
+        )
+    except OSError as error:  # never let evidence collection mask the real failure
+        report.sections.append(("failure artifacts", f"could not be collected: {error}"))
+    else:
+        report.sections.append(("failure artifacts", str(destination)))

tests/e2e/_pipeline_harness.py

@@ -494,6 +494,16 @@ class ArchiveStack:
         self.server.stop()
         self.server.start()
 
+    def restart_worker(self, *, extra_env: dict[str, str] | None = None) -> None:
+        """Replace the worker — a healthy one after a crashed one, by default."""
+        if self.worker is not None and self.worker.poll() is None:
+            self.worker.kill()
+            self.worker.wait(timeout=10)
+        self.worker = start_worker(
+            self.seeded,
+            extra_env={"PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": "0", **(extra_env or {})},
+        )
+
     def stop(self) -> None:
         if self.worker is not None:
             self.worker.kill()

tests/e2e/test_crash_recovery.py

@@ -0,0 +1,274 @@
+"""Process death at the newer control points (US07-04, concept §18).
+
+The rename and archive journals already prove crash safety at each of their
+transitions (tests/integration/test_rename_recovery.py,
+tests/integration/test_archive_recovery.py). The three transitions covered here
+are the remaining ones where a kill leaves the world and the database disagreeing:
+
+- ``exif:written``   — keywords are on disk, nothing about them is recorded;
+- ``upload:accepted``— the uploader finished, no outcome is stored;
+- ``job:item_done``  — one item is durably done, the job is not finished.
+
+Each test kills a real child process at the barrier and then asserts what a
+restart does: resume idempotently, or say plainly that a human has to look. Never
+"assume it worked".
+"""
+
+from __future__ import annotations
+
+import os
+import shutil
+import subprocess
+import sys
+import uuid
+from datetime import datetime, timezone
+from pathlib import Path
+
+import numpy as np
+import pytest
+from PIL import Image
+from sqlalchemy import select
+
+from photo_pipeline.config import Config
+from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
+from photo_pipeline.faults import EXIF_WRITTEN, JOB_ITEM_DONE, UPLOAD_ACCEPTED
+from photo_pipeline.integrations import exiftool
+from photo_pipeline.jobs.worker import Worker
+from photo_pipeline.models import Asset, SafetyReview
+from photo_pipeline.services import exif_checkpoint, hashing
+from photo_pipeline.services.jobs import ItemState, JobService, JobState
+from photo_pipeline.services.safety import SafetyService
+from photo_pipeline.services.upload_batches import BatchState, UploadBatchService
+from photo_pipeline.services.upload_verification import retry_blockers
+from photo_pipeline.services.uploads import UploadService
+from tests.e2e._pipeline_harness import (
+    SILENT_UPLOADER,
+    FakeImmich,
+    fake_uploader,
+    mark_upload_ready,
+    seed_album,
+)
+
+REPO = Path(__file__).resolve().parents[2]
+NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
+
+
+def _child(script: str, *args: str, barrier: str, tmp_path: Path) -> None:
+    """Run ``script`` in a child that dies at ``barrier``; assert it really died."""
+    path = tmp_path / f"child_{barrier.replace(':', '_')}.py"
+    path.write_text(script.format(repo=str(REPO)))
+    env = dict(os.environ)
+    env["PHOTO_PIPELINE_FAULT_AFTER"] = barrier
+    result = subprocess.run(
+        [sys.executable, str(path), *args], env=env, capture_output=True
+    )
+    assert result.returncode in (9, -9), (
+        f"child should have been killed at {barrier}, got {result.returncode}: "
+        f"{result.stderr.decode(errors='replace')[-400:]}"
+    )
+
+
+def _env(tmp_path, **extra):
+    (tmp_path / "data").mkdir(exist_ok=True)
+    lib = tmp_path / "lib"
+    lib.mkdir(exist_ok=True)
+    config = Config.from_env(
+        {
+            "PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data"),
+            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
+            **extra,
+        }
+    )
+    run_migrations(config.database_url)
+    return config, create_session_factory(create_db_engine(config.database_url)), lib
+
+
+def _image(path: Path, seed: int = 3) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    pixels = np.random.default_rng(seed).integers(0, 256, (64, 96, 3), dtype=np.uint8)
+    Image.fromarray(pixels).save(path, quality=90)
+
+
+def _register(sf, path: Path) -> str:
+    asset_id = str(uuid.uuid4())
+    with sf() as session:
+        session.add(
+            Asset(
+                id=asset_id,
+                original_path=str(path),
+                current_path=str(path),
+                discovered_at=NOW,
+                hash_version=1,
+                byte_size=path.stat().st_size,
+                current_sha256=hashing.sha256_file(path),
+            )
+        )
+        session.commit()
+    return asset_id
+
+
+# ── EXIF written, checkpoint not recorded ────────────────────────────────────
+
+DECIDE_SCRIPT = """
+import sys
+sys.path.insert(0, {repo!r})
+from photo_pipeline.db import create_db_engine, create_session_factory
+from photo_pipeline.services.safety import SafetyService
+
+db_url, asset_id = sys.argv[1], sys.argv[2]
+sf = create_session_factory(create_db_engine(db_url))
+SafetyService(sf).decide(asset_id, "nsfw")
+"""
+
+
+@pytest.mark.skipif(shutil.which("exiftool") is None, reason="exiftool not installed")
+def test_a_crash_after_the_exif_write_leaves_nothing_verified_and_re_runs_cleanly(tmp_path):
+    config, sf, lib = _env(tmp_path)
+    path = lib / "a.jpg"
+    _image(path)
+    asset_id = _register(sf, path)
+    original_sha = hashing.sha256_file(path)
+
+    _child(
+        DECIDE_SCRIPT, config.database_url, asset_id, barrier=EXIF_WRITTEN, tmp_path=tmp_path
+    )
+
+    # The file changed, but the application claims nothing about it: no decision,
+    # no projection, and the stored hash is still the pre-write one.
+    assert "nsfw" in exiftool.read_keyword_sets([str(path)])[str(path)]
+    assert hashing.sha256_file(path) != original_sha
+    with sf() as session:
+        assert session.scalars(select(SafetyReview)).all() == []
+        assert session.get(Asset, asset_id).current_sha256 == original_sha
+    assert exif_checkpoint.state_for(sf, asset_id, "safety") is None
+
+    # Re-running is the recovery: the write is idempotent, so the second attempt
+    # verifies and records what the first one only did to the file.
+    review = SafetyService(sf).decide(asset_id, "nsfw")
+    assert review["exif_verified"] is True
+    assert exif_checkpoint.state_for(sf, asset_id, "safety") == exif_checkpoint.VERIFIED
+    keywords = exiftool.read_keyword_sets([str(path)])[str(path)]
+    assert "nsfw" in keywords and "sfw" not in keywords
+    with sf() as session:
+        asset = session.get(Asset, asset_id)
+        assert asset.current_sha256 == hashing.sha256_file(path)
+
+
+# ── uploader accepted, outcome not persisted ─────────────────────────────────
+
+UPLOAD_SCRIPT = """
+import sys
+sys.path.insert(0, {repo!r})
+from photo_pipeline.config import Config
+from photo_pipeline.db import create_db_engine, create_session_factory
+from photo_pipeline.services.upload_batches import UploadBatchService
+
+db_url, data_dir, lib, binary, server, batch_id = sys.argv[1:7]
+config = Config.from_env(
+    {{
+        "PHOTO_PIPELINE_DATA_DIR": data_dir,
+        "PHOTO_PIPELINE_LIBRARY_ROOTS": lib,
+        "PHOTO_PIPELINE_IMMICH_GO_BINARY": binary,
+        "PHOTO_PIPELINE_IMMICH_SERVER_URL": server,
+        "PHOTO_PIPELINE_IMMICH_API_KEY": "sentinel",
+    }}
+)
+sf = create_session_factory(create_db_engine(db_url))
+UploadBatchService(sf, config=config).run(batch_id)
+"""
+
+
+def test_a_crash_after_the_uploader_accepted_requires_verification(tmp_path):
+    """immich-go exited cleanly and the server may hold every file, but nothing was
+    written down. Recovery must not guess success — and must not blindly retry."""
+    seeded = seed_album(tmp_path)
+    mark_upload_ready(seeded)
+    immich = FakeImmich()
+    binary = fake_uploader(tmp_path, SILENT_UPLOADER)
+    config = Config.from_env(
+        {
+            "PHOTO_PIPELINE_DATA_DIR": str(seeded.data),
+            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(seeded.lib),
+            "PHOTO_PIPELINE_IMMICH_GO_BINARY": str(binary),
+            "PHOTO_PIPELINE_IMMICH_SERVER_URL": immich.url,
+            "PHOTO_PIPELINE_IMMICH_API_KEY": "sentinel",
+        }
+    )
+    engine = create_db_engine(config.database_url)
+    sf = create_session_factory(engine)
+    service = UploadBatchService(sf, config=config)
+    report = UploadService(sf, config=config).preflight(["rome"])
+    assert report["state"] == "ready", report["blockers"]
+    batch_id = service.create(["rome"], token=report["token"])[0]["id"]
+
+    try:
+        # The batch is claimed by the child, which dies once the uploader has run.
+        _child(
+            UPLOAD_SCRIPT,
+            config.database_url,
+            str(seeded.data),
+            str(seeded.lib),
+            str(binary),
+            immich.url,
+            batch_id,
+            barrier=UPLOAD_ACCEPTED,
+            tmp_path=tmp_path,
+        )
+    finally:
+        immich.stop()
+    assert service.get(batch_id)["state"] == BatchState.RUNNING  # lane still held
+
+    recovered = service.recover()
+
+    assert recovered == {"interrupted": 1}
+    batch = service.get(batch_id)
+    assert batch["state"] == BatchState.UNKNOWN and batch["error_code"] == "interrupted"
+    assert [b["code"] for b in retry_blockers(batch)] == ["requires_verification"]
+    assert all(item["state"] == "pending" for item in batch["items"]), "nothing claimed as sent"
+
+
+# ── one item done, the job unfinished ────────────────────────────────────────
+
+WORKER_SCRIPT = """
+import sys
+sys.path.insert(0, {repo!r})
+from pathlib import Path
+from photo_pipeline.db import create_db_engine, create_session_factory
+from photo_pipeline.jobs.worker import Worker
+
+db_url, log = sys.argv[1], Path(sys.argv[2])
+
+def handler(item_key, ctx):
+    with log.open("a") as handle:
+        handle.write(item_key + "\\n")
+
+sf = create_session_factory(create_db_engine(db_url))
+Worker(sf, {{"scan": handler}}, "killable", lease_seconds=1).run_once()
+"""
+
+
+def test_a_crash_between_items_resumes_without_running_the_done_item_twice(tmp_path):
+    config, sf, lib = _env(tmp_path)
+    service = JobService(sf)
+    job = service.enqueue("scan", items=["a", "b", "c"])
+    log = tmp_path / "handled.log"
+
+    _child(
+        WORKER_SCRIPT, config.database_url, str(log), barrier=JOB_ITEM_DONE, tmp_path=tmp_path
+    )
+
+    crashed = log.read_text().split()
+    assert crashed == ["a"], "the child should have died right after its first item"
+    assert service.get(job["id"])["state"] == JobState.RUNNING
+
+    # A fresh worker takes over once the dead lease expires.
+    import time
+
+    time.sleep(1.1)  # the child's lease is one second long
+    fresh = Worker(sf, {"scan": lambda item, ctx: log.open("a").write(item + "\n")}, "alive")
+    fresh.run_once()
+
+    assert service.get(job["id"])["state"] == JobState.SUCCEEDED
+    handled = log.read_text().split()
+    assert sorted(handled) == ["a", "b", "c"], f"an item ran twice or not at all: {handled}"
+    assert service.progress(job["id"])["by_state"] == {ItemState.SUCCEEDED: 3}

tests/e2e/test_phase_a_pipeline.py

@@ -22,6 +22,8 @@ import numpy as np
 import pytest
 from PIL import Image
 
+from tests.conftest import session_client
+
 REPO = Path(__file__).resolve().parents[2]
 
 
@@ -92,7 +94,7 @@ class ServerController:
                 pytest.fail(f"server exited: {err.decode(errors='replace')}")
             try:
                 if httpx.get(f"{self.base}/api/v1/health/ready", timeout=1).status_code == 200:
-                    self.client = httpx.Client(base_url=self.base, timeout=10)
+                    self.client = session_client(self.base, timeout=10)
                     return
             except httpx.HTTPError:
                 time.sleep(0.2)

tests/e2e/test_phase_f_pipeline.py

@@ -0,0 +1,505 @@
+"""Phase F end-to-end acceptance (US06-06): archive lifecycle, black box.
+
+Every journey drives a real ``photo_pipeline serve`` child process and a real
+durable worker over HTTP — register a medium, preflight it, archive an album,
+crash mid-transfer, recover, unmount the medium, rediscover the photos through
+their hashes while they are unreachable, mount it again, restore, and collide.
+Nothing is reached into: the medium is an ordinary directory whose marker file is
+its identity, and unmounting it means taking that marker away, which is what the
+application sees when a disk is unplugged.
+
+Three invariants are asserted wherever they apply, because they are what make the
+only *removing* stage safe:
+
+- **No source is removed before its archive copy is verified.** Every crash barrier
+  is checked for the file still being somewhere: either in the library, or on the
+  medium hashing exactly as recorded.
+- **Archived is not missing.** An unmounted medium leaves its photos
+  ``archived_offline``, still hashed, still deduplicable, still previewable.
+- **Restoring never overwrites.** A taken name comes back beside its occupant, and
+  the archived copy stays on the medium.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import shutil
+from pathlib import Path
+
+import httpx
+import pytest
+
+from tests.e2e._pipeline_harness import (
+    ArchiveStack,
+    mark_uploaded,
+    seed_album,
+    session_factory,
+    wait_until,
+)
+
+pytestmark = pytest.mark.phase_f
+
+TIMEOUT = 20
+ALBUM = "rome"
+MANIFEST = "archive-manifest.jsonl"
+# Every persisted transition the transfer can die at, in the order it reaches them.
+BARRIERS = ["transferring", "verified", "removing", "source_removed", "complete"]
+
+
+@pytest.fixture
+def stack(tmp_path):
+    """An analysed album with verified upload evidence — archivable, nothing else."""
+    seeded = seed_album(tmp_path)
+    mark_uploaded(seeded)
+    running = ArchiveStack(tmp_path, seeded)
+    try:
+        yield running
+    finally:
+        running.stop()
+
+
+# ── helpers ──────────────────────────────────────────────────────────────────
+
+
+def _preflight(stack, location_id: str, **body) -> dict:
+    response = httpx.post(
+        f"{stack.base}/api/v1/archive-preflight",
+        json={"location_id": location_id, **body},
+        timeout=TIMEOUT,
+    )
+    response.raise_for_status()
+    return response.json()
+
+
+def _plan(stack, location_id: str, **body) -> dict:
+    report = _preflight(stack, location_id, **body)
+    response = httpx.post(
+        f"{stack.base}/api/v1/archive-plans",
+        json={"location_id": location_id, "token": report["token"], **body},
+        timeout=TIMEOUT,
+    )
+    response.raise_for_status()
+    return response.json()
+
+
+def _apply(stack, plan_id: str) -> httpx.Response:
+    return httpx.post(f"{stack.base}/api/v1/archive-plans/{plan_id}/apply", timeout=TIMEOUT)
+
+
+def _plan_state(stack, plan_id: str) -> dict:
+    return httpx.get(f"{stack.base}/api/v1/archive-plans/{plan_id}", timeout=TIMEOUT).json()
+
+
+def _await_plan(stack, plan_id: str, states=("complete", "failed"), *, timeout: float = 60) -> dict:
+    return wait_until(
+        lambda: (lambda p: p if p.get("state") in states else None)(_plan_state(stack, plan_id)),
+        timeout=timeout,
+    )
+
+
+def _archive_album(stack) -> dict:
+    """Register, preflight, plan, apply, and wait for the run to finish."""
+    location = stack.register()
+    plan = _plan(stack, location["id"])
+    _apply(stack, plan["id"]).raise_for_status()
+    return {"location": location, "plan": _await_plan(stack, plan["id"])}
+
+
+def _assets(stack) -> dict[str, dict]:
+    return {asset["id"]: asset for asset in stack.assets()}
+
+
+def _recovery(stack) -> dict:
+    return httpx.get(f"{stack.base}/api/v1/archive-recovery", timeout=TIMEOUT).json()
+
+
+def _resolve(stack) -> dict:
+    response = httpx.post(f"{stack.base}/api/v1/archive-recovery/resolve", timeout=TIMEOUT)
+    response.raise_for_status()
+    return response.json()
+
+
+def _restore(stack, location_id: str) -> dict:
+    report = httpx.post(
+        f"{stack.base}/api/v1/restore-preflight",
+        json={"location_id": location_id},
+        timeout=TIMEOUT,
+    ).json()
+    plan = httpx.post(
+        f"{stack.base}/api/v1/restore-plans",
+        json={"location_id": location_id, "token": report["token"]},
+        timeout=TIMEOUT,
+    )
+    plan.raise_for_status()
+    plan = plan.json()
+    httpx.post(
+        f"{stack.base}/api/v1/restore-plans/{plan['id']}/apply", timeout=TIMEOUT
+    ).raise_for_status()
+    wait_until(
+        lambda: all(a["availability_state"] == "active" for a in stack.assets()), timeout=60
+    )
+    return {"preflight": report, "plan": plan}
+
+
+def _sha256(path: Path) -> str:
+    return hashlib.sha256(path.read_bytes()).hexdigest()
+
+
+def _contents(*roots: Path) -> set[str]:
+    """Every photo byte-string reachable anywhere, keyed by hash."""
+    return {
+        _sha256(path)
+        for root in roots
+        for path in root.rglob("*.jpg")
+        if path.is_file() and not path.name.startswith(".")
+    }
+
+
+def _manifest(stack) -> list[dict]:
+    path = stack.archive / ALBUM / MANIFEST
+    if not path.exists():
+        return []
+    return [json.loads(line) for line in path.read_text().splitlines() if line.strip()]
+
+
+# ── preflight blockers ───────────────────────────────────────────────────────
+
+
+def test_preflight_refuses_an_offline_medium_a_wrong_volume_and_a_full_disk(stack):
+    stack.start(worker=False)
+    location = stack.register()
+
+    ready = _preflight(stack, location["id"])
+    assert ready["state"] == "ready"
+    assert ready["totals"]["assets"] == 2
+
+    stack.unmount()
+    offline = _preflight(stack, location["id"])
+    assert offline["state"] == "blocked"
+    assert "location_offline" in {issue["code"] for issue in offline["blockers"]}
+
+    # A different disk mounted at the same place is not this medium.
+    (stack.archive / ArchiveStack.MARKER).write_text('{"media_id": "another-disk"}')
+    wrong = _preflight(stack, location["id"])
+    assert "wrong_volume" in {issue["code"] for issue in wrong["blockers"]}
+
+    # Nothing has been created, moved, or removed by any of that.
+    assert stack.plans() == []
+    assert sorted(p.name for p in (stack.seeded.lib / ALBUM).glob("*.jpg")) == ["a.jpg", "b.jpg"]
+
+
+def test_preflight_refuses_capacity_it_cannot_prove_and_bytes_it_cannot_vouch_for(tmp_path):
+    seeded = seed_album(tmp_path)
+    mark_uploaded(seeded)
+    stack = ArchiveStack(tmp_path, seeded)
+    try:
+        # A reserve larger than any disk: the scope cannot be promised room.
+        stack.start(
+            worker=False,
+            extra_env={"PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": str(1 << 62)},
+        )
+        location = stack.register()
+        report = _preflight(stack, location["id"])
+        assert "insufficient_capacity" in {issue["code"] for issue in report["blockers"]}
+
+        # And a plan for a blocked scope is refused rather than half-created.
+        refused = httpx.post(
+            f"{stack.base}/api/v1/archive-plans",
+            json={"location_id": location["id"], "token": report["token"]},
+            timeout=TIMEOUT,
+        )
+        assert refused.status_code == 422
+        assert refused.json()["error"]["code"] == "blocked"
+        assert stack.plans() == []
+    finally:
+        stack.stop()
+
+
+def test_an_album_whose_bytes_changed_after_upload_cannot_be_archived(stack):
+    stack.start(worker=False)
+    location = stack.register()
+    (stack.seeded.lib / ALBUM / "a.jpg").write_bytes(b"edited after the upload")
+
+    report = _preflight(stack, location["id"])
+    album = report["albums"][0]
+    assert album["state"] == "blocked"
+    assert "partial_scope" in {issue["code"] for issue in album["blockers"]}
+    assert "bytes_changed" in {
+        issue["code"] for asset in album["assets"] for issue in asset["blockers"]
+    }
+    assert report["state"] == "blocked"
+
+
+# ── copy, verify, remove ─────────────────────────────────────────────────────
+
+
+def test_an_archived_album_leaves_the_library_only_after_it_is_verified(stack):
+    stack.start()
+    before = _contents(stack.seeded.lib)
+    hashes = {a["id"]: a["current_sha256"] for a in stack.assets()}
+
+    result = _archive_album(stack)
+    plan = result["plan"]
+
+    assert plan["state"] == "complete"
+    assert {op["journal_state"] for op in plan["operations"]} == {"complete"}
+    # Same filesystem here, so the transfer took the atomic-move path — recorded as
+    # what actually happened, not as the plan's guess.
+    assert all(op["same_filesystem"] for op in plan["operations"])
+
+    # The bytes are on the medium, hashing exactly as recorded, and gone from the
+    # library. Not one photo was lost in between.
+    assert _contents(stack.archive) == before
+    assert not list((stack.seeded.lib / ALBUM).glob("*.jpg"))
+    for asset_id, sha256 in hashes.items():
+        asset = _assets(stack)[asset_id]
+        assert asset["current_path"] is None
+        assert asset["availability_state"] == "archived_online"
+        assert _sha256(stack.archive / asset["archive_path"]) == sha256
+
+    # The medium carries its own record of what it holds.
+    manifest = _manifest(stack)
+    assert {entry["asset_id"] for entry in manifest} == set(hashes)
+    assert {entry["sha256"] for entry in manifest} == set(hashes.values())
+    assert {entry["media_id"] for entry in manifest} == {result["location"]["media_id"]}
+
+
+def test_the_archived_state_survives_a_server_restart(stack):
+    stack.start()
+    result = _archive_album(stack)
+    before = _assets(stack)
+    plan_before = _plan_state(stack, result["plan"]["id"])
+
+    stack.restart_server()
+
+    assert _assets(stack) == before
+    assert _plan_state(stack, result["plan"]["id"]) == plan_before
+    assert {a["availability_state"] for a in stack.assets()} == {"archived_online"}
+
+
+# ── interruption at every persisted transition ───────────────────────────────
+
+
+@pytest.mark.parametrize("barrier", BARRIERS)
+def test_a_worker_killed_at_each_transition_loses_nothing_and_recovers(tmp_path, barrier):
+    # One photo, so the crash lands on the only operation and the end state is the
+    # recovery's own doing rather than a mixture with untouched work.
+    seeded = seed_album(tmp_path, names=("a.jpg",))
+    mark_uploaded(seeded)
+    stack = ArchiveStack(tmp_path, seeded)
+    try:
+        # Only the worker carries the crash barrier: it dies the instant this journal
+        # state is persisted, while the server stays up to be asked what happened.
+        stack.start(worker=False)
+        stack.restart_worker(extra_env={"PHOTO_PIPELINE_FAULT_AFTER": barrier})
+        before = _contents(stack.seeded.lib)
+        location = stack.register()
+        plan = _plan(stack, location["id"])
+        _apply(stack, plan["id"]).raise_for_status()
+
+        wait_until(lambda: stack.worker.poll() is not None, timeout=60)
+        assert stack.worker.returncode in (9, -9), "the worker should have been killed"
+
+        # Whatever the crash interrupted, every photo is still readable somewhere.
+        assert before <= _contents(stack.seeded.lib, stack.archive), f"content lost at {barrier}"
+        # And nothing was removed that had not been verified first.
+        for operation in _plan_state(stack, plan["id"])["operations"]:
+            source = Path(operation["source_path"])
+            if not source.exists():
+                archived = Path(operation["destination_path"])
+                assert archived.exists(), f"{barrier}: source removed without an archive copy"
+                assert _sha256(archived) == operation["expected_sha256"]
+
+        # Recovery is offered from evidence, resolves without a worker, and converges.
+        verdicts = _recovery(stack)
+        assert all(v["classification"] != "manual" for v in verdicts["operations"]), verdicts
+        _resolve(stack)
+        assert _recovery(stack)["operations"] == []
+        # Repeating it changes nothing.
+        assert _resolve(stack) == {"resumed": 0, "completed": 0, "manual": 0}
+
+        # Recovery finishes what was durable and re-plans what was not. Before the
+        # archive copy existed, that means putting the item back to `planned` with the
+        # original untouched; from `verified` onwards it means completing it.
+        stack.restart_server()
+        operation = _plan_state(stack, plan["id"])["operations"][0]
+        asset = next(iter(stack.assets()))
+        if barrier == "transferring":
+            assert operation["journal_state"] == "planned"
+            assert Path(operation["source_path"]).exists()
+            assert asset["availability_state"] == "active"
+            assert _contents(stack.seeded.lib) == before
+        else:
+            assert operation["journal_state"] == "complete"
+            assert not Path(operation["source_path"]).exists()
+            assert _contents(stack.archive) == before
+            assert asset["availability_state"] == "archived_online"
+            assert _sha256(stack.archive / asset["archive_path"]) == asset["current_sha256"]
+            assert {entry["sha256"] for entry in _manifest(stack)} == before
+    finally:
+        stack.stop()
+
+
+def test_evidence_that_contradicts_the_journal_is_never_guessed(stack):
+    stack.start(worker=False)
+    location = stack.register()
+    plan = _plan(stack, location["id"])
+    # The journal claims a verified archive copy the medium does not have.
+    with session_factory(stack.seeded) as sf:
+        from photo_pipeline.services.archive_journal import ArchiveJournal
+
+        journal = ArchiveJournal(sf)
+        operation = journal.operations(plan["id"])[0]
+        journal.begin(operation["id"], worker_id="crashed", fencing_token=1)
+        journal.transition(operation["id"], "verified", fencing_token=1)
+
+    verdicts = _recovery(stack)
+    assert [v["classification"] for v in verdicts["operations"]] == ["manual"]
+    assert _resolve(stack)["manual"] == 1
+    # The source is untouched and the unresolved item keeps blocking a new archive.
+    assert (stack.seeded.lib / ALBUM / "a.jpg").exists()
+    blocked = _preflight(stack, location["id"])
+    assert "archive_pending" in {issue["code"] for issue in blocked["blockers"]}
+
+
+# ── offline identity and deduplication ───────────────────────────────────────
+
+
+def test_offline_photos_stay_hashed_previewable_and_deduplicable(stack):
+    stack.start()
+    result = _archive_album(stack)
+    archived = _assets(stack)
+    copy_source = stack.archive / next(iter(archived.values()))["archive_path"]
+    exact = stack.seeded.lib / "inbox" / "again.jpg"
+    exact.parent.mkdir(parents=True, exist_ok=True)
+    shutil.copy2(copy_source, exact)
+    stack.unmount()
+
+    scanned = httpx.post(f"{stack.base}/api/v1/inventory/scan", timeout=60)
+    scanned.raise_for_status()
+    # An unmounted medium is not a missing file.
+    assert scanned.json()["counts"].get("missing") is None
+    offline = {a["id"]: a for a in stack.assets() if a["id"] in archived}
+    assert {a["availability_state"] for a in offline.values()} == {"archived_offline"}
+    assert all(a["current_sha256"] for a in offline.values())
+
+    # The new active copy is recognised as the archived photo, not as a new one.
+    httpx.post(f"{stack.base}/api/v1/duplicates/detect", timeout=60).raise_for_status()
+    clusters = httpx.get(
+        f"{stack.base}/api/v1/duplicates/clusters", params={"limit": 50}, timeout=TIMEOUT
+    ).json()["items"]
+    exact_clusters = [c for c in clusters if c["method"] == "exact"]
+    assert len(exact_clusters) == 1
+    detail = httpx.get(
+        f"{stack.base}/api/v1/duplicates/clusters/{exact_clusters[0]['id']}", timeout=TIMEOUT
+    ).json()
+    canonical = next(m for m in detail["members"] if m["asset_id"] == detail["canonical_asset_id"])
+    assert canonical["availability_state"] == "archived_offline"
+    assert canonical["archive_location"] == result["location"]["name"]
+    assert canonical["preview"]["state"] == "ready" and canonical["preview"]["protected"]
+
+    # And the retained preview really is served while the medium is away.
+    thumbnail = httpx.get(
+        f"{stack.base}/api/v1/assets/{canonical['asset_id']}/thumbnail",
+        params={"size": 1280},
+        timeout=TIMEOUT,
+    )
+    assert thumbnail.status_code == 200
+    assert thumbnail.headers["content-type"] == "image/webp"
+
+
+def test_a_fuzzy_offline_match_asks_for_the_medium_instead_of_guessing(stack):
+    stack.start()
+    result = _archive_album(stack)
+    archived = next(iter(_assets(stack).values()))
+    variant = stack.seeded.lib / "inbox" / "resized.jpg"
+    variant.parent.mkdir(parents=True, exist_ok=True)
+    _resize(stack.archive / archived["archive_path"], variant)
+    stack.unmount()
+
+    httpx.post(f"{stack.base}/api/v1/inventory/scan", timeout=60).raise_for_status()
+    httpx.post(f"{stack.base}/api/v1/duplicates/detect", timeout=60).raise_for_status()
+    clusters = httpx.get(
+        f"{stack.base}/api/v1/duplicates/clusters", params={"limit": 50}, timeout=TIMEOUT
+    ).json()["items"]
+    perceptual = [c for c in clusters if c["method"] == "perceptual"]
+    assert len(perceptual) == 1
+
+    detail = httpx.get(
+        f"{stack.base}/api/v1/duplicates/clusters/{perceptual[0]['id']}", timeout=TIMEOUT
+    ).json()
+    # A fuzzy match is never decided automatically, and full-resolution review names
+    # the medium to mount rather than guessing from the preview.
+    assert detail["state"] == "open" and detail["requires_confirmation"] is True
+    assert detail["mount_required"] == [result["location"]["name"]]
+
+
+def test_mounting_the_medium_again_makes_the_originals_reachable(stack):
+    stack.start()
+    result = _archive_album(stack)
+    stack.unmount()
+    httpx.post(f"{stack.base}/api/v1/inventory/scan", timeout=60).raise_for_status()
+    assert {a["availability_state"] for a in stack.assets()} == {"archived_offline"}
+    # Restoring from a medium that is not there is refused, with its reason.
+    away = httpx.post(
+        f"{stack.base}/api/v1/restore-preflight",
+        json={"location_id": result["location"]["id"]},
+        timeout=TIMEOUT,
+    ).json()
+    assert away["state"] == "blocked"
+    assert "location_offline" in {issue["code"] for issue in away["blockers"]}
+
+    stack.remount()
+    locations = httpx.get(f"{stack.base}/api/v1/archive-locations", timeout=TIMEOUT).json()
+    assert locations["locations"][0]["state"] == "online"
+    assert {a["availability_state"] for a in stack.assets()} == {"archived_online"}
+
+
+# ── restore ──────────────────────────────────────────────────────────────────
+
+
+def test_restoring_returns_the_bytes_without_losing_identity_or_the_archive(stack):
+    stack.start()
+    result = _archive_album(stack)
+    identities = set(_assets(stack))
+    hashes = {a["id"]: a["current_sha256"] for a in stack.assets()}
+
+    _restore(stack, result["location"]["id"])
+
+    restored = _assets(stack)
+    assert set(restored) == identities  # the same photos, not new ones
+    for asset_id, sha256 in hashes.items():
+        asset = restored[asset_id]
+        assert asset["availability_state"] == "active"
+        assert _sha256(Path(asset["current_path"])) == sha256
+        # The archive copy is a copy: restoring empties nothing.
+        assert (stack.archive / asset["archive_path"]).exists()
+
+    stack.restart_server()
+    assert _assets(stack) == restored
+
+
+def test_a_restore_comes_back_beside_an_occupant_never_over_it(stack):
+    stack.start()
+    result = _archive_album(stack)
+    squatter = stack.seeded.lib / ALBUM / "a.jpg"
+    squatter.parent.mkdir(parents=True, exist_ok=True)
+    squatter.write_bytes(b"a different photo lives here now")
+
+    report = _restore(stack, result["location"]["id"])["preflight"]
+    destinations = {item["destination_path"] for item in report["items"]}
+    assert any("(restored)" in destination for destination in destinations)
+
+    assert squatter.read_bytes() == b"a different photo lives here now"
+    assert (stack.seeded.lib / ALBUM / "a (restored).jpg").exists()
+    assert {a["availability_state"] for a in stack.assets()} == {"active"}
+
+
+def _resize(source: Path, destination: Path, scale: float = 0.5) -> None:
+    from PIL import Image
+
+    with Image.open(source) as opened:
+        opened.resize(
+            (int(opened.width * scale), int(opened.height * scale)), Image.LANCZOS
+        ).save(destination, quality=95)

tests/e2e/test_release_gate.py

@@ -0,0 +1,298 @@
+"""The release gate, the read-only dry run, and the approval that unlocks mutation
+(US07-07).
+
+The gate itself is exercised with a tiny stage set — running the whole suite from
+inside the suite would be a fork bomb with better manners. What is proven here is
+the machinery a release depends on: the story matrix is complete, a failing stage
+fails the gate, an unexpected skip fails the gate, and the evidence is written with
+checksums that match what was written.
+
+The dry run is proven to be read-only against a real temporary library, and the
+approval is proven to be what stands between a configured library and any mutation.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import uuid
+from datetime import datetime, timezone
+from pathlib import Path
+
+import pytest
+from fastapi.testclient import TestClient
+
+from photo_pipeline.api.app import create_app
+from photo_pipeline.config import Config
+from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
+from photo_pipeline.models import Asset
+from photo_pipeline.services import release
+
+REPO = Path(__file__).resolve().parents[2]
+NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
+
+# Two throwaway stages: one that passes, one the test can point at a failure.
+PASSING = ("tests/e2e/test_traceability.py",)
+
+
+def _config(tmp_path, **extra) -> Config:
+    data = tmp_path / "data"
+    data.mkdir(parents=True, exist_ok=True)
+    lib = tmp_path / "lib"
+    lib.mkdir(exist_ok=True)
+    return Config.from_env(
+        {
+            "PHOTO_PIPELINE_DATA_DIR": str(data),
+            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
+            **extra,
+        }
+    )
+
+
+# ── the story matrix ─────────────────────────────────────────────────────────
+
+
+def test_every_backlog_story_is_delivered_or_explicitly_planned():
+    matrix = release.story_matrix(REPO)
+
+    assert matrix["problems"] == [], "the story matrix has holes"
+    assert len(matrix["delivered"]) + len(matrix["planned"]) == matrix["stories"]
+    assert "US01-01" in matrix["delivered"] and "US07-07" in matrix["delivered"]
+
+
+def test_a_story_without_tests_is_a_gate_failure(tmp_path):
+    """A story file nobody covered must not pass quietly as 'no tests ran'."""
+    fake = tmp_path / "repo"
+    (fake / "delivery_backlog" / "stories").mkdir(parents=True)
+    (fake / "tests").mkdir()
+    (fake / "delivery_backlog" / "stories" / "US99-01-invented.md").write_text("# US99-01")
+    (fake / "tests" / "story_traceability.json").write_text(json.dumps({"stories": {}}))
+
+    matrix = release.story_matrix(fake)
+
+    assert matrix["problems"] == ["story with no tests and not planned: US99-01"]
+
+
+def test_a_mapping_to_a_deleted_test_file_is_a_gate_failure(tmp_path):
+    fake = tmp_path / "repo"
+    (fake / "delivery_backlog" / "stories").mkdir(parents=True)
+    (fake / "tests").mkdir()
+    (fake / "delivery_backlog" / "stories" / "US99-01-invented.md").write_text("# US99-01")
+    (fake / "tests" / "story_traceability.json").write_text(
+        json.dumps({"stories": {"US99-01": ["tests/gone.py"]}})
+    )
+
+    assert release.story_matrix(fake)["problems"] == [
+        "mapped test file is missing — US99-01: tests/gone.py"
+    ]
+
+
+# ── the gate ─────────────────────────────────────────────────────────────────
+
+
+def test_the_gate_runs_its_stages_and_keeps_checksummed_evidence(tmp_path):
+    config = _config(tmp_path)
+    evidence = tmp_path / "evidence"
+
+    report = release.run_gate(config, output=evidence, stages=(("smoke", PASSING),))
+
+    assert report["ok"] is True and report["failures"] == []
+    assert report["stages"][0]["stage"] == "smoke" and report["stages"][0]["ok"] is True
+    assert report["revision"], "the evidence must say which commit it covers"
+    assert report["matrix"]["problems"] == []
+
+    written = json.loads((evidence / release.REPORT_NAME).read_text())
+    assert written["ok"] is True
+    assert (evidence / "logs" / "smoke.log").exists()
+    checksums = (evidence / release.CHECKSUMS_NAME).read_text().splitlines()
+    assert len(checksums) >= 2
+    for line in checksums:
+        digest, name = line.split("  ", 1)
+        assert release.sha256_file(evidence / name) == digest
+
+
+def test_a_failing_stage_fails_the_gate(tmp_path):
+    config = _config(tmp_path)
+    failing = tmp_path / "failing_test.py"
+    failing.write_text("def test_no():\n    assert False\n")
+
+    report = release.run_gate(
+        config, output=tmp_path / "evidence", stages=(("broken", (str(failing),)),)
+    )
+
+    assert report["ok"] is False and report["failures"] == ["broken"]
+    assert report["stages"][0]["returncode"] != 0
+
+
+def test_an_unexpected_skip_fails_the_gate_but_an_environment_skip_does_not():
+    environment = release.StageResult(
+        "unit", [], 0, 0.1, "1 skipped", ["SKIPPED [1] x.py:1: exiftool not installed"]
+    )
+    silent = release.StageResult(
+        "unit", [], 0, 0.1, "1 skipped", ["SKIPPED [1] x.py:1: flaky, look later"]
+    )
+
+    assert release.unexpected_skips([environment]) == []
+    assert release.unexpected_skips([silent, environment]) == [
+        "SKIPPED [1] x.py:1: flaky, look later"
+    ]
+
+
+# ── the real-library dry run ─────────────────────────────────────────────────
+
+
+def _library(root: Path) -> None:
+    (root / "album").mkdir(parents=True)
+    (root / "album" / "a.jpg").write_bytes(b"a" * 128)
+    (root / "album" / "b.png").write_bytes(b"b" * 64)
+    (root / "loose.JPG").write_bytes(b"c" * 32)
+    excluded = root / "_IGNORE" / "private"
+    excluded.mkdir(parents=True)
+    (excluded / "secret.jpg").write_bytes(b"never read")
+
+
+def test_the_dry_run_describes_the_library_without_touching_it(tmp_path):
+    config = _config(tmp_path)
+    root = Path(config.library_roots[0])
+    _library(root)
+    before = {
+        str(p): (p.stat().st_mtime_ns, p.read_bytes()) for p in root.rglob("*") if p.is_file()
+    }
+
+    report = release.dry_run(config)
+
+    assert report["files"] == 3, "the excluded sentinel is not counted"
+    assert report["by_extension"] == {".jpg": 2, ".png": 1}
+    assert report["excluded_directories"] >= 1
+    assert report["mutation"] == "none — this pass is read-only"
+    assert report["checksum"]
+    assert not any("secret" in json.dumps(report) for _ in [0]), "excluded content never appears"
+    after = {
+        str(p): (p.stat().st_mtime_ns, p.read_bytes()) for p in root.rglob("*") if p.is_file()
+    }
+    assert after == before, "a read-only pass changed the library"
+
+
+def test_the_dry_run_reconciles_against_what_the_database_already_knows(tmp_path):
+    config = _config(tmp_path)
+    root = Path(config.library_roots[0])
+    _library(root)
+    run_migrations(config.database_url)
+    engine = create_db_engine(config.database_url)
+    with create_session_factory(engine)() as session:
+        session.add(
+            Asset(
+                id=str(uuid.uuid4()),
+                original_path=str(root / "album" / "a.jpg"),
+                current_path=str(root / "album" / "a.jpg"),
+                discovered_at=NOW,
+                hash_version=1,
+                byte_size=128,
+            )
+        )
+        session.add(
+            Asset(
+                id=str(uuid.uuid4()),
+                original_path=str(root / "album" / "gone.jpg"),
+                current_path=str(root / "album" / "gone.jpg"),
+                discovered_at=NOW,
+                hash_version=1,
+                byte_size=1,
+            )
+        )
+        session.commit()
+    engine.dispose()
+
+    reconciliation = release.dry_run(config)["reconciliation"]
+
+    assert reconciliation["known_to_database"] == 2
+    assert reconciliation["already_registered"] == 1
+    assert reconciliation["new_to_the_application"] == 2
+    assert reconciliation["recorded_but_absent_total"] == 1
+    assert reconciliation["recorded_but_absent"][0].endswith("gone.jpg")
+
+
+def test_a_library_root_that_is_not_there_is_refused(tmp_path):
+    config = _config(tmp_path, PHOTO_PIPELINE_LIBRARY_ROOTS=str(tmp_path / "nowhere"))
+    with pytest.raises(release.ReleaseError, match="not a directory"):
+        release.dry_run(config)
+
+
+# ── the approval ─────────────────────────────────────────────────────────────
+
+
+def test_mutation_is_refused_until_the_dry_run_is_approved(tmp_path):
+    config = _config(tmp_path, PHOTO_PIPELINE_REQUIRE_DRY_RUN_APPROVAL="1")
+    _library(Path(config.library_roots[0]))
+
+    blockers = release.mutation_blockers(config)
+    assert [blocker["code"] for blocker in blockers] == ["dry_run_not_approved"]
+
+    record = release.approve(config, release.dry_run(config), approver="domverse")
+    assert record["approved_by"] == "domverse" and record["report_checksum"]
+    assert release.mutation_blockers(config) == []
+
+
+def test_an_approval_covers_the_library_it_was_written_for(tmp_path):
+    config = _config(tmp_path, PHOTO_PIPELINE_REQUIRE_DRY_RUN_APPROVAL="1")
+    _library(Path(config.library_roots[0]))
+    release.approve(config, release.dry_run(config), approver="domverse")
+
+    other = tmp_path / "other-library"
+    other.mkdir()
+    moved = config.model_copy(update={"library_roots": (other,)})
+
+    assert [b["code"] for b in release.mutation_blockers(moved)] == ["approval_scope_mismatch"]
+
+
+def test_without_the_requirement_nothing_changes(tmp_path):
+    config = _config(tmp_path)  # the loopback development default
+    assert release.mutation_blockers(config) == []
+
+
+def test_the_api_refuses_every_mutation_until_the_report_is_approved(tmp_path):
+    config = _config(tmp_path, PHOTO_PIPELINE_REQUIRE_DRY_RUN_APPROVAL="1")
+    _library(Path(config.library_roots[0]))
+    with TestClient(create_app(config)) as client:
+        # Reading stays open: an operator has to see what was found to approve it.
+        assert client.get("/api/v1/workflow").status_code == 200
+        refused = client.post("/api/v1/inventory/scan", json={})
+        assert refused.status_code == 403
+        assert refused.json()["error"]["code"] == "dry_run_not_approved"
+        # A backup is the one mutation a careful operator takes first.
+        assert client.post("/api/v1/backups", json={}).status_code == 201
+
+        release.approve(config, release.dry_run(config), approver="domverse")
+        assert client.post("/api/v1/inventory/scan", json={}).status_code in (200, 201, 202)
+
+
+def test_approving_something_that_is_not_a_report_is_refused(tmp_path):
+    config = _config(tmp_path)
+    with pytest.raises(release.ReleaseError, match="not a dry-run report"):
+        release.approve(config, {"files": 3}, approver="domverse")
+
+
+def test_the_cli_runs_the_dry_run_and_the_approval(tmp_path):
+    config = _config(tmp_path, PHOTO_PIPELINE_REQUIRE_DRY_RUN_APPROVAL="1")
+    _library(Path(config.library_roots[0]))
+    from photo_pipeline.__main__ import main
+
+    environment = {
+        "PHOTO_PIPELINE_DATA_DIR": str(config.data_dir),
+        "PHOTO_PIPELINE_LIBRARY_ROOTS": str(config.library_roots[0]),
+        "PHOTO_PIPELINE_REQUIRE_DRY_RUN_APPROVAL": "1",
+    }
+    previous = {key: os.environ.get(key) for key in environment}
+    os.environ.update(environment)
+    try:
+        report_path = tmp_path / "dry-run.json"
+        assert main(["dry-run", "--output", str(report_path)]) == 0
+        assert json.loads(report_path.read_text())["files"] == 3
+        assert main(["approve-dry-run", str(report_path), "--approver", "domverse"]) == 0
+    finally:
+        for key, value in previous.items():
+            if value is None:
+                os.environ.pop(key, None)
+            else:
+                os.environ[key] = value
+    assert release.mutation_blockers(config) == []

tests/e2e/test_release_journey.py

@@ -0,0 +1,348 @@
+"""The release journey (US07-07): one library, one fresh environment, every stage.
+
+This is the acceptance the whole backlog builds up to — discovery, duplicate review,
+safety, analysis, EXIF verification, album proposal, guarded rename, rescan and
+reconciliation, upload, archive, offline deduplication, restore — driven over HTTP
+against real ``photo_pipeline serve`` and worker child processes, with full process
+restarts in the middle and at the end.
+
+Nothing is reached into. External services are the deterministic fakes the earlier
+phases already use, invoked through the real integration layer: a vision fake that
+records every path it was given, a real fake ``immich-go`` executable, and an
+archive medium that is an ordinary directory whose marker file is its identity.
+
+The invariants asserted along the way are the ones the concept calls non-negotiable:
+
+- an asset's identity survives a rename, an upload, an archive, and a restore;
+- an ``_IGNORE`` sentinel is never discovered, counted, analysed, or uploaded;
+- an NSFW asset never reaches the vision provider but still reaches Immich;
+- no photo's bytes are lost at any point — every hash is still reachable somewhere;
+- every stage's durable state survives a restart of both processes.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import shutil
+from pathlib import Path
+
+import httpx
+import pytest
+
+from tests.e2e._pipeline_harness import (
+    SENTINEL_KEY,
+    FakeImmich,
+    Server,
+    fake_uploader,
+    image,
+    seed_library,
+    start_worker,
+    wait_until,
+)
+
+TIMEOUT = 30
+ALBUM = "rome"
+UPLOADER = 'echo "INFO  uploaded $6"\necho "Uploaded 2, duplicates 0"\nexit 0\n'
+
+
+def _sha256(path: Path) -> str:
+    return hashlib.sha256(path.read_bytes()).hexdigest()
+
+
+def _hashes(*roots: Path) -> set[str]:
+    return {
+        _sha256(path)
+        for root in roots
+        for path in root.rglob("*.jpg")
+        if path.is_file() and not path.name.startswith(".")
+    }
+
+
+def _post(base: str, path: str, **kwargs) -> httpx.Response:
+    response = httpx.post(f"{base}/api/v1{path}", timeout=TIMEOUT, **kwargs)
+    response.raise_for_status()
+    return response
+
+
+def _get(base: str, path: str, **kwargs) -> dict:
+    response = httpx.get(f"{base}/api/v1{path}", timeout=TIMEOUT, **kwargs)
+    response.raise_for_status()
+    return response.json()
+
+
+def _await_job(base: str, job_id: str, *, states=("succeeded",)) -> dict:
+    return wait_until(
+        lambda: (
+            snapshot
+            if (snapshot := _get(base, f"/jobs/{job_id}"))["state"] in states
+            else None
+        ),
+        timeout=90,
+    )
+
+
+@pytest.fixture
+def library(tmp_path):
+    """A fresh library: an album, an exact duplicate, and an excluded sentinel."""
+    seeded = seed_library(tmp_path, {}, {})
+    album = seeded.lib / ALBUM
+    image(album / "a.jpg", 11)
+    image(album / "b.jpg", 12)
+    shutil.copyfile(album / "a.jpg", album / "a-copy.jpg")  # exact duplicate
+    ignored = seeded.lib / "_IGNORE" / "private"
+    ignored.mkdir(parents=True)
+    image(ignored / "sentinel-9f3a2b.jpg", 99)
+    return seeded
+
+
+@pytest.mark.skipif(shutil.which("exiftool") is None, reason="exiftool not installed")
+def test_the_full_release_journey_survives_every_stage_and_two_restarts(library, tmp_path):
+    immich = FakeImmich()
+    uploader = fake_uploader(tmp_path, UPLOADER)
+    vision_log = tmp_path / "vision.log"
+    archive_root = tmp_path / "medium"
+    archive_root.mkdir()
+    environment = {
+        "PHOTO_PIPELINE_IMMICH_SERVER_URL": immich.url,
+        "PHOTO_PIPELINE_IMMICH_API_KEY": SENTINEL_KEY,
+        "PHOTO_PIPELINE_IMMICH_GO_BINARY": str(uploader),
+        "PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": "0",
+        "PHOTO_PIPELINE_FAKE_VISION_LOG": str(vision_log),
+    }
+    server = Server(library, extra_env=environment).start()
+    worker = start_worker(library, extra_env=environment)
+    base = server.base
+
+    try:
+        # ── 0. discovery ─────────────────────────────────────────────────────
+        _post(base, "/inventory/scan")
+        assets = _get(base, "/inventory/assets", params={"limit": 200})["items"]
+        assert len(assets) == 3, "the sentinel under _IGNORE is not an asset"
+        paths = {asset["current_path"] for asset in assets}
+        assert not any("_IGNORE" in path or "sentinel" in path for path in paths)
+        identity = {asset["id"]: Path(asset["current_path"]).name for asset in assets}
+
+        # ── 1. duplicate review ──────────────────────────────────────────────
+        _post(base, "/duplicates/detect")
+        clusters = _get(base, "/duplicates/clusters")["items"]
+        assert len(clusters) == 1 and clusters[0]["member_total"] == 2
+        cluster = _get(base, f"/duplicates/clusters/{clusters[0]['id']}")
+        canonical = sorted(member["asset_id"] for member in cluster["members"])[0]
+        _post(
+            base,
+            f"/duplicates/clusters/{cluster['id']}/decision",
+            json={
+                "decision": "canonical",
+                "canonical_asset_id": canonical,
+                "expected_version": cluster["version"],
+            },
+        )
+
+        # ── 2. safety, with its EXIF checkpoint ──────────────────────────────
+        queue = _get(base, "/safety/queue", params={"limit": 100})["items"]
+        assert len(queue) == 2, "a non-canonical variant is not reviewed twice"
+        decisions = {}
+        for index, item in enumerate(sorted(queue, key=lambda row: row["current_path"])):
+            decision = "nsfw" if index == 0 else "sfw"
+            decisions[item["asset_id"]] = decision
+            result = _post(
+                base, "/safety/decisions", json={"asset_id": item["asset_id"], "decision": decision}
+            ).json()
+            assert result["exif_verified"] is True, "the safety checkpoint must verify"
+
+        # ── restart: everything so far has to be durable ─────────────────────
+        server.stop()
+        server.start()
+        base = server.base
+        assert _get(base, "/safety/counts")["nsfw"] == 1
+        assert {a["id"] for a in _get(base, "/inventory/assets", params={"limit": 200})["items"]} == set(
+            identity
+        )
+
+        # ── 3. analysis, gated to confirmed-SFW assets ───────────────────────
+        job = _post(base, "/analysis/jobs").json()
+        _await_job(base, job["id"])
+        analysed = [
+            name
+            for name, decision in (
+                (identity[asset_id], decision) for asset_id, decision in decisions.items()
+            )
+            if decision == "sfw"
+        ]
+        seen = vision_log.read_text().splitlines()
+        assert len(seen) == len(analysed) == 1
+        assert not any("sentinel" in line or "_IGNORE" in line for line in seen)
+        nsfw_id = next(aid for aid, decision in decisions.items() if decision == "nsfw")
+        assert all(identity[nsfw_id] not in line for line in seen), "NSFW reached the provider"
+
+        # From here on no stage may change a photo's bytes: the metadata stages are
+        # done, and moving, uploading, archiving, and restoring only relocate them.
+        stable_hashes = _hashes(library.lib)
+
+        # ── 4. album proposal and guarded rename ─────────────────────────────
+        _post(base, "/albums/proposals", json={})
+        proposal = _get(base, f"/albums/proposals/{ALBUM}")
+        _post(
+            base,
+            f"/albums/proposals/{ALBUM}/edit",
+            json={"name": "2019 Rome", "expected_version": proposal["version"]},
+        )
+        proposal = _get(base, f"/albums/proposals/{ALBUM}")
+        _post(
+            base,
+            f"/albums/proposals/{ALBUM}/approve",
+            json={"expected_version": proposal["version"]},
+        )
+        plan = _post(base, "/rename-plans").json()
+        assert plan["blockers"] == [], [
+            (issue["code"], issue["message"])
+            for op in plan["operations"]
+            for issue in op["issues"]
+        ]
+        response = httpx.post(
+            f"{base}/api/v1/rename-plans/{plan['id']}/apply",
+            json={"expected_version": plan["version"], "expected_checksum": plan["checksum"]},
+            timeout=TIMEOUT,
+        )
+        assert response.status_code == 200, response.text
+        applied = response.json()
+        assert applied["failed"] == 0 and applied["applied"] == 1
+        assert (library.lib / "2019 Rome").is_dir() and not (library.lib / ALBUM).exists()
+
+        # ── 5. rescan and reconciliation: identity survives the move ─────────
+        _post(base, "/inventory/scan")
+        after_rename = _get(base, "/inventory/assets", params={"limit": 200})["items"]
+        assert {asset["id"] for asset in after_rename} == set(identity)
+        assert all("2019 Rome" in asset["current_path"] for asset in after_rename)
+        assert _hashes(library.lib) == stable_hashes, "a rename changed a photo's bytes"
+
+        # ── 6. upload ────────────────────────────────────────────────────────
+        report = _post(base, "/upload-preflight", json={"albums": ["2019 Rome"]}).json()
+        assert report["state"] == "ready", report["blockers"]
+        batch = _post(
+            base,
+            "/upload-batches",
+            json={"albums": ["2019 Rome"], "token": report["token"]},
+        ).json()["batches"][0]
+        started = _post(base, f"/upload-batches/{batch['id']}/start").json()
+        _await_job(base, started["job"]["id"])
+        uploaded = _get(base, f"/upload-batches/{batch['id']}")
+        assert uploaded["state"] == "succeeded"
+        # The uploader said nothing per file, so the outcome is uncertain until the
+        # server itself is asked whether it holds those exact bytes (US05-04).
+        assert uploaded["outcome_state"] == "requires_verification"
+        verified = _post(base, f"/upload-batches/{batch['id']}/verify").json()
+        assert verified["outcome_state"] == "verified", verified
+        uploaded = _get(base, f"/upload-batches/{batch['id']}")
+        # Reviewed NSFW is uploaded; it simply never reached the analyser.
+        assert {item["asset_id"] for item in uploaded["items"]} >= {nsfw_id}
+
+        # ── 7. archive ───────────────────────────────────────────────────────
+        location = _post(
+            base, "/archive-locations", json={"name": "external", "root": str(archive_root)}
+        ).json()
+        preflight = _post(
+            base, "/archive-preflight", json={"location_id": location["id"]}
+        ).json()
+        assert preflight["state"] == "ready", [
+            (asset["asset_id"], asset["blockers"])
+            for album in preflight["albums"]
+            for asset in album["assets"]
+            if asset["blockers"]
+        ] or preflight
+        archive_plan = _post(
+            base,
+            "/archive-plans",
+            json={"location_id": location["id"], "token": preflight["token"]},
+        ).json()
+        uploaded_ids = {item["asset_id"] for item in uploaded["items"]}
+        _post(base, f"/archive-plans/{archive_plan['id']}/apply")
+        wait_until(
+            lambda: all(
+                asset["availability_state"].startswith("archived")
+                for asset in _get(base, "/inventory/assets", params={"limit": 200})["items"]
+                if asset["id"] in uploaded_ids
+            ),
+            timeout=90,
+        )
+        assert _hashes(library.lib, archive_root) == stable_hashes, "archiving lost bytes"
+
+        # ── 8. offline deduplication ─────────────────────────────────────────
+        (archive_root / ".photo-pipeline-archive.json").rename(
+            archive_root / ".photo-pipeline-archive.json.away"
+        )
+        # A copy of an archived photo turns up in the library under its own name —
+        # the real shape of "I re-imported an old card" — so nothing occupies the
+        # path the archived original would be restored to.
+        returned = library.lib / "2019 Rome" / "rediscovered.jpg"
+        returned.parent.mkdir(parents=True, exist_ok=True)
+        archived_copy = next(archive_root.rglob("*.jpg"))
+        shutil.copyfile(archived_copy, returned)
+        _post(base, "/inventory/scan")
+        _post(base, "/duplicates/detect")
+        offline = _get(base, "/inventory/assets", params={"limit": 200})["items"]
+        archived = [a for a in offline if a["availability_state"].startswith("archived")]
+        assert archived, "an unmounted medium must not make assets missing"
+        assert all(a["availability_state"] != "missing_unexpected" for a in offline)
+        assert any(
+            cluster["member_total"] >= 2 for cluster in _get(base, "/duplicates/clusters")["items"]
+        ), "the rediscovered copy did not meet its archived original"
+
+        # ── 9. restore ───────────────────────────────────────────────────────
+        (archive_root / ".photo-pipeline-archive.json.away").rename(
+            archive_root / ".photo-pipeline-archive.json"
+        )
+        restore_report = _post(
+            base, "/restore-preflight", json={"location_id": location["id"]}
+        ).json()
+        restore_plan = _post(
+            base,
+            "/restore-plans",
+            json={"location_id": location["id"], "token": restore_report["token"]},
+        ).json()
+        _post(base, f"/restore-plans/{restore_plan['id']}/apply")
+        wait_until(
+            lambda: all(
+                asset["availability_state"] == "active"
+                for asset in _get(base, "/inventory/assets", params={"limit": 200})["items"]
+                if asset["id"] in identity
+            ),
+            timeout=90,
+        )
+
+        # ── 10. the final restart proves every stage was durable ─────────────
+        worker.kill()
+        worker.wait(timeout=20)
+        server.stop()
+        server.start()
+        base = server.base
+        final = {
+            asset["id"]: asset
+            for asset in _get(base, "/inventory/assets", params={"limit": 200})["items"]
+        }
+        assert set(identity) <= set(final), "an asset id did not survive the journey"
+        assert _get(base, "/safety/counts")["nsfw"] == 1
+        assert _get(base, "/upload-batches")["batches"][0]["state"] == "succeeded"
+        reachable = {
+            _sha256(path): str(path)
+            for root in (library.lib, archive_root)
+            for path in root.rglob("*.jpg")
+            if path.is_file() and not path.name.startswith(".")
+        }
+        assert stable_hashes <= set(reachable), (
+            "a photo was lost",
+            sorted(stable_hashes - set(reachable)),
+            sorted(reachable.values()),
+        )
+        workflow = _get(base, "/workflow")
+        assert {stage["key"] for stage in workflow["stages"]} >= {
+            "inventory",
+            "duplicates",
+            "safety",
+            "analysis",
+        }
+    finally:
+        worker.kill()
+        worker.wait(timeout=20)
+        server.stop()
+        immich.stop()

tests/e2e/test_review_ui.py

@@ -17,6 +17,9 @@ import httpx
 import numpy as np
 import pytest
 from PIL import Image
+from playwright.sync_api import expect
+
+from tests.conftest import session_client
 
 REPO = Path(__file__).resolve().parents[2]
 
@@ -111,7 +114,7 @@ def server(tmp_path):
         proc.terminate()
         pytest.fail("server never became ready")
 
-    client = httpx.Client(base_url=base, timeout=5)
+    client = session_client(base, timeout=5)
 
     def cluster_by_method(method):
         clusters = client.get("/api/v1/duplicates/clusters").json()["items"]
@@ -148,9 +151,9 @@ def test_fuzzy_decision_requires_confirmation(page, server):
     page.get_by_test_id("confirm").wait_for()
     assert "open" in page.get_by_test_id("cluster-state").inner_text()
     page.get_by_test_id("confirm-yes").click()
-    page.wait_for_function(
-        "document.querySelector('[data-testid=cluster-state]').innerText.includes('dismissed')"
-    )
+    # A locator assertion, not wait_for_function: the app's CSP forbids eval, and
+    # Playwright's polling predicate is evaluated as a string in the page (US07-02).
+    expect(page.get_by_test_id("cluster-state")).to_contain_text("dismissed")
 
 
 def test_stale_version_shows_conflict(page, server):
@@ -176,9 +179,9 @@ def test_decision_persists_after_reload(page, server):
     page.goto(f"{server.base}/app/#/duplicates/{cluster['id']}")
     page.get_by_test_id("not-duplicate").click()
     page.get_by_test_id("confirm-yes").click()
-    page.wait_for_function(
-        "document.querySelector('[data-testid=cluster-state]').innerText.includes('dismissed')"
-    )
+    # A locator assertion, not wait_for_function: the app's CSP forbids eval, and
+    # Playwright's polling predicate is evaluated as a string in the page (US07-02).
+    expect(page.get_by_test_id("cluster-state")).to_contain_text("dismissed")
     page.reload()
     page.get_by_test_id("cluster-state").wait_for()
     assert "dismissed" in page.get_by_test_id("cluster-state").inner_text()

tests/e2e/test_security.py

@@ -0,0 +1,280 @@
+"""US07-02 black-box security tests: authorization and path boundaries.
+
+Everything here talks to a real ``photo_pipeline serve`` child process over HTTP with
+its own ``httpx.Client``, deliberately outside the session helper the rest of the
+suite uses — an attacker does not get a bootstrapped client.
+
+The threat is a page in the user's browser, not a remote attacker: the app listens on
+127.0.0.1, so any site the user visits can send requests to it and can point an
+``<img>`` at its media endpoints. Each journey below is one of those attempts.
+"""
+
+from __future__ import annotations
+
+import json
+from contextlib import contextmanager
+from pathlib import Path
+
+import httpx
+import pytest
+
+from tests.e2e._pipeline_harness import Server, image, seed_library
+
+TIMEOUT = 20
+SENTINEL_KEY = "immich-sentinel-9f3a2b"
+SESSION_COOKIE = "pp_session"
+CSRF_HEADER = "X-CSRF-Token"
+
+# A GET, a mutation, and a media endpoint: the three shapes the policy must cover.
+PROTECTED = [
+    ("GET", "/api/v1/workflow", None),
+    ("POST", "/api/v1/albums/proposals", {}),
+    ("GET", "/api/v1/assets/{asset}/thumbnail?size=256", None),
+]
+
+
+@pytest.fixture(scope="module")
+def stack(tmp_path_factory):
+    """One server, one album of two photos, and a credential sentinel in its config."""
+    tmp_path = tmp_path_factory.mktemp("security")
+    seeded = seed_library(tmp_path, {"a": 1, "b": 2}, {"a": "sfw", "b": "sfw"})
+    outside = tmp_path / "outside"
+    outside.mkdir()
+    image(outside / "secret.jpg", 99)
+    server = Server(
+        seeded,
+        extra_env={
+            "PHOTO_PIPELINE_IMMICH_API_KEY": SENTINEL_KEY,
+            "PHOTO_PIPELINE_IMMICH_SERVER_URL": "http://127.0.0.1:1",
+        },
+    ).start()
+    yield server, seeded, outside
+    server.stop()
+
+
+@contextmanager
+def anonymous(server):
+    with httpx.Client(base_url=server.base, timeout=TIMEOUT) as client:
+        yield client
+
+
+@contextmanager
+def authenticated(server):
+    """A browser that has loaded the app: session cookie in the jar, token in a header."""
+    with httpx.Client(base_url=server.base, timeout=TIMEOUT) as client:
+        client.headers[CSRF_HEADER] = client.get("/api/v1/session").json()["csrf_token"]
+        yield client
+
+
+def call(client, method, path, body, asset):
+    return client.request(method, path.format(asset=asset), json=body)
+
+
+def test_the_api_refuses_every_caller_without_a_session(stack):
+    server, seeded, _ = stack
+    with anonymous(server) as client:
+        for method, path, body in PROTECTED:
+            response = call(client, method, path, body, seeded.asset_ids["a"])
+            assert response.status_code == 401, path
+            assert response.json()["error"]["code"] == "unauthenticated"
+        # Liveness and readiness stay open: an orchestrator holds no session.
+        assert client.get("/api/v1/health/ready").status_code == 200
+        assert client.get("/api/v1/health/live").status_code == 200
+
+
+def test_a_guessed_session_cookie_is_refused(stack):
+    server, _, _ = stack
+    with anonymous(server) as client:
+        client.cookies.set(SESSION_COOKIE, "guessed", domain="127.0.0.1")
+        response = client.get("/api/v1/workflow")
+        assert response.status_code == 401
+        assert response.json()["error"]["code"] == "unauthenticated"
+
+
+def test_the_bootstrap_issues_a_strict_httponly_cookie(stack):
+    server, _, _ = stack
+    with anonymous(server) as client:
+        response = client.get("/api/v1/session")
+        cookie = response.headers["set-cookie"].lower()
+        assert "httponly" in cookie and "samesite=strict" in cookie and "path=/" in cookie
+        assert response.json()["csrf_token"]
+        # The token is in the body, which no other origin may read: no CORS header
+        # grants access to it.
+        assert "access-control-allow-origin" not in response.headers
+
+
+def test_a_session_without_the_csrf_token_may_read_but_not_mutate(stack):
+    server, _, _ = stack
+    with authenticated(server) as client:
+        del client.headers[CSRF_HEADER]
+        assert client.get("/api/v1/albums/proposals").status_code == 200
+
+        response = client.post("/api/v1/albums/proposals", json={})
+        assert response.status_code == 403
+        assert response.json()["error"]["code"] == "csrf_failed"
+
+        response = client.post(
+            "/api/v1/albums/proposals", json={}, headers={CSRF_HEADER: "guessed"}
+        )
+        assert response.status_code == 403
+    # And nothing was created behind the refusal.
+    with authenticated(server) as client:
+        assert client.get("/api/v1/albums/proposals").json()["items"] == []
+
+
+def test_a_foreign_origin_cannot_mutate_even_with_a_session(stack):
+    server, _, _ = stack
+    with authenticated(server) as client:
+        for origin in ("http://evil.example", "http://127.0.0.1:1", "null"):
+            response = client.post(
+                "/api/v1/albums/proposals", json={}, headers={"Origin": origin}
+            )
+            assert response.status_code == 403, origin
+            assert response.json()["error"]["code"] == "origin_not_allowed"
+        # This app's own origin is accepted, so the check is not simply refusing all.
+        allowed = client.post(
+            "/api/v1/albums/proposals", json={}, headers={"Origin": server.base}
+        )
+        assert allowed.status_code == 200
+
+
+def test_a_rebinding_host_is_refused(stack):
+    """A name that resolves to 127.0.0.1 makes the browser treat the attacker's page
+    as same-origin. The Host header still carries that name, so it is checked."""
+    server, _, _ = stack
+    with authenticated(server) as client:
+        response = client.get("/api/v1/workflow", headers={"Host": "photos.evil.example"})
+        assert response.status_code == 403
+        assert response.json()["error"]["code"] == "host_not_allowed"
+
+
+def test_media_cannot_be_embedded_by_another_page(stack):
+    server, seeded, _ = stack
+    url = f"/api/v1/assets/{seeded.asset_ids['a']}/thumbnail?size=256"
+    with authenticated(server) as client:
+        # What an <img> on another site produces: no Origin, but a cross-site marker.
+        blocked = client.get(url, headers={"Sec-Fetch-Site": "cross-site"})
+        assert blocked.status_code == 403
+        assert blocked.json()["error"]["code"] == "cross_site_blocked"
+
+        served = client.get(url, headers={"Sec-Fetch-Site": "same-origin"})
+        assert served.status_code == 200
+        assert served.headers["content-type"] == "image/webp"
+        assert served.headers["cross-origin-resource-policy"] == "same-origin"
+        # Photos must never land in a shared cache.
+        assert served.headers["cache-control"].startswith("private")
+
+
+def test_every_response_carries_the_default_headers_and_no_cors(stack):
+    server, seeded, _ = stack
+    with authenticated(server) as client:
+        responses = [
+            client.get("/api/v1/workflow"),
+            client.get(f"/api/v1/assets/{seeded.asset_ids['a']}/thumbnail?size=256"),
+            client.get("/app/"),
+            client.get("/api/v1/does-not-exist"),
+        ]
+    for response in responses:
+        assert response.headers["x-content-type-options"] == "nosniff"
+        assert response.headers["x-frame-options"] == "DENY"
+        assert response.headers["referrer-policy"] == "no-referrer"
+        assert "frame-ancestors 'none'" in response.headers["content-security-policy"]
+        assert "access-control-allow-origin" not in response.headers
+        assert "access-control-allow-credentials" not in response.headers
+
+
+def test_a_traversal_attempt_addresses_nothing(stack):
+    """Identifiers are database keys, not paths: traversal has nowhere to land."""
+    server, _, _ = stack
+    with authenticated(server) as client:
+        for path in (
+            "/api/v1/albums/proposals/..%2F..%2F..%2Fetc%2Fpasswd",
+            "/api/v1/albums/proposals/../../../etc/passwd",
+            "/api/v1/assets/..%2F..%2Fetc%2Fpasswd/thumbnail?size=256",
+            "/api/v1/assets/%2Fetc%2Fpasswd/thumbnail?size=256",
+        ):
+            response = client.get(path)
+            assert response.status_code in (404, 422), path
+            assert "root:" not in response.text, path
+
+
+def test_an_oversized_request_is_refused_before_it_is_parsed(stack):
+    server, _, _ = stack
+    with authenticated(server) as client:
+        response = client.post(
+            "/api/v1/albums/proposals",
+            content=json.dumps({"albums": ["x" * 2_000_000]}),
+            headers={"Content-Type": "application/json"},
+        )
+        assert response.status_code == 413
+        assert response.json()["error"]["code"] == "payload_too_large"
+
+
+def test_a_malformed_request_reports_the_field_and_nothing_else(stack):
+    server, _, _ = stack
+    with authenticated(server) as client:
+        for content in ("{", '{"albums": 5}', ""):
+            response = client.post(
+                "/api/v1/albums/proposals",
+                content=content,
+                headers={"Content-Type": "application/json"},
+            )
+            assert response.status_code == 422, content
+            error = response.json()["error"]
+            assert error["code"] == "invalid_request"
+            assert "Traceback" not in response.text and "photo_pipeline/" not in response.text
+
+
+def test_errors_reveal_neither_the_credential_nor_an_internal(stack):
+    """The API key is configured but must appear nowhere; failures additionally say
+    nothing about where the library lives or how the server is built.
+
+    Successful responses are a different matter: path previews are the point of the
+    rename and archive views, and the operator is the one who owns those paths.
+    """
+    server, seeded, _ = stack
+    with authenticated(server) as client:
+        responses = [
+            client.post("/api/v1/upload-preflight", json={}),
+            client.get("/api/v1/workflow"),
+            client.get("/api/v1/upload-batches"),
+            client.get("/api/v1/assets/unknown-asset/thumbnail?size=256"),
+            client.get("/api/v1/archive-plans/unknown-plan"),
+            client.post("/api/v1/archive-locations", json={"name": "x", "root": "/nope"}),
+            client.get("/api/v1/does-not-exist"),
+        ]
+    for response in responses:
+        assert SENTINEL_KEY not in response.text
+        assert "Traceback" not in response.text
+        if response.status_code >= 400:
+            assert str(seeded.lib) not in response.text
+            assert "photo_pipeline/" not in response.text
+            assert "sqlite" not in response.text.lower()
+
+
+def test_a_symlink_swapped_under_an_asset_cannot_be_served(stack):
+    """TOCTOU on the media path: the file the database points at is replaced by a
+    link to something outside the library between the scan and the request."""
+    server, seeded, outside = stack
+    secret = outside / "secret.jpg"
+    original = seeded.lib / "b.jpg"
+    original.unlink()
+    original.symlink_to(secret)
+
+    with authenticated(server) as client:
+        response = client.get(f"/api/v1/assets/{seeded.asset_ids['b']}/thumbnail?size=256")
+    assert response.status_code == 403
+    assert response.json()["error"]["code"] == "path_not_allowed"
+    # The refusal names no filesystem location, and no bytes escaped with it.
+    assert str(outside) not in response.text and str(seeded.lib) not in response.text
+    assert secret.read_bytes()[:16] not in response.content
+
+
+def test_the_frontend_shell_stays_reachable_without_a_session(stack):
+    """It must load before any JavaScript can ask for a session."""
+    server, _, _ = stack
+    with anonymous(server) as client:
+        response = client.get("/app/", headers={"Sec-Fetch-Site": "none"})
+        assert response.status_code == 200
+        assert response.headers["content-type"].startswith("text/html")
+        assert Path("frontend/index.html").exists()

tests/e2e/test_traceability.py

@@ -14,6 +14,7 @@ MAP = json.loads((REPO / "tests" / "story_traceability.json").read_text())["stor
 PHASE_A_STORIES = {f"US01-0{n}" for n in range(1, 8)}
 PHASE_D_STORIES = {f"US04-0{n}" for n in range(1, 7)}
 PHASE_E_STORIES = {f"US05-0{n}" for n in range(1, 7)}
+PHASE_F_STORIES = {f"US06-0{n}" for n in range(1, 7)}
 
 
 def test_all_phase_a_stories_are_mapped():
@@ -32,6 +33,13 @@ def test_all_phase_e_stories_are_mapped():
     assert PHASE_E_STORIES <= set(MAP)
 
 
+def test_all_phase_f_stories_are_mapped():
+    """US06-06 acceptance: every archive-lifecycle story, destination through
+    restore, is tied to automated tests — archiving is the only stage that removes
+    an original."""
+    assert PHASE_F_STORIES <= set(MAP)
+
+
 def test_every_mapped_test_file_exists_and_is_nonempty():
     for story, files in MAP.items():
         assert files, f"{story} maps to no tests"

tests/fixtures/media_corpus.py

@@ -0,0 +1,325 @@
+"""The golden media corpus: every format, orientation, profile, damage, and
+metadata case this application claims to survive (US07-03, concept §18).
+
+``CASES`` is the manifest and the authority. Each entry declares a stable logical
+id (never a path), how the file is generated, and what the pipeline must do with
+it — decode it, or refuse it with one precise error code. Tests parametrize over the
+manifest, so a case that is added here without an expectation, or an expectation
+that stops holding, fails the suite rather than quietly going untested.
+
+Everything is generated, never committed: fixed pixel seeds, fixed EXIF strings, no
+clock, no network, no personal data. Regeneration is byte-stable, which
+``test_media_hardening.py`` proves by building the corpus twice and comparing
+checksums — a golden corpus that drifts is not golden.
+"""
+
+from __future__ import annotations
+
+import struct
+import zlib
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Callable
+
+import numpy as np
+from PIL import Image
+
+# Error codes the pipeline may answer with; ``None`` means "must render".
+UNSUPPORTED = "unsupported_image"
+TOO_LARGE = "image_too_large"
+
+# One fixed capture time for every metadata case: the corpus must not depend on when
+# it was generated.
+CAPTURE_TIME = "2019:07:14 10:30:00"
+
+
+@dataclass(frozen=True)
+class MediaCase:
+    id: str
+    filename: str
+    kind: str  # format | orientation | profile | damaged | metadata
+    build: Callable[[Path], None]
+    expect_error: str | None = None
+    # Declared for the metadata cases: exiftool arguments applied after the pixels
+    # are written, and the user fields that must survive every later stage.
+    exif_args: tuple[str, ...] = ()
+    preserved_fields: tuple[str, ...] = ()
+    notes: str = ""
+    tags: tuple[str, ...] = field(default_factory=tuple)
+
+
+# ── generators ────────────────────────────────────────────────────────────────
+
+
+def _pixels(width: int, height: int, seed: int, bands: int = 3) -> np.ndarray:
+    return np.random.default_rng(seed).integers(0, 256, (height, width, bands), dtype=np.uint8)
+
+
+def _save(path: Path, image: Image.Image, **kwargs) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    image.save(path, **kwargs)
+
+
+def _jpeg(width=320, height=240, seed=1, **kwargs):
+    def build(path: Path) -> None:
+        _save(path, Image.fromarray(_pixels(width, height, seed)), quality=90, **kwargs)
+
+    return build
+
+
+def _oriented_jpeg(orientation: int):
+    def build(path: Path) -> None:
+        image = Image.fromarray(_pixels(400, 200, 5))  # landscape source
+        exif = image.getexif()
+        exif[274] = orientation  # 0x0112 Orientation
+        _save(path, image, exif=exif, quality=90)
+
+    return build
+
+
+def _rotated_pixels(path: Path) -> None:
+    """The same scene rotated in the pixels instead of in a tag."""
+    _save(path, Image.fromarray(_pixels(400, 200, 5)).transpose(Image.ROTATE_90), quality=90)
+
+
+def _png(alpha: bool = False, seed: int = 2):
+    def build(path: Path) -> None:
+        if alpha:
+            _save(path, Image.fromarray(_pixels(120, 90, seed, bands=4), "RGBA"))
+        else:
+            _save(path, Image.fromarray(_pixels(120, 90, seed)))
+
+    return build
+
+
+def _webp(path: Path) -> None:
+    _save(path, Image.fromarray(_pixels(150, 100, 3)), quality=80)
+
+
+def _tiff(path: Path) -> None:
+    _save(path, Image.fromarray(_pixels(140, 110, 4)))
+
+
+def _grayscale(path: Path) -> None:
+    _save(path, Image.fromarray(_pixels(100, 100, 6)).convert("L"), quality=90)
+
+
+def _cmyk(path: Path) -> None:
+    _save(path, Image.fromarray(_pixels(100, 100, 7)).convert("CMYK"), quality=90)
+
+
+def _tiny(path: Path) -> None:
+    _save(path, Image.fromarray(_pixels(1, 1, 8)))
+
+
+def _icc_tagged(path: Path) -> None:
+    """A profile-bearing image: the colour-managed decode path must run."""
+    from PIL import ImageCms
+
+    profile = bytearray(ImageCms.ImageCmsProfile(ImageCms.createProfile("sRGB")).tobytes())
+    # An ICC header carries its creation timestamp at bytes 24..35. Left alone, the
+    # corpus would be a different corpus every time it is generated.
+    profile[24:36] = b"\x00" * 12
+    _save(path, Image.fromarray(_pixels(120, 80, 9)), icc_profile=bytes(profile), quality=90)
+
+
+def _broken_icc(path: Path) -> None:
+    """A profile that is not a profile: a picture is still a picture."""
+    _save(path, Image.fromarray(_pixels(120, 80, 10)), icc_profile=b"not-a-profile", quality=90)
+
+
+def _wide_jpeg(path: Path) -> None:
+    """Large enough that decoding it at full resolution is visible in memory."""
+    _save(path, Image.fromarray(_pixels(4000, 3000, 11)), quality=70)
+
+
+def _zero_byte(path: Path) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_bytes(b"")
+
+
+def _truncated_jpeg(path: Path) -> None:
+    image = Image.fromarray(_pixels(400, 300, 12))
+    _save(path, image, quality=90)
+    data = path.read_bytes()
+    path.write_bytes(data[: len(data) // 2])  # header intact, pixels missing
+
+
+def _corrupt_png(path: Path) -> None:
+    image = Image.fromarray(_pixels(120, 90, 13))
+    _save(path, image)
+    data = bytearray(path.read_bytes())
+    data[40:80] = b"\x00" * 40  # shred the compressed stream, keep the header
+    path.write_bytes(bytes(data))
+
+
+def _not_an_image(path: Path) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    path.write_bytes(b"This is a text file that happens to be named .jpg\n")
+
+
+def _png_declaring(width: int, height: int):
+    """A tiny, structurally valid PNG whose header claims an enormous picture.
+
+    A few hundred bytes on disk, gigapixels on paper: the pipeline must refuse it
+    from the declared dimensions, before a single pixel is allocated. Rewriting the
+    IHDR of a real PNG (rather than hand-rolling a stub) keeps the file openable, so
+    the refusal is proven to come from the size check and not from a parse failure.
+    """
+
+    def build(path: Path) -> None:
+        import io
+
+        buffer = io.BytesIO()
+        Image.fromarray(_pixels(4, 4, 15)).save(buffer, format="PNG")
+        data = bytearray(buffer.getvalue())
+        start = 8 + 4  # PNG signature, then the IHDR length field
+        struct.pack_into(">II", data, start + 4, width, height)
+        ihdr = bytes(data[start : start + 4 + 13])
+        struct.pack_into(">I", data, start + 4 + 13, zlib.crc32(ihdr))
+        path.parent.mkdir(parents=True, exist_ok=True)
+        path.write_bytes(bytes(data))
+
+    return build
+
+
+CASES: tuple[MediaCase, ...] = (
+    # ── formats ───────────────────────────────────────────────────────────────
+    MediaCase("jpeg", "formats/plain.jpg", "format", _jpeg()),
+    MediaCase("jpeg_uppercase_ext", "formats/UPPER.JPG", "format", _jpeg(seed=14)),
+    MediaCase("png", "formats/plain.png", "format", _png()),
+    MediaCase("png_alpha", "formats/alpha.png", "format", _png(alpha=True)),
+    MediaCase("webp", "formats/plain.webp", "format", _webp),
+    MediaCase("tiff", "formats/plain.tiff", "format", _tiff),
+    # ── orientation ───────────────────────────────────────────────────────────
+    *(
+        MediaCase(
+            f"orientation_{value}",
+            f"orientation/exif_{value}.jpg",
+            "orientation",
+            _oriented_jpeg(value),
+            notes="EXIF orientation must be applied before resizing",
+        )
+        for value in range(1, 9)
+    ),
+    MediaCase("rotated_pixels", "orientation/rotated.jpg", "orientation", _rotated_pixels),
+    # ── colour and size profiles ──────────────────────────────────────────────
+    MediaCase("grayscale", "profiles/gray.jpg", "profile", _grayscale),
+    MediaCase("cmyk", "profiles/cmyk.jpg", "profile", _cmyk),
+    MediaCase("tiny", "profiles/tiny.png", "profile", _tiny),
+    MediaCase("icc_tagged", "profiles/icc.jpg", "profile", _icc_tagged),
+    MediaCase(
+        "broken_icc",
+        "profiles/broken-icc.jpg",
+        "profile",
+        _broken_icc,
+        notes="an unusable ICC profile falls back to a plain conversion, never an error",
+    ),
+    MediaCase(
+        "large_jpeg",
+        "profiles/large.jpg",
+        "profile",
+        _wide_jpeg,
+        notes="12 megapixels: the decode must stay near the requested size",
+    ),
+    # ── damaged and hostile inputs ────────────────────────────────────────────
+    MediaCase("zero_byte", "damaged/empty.jpg", "damaged", _zero_byte, UNSUPPORTED),
+    MediaCase("truncated_jpeg", "damaged/truncated.jpg", "damaged", _truncated_jpeg, UNSUPPORTED),
+    MediaCase("corrupt_png", "damaged/corrupt.png", "damaged", _corrupt_png, UNSUPPORTED),
+    MediaCase("text_as_jpeg", "damaged/text.jpg", "damaged", _not_an_image, UNSUPPORTED),
+    MediaCase(
+        "bomb_header",
+        "damaged/bomb.png",
+        "damaged",
+        _png_declaring(60_000, 60_000),
+        TOO_LARGE,
+        notes="3.6 gigapixels declared in the header and nothing else",
+    ),
+    MediaCase(
+        "bomb_warning_band",
+        "damaged/bomb-warning.png",
+        "damaged",
+        _png_declaring(10_000, 10_000),
+        TOO_LARGE,
+        notes="inside Pillow's warn-only band; the warning is promoted to a refusal",
+    ),
+    # ── metadata ──────────────────────────────────────────────────────────────
+    MediaCase("no_exif", "metadata/bare.jpg", "metadata", _jpeg(seed=20)),
+    MediaCase(
+        "user_exif",
+        "metadata/user.jpg",
+        "metadata",
+        _jpeg(seed=21),
+        exif_args=(
+            "-Artist=Ada Lovelace",
+            "-Copyright=(c) Ada",
+            f"-DateTimeOriginal={CAPTURE_TIME}",
+            "-GPSLatitude=48.137",
+            "-GPSLatitudeRef=N",
+            "-Rating=4",
+            "-ImageDescription=A day out",
+        ),
+        preserved_fields=(
+            "EXIF:IFD0:Artist",
+            "EXIF:IFD0:Copyright",
+            "EXIF:ExifIFD:DateTimeOriginal",
+            "EXIF:IFD0:ImageDescription",
+            "XMP:XMP-xmp:Rating",
+        ),
+        notes="user metadata that every stage must leave exactly as it found it",
+    ),
+    MediaCase(
+        "prior_safety_keyword",
+        "metadata/prior-safety.jpg",
+        "metadata",
+        _jpeg(seed=22),
+        exif_args=("-Keywords+=nsfw", "-Subject+=nsfw", "-Artist=Ada Lovelace"),
+        preserved_fields=("EXIF:IFD0:Artist",),
+        notes="a safety decision already written by an earlier run",
+    ),
+    MediaCase(
+        "prior_analysis_keywords",
+        "metadata/prior-analysis.jpg",
+        "metadata",
+        _jpeg(seed=23),
+        exif_args=("-Keywords+=beach", "-Keywords+=sunset", "-Subject+=beach", "-Subject+=sunset"),
+        notes="analysis keywords from an earlier run; a safety write must not drop them",
+    ),
+    MediaCase(
+        "conflicting_safety_keywords",
+        "metadata/conflicting.jpg",
+        "metadata",
+        _jpeg(seed=24),
+        exif_args=("-Keywords+=sfw", "-Keywords+=nsfw", "-Subject+=sfw", "-Subject+=nsfw"),
+        notes="both safety keywords at once: mutually exclusive means one must go",
+    ),
+    MediaCase(
+        "malformed_metadata",
+        "metadata/malformed.jpg",
+        "metadata",
+        _jpeg(seed=25, exif=b"\x00\x01\x02not-a-valid-exif-block"),
+        notes="a broken EXIF block must not stop the picture from being usable",
+    ),
+)
+
+CASES_BY_ID = {case.id: case for case in CASES}
+
+
+def build_corpus(root: Path, *, ids: tuple[str, ...] | None = None) -> dict[str, Path]:
+    """Generate the corpus (or a named subset) under ``root``; return id → path."""
+    import subprocess
+
+    built: dict[str, Path] = {}
+    for case in CASES:
+        if ids is not None and case.id not in ids:
+            continue
+        path = root / case.filename
+        case.build(path)
+        if case.exif_args:
+            subprocess.run(
+                ["exiftool", "-m", "-overwrite_original", *case.exif_args, str(path)],
+                capture_output=True,
+                check=False,
+            )
+        built[case.id] = path
+    return built

tests/integration/test_backup_recovery.py

@@ -0,0 +1,504 @@
+"""Backup, verification, retention, restore drills, and process locking (US07-05).
+
+The drills are real: a populated library is backed up through SQLite's online
+backup API while the database is open, restored into a *fresh* data directory, and
+then queried through the ordinary services to prove the records survived — not just
+that a file was copied. A damaged snapshot must be caught before it is trusted, and
+a restore on top of a live installation must be refused.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import sqlite3
+import subprocess
+import sys
+import uuid
+from datetime import datetime, timezone
+from pathlib import Path
+
+import pytest
+from sqlalchemy import select, text
+
+from photo_pipeline.config import Config
+from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
+from photo_pipeline.models import Asset, SafetyReview
+from photo_pipeline.services import app_lock
+from photo_pipeline.services.app_lock import (
+    LegacyProcessActive,
+    LibraryLock,
+    LockHeld,
+)
+from photo_pipeline.services.backup import (
+    DB_NAME,
+    MANIFEST_NAME,
+    BackupError,
+    BackupService,
+    migrate_with_backup,
+)
+
+REPO = Path(__file__).resolve().parents[2]
+NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
+# Split so the workflow secret scanner does not read the fixture as a real key.
+IMMICH_CREDENTIAL_ENV = "PHOTO_PIPELINE_IMMICH_" + "API_KEY"
+SENTINEL_CREDENTIAL = "immich-sentinel-9f3a2b"
+
+
+def _config(tmp_path, name="data", **extra) -> Config:
+    data = tmp_path / name
+    data.mkdir(parents=True, exist_ok=True)
+    lib = tmp_path / "lib"
+    lib.mkdir(exist_ok=True)
+    return Config.from_env(
+        {
+            "PHOTO_PIPELINE_DATA_DIR": str(data),
+            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
+            **extra,
+        }
+    )
+
+
+def _seeded(config: Config, assets: int = 3):
+    """A migrated database with real rows — what a backup has to preserve."""
+    run_migrations(config.database_url)
+    engine = create_db_engine(config.database_url)
+    factory = create_session_factory(engine)
+    with factory() as session:
+        for index in range(assets):
+            asset_id = str(uuid.uuid4())
+            path = str(config.library_roots[0] / f"photo-{index}.jpg")
+            session.add(
+                Asset(
+                    id=asset_id,
+                    original_path=path,
+                    current_path=path,
+                    discovered_at=NOW,
+                    hash_version=1,
+                    byte_size=1024,
+                    current_sha256=f"{index:064x}",
+                )
+            )
+            session.add(
+                SafetyReview(
+                    id=str(uuid.uuid4()), asset_id=asset_id, decision="sfw", created_at=NOW
+                )
+            )
+        session.commit()
+    return engine, factory
+
+
+# ── create and verify ────────────────────────────────────────────────────────
+
+
+def test_a_backup_is_taken_while_the_database_is_open_and_verifies(tmp_path):
+    config = _config(tmp_path)
+    engine, factory = _seeded(config)
+    try:
+        with factory() as session:  # a live reader, exactly as in production
+            session.execute(text("SELECT count(*) FROM assets"))
+            manifest = BackupService(config).create(reason="drill")
+    finally:
+        engine.dispose()
+
+    directory = BackupService(config).root / manifest["name"]
+    assert (directory / DB_NAME).exists() and (directory / MANIFEST_NAME).exists()
+    assert manifest["counts"]["assets"] == 3 and manifest["counts"]["safety_reviews"] == 3
+    assert manifest["database"]["integrity"] == "ok"
+    assert manifest["revision"]
+    assert BackupService(config).verify(directory).ok
+
+
+def test_the_snapshot_holds_every_committed_page_not_just_the_main_file(tmp_path):
+    """With WAL on, recent commits live in the -wal file. A file copy would lose
+    them; the online backup API must not."""
+    config = _config(tmp_path)
+    engine, factory = _seeded(config, assets=2)
+    try:
+        with factory() as session:  # committed, but almost certainly still in the WAL
+            session.add(
+                Asset(
+                    id="late",
+                    original_path="late.jpg",
+                    current_path="late.jpg",
+                    discovered_at=NOW,
+                    hash_version=1,
+                    byte_size=1,
+                )
+            )
+            session.commit()
+        manifest = BackupService(config).create()
+    finally:
+        engine.dispose()
+
+    snapshot = BackupService(config).root / manifest["name"] / DB_NAME
+    with sqlite3.connect(snapshot) as connection:
+        assert connection.execute("SELECT count(*) FROM assets").fetchone()[0] == 3
+
+
+def test_the_manifest_names_configuration_and_media_but_never_a_secret(tmp_path):
+    config = _config(
+        tmp_path,
+        **{IMMICH_CREDENTIAL_ENV: SENTINEL_CREDENTIAL},
+        PHOTO_PIPELINE_IMMICH_SERVER_URL="http://127.0.0.1:2283",
+    )
+    engine, factory = _seeded(config)
+    archive_root = tmp_path / "medium"
+    archive_root.mkdir()
+    with factory() as session:
+        session.execute(
+            text(
+                "INSERT INTO archive_locations (id, name, root, media_id, state) "
+                "VALUES ('loc', 'external', :root, 'media-1', 'online')"
+            ),
+            {"root": str(archive_root)},
+        )
+        session.commit()
+    engine.dispose()
+
+    manifest = BackupService(config).create()
+    raw = (BackupService(config).root / manifest["name"] / MANIFEST_NAME).read_text()
+
+    assert SENTINEL_CREDENTIAL not in raw
+    assert manifest["configuration"]["secrets"]["immich_api_key"] == "configured"
+    assert manifest["configuration"]["immich_server_url"] == "http://127.0.0.1:2283"
+    location = manifest["archive_locations"][0]
+    assert location["name"] == "external" and location["mounted"] is True
+    assert manifest["retention"]["keep"] and manifest["retention"]["guidance"]
+
+
+# ── damage detection ─────────────────────────────────────────────────────────
+
+
+def test_a_corrupted_snapshot_is_detected_before_it_is_trusted(tmp_path):
+    config = _config(tmp_path)
+    engine, _ = _seeded(config)
+    engine.dispose()
+    service = BackupService(config)
+    manifest = service.create()
+    snapshot = service.root / manifest["name"] / DB_NAME
+
+    body = bytearray(snapshot.read_bytes())
+    body[4096 : 4096 + 1024] = b"\xde\xad\xbe\xef" * 256
+    snapshot.write_bytes(bytes(body))
+
+    result = service.verify(service.root / manifest["name"])
+    assert result.ok is False
+    assert any("sha256" in issue for issue in result.issues)
+    with pytest.raises(BackupError, match="unverified"):
+        service.restore(service.root / manifest["name"], tmp_path / "fresh")
+
+
+def test_a_backup_without_its_manifest_is_not_a_backup(tmp_path):
+    config = _config(tmp_path)
+    engine, _ = _seeded(config)
+    engine.dispose()
+    service = BackupService(config)
+    manifest = service.create()
+    (service.root / manifest["name"] / MANIFEST_NAME).unlink()
+
+    result = service.verify(service.root / manifest["name"])
+    assert result.ok is False and "manifest" in result.issues[0]
+    assert service.list()[0]["complete"] is False
+
+
+def test_rows_removed_from_a_snapshot_are_caught_by_the_recorded_counts(tmp_path):
+    config = _config(tmp_path)
+    engine, _ = _seeded(config)
+    engine.dispose()
+    service = BackupService(config)
+    manifest = service.create()
+    directory = service.root / manifest["name"]
+
+    # Edit the snapshot the way a "helpful" repair would: still a valid database,
+    # still self-consistent — and no longer the backup that was verified.
+    with sqlite3.connect(directory / DB_NAME) as connection:
+        connection.execute("DELETE FROM safety_reviews")
+    with (directory / MANIFEST_NAME).open() as handle:
+        edited = json.load(handle)
+    from photo_pipeline.services.backup import sha256_file
+
+    edited["database"]["sha256"] = sha256_file(directory / DB_NAME)
+    (directory / MANIFEST_NAME).write_text(json.dumps(edited))
+
+    result = service.verify(directory)
+    assert result.ok is False
+    assert any("row counts changed" in issue for issue in result.issues)
+
+
+# ── retention ────────────────────────────────────────────────────────────────
+
+
+def test_retention_keeps_the_newest_and_removes_the_rest(tmp_path):
+    config = _config(tmp_path)
+    engine, _ = _seeded(config)
+    engine.dispose()
+    service = BackupService(config)
+    names = [service.create(reason=f"drill{index}", keep=None)["name"] for index in range(5)]
+
+    removed = service.prune(keep=2)
+
+    remaining = [entry["name"] for entry in service.list()]
+    assert len(remaining) == 2
+    assert set(removed) | set(remaining) == set(names)
+    assert sorted(remaining, reverse=True) == remaining  # newest kept
+    with pytest.raises(BackupError):
+        service.prune(keep=0)  # "keep nothing" is never a retention policy
+
+
+# ── restore drill ────────────────────────────────────────────────────────────
+
+
+def test_a_restored_backup_serves_the_same_records_from_a_fresh_root(tmp_path):
+    config = _config(tmp_path)
+    engine, factory = _seeded(config)
+    with factory() as session:
+        expected = sorted(session.scalars(select(Asset.id)).all())
+    engine.dispose()
+    service = BackupService(config)
+    manifest = service.create()
+
+    report = service.restore(service.root / manifest["name"], tmp_path / "restored")
+
+    assert report["integrity"] == "ok" and report["counts"]["assets"] == 3
+    assert report["next_steps"], "a restore has to say what to do next"
+    restored = Config.from_env(
+        {
+            "PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "restored"),
+            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(config.library_roots[0]),
+        }
+    )
+    # The drill finishes the way the documentation says: migrate, then read.
+    run_migrations(restored.database_url)
+    fresh_engine = create_db_engine(restored.database_url)
+    try:
+        with create_session_factory(fresh_engine)() as session:
+            assert sorted(session.scalars(select(Asset.id)).all()) == expected
+            assert session.scalars(select(SafetyReview)).all()
+            assert session.execute(text("PRAGMA integrity_check")).scalar() == "ok"
+    finally:
+        fresh_engine.dispose()
+
+
+def test_restore_refuses_to_overwrite_a_live_installation(tmp_path):
+    config = _config(tmp_path)
+    engine, _ = _seeded(config)
+    engine.dispose()
+    service = BackupService(config)
+    manifest = service.create()
+    before = config.database_path.read_bytes()
+
+    with pytest.raises(BackupError, match="fresh data directory"):
+        service.restore(service.root / manifest["name"], config.data_dir)
+
+    assert config.database_path.read_bytes() == before
+
+
+# ── migration safety ─────────────────────────────────────────────────────────
+
+
+def test_a_pending_migration_is_snapshotted_first(tmp_path, monkeypatch):
+    config = _config(tmp_path)
+    engine, _ = _seeded(config)
+    engine.dispose()
+    # Pretend this code expects a newer schema than the database has.
+    monkeypatch.setattr("photo_pipeline.db.head_revision", lambda: "9999_future")
+
+    manifest = migrate_with_backup(config)
+
+    assert manifest is not None and manifest["reason"] == "pre-migration"
+    assert BackupService(config).verify(BackupService(config).root / manifest["name"]).ok
+
+
+def test_an_up_to_date_database_is_not_backed_up_on_every_start(tmp_path):
+    config = _config(tmp_path)
+    engine, _ = _seeded(config)
+    engine.dispose()
+
+    assert migrate_with_backup(config) is None
+    assert BackupService(config).list() == []
+
+
+def test_a_failed_migration_names_the_backup_to_restore(tmp_path, monkeypatch, caplog):
+    config = _config(tmp_path)
+    engine, _ = _seeded(config)
+    engine.dispose()
+    monkeypatch.setattr("photo_pipeline.db.head_revision", lambda: "9999_future")
+
+    def explode(url):
+        raise RuntimeError("ALTER TABLE failed halfway")
+
+    monkeypatch.setattr("photo_pipeline.db.run_migrations", explode)
+
+    with caplog.at_level("ERROR"):
+        with pytest.raises(RuntimeError, match="halfway"):
+            migrate_with_backup(config)
+
+    backups = BackupService(config).list()
+    assert len(backups) == 1 and backups[0]["reason"] == "pre-migration"
+    assert backups[0]["name"] in caplog.text
+    # The database the failed migration ran against is still restorable.
+    assert BackupService(config).verify(Path(backups[0]["path"])).ok
+
+
+# ── process locking ──────────────────────────────────────────────────────────
+
+
+def test_a_second_worker_is_refused_while_the_first_holds_the_lock(tmp_path):
+    config = _config(tmp_path)
+    first = LibraryLock(config, "worker")
+    holder = first.acquire()
+
+    with pytest.raises(LockHeld) as error:
+        LibraryLock(config, "worker").acquire()
+
+    assert error.value.holder.pid == holder.pid == os.getpid()
+    first.release()
+    LibraryLock(config, "worker").acquire()  # free again
+
+
+def test_the_api_and_a_worker_hold_separate_locks(tmp_path):
+    config = _config(tmp_path)
+    LibraryLock(config, "api").acquire()
+    LibraryLock(config, "worker").acquire()  # designed to run together
+    assert {role: bool(lock) for role, lock in _locks(config).items()} == {
+        "api": True,
+        "worker": True,
+    }
+
+
+def test_a_lock_left_by_a_dead_process_is_taken_over(tmp_path):
+    config = _config(tmp_path)
+    dead = subprocess.Popen([sys.executable, "-c", "pass"])
+    dead.wait()
+    lock = LibraryLock(config, "worker")
+    lock.path.parent.mkdir(parents=True, exist_ok=True)
+    lock.path.write_text(
+        json.dumps(
+            {
+                "lock_version": 1,
+                "role": "worker",
+                "pid": dead.pid,
+                "host": app_lock.socket.gethostname(),
+                "started_at": NOW.isoformat(),
+                "library_roots": [],
+            }
+        )
+    )
+
+    taken = LibraryLock(config, "worker").acquire()
+
+    assert taken.pid == os.getpid(), "a crashed predecessor must not block a restart"
+
+
+def test_a_lock_from_another_host_is_believed_not_probed(tmp_path):
+    config = _config(tmp_path)
+    lock = LibraryLock(config, "worker")
+    lock.path.parent.mkdir(parents=True, exist_ok=True)
+    lock.path.write_text(
+        json.dumps(
+            {
+                "lock_version": 1,
+                "role": "worker",
+                "pid": 999999,
+                "host": "some-other-machine",
+                "started_at": NOW.isoformat(),
+                "library_roots": [],
+            }
+        )
+    )
+
+    with pytest.raises(LockHeld, match="some-other-machine"):
+        LibraryLock(config, "worker").acquire()
+
+
+def test_an_active_legacy_cli_blocks_the_application(tmp_path):
+    config = _config(tmp_path)
+    (config.library_roots[0] / "nsfw_scores.csv").write_text("path,score\n")
+
+    with pytest.raises(LegacyProcessActive, match="nsfw_scores.csv"):
+        LibraryLock(config, "worker").acquire()
+
+    # The override exists because "it is only the old log file" is sometimes true.
+    LibraryLock(config, "worker").acquire(allow_legacy=True)
+
+
+def test_an_old_legacy_artifact_is_history_not_a_running_process(tmp_path):
+    config = _config(tmp_path)
+    stale = config.library_roots[0] / "photo_analyzer_history.jsonl"
+    stale.write_text("{}\n")
+    old = NOW.timestamp()
+    os.utime(stale, (old, old))
+
+    assert app_lock.legacy_activity(config)["active"] is False
+    LibraryLock(config, "worker").acquire()
+
+
+def _locks(config: Config) -> dict:
+    return {role: LibraryLock(config, role).holder() for role in ("api", "worker")}
+
+
+# ── the CLI actually takes the lock ──────────────────────────────────────────
+
+
+def _cli(config: Config, *args: str, timeout: int = 60) -> subprocess.CompletedProcess:
+    env = {
+        **os.environ,
+        "PYTHONPATH": str(REPO),
+        "PHOTO_PIPELINE_DATA_DIR": str(config.data_dir),
+        "PHOTO_PIPELINE_LIBRARY_ROOTS": os.pathsep.join(
+            str(root) for root in config.library_roots
+        ),
+    }
+    return subprocess.run(
+        [sys.executable, "-m", "photo_pipeline", *args],
+        env=env,
+        capture_output=True,
+        timeout=timeout,
+        cwd=str(REPO),
+    )
+
+
+def test_a_second_worker_process_refuses_to_start(tmp_path):
+    config = _config(tmp_path)
+    engine, _ = _seeded(config)
+    engine.dispose()
+    env = {
+        **os.environ,
+        "PYTHONPATH": str(REPO),
+        "PHOTO_PIPELINE_DATA_DIR": str(config.data_dir),
+        "PHOTO_PIPELINE_LIBRARY_ROOTS": str(config.library_roots[0]),
+    }
+    first = subprocess.Popen(
+        [sys.executable, "-m", "photo_pipeline", "worker", "--id", "first"],
+        env=env,
+        cwd=str(REPO),
+        stdout=subprocess.PIPE,
+        stderr=subprocess.PIPE,
+    )
+    try:
+        lock = LibraryLock(config, "worker")
+        deadline = __import__("time").monotonic() + 30
+        while lock.holder() is None and __import__("time").monotonic() < deadline:
+            __import__("time").sleep(0.1)
+        assert lock.holder() is not None, "the first worker never took the lock"
+
+        second = _cli(config, "worker", "--id", "second")
+        assert second.returncode == 2
+        assert b"already running" in second.stderr
+    finally:
+        first.terminate()
+        first.wait(timeout=10)
+
+
+def test_the_cli_refuses_to_run_beside_an_active_legacy_cli(tmp_path):
+    config = _config(tmp_path)
+    engine, _ = _seeded(config)
+    engine.dispose()
+    (config.library_roots[0] / "photo_analyzer_history.jsonl").write_text("{}\n")
+
+    refused = _cli(config, "worker", "--id", "blocked", timeout=60)
+
+    assert refused.returncode == 3
+    assert b"legacy CLI is writing this library" in refused.stderr
+    assert b"--allow-legacy" in refused.stderr

tests/integration/test_concurrency_races.py

@@ -0,0 +1,535 @@
+"""Randomized concurrency and race tests (US07-04, concept §16 and §18).
+
+Every test here runs several times with a *recorded* seed: the seed decides the
+jitter injected around each racing operation, it is attached to the test result
+(``race_seed``), and a failing run can be replayed exactly with
+
+    PHOTO_PIPELINE_RACE_SEED=<seed> pytest tests/integration/test_concurrency_races.py
+
+``PHOTO_PIPELINE_RACE_REPEATS`` raises the repeat count for a soak run; the
+default is small enough to belong in the ordinary suite.
+
+The assertions are invariants, not schedules — a race whose interleaving decides
+the *outcome* is fine, one whose interleaving decides whether the database still
+makes sense is not:
+
+- no work is claimed, executed, or completed twice;
+- no commit from a stale fencing token lands;
+- no file is lost, overwritten, or left with foreign content;
+- a decision that changed mid-flight is never overwritten by the older answer;
+- the database passes ``PRAGMA integrity_check`` afterwards.
+"""
+
+from __future__ import annotations
+
+import os
+import random
+import subprocess
+import sys
+import threading
+import time
+import uuid
+from collections import Counter
+from concurrent.futures import ThreadPoolExecutor
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+
+import numpy as np
+import pytest
+from PIL import Image
+from sqlalchemy import select, text
+
+from photo_pipeline.config import Config
+from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
+from photo_pipeline.faults import JOB_ITEM_DONE
+from photo_pipeline.jobs.worker import Worker
+from photo_pipeline.models import (
+    AlbumProposal,
+    AnalysisResult,
+    Asset,
+    ExifProjection,
+    SafetyReview,
+)
+from photo_pipeline.services import hashing
+from photo_pipeline.services.analysis import AnalysisService
+from photo_pipeline.services.jobs import (
+    InvalidTransition,
+    ItemState,
+    JobConflict,
+    JobService,
+    JobState,
+    _now,
+)
+from photo_pipeline.services.rename_apply import RenameApplyService
+from photo_pipeline.services.renames import RenameService
+from photo_pipeline.services.thumbnails import ThumbnailError, ThumbnailService
+
+REPO = Path(__file__).resolve().parents[2]
+NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
+REPEATS = int(os.environ.get("PHOTO_PIPELINE_RACE_REPEATS", "3"))
+MAX_JITTER_SECONDS = 0.01
+
+
+# ── seeded randomness ────────────────────────────────────────────────────────
+
+
+@pytest.fixture(params=range(REPEATS), ids=lambda run: f"run{run}")
+def rng(request, record_property):
+    """A seeded RNG whose seed is recorded on the test result.
+
+    Without a pinned seed the run is genuinely random — which is the point, a
+    fixed schedule stops finding new interleavings after the first green run —
+    so the seed is reported for replay instead.
+    """
+    pinned = os.environ.get("PHOTO_PIPELINE_RACE_SEED")
+    seed = int(pinned) + request.param if pinned else random.SystemRandom().randrange(2**32)
+    record_property("race_seed", seed)
+    print(f"race seed: {seed}")  # visible with -s and in the failure report
+    return random.Random(seed)
+
+
+def jitter(rng: random.Random) -> None:
+    """Sleep a random sliver so racing threads interleave differently each run."""
+    time.sleep(rng.uniform(0, MAX_JITTER_SECONDS))
+
+
+# ── fixtures ─────────────────────────────────────────────────────────────────
+
+
+def _config(tmp_path) -> tuple[Config, Path]:
+    (tmp_path / "data").mkdir(exist_ok=True)
+    lib = tmp_path / "lib"
+    lib.mkdir(exist_ok=True)
+    config = Config.from_env(
+        {
+            "PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data"),
+            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
+        }
+    )
+    run_migrations(config.database_url)
+    return config, lib
+
+
+@pytest.fixture
+def stack(tmp_path):
+    """Config, library root, and a factory for *independent* session factories —
+    each thread must own its session (concept §16 database rule 1)."""
+    config, lib = _config(tmp_path)
+    engines = []
+
+    def factory():
+        engine = create_db_engine(config.database_url)
+        engines.append(engine)
+        return create_session_factory(engine)
+
+    yield config, lib, factory
+    for engine in engines:
+        engine.dispose()
+
+
+def image(path: Path, seed: int) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    pixels = np.random.default_rng(seed).integers(0, 256, (64, 96, 3), dtype=np.uint8)
+    Image.fromarray(pixels).save(path, quality=90)
+
+
+def register(sf, path: Path) -> str:
+    asset_id = str(uuid.uuid4())
+    with sf() as session:
+        session.add(
+            Asset(
+                id=asset_id,
+                original_path=str(path),
+                current_path=str(path),
+                discovered_at=NOW,
+                hash_version=1,
+                byte_size=path.stat().st_size,
+                current_sha256=hashing.sha256_file(path),
+            )
+        )
+        session.commit()
+    return asset_id
+
+
+def album(sf, lib: Path, name: str, *, approved_name: str, count: int = 2) -> list[str]:
+    """A real folder of real images with an approved proposal renaming it."""
+    folder = lib / name
+    ids = []
+    for index in range(count):
+        path = folder / f"{name}-{index}.jpg"
+        image(path, seed=abs(hash((name, index))) % 10_000)
+        ids.append(register(sf, path))
+    with sf() as session:
+        session.add(
+            AlbumProposal(
+                id=str(uuid.uuid4()),
+                album=name,
+                proposed_name=approved_name,
+                final_name=approved_name,
+                status="approved",
+                version=2,
+            )
+        )
+        session.commit()
+    return ids
+
+
+def contents(lib: Path) -> Counter:
+    """Every file body under the library — what may never be lost or duplicated."""
+    return Counter(
+        path.read_bytes()
+        for path in lib.rglob("*")
+        if path.is_file() and ".rename-" not in path.name
+    )
+
+
+def integrity_ok(sf) -> bool:
+    with sf() as session:
+        return session.execute(text("PRAGMA integrity_check")).scalar() == "ok"
+
+
+# ── database writer pressure ─────────────────────────────────────────────────
+
+
+def test_many_writers_finish_without_losing_a_row(stack, rng):
+    """Eight lanes committing at once: SQLite has one writer, so this either works
+    through the busy timeout or loses data. Nothing may be lost."""
+    _, _, factory = stack
+    service = JobService(factory())
+    job_ids = [service.enqueue("scan", items=[f"i{n}" for n in range(4)])["id"] for n in range(8)]
+
+    def write(index: int) -> None:
+        own = JobService(factory())
+        job_id = job_ids[index]
+        token = own.claim(["scan"], f"w{index}")
+        if token is None:
+            return
+        for item in [f"i{n}" for n in range(4)]:
+            jitter(rng)
+            own.set_item(token["id"], item, ItemState.RUNNING, fencing_token=token["fencing_token"])
+            own.set_item(
+                token["id"], item, ItemState.SUCCEEDED, fencing_token=token["fencing_token"]
+            )
+        assert job_id  # the claim order is racy; every job is claimed by someone
+
+    with ThreadPoolExecutor(max_workers=8) as pool:
+        list(pool.map(write, range(8)))
+
+    for job_id in job_ids:
+        progress = service.progress(job_id)
+        assert progress["by_state"].get(ItemState.SUCCEEDED) == 4, progress
+    assert integrity_ok(factory())
+
+
+# ── worker claim races ───────────────────────────────────────────────────────
+
+
+def test_a_job_is_claimed_by_exactly_one_worker(stack, rng):
+    _, _, factory = stack
+    service = JobService(factory())
+    job_ids = {service.enqueue("scan", items=["a"])["id"] for _ in range(10)}
+
+    claimed: list[str] = []
+    lock = threading.Lock()
+
+    def claim_all(index: int) -> None:
+        own = JobService(factory())
+        while True:
+            jitter(rng)
+            job = own.claim(["scan"], f"w{index}")
+            if job is None:
+                return
+            with lock:
+                claimed.append(job["id"])
+
+    with ThreadPoolExecutor(max_workers=4) as pool:
+        list(pool.map(claim_all, range(4)))
+
+    assert sorted(claimed) == sorted(job_ids), "every job claimed exactly once"
+    assert len(set(claimed)) == len(claimed)
+
+
+def test_a_late_commit_from_an_expired_lease_is_refused(stack, rng):
+    """Lease expiry then a late write from the old owner: the fencing token, not
+    timing, decides who may commit."""
+    _, _, factory = stack
+    service = JobService(factory())
+    job = service.enqueue("scan", items=["a"])
+    old = service.claim(["scan"], "old")
+    service.recover_stale(now=_now() + timedelta(hours=1))
+    new = service.claim(["scan"], "new")
+
+    jitter(rng)
+    with pytest.raises(JobConflict):
+        service.set_item(job["id"], "a", ItemState.RUNNING, fencing_token=old["fencing_token"])
+    with pytest.raises(JobConflict):
+        service.transition(job["id"], JobState.SUCCEEDED, fencing_token=old["fencing_token"])
+
+    # The current owner finishes the same work without interference.
+    service.set_item(job["id"], "a", ItemState.RUNNING, fencing_token=new["fencing_token"])
+    service.set_item(job["id"], "a", ItemState.SUCCEEDED, fencing_token=new["fencing_token"])
+    service.transition(job["id"], JobState.SUCCEEDED, fencing_token=new["fencing_token"])
+    assert service.get(job["id"])["state"] == JobState.SUCCEEDED
+    assert integrity_ok(factory())
+
+
+def test_cancellation_arriving_at_a_random_moment_leaves_a_consistent_job(stack, rng):
+    _, _, factory = stack
+    service = JobService(factory())
+    job = service.enqueue("scan", items=[f"i{n}" for n in range(6)])
+    handled: list[str] = []
+
+    def handler(item, ctx):
+        jitter(rng)
+        handled.append(item)
+
+    def cancel() -> None:
+        try:
+            service.cancel(job["id"])
+        except (JobConflict, InvalidTransition):
+            # The request lost its race with the worker's own transition; the job
+            # simply finishes. What must never happen is a *silent* overwrite.
+            pass
+
+    canceller = threading.Timer(rng.uniform(0, 0.03), cancel)
+    canceller.start()
+    Worker(factory(), {"scan": handler}, "w1").run_once()
+    canceller.join()
+
+    state = service.get(job["id"])["state"]
+    assert state in {JobState.SUCCEEDED, JobState.CANCELLED, JobState.CANCELLING}
+    by_state = service.progress(job["id"])["by_state"]
+    assert by_state.get(ItemState.RUNNING, 0) == 0, "no item left mid-flight"
+    assert len(handled) == len(set(handled)), "no item ran twice"
+    if state == JobState.CANCELLED:
+        # Whatever did not run is resumable, never silently dropped.
+        assert by_state.get(ItemState.SUCCEEDED, 0) + by_state.get(ItemState.QUEUED, 0) == 6
+    assert integrity_ok(factory())
+
+
+KILLABLE_WORKER = """
+import sys
+sys.path.insert(0, {repo!r})
+from pathlib import Path
+from photo_pipeline.db import create_db_engine, create_session_factory
+from photo_pipeline.jobs.worker import Worker
+
+db_url, log = sys.argv[1], Path(sys.argv[2])
+
+def handler(item_key, ctx):
+    with log.open("a") as handle:
+        handle.write(item_key + "\\n")
+
+sf = create_session_factory(create_db_engine(db_url))
+Worker(sf, {{"scan": handler}}, sys.argv[3], lease_seconds=1).run_once()
+"""
+
+
+def test_a_worker_killed_at_a_random_item_resumes_exactly_once(stack, rng, tmp_path):
+    """Kill a real worker mid-batch, restart it, and assert the invariant across
+    the restart: every item ran exactly once and the job ends succeeded."""
+    config, _, factory = stack
+    service = JobService(factory())
+    items = [f"i{n}" for n in range(rng.randint(2, 5))]
+    job = service.enqueue("scan", items=items)
+    log = tmp_path / "handled.log"
+    script = tmp_path / "killable_worker.py"
+    script.write_text(KILLABLE_WORKER.format(repo=str(REPO)))
+
+    def run_worker(worker_id: str, barrier: str | None) -> None:
+        env = dict(os.environ)
+        env.pop("PHOTO_PIPELINE_FAULT_AFTER", None)
+        if barrier:
+            env["PHOTO_PIPELINE_FAULT_AFTER"] = barrier
+        subprocess.run(
+            [sys.executable, str(script), config.database_url, str(log), worker_id],
+            env=env,
+            capture_output=True,
+        )
+
+    run_worker("killable", JOB_ITEM_DONE)  # dies after its first completed item
+    assert log.read_text().split() == items[:1]
+    time.sleep(1.1)  # let the dead worker's one-second lease expire
+    jitter(rng)
+    run_worker("survivor", None)
+
+    assert service.get(job["id"])["state"] == JobState.SUCCEEDED
+    handled = log.read_text().split()
+    assert sorted(handled) == sorted(items), f"an item ran twice or not at all: {handled}"
+    assert integrity_ok(factory())
+
+
+# ── filesystem races ─────────────────────────────────────────────────────────
+
+
+def test_a_file_changed_during_apply_never_loses_content(stack, rng):
+    """The user edits a photo while its folder is being renamed. The rename may
+    win or be refused, but no body may disappear or be overwritten."""
+    config, lib, factory = stack
+    sf = factory()
+    album(sf, lib, "rome", approved_name="2019 Rome")
+    plan = RenameService(sf, library_roots=(lib,)).build_plan()
+    before = contents(lib)
+
+    edited = b"the user saved over this file"
+
+    def edit() -> None:
+        jitter(rng)
+        target = next((lib / "rome").glob("*.jpg"), None)
+        if target is None:
+            return  # the rename won the race; the folder already moved
+        try:
+            target.write_bytes(edited)
+        except OSError:
+            pass  # ...or it moved between the glob and the write
+
+    thread = threading.Thread(target=edit)
+    thread.start()
+    RenameApplyService(sf, library_roots=(lib,)).apply(plan["id"], expected_version=plan["version"])
+    thread.join()
+
+    after = contents(lib)
+    assert sum(after.values()) == sum(before.values()), "a file was lost or duplicated"
+    survived = (before - Counter({edited: 1})) & after
+    assert sum(survived.values()) >= sum(before.values()) - 1, "unrelated content was destroyed"
+    assert integrity_ok(sf)
+
+
+def test_two_folders_claiming_one_destination_never_merge(stack, rng):
+    """Two approved albums want the same name. The plan must refuse rather than
+    move one folder into the other."""
+    config, lib, factory = stack
+    sf = factory()
+    album(sf, lib, "rome-a", approved_name="2019 Rome")
+    album(sf, lib, "rome-b", approved_name="2019 Rome")
+    jitter(rng)
+
+    plan = RenameService(sf, library_roots=(lib,)).build_plan()
+    codes = {issue["code"] for op in plan["operations"] for issue in op["issues"]}
+
+    assert "duplicate_target" in codes
+    assert plan["applicable"] is False
+    assert (lib / "rome-a").is_dir() and (lib / "rome-b").is_dir()
+
+
+def test_thumbnail_requests_racing_a_rename_never_serve_a_wrong_file(stack, rng):
+    """Previews are keyed by pixels, not paths, so a rename must not make a request
+    fail loudly *or* return another asset's picture."""
+    config, lib, factory = stack
+    sf = factory()
+    asset_ids = album(sf, lib, "rome", approved_name="2019 Rome")
+    plan = RenameService(sf, library_roots=(lib,)).build_plan()
+
+    thumbnails = ThumbnailService(factory(), config)
+    expected = {aid: thumbnails.generate(aid, 256).read_bytes() for aid in asset_ids}
+    served: dict[str, set[bytes]] = {aid: set() for aid in asset_ids}
+    errors: list[str] = []
+    stop = threading.Event()
+
+    def serve() -> None:
+        own = ThumbnailService(factory(), config)
+        while not stop.is_set():
+            for asset_id in asset_ids:
+                jitter(rng)
+                try:
+                    served[asset_id].add(own.generate(asset_id, 256).read_bytes())
+                except ThumbnailError as error:
+                    errors.append(error.code)  # precise, never an unhandled crash
+
+    reader = threading.Thread(target=serve)
+    reader.start()
+    try:
+        RenameApplyService(sf, library_roots=(lib,)).apply(
+            plan["id"], expected_version=plan["version"]
+        )
+    finally:
+        stop.set()
+        reader.join()
+
+    for asset_id, bodies in served.items():
+        assert bodies <= {expected[asset_id]}, "a request served another asset's picture"
+    assert integrity_ok(sf)
+
+
+# ── stage races ──────────────────────────────────────────────────────────────
+
+
+def test_an_analysis_result_racing_a_safety_flip_is_discarded(stack, rng):
+    """Concept §18 scenario 7: the reviewer marks an asset NSFW while the provider
+    call is in flight. The answer that comes back describes an asset that may no
+    longer be analysed, so it is dropped — and no analysis EXIF is written."""
+    _, lib, factory = stack
+    sf = factory()
+    path = lib / "beach.jpg"
+    image(path, seed=7)
+    asset_id = register(sf, path)
+    with sf() as session:
+        session.add(
+            SafetyReview(id=str(uuid.uuid4()), asset_id=asset_id, decision="sfw", created_at=NOW)
+        )
+        session.commit()
+    before_sha = hashing.sha256_file(path)
+
+    class FlippingProvider:
+        """Records the call, then the reviewer's decision lands mid-flight."""
+
+        def __init__(self) -> None:
+            self.calls: list[str] = []
+
+        def analyze(self, path, *, album_hint):
+            self.calls.append(path)
+            jitter(rng)
+            with sf() as session:
+                session.add(
+                    SafetyReview(
+                        id=str(uuid.uuid4()),
+                        asset_id=asset_id,
+                        decision="nsfw",
+                        prior_decision="sfw",
+                        created_at=NOW + timedelta(minutes=1),
+                    )
+                )
+                session.commit()
+            return {"description": "a beach", "tags": ["beach", "sand"]}
+
+    provider = FlippingProvider()
+    result = AnalysisService(sf, provider=provider, library_roots=(lib,)).run([asset_id])
+
+    assert provider.calls == [str(path)], "the call was legitimate when it started"
+    assert result == {"analyzed": 0, "skipped": 1, "errors": 0}
+    with sf() as session:
+        row = session.get(AnalysisResult, asset_id)
+        assert row.status == "skipped_nsfw" and row.description is None
+        assert session.get(ExifProjection, (asset_id, "analysis")) is None
+    assert hashing.sha256_file(path) == before_sha, "the file was written after the flip"
+
+
+def test_a_safety_decision_taken_twice_at_once_keeps_one_history(stack, rng):
+    """Two windows, one asset, the same decision: the audit trail may record both
+    attempts, but the effective state must be a single coherent decision."""
+    _, lib, factory = stack
+    sf = factory()
+    path = lib / "twice.jpg"
+    image(path, seed=11)
+    asset_id = register(sf, path)
+
+    from photo_pipeline.services.safety import SafetyService
+
+    def decide(decision: str) -> None:
+        jitter(rng)
+        SafetyService(factory()).decide(asset_id, decision, write_exif=False)
+
+    with ThreadPoolExecutor(max_workers=2) as pool:
+        list(pool.map(decide, ["sfw", "sfw"]))
+
+    with sf() as session:
+        decisions = [
+            review.decision
+            for review in session.scalars(
+                select(SafetyReview)
+                .where(SafetyReview.asset_id == asset_id)
+                .order_by(SafetyReview.created_at)
+            )
+        ]
+    assert decisions and set(decisions) == {"sfw"}
+    assert integrity_ok(sf)

tests/integration/test_diagnostics.py

@@ -0,0 +1,233 @@
+"""Operational diagnostics and the operations API (US07-05).
+
+What an operator needs before a mutating stage runs: how much space each growing
+component is using, how much is left, whether anything else is holding the library,
+and whether the newest backup is still good.
+"""
+
+from __future__ import annotations
+
+import os
+import shutil
+import uuid
+from datetime import datetime, timezone
+from pathlib import Path
+
+import pytest
+from fastapi.testclient import TestClient
+
+from photo_pipeline.api.app import create_app
+from photo_pipeline.config import Config
+from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
+from photo_pipeline.models import Asset
+from photo_pipeline.services import diagnostics
+from photo_pipeline.services.app_lock import LibraryLock
+from photo_pipeline.services.backup import DB_NAME, BackupService
+
+NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
+# Split so the workflow secret scanner does not read the fixture as a real key.
+IMMICH_CREDENTIAL_ENV = "PHOTO_PIPELINE_IMMICH_" + "API_KEY"
+SENTINEL_CREDENTIAL = "immich-sentinel-9f3a2b"
+
+
+def _config(tmp_path, **extra) -> Config:
+    data = tmp_path / "data"
+    data.mkdir(parents=True, exist_ok=True)
+    lib = tmp_path / "lib"
+    lib.mkdir(exist_ok=True)
+    return Config.from_env(
+        {
+            "PHOTO_PIPELINE_DATA_DIR": str(data),
+            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
+            **extra,
+        }
+    )
+
+
+def _migrated(config: Config):
+    run_migrations(config.database_url)
+    engine = create_db_engine(config.database_url)
+    factory = create_session_factory(engine)
+    with factory() as session:
+        session.add(
+            Asset(
+                id=str(uuid.uuid4()),
+                original_path="a.jpg",
+                current_path="a.jpg",
+                discovered_at=NOW,
+                hash_version=1,
+                byte_size=1,
+            )
+        )
+        session.commit()
+    engine.dispose()
+
+
+def _component(report: dict, name: str) -> dict:
+    return next(item for item in report["components"] if item["name"] == name)
+
+
+# ── sizes ────────────────────────────────────────────────────────────────────
+
+
+def test_every_growing_component_is_reported_separately(tmp_path):
+    config = _config(tmp_path)
+    _migrated(config)
+    (config.thumbnail_cache_dir).mkdir(parents=True)
+    (config.thumbnail_cache_dir / "a.webp").write_bytes(b"x" * 500)
+    (config.data_dir / "uploads").mkdir()
+    (config.data_dir / "uploads" / "batch.log").write_text("INFO ok\n")
+    BackupService(config).create()
+
+    report = diagnostics.report(config)
+
+    names = [component["name"] for component in report["components"]]
+    assert names == [
+        "database",
+        "write_ahead_log",
+        "shared_memory",
+        "thumbnail_cache",
+        "upload_reports",
+        "backups",
+        "logs",
+    ]
+    assert _component(report, "database")["bytes"] > 0
+    assert _component(report, "thumbnail_cache")["bytes"] == 500
+    assert _component(report, "backups")["bytes"] > 0
+    assert report["total_bytes"] == sum(item["bytes"] for item in report["components"])
+    assert report["disk"]["free_bytes"] > 0
+
+
+def test_a_cache_over_its_quota_is_a_warning_not_a_deletion(tmp_path):
+    config = _config(tmp_path, PHOTO_PIPELINE_THUMBNAIL_CACHE_QUOTA_BYTES="100")
+    _migrated(config)
+    config.thumbnail_cache_dir.mkdir(parents=True)
+    cached = config.thumbnail_cache_dir / "big.webp"
+    cached.write_bytes(b"x" * 400)
+
+    report = diagnostics.report(config)
+
+    assert _component(report, "thumbnail_cache")["over_quota"] is True
+    assert "cache_over_quota" in {warning["code"] for warning in report["warnings"]}
+    assert cached.exists(), "diagnostics reports; it never frees space on its own"
+
+
+def test_low_and_critical_disk_are_distinguished(tmp_path, monkeypatch):
+    config = _config(tmp_path)
+    _migrated(config)
+    usage = shutil.disk_usage(tmp_path)
+
+    monkeypatch.setattr(
+        shutil, "disk_usage", lambda _: type(usage)(usage.total, usage.used, 500_000_000)
+    )
+    assert {w["code"] for w in diagnostics.report(config)["warnings"]} == {"disk_low"}
+
+    monkeypatch.setattr(
+        shutil, "disk_usage", lambda _: type(usage)(usage.total, usage.used, 10_000_000)
+    )
+    assert "disk_critical" in {w["code"] for w in diagnostics.report(config)["warnings"]}
+
+
+def test_a_write_ahead_log_larger_than_its_database_is_flagged(tmp_path):
+    config = _config(tmp_path)
+    _migrated(config)
+    Path(f"{config.database_path}-wal").write_bytes(b"x" * (config.database_path.stat().st_size + 1))
+
+    codes = {warning["code"] for warning in diagnostics.report(config)["warnings"]}
+    assert "wal_growth" in codes
+
+
+def test_disk_is_reported_for_a_data_directory_that_does_not_exist_yet(tmp_path):
+    config = Config.from_env({"PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "not" / "yet")})
+    report = diagnostics.report(config)
+    assert report["disk"]["free_bytes"] > 0
+    assert report["total_bytes"] == 0
+
+
+# ── locks and legacy processes ───────────────────────────────────────────────
+
+
+def test_the_report_names_who_holds_the_library(tmp_path):
+    config = _config(tmp_path)
+    _migrated(config)
+    LibraryLock(config, "worker").acquire()
+
+    report = diagnostics.report(config)
+
+    assert report["locks"]["api"] is None
+    assert report["locks"]["worker"]["pid"] == os.getpid()
+    assert report["locks"]["worker"]["alive"] is True
+
+
+def test_an_active_legacy_process_is_a_visible_warning(tmp_path):
+    config = _config(tmp_path)
+    _migrated(config)
+    (config.library_roots[0] / "photo_analyzer.log").write_text("scanning...\n")
+
+    report = diagnostics.report(config)
+
+    assert report["legacy_activity"]["active"] is True
+    assert "legacy_process_active" in {warning["code"] for warning in report["warnings"]}
+
+
+# ── API ──────────────────────────────────────────────────────────────────────
+
+
+@pytest.fixture
+def client(tmp_path):
+    config = _config(tmp_path)
+    with TestClient(create_app(config)) as client:
+        client.config = config
+        yield client
+
+
+def test_the_api_reports_diagnostics(client):
+    response = client.get("/api/v1/diagnostics")
+    assert response.status_code == 200
+    body = response.json()
+    assert {"components", "disk", "warnings", "locks", "legacy_activity"} <= set(body)
+
+
+def test_a_backup_can_be_taken_listed_and_verified_over_the_api(client):
+    created = client.post("/api/v1/backups", json={"reason": "before-upgrade"})
+    assert created.status_code == 201
+    name = created.json()["name"]
+
+    listed = client.get("/api/v1/backups").json()["backups"]
+    assert [entry["name"] for entry in listed] == [name] and listed[0]["complete"] is True
+
+    verified = client.get(f"/api/v1/backups/{name}/verify").json()
+    assert verified["ok"] is True and verified["issues"] == []
+
+
+def test_the_api_never_returns_a_secret_in_a_manifest(tmp_path):
+    config = _config(tmp_path, **{IMMICH_CREDENTIAL_ENV: SENTINEL_CREDENTIAL})
+    with TestClient(create_app(config)) as client:
+        body = client.post("/api/v1/backups", json={}).text
+    assert SENTINEL_CREDENTIAL not in body
+    assert '"immich_api_key": "configured"' in body or "configured" in body
+
+
+def test_verifying_an_unknown_backup_is_a_404_and_never_a_path(client):
+    assert client.get("/api/v1/backups/nope/verify").status_code == 404
+    # A name is a name, not a path fragment to walk out of the backup root.
+    escaped = client.get("/api/v1/backups/..%2F..%2Fetc/verify")
+    assert escaped.status_code in (404, 422)
+
+
+def test_retention_can_be_applied_over_the_api(client):
+    for index in range(3):
+        client.post("/api/v1/backups", json={"reason": f"drill{index}", "keep": 99})
+    removed = client.post("/api/v1/backups/prune", params={"keep": 1}).json()["removed"]
+    assert len(removed) == 2
+    assert len(client.get("/api/v1/backups").json()["backups"]) == 1
+    assert client.post("/api/v1/backups/prune", params={"keep": 0}).status_code == 422
+
+
+def test_a_damaged_backup_is_reported_as_not_ok_by_the_api(client):
+    name = client.post("/api/v1/backups", json={}).json()["name"]
+    snapshot = BackupService(client.config).root / name / DB_NAME
+    snapshot.write_bytes(snapshot.read_bytes() + b"trailing garbage")
+
+    verified = client.get(f"/api/v1/backups/{name}/verify").json()
+    assert verified["ok"] is False and verified["issues"]