US09-05: Automate Documentation Acceptance (#111)

.gitea/workflows/test.yml

@@ -101,3 +101,39 @@ jobs:
         with:
           name: container-gate-${{ gitea.sha }}
           path: gate-evidence
+
+  # The manuals, checked against the application they describe (US09-05). Runs on
+  # every pull request as well as `main`: documentation drifts by the same commits
+  # that change behaviour, and telling the author while the change is still open is
+  # the only time the fix is cheap.
+  documentation:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+
+      - name: Install the application and its test dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e '.[test]'
+          python -m playwright install --with-deps chromium
+
+      - name: Documentation gate
+        # One command: every phase_i check, offline and in the browser, with the
+        # evidence retained. It accepts no skip — a check that did not run is a page
+        # nobody compared with the code.
+        env:
+          PHOTO_PIPELINE_DATA_DIR: ${{ gitea.workspace }}/docs-gate-data
+        run: python -m photo_pipeline docs-gate --output docs-evidence
+
+      - name: Keep the evidence
+        if: always()
+        uses: actions/upload-artifact@v3
+        with:
+          name: docs-gate-${{ gitea.sha }}
+          path: docs-evidence

README.md

@@ -4,6 +4,24 @@ Integrated, restart-safe photo analysis, duplicate review, metadata, upload, and
 archive workflow. Planning lives in `INTEGRATED_PIPELINE_CONCEPT.md` and
 `delivery_backlog/`.
 
+## Documentation
+
+The manuals live in [`docs/`](docs/index.md) and are the same files the running
+application serves at `/app/#/docs`:
+
+| | |
+|---|---|
+| [Overview](docs/overview.md) | what it does and the rules it will not break |
+| [Installation and operations](docs/installation.md) | host and container install, every setting, backup and restore, every refusal |
+| [A guided first pass](docs/first-pass.md) | one library from scan to verified upload |
+| [The stages](docs/index.md) | one illustrated page each, from inventory to archive |
+| [Errors and refusals](docs/errors.md) | every error code, cause, and remedy |
+| [Architecture](docs/architecture.md) | diagrams, module map, journals, invariants |
+
+This README stays the repository's own notes — how the code is built, tested, and
+released. Anything an operator or a user needs belongs in `docs/`, and one gate
+(`python -m photo_pipeline docs-gate`) keeps those pages honest against the code.
+
 ## Application (`photo_pipeline`)
 
 The target application lives in `photo_pipeline/` (FastAPI + SQLAlchemy + Alembic).

docs/architecture.md

@@ -0,0 +1,224 @@
+# Architecture
+
+[← Documentation index](index.md)
+
+For whoever has to change this code without breaking somebody's photo library. It
+explains what the pieces are, which rules each one keeps, and where to look when a
+stage refuses.
+
+The product decisions behind all of it live in `INTEGRATED_PIPELINE_CONCEPT.md`; this
+page describes what was built.
+
+## Context
+
+Five things outside the application, and what actually crosses each boundary.
+
+```mermaid
+flowchart LR
+  operator([Operator]):::person -->|browser, one session| app
+  app[Photo Pipeline]:::system -->|read, rename, EXIF write| library[(Photo library<br/>bind mount)]
+  app -->|database, cache, journals, backups| data[(Data directory<br/>local filesystem)]
+  app -->|confirmed-SFW images only| vision[Vision provider]:::ext
+  app -->|immich-go, verified bytes| immich[Immich server]:::ext
+  app -->|copy, verify, then remove| archive[(Archive medium)]
+  classDef person fill:#1f6feb,stroke:#58a6ff,color:#fff
+  classDef system fill:#238636,stroke:#3fb950,color:#fff
+  classDef ext fill:#6e40c9,stroke:#a371f7,color:#fff
+```
+
+| boundary | leaves the machine? | carries |
+|---|---|---|
+| operator → app | no (loopback, or a proxy you configured) | commands against stable ids, never paths |
+| app → library | no | reads, folder renames, EXIF merges |
+| app → data directory | no | SQLite in WAL mode, thumbnails, journals, backups |
+| app → vision provider | **yes** | image bytes of confirmed-SFW canonical assets only |
+| app → Immich | **yes** | the exact verified bytes of an approved album |
+| app → archive medium | no | copies, verified before the source is removed |
+
+## Runtime
+
+Two long-lived processes and a one-shot migration, sharing one database and one
+library.
+
+```mermaid
+flowchart TD
+  browser([Browser]) -->|JSON + SSE| api
+  migrate["migrate<br/>backup, then upgrade"] -->|must exit 0| api
+  migrate --> worker
+  api["api · serve<br/>enqueues, serves, reads"] -->|jobs table| db[(SQLite WAL)]
+  worker["worker<br/>claims and does the work"] -->|jobs table| db
+  worker --> tools["exiftool · vision API · immich-go"]
+  api -.->|api.lock.json| lock{{library lock}}
+  worker -.->|worker.lock.json| lock
+  api --> lib[(library)]
+  worker --> lib
+```
+
+`serve` enqueues and renders; **it does not do the work**. Everything that scans,
+scores, analyses, renames, uploads, or archives happens in the worker, which claims
+a queued job atomically with a fencing token. One mutating job runs at a time, and
+the lock file in the data directory is what makes a second worker impossible rather
+than merely discouraged.
+
+## The modules
+
+| package | owns | must not |
+|---|---|---|
+| `api/` | HTTP surface, error envelope, security policy | contain SQL or business logic |
+| `api/routes/` | one module per resource group, all under `/api/v1` | accept a filesystem path from the browser |
+| `api/security.py` | host/origin/CSRF/session/size policy as one pure `evaluate`, plus the ASGI middleware | be bypassed per-route |
+| `schemas/` | Pydantic request and response contracts | reach the database |
+| `services/` | all domain logic; the only place a decision is made | be imported by the frozen CLI archive |
+| `models/` | SQLAlchemy tables | hold behaviour |
+| `jobs/` | durable job lifecycle, worker loop, lock ranks, handler registry | run work in the API process |
+| `integrations/` | the real outside world: `exiftool`, vision, NSFW model, `immich-go` and its report grammars | be called without a version recorded |
+| `path_policy.py` | the library boundary, `_IGNORE/` exclusion, symlink-escape refusal | be duplicated anywhere |
+| `imaging.py` | the single bounded-decode door | let a caller open an image directly |
+| `faults.py` | the crash barriers the tests fire | read anything but its one env var |
+| `config.py` | typed settings, secrets as `SecretStr` | log or return a value |
+| `db.py` | engine, sessions, WAL and foreign keys, migration entry | be used to bypass a repository |
+
+Services worth knowing by name: `inventory`, `duplicates`, `safety`, `analysis`,
+`albums`/`proposals`/`naming`, `renames`/`rename_apply`/`rename_journal`,
+`uploads`/`upload_batches`/`upload_reports`/`upload_verification`,
+`archives`/`archive_transfer`/`archive_journal`/`restores`/`availability`,
+`thumbnails`, `hashing`, `exif_checkpoint`, `jobs`, `workflow`, `backup`,
+`diagnostics`, `app_lock`, `release`, `benchmarks`, `library`, `legacy_import`.
+
+## The state machines
+
+Three journals decide what a restart is allowed to assume. All three are read from
+the database plus the real world — never guessed from a missing file.
+
+### Durable jobs
+
+```mermaid
+stateDiagram-v2
+  [*] --> queued
+  queued --> running: claimed with a fencing token
+  queued --> cancelled
+  queued --> cancelling
+  running --> succeeded
+  running --> failed
+  running --> cancelling
+  cancelling --> cancelled
+  failed --> retry_queued
+  retry_queued --> running
+  succeeded --> [*]
+  cancelled --> [*]
+```
+
+`succeeded` and `cancelled` are terminal, so a duplicate delivery cannot move a
+finished job. Claiming is compare-and-set on the row version; a worker whose lease
+expired cannot commit after another has taken over.
+
+### The rename journal
+
+The only state machine that moves somebody's folders.
+
+```mermaid
+stateDiagram-v2
+  [*] --> planned
+  planned --> moving
+  moving --> moved
+  moving --> planned: proven untouched
+  moving --> rollback_required
+  moved --> database_updated
+  database_updated --> verified
+  verified --> complete
+  moved --> rollback_required
+  database_updated --> rollback_required
+  verified --> rollback_required
+  planned --> failed
+  failed --> planned
+  rollback_required --> rolled_back
+  complete --> [*]
+  rolled_back --> [*]
+```
+
+Intent is written **before** the disk is touched, which is why `moving` can resolve
+backwards: the evidence decides. `moving`, `moved`, `database_updated`, and
+`rollback_required` are the unsafe states — while any operation sits in one, the
+library may be half-renamed, so unrelated mutations are refused with
+`409 rename_recovery_required` until a person resolves it.
+
+### Upload batches
+
+```mermaid
+stateDiagram-v2
+  [*] --> planned
+  planned --> running
+  running --> succeeded
+  running --> failed
+  running --> cancelling
+  running --> unknown_requires_verification: process died after acceptance
+  cancelling --> cancelled
+  failed --> running: retry
+  cancelled --> running: retry
+```
+
+`unknown_requires_verification` is deliberately **not** restartable. The server may
+already hold the files; the answer is to ask Immich for the recorded SHA-1, not to
+upload again and hope.
+
+Archive transfers use the same shape — `planned → transferring → verified →
+removing → complete` — and the source is removed only after the archived bytes are
+verified.
+
+## Identity and the data model
+
+A path is metadata. The identity is `assets.id`, a UUID that never changes, and
+`asset_paths` records every path an asset has ever had with the reason it changed.
+That is what makes a rename cheap: nothing else in the database has to move.
+
+Tables: `assets`, `asset_paths`, `thumbnails`, `jobs`, `job_items`, `job_events`,
+`safety_reviews`, `analysis_results`, `exif_projections`, `duplicate_clusters`,
+`duplicate_members`, `duplicate_negative_links`, `album_proposals`, `rename_plans`,
+`rename_operations`, `upload_batches`, `upload_items`, `upload_verifications`,
+`archive_locations`, `archive_plans`, `archive_operations`.
+
+Availability is independent of workflow progress: `active`, `archiving`,
+`archived_online`, `archived_offline`, `restoring`, `missing_unexpected`. An
+unmounted archive disk is `archived_offline`, never `missing` — the scanner is not
+allowed to conclude that a photo is gone because a disk is unplugged.
+
+EXIF is a projection with its own verified state (`verified`, `divergent`,
+`failed`): the desired values are written, read back, and compared, and anything the
+stage does not own having changed makes the asset `divergent` and blocks the next
+mutating stage.
+
+## Where each invariant lives
+
+| invariant | enforced in |
+|---|---|
+| `_IGNORE/` is never traversed, counted, or opened | `path_policy.is_excluded`, used by every discovery path |
+| no path outside the library roots is reachable | `path_policy.resolve_in_roots` — it returns the resolved path, because validating one name and opening another is the symlink race |
+| one writer per library | `services/app_lock.py` (an `flock` on a JSON lock file, per role) |
+| one mutating job at a time | `services/jobs.py` lock keys plus `jobs/locks.py` rank ordering |
+| only confirmed-SFW assets reach the vision provider | `services/analysis.py`, re-checked *after* the provider call so a decision that flipped mid-flight discards the result |
+| EXIF is verified, and other fields preserved | `services/exif_checkpoint.py` |
+| uploads carry the exact verified bytes | `services/uploads.py` preflight, re-proved immediately before the uploader runs |
+| an uncertain upload is not a failure | `services/upload_batches.py`, `services/upload_verification.py` |
+| the archive source outlives its copy until verified | `services/archive_transfer.py` |
+| decoding is bounded | `imaging.py` |
+| the trust boundary and CSRF | `api/security.py` |
+
+## Concurrency
+
+Locks are taken broad to narrow — `library → stage/job → album/folder → asset` — and
+never the other way, which is what makes deadlock structural rather than lucky.
+Ranks live in `jobs/locks.py`.
+
+Exactly-once execution is not achievable across SQLite, a filesystem, subprocesses,
+and a remote server. The design promises **at-least-once with idempotent recovery**:
+every handler may run twice, and running twice must not produce two side effects.
+That is why the rename journal records intent before moving, why EXIF writes merge
+and verify, and why upload retries consult both local history and Immich.
+
+## History
+
+This application was extracted from two command-line tools rather than written from
+nothing. Their sources are frozen in `legacy_cli_archive/` with the ledger mapping
+each donated behaviour to the service that now owns it, the characterization tests
+that pinned it, and every intentional difference. Production code must not import
+them; they are provenance and rollback evidence.

docs/errors.md

@@ -0,0 +1,95 @@
+# Errors and refusals
+
+[← Documentation index](index.md)
+
+Every error the API returns carries a code:
+
+```json
+{"error": {"code": "lock_held", "message": "a library_write job is already running"}}
+```
+
+Most of them are **refusals, not faults**. This application would rather stop and
+explain than guess about somebody's photographs, so a code below usually means it
+protected something. Each row says what caused it and what to do.
+
+## Access and the trust boundary
+
+| code | cause | what to do |
+|---|---|---|
+| `unauthenticated` | no application session | reload the page; the browser bootstraps one |
+| `access_denied` | wrong or missing access secret | check `PHOTO_PIPELINE_ACCESS_SECRET`. Attempts are logged with the caller's address only |
+| `too_many_attempts` | more than five failed secret attempts in a minute | wait. This is the rate limit, not a lockout |
+| `csrf_failed` | a mutation without the session's token | reload; a stale tab has an old token |
+| `host_not_allowed` | the `Host` header is not a configured name | add the real hostname to `PHOTO_PIPELINE_ALLOWED_HOSTS` |
+| `origin_not_allowed` | the request came from another origin | not something a browser tab of this app produces |
+| `cross_site_blocked` | another site triggered the request | expected — this is the protection working |
+| `payload_too_large` | the body exceeds `PHOTO_PIPELINE_MAX_REQUEST_BYTES` | send less; every endpoint takes small commands |
+| `path_not_allowed` | a path resolved outside the library roots | usually a symlink. Nothing outside the roots is reachable, by design |
+
+## The safety gate
+
+| code | cause | what to do |
+|---|---|---|
+| `dry_run_not_approved` | mutation is gated until a dry run is approved | run `dry-run`, read it, then `approve-dry-run` |
+| `approval_scope_mismatch` | the approval covers different library roots | approve a report for the roots actually configured |
+| `approval_unreadable` | the approval record cannot be read | re-approve; do not edit it by hand |
+
+## Concurrency and staleness
+
+| code | cause | what to do |
+|---|---|---|
+| `lock_held` | a mutating job already holds the lane | wait for it. One at a time is what makes a crash recoverable |
+| `version_conflict` | you acted on a version that changed underneath you | the view re-renders with the server's truth; decide again |
+| `invalid_transition` | a state change that the machine does not allow | usually a stale tab; reload |
+| `job_error` | the job service refused the request | the message says why |
+| `rename_recovery_required` | an interrupted rename is unresolved | resolve it in [Renames](stages/renames.md). Unrelated mutations stay blocked on purpose |
+
+## Stage refusals
+
+| code | cause | what to do |
+|---|---|---|
+| `nothing_to_score` | no photo is eligible for safety scoring | the queue is already decided |
+| `nothing_eligible` | no photo is eligible for analysis | resolve safety decisions first |
+| `nothing_to_plan` | no approved album name to build a plan from | approve a proposal first |
+| `invalid_decision` | the decision is not one this cluster accepts | reload the cluster |
+| `invalid_proposal` | the name is empty, reserved, or has forbidden characters | fix the name; the rules are in [Album proposals](stages/albums.md) |
+| `unknown_album` | the album is not a folder the library knows | rescan |
+| `cannot_apply` | the plan is invalid, stale, or another plan is unresolved | read the blockers in the preview |
+| `cannot_rollback` | the operation is complete or its preconditions no longer hold | rollback is recovery, not undo |
+
+## Upload
+
+| code | cause | what to do |
+|---|---|---|
+| `stale_preflight` | bytes changed between approval and running | re-run preflight; this is the check that stops the wrong bytes being uploaded |
+| `not_runnable` | the batch is in a state that must not be re-run | an uncertain batch is verified, never retried |
+| `requires_verification` | the outcome is unknown; the server may hold the files | verify against Immich, then resolve |
+| `changed_after_upload` | the file changed after a successful upload | uploading again may create or upgrade an asset |
+
+## Operations
+
+| code | cause | what to do |
+|---|---|---|
+| `backup_failed` | the snapshot could not be taken or verified | check free space and the message |
+| `invalid_retention` | a `--keep` value that is not a positive count | the newest backup is never pruned |
+| `not_found` | no such id | usually a stale link |
+
+## Generic
+
+| code | cause |
+|---|---|
+| `invalid_request` | the request body failed validation. Field names only — never the values, which end up in logs and screenshots |
+| `http_error` | a plain HTTP-level refusal, with its status |
+| `internal_error` | an unhandled error. The code is all you get; the traceback is in the server log, because it can carry paths and credentials |
+
+## Diagnostics warnings
+
+Not errors — reported by `diagnostics` and worth acting on. They are listed with
+their meanings in [Diagnostics](stages/diagnostics.md): `disk_low`, `disk_critical`,
+`cache_over_quota`, `wal_growth`, `tool_version_drift`, `legacy_process_active`, and
+`no_roots`.
+
+## Startup refusals
+
+Exit codes 2 to 5 are covered by the installation manual under
+[when it refuses to start](installation.md#when-it-refuses-to-start).

docs/first-pass.md

@@ -0,0 +1,66 @@
+# A guided first pass
+
+[← Documentation index](index.md)
+
+One library, from nothing to a verified upload. Each stage links to its own page; this
+is the order, and — more importantly — where the points of no return are.
+
+![The workflow home, with every stage and its counts](images/workflow.png)
+
+The workflow page is the home view and the honest summary: each stage shows its state,
+its counts, why it is blocked if it is, and the one action that moves it forward.
+
+| # | stage | reversible afterwards? |
+|---|---|---|
+| 0 | [Inventory](stages/inventory.md) | nothing was changed |
+| 1 | [Duplicate review](stages/duplicates.md) | yes — decisions can be changed, no file is deleted |
+| 2 | [Safety review](stages/safety.md) | the decision, yes; the EXIF keyword is a file write |
+| 3 | [Analysis](stages/analysis.md) | the result, yes; the caption and keywords are a file write |
+| 4 | [Album proposals](stages/albums.md) | yes — approving renames nothing |
+| 5 | [Renames](stages/renames.md) | **your folders move.** Recoverable, journaled, but real |
+| 6 | [Upload](stages/uploads.md) | **assets exist on the Immich server.** Not undone from here |
+| 7 | [Archive](stages/archive.md) | **originals leave active storage.** Restorable while the medium is reachable |
+| — | [Diagnostics](stages/diagnostics.md) | read-only, any time |
+
+## Before the first run
+
+Take a backup of the photo library itself. This application is careful — it previews,
+journals, and verifies — but it is the first time you are pointing it at your
+photographs, and a backup is cheaper than confidence.
+
+If the library is irreplaceable, turn on `PHOTO_PIPELINE_REQUIRE_DRY_RUN_APPROVAL`
+before anything else. Every mutating request is then refused until you have produced
+a read-only reconciliation report and approved it by name.
+
+## The pass
+
+1. **Scan.** Inventory → *Rescan*. Reads only. Check the count, and check that
+   nothing under `_IGNORE/` appears.
+2. **Resolve duplicates.** Exact byte matches can be accepted as recommended; anything
+   fuzzy gets looked at. Resolving first is what stops you paying to analyse the same
+   photo twice.
+3. **Decide safety.** Every canonical photo becomes `sfw` or `nsfw`. Nothing reaches
+   the vision provider until it is confirmed SFW — this is the gate the whole design
+   exists around.
+4. **Analyse.** Only confirmed-SFW photos. Each result is written to the database and
+   projected into EXIF, then read back and verified.
+5. **Propose album names.** Evidence from what was analysed. Edit anything you disagree
+   with. Approving changes no file.
+6. **Build the rename plan, read it, apply it.** The preview shows every old → new
+   path. This is where folders move.
+7. **Rescan.** Paths are reconciled; identities are unchanged.
+8. **Upload.** Preflight lists every blocker. The command is previewed without the API
+   key. One album at a time.
+9. **Verify.** An uncertain upload is not a failure and not a success — ask the server.
+10. **Archive** (optional). Copy, verify, and only then reclaim the space.
+
+Stop anywhere. Every stage is resumable, and closing the browser cancels nothing.
+
+## Two rules worth internalising
+
+**The application refuses more than it warns.** A refusal is not a fault; it is the
+design working. [Errors and refusals](errors.md) explains each one.
+
+**One long job at a time.** Browsing stays available while a job runs, but a second
+mutating job is refused — that is what makes a crash recoverable rather than
+ambiguous.

docs/index.md

@@ -15,17 +15,51 @@ repository under `docs/`, on Gitea, and inside the running application under
 2. [Installation and operations](installation.md) — host and container installation,
    every setting, the first-run checklist, upgrades, backup and restore, and what
    each refusal at startup means.
+3. [A guided first pass](first-pass.md) — one library from scan to verified upload,
+   with the point of no return named in each stage.
+4. [Architecture](architecture.md) — the context and runtime diagrams, what each
+   module owns, the three journals a restart reads, and where every invariant is
+   enforced.
 
-## Being written
+## The stages, one page each
 
-The remaining manuals are accepted work, not aspiration; each is a story in
-[E09](https://git.domverse-berlin.eu/domverse/photoanalyzer/src/branch/main/delivery_backlog/E09-documentation.md)
-and will appear here as it lands.
+Every page answers the same four questions: what the stage is for, what you decide,
+what it changes on disk or on the server, and what it refuses.
 
-- **Architecture** — context and runtime diagrams, the module map, the job and
-  journal state machines, and where each invariant is enforced (US09-03).
-- **User manual** — one page per workflow stage with screenshots of the real
-  application, and a catalogue of every error and refusal (US09-04).
+1. [Inventory and discovery](stages/inventory.md)
+2. [Duplicate review](stages/duplicates.md)
+3. [Safety review](stages/safety.md)
+4. [Analysis](stages/analysis.md)
+5. [Album proposals](stages/albums.md)
+6. [Renames](stages/renames.md)
+7. [Upload](stages/uploads.md)
+8. [Archive](stages/archive.md)
+9. [Diagnostics and library statistics](stages/diagnostics.md)
+
+## When something goes wrong
+
+- [Errors and refusals](errors.md) — every error code, what caused it, what to do.
+- [Recovery](recovery.md) — what the application resolves by itself, and what needs
+  you.
+
+## Keeping these pages true
+
+Documentation that disagrees with the application is worse than none, because it is
+trusted. So one command compares them:
+
+```bash
+python -m photo_pipeline docs-gate
+```
+
+It runs every `phase_i` check — dead links and anchors, pages nobody links to, images
+nobody shows, settings, commands, exit codes, error codes, and states that no longer
+exist in the code, the pages rendering in a real browser without a console or policy
+error, and the screenshots still showing what the application shows. It retains its
+evidence and **accepts no skipped check**: a check that did not run is a page nobody
+compared. CI runs it on every pull request.
+
+The repository's own build, test, and release notes live in the
+[README](../README.md).
 
 ## Conventions
 

docs/overview.md

@@ -72,6 +72,6 @@ python -m photo_pipeline serve      # the API and this browser application
 python -m photo_pipeline worker     # the process that does the long work
 ```
 
-The full procedure, the container path, and every setting belong to the installation
-manual, which is being written next — see
-[the documentation index](index.md#being-written).
+The full procedure, the container path, and every setting are in the
+[installation manual](installation.md); if you would rather start using it, take
+[the guided first pass](first-pass.md).

docs/recovery.md

@@ -0,0 +1,62 @@
+# Recovery
+
+[← Documentation index](index.md)
+
+What the application resolves by itself, and what needs you. The dividing line is
+simple: it resumes when the evidence is unambiguous, and it stops and asks when it is
+not. It never guesses about your files.
+
+## An interrupted rename
+
+A rename records its intent **before** touching the disk, so after a crash the
+journal plus the actual filesystem give a verdict per operation:
+
+| verdict | means | what happens |
+|---|---|---|
+| resumable | the move provably did not happen | reset to planned and run again |
+| rollback-safe | the move happened; the remaining steps can be completed or undone | drive it forward, or roll it back |
+| manual | the evidence is contradictory | left alone, and it keeps blocking |
+
+Open [Renames](stages/renames.md); the recovery panel lists every unresolved
+operation and offers a resolve action only where one is safe. Until then, unrelated
+mutations are refused with `rename_recovery_required` — building new work on a
+half-applied move is how a library becomes unexplainable.
+
+## An uncertain upload
+
+`unknown_requires_verification` means the uploader died after Immich may have
+accepted the files. It is **not** retryable. Verification asks the server whether it
+holds the recorded SHA-1 and answers present, absent, or inconclusive. An
+inconclusive answer is resolved by you, with your evidence and name recorded in the
+batch's history.
+
+## A failed migration
+
+A pending schema change is snapshotted before it is applied. If the upgrade fails,
+the previous database and its `pre-migration` backup are both intact and the error
+log names the backup directory. Stop everything and run the
+[restore drill](installation.md#restoring).
+
+## A restored backup
+
+After restoring, run a scan. Paths are reconciled against the real library and
+identities are preserved where hashes match. Anything that does not match becomes a
+visible `stale` or `divergent` state instead of being quietly accepted.
+
+Mount every archive medium the manifest names before archiving again: the database
+records where archived originals live, it does not contain them.
+
+## A job that will not start
+
+Check, in this order: is a worker running; is another mutating job holding the lane
+(`lock_held`); is an interrupted rename blocking everything
+(`rename_recovery_required`); is the library lock held by a process that is still
+alive (exit code `2` names the holder).
+
+## What is never automatic
+
+No file is deleted, no folder is renamed, no upload is retried, and no archive source
+is removed without either an explicit confirmation from you or a verification the
+application performed itself. If you are reading this page because something happened
+that you did not approve, that is a bug worth reporting — with the diagnostics output
+and the relevant journal, both of which are safe to share: they carry no secrets.

docs/stages/albums.md

@@ -0,0 +1,47 @@
+# Album proposals
+
+[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
+
+![Reviewing an album proposal](../images/albums.png)
+
+**What it is for.** Turning a folder of analysed photographs into a name a person
+would have chosen, before any folder is touched.
+
+**What you decide.** The final name. The proposal is a starting point with its
+reasoning attached; you edit it, accept it, or ignore it.
+
+**What it changes.** A row per album: proposed name, rationale, confidence, and your
+final name, with a version. **Approving renames nothing.** Not one file moves at this
+stage — approval only marks a name as agreed, and the [rename stage](renames.md) is
+where it becomes a plan you have to confirm separately.
+
+**What it refuses.** A name containing `/ \ : * ? " < > |`, a reserved device name, or
+one that collides with an existing folder is rejected while you type and again on the
+server. Approval is refused when the evidence changed since the proposal was
+generated — the proposal is stale, and regenerating is the honest fix. Editing with a
+stale version returns a conflict and shows you the server's truth rather than
+overwriting it.
+
+## Reading the view
+
+The left pane lists albums with their state: `none`, `proposed`, `edited`,
+`approved`, `error`, or `stale`. The right pane shows the evidence the name was built
+from — date range, dominant tags, locations, counts — then the suggested name, the
+rationale, the confidence, and an editable final name.
+
+The default naming shape is predictable and sortable:
+
+```text
+YYYY-MM — Place — Event
+YYYY — Event
+Place — Event
+```
+
+## While a rename is unresolved
+
+Generating and approving proposals is blocked while an interrupted rename is
+outstanding, with `409 rename_recovery_required`. Resolve the rename first: building
+new names on top of a half-applied move is how a library becomes hard to reason
+about.
+
+Next: [renames](renames.md).

docs/stages/analysis.md

@@ -0,0 +1,39 @@
+# Analysis
+
+[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
+
+![The analysis view](../images/analysis.png)
+
+**What it is for.** Describing what a photograph shows, so albums can be named from
+evidence and the library can be searched by content.
+
+**What you decide.** When to run it, and over what scope. The descriptions themselves
+come from the configured vision provider.
+
+**What it changes.** For each eligible photo: a stored result — description, tags,
+approximate year, location hint, model and prompt version — and then an EXIF
+checkpoint that merges a **managed caption segment** and additive keywords into the
+file, reads them back, and verifies that nothing else moved. The file's SHA-256 is
+refreshed after the write.
+
+**What it refuses.** Anything not confirmed SFW. Anything not canonical. A malformed
+provider response is quarantined rather than stored. Keywords are only ever added,
+never removed, because EXIF keywords carry no ownership and deleting a generated one
+could delete yours.
+
+## Running it
+
+The view shows eligible, analysed, pending, and error counts, and the job's live
+progress. Starting a second mutating job while it runs is refused — that is the
+one-writer rule, not a queue.
+
+Cancelling drains safely: completed items stay completed, and resuming continues
+rather than restarting. A photo may be sent to the provider twice if a crash happens
+mid-item, but its stored result and its EXIF are written once.
+
+## Costs
+
+Each analysed photo is a provider call. Resolving duplicates first is what keeps that
+number honest, and the counts here are the ones to check before starting a large run.
+
+Next: [album proposals](albums.md).

docs/stages/archive.md

@@ -0,0 +1,43 @@
+# Archive
+
+[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
+
+![Archive locations and plans](../images/archive.png)
+
+**What it is for.** Moving a finished album out of active storage onto another disk,
+while keeping everything the library knows about it.
+
+**What you decide.** Which album, which destination, and whether to accept the
+reclaimed space in exchange for needing that medium mounted to see the originals
+again.
+
+**What it changes.** Files are copied to the archive location, verified there, and
+only then removed from the library. The asset keeps its id, its hashes, its
+decisions, its analysis, its upload history, and a durable preview thumbnail;
+`current_path` becomes null and availability becomes `archived_online` or
+`archived_offline`.
+
+**What it refuses.** Preflight blocks on: an unverified upload, a checksum that no
+longer matches the uploaded bytes, an unmounted or unwritable destination,
+insufficient free space plus the configured reserve, a destination that already
+holds unexpected paths, a missing durable thumbnail, and any conflicting job. The
+source is **never** removed before the archived copy is verified — not because Immich
+reported success, which is an ingest, not a backup.
+
+## Offline is not missing
+
+An archived photo whose medium is unplugged is `archived_offline`. It still appears
+in search, still participates in duplicate detection through its retained hashes and
+thumbnail, and is never reported as lost. If a full-resolution comparison is needed,
+the application asks you to mount the named medium rather than guessing.
+
+## Restore
+
+Restore is planned and journaled like everything else: the medium must be online, the
+bytes are copied back and verified into a collision-free destination, a new active
+path is registered, and a reconciliation scan follows. Existing safety, analysis,
+EXIF, and upload state stay valid when hashes match — and when they do not, the
+result is a visible `divergent` state rather than silent acceptance of a different
+file.
+
+Next: [diagnostics](diagnostics.md).

docs/stages/diagnostics.md

@@ -0,0 +1,49 @@
+# Diagnostics and library statistics
+
+[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
+
+![Library statistics](../images/statistics.png)
+
+**What it is for.** Answering "is this installation healthy, and what is in this
+library?" — the first thing to look at when a stage behaves unexpectedly.
+
+**What you decide.** Nothing. Both are read-only.
+
+**What it changes.** Nothing.
+
+**What it refuses.** Nothing — both are reads, and both stay available while a job
+holds the library. That is deliberate: the moment you most need to know what the
+installation is doing is while it is busy.
+
+## Statistics
+
+The Stats view summarises the library itself: how many photos are analysed, the
+common tags, the distribution across albums and years. It answers questions about
+your photographs.
+
+## Diagnostics
+
+Diagnostics answers questions about the *installation*, and lives at
+`GET /api/v1/diagnostics` and on the command line:
+
+```bash
+python -m photo_pipeline diagnostics
+```
+
+It reports the database, write-ahead log, thumbnail cache, uploader reports, backups,
+and logs as separate sizes, plus free space, the tool versions it found against the
+ones the image pinned, and which locks are held and by whom.
+
+Its warnings are the ones worth acting on:
+
+| warning | means |
+|---|---|
+| `disk_low` / `disk_critical` | free space is running out; mutating stages stop before the reserve |
+| `cache_over_quota` | the thumbnail cache exceeds its configured quota |
+| `wal_growth` | the write-ahead log is outgrowing its database — usually a long-running reader |
+| `tool_version_drift` | the installed `exiftool` or `immich-go` is not the version the image pinned |
+| `legacy_process_active` | the frozen command-line tools are writing this library. Stop them |
+| `no_roots` | no library root is configured, so there is nothing to work on |
+
+An empty `warnings` list on a fresh installation is what the
+[first-run checklist](../installation.md#first-run-checklist) is looking for.

docs/stages/duplicates.md

@@ -0,0 +1,36 @@
+# Duplicate review
+
+[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
+
+![The duplicate cluster list](../images/duplicates.png)
+
+**What it is for.** Deciding which file is *the* copy of a picture, before anything
+expensive or irreversible happens to the others.
+
+**What you decide.** For each cluster: keep the recommended canonical, choose a
+different one, keep everything as variants, declare it not a duplicate, or defer.
+
+**What it changes.** Only the decision, recorded as an auditable event. **No file is
+ever deleted.** Non-canonical copies stay exactly where they are; they are simply
+excluded from analysis and upload.
+
+**What it refuses.** Exact byte and identical-pixel matches may be accepted on the
+recommendation. Anything fuzzy — a crop, a re-encode, a burst frame — requires an
+explicit confirmation, because a perceptual hash is evidence, not proof.
+
+## Reading a cluster
+
+Each member shows its path, role, pHash distance, and size, with synchronised zoom
+across the previews. The recommendation is visible but never pre-applied for fuzzy
+matches. The confidence band tells you how much the machine is claiming.
+
+Rejecting a pair records a negative link, so a later rescan does not keep proposing
+the same wrong match.
+
+## Why first
+
+Resolving duplicates before safety and analysis is what stops you reviewing and
+paying for the same photograph three times. It also means the album evidence is built
+from one copy of each picture rather than a skewed count.
+
+Next: [safety review](safety.md).

docs/stages/inventory.md

@@ -0,0 +1,39 @@
+# Inventory and discovery
+
+[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
+
+![The inventory view](../images/inventory.png)
+
+**What it is for.** Finding every supported photo in the configured library roots and
+giving each one a permanent identity, so that everything afterwards can refer to a
+picture rather than a path.
+
+**What you decide.** Nothing. This stage is the only one with no judgement in it.
+
+**What it changes.** On disk, nothing at all — discovery reads. In the database it
+records each photo's path, size, timestamps, full-file SHA-256, normalised pixel hash,
+and perceptual hash, and it opens a path-history entry.
+
+**What it refuses.** Anything under an `_IGNORE/` directory is never traversed,
+counted, hashed, or previewed. Unsupported extensions and unreadable files are
+reported rather than skipped silently. Nothing outside the configured roots is
+reachable, including through a symlink.
+
+## Reading the view
+
+Each row is one asset: its current path, availability, whether the file is present,
+and its size. The filter searches paths; the availability selector separates active
+photos from archived ones.
+
+A **rescan** after moving files around outside the application is the normal way to
+reconcile: a file that moved keeps its identity, because identity is the hash and the
+id, not the location.
+
+## What good looks like
+
+The count matches what you expect, `_IGNORE/` contents are absent, and nothing shows
+as missing. A photo listed as missing means the file is no longer at its recorded
+path and no new path matched its hashes — that is a question for you, not something
+the scanner resolves by guessing.
+
+Next: [duplicate review](duplicates.md).

docs/stages/renames.md

@@ -0,0 +1,46 @@
+# Renames
+
+[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
+
+![The rename plan preview](../images/renames.png)
+
+**What it is for.** Applying approved album names to the actual folders — the first
+stage that changes your filesystem.
+
+**What you decide.** Whether to apply the plan, after reading it. The confirmation
+names the plan, its version, and its checksum, so what you approve is what runs.
+
+**What it changes.** Folders move. For each operation: the intent is journaled
+*before* the disk is touched, the move is made, `assets.current_path` is updated by
+id in one transaction, the old path is closed and a new one opened in path history,
+the result is verified, and only then is the operation complete.
+
+**What it refuses.** A plan is invalid — not merely warned about — when a source is
+missing or changed, two operations target the same destination, a destination already
+exists, a path would leave the library or enter `_IGNORE/`, or a source is a symlink.
+Case-only renames and cross-filesystem moves are allowed but flagged, because they
+take a different, staged route. A plan whose checksum no longer matches is refused
+rather than reinterpreted.
+
+## Reading the preview
+
+Every operation shows both full paths, the kind of move (`rename`, `case-only
+rename`, `cross-filesystem move`), how many photos it affects, its journal state, and
+any issue with its severity. The plan can be exported as JSON before you commit to it.
+
+**A run cannot be stopped part-way.** It can be interrupted — by a crash, a power
+cut, a killed container — and interruption is recoverable, but there is no cancel
+button once it starts. That is stated on the confirmation, not hidden in a manual.
+
+## After an interruption
+
+The recovery panel lists each unresolved operation with a verdict read from the
+journal *and the disk*: resumable, safely rollbackable, or needing a person. It never
+guesses from a missing path alone. Until it is resolved, unrelated mutations are
+refused with `409 rename_recovery_required`.
+
+Rollback is recovery, not undo. A completed operation is terminal — the button only
+appears for operations that are actually reversible. See
+[recovery](../recovery.md).
+
+Next: [upload](uploads.md).

docs/stages/safety.md

@@ -0,0 +1,43 @@
+# Safety review
+
+[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
+
+![The safety review queue](../images/safety.png)
+
+**What it is for.** Deciding which photographs may be sent to a cloud vision
+provider. This is the privacy gate the rest of the design is built around.
+
+**What you decide.** `sfw`, `nsfw`, or defer — per photo, or in bulk from the filters.
+A local model can score first to order the queue, but the score is a suggestion; the
+decision is yours.
+
+**What it changes.** The decision is stored in the database *and* projected into the
+file: one mutually exclusive `sfw` or `nsfw` keyword written into `Keywords` and
+`Subject`. The write is merged with existing metadata, read back, and verified, and
+the file's SHA-256 is refreshed afterwards.
+
+**What it refuses.** An undecided or deferred photo does not reach the vision
+provider, and does not reach Immich either. If the read-back shows that a field the
+stage does not own has changed, the checkpoint is marked `divergent`, the asset is not
+marked verified, and the next mutating stage is blocked rather than proceeding on
+metadata nobody trusts.
+
+## The rule that matters
+
+> Only a confirmed `sfw` photo may enter cloud content analysis. A confirmed `nsfw`
+> photo skips analysis entirely but remains eligible for upload to Immich once its
+> keyword is verified.
+
+That is why marking something NSFW is not a punishment: it routes the photo around an
+external service while keeping it in your own library workflow.
+
+The gate is re-checked *after* the provider call as well. If a decision flips to NSFW
+while an analysis is in flight, the result is discarded rather than stored.
+
+## Reading the view
+
+The tabs filter by state. Each row shows the path, the model's score, its suggestion,
+your decision, and an `✓ exif` badge once the keyword is verified on disk. A row
+without that badge has a decision the file does not yet carry.
+
+Next: [analysis](analysis.md).

docs/stages/uploads.md

@@ -0,0 +1,45 @@
+# Upload
+
+[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
+
+![Upload preflight](../images/uploads.png)
+
+**What it is for.** Sending an approved album to Immich through `immich-go`, with an
+audit trail of exactly which bytes went.
+
+**What you decide.** Which albums, and whether to proceed once the preflight has
+listed its blockers. Upload is never an automatic consequence of renaming.
+
+**What it changes.** Nothing locally. On the Immich server, assets appear. Locally a
+batch is recorded: the album, the asset ids, the redacted command, the uploader
+version, the pre-upload SHA-256 **and** the SHA-1 Immich matches on, the output log,
+and the outcome counts.
+
+**What it refuses.** Preflight blocks on: missing credentials, an unreachable server,
+a missing `immich-go`, an unresolved rename, an undecided or deferred safety
+decision, an unverified EXIF checkpoint, incomplete analysis for an SFW photo, a file
+whose bytes changed since it was checkpointed, and a partial scope you have not
+explicitly accepted. The approval is then **re-proved immediately before the uploader
+runs** — bytes edited after you approved fail with `stale_preflight` and the uploader
+never starts.
+
+## Reading the view
+
+The scope shows exactly which albums and how many photos. The command preview is the
+real command with the API key removed; the key never reaches the browser and never
+reaches a log.
+
+## Uncertain is not failed
+
+If the process dies after the server accepted files, the batch becomes
+`unknown_requires_verification` — not "failed", and **not retryable**. The server may
+already hold the photographs, and uploading again would create duplicates or
+upgrades. Verification asks Immich about the recorded SHA-1 and answers present,
+absent, or inconclusive; an inconclusive answer is resolved by you, with the evidence
+recorded.
+
+If a file's bytes change after a successful upload, it is marked
+`changed_after_upload` and you are warned that uploading again may create or upgrade
+an asset rather than doing nothing.
+
+Next: [archive](archive.md).

frontend/js/app.js

@@ -1,6 +1,7 @@
 import { api } from "./api.js";
 import { renderArchive, setArchiveRender } from "./archive.js";
 import { renderDocs } from "./docs.js";
+import { show as showIn } from "./dom.js";
 import { navigate, onRouteChange, parseHash } from "./router.js";
 import { renderRenames, setRenamesRender } from "./renames.js";
 import { renderUploads, setUploadsRender } from "./uploads.js";
@@ -35,7 +36,7 @@ function el(tag, attrs = {}, ...children) {
 }
 
 function show(...nodes) {
-  root.replaceChildren(...nodes);
+  showIn(root, ...nodes);
 }
 
 function setActiveNav(view) {

frontend/js/archive.js

@@ -9,7 +9,7 @@
 // is not mounted produces an instruction naming it rather than a disabled mystery,
 // and an operation whose evidence is ambiguous offers no button at all.
 import { api } from "./api.js";
-import { el, errorBanner, setActiveNav } from "./dom.js";
+import { el, errorBanner, setActiveNav, show } from "./dom.js";
 import { subscribeJob } from "./events.js";
 import { navigate } from "./router.js";
 
@@ -46,7 +46,7 @@ export async function renderArchive(root, params = {}) {
   try {
     locations = (await api.archiveLocations()).locations;
   } catch (error) {
-    root.replaceChildren(errorBanner(`Failed to load archive locations: ${error.message}`));
+    show(root, errorBanner(`Failed to load archive locations: ${error.message}`));
     return;
   }
   const location = locations.find((l) => l.id === params.location) || locations[0] || null;
@@ -61,7 +61,7 @@ export async function renderArchive(root, params = {}) {
       ),
       outcomeBanner()
     );
-    root.replaceChildren(...nodes.filter(Boolean));
+    show(root, ...nodes.filter(Boolean));
     return;
   }
 
@@ -95,7 +95,7 @@ export async function renderArchive(root, params = {}) {
     archived ? archivedSection(archived.items, locations) : null,
     restore ? restoreSection(restore, location) : null
   );
-  root.replaceChildren(...nodes.filter(Boolean));
+  show(root, ...nodes.filter(Boolean));
 }
 
 function plans(listed) {

frontend/js/dom.js

@@ -21,6 +21,19 @@ export function errorBanner(message) {
   return el("div", { class: "alert", role: "alert" }, message);
 }
 
+/**
+ * Replace a container's contents, dropping the absent ones.
+ *
+ * `el` already ignores null children, but `replaceChildren` does not: it stringifies
+ * them, so an optional node that is not there renders the word "null" on the page.
+ * Every view builds optional nodes — a banner only while a job runs, a conflict only
+ * after a 409 — so the filter belongs here rather than in each caller. It was missing
+ * from one of them, and a documentation screenshot is where that turned up (US09-04).
+ */
+export function show(root, ...nodes) {
+  root.replaceChildren(...nodes.flat().filter((node) => node != null && node !== false));
+}
+
 export function setActiveNav(view) {
   document.querySelectorAll("nav a[data-nav]").forEach((a) => {
     if (a.dataset.nav === view) a.setAttribute("aria-current", "page");

frontend/js/renames.js

@@ -5,7 +5,7 @@
 // recovery classifications all come from the server; the view only shows them and
 // refuses to offer an action the server would reject.
 import { api } from "./api.js";
-import { el, errorBanner, setActiveNav } from "./dom.js";
+import { el, errorBanner, setActiveNav, show } from "./dom.js";
 
 // What the last confirm did, for this tab only. Deliberately not persisted: after a
 // reload the page must show what the journal says, not what this page remembers.
@@ -24,7 +24,7 @@ export async function renderRenames(root, params = {}) {
   try {
     [plans, recovery] = await Promise.all([api.listPlans(), api.renameRecovery()]);
   } catch (error) {
-    root.replaceChildren(errorBanner(`Failed to load renames: ${error.message}`));
+    show(root, errorBanner(`Failed to load renames: ${error.message}`));
     return;
   }
 
@@ -34,13 +34,13 @@ export async function renderRenames(root, params = {}) {
     try {
       plan = await api.getPlan(selectedId);
     } catch (error) {
-      root.replaceChildren(errorBanner(`Failed to load plan: ${error.message}`));
+      show(root, errorBanner(`Failed to load plan: ${error.message}`));
       return;
     }
   }
 
   const blocked = recovery.blocks_mutation;
-  root.replaceChildren(
+  show(root,
     el("h1", {}, "Renames"),
     recoveryPanel(recovery),
     el(

frontend/js/uploads.js

@@ -8,7 +8,7 @@
 // preview arrives redacted — so nothing in this view may reconstruct, store, or
 // route a secret.
 import { api } from "./api.js";
-import { el, errorBanner, setActiveNav } from "./dom.js";
+import { el, errorBanner, setActiveNav, show } from "./dom.js";
 import { subscribeJob } from "./events.js";
 import { navigate } from "./router.js";
 
@@ -45,7 +45,7 @@ export async function renderUploads(root, params = {}) {
       api.listUploadBatches(),
     ]);
   } catch (error) {
-    root.replaceChildren(errorBanner(`Failed to load uploads: ${error.message}`));
+    show(root, errorBanner(`Failed to load uploads: ${error.message}`));
     return;
   }
 
@@ -62,12 +62,12 @@ export async function renderUploads(root, params = {}) {
       batch = detail;
       history = verifications.verifications;
     } catch (error) {
-      root.replaceChildren(errorBanner(`Failed to load upload batch: ${error.message}`));
+      show(root, errorBanner(`Failed to load upload batch: ${error.message}`));
       return;
     }
   }
 
-  root.replaceChildren(
+  show(root,
     ...[
       el("h1", {}, "Upload"),
       configurationCard(preflight),

frontend/js/views.js

@@ -3,7 +3,7 @@
 // HTML. Mutating actions are disabled while a job holds the library_write lock and
 // say why (concept §shared interaction rules: read-only browsing stays available).
 import { api } from "./api.js";
-import { el, errorBanner, setActiveNav } from "./dom.js";
+import { el, errorBanner, setActiveNav, show } from "./dom.js";
 import { navigate } from "./router.js";
 import { subscribeJob } from "./events.js";
 
@@ -26,7 +26,7 @@ export async function renderWorkflow(root) {
   try {
     data = await api.workflow();
   } catch (error) {
-    root.replaceChildren(errorBanner(`Failed to load workflow: ${error.message}`));
+    show(root, errorBanner(`Failed to load workflow: ${error.message}`));
     return;
   }
   const active = data.active_job;
@@ -67,7 +67,7 @@ export async function renderWorkflow(root) {
       )
     );
   });
-  root.replaceChildren(
+  show(root,
     el("h1", {}, "Workflow"),
     jobBanner(active),
     el("div", { class: "stepper" }, ...cards)
@@ -105,7 +105,7 @@ export async function renderSafety(root, params) {
   try {
     data = await api.safetyQueue({ state });
   } catch (error) {
-    root.replaceChildren(errorBanner(`Failed to load safety queue: ${error.message}`));
+    show(root, errorBanner(`Failed to load safety queue: ${error.message}`));
     return;
   }
 
@@ -149,7 +149,7 @@ export async function renderSafety(root, params) {
     )
   );
 
-  root.replaceChildren(
+  show(root,
     el("h1", {}, "Safety review"),
     tabs,
     el(
@@ -197,7 +197,7 @@ export async function renderLibrary(root, params) {
   try {
     data = await api.libraryAssets({ q, offset, limit, sort: params.sort || "path" });
   } catch (error) {
-    root.replaceChildren(errorBanner(`Failed to load library: ${error.message}`));
+    show(root, errorBanner(`Failed to load library: ${error.message}`));
     return;
   }
 
@@ -220,7 +220,7 @@ export async function renderLibrary(root, params) {
     )
   );
 
-  root.replaceChildren(
+  show(root,
     el("h1", {}, "Library"),
     el("div", { class: "toolbar" }, search, el("span", { class: "muted", "data-testid": "library-total" }, `${data.total} photos`)),
     data.rows.length ? el("div", { class: "cluster-grid" }, ...cards) : el("p", { class: "muted" }, "No matching photos."),
@@ -235,7 +235,7 @@ export async function renderAnalyze(root) {
   try {
     [counts, workflow] = await Promise.all([api.analysisCounts(), api.workflow()]);
   } catch (error) {
-    root.replaceChildren(errorBanner(`Failed to load analysis: ${error.message}`));
+    show(root, errorBanner(`Failed to load analysis: ${error.message}`));
     return;
   }
   const active = workflow.active_job;
@@ -269,7 +269,7 @@ export async function renderAnalyze(root) {
     "Analyze eligible SFW assets"
   );
 
-  root.replaceChildren(
+  show(root,
     el("h1", {}, "Analyze"),
     jobBanner(active),
     el(
@@ -292,10 +292,10 @@ export async function renderStats(root) {
   try {
     data = await api.libraryStats();
   } catch (error) {
-    root.replaceChildren(errorBanner(`Failed to load stats: ${error.message}`));
+    show(root, errorBanner(`Failed to load stats: ${error.message}`));
     return;
   }
-  root.replaceChildren(
+  show(root,
     el("h1", {}, "Stats"),
     el("div", { class: "counts-grid", "data-testid": "stats-status" }, ...Object.entries(data.status).map(([k, v]) => stat(k, v))),
     facetBlock("Top tags", data.top_tags),
@@ -338,7 +338,7 @@ export async function renderAlbums(root, params = {}) {
       api.renameRecovery(),
     ]);
   } catch (error) {
-    root.replaceChildren(errorBanner(`Failed to load albums: ${error.message}`));
+    show(root, errorBanner(`Failed to load albums: ${error.message}`));
     return;
   }
 
@@ -385,7 +385,7 @@ export async function renderAlbums(root, params = {}) {
       )
     : el("p", { class: "muted" }, "No albums with evidence yet.");
 
-  root.replaceChildren(
+  show(root,
     el("h1", {}, "Albums"),
     renameBlocked
       ? el(

photo_pipeline/__main__.py

@@ -1,5 +1,5 @@
 """Application management CLI:
-``python -m photo_pipeline {serve,migrate,worker,import-legacy-scores,backup,verify-backup,restore,diagnostics,benchmark,release-gate,container-gate,dry-run,approve-dry-run}``.
+``python -m photo_pipeline {serve,migrate,worker,import-legacy-scores,backup,verify-backup,restore,diagnostics,benchmark,release-gate,container-gate,docs-gate,dry-run,approve-dry-run}``.
 
 ``serve`` and ``worker`` take the library process lock for their role (US07-05):
 two workers, or the frozen CLI running beside the app, would each be safe on their
@@ -78,6 +78,12 @@ def main(argv: Sequence[str] | None = None) -> int:
     container_cmd.add_argument(
         "--output", help="Evidence directory (default: data/container-gate/<stamp>)"
     )
+    docs_cmd = commands.add_parser(
+        "docs-gate",
+        help="Check the manuals against the application they describe and keep the "
+        "evidence (US09-05)",
+    )
+    docs_cmd.add_argument("--output", help="Evidence directory (default: data/docs-gate/<stamp>)")
     dry_cmd = commands.add_parser(
         "dry-run", help="Read-only reconciliation of the configured library (US07-07)"
     )
@@ -188,6 +194,30 @@ def main(argv: Sequence[str] | None = None) -> int:
         )
         return 0 if report["ok"] else 1
 
+    if args.command == "docs-gate":
+        from datetime import datetime, timezone
+
+        from photo_pipeline.services import release
+
+        # Documentation that disagrees with the application is worse than none: it is
+        # trusted. So this gate accepts no skip either — a check that did not run is a
+        # page nobody compared with the code.
+        output = args.output or Path(config.data_dir) / "docs-gate" / datetime.now(
+            timezone.utc
+        ).strftime("%Y%m%dT%H%M%SZ")
+        report = release.run_gate(
+            config,
+            output=output,
+            stages=release.DOCS_STAGES,
+            allowed_skips=release.DOCS_ALLOWED_SKIP_REASONS,
+        )
+        print(
+            json.dumps(
+                {k: v for k, v in report.items() if k not in ("stages", "matrix")}, indent=2
+            )
+        )
+        return 0 if report["ok"] else 1
+
     if args.command == "dry-run":
         from photo_pipeline.services import release
 

photo_pipeline/services/release.py

@@ -71,6 +71,18 @@ CONTAINER_STAGES: tuple[tuple[str, tuple[str, ...]], ...] = (
 # would skip here — no daemon, no compose, no browser — means it was not proven.
 CONTAINER_ALLOWED_SKIP_REASONS: tuple[str, ...] = ()
 
+# The documentation gate (US09-05): the manuals, held to the application. Selected by
+# marker across the whole suite, because each check lives beside what it describes —
+# the offline contracts under tests/integration, the browser and screenshot journeys
+# under tests/e2e. Adding a phase_i test is enough to put it in front of a release.
+DOCS_STAGES: tuple[tuple[str, tuple[str, ...]], ...] = (
+    ("documentation", ("tests", "-m", "phase_i")),
+)
+
+# Also nothing. Every check here is either offline or needs the browser the rest of
+# the suite already needs, so a skip means the manuals were not compared with the code.
+DOCS_ALLOWED_SKIP_REASONS: tuple[str, ...] = ()
+
 
 class ReleaseError(RuntimeError):
     pass

pyproject.toml

@@ -54,4 +54,5 @@ markers = [
     "phase_f: Phase F end-to-end acceptance (US06-06) — archive destination, transfer, and restore journeys",
     "container: builds and runs the container image and its composition (US08-02, US08-03) — needs a Docker daemon, the compose plugin, and the network",
     "phase_h: Phase H container deployment acceptance (US08-05) — the browser, upgrade, restart, and security journeys against the composed stack",
+    "phase_i: Phase I documentation acceptance (US09-05) — the manuals checked against the application, in the browser and offline",
 ]

tests/e2e/test_docs_ui.py

@@ -12,10 +12,15 @@ The offline contract — reachability, dead links, pinned checksums — is
 
 from __future__ import annotations
 
+from pathlib import Path
+
 import pytest
 
 from tests.e2e._pipeline_harness import Server, seed_library
 
+# Every check here belongs to the documentation gate (US09-05).
+pytestmark = pytest.mark.phase_i
+
 
 @pytest.fixture(scope="module")
 def server(tmp_path_factory):
@@ -90,6 +95,34 @@ def test_a_mermaid_block_becomes_a_diagram_under_the_unchanged_script_policy(pag
     assert "unsafe-eval" not in policy
 
 
+def test_every_diagram_in_the_architecture_overview_draws(page, server, quiet):
+    """Five diagrams, three of them state machines (US09-03). A mermaid block with a
+    syntax error renders an error node instead of throwing, so a page that merely
+    loaded proves nothing — count the drawings and read the source for the states."""
+    source = (Path(__file__).resolve().parents[2] / "docs" / "architecture.md").read_text()
+    expected = source.count("```mermaid")
+    assert expected >= 5, "the overview lost its diagrams"
+
+    page.goto(f"{server.base}/app/#/docs?page=architecture")
+    page.get_by_test_id("diagram").first.wait_for()
+    page.wait_for_function(
+        "count => document.querySelectorAll('[data-testid=\"diagram\"] svg').length === count",
+        arg=expected,
+    )
+
+    drawn = page.locator('[data-testid="diagram"] svg')
+    assert drawn.count() == expected
+    for index in range(expected):
+        diagram = drawn.nth(index)
+        # Labels may be <text> or HTML inside a <foreignObject> depending on the
+        # diagram type, so ask for the rendered text rather than a specific element.
+        labels = diagram.evaluate("node => node.textContent.trim().length")
+        assert labels > 0, f"diagram {index} drew no labels"
+        assert diagram.locator(".error-icon, .error-text").count() == 0, f"diagram {index} errored"
+    assert page.locator("pre code.language-mermaid").count() == 0
+    assert quiet == []
+
+
 def test_an_unknown_page_says_so_without_naming_a_path(page, server, quiet):
     page.goto(f"{server.base}/app/#/docs?page=no-such-manual")
     message = page.get_by_test_id("doc-not-found")

tests/e2e/test_user_manual_screenshots.py

@@ -0,0 +1,215 @@
+"""US09-04: the user manual's screenshots, produced from the running application.
+
+A screenshot nobody can regenerate is a screenshot that silently stops being true.
+So every image in the manual is captured here, from a real server and a real worker
+driving a temporary fixture library, and never pasted in by hand.
+
+Regenerate the committed images with one command:
+
+    PHOTO_PIPELINE_WRITE_SCREENSHOTS=1 work_item/scripts/python -m pytest \
+        tests/e2e/test_user_manual_screenshots.py -q
+
+Without that variable the capture still runs on every suite — the generator has to
+keep working, and a view that stops rendering must fail here rather than in a
+reader's browser — but the images go to a temporary directory. Regenerating on every
+run would leave the working tree permanently dirty, because PNG output is not
+byte-stable.
+"""
+
+from __future__ import annotations
+
+import os
+import re
+from contextlib import closing
+from pathlib import Path
+
+import pytest
+from PIL import Image
+from playwright.sync_api import TimeoutError as PlaywrightTimeout
+
+from tests.conftest import session_client
+from tests.e2e._pipeline_harness import (
+    Server,
+    approve_album,
+    seed_album,
+    start_worker,
+    wait_until,
+)
+
+REPO = Path(__file__).resolve().parents[2]
+DOCS = REPO / "docs"
+IMAGES = DOCS / "images"
+ALBUM = "rome"
+VIEWPORT = {"width": 1280, "height": 900}
+
+# One per stage page of the manual: the route, the heading that view renders, and the
+# element whose presence means it has finished.
+#
+# The heading matters more than it looks. Every route replaces the same container, so
+# waiting for "an h1" matches the *previous* view's heading and photographs the screen
+# you just left — which is how the statistics page first shipped a picture of the
+# archive view.
+SHOTS = (
+    ("workflow", "#/workflow", "Workflow", "stage-safety"),
+    ("inventory", "#/inventory", "Inventory", "asset-row"),
+    ("duplicates", "#/duplicates", "Duplicate clusters", None),
+    ("safety", "#/safety", "Safety review", None),
+    ("analysis", "#/analyze", "Analyze", "analyze-counts"),
+    ("albums", f"#/albums?album={ALBUM}", "Albums", "suggested-name"),
+    ("renames", "#/renames", "Renames", None),
+    ("uploads", "#/uploads", "Upload", "upload-scope"),
+    ("archive", "#/archive", "Archive", "archive-locations"),
+    ("statistics", "#/stats", "Stats", None),
+)
+
+# Every check here belongs to the documentation gate (US09-05).
+pytestmark = pytest.mark.phase_i
+
+
+def writing() -> bool:
+    return os.environ.get("PHOTO_PIPELINE_WRITE_SCREENSHOTS") == "1"
+
+
+@pytest.fixture(scope="module")
+def stack(tmp_path_factory):
+    """A library in the state the manual describes: one analysed album, a duplicate
+    to review, an archive destination, and a worker that claims jobs."""
+    tmp_path = tmp_path_factory.mktemp("manual")
+    seeded = seed_album(tmp_path, ALBUM, ("forum.jpg", "colosseum.jpg"))
+    worker = start_worker(seeded, fake_vision_log=tmp_path / "vision.log")
+    server = Server(seeded).start()
+    try:
+        _prepare(server.base, seeded)
+        yield server
+    finally:
+        server.stop()
+        worker.terminate()
+        worker.wait(timeout=10)
+
+
+def _prepare(base: str, seeded) -> None:
+    """Everything the views need, established over the public API — the same calls a
+    person would make, so the screenshots show reachable states."""
+    with closing(session_client(base, timeout=30)) as client:
+        client.post("/api/v1/duplicates/detect").raise_for_status()
+        client.post("/api/v1/albums/proposals", json={}).raise_for_status()
+        destination = seeded.data / "archive"
+        destination.mkdir(exist_ok=True)
+        client.post(
+            "/api/v1/archive-locations",
+            json={"name": "external disk", "root": str(destination)},
+        ).raise_for_status()
+        wait_until(lambda: client.get("/api/v1/workflow").status_code == 200)
+
+    # An approved name and a built — deliberately unapplied — plan, so the renames
+    # screenshot shows the preview its page describes rather than an empty state.
+    # Building a plan moves nothing; applying it is what would, and nothing here does.
+    approve_album(base, album=ALBUM, name="2019 — Rome")
+    with closing(session_client(base, timeout=30)) as client:
+        client.post("/api/v1/rename-plans").raise_for_status()
+
+
+def _capture(page, server, target: Path) -> list[str]:
+    target.mkdir(parents=True, exist_ok=True)
+    page.set_viewport_size(VIEWPORT)
+    written = []
+    for name, route, heading, ready in SHOTS:
+        page.goto(f"{server.base}/app/{route}")
+        page.locator("main h1", has_text=heading).first.wait_for(timeout=30_000)
+        if ready:
+            page.get_by_test_id(ready).first.wait_for(timeout=30_000)
+        page.screenshot(path=str(target / f"{name}.png"))
+        written.append(name)
+    return written
+
+
+def test_the_generator_produces_every_screenshot_the_manual_references(page, stack, tmp_path):
+    destination = IMAGES if writing() else tmp_path / "images"
+    written = _capture(page, stack, destination)
+
+    assert sorted(written) == sorted(name for name, *_ in SHOTS)
+    for name in written:
+        produced = destination / f"{name}.png"
+        assert produced.stat().st_size > 5_000, f"{name}.png is too small to be a view"
+
+    referenced = set(re.findall(r"images/([a-z-]+)\.png", "\n".join(
+        path.read_text() for path in DOCS.rglob("*.md")
+    )))
+    assert referenced <= set(written), f"the manual references images nobody generates: {referenced - set(written)}"
+
+
+def test_no_screenshot_shows_a_real_path_or_a_secret(page, stack, tmp_path):
+    """The fixture library is synthetic and lives in a temporary directory, so the
+    pixels cannot carry someone's photographs — but the view can still print a path,
+    and that path must be the fixture's, never the operator's."""
+    with closing(session_client(stack.base, timeout=30)) as client:
+        rendered = client.get("/api/v1/inventory/assets", params={"limit": 200}).json()["items"]
+    paths = [item["current_path"] for item in rendered if item["current_path"]]
+    assert paths, "nothing was rendered, so nothing was proven"
+    assert all("/manual" in path or "pytest" in path or "/tmp" in path or "/var" in path for path in paths), paths
+    assert all(ALBUM in path or "images" in path for path in paths)
+
+
+def test_the_committed_screenshots_still_show_what_the_application_shows(page, stack, tmp_path):
+    """A UI change that invalidates the manual should be a red build, not a discovery
+    months later by somebody following a picture of a screen that no longer exists.
+
+    The tolerance is on *content*, not pixels. Comparing the committed PNGs with a
+    fresh capture was tried first and rejected: PNG output is not reproducible, font
+    rasterisation differs between the machine that generated an image and the machine
+    running the gate, and several views legitimately print the fixture library's
+    absolute path, which is a fresh temporary directory every run. A pixel or
+    perceptual comparison therefore fails for reasons that have nothing to do with
+    the manual being wrong — and a check that cries wolf gets deleted.
+
+    What actually invalidates a screenshot is the view no longer showing what its
+    page says it shows. That is what is compared here: the heading, the elements the
+    page describes, and the fact that the image was captured at the same size.
+
+    ponytail: content comparison, not pixels. A visual-diff service with per-platform
+    baselines is the upgrade if cosmetic regressions ever need catching too.
+    """
+    if not IMAGES.is_dir():
+        pytest.skip("no screenshots have been generated into the repository yet")
+
+    fresh = tmp_path / "fresh"
+    _capture(page, stack, fresh)
+
+    drifted = []
+    for name, route, heading, ready in SHOTS:
+        committed = IMAGES / f"{name}.png"
+        if not committed.is_file():
+            drifted.append(f"{name}: never committed")
+            continue
+        with Image.open(committed) as image:
+            if image.size != (VIEWPORT["width"], VIEWPORT["height"]):
+                drifted.append(f"{name}: committed at {image.size}, captured at {VIEWPORT}")
+        # The views render asynchronously, so each check waits rather than asking a
+        # question the page has not finished answering.
+        page.goto(f"{stack.base}/app/{route}")
+        try:
+            page.locator("main h1", has_text=heading).first.wait_for(timeout=15_000)
+        except PlaywrightTimeout:
+            drifted.append(f"{name}: the view no longer shows the heading '{heading}'")
+            continue
+        if ready:
+            try:
+                page.get_by_test_id(ready).first.wait_for(timeout=15_000)
+            except PlaywrightTimeout:
+                drifted.append(f"{name}: the view no longer renders '{ready}'")
+
+    assert drifted == [], (
+        f"the manual's screenshots no longer match the application: {drifted}. "
+        "Regenerate them with PHOTO_PIPELINE_WRITE_SCREENSHOTS=1 and review the result."
+    )
+
+
+def test_every_committed_screenshot_is_referenced_by_a_page():
+    """An image nobody shows is an image nobody updates."""
+    if not IMAGES.is_dir():
+        pytest.skip("no screenshots have been generated into the repository yet")
+    text = "\n".join(path.read_text() for path in DOCS.rglob("*.md"))
+    orphans = sorted(
+        image.name for image in IMAGES.glob("*.png") if f"images/{image.name}" not in text
+    )
+    assert orphans == [], f"committed but unreferenced: {orphans}"

tests/integration/test_architecture_overview.py

@@ -0,0 +1,169 @@
+"""US09-03: the architecture overview, checked against the architecture.
+
+An architecture document is the one that rots most quietly: nothing breaks when it
+describes a module that was renamed two epics ago, it just quietly misleads the next
+person. So the parts of it that the code also knows — module names, state machines,
+table names, the modules an invariant is claimed to live in — are compared with the
+code, and a missing one fails the suite.
+
+The diagrams' rendering is proven in ``tests/e2e/test_docs_ui.py``.
+"""
+
+from __future__ import annotations
+
+import re
+from pathlib import Path
+
+import pytest
+
+import photo_pipeline.models  # noqa: F401  (registers every table on Base.metadata)
+from photo_pipeline.db import Base
+from photo_pipeline.services import jobs, rename_journal, upload_batches
+
+# Every check here belongs to the documentation gate (US09-05).
+pytestmark = pytest.mark.phase_i
+
+REPO = Path(__file__).resolve().parents[2]
+PACKAGE = REPO / "photo_pipeline"
+OVERVIEW = REPO / "docs" / "architecture.md"
+TEXT = OVERVIEW.read_text()
+
+# Names the map does not owe the reader individually: the package markers, the CLI
+# entry point, and the two modules that exist to be small and obvious.
+UNMAPPED = {"__init__", "__main__", "logging"}
+
+
+def modules() -> set[str]:
+    """Every module and package under photo_pipeline, by the name it is imported as."""
+    names = set()
+    for path in PACKAGE.rglob("*.py"):
+        relative = path.relative_to(PACKAGE)
+        names.add(relative.parts[0] if len(relative.parts) > 1 else relative.stem)
+    return {name.removesuffix(".py") for name in names} - UNMAPPED
+
+
+def states(container: type) -> set[str]:
+    return {
+        value
+        for name, value in vars(container).items()
+        if name.isupper() and isinstance(value, str)
+    }
+
+
+# ── the module map ───────────────────────────────────────────────────────────
+
+
+def test_every_module_appears_in_the_map():
+    """A service nobody documented is a service the next person re-implements."""
+    missing = sorted(name for name in modules() if name not in TEXT)
+    assert missing == [], f"absent from the architecture overview: {missing}"
+
+
+def test_every_service_is_named_individually():
+    """The package table says what `services/` is for; this is the list that actually
+    drifts, because a new service is added roughly every story."""
+    services = {path.stem for path in (PACKAGE / "services").glob("*.py")} - UNMAPPED
+    missing = sorted(name for name in services if not re.search(rf"\b{name}\b", TEXT))
+    assert missing == [], f"services missing from the overview: {missing}"
+
+
+def test_the_map_names_no_module_that_does_not_exist():
+    """Every `services/<name>` or backticked module path in the document resolves."""
+    referenced = set(re.findall(r"`(?:photo_pipeline/)?([a-z_]+)/([a-z_]+)\.py`", TEXT))
+    referenced |= {("", name) for name in re.findall(r"`([a-z_]+)\.py`", TEXT)}
+    missing = [
+        f"{package}/{module}.py" if package else f"{module}.py"
+        for package, module in referenced
+        if not (PACKAGE / package / f"{module}.py").is_file()
+    ]
+    assert missing == [], f"documented but absent from the source: {missing}"
+
+
+# ── the state machines ───────────────────────────────────────────────────────
+
+
+def test_every_job_state_is_documented():
+    missing = sorted(state for state in states(jobs.JobState) if state not in TEXT)
+    assert missing == [], f"job states missing from the overview: {missing}"
+
+
+def test_every_rename_journal_state_is_documented():
+    missing = sorted(state for state in states(rename_journal.JournalState) if state not in TEXT)
+    assert missing == [], f"journal states missing from the overview: {missing}"
+
+
+def test_every_upload_batch_state_is_documented():
+    missing = sorted(state for state in states(upload_batches.BatchState) if state not in TEXT)
+    assert missing == [], f"batch states missing from the overview: {missing}"
+
+
+def test_the_unsafe_journal_states_are_named_as_the_ones_that_block():
+    """The reason an unrelated mutation is refused has to be findable."""
+    for state in rename_journal.UNSAFE_STATES:
+        assert state in TEXT
+    assert "rename_recovery_required" in TEXT
+
+
+def test_an_uncertain_upload_is_documented_as_not_retryable():
+    assert upload_batches.BatchState.UNKNOWN not in upload_batches.RUNNABLE_STATES
+    assert "not** restartable" in TEXT or "not restartable" in TEXT
+
+
+# ── the data model ───────────────────────────────────────────────────────────
+
+
+def test_every_table_is_listed():
+    missing = sorted(name for name in Base.metadata.tables if name not in TEXT)
+    assert missing == [], f"tables missing from the overview: {missing}"
+
+
+def test_the_document_lists_no_table_that_does_not_exist():
+    listed = set(re.findall(r"`([a-z_]+)`", TEXT))
+    plausible = {name for name in listed if name.endswith("s") and "_" in name}
+    invented = sorted(
+        name
+        for name in plausible
+        if name not in Base.metadata.tables
+        and not (PACKAGE / "services" / f"{name}.py").is_file()
+        and name not in {"library_roots", "trusted_proxies", "allowed_hosts", "asset_paths"}
+    )
+    assert invented == [], f"looks like a table but is not one: {invented}"
+
+
+# ── the invariants ───────────────────────────────────────────────────────────
+
+
+def test_each_invariant_points_at_a_module_that_enforces_it():
+    """The point of the table is to answer 'where do I look'. A wrong answer there
+    costs more than no answer."""
+    claims = {
+        "path_policy.is_excluded": PACKAGE / "path_policy.py",
+        "path_policy.resolve_in_roots": PACKAGE / "path_policy.py",
+        "services/app_lock.py": PACKAGE / "services" / "app_lock.py",
+        "services/exif_checkpoint.py": PACKAGE / "services" / "exif_checkpoint.py",
+        "services/archive_transfer.py": PACKAGE / "services" / "archive_transfer.py",
+        "api/security.py": PACKAGE / "api" / "security.py",
+        "imaging.py": PACKAGE / "imaging.py",
+    }
+    for claim, path in claims.items():
+        assert claim in TEXT, f"the invariant table does not mention {claim}"
+        assert path.is_file(), f"{claim} does not exist"
+        function = claim.rpartition(".")[2] if "/" not in claim else ""
+        if function and function != "py":
+            assert f"def {function}" in path.read_text(), f"{claim} is not defined there"
+
+
+def test_the_lock_order_matches_the_ranks_in_the_code():
+    from photo_pipeline.jobs import locks
+
+    order = [name for name, _ in sorted(locks.LOCK_RANK.items(), key=lambda item: item[1])]
+    assert order[0].startswith("library"), "the broadest lock is no longer the library"
+    assert "library → stage/job → album/folder → asset" in TEXT
+
+
+def test_the_frozen_donor_archive_is_described_as_provenance_not_a_dependency():
+    """That production never imports the frozen sources is proven by
+    ``tests/unit/test_legacy_archive.py`` — which also forbids any other test from
+    naming that directory, so this one checks the claim by its consequence."""
+    assert "must not import" in TEXT
+    assert "provenance and rollback evidence" in TEXT

tests/integration/test_docs_gate.py

@@ -0,0 +1,196 @@
+"""US09-05: the documentation gate's own contract, checked without a browser.
+
+The gate itself needs a browser and a server and takes minutes; what is checkable
+offline is what makes it a *gate* rather than a long test run:
+
+* it selects every documentation check by marker, so adding one is enough to put it
+  in front of a release;
+* it accepts no skip at all — a check that did not run is a page nobody compared
+  with the code;
+* it retains checksummed evidence per run;
+* CI runs it on pull requests, which is when a fix is still cheap.
+
+The checks it runs are the four suites written by US09-01 through US09-04.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+from pathlib import Path
+
+import pytest
+import yaml
+
+from photo_pipeline.config import Config
+from photo_pipeline.services import release
+
+REPO = Path(__file__).resolve().parents[2]
+DOCS = REPO / "docs"
+TEST_WORKFLOW = yaml.safe_load((REPO / ".gitea" / "workflows" / "test.yml").read_text())
+
+pytestmark = pytest.mark.phase_i
+
+# The suites the gate exists to run. Each is mapped to a story in the traceability
+# matrix, so this list is also what stops one being quietly dropped.
+DOCUMENTATION_SUITES = (
+    "tests/integration/test_documentation.py",
+    "tests/integration/test_installation_manual.py",
+    "tests/integration/test_architecture_overview.py",
+    "tests/integration/test_user_manual.py",
+    "tests/integration/test_docs_gate.py",
+    "tests/e2e/test_docs_ui.py",
+    "tests/e2e/test_user_manual_screenshots.py",
+)
+
+
+def _config(tmp_path) -> Config:
+    (tmp_path / "data").mkdir(exist_ok=True)
+    return Config.from_env({"PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data")})
+
+
+# ── what the gate runs ───────────────────────────────────────────────────────
+
+
+def test_the_gate_selects_every_documentation_check_by_marker():
+    assert release.DOCS_STAGES == (("documentation", ("tests", "-m", "phase_i")),)
+    declared = [
+        line
+        for line in (REPO / "pyproject.toml").read_text().splitlines()
+        if line.strip().startswith('"phase_i:')
+    ]
+    assert declared, "an unregistered marker selects nothing and fails no gate"
+
+
+def test_every_documentation_suite_carries_the_marker():
+    """A documentation test without the marker is a check the gate never runs."""
+    unmarked = [
+        suite
+        for suite in DOCUMENTATION_SUITES
+        if "pytest.mark.phase_i" not in (REPO / suite).read_text()
+    ]
+    assert unmarked == [], f"suites missing the phase_i marker: {unmarked}"
+
+
+def test_the_marker_selects_nothing_outside_the_documentation_suites():
+    """Scope matters: the gate's promise is that it ran *the documentation checks*."""
+    marked = sorted(
+        str(path.relative_to(REPO))
+        for path in (REPO / "tests").rglob("test_*.py")
+        if "pytest.mark.phase_i" in path.read_text()
+    )
+    assert marked == sorted(DOCUMENTATION_SUITES)
+
+
+# ── no skip is an environment limit here ─────────────────────────────────────
+
+
+def test_the_gate_accepts_no_skipped_check_at_all():
+    assert release.DOCS_ALLOWED_SKIP_REASONS == ()
+    skipped = release.StageResult(
+        "documentation", [], 0, 0.1, "1 skipped", ["SKIPPED [1] x.py:1: exiftool not installed"]
+    )
+    # The release gate tolerates this one because it describes the machine. The
+    # documentation gate cannot: nothing here depends on the machine.
+    assert release.unexpected_skips([skipped]) == []
+    assert release.unexpected_skips([skipped], release.DOCS_ALLOWED_SKIP_REASONS) == [
+        "SKIPPED [1] x.py:1: exiftool not installed"
+    ]
+
+
+def test_a_skipped_check_fails_the_run_and_the_evidence_says_so(tmp_path):
+    skipping = tmp_path / "test_skipping.py"
+    skipping.write_text(
+        "import pytest\n\ndef test_x():\n    pytest.skip('exiftool not installed')\n"
+    )
+    evidence = tmp_path / "evidence"
+
+    report = release.run_gate(
+        _config(tmp_path),
+        output=evidence,
+        stages=(("documentation", (str(skipping),)),),
+        allowed_skips=release.DOCS_ALLOWED_SKIP_REASONS,
+    )
+
+    assert report["ok"] is False
+    assert report["failures"] == [], "the stage passed; the skip is what fails the gate"
+    assert report["unexpected_skips"], report
+    written = json.loads((evidence / release.REPORT_NAME).read_text())
+    assert written["ok"] is False
+    assert (evidence / "logs" / "documentation.log").exists()
+    for line in (evidence / release.CHECKSUMS_NAME).read_text().splitlines():
+        digest, name = line.split("  ", 1)
+        assert release.sha256_file(evidence / name) == digest
+
+
+def test_a_broken_documentation_check_fails_the_gate(tmp_path):
+    """The failure mode that matters: a page and the code disagreeing."""
+    failing = tmp_path / "test_disagreement.py"
+    failing.write_text(
+        "def test_the_manual_matches_the_code():\n"
+        "    documented = {'lock_held'}\n"
+        "    emitted = {'lock_held', 'renamed_since'}\n"
+        "    assert emitted <= documented\n"
+    )
+
+    report = release.run_gate(
+        _config(tmp_path),
+        output=tmp_path / "evidence",
+        stages=(("documentation", (str(failing),)),),
+        allowed_skips=release.DOCS_ALLOWED_SKIP_REASONS,
+    )
+
+    assert report["ok"] is False
+    assert report["failures"], report
+
+
+def test_a_clean_tree_passes(tmp_path):
+    passing = tmp_path / "test_passing.py"
+    passing.write_text("def test_x():\n    assert True\n")
+
+    report = release.run_gate(
+        _config(tmp_path),
+        output=tmp_path / "evidence",
+        stages=(("documentation", (str(passing),)),),
+        allowed_skips=release.DOCS_ALLOWED_SKIP_REASONS,
+    )
+
+    assert report["ok"] is True and report["unexpected_skips"] == []
+    assert report["revision"], "the evidence must say which commit it covers"
+
+
+# ── the command, and CI ──────────────────────────────────────────────────────
+
+
+def test_the_command_is_documented_and_wired():
+    assert "docs-gate" in (REPO / "photo_pipeline" / "__main__.py").read_text()
+    assert "docs-gate" in (DOCS / "index.md").read_text()
+
+
+def test_ci_runs_the_gate_on_pull_requests_and_keeps_its_evidence():
+    job = TEST_WORKFLOW["jobs"]["documentation"]
+    # No `if:` — documentation drifts on the same commits that change behaviour, and
+    # the pull request is when saying so is still cheap.
+    assert "if" not in job
+    assert "pull_request" in TEST_WORKFLOW[True] or "pull_request" in TEST_WORKFLOW.get("on", {})
+    script = "\n".join(step["run"] for step in job["steps"] if "run" in step)
+    assert "photo_pipeline docs-gate" in script
+    evidence = next(step for step in job["steps"] if "upload-artifact" in str(step.get("uses")))
+    assert evidence["if"] == "always()", "a failed gate's evidence is the one worth keeping"
+
+
+# ── the pages and the repository point at each other ─────────────────────────
+
+
+def test_the_readme_and_the_documentation_index_link_to_each_other():
+    """Neither may become the forgotten copy."""
+    readme = (REPO / "README.md").read_text()
+    index = (DOCS / "index.md").read_text()
+    assert "docs/index.md" in readme
+    assert re.search(r"\.\./README\.md", index), "the index does not link back to the README"
+
+
+def test_every_documentation_story_is_mapped_in_the_traceability_matrix():
+    matrix = json.loads((REPO / "tests" / "story_traceability.json").read_text())["stories"]
+    for story in ("US09-01", "US09-02", "US09-03", "US09-04", "US09-05"):
+        assert matrix.get(story), f"{story} is not mapped to any test"

tests/integration/test_documentation.py

@@ -16,12 +16,17 @@ import json
 import re
 from pathlib import Path
 
+import pytest
+
 from starlette.testclient import TestClient
 
 from photo_pipeline.api.app import create_app
 from photo_pipeline.api.security import DEFAULT_HEADERS
 from photo_pipeline.config import Config
 
+# Every check here belongs to the documentation gate (US09-05).
+pytestmark = pytest.mark.phase_i
+
 REPO = Path(__file__).resolve().parents[2]
 DOCS = REPO / "docs"
 VENDOR = REPO / "frontend" / "js" / "vendor"

tests/integration/test_installation_manual.py

@@ -16,8 +16,13 @@ import sys
 from functools import lru_cache
 from pathlib import Path
 
+import pytest
+
 from photo_pipeline.config import ENV_PREFIX, LEGACY_ALIASES, Config
 
+# Every check here belongs to the documentation gate (US09-05).
+pytestmark = pytest.mark.phase_i
+
 REPO = Path(__file__).resolve().parents[2]
 MANUAL = REPO / "docs" / "installation.md"
 MAIN = REPO / "photo_pipeline" / "__main__.py"

tests/integration/test_user_manual.py

@@ -0,0 +1,177 @@
+"""US09-04: the user manual, checked against the application it describes.
+
+The manual's most perishable claim is its error catalogue: a code renamed in a route
+leaves a page confidently explaining something that can no longer happen, and the
+operator meeting the new code finds nothing. So the catalogue is compared with the
+codes the application can actually emit, in both directions.
+
+The screenshots are generated and checked by
+``tests/e2e/test_user_manual_screenshots.py``.
+"""
+
+from __future__ import annotations
+
+import re
+from pathlib import Path
+
+import pytest
+
+REPO = Path(__file__).resolve().parents[2]
+DOCS = REPO / "docs"
+STAGES = DOCS / "stages"
+PACKAGE = REPO / "photo_pipeline"
+ERRORS = (DOCS / "errors.md").read_text()
+ALL_DOCS = "\n".join(path.read_text() for path in DOCS.rglob("*.md"))
+
+# The shapes an error code is emitted in. Everything the API returns goes through one
+# of these, so the set they find is the set an operator can meet.
+EMITTERS = (
+    re.compile(r'_error\(\s*\d+,\s*"([a-z_]+)"'),
+    # The status may be a literal or an exception's own code, so match either.
+    re.compile(r'_envelope\(\s*[\w.]+,\s*"([a-z_]+)"'),
+    re.compile(r'"code":\s*"([a-z_]+)"'),
+    re.compile(r'code="([a-z_]+)"'),
+    re.compile(r'Refusal\(\s*\d+,\s*"([a-z_]+)"'),
+)
+
+# Codes raised deep in a service and mapped to a response by its route; they are what
+# the browser shows, so the manual owes the reader an entry.
+SERVICE_CODES = {
+    "lock_held",
+    "path_not_allowed",
+    "rename_recovery_required",
+    "stale_preflight",
+    "access_denied",
+    "too_many_attempts",
+}
+
+STAGE_PAGES = (
+    "inventory",
+    "duplicates",
+    "safety",
+    "analysis",
+    "albums",
+    "renames",
+    "uploads",
+    "archive",
+    "diagnostics",
+)
+
+# Every check here belongs to the documentation gate (US09-05).
+pytestmark = pytest.mark.phase_i
+
+
+def emitted_codes() -> set[str]:
+    found: set[str] = set()
+    for path in PACKAGE.rglob("*.py"):
+        text = path.read_text()
+        for pattern in EMITTERS:
+            found |= set(pattern.findall(text))
+    for code in SERVICE_CODES:
+        assert re.search(rf'"{code}"', "\n".join(p.read_text() for p in PACKAGE.rglob("*.py"))), (
+            f"{code} is documented as a service code but no longer exists"
+        )
+    return (found | SERVICE_CODES) - {"code"}
+
+
+def documented_codes() -> set[str]:
+    return set(re.findall(r"`([a-z][a-z_]{3,})`", ERRORS))
+
+
+# ── the error catalogue ──────────────────────────────────────────────────────
+
+
+def test_every_error_the_application_can_emit_is_documented():
+    missing = sorted(emitted_codes() - documented_codes())
+    assert missing == [], f"undocumented error codes: {missing}"
+
+
+def test_the_catalogue_documents_no_code_that_cannot_happen():
+    """Backticked snake_case in the catalogue is either a code, a setting, or one of
+    the handful of prose terms below."""
+    prose = {
+        "diagnostics",
+        "dry_run",
+        "approve_dry_run",
+        "current_path",
+        "archived_online",
+        "archived_offline",
+    }
+    settings = set(re.findall(r"PHOTO_PIPELINE_[A-Z_]+", ERRORS))
+    invented = sorted(
+        code
+        for code in documented_codes()
+        if code not in emitted_codes()
+        and code not in prose
+        and f"PHOTO_PIPELINE_{code.upper()}" not in settings
+    )
+    assert invented == [], f"documented but not emitted anywhere: {invented}"
+
+
+def test_the_protective_refusals_are_explained_as_intentional():
+    """These are the ones a reader is most likely to mistake for a broken
+    application, which is exactly when they start working around them."""
+    for code in ("lock_held", "rename_recovery_required", "stale_preflight", "not_runnable"):
+        row = next(line for line in ERRORS.splitlines() if line.startswith(f"| `{code}`"))
+        assert len(row.split("|")) >= 4, f"{code} has no cause and remedy"
+
+
+# ── the stage pages ──────────────────────────────────────────────────────────
+
+
+def test_there_is_one_page_per_workflow_stage():
+    missing = [name for name in STAGE_PAGES if not (STAGES / f"{name}.md").is_file()]
+    assert missing == [], f"stages without a page: {missing}"
+
+
+def test_every_stage_page_answers_the_four_questions():
+    """The shape is the point: a page that skips 'what it changes' is the page
+    somebody needed."""
+    for name in STAGE_PAGES:
+        text = (STAGES / f"{name}.md").read_text()
+        for question in ("What it is for.", "What you decide.", "What it changes.", "What it refuses."):
+            assert question in text, f"{name}.md does not answer: {question}"
+
+
+def test_every_stage_page_shows_a_screenshot():
+    for name in STAGE_PAGES:
+        text = (STAGES / f"{name}.md").read_text()
+        assert re.search(r"!\[[^\]]+\]\(\.\./images/[a-z-]+\.png\)", text), (
+            f"{name}.md carries no screenshot"
+        )
+
+
+def test_the_guided_pass_names_the_irreversible_stages():
+    first_pass = (DOCS / "first-pass.md").read_text()
+    for stage in ("renames", "uploads", "archive"):
+        assert f"stages/{stage}.md" in first_pass
+    assert "point" in first_pass.lower() and "no return" in first_pass.lower()
+
+
+def test_recovery_covers_each_interruption_the_application_can_survive():
+    recovery = (DOCS / "recovery.md").read_text()
+    for topic in (
+        "interrupted rename",
+        "uncertain upload",
+        "failed migration",
+        "restored backup",
+    ):
+        assert topic in recovery.lower(), f"recovery does not cover: {topic}"
+
+
+def test_the_manual_is_reachable_from_the_index():
+    index = (DOCS / "index.md").read_text()
+    for page in [f"stages/{name}.md" for name in STAGE_PAGES] + [
+        "first-pass.md",
+        "errors.md",
+        "recovery.md",
+    ]:
+        assert page in index, f"{page} is not linked from the index"
+
+
+def test_the_screenshots_are_declared_as_generated():
+    """The allow list had to be widened for them, so the reason has to be visible
+    where the widening is."""
+    config = (REPO / "work_item" / ".work-item.yml").read_text()
+    assert "docs/images/**" in config
+    assert "test_user_manual_screenshots.py" in config, "no note saying who generates them"

tests/story_traceability.json

@@ -200,12 +200,19 @@
     ],
     "US09-02": [
       "tests/integration/test_installation_manual.py"
+    ],
+    "US09-03": [
+      "tests/integration/test_architecture_overview.py",
+      "tests/e2e/test_docs_ui.py"
+    ],
+    "US09-04": [
+      "tests/integration/test_user_manual.py",
+      "tests/e2e/test_user_manual_screenshots.py"
+    ],
+    "US09-05": [
+      "tests/integration/test_docs_gate.py"
     ]
   },
-  "planned": [
-    "US09-03",
-    "US09-04",
-    "US09-05"
-  ],
+  "planned": [],
   "_planned_comment": "Accepted backlog stories that are not implemented yet. The release gate (US07-07) requires every story file to be either mapped to tests or listed here, so an unimplemented story is a visible decision rather than a hole in the matrix."
 }

work_item/.work-item.yml

@@ -16,6 +16,9 @@ safety:
   max_file_bytes: 5000000
   allow:
     - tests/fixtures/**
+    # Generated by tests/e2e/test_user_manual_screenshots.py from a synthetic fixture
+    # library, never a real photo. The deny list below blocks images by default.
+    - docs/images/**
   deny:
     - _IGNORE/**
     - "**/_IGNORE/**"