US06-01: Configure and Preflight Archive Destinations

.gitignore

@@ -17,6 +17,3 @@ _todo/
 pictures/
 photos/
 _IGNORE/
-
-# Test failure evidence (US07-04)
-.artifacts/

AGENTS.md

@@ -80,8 +80,7 @@ are the authoritative backlog.
    uncertain. Work only on the claimed story and its generated feature branch.
 4. Read the entire issue, linked specification, dependencies, and acceptance criteria.
    Reconcile them with the concept before designing or changing code.
-5. Inspect the legacy CLI donors — frozen in `legacy_cli_archive/` since US07-01,
+5. Inspect the legacy CLI donors before replacing applicable behavior. Update the donor
-   with their ledger — before replacing applicable behavior. Update the donor
    ledger and characterization tests required by the story.
 6. Implement every acceptance criterion and its automated tests.
 7. Run story-specific tests and the accumulated regression suite required by the epic.

README.md

@@ -17,25 +17,6 @@ python -m photo_pipeline serve     # start the API + static review UI (127.0.0.1
 Configuration comes from `PHOTO_PIPELINE_*` environment variables (see
 `photo_pipeline/config.py`); secrets are referenced, never logged.
 
-### API access (US07-02)
-
-The app listens on loopback, so its attacker is another page in the same browser.
-Every `/api/v1` route except `health/live`, `health/ready`, and `session` requires
-the application session, and every mutation requires its CSRF token as well:
-
-```bash
-BASE=http://127.0.0.1:8000
-TOKEN=$(curl -sc /tmp/pp.jar $BASE/api/v1/session | python -c 'import json,sys; print(json.load(sys.stdin)["csrf_token"])')
-curl -sb /tmp/pp.jar -H "X-CSRF-Token: $TOKEN" -X POST $BASE/api/v1/albums/proposals -d '{}' -H 'Content-Type: application/json'
-```
-
-The session is per server process — restarting `serve` invalidates it, and the
-browser client re-bootstraps by itself. Requests are also refused when the `Host` is
-not a loopback name (DNS rebinding), when `Origin` is any other origin, when
-`Sec-Fetch-Site` says the request came from another site (an `<img>` pointed at a
-thumbnail), or when the body exceeds `PHOTO_PIPELINE_MAX_REQUEST_BYTES`. There is no
-CORS middleware at all, so no other origin can read a response.
-
 ## Testing
 
 One offline command runs the whole suite (unit, integration, and browser
@@ -188,128 +169,3 @@ work_item/scripts/python -m pytest -m phase_e -q
   bytes, and recovery after a restart).
 
 Phases A–D remain green in the full run above.
-
-### Phase F acceptance gate
-
-Phase F (Epic E06: archive lifecycle) is the only stage that *removes* originals
-from the library, and the only one whose storage can walk away in someone's bag.
-One command runs the archive fault-injection suites, the black-box archive and
-restore API journeys, and the browser suite:
-
-```bash
-work_item/scripts/python -m pytest -m phase_f -q
-```
-
-- `tests/integration/test_archive_*.py` and `tests/integration/test_restore.py`
-  drive real files on real filesystems: preflight against a mounted, missing,
-  swapped, read-only, or full medium; copy-verify-remove and the same-filesystem
-  move path; and a crash at **every** persisted journal transition in both transfer
-  modes, asserting that no source is ever removed without a durable, byte-identical
-  archive copy.
-- `tests/e2e/test_phase_f_pipeline.py` drives a real server and a real durable
-  worker over HTTP: preflight blockers (offline medium, wrong volume, insufficient
-  capacity, bytes changed after upload), a verified archive whose manifest, hashes,
-  and path history are checked on the medium itself, a worker killed at each of
-  `transferring`, `verified`, `removing`, `source_removed`, and `complete`, the
-  evidence-based recovery that follows, offline deduplication of an exact and a
-  fuzzy copy while the medium is away, mount return, restore, and a collision that
-  restores beside its occupant.
-- **Archived is not missing.** An unmounted medium leaves its photos
-  `archived_offline` — still hashed, still in the duplicate indexes, still
-  previewable through their protected thumbnails — and a rescan neither prunes nor
-  flags them.
-- **Ambiguity is never guessed.** A journal state the medium contradicts stays
-  `manual`, offers no automatic action, and keeps blocking further archiving until
-  a human decides.
-- `tests/e2e/test_archive_ui.py` covers the browser journeys (preview with
-  destination identity and reclaimable bytes, blockers and mount instructions,
-  progress split into transfer/verification/removal, interruption and recovery,
-  offline browsing, restore, collision, keyboard confirmation, and reload).
-
-Phases A–E remain green in the full run above.
-
-## Media and metadata hardening (US07-03)
-
-Every pixel the application reads goes through `photo_pipeline/imaging.py`: the
-declared dimensions are checked before anything is decoded, Pillow's
-decompression-bomb warning is treated as a refusal, JPEG decodes near the requested
-size, and each decoder failure becomes one of two typed errors. A damaged file is a
-per-item error with a persisted code, never a failed scan or a dead worker.
-
-Every metadata stage ends with an EXIF checkpoint (`services/exif_checkpoint.py`):
-snapshot, write the owned keywords, read back, prove the owned fields landed and that
-nothing else moved, refresh the file hash. A field the stage does not own that
-changed anyway makes the checkpoint `divergent` — recorded in `exif_projections`,
-shown in the review queue, never repaired behind the user's back, and not counted as
-verified, so upload stays blocked.
-
-The golden corpus that proves all of it is generated, not committed:
-`tests/fixtures/media_corpus.py` declares every format, orientation, profile,
-damage, and metadata case with its expected outcome, and the suite regenerates it
-twice to prove it does not drift.
-
-```bash
-work_item/scripts/python -m pytest tests/integration/test_media_hardening.py tests/integration/test_exif_checkpoints.py -q
-```
-
-## Concurrency and crash recovery (US07-04)
-
-Crash safety is proven by crashing. `photo_pipeline/faults.py` defines the control
-points — the persisted transitions of the rename, archive, EXIF, upload, and job
-lanes — and arms one only when `PHOTO_PIPELINE_FAULT_AFTER` names it, at which
-point the process dies the way a `SIGKILL` does. There is no endpoint and no
-configuration field that can reach a barrier; a deployment that never sets the
-variable can never hit one.
-
-The race suite runs each scenario several times with a seed recorded on the test
-result (`race_seed`) and asserts invariants rather than schedules: work is never
-claimed or executed twice, a stale fencing token never commits, no file body is
-lost or overwritten, and the database still passes `PRAGMA integrity_check`.
-
-```bash
-work_item/scripts/python -m pytest tests/integration/test_concurrency_races.py \
-    tests/integration/test_fault_matrix.py tests/e2e/test_crash_recovery.py -q
-
-# replay a failure, or soak for new interleavings
-PHOTO_PIPELINE_RACE_SEED=1234 PHOTO_PIPELINE_RACE_REPEATS=50 \
-    work_item/scripts/python -m pytest tests/integration/test_concurrency_races.py -q
-```
-
-Any failing test keeps its evidence: the temporary database (with its write-ahead
-log), the journals, the logs, the recorded seed, and a SHA-256 manifest of every
-file in the temporary library are copied to `.artifacts/<test id>/` before pytest
-deletes the directory. Point `PHOTO_PIPELINE_TEST_ARTIFACTS` elsewhere to collect
-them from CI.
-
-## Legacy CLI archive
-
-The command-line tools this application was extracted from are frozen in
-`legacy_cli_archive/` (US07-01): the original sources, their docs, the dependency
-lock they were last verified against, schema notes, a redacted sample
-configuration, the donor ledger, and a checksum for every file.
-
-```bash
-cd legacy_cli_archive && shasum -a 256 -c CHECKSUMS.sha256   # verify the archive
-work_item/scripts/python -m pytest tests/unit/test_legacy_archive.py -q   # lint it
-```
-
-They are reference material and rollback evidence only. No module under
-`photo_pipeline/` imports or executes them, the archive is not on the application's
-import path, and `tests/unit/test_legacy_archive.py` enforces that along with the
-checksums and the redaction. Only the two suites that compare *against* the donors —
-`tests/characterization/` and `tests/integration/test_safety_parity.py` — put the
-archived sources on `sys.path`.
-
-The last path-keyed state they owned, `nsfw_scores.csv`, is imported once and then
-left alone:
-
-```bash
-work_item/scripts/python -m photo_pipeline import-legacy-scores /path/to/nsfw_scores.csv --dry-run
-```
-
-The import writes scored-but-unreviewed `safety_reviews` rows onto stable asset ids,
-never invents an asset for an unknown path, never overwrites a human decision, and
-writes a reconciliation report to the data directory saying exactly what it did.
-`legacy_cli_archive/donor_ledger.yaml` records every migrated behavior with its
-target, the tests that pin the donor, the tests that prove the replacement, and each
-intentional delta; rows still marked `pending` name the story that will resolve them.

donor_ledger.yaml

@@ -12,15 +12,7 @@
 #                            (if anything) carries over
 # Every row needs either `tests` (existing test IDs, module::function) or
 # `pending_story` (the backlog story that will characterize/deliver it).
-# status: characterized — donor behavior pinned by characterization tests
+# status: characterized | pending
-#         resolved      — replacement shipped; `parity` names the tests that prove
-#                         it, and `delta` states every intentional difference
-#         pending       — not migrated yet; `pending_story` says which story will
-# parity: test ids (path::function) in any suite, proving the replacement
-# delta:  what the replacement deliberately does differently, or not at all
-#
-# Archived by US07-01: the sources referenced below now live beside this file in
-# src/ and are frozen (see README.md). Nothing in photo_pipeline imports them.
 
 rows:
   # ── photo_analyzer.py ──────────────────────────────────────────────────────
@@ -88,12 +80,8 @@ rows:
       Copying a primary's analysis into variant rows survives, but keyed by
       asset_id and recorded as stage state instead of raw row copies.
     target: photo_pipeline/services/duplicates.py
-    parity:
+    pending_story: US01-04
-      - tests/integration/test_duplicate_engine.py::test_exact_copies_form_auto_decided_cluster
+    status: pending
-      - tests/integration/test_duplicate_engine.py::test_perceptual_variant_is_review_only
-    delta: >
-      The donor propagated variant links implicitly while writing rows; the replacement makes the canonical link a reviewable, reversible cluster decision, so a propagated link can always be undone.
-    status: resolved
 
   - id: pa-hashing
     area: hashing
@@ -153,12 +141,8 @@ rows:
     classification: replace
     rationale: Console report; superseded by the duplicate-review API/UI (US01-06).
     target: photo_pipeline/api/routes + frontend duplicate review
-    parity:
+    pending_story: US01-06
-      - tests/integration/test_review_api.py::test_clusters_list_and_detail
+    status: pending
-      - tests/e2e/test_review_ui.py::test_fuzzy_decision_requires_confirmation
-    delta: >
-      The text listing became the paged cluster API and the comparison UI; no textual report is produced.
-    status: resolved
 
   - id: pa-reconcile
     area: database
@@ -218,7 +202,7 @@ rows:
       RGB-normalize (drops alpha, converts HEIC), LANCZOS resize to 2048px
       long-edge, JPEG q85 base64 — the provider-input contract. Truncated-image
       tolerance (ImageFile.LOAD_TRUNCATED_IMAGES) carries with it.
-    target: photo_pipeline/services/analysis.py
+    target: photo_pipeline/integrations/vision.py
     tests:
       - test_pa_imaging::test_prepare_image_small_passthrough_jpeg
       - test_pa_imaging::test_prepare_image_resizes_to_max_long_edge
@@ -246,12 +230,9 @@ rows:
       response validation, 429/503 retry with exponential backoff. Prompt and
       model/config version must be persisted per analysis_runs. Characterized
       against a deterministic fake provider when the analysis service is ported.
-    target: photo_pipeline/services/analysis.py
+    target: photo_pipeline/integrations/vision.py
-    parity:
+    pending_story: US02-06
-      - tests/integration/test_safety_analysis.py::test_provider_called_only_for_confirmed_sfw
+    status: pending
-    delta: >
-      The prompt and response schema carry over; the provider is an injected adapter so the privacy gate is testable, and results are keyed to asset ids rather than paths.
-    status: resolved
 
   - id: pa-throttle
     area: logging
@@ -260,8 +241,8 @@ rows:
     rationale: >
       Rolling throttle window + persistent throttle_events.jsonl + RPD day
       counter become job metrics/events on the durable job model.
-    target: photo_pipeline/services/jobs.py + photo_pipeline/services/analysis.py
+    target: photo_pipeline/jobs/coordinator.py
-    pending_story: US07-04
+    pending_story: US02-02
     status: pending
 
   - id: pa-nsfw-filter
@@ -331,12 +312,8 @@ rows:
       checked between items, double-SIGINT force quit — becomes the durable
       JobRunner worker loop with the same drain-and-resume semantics.
     target: photo_pipeline/jobs/worker.py
-    parity:
+    pending_story: US02-02
-      - tests/integration/test_worker.py::test_worker_processes_all_items
+    status: pending
-      - tests/integration/test_worker.py::test_cooperative_cancellation_leaves_items_resumable
-    delta: >
-      The in-process folder loop with SIGINT handling became durable jobs claimed by a worker: cancellation is a persisted request, not a signal, and an interrupted run resumes from the database instead of restarting.
-    status: resolved
 
   - id: pa-ui-terminal
     area: ui
@@ -394,12 +371,8 @@ rows:
       the JSONL history logger become structured JSON logging with job_id/
       asset_id and job_events rows; per-photo history maps to job events.
     target: photo_pipeline structured logging + jobs/job_events
-    parity:
+    pending_story: US02-02
-      - tests/integration/test_jobs.py::test_enqueue_persists_items_and_event
+    status: pending
-      - tests/integration/test_jobs_sse.py::test_sse_streams_all_events_then_closes
-    delta: >
-      The JSONL history file and rich console handler are replaced by structured JSON logs plus durable job_events; the browser reads events over SSE rather than tailing a file.
-    status: resolved
 
   - id: pa-balance
     area: vision
@@ -409,11 +382,8 @@ rows:
       Provider balance/quota probes (report 'unsupported' on providers without
       the endpoint). Network-bound; characterized against the fake provider.
     target: photo_pipeline/services/analysis.py
-    parity:
+    pending_story: US02-06
-      - tests/integration/test_safety_analysis.py::test_provider_called_only_for_confirmed_sfw
+    status: pending
-    delta: >
-      Not carried over: balance/quota polling was provider-specific (Gemini/OpenAI billing endpoints) and key-scoped. Cost reporting, when a story asks for it, comes from the per-run usage recorded with each analysis result rather than from a vendor endpoint.
-    status: resolved
 
   - id: pa-cli
     area: configuration
@@ -423,13 +393,9 @@ rows:
       argparse surface is superseded by the API; flags map to job configs
       (documented in WEBAPP_CONCEPT.md §8 parity table). Transitional CLI calls
       the shared services until archival (E07).
-    target: photo_pipeline/__main__.py (serve | worker | migrate | import-legacy-scores) + /api/v1
+    target: photo_pipeline/api + transitional CLI
-    parity:
+    pending_story: US07-01
-      - tests/integration/test_app_lifecycle.py::test_restart_preserves_data_and_reruns_migrations
+    status: pending
-      - tests/unit/test_legacy_archive.py::test_production_code_never_imports_an_archived_module
-    delta: >
-      The argparse surface is not reproduced. Every flag that drove work became an API command or a job configuration; the CLI keeps only what an application needs to be operated (serve, worker, migrate) plus the one-off legacy CSV import.
-    status: resolved
 
   # ── nsfwtag/ ───────────────────────────────────────────────────────────────
   - id: nt-discovery
@@ -456,17 +422,9 @@ rows:
       nsfw_scores.csv stops being the source of truth (concept: DB state).
       Format characterized (4-decimal scores, bad rows dropped) because the
       existing CSV must migrate into assets.safety_score.
-    target: photo_pipeline/services/legacy_import.py (one-off import into safety_reviews)
+    target: photo_pipeline/repositories (safety), CSV import in US01-02 migration
     tests: [test_nsfwtag::test_score_cache_roundtrip_and_tolerance]
-    parity:
+    status: characterized
-      - tests/integration/test_legacy_import.py::test_scores_are_imported_onto_asset_identity
-      - tests/integration/test_legacy_import.py::test_a_reviewed_asset_is_never_overwritten_by_the_csv
-    delta: >
-      The CSV is no longer read at runtime at all: it is imported once into
-      scored-but-unreviewed safety_reviews rows and left on disk untouched. A path
-      that matches nothing is reported, never turned into an asset, and a human
-      decision always outranks an imported score.
-    status: resolved
 
   - id: nt-score-model
     area: nsfw
@@ -480,13 +438,6 @@ rows:
       inference path needs the local model + deterministic fake.
     target: photo_pipeline/integrations/nsfw_model.py
     tests: [test_nsfwtag::test_score_images_cache_hit_skips_model]
-    delta: >
-      The donor set Pillow's process-global ImageFile.LOAD_TRUNCATED_IMAGES so a
-      partially downloaded file still scored. Here the same process also hashes
-      files and renders previews, and those must keep refusing a truncated file
-      rather than silently working on half of one; scoring opens images through
-      the bounded photo_pipeline.imaging door instead and skips the ones it cannot
-      read, leaving them unscored and visibly undecided (US07-03).
     status: characterized
 
   - id: nt-exif-keyword
@@ -534,11 +485,8 @@ rows:
       Newline-list bulk tagging (nsfw_confirmed.txt flow) is superseded by DB
       review decisions; the existing list is a one-time migration input.
     target: photo_pipeline/services/safety.py (decision import in US01-02)
-    parity:
+    pending_story: US01-02
-      - tests/integration/test_safety_parity.py::test_extracted_marks_and_partition_match_donor
+    status: pending
-    delta: >
-      Bulk keyword application from a file list is replaced by decisions against asset ids; the EXIF write itself is the extracted, read-back-verified one.
-    status: resolved
 
   - id: nt-ui
     area: ui
@@ -550,11 +498,8 @@ rows:
       threshold/score review flow, lightbox and keyboard model are the frontend
       donor for the Safety view (preserved per concept §10; ported in US02-01).
     target: photo_pipeline/api + frontend Safety view
-    parity:
+    pending_story: US02-01
-      - tests/e2e/test_workflow_views.py::test_safety_review_decide_persists_across_reload
+    status: pending
-    delta: >
-      The stdlib review server is replaced by the API plus the Safety view; the donor's layout, thresholds, and keyboard flow carry over, its embedded HTML generation does not.
-    status: resolved
 
   - id: nt-bench
     area: nsfw
@@ -564,11 +509,8 @@ rows:
       Dev-only model benchmark; archived without webapp replacement (recorded
       basis of the AdamCodd model choice). No production caller.
     target: none (archive as reference)
-    parity:
+    pending_story: US07-01
-      - tests/unit/test_legacy_archive.py::test_every_archived_source_matches_its_checksum
+    status: pending
-    delta: >
-      No replacement: a dev-only benchmark whose result (the AdamCodd model choice) is already recorded. Kept in the archive as the basis of that choice.
-    status: resolved
 
   # ── webapp/ ────────────────────────────────────────────────────────────────
   - id: wa-query-search
@@ -637,13 +579,9 @@ rows:
       Subprocess-driving-the-CLI job control is superseded by durable DB jobs
       with a worker process. Two ideas carry over: progress derived from DB
       counts (not job-private state) and single-mutating-job enforcement.
-    target: photo_pipeline/services/jobs.py + photo_pipeline/jobs/worker.py
+    target: photo_pipeline/jobs/coordinator.py
-    parity:
+    pending_story: US02-02
-      - tests/integration/test_jobs.py::test_idempotency_key_returns_same_job
+    status: pending
-      - tests/integration/test_worker.py::test_handler_failure_fails_the_job
-    delta: >
-      Subprocess supervision of a CLI is replaced by durable jobs in the same process family: there is no subprocess to supervise, and progress is persisted rather than scraped from stdout.
-    status: resolved
 
   - id: wa-server
     area: ui
@@ -655,9 +593,5 @@ rows:
       browser. analyzer.html + page.py design (dark OLED tokens, Library/
       Analyze/Stats views) is frontend donor material per concept §10.
     target: photo_pipeline/api/app.py + frontend
-    parity:
+    pending_story: US02-05
-      - tests/e2e/test_frontend_shell.py::test_shell_loads_assets_without_console_or_network_errors
+    status: pending
-      - tests/e2e/test_frontend_shell.py::test_deep_link_and_reload_restore_view_and_filters
-    delta: >
-      The stdlib HTTP routes become the versioned FastAPI API and the static frontend shell; Python no longer interpolates HTML.
-    status: resolved

frontend/index.html

@@ -19,7 +19,6 @@
         <a href="#/albums" data-nav="albums">Albums</a>
         <a href="#/renames" data-nav="renames">Renames</a>
         <a href="#/uploads" data-nav="uploads">Upload</a>
-        <a href="#/archive" data-nav="archive">Archive</a>
         <a href="#/stats" data-nav="stats">Stats</a>
       </nav>
     </header>

frontend/js/api.js

@@ -2,43 +2,14 @@
 // cancellation. Every method accepts an optional { signal } from cancellable().
 export const BASE = "/api/v1";
 
-// The API refuses every request without the session cookie, and every mutation
-// without this token echoed back. The token is readable only same-origin, which is
-// what makes it proof that the caller is this app and not another page.
-let csrfToken = null;
-
-async function session() {
-  if (csrfToken === null) {
-    const response = await fetch(BASE + "/session", { credentials: "same-origin" });
-    const body = await response.json().catch(() => null);
-    csrfToken = (body && body.csrf_token) || null;
-  }
-  return csrfToken || "";
-}
-
-async function send(path, { signal, ...options }) {
-  return fetch(BASE + path, {
-    credentials: "same-origin",
-    signal,
-    ...options,
-    headers: {
-      "Content-Type": "application/json",
-      "X-CSRF-Token": await session(),
-      ...(options.headers || {}),
-    },
-  });
-}
-
 async function request(path, { signal, ...options } = {}) {
   let response;
   try {
-    response = await send(path, { signal, ...options });
+    response = await fetch(BASE + path, {
-    // A restarted server issues a new session; re-bootstrap once rather than
+      headers: { "Content-Type": "application/json" },
-    // stranding an open tab on 401.
+      signal,
-    if (response.status === 401) {
+      ...options,
-      csrfToken = null;
+    });
-      response = await send(path, { signal, ...options });
-    }
   } catch (error) {
     // A caller-cancelled fetch is not a failure; tag it so views can ignore it.
     if (error.name === "AbortError") {
@@ -167,31 +138,4 @@ export const api = {
     }),
   uploadVerifications: (id, opts = {}) =>
     request(`/upload-batches/${encodeURIComponent(id)}/verifications`, opts),
-
-  // ── Archive and restore: destinations, preflight, plans, recovery ────────
-  archiveLocations: (opts = {}) => request("/archive-locations", opts),
-  registerArchiveLocation: (payload, opts = {}) =>
-    request("/archive-locations", { method: "POST", body: JSON.stringify(payload), ...opts }),
-  archivePreflight: (payload, opts = {}) =>
-    request("/archive-preflight", { method: "POST", body: JSON.stringify(payload), ...opts }),
-  createArchivePlan: (payload, opts = {}) =>
-    request("/archive-plans", { method: "POST", body: JSON.stringify(payload), ...opts }),
-  listArchivePlans: (opts = {}) => request("/archive-plans", opts),
-  getArchivePlan: (id, opts = {}) => request(`/archive-plans/${encodeURIComponent(id)}`, opts),
-  applyArchivePlan: (id, opts = {}) =>
-    request(`/archive-plans/${encodeURIComponent(id)}/apply`, { method: "POST", ...opts }),
-  archiveRecovery: (opts = {}) => request("/archive-recovery", opts),
-  resolveArchiveRecovery: (opts = {}) =>
-    request("/archive-recovery/resolve", { method: "POST", ...opts }),
-  restorePreflight: (payload, opts = {}) =>
-    request("/restore-preflight", { method: "POST", body: JSON.stringify(payload), ...opts }),
-  createRestorePlan: (payload, opts = {}) =>
-    request("/restore-plans", { method: "POST", body: JSON.stringify(payload), ...opts }),
-  listRestorePlans: (opts = {}) => request("/restore-plans", opts),
-  getRestorePlan: (id, opts = {}) => request(`/restore-plans/${encodeURIComponent(id)}`, opts),
-  applyRestorePlan: (id, opts = {}) =>
-    request(`/restore-plans/${encodeURIComponent(id)}/apply`, { method: "POST", ...opts }),
-  restoreRecovery: (opts = {}) => request("/restore-recovery", opts),
-  resolveRestoreRecovery: (opts = {}) =>
-    request("/restore-recovery/resolve", { method: "POST", ...opts }),
 };

frontend/js/app.js

@@ -1,5 +1,4 @@
 import { api } from "./api.js";
-import { renderArchive, setArchiveRender } from "./archive.js";
 import { navigate, onRouteChange, parseHash } from "./router.js";
 import { renderRenames, setRenamesRender } from "./renames.js";
 import { renderUploads, setUploadsRender } from "./uploads.js";
@@ -367,7 +366,6 @@ function render() {
   else if (path === "/albums") renderAlbums(root, params);
   else if (path === "/renames") renderRenames(root, params);
   else if (path === "/uploads") renderUploads(root, params);
-  else if (path === "/archive") renderArchive(root, params);
   else if (path === "/stats") renderStats(root, params);
   else show(errorBanner("Unknown view"));
 }
@@ -376,6 +374,5 @@ function render() {
 setRender(render);
 setRenamesRender(render);
 setUploadsRender(render);
-setArchiveRender(render);
 onRouteChange(render);
 render();

frontend/js/archive.js

@@ -1,870 +0,0 @@
-// Archive view (US06-05): preview what would leave active storage, confirm it
-// exactly, watch the transfer, recover an interrupted one, browse what is already
-// archived, and bring it back.
-//
-// Archiving is the only stage that removes originals, so this view never decides
-// anything itself: the destination's identity, every blocker, the confirmation
-// token, and what recovery may do all come from the server, and an action the
-// server would refuse is not offered. Two things follow from that. A medium that
-// is not mounted produces an instruction naming it rather than a disabled mystery,
-// and an operation whose evidence is ambiguous offers no button at all.
-import { api } from "./api.js";
-import { el, errorBanner, setActiveNav } from "./dom.js";
-import { subscribeJob } from "./events.js";
-import { navigate } from "./router.js";
-
-let outcome = null;
-let activity = [];
-
-let render = () => {};
-export function setArchiveRender(fn) {
-  render = fn;
-}
-
-// The per-file journal states, split into the three things an operator actually
-// wants told apart: bytes moving, bytes proven, original removed (concept §9).
-const PHASES = [
-  ["planned", "planned", "waiting"],
-  ["transferring", "transfer", "copying to the medium"],
-  ["verified", "verified", "archive copy hashed and manifested"],
-  ["removing", "removing", "removing the active original"],
-  ["complete", "complete", "archived and removed"],
-  ["failed", "failed", "left alone for a decision"],
-];
-
-const AVAILABILITY_LABEL = {
-  active: "in the library",
-  archived_online: "archived · medium mounted",
-  archived_offline: "archived · medium away",
-  missing_unexpected: "missing — unexplained",
-};
-
-export async function renderArchive(root, params = {}) {
-  setActiveNav("archive");
-
-  let locations;
-  try {
-    locations = (await api.archiveLocations()).locations;
-  } catch (error) {
-    root.replaceChildren(errorBanner(`Failed to load archive locations: ${error.message}`));
-    return;
-  }
-  const location = locations.find((l) => l.id === params.location) || locations[0] || null;
-
-  const nodes = [el("h1", {}, "Archive"), locationsCard(locations, location, params)];
-  if (!location) {
-    nodes.push(
-      el(
-        "p",
-        { class: "muted", "data-testid": "no-locations" },
-        "Register the disk, NAS share, or removable medium that will hold archived originals."
-      ),
-      outcomeBanner()
-    );
-    root.replaceChildren(...nodes.filter(Boolean));
-    return;
-  }
-
-  const [preflight, archivePlans, restorePlans, recovery, restoreRecovery, archived, restore] =
-    await Promise.all([
-      load(() => api.archivePreflight({ location_id: location.id })),
-      load(() => api.listArchivePlans()),
-      load(() => api.listRestorePlans()),
-      load(() => api.archiveRecovery()),
-      load(() => api.restoreRecovery()),
-      load(() => api.listAssets({ limit: 200 })),
-      load(() => api.restorePreflight({ location_id: location.id })),
-    ]);
-
-  // Both directions share the runs table: one lane moves these originals, so one
-  // history is what an operator has to reason about.
-  const runs = [...plans(archivePlans), ...plans(restorePlans)].sort((a, b) =>
-    (a.created_at || "").localeCompare(b.created_at || "")
-  );
-  const chosen = runs.find((run) => run.id === params.plan) || runs[runs.length - 1] || null;
-  const selectedPlan = chosen ? await load(() => planDetailOf(chosen)) : null;
-
-  nodes.push(
-    preflight ? previewSection(preflight, location) : null,
-    preflight ? confirmBlock(preflight, location) : null,
-    outcomeBanner(),
-    activityLog(),
-    recoverySection(mergeRecovery(recovery, restoreRecovery)),
-    planList(runs, chosen && chosen.id),
-    selectedPlan ? planDetail(selectedPlan) : null,
-    archived ? archivedSection(archived.items, locations) : null,
-    restore ? restoreSection(restore, location) : null
-  );
-  root.replaceChildren(...nodes.filter(Boolean));
-}
-
-function plans(listed) {
-  return listed ? listed.plans : [];
-}
-
-function planDetailOf(run) {
-  return run.direction === "restore" ? api.getRestorePlan(run.id) : api.getArchivePlan(run.id);
-}
-
-// Archive and restore recovery answer the same question about the same lane, so
-// they are one list; an unresolved item of either kind blocks the other.
-function mergeRecovery(archive, restore) {
-  if (!archive && !restore) return null;
-  return {
-    operations: [...((archive || {}).operations || []), ...((restore || {}).operations || [])],
-    manual: [...((archive || {}).manual || []), ...((restore || {}).manual || [])],
-  };
-}
-
-// A section whose data failed to load must not take the rest of the view with it:
-// the medium being away is exactly when the archived-asset list matters most.
-async function load(call) {
-  try {
-    return await call();
-  } catch (_) {
-    return null;
-  }
-}
-
-// ── locations ────────────────────────────────────────────────────────────────
-// A location is a medium, not a path: the marker's ``media_id`` is what proves the
-// right disk is mounted, so it is shown next to the state it produced.
-function locationsCard(locations, selected, params) {
-  const name = el("input", {
-    type: "text",
-    "data-testid": "location-name",
-    "aria-label": "Archive location name",
-    placeholder: "External disk",
-  });
-  const root = el("input", {
-    type: "text",
-    "data-testid": "location-root",
-    "aria-label": "Archive location path",
-    placeholder: "/Volumes/archive",
-  });
-
-  return el(
-    "div",
-    { class: "card", "data-testid": "archive-locations" },
-    el("h2", {}, "Destinations"),
-    locations.length
-      ? el(
-          "table",
-          { class: "grid", "data-testid": "locations" },
-          el(
-            "thead",
-            {},
-            el(
-              "tr",
-              {},
-              ...["", "Name", "Root", "Medium", "State", "Last seen"].map((label) =>
-                el("th", { scope: "col" }, label)
-              )
-            )
-          ),
-          el(
-            "tbody",
-            {},
-            ...locations.map((location) =>
-              el(
-                "tr",
-                {
-                  "data-testid": "location-row",
-                  "data-name": location.name,
-                  "aria-current": selected && location.id === selected.id ? "true" : false,
-                },
-                el(
-                  "td",
-                  {},
-                  el("input", {
-                    type: "radio",
-                    name: "archive-location",
-                    "data-testid": "select-location",
-                    "aria-label": `Use ${location.name}`,
-                    checked: selected && location.id === selected.id ? "checked" : false,
-                    onchange: () => navigate("/archive", { ...params, location: location.id }),
-                  })
-                ),
-                el("td", { "data-testid": "location-label" }, location.name),
-                el("td", { class: "path", "data-testid": "location-path" }, location.root),
-                el("td", { class: "path", "data-testid": "location-media" }, location.media_id),
-                el(
-                  "td",
-                  {},
-                  el(
-                    "span",
-                    {
-                      class: `badge ${location.state === "online" ? "complete" : "attention"}`,
-                      "data-testid": "location-state",
-                    },
-                    location.state
-                  )
-                ),
-                el("td", { class: "muted" }, location.last_seen_at || "never")
-              )
-            )
-          )
-        )
-      : null,
-    selected && selected.state !== "online" ? mountInstruction(selected) : null,
-    el(
-      "div",
-      { class: "toolbar" },
-      name,
-      root,
-      el(
-        "button",
-        {
-          "data-testid": "register-location",
-          onclick: () =>
-            run(() => api.registerArchiveLocation({ name: name.value, root: root.value })),
-        },
-        "Register destination"
-      )
-    )
-  );
-}
-
-// The one thing the app cannot do for the user: name the medium to connect.
-function mountInstruction(location) {
-  const detail =
-    location.state === "wrong_volume"
-      ? `A different medium is mounted at ${location.root}.`
-      : `Nothing is mounted at ${location.root}.`;
-  return el(
-    "div",
-    { class: "confirm", role: "status", "data-testid": "mount-instruction" },
-    `${detail} Connect “${location.name}” (medium ${location.media_id}) and mount it there, ` +
-      "then reload this view. Archived photos stay listed and searchable meanwhile."
-  );
-}
-
-// ── preview ──────────────────────────────────────────────────────────────────
-function previewSection(preflight, location) {
-  const totals = preflight.totals;
-  const capacity = preflight.capacity;
-  const rows = preflight.albums.map((album) =>
-    el(
-      "tr",
-      { "data-testid": "archive-album-row", "data-album": album.album },
-      el("td", { "data-testid": "album-name" }, album.album),
-      el("td", { class: "path", "data-testid": "album-folder" }, album.folder),
-      el("td", { class: "path", "data-testid": "album-destination" }, album.destination),
-      el(
-        "td",
-        { "data-testid": "album-method" },
-        album.transfer_method === "move" ? "move (same filesystem)" : "copy · verify · remove"
-      ),
-      el("td", { "data-testid": "album-assets" }, String(album.asset_count)),
-      el("td", { "data-testid": "album-reclaim" }, bytes(album.reclaimable_bytes)),
-      el(
-        "td",
-        {},
-        el("span", { class: `badge ${album.state}`, "data-testid": "album-state" }, album.state),
-        ...album.blockers.map((blocker) =>
-          el(
-            "div",
-            { class: "blocker", "data-testid": "album-blocker", "data-code": blocker.code },
-            blocker.message
-          )
-        )
-      )
-    )
-  );
-
-  return el(
-    "div",
-    { "data-testid": "archive-preview" },
-    el("h2", {}, "Preview"),
-    el(
-      "div",
-      { class: "decision-bar" },
-      el(
-        "span",
-        { class: "badge", "data-testid": "destination-identity" },
-        `${location.name} · ${location.media_id}`
-      ),
-      el("span", { class: "badge", "data-testid": "total-albums" }, `${totals.albums} album(s)`),
-      el("span", { class: "badge", "data-testid": "total-assets" }, `${totals.assets} photo(s)`),
-      el(
-        "span",
-        { class: "badge", "data-testid": "total-reclaim" },
-        `${bytes(totals.bytes)} reclaimable`
-      ),
-      el(
-        "span",
-        {
-          class: `badge ${capacity.sufficient ? "complete" : "blocked"}`,
-          "data-testid": "capacity",
-        },
-        `free ${bytes(capacity.free_bytes)} · reserve ${bytes(capacity.reserve_bytes)}`
-      )
-    ),
-    blockerList(preflight.blockers, "preflight-blockers", "preflight-blocker", "This scope cannot be archived yet"),
-    rows.length
-      ? el(
-          "table",
-          { class: "grid", "data-testid": "archive-albums" },
-          el(
-            "thead",
-            {},
-            el(
-              "tr",
-              {},
-              ...["Album", "Folder", "Destination", "Transfer", "Photos", "Reclaims", "State"].map(
-                (label) => el("th", { scope: "col" }, label)
-              )
-            )
-          ),
-          el("tbody", {}, ...rows)
-        )
-      : el(
-          "p",
-          { class: "muted", "data-testid": "no-albums" },
-          "No album has a verified upload whose bytes are still unchanged, so nothing may be archived."
-        )
-  );
-}
-
-function blockerList(blockers, containerId, itemId, title) {
-  if (!blockers || !blockers.length) return null;
-  return el(
-    "div",
-    { class: "alert", role: "alert", "data-testid": containerId },
-    el("strong", {}, title),
-    el(
-      "ul",
-      {},
-      ...blockers.map((blocker) =>
-        el("li", { "data-testid": itemId, "data-code": blocker.code }, `${blocker.code}: ${blocker.message}`)
-      )
-    )
-  );
-}
-
-// ── confirmation ─────────────────────────────────────────────────────────────
-function confirmBlock(preflight, location) {
-  const ready = preflight.state === "ready";
-  const totals = preflight.totals;
-  return el(
-    "div",
-    { class: "card", "data-testid": "confirm" },
-    el("h2", {}, "Confirm"),
-    el(
-      "p",
-      { class: "muted", "data-testid": "confirm-token" },
-      `Preflight ${preflight.token.slice(0, 20)}… · ${location.name}`
-    ),
-    el(
-      "p",
-      { "data-testid": "archive-note" },
-      "Archiving removes each original from the library — but only after its copy on " +
-        "the medium has been written, hashed, and recorded in the manifest. The photos " +
-        "stay searchable and deduplicable while the medium is away, and can be restored " +
-        "from this view."
-    ),
-    el(
-      "div",
-      { class: "toolbar" },
-      el(
-        "button",
-        {
-          class: "primary",
-          "data-testid": "start-archive",
-          disabled: ready ? false : "disabled",
-          title: ready ? false : "resolve the blockers above first",
-          onclick: () =>
-            run(async () => {
-              const plan = await api.createArchivePlan({
-                location_id: location.id,
-                token: preflight.token,
-              });
-              const started = await api.applyArchivePlan(plan.id);
-              watch(started.job.id, plan.id, "archive");
-              return { archiving: plan.asset_count };
-            }),
-        },
-        `Archive ${totals.ready_albums} album(s) · reclaim ${bytes(totals.bytes)}`
-      )
-    )
-  );
-}
-
-// ── plans and progress ───────────────────────────────────────────────────────
-function planList(runs, selectedId) {
-  if (!runs.length) {
-    return el("p", { class: "muted", "data-testid": "no-plans" }, "Nothing has been archived yet.");
-  }
-  return el(
-    "div",
-    { "data-testid": "archive-plans" },
-    el("h2", {}, "Runs"),
-    el(
-      "table",
-      { class: "grid", "data-testid": "plans" },
-      el(
-        "thead",
-        {},
-        el(
-          "tr",
-          {},
-          ...["Created", "Direction", "State", "Photos", "Bytes"].map((l) =>
-            el("th", { scope: "col" }, l)
-          )
-        )
-      ),
-      el(
-        "tbody",
-        {},
-        ...runs.map((plan) =>
-          el(
-            "tr",
-            {
-              "data-testid": "plan-row",
-              "data-plan": plan.id,
-              "data-direction": plan.direction,
-              "aria-current": plan.id === selectedId ? "true" : false,
-            },
-            el(
-              "td",
-              {},
-              el("a", { class: "link", href: `#/archive?plan=${encodeURIComponent(plan.id)}` }, plan.created_at || plan.id)
-            ),
-            el("td", { "data-testid": "plan-direction" }, plan.direction),
-            el("td", {}, el("span", { class: `badge ${plan.state}`, "data-testid": "plan-state" }, plan.state)),
-            el("td", {}, String(plan.asset_count)),
-            el("td", {}, bytes(plan.byte_size))
-          )
-        )
-      )
-    )
-  );
-}
-
-function planDetail(plan) {
-  const operations = plan.operations || [];
-  const counts = {};
-  for (const operation of operations) {
-    counts[operation.journal_state] = (counts[operation.journal_state] || 0) + 1;
-  }
-  return el(
-    "div",
-    { class: "card", "data-testid": "plan-detail", "data-plan": plan.id },
-    el("h2", {}, `${plan.direction === "restore" ? "Restore" : "Archive"} run ${plan.created_at || plan.id}`),
-    // Transfer, verification, and removal are separate answers to separate
-    // questions: what has moved, what is proven, and what is already gone.
-    el(
-      "div",
-      { class: "decision-bar", "data-testid": "plan-progress" },
-      el("span", { class: `badge ${plan.state}`, "data-testid": "detail-state" }, plan.state),
-      ...PHASES.map(([key, label, title]) =>
-        el(
-          "span",
-          { class: `badge ${key}`, "data-testid": `count-${label}`, title },
-          `${label}: ${counts[key] ?? 0}`
-        )
-      )
-    ),
-    el(
-      "table",
-      { class: "grid", "data-testid": "operations" },
-      el(
-        "thead",
-        {},
-        el(
-          "tr",
-          {},
-          ...["Photo", "Destination", "Phase", "Attempts", "Problem"].map((l) =>
-            el("th", { scope: "col" }, l)
-          )
-        )
-      ),
-      el(
-        "tbody",
-        {},
-        ...operations.map((operation) =>
-          el(
-            "tr",
-            { "data-testid": "operation-row", "data-asset-id": operation.asset_id },
-            el("td", { class: "path", "data-testid": "operation-source" }, operation.source_path),
-            el("td", { class: "path", "data-testid": "operation-destination" }, operation.destination_path),
-            el(
-              "td",
-              {},
-              el(
-                "span",
-                { class: `badge ${operation.journal_state}`, "data-testid": "operation-phase" },
-                phaseLabel(operation.journal_state)
-              )
-            ),
-            el("td", {}, String(operation.attempt_count)),
-            el(
-              "td",
-              { class: "muted", "data-testid": "operation-error", "data-code": operation.error_code || "" },
-              operation.error_code ? `${operation.error_code}: ${operation.error_message || ""}` : "—"
-            )
-          )
-        )
-      )
-    )
-  );
-}
-
-function phaseLabel(state) {
-  const found = PHASES.find(([key]) => key === state);
-  return found ? found[1] : state;
-}
-
-// ── recovery ─────────────────────────────────────────────────────────────────
-// What an interrupted run left behind, straight from the journal plus the files on
-// disk. Only the operations the server itself classified as resolvable get an
-// action; ambiguous ones are shown with their evidence and no button.
-function recoverySection(recovery) {
-  const operations = recovery ? recovery.operations : [];
-  if (!operations.length) return null;
-  const manual = recovery.manual || [];
-  const resolvable = operations.length - manual.length;
-
-  return el(
-    "div",
-    { class: "card", "data-testid": "archive-recovery" },
-    el("h2", {}, "Interrupted work"),
-    el(
-      "p",
-      { "data-testid": "recovery-summary" },
-      `${operations.length} operation(s) did not finish · ${resolvable} resolvable · ` +
-        `${manual.length} need a decision`
-    ),
-    el(
-      "table",
-      { class: "grid", "data-testid": "recovery-operations" },
-      el(
-        "thead",
-        {},
-        el(
-          "tr",
-          {},
-          ...["Photo", "Phase", "Verdict", "Evidence"].map((l) => el("th", { scope: "col" }, l))
-        )
-      ),
-      el(
-        "tbody",
-        {},
-        ...operations.map((verdict) =>
-          el(
-            "tr",
-            {
-              "data-testid": "recovery-row",
-              "data-classification": verdict.classification,
-              "data-direction": verdict.direction,
-            },
-            el("td", { class: "path", "data-testid": "recovery-source" }, verdict.source_path),
-            el("td", {}, el("span", { class: "badge" }, phaseLabel(verdict.journal_state))),
-            el(
-              "td",
-              {},
-              el(
-                "span",
-                {
-                  class: `badge ${verdict.classification === "manual" ? "blocked" : "attention"}`,
-                  "data-testid": "recovery-verdict",
-                },
-                verdict.classification
-              )
-            ),
-            el(
-              "td",
-              { class: "muted", "data-testid": "recovery-reason" },
-              `${verdict.reason} (source ${verdict.source_exists ? "present" : "absent"}, ` +
-                `archive copy ${verdict.destination_matches ? "verified" : verdict.destination_exists ? "different bytes" : "absent"})`
-            )
-          )
-        )
-      )
-    ),
-    manual.length
-      ? el(
-          "div",
-          { class: "alert", role: "alert", "data-testid": "recovery-manual" },
-          `${manual.length} operation(s) cannot be resolved from the evidence. Nothing will be ` +
-            "removed or retried for them here: inspect the medium and the library, then decide."
-        )
-      : null,
-    el(
-      "div",
-      { class: "toolbar" },
-      resolvable
-        ? el(
-            "button",
-            {
-              class: "primary",
-              "data-testid": "resolve-recovery",
-              onclick: () => run(() => api.resolveArchiveRecovery()),
-            },
-            `Finish ${resolvable} recoverable operation(s)`
-          )
-        : el(
-            "span",
-            { class: "muted", "data-testid": "no-safe-recovery" },
-            "No operation can be finished safely from here."
-          )
-    )
-  );
-}
-
-// ── archived assets ──────────────────────────────────────────────────────────
-// Browsing what is already archived, including while the medium is away: the
-// retained protected preview and the recorded hashes are the evidence, so the row
-// stays complete and honest instead of turning into a missing file.
-function archivedSection(assets, locations) {
-  const archived = assets.filter((asset) => asset.availability_state !== "active");
-  if (!archived.length) return null;
-  const names = Object.fromEntries(locations.map((location) => [location.id, location.name]));
-
-  return el(
-    "div",
-    { "data-testid": "archived-assets" },
-    el("h2", {}, "Archived photos"),
-    el(
-      "table",
-      { class: "grid", "data-testid": "archived" },
-      el(
-        "thead",
-        {},
-        el(
-          "tr",
-          {},
-          ...["Preview", "Archived as", "Medium", "Availability", "Size"].map((l) =>
-            el("th", { scope: "col" }, l)
-          )
-        )
-      ),
-      el(
-        "tbody",
-        {},
-        ...archived.map((asset) =>
-          el(
-            "tr",
-            { "data-testid": "archived-row", "data-asset-id": asset.id },
-            el(
-              "td",
-              {},
-              el("img", {
-                "data-testid": "archived-preview",
-                width: 96,
-                loading: "lazy",
-                src: api.thumbnailUrl(asset.id, 256),
-                alt: `Preview of ${asset.archive_path || asset.id}`,
-              })
-            ),
-            el("td", { class: "path", "data-testid": "archived-path" }, asset.archive_path || "—"),
-            el(
-              "td",
-              { "data-testid": "archived-medium" },
-              names[asset.archive_location_id] || asset.archive_location_id || "—"
-            ),
-            el(
-              "td",
-              {},
-              el(
-                "span",
-                {
-                  class: `badge ${asset.availability_state === "archived_online" ? "complete" : "attention"}`,
-                  "data-testid": "archived-availability",
-                },
-                AVAILABILITY_LABEL[asset.availability_state] || asset.availability_state
-              )
-            ),
-            el("td", {}, bytes(asset.byte_size))
-          )
-        )
-      )
-    )
-  );
-}
-
-// ── restore ──────────────────────────────────────────────────────────────────
-function restoreSection(restore, location) {
-  const ready = restore.state === "ready";
-  const items = restore.items || [];
-  if (!items.length && !restore.blockers.length) return null;
-
-  return el(
-    "div",
-    { class: "card", "data-testid": "restore" },
-    el("h2", {}, "Restore"),
-    el(
-      "p",
-      { "data-testid": "restore-note" },
-      "Restoring copies the archived bytes back into the library and leaves the archive " +
-        "copy where it is. A name that is already taken is never overwritten: the photo " +
-        "comes back beside it under a visibly different name."
-    ),
-    blockerList(restore.blockers, "restore-blockers", "restore-blocker", "This restore cannot run yet"),
-    items.length
-      ? el(
-          "table",
-          { class: "grid", "data-testid": "restore-items" },
-          el(
-            "thead",
-            {},
-            el(
-              "tr",
-              {},
-              ...["Archived as", "Comes back as", "Size", "State"].map((l) =>
-                el("th", { scope: "col" }, l)
-              )
-            )
-          ),
-          el(
-            "tbody",
-            {},
-            ...items.map((item) =>
-              el(
-                "tr",
-                { "data-testid": "restore-row", "data-asset-id": item.asset_id },
-                el("td", { class: "path", "data-testid": "restore-source" }, item.archive_path),
-                el("td", { class: "path", "data-testid": "restore-destination" }, item.destination_path || "—"),
-                el("td", {}, bytes(item.byte_size)),
-                el(
-                  "td",
-                  {},
-                  item.blockers.length
-                    ? el(
-                        "span",
-                        {
-                          class: "badge blocked",
-                          "data-testid": "restore-item-blocker",
-                          "data-code": item.blockers[0].code,
-                        },
-                        item.blockers[0].code
-                      )
-                    : el("span", { class: "badge ready", "data-testid": "restore-item-state" }, "ready")
-                )
-              )
-            )
-          )
-        )
-      : null,
-    el(
-      "div",
-      { class: "toolbar" },
-      el(
-        "button",
-        {
-          class: "primary",
-          "data-testid": "start-restore",
-          disabled: ready ? false : "disabled",
-          title: ready ? false : "the medium and every archived copy must check out first",
-          onclick: () =>
-            run(async () => {
-              const plan = await api.createRestorePlan({
-                location_id: location.id,
-                token: restore.token,
-              });
-              const started = await api.applyRestorePlan(plan.id);
-              watch(started.job.id, plan.id, "restore");
-              return { restoring: plan.asset_count };
-            }),
-        },
-        `Restore ${items.length} photo(s) from ${location.name}`
-      )
-    )
-  );
-}
-
-// ── running commands ─────────────────────────────────────────────────────────
-async function run(action) {
-  try {
-    outcome = { kind: "ok", result: await action() };
-  } catch (error) {
-    outcome = error.status === 409 ? { kind: "conflict", error } : { kind: "error", error };
-  }
-  render();
-}
-
-// Live job activity. The plan panel is refreshed on its own tick because the
-// journal advances per file, not per job event; a full re-render would re-run
-// preflight (which re-hashes the library), so that happens once when the job ends.
-const REFRESH_MS = 500;
-
-function watch(jobId, planId, kind) {
-  activity = [`Started ${kind} job ${jobId}`];
-  const tick = setInterval(() => refreshPlan(planId, kind), REFRESH_MS);
-  subscribeJob(jobId, {
-    onEvent: (event) => {
-      activity.push(`${event.type}${event.message ? ": " + event.message : ""}`);
-      const log = document.querySelector('[data-testid="archive-activity"]');
-      if (log) log.textContent = activity.join("\n");
-    },
-    onDone: () => {
-      clearInterval(tick);
-      activity.push("done");
-      render();
-    },
-  });
-}
-
-async function refreshPlan(planId, kind) {
-  const node = document.querySelector(`[data-testid="plan-detail"][data-plan="${planId}"]`);
-  if (!node) return; // the user navigated away from the running plan
-  try {
-    const plan = await (kind === "restore" ? api.getRestorePlan(planId) : api.getArchivePlan(planId));
-    node.replaceWith(planDetail(plan));
-  } catch (_) {
-    // Transient; the next tick tries again and the job's end re-renders anyway.
-  }
-}
-
-function activityLog() {
-  return el(
-    "pre",
-    {
-      class: "activity-log",
-      role: "status",
-      "aria-live": "polite",
-      "data-testid": "archive-activity",
-    },
-    activity.join("\n")
-  );
-}
-
-function outcomeBanner() {
-  if (!outcome) return null;
-  if (outcome.kind === "conflict") {
-    return el(
-      "div",
-      { class: "alert", role: "alert", "data-testid": "conflict" },
-      `The server refused this: ${outcome.error.message}. Nothing was moved or removed; ` +
-        "the state below is the server's current one — review it and decide again."
-    );
-  }
-  if (outcome.kind === "error") {
-    return el(
-      "div",
-      { class: "alert", role: "alert", "data-testid": "archive-error" },
-      `Failed: ${outcome.error.message}`
-    );
-  }
-  const result = outcome.result || {};
-  const message =
-    result.archiving !== undefined
-      ? `Archiving ${result.archiving} photo(s). Originals are removed only after their copies verify.`
-      : result.restoring !== undefined
-        ? `Restoring ${result.restoring} photo(s) into the library.`
-        : "Done — the state below is the server's.";
-  return el("div", { class: "alert", role: "status", "data-testid": "archive-result" }, message);
-}
-
-// ── formatting ───────────────────────────────────────────────────────────────
-function bytes(value) {
-  if (value == null) return "unknown";
-  const units = ["B", "kB", "MB", "GB", "TB"];
-  let size = value;
-  let unit = 0;
-  while (size >= 1000 && unit < units.length - 1) {
-    size /= 1000;
-    unit += 1;
-  }
-  return `${unit === 0 ? size : size.toFixed(1)} ${units[unit]}`;
-}

frontend/js/tests/unit.js

@@ -29,10 +29,6 @@ function jsonResponse(status, body) {
 const tick = (ms = 0) => new Promise((r) => setTimeout(r, ms));
 
 async function run() {
-  // The client fetches its CSRF token once, lazily (US07-02). Do that against the
-  // real server first, so the stubbed fetch below only ever sees the call under test.
-  await api.workflow().catch(() => {});
-
   // ── store ────────────────────────────────────────────────────────────────
   {
     const store = createStore({ n: 0 });

legacy_cli_archive/CHECKSUMS.sha256

@@ -1,22 +0,0 @@
-d098bbde13d2ebc872ca781e244cc1e48b97cc551fedaca3251d6a5278a15234  src/compare_models.py
-7d0f68cb95fbd6603e4c558620098b46929ebdb6fd91a598bbf84f26eea35e6c  src/nsfw_tag.py
-35329c53570e215cef15f59429c1a5251a0448b90522b0910ab9d68efe1bc307  src/nsfwtag/__init__.py
-7f52b55e4f3b77eda3657d7cd2272c7cca8422100e241671de495603c1126ee2  src/nsfwtag/__main__.py
-6f5a97114e0d87d272ce22d065dc31fa0dd72ff8590d11ad14cb1c1f486d339c  src/nsfwtag/bench.py
-29498fac1d73ba2b7420ffe1ffe49a87f684ec299e2c3b9c6e14e63e95643ba4  src/nsfwtag/exif.py
-934e82c402813ebf503e5a20eb03d5103f84df13117bc4dd1fd95deaa01ab263  src/nsfwtag/README.md
-68125e6184c7e4d2a5b0153f2155675753ab0b1329d1dbc4933d4769492ec4be  src/nsfwtag/review.html
-a597eab74803dd31452b5d70abf7d6d6320eb17a61596c891b009da15d624e98  src/nsfwtag/scoring.py
-674969a18e58ee511a2abf574a875db92f2f524769b1b614d0d3662ab879a709  src/nsfwtag/server.py
-24a7c8d029da6e97d46b110a9fe9dbb3900127f8ae3c142f634fdd91d6c45243  src/nsfwtag/webapp.py
-2c2ea558f1b9095c1150f078ea29c8e0f180cc045a9d1097b83f61bfb145295b  src/photo_analyzer.py
-5fb7ce2f977a17da1f501c45d7985328318ae14298a3f347eab46a6e3f02aab3  src/test_dedup.py
-1673dc76cc60aba56a5b065d9bc9f5dda00342cf2a79d000a3f74686fb7cffec  src/test_nsfw_skip.py
-55aa82f348e0e90be0163ba6aa278e5fc8e445996e5335774917e5ef59ad3563  src/webapp/__init__.py
-ba4342bd0175591a2121f063f14a78a602d5679bcfcf24aa14c8d4b8cd5da1a2  src/webapp/__main__.py
-a60f24035a989909467778f888d62854c1afc2702f47b8d3cca32bf3cf04d5cf  src/webapp/analyzer.html
-018bcc2f770d716444b456db58d6b4a41800138a98598b3c27ba503ab22d939e  src/webapp/page.py
-5f599b107b2b117ca118b6fbdec5e5786a9ab4eea424aa9af1bfea7bf87cebab  src/webapp/query.py
-875cce697caa02717c85a2707ba935c46dd9ace18428fd4abb13105b7a8e24d9  src/webapp/README.md
-af7d0d72d245b4bbb1e0e30f9708239697543212d4171defd1bfe26dd24a4f05  src/webapp/runner.py
-cddf0555b06fe7e4daa14309baaaf91130f8506922cb0ffb6507d76cbe39add4  src/webapp/server.py

legacy_cli_archive/README.md

@@ -1,80 +0,0 @@
-# Legacy CLI archive (US07-01)
-
-Frozen, read-only sources of the command-line tools this application was extracted
-from. They are **reference material and rollback evidence** — provenance for
-behavior that now lives in `photo_pipeline/`, and the only way to answer "what did
-the original actually do?" once the replacement has drifted.
-
-> **Nothing here is production code.** No module under `photo_pipeline/` imports or
-> executes anything in this directory, and this directory is not on the application's
-> import path. `tests/unit/test_legacy_archive.py` enforces both, along with the
-> checksums and the redaction below.
-
-## What is here
-
-| Path | Role |
-|---|---|
-| `src/photo_analyzer.py` | the analysis CLI: discovery, hashing/dedup, vision analysis, EXIF writing, SQLite schema, album naming |
-| `src/nsfwtag/` | NSFW scoring, EXIF safety keywords, and the review server (`__init__` 1.1.0) |
-| `src/webapp/` | the stdlib review web app: FTS search, stats, subprocess runner, HTML shell (`__init__` 0.1.0) |
-| `src/nsfw_tag.py` | thin backwards-compatible entry point for `nsfwtag` |
-| `src/compare_models.py` | dev-only model comparison script |
-| `src/test_dedup.py`, `src/test_nsfw_skip.py` | the CLIs' own standalone self-checks (never pytest suites) |
-| `donor_ledger.yaml` | the donor ledger: every migrated behavior, its target, its tests, and every intentional delta |
-| `requirements-lock.txt` | the dependency versions the frozen sources were last verified against |
-| `photo_analyzer.env.sample` | the CLI's configuration surface, with every value replaced by a placeholder |
-| `CHECKSUMS.sha256` | SHA-256 of every archived source file |
-
-`photo_analyzer.py` carries no `__version__`; its identity is its checksum, recorded
-in `CHECKSUMS.sha256` and taken at commit `9b7ee6b` (the merge of US06-06, the last
-commit before archival).
-
-## Verifying the archive
-
-```bash
-cd legacy_cli_archive && shasum -a 256 -c CHECKSUMS.sha256
-```
-
-Any edit to an archived source must be accompanied by a regenerated checksum file
-and a note here explaining why a *frozen* archive changed — the normal answer being
-that it should not.
-
-## Schema notes
-
-`photo_analyzer.py` owned a path-keyed SQLite database (`SCHEMA`, near the top of the
-file):
-
-- `photos(id, path UNIQUE, status, phash, file_sha1, dup_of, description, tags,
-  people_count, setting, time_of_day, season, mood, location_hint, approx_year,
-  raw_response, error_message, analyzed_at, exif_written_at)`;
-- `photos_fts` — an FTS5 external-content index over `path, description, tags, mood,
-  location_hint`, kept in sync by insert/update/delete triggers;
-- late columns (`phash`, `file_sha1`, `dup_of`) were added by an in-code
-  `_migrate_schema()` rather than a migration tool, and their indexes are created
-  only after the `ALTER`.
-
-The replacement keeps the same analysis fields but re-keys everything to a stable
-`assets.id` (Alembic migrations `0001`…), because a path is not an identity: the
-donor's `path UNIQUE` is exactly what broke on every move and rename.
-
-`nsfwtag` kept its safety scores outside the database in `nsfw_scores.csv`
-(`path,nsfw_score`, four decimals, unreadable rows dropped). That file is no longer a
-source of truth; `photo_pipeline/services/legacy_import.py` imports it into
-`assets.safety_score` and reports exactly what matched, what did not, and why.
-
-## Redaction
-
-The archive contains no credentials. `photo_analyzer.env.sample` documents the
-configuration surface (`LLM_API_KEY`, `LLM_BASE_URL`, `LLM_MODEL`, and the tuning
-variables) with placeholder values only; the CLI itself never contained a key, it
-read one from `photo_analyzer.env` or the environment. No `.env`, database, log, CSV,
-or photo from the author's library is archived.
-
-## Why these tools were retired
-
-Each behavior's fate is recorded per row in `donor_ledger.yaml`: `reuse`, `extract`,
-`refactor`, or `replace`, with the target module, the characterization tests that
-pinned the donor's behavior, the parity tests the replacement passes, and — where the
-replacement deliberately does something else or nothing at all — a `delta` saying so.
-Rows still marked `pending` name the backlog story that will resolve them; they are
-the honest list of what has *not* been carried over yet.

legacy_cli_archive/photo_analyzer.env.sample

@@ -1,18 +0,0 @@
-# Sample configuration for the archived photo_analyzer.py CLI (US07-01).
-#
-# REDACTED: no value below is real. Each line names a variable the CLI read and
-# what belongs there; the placeholders are deliberately not key-shaped, so this
-# file can never be mistaken for — or scanned as — a credential.
-
-LLM_API_KEY=<paste your provider key here>
-LLM_BASE_URL=<provider base url, e.g. the OpenAI-compatible Gemini endpoint>
-LLM_MODEL=<model id, e.g. a Gemini Flash release>
-
-# Optional tuning the CLI read from the same file:
-PHASH_THRESHOLD=8
-MAX_WORKERS=4
-RETRY_ATTEMPTS=3
-RPD_LIMIT=0
-
-# The CLI ignored its own shipped placeholder (a literal "sk-REPLACE..." string)
-# until it was replaced, and a shell variable always won over this file.

legacy_cli_archive/requirements-lock.txt

@@ -1,20 +0,0 @@
-# Final dependency lock of the archived CLIs (US07-01).
-#
-# These are the versions present in the environment the archive was taken from —
-# what the frozen sources were last verified against by the characterization
-# suite. Restoring a donor for forensics means pinning these, not "latest".
-#
-# Python 3.14.6
-
-openai==3.0.0          # photo_analyzer: OpenAI-compatible vision client
-numpy==2.4.6           # photo_analyzer, nsfwtag: pixel work
-Pillow==12.3.0         # photo_analyzer, nsfwtag: decode/resize
-rich==15.0.0           # photo_analyzer: console output
-scipy==1.18.0          # photo_analyzer: perceptual-hash DCT
-PyYAML==6.0.3          # tooling that reads the donor ledger
-
-# NSFW inference (nsfwtag/scoring.py, nsfwtag/bench.py) was never installed in the
-# archiving environment; the model stack is recorded here from the sources so a
-# forensic run can reproduce it, not from a resolved lock:
-#   torch, transformers, timm  — AdamCodd/vit-base-nsfw-detector (see nsfwtag/README.md)
-# exiftool is an external binary, not a Python package.

migrations/versions/0012_archive_plans.py

@@ -1,96 +0,0 @@
-"""Archive plans, per-item transfer journal, and archived asset location (US06-02).
-
-Revision ID: 0012_archive_plans
-Revises: 0011_archive_locations
-Create Date: 2026-08-16
-
-The journal is what makes removing an original recoverable: every item records its
-source, destination, expected hash, and the state it had reached before the process
-died. Assets gain the archive location and relative path so an offline original is
-still explained rather than looking missing.
-"""
-
-import sqlalchemy as sa
-from alembic import op
-
-revision = "0012_archive_plans"
-down_revision = "0011_archive_locations"
-branch_labels = None
-depends_on = None
-
-
-def upgrade() -> None:
-    op.create_table(
-        "archive_plans",
-        sa.Column("id", sa.String(), primary_key=True),
-        sa.Column(
-            "location_id",
-            sa.String(),
-            sa.ForeignKey("archive_locations.id"),
-            nullable=False,
-            index=True,
-        ),
-        # The preflight token this plan was approved against.
-        sa.Column("token", sa.String(), nullable=False),
-        sa.Column("albums", sa.String(), nullable=True),  # JSON array
-        # planned | applying | complete | failed
-        sa.Column("state", sa.String(), nullable=False, server_default="planned"),
-        sa.Column("schema_version", sa.Integer(), nullable=False, server_default="1"),
-        sa.Column("asset_count", sa.Integer(), nullable=False, server_default="0"),
-        sa.Column("byte_size", sa.Integer(), nullable=False, server_default="0"),
-        # Bumped on every claim and used as the fencing token.
-        sa.Column("version", sa.Integer(), nullable=False, server_default="1"),
-        sa.Column("worker_id", sa.String(), nullable=True),
-        sa.Column("completed_at", sa.DateTime(timezone=True), nullable=True),
-        sa.Column(
-            "created_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()
-        ),
-        sa.Column(
-            "updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()
-        ),
-    )
-    op.create_table(
-        "archive_operations",
-        sa.Column("id", sa.String(), primary_key=True),
-        sa.Column(
-            "plan_id",
-            sa.String(),
-            sa.ForeignKey("archive_plans.id", ondelete="CASCADE"),
-            nullable=False,
-            index=True,
-        ),
-        sa.Column("sequence", sa.Integer(), nullable=False),
-        sa.Column("album", sa.String(), nullable=False),
-        sa.Column("asset_id", sa.String(), sa.ForeignKey("assets.id"), nullable=False, index=True),
-        sa.Column("source_path", sa.String(), nullable=False),
-        sa.Column("destination_path", sa.String(), nullable=False),
-        # Relative to the location root: the medium can be mounted anywhere later.
-        sa.Column("archive_path", sa.String(), nullable=False),
-        sa.Column("expected_sha256", sa.String(), nullable=False),
-        sa.Column("byte_size", sa.Integer(), nullable=True),
-        sa.Column("same_filesystem", sa.Boolean(), nullable=True),
-        # planned | transferring | verified | removing | complete | failed
-        sa.Column("journal_state", sa.String(), nullable=False, server_default="planned"),
-        sa.Column("attempt_count", sa.Integer(), nullable=False, server_default="0"),
-        sa.Column("fencing_token", sa.Integer(), nullable=True),
-        sa.Column("worker_id", sa.String(), nullable=True),
-        sa.Column("verified_at", sa.DateTime(timezone=True), nullable=True),
-        sa.Column("removed_at", sa.DateTime(timezone=True), nullable=True),
-        sa.Column("error_code", sa.String(), nullable=True),
-        sa.Column("error_message", sa.String(), nullable=True),
-        sa.Column(
-            "updated_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()
-        ),
-        sa.UniqueConstraint("plan_id", "sequence", name="uq_archive_operations_plan_sequence"),
-    )
-    # Plain columns: SQLite cannot ALTER a table to add a foreign key, and the
-    # relationship is enforced by the service that writes them.
-    op.add_column("assets", sa.Column("archive_location_id", sa.String(), nullable=True))
-    op.add_column("assets", sa.Column("archive_path", sa.String(), nullable=True))
-
-
-def downgrade() -> None:
-    op.drop_column("assets", "archive_path")
-    op.drop_column("assets", "archive_location_id")
-    op.drop_table("archive_operations")
-    op.drop_table("archive_plans")

migrations/versions/0013_protected_thumbnails.py

@@ -1,29 +0,0 @@
-"""Protected thumbnails (US06-03).
-
-Revision ID: 0013_protected_thumbnails
-Revises: 0012_archive_plans
-Create Date: 2026-08-16
-
-A protected thumbnail is the durable comparison preview of an asset whose
-original has left active storage. It is evidence rather than cache, so the LRU
-quota must not evict it: the archive medium may be offline when it is needed.
-"""
-
-import sqlalchemy as sa
-from alembic import op
-
-revision = "0013_protected_thumbnails"
-down_revision = "0012_archive_plans"
-branch_labels = None
-depends_on = None
-
-
-def upgrade() -> None:
-    op.add_column(
-        "thumbnails",
-        sa.Column("protected", sa.Boolean(), nullable=False, server_default=sa.false()),
-    )
-
-
-def downgrade() -> None:
-    op.drop_column("thumbnails", "protected")

migrations/versions/0014_restore_plans.py

@@ -1,37 +0,0 @@
-"""Restore plans and archive divergence (US06-04).
-
-Revision ID: 0014_restore_plans
-Revises: 0013_protected_thumbnails
-Create Date: 2026-08-16
-
-Restore reuses the archive plan and journal tables: the crash-safe question is the
-same one in the opposite direction (copy, verify, publish, register), so the rows
-gain a ``direction`` instead of a parallel pair of tables. ``archive_divergent_at``
-records the moment an archived copy was proven to hold bytes that are not the ones
-the database recorded — a restore must never silently accept a different file.
-"""
-
-import sqlalchemy as sa
-from alembic import op
-
-revision = "0014_restore_plans"
-down_revision = "0013_protected_thumbnails"
-branch_labels = None
-depends_on = None
-
-
-def upgrade() -> None:
-    for table in ("archive_plans", "archive_operations"):
-        op.add_column(
-            table,
-            sa.Column("direction", sa.String(), nullable=False, server_default="archive"),
-        )
-    op.add_column(
-        "assets", sa.Column("archive_divergent_at", sa.DateTime(timezone=True), nullable=True)
-    )
-
-
-def downgrade() -> None:
-    op.drop_column("assets", "archive_divergent_at")
-    for table in ("archive_plans", "archive_operations"):
-        op.drop_column(table, "direction")

migrations/versions/0015_exif_projections.py

@@ -1,42 +0,0 @@
-"""Durable EXIF projections per asset and stage (US07-03).
-
-Revision ID: 0015_exif_projections
-Revises: 0014_restore_plans
-Create Date: 2026-08-16
-
-The concept's ``exif_projections`` table, added at the point it earns its keep: a
-checkpoint that finds a field it does not own changed must be able to say so after
-a restart. ``state`` is verified | divergent | failed, and only ``verified`` counts
-as a completed metadata stage.
-"""
-
-import sqlalchemy as sa
-from alembic import op
-
-revision = "0015_exif_projections"
-down_revision = "0014_restore_plans"
-branch_labels = None
-depends_on = None
-
-
-def upgrade() -> None:
-    op.create_table(
-        "exif_projections",
-        sa.Column("asset_id", sa.String(), sa.ForeignKey("assets.id"), primary_key=True),
-        sa.Column("stage", sa.String(), primary_key=True),  # safety | analysis
-        sa.Column("id", sa.String(), nullable=False),
-        sa.Column("projection_version", sa.Integer(), nullable=False, server_default="0"),
-        # What the stage asked for: {"add": [...], "remove": [...]}.
-        sa.Column("desired_json", sa.String(), nullable=True),
-        # Fields outside the stage's ownership that did not survive the write.
-        sa.Column("divergent_fields", sa.String(), nullable=True),
-        sa.Column("result_file_sha256", sa.String(), nullable=True),
-        sa.Column("state", sa.String(), nullable=False),
-        sa.Column("error_code", sa.String(), nullable=True),
-        sa.Column("verified_at", sa.DateTime(timezone=True), nullable=True),
-        sa.Column("updated_at", sa.DateTime(timezone=True), nullable=True),
-    )
-
-
-def downgrade() -> None:
-    op.drop_table("exif_projections")

photo_pipeline/__main__.py

@@ -1,4 +1,4 @@
-"""Application management CLI: ``python -m photo_pipeline {serve,migrate,worker,import-legacy-scores}``."""
+"""Application management CLI: ``python -m photo_pipeline {serve,migrate}``."""
 
 from __future__ import annotations
 
@@ -16,17 +16,6 @@ def main(argv: Sequence[str] | None = None) -> int:
     commands.add_parser("migrate", help="Upgrade the database to the latest revision")
     worker_cmd = commands.add_parser("worker", help="Run a durable-job worker")
     worker_cmd.add_argument("--id", default="worker-1", help="Worker id (lease owner)")
-    import_cmd = commands.add_parser(
-        "import-legacy-scores",
-        help="Import the archived CLI's nsfw_scores.csv into the database (US07-01)",
-    )
-    import_cmd.add_argument("csv", help="Path to nsfw_scores.csv")
-    import_cmd.add_argument(
-        "--overwrite", action="store_true", help="Replace differing imported scores"
-    )
-    import_cmd.add_argument(
-        "--dry-run", action="store_true", help="Report what would happen and change nothing"
-    )
     args = parser.parse_args(argv)
 
     config = Config.from_env()
@@ -36,24 +25,6 @@ def main(argv: Sequence[str] | None = None) -> int:
         run_migrations(config.database_url)
         return 0
 
-    if args.command == "import-legacy-scores":
-        import json
-
-        from photo_pipeline.db import create_db_engine, create_session_factory
-        from photo_pipeline.services.legacy_import import LegacyImportService, write_report
-
-        run_migrations(config.database_url)
-        engine = create_db_engine(config.database_url)
-        service = LegacyImportService(create_session_factory(engine))
-        report = service.import_nsfw_scores(
-            args.csv, overwrite=args.overwrite, dry_run=args.dry_run
-        )
-        # The report is the point: an import nobody can audit is not a migration.
-        if not args.dry_run:
-            write_report(report, config.data_dir)
-        print(json.dumps(report.counts, indent=2))
-        return 0
-
     if args.command == "worker":
         from photo_pipeline.db import create_db_engine, create_session_factory
 

photo_pipeline/api/app.py

@@ -8,15 +8,11 @@ and exposes the versioned ``/api/v1`` surface; US01-02 ships only health.
 
 from __future__ import annotations
 
-import logging
 from contextlib import asynccontextmanager
 from pathlib import Path
 
-from fastapi import FastAPI, Request
+from fastapi import FastAPI
-from fastapi.exceptions import RequestValidationError
-from fastapi.responses import JSONResponse
 from fastapi.staticfiles import StaticFiles
-from starlette.exceptions import HTTPException as StarletteHTTPException
 
 from photo_pipeline.api.routes import (
     albums,
@@ -29,58 +25,20 @@ from photo_pipeline.api.routes import (
     library,
     renames,
     safety,
-    session as session_routes,
     thumbnails,
     uploads,
     workflow,
 )
-from photo_pipeline.api.security import DEFAULT_HEADERS, SecurityMiddleware, Session
 
 # Registers the safety_score / analysis job handlers on import.
 import photo_pipeline.jobs.domain_handlers  # noqa: F401
 from photo_pipeline.config import Config
 from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
 from photo_pipeline.logging import configure_logging
-from photo_pipeline.services.thumbnails import ThumbnailService
 from photo_pipeline.services.upload_batches import UploadBatchService
 
 FRONTEND_DIR = Path(__file__).resolve().parents[2] / "frontend"
 
-log = logging.getLogger(__name__)
-
-
-def _envelope(status: int, code: str, message: str) -> JSONResponse:
-    return JSONResponse(
-        status_code=status,
-        content={"error": {"code": code, "message": message}},
-        headers=DEFAULT_HEADERS,
-    )
-
-
-def _install_error_handlers(app: FastAPI) -> None:
-    """One JSON error envelope everywhere, and nothing behind it.
-
-    An unhandled exception carries the library's absolute paths, SQL, and sometimes
-    a credential in its text; the client gets a code, the operator gets the traceback
-    in the server log (US07-02).
-    """
-
-    @app.exception_handler(StarletteHTTPException)
-    async def _http_error(request: Request, exc: StarletteHTTPException):
-        return _envelope(exc.status_code, "http_error", str(exc.detail))
-
-    @app.exception_handler(RequestValidationError)
-    async def _validation_error(request: Request, exc: RequestValidationError):
-        # Field locations only: the echoed input can be the caller's own data, but it
-        # is also what ends up in shared logs and screenshots.
-        fields = sorted(".".join(str(part) for part in error["loc"]) for error in exc.errors())
-        return _envelope(422, "invalid_request", f"invalid request fields: {', '.join(fields)}")
-
-    @app.exception_handler(Exception)
-    async def _unhandled(request: Request, exc: Exception):
-        log.exception("unhandled error serving %s", request.url.path)
-        return _envelope(500, "internal_error", "internal error")
-
 
 def create_app(config: Config | None = None) -> FastAPI:
     config = config or Config.from_env()
@@ -97,9 +55,6 @@ def create_app(config: Config | None = None) -> FastAPI:
         # An upload whose process died left no outcome behind; resolve it now so the
         # uploader lane is free and the uncertain batch is visible (US05-02).
         UploadBatchService(app.state.session_factory, config=config).recover()
-        # A render killed mid-write leaves its temporary beside the cache entry;
-        # remove those recognized leftovers, and only those (US07-03).
-        ThumbnailService(app.state.session_factory, config).cleanup_temp_files()
         try:
             yield
         finally:
@@ -107,12 +62,6 @@ def create_app(config: Config | None = None) -> FastAPI:
             app.state.engine = None
 
     app = FastAPI(title="Photo Pipeline", version="0.1.0", lifespan=lifespan)
-    # One session per process: the browser exchanges it for a cookie + CSRF token,
-    # and every other origin is refused before a route ever runs (US07-02).
-    app.state.session = Session.create()
-    app.add_middleware(SecurityMiddleware, session=app.state.session, config=config)
-    _install_error_handlers(app)
-    app.include_router(session_routes.router, prefix="/api/v1")
     app.include_router(health.router, prefix="/api/v1")
     app.include_router(inventory.router, prefix="/api/v1")
     app.include_router(duplicates.router, prefix="/api/v1")

photo_pipeline/api/routes/analysis.py

@@ -18,10 +18,7 @@ router = APIRouter(tags=["analysis"])
 
 
 def _service(request: Request) -> AnalysisService:
-    return AnalysisService(
+    return AnalysisService(request.app.state.session_factory)
-        request.app.state.session_factory,
-        library_roots=tuple(request.app.state.config.library_roots),
-    )
 
 
 def _error(status: int, code: str, message: str) -> JSONResponse:

photo_pipeline/api/routes/archives.py

@@ -1,10 +1,9 @@
-"""Archive location, preflight, plan, and restore API (US06-01, US06-02, US06-04).
+"""Archive location and preflight API (US06-01).
 
 Registering a location writes a marker onto the medium; preflight is a command
 rather than a read, because it probes the destination, hashes the scope, and issues
-the token an archive plan must present. Creating a plan writes only database rows —
+the token a later archive plan must present (US06-02). Neither endpoint moves or
-the transfer itself runs on the durable ``archive`` lane, never in the request
+removes a single library file.
-thread, because it removes originals.
 """
 
 from __future__ import annotations
@@ -13,18 +12,12 @@ from fastapi import APIRouter, Request
 from fastapi.responses import JSONResponse
 from pydantic import BaseModel
 
-from photo_pipeline.jobs.domain_handlers import ARCHIVE_LOCK, ARCHIVE_PLAN, RESTORE_PLAN
 from photo_pipeline.services.archives import ArchiveError, ArchiveService
-from photo_pipeline.services.archive_transfer import ArchiveTransferService
-from photo_pipeline.services.jobs import JobBlocked, JobService
-from photo_pipeline.services.restores import RestoreService
 
 router = APIRouter(tags=["archives"])
 
-# Which failures are the caller's request (422), a missing thing (404), or state
+# Which failures are the caller's request (422) and which are a missing thing (404).
-# that changed under the caller (409).
+NOT_FOUND_CODES = {"unknown_location"}
-NOT_FOUND_CODES = {"unknown_location", "unknown_plan"}
-CONFLICT_CODES = {"stale_token", "stale_plan", "archive_pending"}
 
 
 class RegisterLocationRequest(BaseModel):
@@ -38,42 +31,12 @@ class PreflightRequest(BaseModel):
     albums: list[str] | None = None
 
 
-class CreatePlanRequest(PreflightRequest):
-    # The token of the preflight the user approved; a stale one is refused.
-    token: str
-
-
-class RestoreRequest(BaseModel):
-    location_id: str
-    # ``None`` means every asset archived at this location.
-    asset_ids: list[str] | None = None
-
-
-class CreateRestoreRequest(RestoreRequest):
-    token: str
-
-
 def _service(request: Request) -> ArchiveService:
     return ArchiveService(request.app.state.session_factory, config=request.app.state.config)
 
 
-def _restores(request: Request) -> RestoreService:
-    return RestoreService(request.app.state.session_factory, config=request.app.state.config)
-
-
-def _transfers(request: Request) -> ArchiveTransferService:
-    return ArchiveTransferService(
-        request.app.state.session_factory, config=request.app.state.config
-    )
-
-
 def _error(error: ArchiveError) -> JSONResponse:
-    if error.code in NOT_FOUND_CODES:
+    status = 404 if error.code in NOT_FOUND_CODES else 422
-        status = 404
-    elif error.code in CONFLICT_CODES:
-        status = 409
-    else:
-        status = 422
     return JSONResponse(
         status_code=status, content={"error": {"code": error.code, "message": str(error)}}
     )
@@ -98,137 +61,3 @@ def preflight(body: PreflightRequest, request: Request):
         return _service(request).preflight(body.location_id, body.albums)
     except ArchiveError as error:
         return _error(error)
-
-
-@router.post("/archive-plans", status_code=201)
-def create_plan(body: CreatePlanRequest, request: Request):
-    """Turn an approved preflight into a durable, journaled plan. Nothing moves."""
-    try:
-        return _transfers(request).create(body.location_id, body.albums, token=body.token)
-    except ArchiveError as error:
-        return _error(error)
-
-
-@router.get("/archive-plans")
-def list_plans(request: Request) -> dict:
-    return {"plans": _transfers(request).list()}
-
-
-@router.get("/archive-plans/{plan_id}")
-def get_plan(plan_id: str, request: Request):
-    plan = _transfers(request).get(plan_id)
-    if plan is None:
-        return _error(ArchiveError("unknown_plan", f"unknown archive plan {plan_id}"))
-    return plan
-
-
-@router.post("/archive-plans/{plan_id}/apply")
-def apply_plan(plan_id: str, request: Request):
-    """Queue the transfer on the archiver lane. The worker removes the sources."""
-    service = _transfers(request)
-    plan = service.get(plan_id)
-    if plan is None:
-        return _error(ArchiveError("unknown_plan", f"unknown archive plan {plan_id}"))
-    if service.journal.blocks_mutation():
-        unresolved = [row for row in service.journal.incomplete() if row["plan_id"] != plan_id]
-        if unresolved:
-            return _error(
-                ArchiveError(
-                    "archive_pending",
-                    f"an unresolved archive operation ({unresolved[0]['id']}) must be recovered",
-                )
-            )
-    try:
-        job = JobService(request.app.state.session_factory).enqueue(
-            ARCHIVE_PLAN,
-            lock=ARCHIVE_LOCK,
-            # One queued attempt per plan version: a double-clicked apply reuses it.
-            idempotency_key=f"archive:{plan_id}:{plan['version']}",
-            items=[plan_id],
-        )
-    except JobBlocked as error:
-        return JSONResponse(
-            status_code=409, content={"error": {"code": error.code, "message": str(error)}}
-        )
-    return {"plan_id": plan_id, "job": job}
-
-
-@router.post("/restore-preflight")
-def restore_preflight(body: RestoreRequest, request: Request):
-    """Validate restoring archived assets back into the library. Nothing moves."""
-    try:
-        return _restores(request).preflight(body.location_id, body.asset_ids)
-    except ArchiveError as error:
-        return _error(error)
-
-
-@router.post("/restore-plans", status_code=201)
-def create_restore_plan(body: CreateRestoreRequest, request: Request):
-    try:
-        return _restores(request).create(body.location_id, body.asset_ids, token=body.token)
-    except ArchiveError as error:
-        return _error(error)
-
-
-@router.get("/restore-plans")
-def list_restore_plans(request: Request) -> dict:
-    return {"plans": _restores(request).list()}
-
-
-@router.get("/restore-plans/{plan_id}")
-def get_restore_plan(plan_id: str, request: Request):
-    plan = _restores(request).get(plan_id)
-    if plan is None:
-        return _error(ArchiveError("unknown_plan", f"unknown restore plan {plan_id}"))
-    return plan
-
-
-@router.post("/restore-plans/{plan_id}/apply")
-def apply_restore_plan(plan_id: str, request: Request):
-    """Queue the restore on the archiver lane — the same single lane as archiving,
-    because both move the same originals."""
-    service = _restores(request)
-    plan = service.get(plan_id)
-    if plan is None:
-        return _error(ArchiveError("unknown_plan", f"unknown restore plan {plan_id}"))
-    unresolved = [row for row in service.journal.incomplete() if row["plan_id"] != plan_id]
-    if unresolved:
-        return _error(
-            ArchiveError(
-                "archive_pending",
-                f"an unresolved archive operation ({unresolved[0]['id']}) must be recovered",
-            )
-        )
-    try:
-        job = JobService(request.app.state.session_factory).enqueue(
-            RESTORE_PLAN,
-            lock=ARCHIVE_LOCK,
-            idempotency_key=f"restore:{plan_id}:{plan['version']}",
-            items=[plan_id],
-        )
-    except JobBlocked as error:
-        return JSONResponse(
-            status_code=409, content={"error": {"code": error.code, "message": str(error)}}
-        )
-    return {"plan_id": plan_id, "job": job}
-
-
-@router.get("/restore-recovery")
-def restore_recovery_status(request: Request) -> dict:
-    return _restores(request).recovery_status()
-
-
-@router.post("/restore-recovery/resolve")
-def resolve_restore_recovery(request: Request) -> dict:
-    return _restores(request).recover()
-
-
-@router.get("/archive-recovery")
-def recovery_status(request: Request) -> dict:
-    """What an interrupted transfer left behind, straight from journal + disk."""
-    return _transfers(request).recovery_status()
-
-
-@router.post("/archive-recovery/resolve")
-def resolve_recovery(request: Request) -> dict:
-    return _transfers(request).recover()

photo_pipeline/api/routes/session.py

@@ -1,30 +0,0 @@
-"""Session bootstrap: the one endpoint reachable without a session (US07-02).
-
-It sets the ``HttpOnly``/``SameSite=Strict`` session cookie and returns the CSRF
-token in the body. A foreign page can call this — it just cannot read the answer,
-because the app sends no CORS headers — and the cookie it received is never attached
-to a request that foreign page initiates.
-"""
-
-from __future__ import annotations
-
-from fastapi import APIRouter, Request
-from fastapi.responses import JSONResponse
-
-from photo_pipeline.api.security import SESSION_COOKIE
-
-router = APIRouter(tags=["session"])
-
-
-@router.get("/session")
-def start_session(request: Request) -> JSONResponse:
-    session = request.app.state.session
-    response = JSONResponse({"csrf_token": session.csrf_token})
-    response.set_cookie(
-        SESSION_COOKIE,
-        session.id,
-        httponly=True,
-        samesite="strict",
-        path="/",
-    )
-    return response

photo_pipeline/api/routes/thumbnails.py

@@ -31,7 +31,5 @@ def get_thumbnail(asset_id: str, request: Request, size: int = Query(512)):
     return FileResponse(
         path,
         media_type="image/webp",
-        # private: the URL is versioned and immutable, but these bytes are the user's
+        headers={"Cache-Control": "public, max-age=31536000, immutable"},
-        # photos and must never sit in a shared cache (US07-02).
-        headers={"Cache-Control": "private, max-age=31536000, immutable"},
     )

photo_pipeline/api/security.py

@@ -1,192 +0,0 @@
-"""Local-web attack surface: session, CSRF, Origin/Host checks, default headers.
-
-The app binds to 127.0.0.1, so the attacker is not a remote client but another page
-in the user's browser (concept §15, "Local web attack"): any site can issue requests
-to ``http://127.0.0.1:8000`` and can embed ``<img src=...>`` against media endpoints.
-The defenses stack, because each one alone has a hole:
-
-* **Host** must be a loopback name — a DNS rebinding host that resolves to 127.0.0.1
-  passes the browser's origin rules but not this check.
-* **Origin**, when the browser sends one, must be this exact origin (scheme, host,
-  port). There is no CORS middleware at all, so a foreign page can never *read* a
-  response even if it manages to send a request.
-* **Sec-Fetch-Site** rejects cross-site loads that carry no Origin, which is what an
-  ``<img>`` or ``<script>`` against a media endpoint looks like.
-* A **session cookie** (``SameSite=Strict``, ``HttpOnly``) is required by every
-  ``/api/v1`` route except liveness/readiness and the bootstrap itself. Strict means
-  the browser never attaches it to a request another site initiated.
-* A **CSRF token** must be echoed in a header on every mutation. It is handed out
-  only in the bootstrap response body, which a foreign page cannot read (no CORS) —
-  so possessing it proves the caller is same-origin.
-
-``evaluate`` is a pure function over the request metadata: the whole policy is one
-table that a unit test can enumerate, and the middleware only applies its verdict.
-"""
-
-from __future__ import annotations
-
-import secrets
-from collections.abc import Mapping
-from dataclasses import dataclass
-from urllib.parse import urlsplit
-
-from starlette.datastructures import Headers, MutableHeaders
-from starlette.responses import JSONResponse
-
-SESSION_COOKIE = "pp_session"
-CSRF_HEADER = "x-csrf-token"
-API_PREFIX = "/api/v1"
-SAFE_METHODS = frozenset({"GET", "HEAD", "OPTIONS"})
-# Reachable without a session: liveness/readiness (an orchestrator has no cookie)
-# and the bootstrap that issues the session in the first place.
-PUBLIC_PATHS = frozenset(
-    {f"{API_PREFIX}/health/live", f"{API_PREFIX}/health/ready", f"{API_PREFIX}/session"}
-)
-LOOPBACK_HOSTS = frozenset({"127.0.0.1", "localhost", "::1", "[::1]"})
-
-# Applied to every response. No inline script/style is used by the frontend, so the
-# policy can stay strict; `frame-ancestors 'none'` and CORP keep other pages from
-# embedding the app or its thumbnails.
-DEFAULT_HEADERS = {
-    "x-content-type-options": "nosniff",
-    "x-frame-options": "DENY",
-    "referrer-policy": "no-referrer",
-    "cross-origin-resource-policy": "same-origin",
-    "cross-origin-opener-policy": "same-origin",
-    "content-security-policy": (
-        "default-src 'self'; img-src 'self' data:; style-src 'self'; script-src 'self'; "
-        "connect-src 'self'; frame-ancestors 'none'; base-uri 'none'; form-action 'none'"
-    ),
-}
-
-
-@dataclass(frozen=True)
-class Session:
-    """One process, one session. A local app has exactly one user; a session store
-    would be bookkeeping without a second subject to distinguish.
-
-    ponytail: per-session rows if the app ever serves more than one operator.
-    """
-
-    id: str
-    csrf_token: str
-
-    @classmethod
-    def create(cls) -> Session:
-        return cls(secrets.token_urlsafe(32), secrets.token_urlsafe(32))
-
-
-@dataclass(frozen=True)
-class Refusal:
-    status: int
-    code: str
-    message: str
-
-
-def split_host(value: str) -> tuple[str, str]:
-    """``"127.0.0.1:8000"`` -> ``("127.0.0.1", "8000")``; bracketed IPv6 aware."""
-    value = value.strip()
-    if value.startswith("["):
-        host, _, port = value.partition("]")
-        return host + "]", port.lstrip(":")
-    host, _, port = value.partition(":")
-    return host, port
-
-
-def evaluate(
-    *,
-    method: str,
-    path: str,
-    headers: Mapping[str, str],
-    session: Session,
-    allowed_hosts: frozenset[str] = LOOPBACK_HOSTS,
-    max_request_bytes: int,
-) -> Refusal | None:
-    """Why this request must be refused, or ``None`` when it may proceed."""
-    host_header = headers.get("host", "")
-    host, port = split_host(host_header)
-    if host.lower() not in allowed_hosts:
-        return Refusal(403, "host_not_allowed", "request host is not a local address")
-
-    origin = headers.get("origin")
-    if origin is not None and origin != "":
-        parts = urlsplit(origin)
-        origin_host, origin_port = split_host(parts.netloc)
-        if (
-            parts.scheme not in ("http", "https")
-            or origin_host.lower() not in allowed_hosts
-            or origin_port != port
-        ):
-            return Refusal(403, "origin_not_allowed", "request origin is not this application")
-
-    # Absent means a non-browser client; "none" is a user-initiated navigation.
-    fetch_site = headers.get("sec-fetch-site")
-    if fetch_site is not None and fetch_site not in ("same-origin", "none"):
-        return Refusal(403, "cross_site_blocked", "cross-site requests are not accepted")
-
-    length = headers.get("content-length")
-    if length and length.isdigit() and int(length) > max_request_bytes:
-        return Refusal(413, "payload_too_large", "request body exceeds the configured limit")
-
-    protected = path.startswith(API_PREFIX) and path not in PUBLIC_PATHS
-    if not protected:
-        return None
-
-    if headers.get("cookie-session") != session.id:
-        return Refusal(401, "unauthenticated", "a valid application session is required")
-    if method.upper() not in SAFE_METHODS and headers.get(CSRF_HEADER) != session.csrf_token:
-        return Refusal(403, "csrf_failed", "missing or invalid CSRF token")
-    return None
-
-
-class SecurityMiddleware:
-    """Pure-ASGI so the SSE stream keeps streaming (BaseHTTPMiddleware buffers)."""
-
-    def __init__(self, app, *, session: Session, config) -> None:
-        self.app = app
-        self.session = session
-        self.max_request_bytes = config.max_request_bytes
-        self.allowed_hosts = frozenset(LOOPBACK_HOSTS | {str(config.host).lower()})
-
-    async def __call__(self, scope, receive, send) -> None:
-        if scope["type"] != "http":
-            await self.app(scope, receive, send)
-            return
-        headers = Headers(scope=scope)
-        # The cookie is read here and handed to the pure policy as one value, so the
-        # policy never has to parse a Cookie header.
-        lookup = dict(headers)
-        lookup["cookie-session"] = _cookie(headers.get("cookie", ""), SESSION_COOKIE)
-        refusal = evaluate(
-            method=scope.get("method", "GET"),
-            path=scope.get("path", "/"),
-            headers=lookup,
-            session=self.session,
-            allowed_hosts=self.allowed_hosts,
-            max_request_bytes=self.max_request_bytes,
-        )
-        if refusal is not None:
-            response = JSONResponse(
-                status_code=refusal.status,
-                content={"error": {"code": refusal.code, "message": refusal.message}},
-                headers=DEFAULT_HEADERS,
-            )
-            await response(scope, receive, send)
-            return
-
-        async def send_with_headers(message):
-            if message["type"] == "http.response.start":
-                out = MutableHeaders(scope=message)
-                for name, value in DEFAULT_HEADERS.items():
-                    out.setdefault(name, value)
-            await send(message)
-
-        await self.app(scope, receive, send_with_headers)
-
-
-def _cookie(header: str, name: str) -> str:
-    for part in header.split(";"):
-        key, _, value = part.strip().partition("=")
-        if key == name:
-            return value
-    return ""

photo_pipeline/config.py

@@ -30,10 +30,6 @@ class Config(BaseModel):
     log_level: str = "INFO"
     log_format: str = "json"  # "json" or "text"
 
-    # Largest request body the API accepts. Every endpoint takes small JSON commands;
-    # anything larger is a mistake or an attempt to exhaust memory (US07-02).
-    max_request_bytes: int = 1_048_576
-
     # Library boundary for path validation (os.pathsep-separated in the env var).
     library_roots: tuple[Path, ...] = ()
     thumbnail_cache_quota_bytes: int = 500_000_000

photo_pipeline/faults.py

@@ -1,45 +0,0 @@
-"""Test-only fault control points (concept §18, US07-04).
-
-Crash safety can only be proven by crashing at the exact moment a transition has
-been persisted but its consequence has not. That needs a barrier *inside* the
-production code path — but not a production capability: there is no endpoint, no
-service method, and no configuration file entry that can trigger one. The only
-switch is an environment variable naming a single point, read at the moment it is
-passed, and the only thing it does is kill the process. A deployment that never
-sets it can never reach the barrier.
-
-``os._exit`` is deliberate: it skips atexit handlers, buffered flushes, and
-``finally`` blocks, which is what a real ``SIGKILL`` or power loss does. A clean
-shutdown would prove nothing.
-
-The points are the persisted transitions of the journalled stages:
-
-    rename    moving | moved | database_updated | verified | complete
-    archive   transferring | verified | removing | source_removed | complete
-    exif      exif:written        — keywords on disk, checkpoint not yet recorded
-    upload    upload:accepted     — uploader exited, outcome not yet persisted
-    jobs      job:item_done       — item committed, job outcome not yet written
-
-Recovery for each is asserted in tests/integration/test_fault_matrix.py and
-tests/e2e/test_crash_recovery.py.
-"""
-
-from __future__ import annotations
-
-import os
-
-ENV_VAR = "PHOTO_PIPELINE_FAULT_AFTER"
-
-EXIF_WRITTEN = "exif:written"
-UPLOAD_ACCEPTED = "upload:accepted"
-JOB_ITEM_DONE = "job:item_done"
-
-
-def maybe_fault(point: str) -> None:
-    """Die abruptly when ``PHOTO_PIPELINE_FAULT_AFTER`` names ``point``.
-
-    Shared by the rename, archive, restore, EXIF, upload, and job lanes, each
-    passing its own state names. Never set the variable outside tests.
-    """
-    if os.environ.get(ENV_VAR) == point:
-        os._exit(9)

photo_pipeline/imaging.py

@@ -1,119 +0,0 @@
-"""Bounded, defensive image decoding — one door for every pixel this app reads.
-
-A photo library contains files nobody planned for: truncated downloads, zero-byte
-placeholders, a PNG whose header claims 200000×200000, a TIFF with a broken ICC
-profile, an extension that lies about its content. None of them may take down a
-request or a worker, and none may decode more pixels than the caller allowed
-(concept §17: decoded pixels, not file size, are what exhausts memory).
-
-``open_image`` is that single door:
-
-* the declared dimensions are checked **before** a pixel is decoded;
-* Pillow's decompression-bomb *warning* is promoted to an error, because the
-  warning band (between Pillow's limit and twice it) still decodes the image;
-* every decoder failure — at open time or during the caller's decode — becomes one
-  of two typed errors, so callers map them to their own item state instead of
-  catching ``Exception``;
-* error text names no path: it reaches API responses, and the full reason goes to
-  the server log instead (US07-02).
-
-``to_srgb`` and ``draft`` are the other two bounded-decode helpers: colour-manage a
-profile-bearing image into sRGB, and let JPEG decode straight to a size near the
-requested one rather than at full resolution.
-"""
-
-from __future__ import annotations
-
-import io
-import logging
-import warnings
-from contextlib import contextmanager
-from pathlib import Path
-
-from PIL import Image, ImageCms, ImageFile, UnidentifiedImageError
-
-log = logging.getLogger(__name__)
-
-# Matches ``Config.thumbnail_max_pixels``; used where no configuration is at hand
-# (hashing runs inside discovery, which takes no config).
-DEFAULT_MAX_PIXELS = 100_000_000
-
-
-class MediaError(Exception):
-    """A file could not be turned into pixels safely."""
-
-
-class UndecodableImage(MediaError):
-    """Corrupt, truncated, empty, or not an image at all."""
-
-
-class ImageTooLarge(MediaError):
-    """More pixels than this operation is allowed to decode."""
-
-
-@contextmanager
-def open_image(path: Path | str, *, max_pixels: int = DEFAULT_MAX_PIXELS):
-    """Yield an open :class:`PIL.Image.Image`, bounded and with typed failures.
-
-    Decoder errors raised inside the ``with`` body are translated too — a truncated
-    JPEG only fails when its pixels are actually pulled, which is the caller's line,
-    not this one.
-    """
-    # Pillow's truncation tolerance is a process-global switch that any library in
-    # the process can flip (the donor CLI did). This door decides the policy for its
-    # own callers: half a file is not a picture.
-    tolerated = ImageFile.LOAD_TRUNCATED_IMAGES
-    ImageFile.LOAD_TRUNCATED_IMAGES = False
-    with warnings.catch_warnings():
-        # The warning band is not a warning for us: it means Pillow was willing to
-        # decode an image large enough to be a denial-of-service.
-        warnings.simplefilter("error", Image.DecompressionBombWarning)
-        try:
-            with Image.open(path) as image:
-                width, height = image.size
-                if width * height > max_pixels:
-                    raise ImageTooLarge(f"{width}x{height} exceeds the {max_pixels} pixel limit")
-                yield image
-        except MediaError:
-            raise
-        except (Image.DecompressionBombError, Image.DecompressionBombWarning) as error:
-            log.info("refused oversized image %s: %s", path, error)
-            raise ImageTooLarge("image exceeds the decompression-bomb limit") from error
-        except (UnidentifiedImageError, OSError, ValueError, SyntaxError, MemoryError) as error:
-            log.info("cannot decode %s: %s", path, error)
-            raise UndecodableImage(f"cannot decode image ({type(error).__name__})") from error
-        finally:
-            ImageFile.LOAD_TRUNCATED_IMAGES = tolerated
-
-
-def draft(image: Image.Image, size: int) -> None:
-    """Ask the decoder for a smaller image where the format allows it (JPEG).
-
-    This is the difference between decoding a 40-megapixel JPEG and decoding the
-    roughly 1-megapixel version a 1280px preview needs.
-    """
-    try:
-        image.draft(None, (size, size))
-    except (AttributeError, ValueError, OSError):  # not a draft-capable format
-        pass
-
-
-def to_srgb(image: Image.Image, *, mode: str) -> Image.Image:
-    """Convert into ``mode``, colour-managing through an embedded ICC profile.
-
-    Without this a wide-gamut original renders with visibly wrong colours, because
-    its numbers are interpreted as sRGB. A broken or unreadable profile is not a
-    reason to fail a preview — the plain conversion is still a correct picture.
-    """
-    profile = image.info.get("icc_profile")
-    if profile:
-        try:
-            return ImageCms.profileToProfile(
-                image,
-                ImageCms.ImageCmsProfile(io.BytesIO(profile)),
-                ImageCms.createProfile("sRGB"),
-                outputMode=mode,
-            )
-        except Exception as error:  # noqa: BLE001 - any ICC failure falls back
-            log.info("ignoring unusable ICC profile on %s: %s", getattr(image, "filename", "?"), error)
-    return image.convert(mode)

photo_pipeline/integrations/exiftool.py

@@ -17,20 +17,6 @@ import os
 import subprocess
 from collections.abc import Iterable
 
-# A hung exiftool must not hang the worker with it: every call is bounded, and a
-# call that runs out of time is treated exactly like a failed one — no metadata
-# answer, nothing marked verified (US07-04). The knob exists because "slow" is a
-# property of the machine, not of the code: huge files on a slow network volume
-# legitimately take longer than the default.
-DEFAULT_TIMEOUT_SECONDS = 120.0
-
-
-def _timeout() -> float:
-    try:
-        return float(os.environ.get("PHOTO_PIPELINE_EXIFTOOL_TIMEOUT", DEFAULT_TIMEOUT_SECONDS))
-    except ValueError:
-        return DEFAULT_TIMEOUT_SECONDS
-
 
 def read_keyword_sets(paths: Iterable[str]) -> dict[str, set[str]]:
     """Map each path to its lowercased set of ``Keywords`` + ``Subject`` values.
@@ -48,9 +34,8 @@ def read_keyword_sets(paths: Iterable[str]) -> dict[str, set[str]]:
             input="\n".join(paths),
             capture_output=True,
             text=True,
-            timeout=_timeout(),
         )
-    except (FileNotFoundError, subprocess.TimeoutExpired):
+    except FileNotFoundError:
         return {}
     out: dict[str, set[str]] = {}
     try:
@@ -71,33 +56,6 @@ def read_keyword_sets(paths: Iterable[str]) -> dict[str, set[str]]:
     return out
 
 
-def read_all(path: str) -> dict | None:
-    """Every tag exiftool can read from ``path``, or ``None`` when it cannot answer.
-
-    This is the snapshot an EXIF checkpoint compares against: proving that a write
-    preserved the fields it does not own requires knowing all of them, not just the
-    ones being written (US07-03). ``None`` (exiftool missing, unreadable file,
-    unparsable output) is not an empty snapshot — a caller must not read it as
-    "nothing was there".
-    """
-    try:
-        result = subprocess.run(
-            ["exiftool", "-m", "-j", "-G0:1", path],
-            capture_output=True,
-            text=True,
-            timeout=_timeout(),
-        )
-    except (FileNotFoundError, subprocess.TimeoutExpired):
-        return None
-    try:
-        records = json.loads(result.stdout or "[]")
-    except ValueError:
-        return None
-    if not records:
-        return None
-    return {k: v for k, v in records[0].items() if k != "SourceFile"}
-
-
 def apply_keywords(path: str, *, add: Iterable[str] = (), remove: Iterable[str] = ()) -> bool:
     """Idempotently add/remove keywords in Keywords + Subject; preserve all else."""
     args = ["exiftool", "-m", "-overwrite_original"]
@@ -109,10 +67,4 @@ def apply_keywords(path: str, *, add: Iterable[str] = (), remove: Iterable[str]
     if len(args) == 3:
         return True
     args.append(path)
-    try:
+    return subprocess.run(args, capture_output=True, text=True).returncode == 0
-        return subprocess.run(
-            args, capture_output=True, text=True, timeout=_timeout()
-        ).returncode == 0
-    except (FileNotFoundError, subprocess.TimeoutExpired):
-        # A write that never returned is not a write that succeeded.
-        return False

photo_pipeline/integrations/nsfw_model.py

@@ -15,8 +15,6 @@ from __future__ import annotations
 
 from pathlib import Path
 
-from photo_pipeline import imaging
-
 MODEL_ID = "AdamCodd/vit-base-nsfw-detector"
 BATCH = 16
 
@@ -56,17 +54,13 @@ class NsfwModel:
         self._ensure_loaded()
         import numpy as np
         import torch
-        from PIL import Image
+        from PIL import Image, ImageFile
 
-        # The donor set ``ImageFile.LOAD_TRUNCATED_IMAGES = True`` here. That flag is
+        ImageFile.LOAD_TRUNCATED_IMAGES = True
-        # process-global: in this application the same process also hashes files and
+
-        # renders previews, and those must keep failing loudly on a truncated file
+        def preprocess(image):
-        # rather than quietly working on half of one (US07-03). An unreadable image
+            image = image.convert("RGB").resize((self._size, self._size), Image.BILINEAR)
-        # is skipped instead — it stays unscored, and therefore visibly undecided.
+            array = (np.asarray(image, dtype="float32") / 255.0 - 0.5) / 0.5
-        def preprocess(path):
-            with imaging.open_image(path) as image:
-                small = image.convert("RGB").resize((self._size, self._size), Image.BILINEAR)
-            array = (np.asarray(small, dtype="float32") / 255.0 - 0.5) / 0.5
             return torch.from_numpy(array).permute(2, 0, 1)
 
         results: list[tuple[str, float]] = []
@@ -75,10 +69,9 @@ class NsfwModel:
             tensors, batch_paths = [], []
             for path in items[start : start + self.batch]:
                 try:
-                    tensors.append(preprocess(path))
+                    tensors.append(preprocess(Image.open(path)))
                     batch_paths.append(path)
-                except (imaging.MediaError, OSError, ValueError):
+                except Exception:
-                    # One bad file must not cost the batch its other fifteen.
                     continue
             if not tensors:
                 continue

photo_pipeline/jobs/domain_handlers.py

@@ -1,12 +1,10 @@
-"""Domain job handlers: safety scoring, content analysis, uploads, archive
+"""Domain job handlers: safety scoring, content analysis, uploads (US02-06, US05-02).
-transfers, restores (US02-06, US05-02, US06-02, US06-04).
 
-Importing this module registers the ``safety_score``, ``analysis``,
+Importing this module registers the ``safety_score``, ``analysis``, and
-``upload_batch``, ``archive_plan``, and ``restore_plan`` job types so the generic
+``upload_batch`` job types so the generic worker can run them per item. Each handler
-worker can run them per item. Each handler delegates to its service, which owns the real work and the
+delegates to its service, which owns the real work and the privacy gate. Handlers
-privacy gate. Handlers are idempotent: re-scoring or re-analyzing one asset is safe
+are idempotent: re-scoring or re-analyzing one asset is safe after an interrupted
-after an interrupted attempt, an upload batch refuses to re-run an attempt whose
+attempt, and an upload batch refuses to re-run an attempt whose outcome is unknown.
-outcome is unknown, and an archive plan skips items it already completed.
 
 Providers/models are the service defaults here (real NsfwModel / vision provider);
 tests exercise the services directly with injected fakes rather than the worker.
@@ -19,13 +17,12 @@ from photo_pipeline.jobs.handlers import Cancelled, JobContext, register
 SAFETY_SCORE = "safety_score"
 ANALYSIS = "analysis"
 UPLOAD_BATCH = "upload_batch"
-ARCHIVE_PLAN = "archive_plan"
-RESTORE_PLAN = "restore_plan"
 # Both mutate the library's metadata/derived state; one at a time (concept §one job).
 LIBRARY_WRITE_LOCK = "library_write"
 # The uploader lane: one album batch at a time (concept §16).
 UPLOAD_LOCK = "upload"
-# The archiver lane: one archive/restore plan at a time (concept §16).
+# The archiver lane: one archive/restore plan at a time (concept §16). No handler
+# runs on it yet (US06-02); preflight already refuses to plan around a held lease.
 ARCHIVE_LOCK = "archive"
 
 
@@ -38,8 +35,7 @@ def _safety_score_item(asset_id: str, ctx: JobContext) -> None:
 def _analysis_item(asset_id: str, ctx: JobContext) -> None:
     from photo_pipeline.services.analysis import AnalysisService
 
-    roots = tuple(getattr(ctx.config, "library_roots", ()) or ())
+    AnalysisService(ctx.session_factory).run([asset_id])
-    AnalysisService(ctx.session_factory, library_roots=roots).run([asset_id])
 
 
 def _upload_batch_item(batch_id: str, ctx: JobContext) -> None:
@@ -58,36 +54,6 @@ def _upload_batch_item(batch_id: str, ctx: JobContext) -> None:
         raise RuntimeError(f"upload batch {batch_id} is {batch['state']}: {batch['error_code']}")
 
 
-def _archive_plan_item(plan_id: str, ctx: JobContext) -> None:
-    """One item = one archive plan. Item-level failures stay in the journal (the
-    source is then still there); only an unusable plan fails the job."""
-    from photo_pipeline.config import Config
-    from photo_pipeline.services.archive_transfer import ArchiveTransferService
-
-    config = ctx.config if ctx.config is not None else Config.from_env()
-    result = ArchiveTransferService(ctx.session_factory, config=config).apply(
-        plan_id, worker_id=ctx.worker_id
-    )
-    if result["failed"]:
-        raise RuntimeError(f"archive plan {plan_id}: {result['failed']} item(s) failed")
-
-
-def _restore_plan_item(plan_id: str, ctx: JobContext) -> None:
-    """One item = one restore plan. A restore removes nothing, so an item failure
-    simply leaves that asset archived (US06-04)."""
-    from photo_pipeline.config import Config
-    from photo_pipeline.services.restores import RestoreService
-
-    config = ctx.config if ctx.config is not None else Config.from_env()
-    result = RestoreService(ctx.session_factory, config=config).apply(
-        plan_id, worker_id=ctx.worker_id
-    )
-    if result["failed"]:
-        raise RuntimeError(f"restore plan {plan_id}: {result['failed']} item(s) failed")
-
-
 register(SAFETY_SCORE, _safety_score_item)
 register(ANALYSIS, _analysis_item)
 register(UPLOAD_BATCH, _upload_batch_item)
-register(ARCHIVE_PLAN, _archive_plan_item)
-register(RESTORE_PLAN, _restore_plan_item)

photo_pipeline/jobs/worker.py

@@ -16,7 +16,6 @@ from collections.abc import Mapping, Sequence
 from sqlalchemy import select
 from sqlalchemy.orm import sessionmaker
 
-from photo_pipeline.faults import JOB_ITEM_DONE, maybe_fault
 from photo_pipeline.jobs.handlers import REGISTRY, Cancelled, Handler, JobContext
 from photo_pipeline.models import JobItem
 from photo_pipeline.services.jobs import ItemState, JobConflict, JobService, JobState
@@ -87,9 +86,6 @@ class Worker:
                 )
             else:
                 self.service.set_item(job_id, item_key, ItemState.SUCCEEDED, fencing_token=token)
-                # One item is durably done, the job outcome is not: the control point
-                # for a crash mid-batch (US07-04). Recovery must not re-run this item.
-                maybe_fault(JOB_ITEM_DONE)
             self.service.heartbeat(job_id, self.worker_id, lease_seconds=self.lease_seconds)
 
         self._finalize(job_id, token, cancelled=cancelled, any_failed=any_failed)

photo_pipeline/models/__init__.py

@@ -5,9 +5,8 @@ Alembic environment relies on.
 """
 
 from photo_pipeline.models.albums import AlbumProposal
-from photo_pipeline.models.archives import ArchiveLocation, ArchiveOperation, ArchivePlan
+from photo_pipeline.models.archives import ArchiveLocation
 from photo_pipeline.models.assets import Asset, AssetPath
-from photo_pipeline.models.exif import ExifProjection
 from photo_pipeline.models.duplicates import (
     DuplicateCluster,
     DuplicateMember,
@@ -22,14 +21,11 @@ from photo_pipeline.models.workflow import AnalysisResult, SafetyReview
 __all__ = [
     "AlbumProposal",
     "ArchiveLocation",
-    "ArchiveOperation",
-    "ArchivePlan",
     "Asset",
     "AssetPath",
     "DuplicateCluster",
     "DuplicateMember",
     "DuplicateNegativeLink",
-    "ExifProjection",
     "Job",
     "JobItem",
     "JobEvent",

photo_pipeline/models/archives.py

@@ -1,4 +1,4 @@
-"""Archive location, plan, and transfer-journal persistence (US06-01, US06-02).
+"""Archive location persistence (US06-01).
 
 An archive location is a *medium*, not a path. External disks get mounted at
 different mountpoints, and a different disk can be mounted at the same one, so a
@@ -9,27 +9,13 @@ location therefore owns a marker file written onto the medium itself; its
 ``capabilities`` and ``state`` are the last probe result, kept so the UI can list
 locations without touching a sleeping disk. Preflight always re-probes — a stored
 state is a hint, never evidence.
-
-An ``ArchivePlan`` is one approved preflight turned into durable work, and each
-``ArchiveOperation`` is one file's crash-safe journal row (US06-02). The row records
-what the transfer *intends* to do before it does it — source, destination, expected
-hash — because after a crash that intent plus the files on disk is the only evidence
-available for deciding whether an original may be removed.
 """
 
 from __future__ import annotations
 
 from datetime import datetime
 
-from sqlalchemy import (
+from sqlalchemy import DateTime, String, func
-    Boolean,
-    DateTime,
-    ForeignKey,
-    Integer,
-    String,
-    UniqueConstraint,
-    func,
-)
 from sqlalchemy.orm import Mapped, mapped_column
 
 from photo_pipeline.db import Base
@@ -54,74 +40,3 @@ class ArchiveLocation(Base):
     updated_at: Mapped[datetime] = mapped_column(
         DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now()
     )
-
-
-class ArchivePlan(Base):
-    __tablename__ = "archive_plans"
-
-    id: Mapped[str] = mapped_column(String, primary_key=True)
-    location_id: Mapped[str] = mapped_column(
-        ForeignKey("archive_locations.id"), nullable=False, index=True
-    )
-    # The preflight token this plan was approved against; re-verified before apply.
-    token: Mapped[str] = mapped_column(String, nullable=False)
-    albums: Mapped[str | None] = mapped_column(String)  # JSON array
-    # archive | restore — the same journal read in the opposite direction (US06-04).
-    direction: Mapped[str] = mapped_column(String, nullable=False, default="archive")
-
-    # planned | applying | complete | failed
-    state: Mapped[str] = mapped_column(String, nullable=False, default="planned")
-    schema_version: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
-    asset_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
-    byte_size: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
-    # Bumped on every claim and used as the fencing token, so a superseded attempt
-    # cannot commit.
-    version: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
-    worker_id: Mapped[str | None] = mapped_column(String)
-    completed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
-
-    created_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now()
-    )
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now()
-    )
-
-
-class ArchiveOperation(Base):
-    __tablename__ = "archive_operations"
-    __table_args__ = (
-        UniqueConstraint("plan_id", "sequence", name="uq_archive_operations_plan_sequence"),
-    )
-
-    id: Mapped[str] = mapped_column(String, primary_key=True)
-    plan_id: Mapped[str] = mapped_column(
-        ForeignKey("archive_plans.id", ondelete="CASCADE"), nullable=False, index=True
-    )
-    sequence: Mapped[int] = mapped_column(Integer, nullable=False)
-    album: Mapped[str] = mapped_column(String, nullable=False)
-    asset_id: Mapped[str] = mapped_column(ForeignKey("assets.id"), nullable=False, index=True)
-
-    # archive: library → medium. restore: medium → library (US06-04). ``source_path``
-    # and ``destination_path`` always mean "from" and "to" for this direction.
-    direction: Mapped[str] = mapped_column(String, nullable=False, default="archive")
-    source_path: Mapped[str] = mapped_column(String, nullable=False)
-    destination_path: Mapped[str] = mapped_column(String, nullable=False)
-    # Relative to the location root, because the medium can be mounted elsewhere.
-    archive_path: Mapped[str] = mapped_column(String, nullable=False)
-    expected_sha256: Mapped[str] = mapped_column(String, nullable=False)
-    byte_size: Mapped[int | None] = mapped_column(Integer)
-    same_filesystem: Mapped[bool | None] = mapped_column(Boolean)
-
-    # planned | transferring | verified | removing | complete | failed
-    journal_state: Mapped[str] = mapped_column(String, nullable=False, default="planned")
-    attempt_count: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
-    fencing_token: Mapped[int | None] = mapped_column(Integer)
-    worker_id: Mapped[str | None] = mapped_column(String)
-    verified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
-    removed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
-    error_code: Mapped[str | None] = mapped_column(String)
-    error_message: Mapped[str | None] = mapped_column(String)
-    updated_at: Mapped[datetime] = mapped_column(
-        DateTime(timezone=True), nullable=False, server_default=func.now(), onupdate=func.now()
-    )

photo_pipeline/models/assets.py

@@ -35,18 +35,7 @@ class Asset(Base):
 
     discovered_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
     missing_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
-    # active | archiving | archived_online | archived_offline | restoring |
-    # missing_unexpected (concept §9). ``current_path`` is NULL once archived; the
-    # original is then explained by the location plus its relative archive path.
     availability_state: Mapped[str] = mapped_column(String, nullable=False, default="active")
-    # Not a declared foreign key: SQLite cannot add one to an existing table, so the
-    # link is written and read by the archive service (US06-02).
-    archive_location_id: Mapped[str | None] = mapped_column(String)
-    archive_path: Mapped[str | None] = mapped_column(String)
-    # Set when the archived copy was proven to hold bytes other than the recorded
-    # ones (US06-04). Restore refuses such an asset instead of accepting a different
-    # file; cleared as soon as a verification matches again.
-    archive_divergent_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
     # Duplicate canonical link: NULL when the asset is itself canonical or undecided.
     canonical_asset_id: Mapped[str | None] = mapped_column(ForeignKey("assets.id"))
 

photo_pipeline/models/exif.py

@@ -1,33 +0,0 @@
-"""The durable EXIF projection per asset and stage (concept §3, US07-03).
-
-One row per ``(asset_id, stage)``: what the stage wanted written, what the file
-looked like afterwards, and whether anything outside the stage's ownership moved.
-``state = divergent`` is the whole point of the table — it survives restarts, keeps
-the asset out of stages that require verified metadata, and gives a human something
-to look at instead of a silent repair.
-"""
-
-from __future__ import annotations
-
-from datetime import datetime
-
-from sqlalchemy import DateTime, ForeignKey, Integer, String
-from sqlalchemy.orm import Mapped, mapped_column
-
-from photo_pipeline.db import Base
-
-
-class ExifProjection(Base):
-    __tablename__ = "exif_projections"
-
-    asset_id: Mapped[str] = mapped_column(ForeignKey("assets.id"), primary_key=True)
-    stage: Mapped[str] = mapped_column(String, primary_key=True)  # safety | analysis
-    id: Mapped[str] = mapped_column(String, nullable=False)
-    projection_version: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
-    desired_json: Mapped[str | None] = mapped_column(String)
-    divergent_fields: Mapped[str | None] = mapped_column(String)  # JSON array
-    result_file_sha256: Mapped[str | None] = mapped_column(String)
-    state: Mapped[str] = mapped_column(String, nullable=False)  # verified|divergent|failed
-    error_code: Mapped[str | None] = mapped_column(String)
-    verified_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
-    updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))

photo_pipeline/models/thumbnails.py

@@ -11,7 +11,7 @@ from __future__ import annotations
 
 from datetime import datetime
 
-from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String, func
+from sqlalchemy import DateTime, ForeignKey, Integer, String, func
 from sqlalchemy.orm import Mapped, mapped_column
 
 from photo_pipeline.db import Base
@@ -29,9 +29,6 @@ class Thumbnail(Base):
     width: Mapped[int | None] = mapped_column(Integer)
     height: Mapped[int | None] = mapped_column(Integer)
     format: Mapped[str | None] = mapped_column(String)
-    # Durable comparison evidence for an archived asset: never evicted by the LRU
-    # quota, because the original may be on a medium that is no longer reachable.
-    protected: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
     created_at: Mapped[datetime] = mapped_column(
         DateTime(timezone=True), nullable=False, server_default=func.now()
     )

photo_pipeline/path_policy.py

@@ -49,29 +49,6 @@ def resolve_within(root: Path, path: os.PathLike | str) -> Path:
     return resolved
 
 
-def resolve_in_roots(roots: Iterable[os.PathLike | str], path: os.PathLike | str) -> Path:
-    """The resolved path, proven to be inside one of ``roots`` and not excluded.
-
-    Callers must use the **returned** path for whatever they do next: validating one
-    name and then opening another is the symlink race this exists to close (US07-02).
-    The message names no path — it reaches API responses.
-
-    With no roots configured there is no boundary to check; that is a property of the
-    configuration, not permission granted to this call.
-    """
-    if is_excluded(path):
-        raise PathPolicyError("path is inside an excluded (_IGNORE/) tree")
-    roots = list(roots)
-    if not roots:
-        return Path(path)
-    for root in roots:
-        try:
-            return resolve_within(Path(root), path)
-        except PathPolicyError:
-            continue
-    raise PathPolicyError("path is outside the configured library roots")
-
-
 def iter_supported_files(root: os.PathLike | str) -> Iterator[Path]:
     """Yield supported, non-excluded files under ``root`` in deterministic order.
 

photo_pipeline/services/analysis.py

@@ -25,9 +25,9 @@ from typing import Protocol
 from sqlalchemy import func, select
 from sqlalchemy.orm import sessionmaker
 
-from photo_pipeline import path_policy
+from photo_pipeline.integrations import exiftool
 from photo_pipeline.models import AnalysisResult, Asset, SafetyReview
-from photo_pipeline.services import exif_checkpoint
+from photo_pipeline.services import hashing
 from photo_pipeline.services.safety import SFW
 
 MODEL = "gemini-2.5-flash"
@@ -59,16 +59,9 @@ def _now() -> datetime:
 
 
 class AnalysisService:
-    def __init__(
+    def __init__(self, session_factory: sessionmaker, *, provider: VisionProvider | None = None) -> None:
-        self,
-        session_factory: sessionmaker,
-        *,
-        provider: VisionProvider | None = None,
-        library_roots: tuple = (),
-    ) -> None:
         self._session_factory = session_factory
         self._provider = provider
-        self._roots = tuple(library_roots)
 
     def _sfw_asset_ids(self, session) -> set[str]:
         """Asset ids whose latest safety decision is ``sfw`` — the ONLY assets that
@@ -78,11 +71,6 @@ class AnalysisService:
             latest[review.asset_id] = review.decision
         return {aid for aid, decision in latest.items() if decision == SFW}
 
-    def _is_still_sfw(self, asset_id: str) -> bool:
-        """Re-read the current safety decision straight from the database."""
-        with self._session_factory() as session:
-            return asset_id in self._sfw_asset_ids(session)
-
     def eligible_asset_ids(self) -> list[str]:
         """Confirmed-SFW canonical active assets without a completed analysis."""
         with self._session_factory() as session:
@@ -148,40 +136,12 @@ class AnalysisService:
             if not path:
                 skipped += 1
                 continue
-            # Second gate, at the moment of use: the database says where the file
-            # was, the filesystem decides what that name means now. A link swapped
-            # under an asset after the scan would otherwise send bytes from outside
-            # the library — the one place that leaves this machine (US07-02).
-            try:
-                path = str(path_policy.resolve_in_roots(self._roots, path))
-            except path_policy.PathPolicyError as error:
-                self._store(
-                    asset_id, status="error", result=None, error=str(error), tokens=0, raw=""
-                )
-                errors += 1
-                continue
             try:
                 result = provider.analyze(path, album_hint=_album_hint(path))
             except Exception as error:  # provider/validation failure is per-asset
                 self._store(asset_id, status="error", result=None, error=str(error), tokens=0, raw="")
                 errors += 1
                 continue
-            # Third gate, after the call: a provider request takes seconds, and the
-            # reviewer may have flipped this asset to NSFW while it was in flight.
-            # The result describes an asset that is no longer analysable, so it is
-            # discarded — not stored, and above all not written into its EXIF
-            # (concept §18 scenario 7, US07-04).
-            if not self._is_still_sfw(asset_id):
-                self._store(
-                    asset_id,
-                    status="skipped_nsfw",
-                    result=None,
-                    error="the safety decision changed while analysis was in flight",
-                    tokens=0,
-                    raw="",
-                )
-                skipped += 1
-                continue
             self._store(
                 asset_id,
                 status="analyzed",
@@ -190,7 +150,7 @@ class AnalysisService:
                 tokens=int(result.get("_tokens", 0)) if isinstance(result, dict) else 0,
                 raw=json.dumps(result, ensure_ascii=False),
             )
-            self._write_analysis_exif(asset_id, path, result)
+            _write_analysis_exif(path, result)
             analyzed += 1
         return {"analyzed": analyzed, "skipped": skipped, "errors": errors}
 
@@ -215,42 +175,10 @@ class AnalysisService:
                 row.location_hint = result.get("location_hint")
                 row.approx_year = result.get("approx_year")
                 row.analyzed_at = now
+                row.exif_written_at = now
             session.add(row)
             session.commit()
 
-    def _write_analysis_exif(self, asset_id: str, path: str, result: dict) -> None:
-        """The analysis EXIF checkpoint: additive keywords, then prove the rest held.
-
-        Additive by design — safety keywords and the user's own keywords are merged
-        with, never replaced (concept §3). ``exif_written_at`` is set only when the
-        read-back verified both the new keywords and every field this stage does not
-        own; a divergent result is recorded and left for a human (US07-03).
-        ponytail: the managed ``AI:`` caption segment, once captions are owned here.
-        """
-        tags = tuple(str(tag) for tag in (result.get("tags") or []))
-        if not tags:
-            return
-        checkpoint = exif_checkpoint.run(path, add=tags)
-        exif_checkpoint.record(
-            self._session_factory,
-            asset_id=asset_id,
-            stage="analysis",
-            result=checkpoint,
-            add=tags,
-        )
-        if not checkpoint.verified:
-            return
-        with self._session_factory() as session:
-            row = session.get(AnalysisResult, asset_id)
-            if row is not None:
-                row.exif_written_at = checkpoint.verified_at
-            asset = session.get(Asset, asset_id)
-            if asset is not None and checkpoint.sha256:
-                # The bytes changed when the container was rewritten; upload must use
-                # the hash of what is actually on disk now (concept §3).
-                asset.current_sha256 = checkpoint.sha256
-            session.commit()
-
     def get(self, asset_id: str) -> dict | None:
         with self._session_factory() as session:
             row = session.get(AnalysisResult, asset_id)
@@ -263,6 +191,17 @@ def _album_hint(path: str) -> str:
     return Path(path).parent.name
 
 
+def _write_analysis_exif(path: str, result: dict) -> None:
+    """Additive analysis keywords into EXIF (Keywords/Subject), preserving safety
+    and user keywords. The donor also wrote a managed caption; only keywords are
+    written here via the shared adapter.
+    ponytail: add the managed ``AI:`` caption segment + read-back preservation check
+    when the analysis EXIF checkpoint is hardened."""
+    tags = result.get("tags") if isinstance(result, dict) else None
+    if tags:
+        exiftool.apply_keywords(path, add=[str(t) for t in tags])
+
+
 def _result_dict(row: AnalysisResult) -> dict:
     data = {field: getattr(row, field) for field in RESULT_FIELDS}
     data["tags"] = json.loads(row.tags) if row.tags else []

photo_pipeline/services/archive_journal.py

@@ -1,359 +0,0 @@
-"""Archive transfer journal — the durable record of every per-file transition
-(US06-02).
-
-Archiving is the only stage that deletes an original, so the journal exists to make
-one question answerable after any crash: *may this source file be removed?* Intent
-is written before the mutation it describes, and the recorded state plus the real
-files on disk are the sole basis for answering it later. This module owns the state
-machine, the durable writes, and the evidence table; it never touches a photo
-(:mod:`photo_pipeline.services.archive_transfer` does).
-
-Per-item state machine (concept §9 "Transfer and removal semantics"):
-
-```
-planned → transferring → verified → removing → complete
-              ↘             ↘          ↘ failed
-```
-
-A restore (US06-04) uses the same rows with ``direction='restore'``: it copies from
-the medium back into the library and removes nothing, so it goes ``verified →
-complete`` directly. ``source_path``/``destination_path`` always mean "from"/"to",
-which is why the evidence table below needs no direction of its own.
-
-- ``transferring`` — intent recorded; a temporary copy may exist, the destination
-  may or may not have been published. Nothing has been removed.
-- ``verified``     — the archived bytes exist at their final path, hash exactly as
-  recorded, and the manifest entry is durable. Only from here may a source go.
-- ``removing``     — the source removal is committed to; the source may already be
-  gone while the database still points at it.
-- ``complete``     — source absent, database updated, availability recorded.
-
-``classify`` labels each incomplete item from the journal plus disk evidence:
-
-- ``resumable`` — nothing was published; the source is intact, so applying again is
-  safe.
-- ``forward``   — the archived copy exists and matches its recorded hash, so the
-  remaining steps (manifest, removal, bookkeeping) can be finished deterministically.
-- ``manual``    — the evidence contradicts the journal (missing archive copy, wrong
-  bytes, source and archive both gone). Nothing is guessed and nothing is removed;
-  the item blocks unrelated mutations until a human decides.
-"""
-
-from __future__ import annotations
-
-from datetime import datetime, timezone
-from pathlib import Path
-
-from sqlalchemy import select
-from sqlalchemy.orm import sessionmaker
-
-from photo_pipeline.models import ArchiveOperation, ArchivePlan
-from photo_pipeline.services.hashing import sha256_file
-
-
-class ArchiveState:
-    PLANNED = "planned"
-    TRANSFERRING = "transferring"
-    VERIFIED = "verified"
-    REMOVING = "removing"
-    COMPLETE = "complete"
-    FAILED = "failed"
-
-
-ALLOWED_TRANSITIONS = {
-    ArchiveState.PLANNED: {ArchiveState.TRANSFERRING, ArchiveState.FAILED},
-    # From `transferring` the outcome is unknown until evidence is gathered, so it
-    # may resolve forward, back to planned (proven nothing was published), or fail.
-    ArchiveState.TRANSFERRING: {
-        ArchiveState.VERIFIED,
-        ArchiveState.PLANNED,
-        ArchiveState.FAILED,
-    },
-    ArchiveState.VERIFIED: {ArchiveState.REMOVING, ArchiveState.FAILED},
-    # No path back: once the source may be gone, only finishing is safe.
-    ArchiveState.REMOVING: {ArchiveState.COMPLETE, ArchiveState.FAILED},
-    ArchiveState.COMPLETE: set(),
-    # A retry re-enters `transferring`, which rechecks every precondition from
-    # scratch; recovery may also reset a failed item to `planned`.
-    ArchiveState.FAILED: {ArchiveState.PLANNED, ArchiveState.TRANSFERRING},
-}
-
-# A restore removes nothing, so it has no ``removing`` step: a verified published
-# copy is the whole job (US06-04). Keeping this as a separate table means the
-# archive direction still cannot reach ``complete`` without going through removal.
-RESTORE_TRANSITIONS = {
-    **ALLOWED_TRANSITIONS,
-    ArchiveState.VERIFIED: {ArchiveState.COMPLETE, ArchiveState.FAILED},
-}
-
-TERMINAL_STATES = frozenset({ArchiveState.COMPLETE})
-# States where this item may already have touched the filesystem.
-UNSAFE_STATES = frozenset({ArchiveState.TRANSFERRING, ArchiveState.VERIFIED, ArchiveState.REMOVING})
-
-# Which way the bytes move. Same rows, same evidence table, opposite direction.
-ARCHIVE = "archive"
-RESTORE = "restore"
-
-RESUMABLE = "resumable"
-FORWARD = "forward"
-MANUAL = "manual"
-
-
-class JournalError(RuntimeError):
-    pass
-
-
-class InvalidTransition(JournalError):
-    pass
-
-
-class JournalConflict(JournalError):
-    """Fencing check failed; a newer owner has taken over this operation."""
-
-
-def can_transition(current: str, target: str, direction: str = ARCHIVE) -> bool:
-    table = RESTORE_TRANSITIONS if direction == RESTORE else ALLOWED_TRANSITIONS
-    return target in table.get(current, set())
-
-
-def _now() -> datetime:
-    return datetime.now(timezone.utc)
-
-
-class ArchiveJournal:
-    def __init__(self, session_factory: sessionmaker) -> None:
-        self._session_factory = session_factory
-
-    # ── intent ────────────────────────────────────────────────────────────────
-
-    def begin(self, operation_id: str, *, worker_id: str, fencing_token: int) -> dict:
-        """Record the intent to transfer **before** touching the filesystem."""
-        with self._session_factory() as session:
-            row = self._require(session, operation_id)
-            if row.fencing_token is not None and fencing_token < row.fencing_token:
-                raise JournalConflict(
-                    f"stale fencing token {fencing_token} (current {row.fencing_token})"
-                )
-            if row.journal_state in TERMINAL_STATES:
-                raise InvalidTransition(f"{row.journal_state} is terminal")
-            if row.journal_state != ArchiveState.TRANSFERRING and not can_transition(
-                row.journal_state, ArchiveState.TRANSFERRING, row.direction
-            ):
-                raise InvalidTransition(f"{row.journal_state} -> {ArchiveState.TRANSFERRING}")
-            if row.journal_state != ArchiveState.TRANSFERRING:
-                row.attempt_count += 1
-            row.journal_state = ArchiveState.TRANSFERRING
-            row.worker_id = worker_id
-            row.fencing_token = fencing_token
-            row.error_code = row.error_message = None
-            row.updated_at = _now()
-            session.commit()
-            return _operation_dict(row)
-
-    # ── transitions ───────────────────────────────────────────────────────────
-
-    def transition(
-        self,
-        operation_id: str,
-        target: str,
-        *,
-        fencing_token: int | None = None,
-        error: tuple[str, str] | None = None,
-        same_filesystem: bool | None = None,
-    ) -> dict:
-        """Move one operation to ``target``, enforcing the state machine.
-
-        Re-entering the state an operation already holds is a no-op, which is what
-        makes recovery idempotent across repeated restarts.
-        """
-        with self._session_factory() as session:
-            row = self._require(session, operation_id)
-            if fencing_token is not None and row.fencing_token is not None:
-                if fencing_token < row.fencing_token:
-                    raise JournalConflict(
-                        f"stale fencing token {fencing_token} (current {row.fencing_token})"
-                    )
-            if same_filesystem is not None:
-                row.same_filesystem = same_filesystem
-            if row.journal_state == target:
-                session.commit()
-                return _operation_dict(row)  # idempotent
-            if not can_transition(row.journal_state, target, row.direction):
-                raise InvalidTransition(f"{row.journal_state} -> {target}")
-
-            row.journal_state = target
-            row.updated_at = _now()
-            if target == ArchiveState.VERIFIED:
-                row.verified_at = _now()
-            if target == ArchiveState.COMPLETE:
-                row.removed_at = _now()
-            if error:
-                row.error_code, row.error_message = error[0], error[1][:500]
-            elif target != ArchiveState.FAILED:
-                row.error_code = row.error_message = None
-            session.commit()
-            return _operation_dict(row)
-
-    # ── reads ─────────────────────────────────────────────────────────────────
-
-    def get(self, operation_id: str) -> dict | None:
-        with self._session_factory() as session:
-            row = session.get(ArchiveOperation, operation_id)
-            return _operation_dict(row) if row else None
-
-    def operations(self, plan_id: str) -> list[dict]:
-        with self._session_factory() as session:
-            rows = session.scalars(
-                select(ArchiveOperation)
-                .where(ArchiveOperation.plan_id == plan_id)
-                .order_by(ArchiveOperation.sequence)
-            )
-            return [_operation_dict(row) for row in rows]
-
-    def incomplete(self, *, direction: str | None = None) -> list[dict]:
-        """Every operation left in a non-terminal, non-planned state — the work a
-        restart has to reason about. Without ``direction`` this spans archives and
-        restores, because either one half-done blocks the other."""
-        with self._session_factory() as session:
-            stmt = select(ArchiveOperation).where(
-                ArchiveOperation.journal_state.not_in([*TERMINAL_STATES, ArchiveState.PLANNED])
-            )
-            if direction is not None:
-                stmt = stmt.where(ArchiveOperation.direction == direction)
-            rows = session.scalars(
-                stmt.order_by(ArchiveOperation.plan_id, ArchiveOperation.sequence)
-            )
-            return [_operation_dict(row) for row in rows]
-
-    # ── startup classification ────────────────────────────────────────────────
-
-    def classify(self, operation_id: str) -> dict:
-        """Classify one incomplete operation from the journal plus disk evidence.
-
-        Hashes the archived copy when one exists: "a file is at the destination" is
-        not evidence that the *right* bytes are, and only the right bytes justify
-        removing an original. Never mutates anything.
-        """
-        row = self.get(operation_id)
-        if row is None:
-            raise JournalError(f"unknown archive operation {operation_id!r}")
-        source = Path(row["source_path"])
-        destination = Path(row["destination_path"])
-        source_exists = source.exists()
-        destination_exists = destination.exists()
-        destination_matches = (
-            destination_exists and sha256_file(destination) == row["expected_sha256"]
-        )
-
-        classification, reason = _classify(
-            row["journal_state"], source_exists, destination_exists, destination_matches
-        )
-        return {
-            "operation_id": operation_id,
-            "plan_id": row["plan_id"],
-            "direction": row["direction"],
-            "album": row["album"],
-            "asset_id": row["asset_id"],
-            "source_path": row["source_path"],
-            "destination_path": row["destination_path"],
-            "journal_state": row["journal_state"],
-            "classification": classification,
-            "reason": reason,
-            "source_exists": source_exists,
-            "destination_exists": destination_exists,
-            "destination_matches": destination_matches,
-        }
-
-    def classify_all(self, *, direction: str | None = None) -> list[dict]:
-        return [self.classify(row["id"]) for row in self.incomplete(direction=direction)]
-
-    def blocks_mutation(self) -> bool:
-        """True when any item may have the library half-archived."""
-        return any(row["journal_state"] in UNSAFE_STATES for row in self.incomplete())
-
-    # ── plan-level ────────────────────────────────────────────────────────────
-
-    def plan_state(self, plan_id: str) -> str:
-        """Derive the plan's state from its items, so the summary can never disagree
-        with the journal."""
-        states = {row["journal_state"] for row in self.operations(plan_id)}
-        if not states:
-            return "planned"
-        if states <= {ArchiveState.COMPLETE}:
-            return "complete"
-        if states & {ArchiveState.FAILED}:
-            return "failed"
-        if states & UNSAFE_STATES:
-            return "applying"
-        return "planned"
-
-    def sync_plan_state(self, plan_id: str) -> str:
-        state = self.plan_state(plan_id)
-        with self._session_factory() as session:
-            plan = session.get(ArchivePlan, plan_id)
-            if plan is None:
-                raise JournalError(f"unknown archive plan {plan_id!r}")
-            if plan.state != state:
-                plan.state = state
-                plan.version += 1
-                plan.updated_at = _now()
-                if state == "complete" and plan.completed_at is None:
-                    plan.completed_at = _now()
-                session.commit()
-        return state
-
-    @staticmethod
-    def _require(session, operation_id: str) -> ArchiveOperation:
-        row = session.get(ArchiveOperation, operation_id)
-        if row is None:
-            raise JournalError(f"unknown archive operation {operation_id!r}")
-        return row
-
-
-def _classify(
-    state: str, source_exists: bool, destination_exists: bool, destination_matches: bool
-) -> tuple[str, str]:
-    """The evidence table. Kept a pure function so every combination is testable."""
-    if destination_exists and not destination_matches and state != ArchiveState.PLANNED:
-        # Someone else's file, or a partial/edited copy: never overwrite it, and
-        # never treat it as the durable archive that justifies a deletion.
-        return MANUAL, "the archived path holds bytes that are not the recorded ones"
-
-    if state in (ArchiveState.TRANSFERRING, ArchiveState.FAILED):
-        if destination_matches:
-            return FORWARD, "the archived copy is durable; finish the remaining steps"
-        if source_exists:
-            return RESUMABLE, "nothing was published; the source is intact"
-        return MANUAL, "neither the source nor a verified archive copy is present"
-
-    if state in (ArchiveState.VERIFIED, ArchiveState.REMOVING):
-        if destination_matches:
-            return FORWARD, "the archived copy is durable; finish the remaining steps"
-        return MANUAL, f"journal says {state} but the archived copy is missing"
-
-    return MANUAL, f"unhandled journal state {state}"
-
-
-def _operation_dict(row: ArchiveOperation) -> dict:
-    return {
-        "id": row.id,
-        "plan_id": row.plan_id,
-        "direction": row.direction,
-        "sequence": row.sequence,
-        "album": row.album,
-        "asset_id": row.asset_id,
-        "source_path": row.source_path,
-        "destination_path": row.destination_path,
-        "archive_path": row.archive_path,
-        "expected_sha256": row.expected_sha256,
-        "byte_size": row.byte_size,
-        "same_filesystem": row.same_filesystem,
-        "journal_state": row.journal_state,
-        "attempt_count": row.attempt_count,
-        "fencing_token": row.fencing_token,
-        "worker_id": row.worker_id,
-        "verified_at": row.verified_at.isoformat() if row.verified_at else None,
-        "removed_at": row.removed_at.isoformat() if row.removed_at else None,
-        "error_code": row.error_code,
-        "error_message": row.error_message,
-    }

photo_pipeline/services/archive_transfer.py

@@ -1,645 +0,0 @@
-"""Transfer an approved archive plan, verify it, and remove the active sources
-(US06-02).
-
-This is the only module that deletes originals from the photo library, so every
-step exists to make one promise keepable: **a source is removed only after the
-archived bytes are durable and proven identical.** The journal (US06-02,
-:mod:`photo_pipeline.services.archive_journal`) records intent before each mutation;
-this module performs the mutations and the recovery that reads that intent back.
-
-Per file the sequence is:
-
-```
-journal.begin (transferring)   ← intent persisted BEFORE any disk change
-recheck preconditions          ← source hash, free destination, no symlink
-copy to a temporary file       ← same directory, so the publish is atomic
-fsync, close, hash it back     ← read from disk; the write is not the evidence
-atomically publish             ← rename onto the final archive path
-append the manifest entry      ← durable on the medium itself, fsynced
-journal → verified
-journal → removing             ← intent to delete, persisted first
-re-verify the archive copy, remove the source, verify its absence
-current_path=NULL, close the path occurrence, availability + location recorded
-journal → complete
-```
-
-Same-filesystem albums may skip the copy and use an atomic ``rename`` instead
-(concept §9), but only when the shared device is proven at run time — never from the
-plan's stored guess — and the published file is hashed afterwards exactly as in the
-copy path.
-
-Rules that are never relaxed:
-
-- An occupied destination is never overwritten; the item fails with the source
-  untouched.
-- A source whose bytes no longer match the plan is never archived and never removed.
-- A crash resolves from journal + disk evidence only: an archive copy that is
-  missing or hashes differently blocks the item for a human instead of being
-  retried or, worse, treated as a successful archive.
-- Recovery is idempotent — repeated passes converge on the same state.
-"""
-
-from __future__ import annotations
-
-import json
-import os
-import shutil
-import uuid
-from datetime import datetime, timezone
-from pathlib import Path
-
-from sqlalchemy import select
-from sqlalchemy.orm import sessionmaker
-
-from photo_pipeline.config import Config
-from photo_pipeline.faults import maybe_fault
-from photo_pipeline.models import ArchiveLocation, ArchiveOperation, ArchivePlan, Asset, AssetPath
-from photo_pipeline.services.archive_journal import (
-    ARCHIVE,
-    MANUAL,
-    RESUMABLE,
-    ArchiveJournal,
-    ArchiveState,
-)
-from photo_pipeline.services.archives import MARKER_NAME, ArchiveError, ArchiveService
-from photo_pipeline.services.duplicates import DuplicateService
-from photo_pipeline.services.hashing import sha256_file
-from photo_pipeline.services.rename_apply import PreconditionFailed
-from photo_pipeline.services.thumbnails import ThumbnailService
-
-# The per-medium manifest: one JSON line per archived file, appended and fsynced
-# before its source is removed. It lives with the bytes so the archive can still be
-# read back if the database is lost.
-MANIFEST_NAME = "archive-manifest.jsonl"
-MANIFEST_VERSION = 1
-TEMP_SUFFIX = ".part"
-TEMP_PREFIX = ".archive-"
-# Fault barrier between "the source is gone" and "the database knows it" — not a
-# journal state, but the transition crash tests care about most.
-SOURCE_REMOVED = "source_removed"
-
-APPLYABLE_PLAN_STATES = frozenset({"planned", "applying", "failed", "complete"})
-
-
-def _now() -> datetime:
-    return datetime.now(timezone.utc)
-
-
-class ArchiveTransferService:
-    def __init__(self, session_factory: sessionmaker, *, config: Config) -> None:
-        self._session_factory = session_factory
-        self._config = config
-        self.journal = ArchiveJournal(session_factory)
-
-    # ── plans ─────────────────────────────────────────────────────────────────
-
-    def create(self, location_id: str, albums: list[str] | None = None, *, token: str) -> dict:
-        """Turn an approved preflight into a durable plan.
-
-        The token is re-derived from a fresh preflight, so a plan can only be
-        created for the exact scope, bytes, and destination the user approved.
-        """
-        preflight = ArchiveService(self._session_factory, config=self._config).preflight(
-            location_id, albums
-        )
-        if not token or token != preflight["token"]:
-            raise ArchiveError("stale_token", "the archive preflight changed since it was approved")
-        if preflight["state"] != "ready":
-            codes = ", ".join(sorted({issue["code"] for issue in preflight["blockers"]})) or "-"
-            raise ArchiveError("blocked", f"the archive scope is blocked: {codes}")
-
-        root = Path(preflight["location"]["root"])
-        plan_id = str(uuid.uuid4())
-        with self._session_factory() as session:
-            session.add(
-                ArchivePlan(
-                    id=plan_id,
-                    location_id=location_id,
-                    token=token,
-                    albums=json.dumps(albums) if albums is not None else None,
-                    direction=ARCHIVE,
-                    state="planned",
-                    schema_version=MANIFEST_VERSION,
-                    asset_count=preflight["totals"]["assets"],
-                    byte_size=preflight["totals"]["bytes"],
-                )
-            )
-            session.flush()  # the plan row must exist before its items reference it
-            sequence = 0
-            for album in preflight["albums"]:
-                destination_dir = Path(album["destination"])
-                for asset in album["assets"]:
-                    source = Path(asset["current_path"])
-                    destination = destination_dir / source.name
-                    session.add(
-                        ArchiveOperation(
-                            id=str(uuid.uuid4()),
-                            plan_id=plan_id,
-                            direction=ARCHIVE,
-                            sequence=sequence,
-                            album=album["album"],
-                            asset_id=asset["asset_id"],
-                            source_path=str(source),
-                            destination_path=str(destination),
-                            archive_path=str(destination.relative_to(root)),
-                            expected_sha256=asset["current_sha256"],
-                            byte_size=asset["byte_size"],
-                            # Recorded as a preview only; the real decision is made
-                            # against the devices at apply time.
-                            same_filesystem=album["transfer_method"] == "move",
-                            journal_state=ArchiveState.PLANNED,
-                        )
-                    )
-                    sequence += 1
-            session.commit()
-        return self.get(plan_id)
-
-    def get(self, plan_id: str) -> dict | None:
-        with self._session_factory() as session:
-            plan = session.get(ArchivePlan, plan_id)
-            if plan is None:
-                return None
-            report = _plan_dict(plan)
-        report["operations"] = self.journal.operations(plan_id)
-        return report
-
-    def list(self) -> list[dict]:
-        with self._session_factory() as session:
-            rows = session.scalars(
-                select(ArchivePlan)
-                .where(ArchivePlan.direction == ARCHIVE)
-                .order_by(ArchivePlan.created_at)
-            )
-            return [_plan_dict(row) for row in rows]
-
-    # ── apply ─────────────────────────────────────────────────────────────────
-
-    def apply(
-        self,
-        plan_id: str,
-        *,
-        expected_version: int | None = None,
-        worker_id: str = "archive",
-    ) -> dict:
-        """Archive every item of a plan, then report what happened.
-
-        Items are independent: one failure records its reason and leaves that
-        source in place; the rest of the album continues.
-        """
-        plan = self._require_plan(plan_id)
-        if expected_version is not None and plan["version"] != expected_version:
-            raise ArchiveError(
-                "stale_plan",
-                f"plan {plan_id} is at version {plan['version']}, expected {expected_version}",
-            )
-        if plan["state"] not in APPLYABLE_PLAN_STATES:
-            raise ArchiveError("invalid_state", f"plan {plan_id} is {plan['state']}")
-
-        # One archiver lane: never start while another plan may be half-archived
-        # (concept §16 lock hierarchy).
-        blocking = [row for row in self.journal.incomplete() if row["plan_id"] != plan_id]
-        if blocking:
-            raise ArchiveError(
-                "archive_pending",
-                f"another archive operation is unresolved ({blocking[0]['id']}); recover it first",
-            )
-
-        token = self._claim_plan(plan_id)
-        location = self._location(plan["location_id"])
-        archived = failed = skipped = 0
-        for operation in self.journal.operations(plan_id):
-            if operation["journal_state"] == ArchiveState.COMPLETE:
-                skipped += 1  # repeated apply is a no-op for finished work
-                continue
-            try:
-                if operation["journal_state"] in (ArchiveState.VERIFIED, ArchiveState.REMOVING):
-                    # The bytes are already archived; never transfer them twice.
-                    self._finish(operation, location, token=token, worker_id=worker_id)
-                else:
-                    self._archive_one(operation, location, token=token, worker_id=worker_id)
-                archived += 1
-            except PreconditionFailed as error:
-                self._fail(operation, token, error.code, str(error))
-                failed += 1
-            except Exception as error:  # unexpected: record and stop touching disk
-                self._fail(operation, token, "archive_error", str(error))
-                failed += 1
-        self._prune_empty_sources(plan_id)
-        state = self.journal.sync_plan_state(plan_id)
-        return {
-            "plan_id": plan_id,
-            "archived": archived,
-            "failed": failed,
-            "skipped": skipped,
-            "state": state,
-        }
-
-    def _prune_empty_sources(self, plan_id: str) -> None:
-        """Drop an album folder once every one of its files is archived.
-
-        ``rmdir`` only: a folder that still holds anything at all — an unarchived
-        file, someone else's file, a subfolder — is left exactly as it is.
-        """
-        folders: dict[Path, set[str]] = {}
-        for operation in self.journal.operations(plan_id):
-            folders.setdefault(Path(operation["source_path"]).parent, set()).add(
-                operation["journal_state"]
-            )
-        for folder, states in folders.items():
-            if states == {ArchiveState.COMPLETE}:
-                try:
-                    folder.rmdir()
-                except OSError:
-                    pass  # not empty, or gone already; either way, leave it alone
-
-    def _archive_one(self, operation: dict, location: dict, *, token: int, worker_id: str) -> None:
-        source = Path(operation["source_path"])
-        destination = Path(operation["destination_path"])
-
-        # 1. Intent first — after this point a crash is recoverable from evidence.
-        self.journal.begin(operation["id"], worker_id=worker_id, fencing_token=token)
-        maybe_fault(ArchiveState.TRANSFERRING)
-
-        # 2. Recheck immediately before mutating; the plan's snapshot is not trusted.
-        self._recheck(operation, source, destination, location)
-        destination.parent.mkdir(parents=True, exist_ok=True)
-
-        # 3. Transfer. Same-filesystem is an optimisation, so it has to be proven
-        #    here rather than believed from the plan.
-        same_filesystem = _same_filesystem(source, destination.parent)
-        if same_filesystem:
-            os.rename(source, destination)
-        else:
-            copy_verify_publish(source, destination, operation["expected_sha256"])
-        _fsync_dir(destination.parent)
-
-        # 4. The published file is the archive only once it hashes as recorded.
-        if sha256_file(destination) != operation["expected_sha256"]:
-            raise PreconditionFailed(
-                "archive_mismatch", f"{destination} does not hold the expected bytes"
-            )
-        _append_manifest(destination.parent, _manifest_entry(operation, location))
-        self.journal.transition(
-            operation["id"],
-            ArchiveState.VERIFIED,
-            fencing_token=token,
-            same_filesystem=same_filesystem,
-        )
-        maybe_fault(ArchiveState.VERIFIED)
-
-        # 5. Only now may the active source go.
-        self._finish(self.journal.get(operation["id"]), location, token=token, worker_id=worker_id)
-
-    def _finish(self, operation: dict, location: dict, *, token: int, worker_id: str) -> None:
-        """Drive an item whose archive copy is durable through removal and
-        bookkeeping. Every step is idempotent, so recovery may replay it."""
-        destination = Path(operation["destination_path"])
-        source = Path(operation["source_path"])
-        state = operation["journal_state"]
-
-        if state == ArchiveState.FAILED:
-            # The archive copy is durable even though the attempt ended badly:
-            # re-enter the transfer state so the remaining steps can run.
-            self.journal.transition(operation["id"], ArchiveState.TRANSFERRING, fencing_token=token)
-            state = ArchiveState.TRANSFERRING
-
-        if state == ArchiveState.TRANSFERRING:
-            if sha256_file(destination) != operation["expected_sha256"]:
-                raise PreconditionFailed(
-                    "archive_mismatch", f"{destination} does not hold the expected bytes"
-                )
-            _append_manifest(destination.parent, _manifest_entry(operation, location))
-            self.journal.transition(operation["id"], ArchiveState.VERIFIED, fencing_token=token)
-            state = ArchiveState.VERIFIED
-
-        if state == ArchiveState.VERIFIED:
-            self.journal.transition(operation["id"], ArchiveState.REMOVING, fencing_token=token)
-            maybe_fault(ArchiveState.REMOVING)
-            state = ArchiveState.REMOVING
-
-        if state == ArchiveState.REMOVING:
-            # Re-verify the archived bytes immediately before deleting the original:
-            # this check is the entire justification for the removal.
-            if not destination.exists() or sha256_file(destination) != operation["expected_sha256"]:
-                raise PreconditionFailed(
-                    "archive_unverified", f"{destination} is not a verified archive copy"
-                )
-            self._require_evidence(operation["asset_id"], destination)
-            if source.exists():
-                if source.is_symlink():
-                    raise PreconditionFailed("symlink", f"{source} became a symlink")
-                if sha256_file(source) != operation["expected_sha256"]:
-                    raise PreconditionFailed(
-                        "source_changed", f"{source} changed; it is not ours to remove"
-                    )
-                source.unlink()
-            if source.exists():
-                raise PreconditionFailed("removal_failed", f"{source} is still present")
-            # The dangerous window: active storage no longer holds the file while the
-            # database still points at it.
-            maybe_fault(SOURCE_REMOVED)
-            self._record_archived(operation, location, destination)
-            self.journal.transition(operation["id"], ArchiveState.COMPLETE, fencing_token=token)
-            maybe_fault(ArchiveState.COMPLETE)
-
-    def _recheck(self, operation: dict, source: Path, destination: Path, location: dict) -> None:
-        if not source.exists():
-            raise PreconditionFailed("source_missing", f"source {source} disappeared")
-        if source.is_symlink() or destination.is_symlink():
-            raise PreconditionFailed("symlink", "refusing to archive through a symlink")
-        if destination.exists():
-            raise PreconditionFailed("destination_exists", f"destination {destination} is occupied")
-        root = Path(location["root"])
-        if root not in destination.parents:
-            raise PreconditionFailed(
-                "destination_escape", f"{destination} is outside the archive location {root}"
-            )
-        if not root.is_dir() or not (root / MARKER_NAME).exists():
-            raise PreconditionFailed("location_offline", f"{root} is not the archive medium")
-        if sha256_file(source) != operation["expected_sha256"]:
-            raise PreconditionFailed(
-                "source_changed", f"{source} changed since the plan was approved"
-            )
-        with self._session_factory() as session:
-            asset = session.get(Asset, operation["asset_id"])
-            if asset is None or asset.current_path != str(source):
-                raise PreconditionFailed(
-                    "asset_moved", f"asset {operation['asset_id']} is no longer at {source}"
-                )
-
-    def _require_evidence(self, asset_id: str, source: Path) -> dict:
-        """Review evidence must be durable before the original goes.
-
-        The perceptual hash keeps the asset in the fuzzy index once its bytes are
-        unreachable, and the protected preview is what duplicate review can still
-        look at. Both are read from the freshly verified archive copy, which holds
-        exactly the bytes being archived. A file that cannot be decoded has neither
-        — recorded, not fatal, since its exact hashes remain — but failing to
-        produce a preview from a decodable original stops the removal (concept §9).
-        """
-        DuplicateService(self._session_factory).ensure_phash(asset_id, source=source)
-        preview = ThumbnailService(self._session_factory, self._config).ensure_protected(
-            asset_id, source=source
-        )
-        if preview["state"] == "unavailable":
-            raise PreconditionFailed(
-                "preview_unavailable",
-                f"no durable comparison preview for asset {asset_id} "
-                f"({preview['error_code']})",
-            )
-        return preview
-
-    # ── database ──────────────────────────────────────────────────────────────
-
-    def _record_archived(self, operation: dict, location: dict, destination: Path) -> None:
-        """The original is gone from active storage: drop ``current_path``, close its
-        occurrence, record where the bytes now live, and set availability."""
-        now = _now()
-        with self._session_factory() as session:
-            asset = session.get(Asset, operation["asset_id"])
-            if asset is None:
-                raise PreconditionFailed(
-                    "asset_missing", f"asset {operation['asset_id']} no longer exists"
-                )
-            if asset.current_path:
-                for row in session.scalars(
-                    select(AssetPath).where(
-                        AssetPath.asset_id == asset.id,
-                        AssetPath.path == asset.current_path,
-                        AssetPath.valid_until.is_(None),
-                    )
-                ):
-                    row.valid_until = now
-            recorded = session.scalar(
-                select(AssetPath).where(
-                    AssetPath.asset_id == asset.id, AssetPath.path == str(destination)
-                )
-            )
-            if recorded is None:  # idempotent: recovery may replay this
-                session.add(
-                    AssetPath(
-                        asset_id=asset.id,
-                        path=str(destination),
-                        valid_from=now,
-                        reason="archive",
-                    )
-                )
-            asset.current_path = None
-            asset.availability_state = (
-                "archived_online" if destination.exists() else "archived_offline"
-            )
-            asset.archive_location_id = location["id"]
-            asset.archive_path = operation["archive_path"]
-            asset.state_version += 1
-            asset.updated_at = now
-            session.commit()
-
-    # ── recovery ──────────────────────────────────────────────────────────────
-
-    def recover(self, *, worker_id: str = "archive-recovery") -> dict:
-        """Resolve every incomplete item from journal + disk evidence.
-
-        Idempotent: running it repeatedly converges. Ambiguous (``manual``) work is
-        left exactly as found and keeps blocking unrelated mutations.
-        """
-        results = {"resumed": 0, "completed": 0, "manual": 0}
-        touched: set[str] = set()
-        for verdict in self.journal.classify_all(direction=ARCHIVE):
-            operation = self.journal.get(verdict["operation_id"])
-            touched.add(operation["plan_id"])
-            token = (operation["fencing_token"] or 0) + 1
-            if verdict["classification"] == MANUAL:
-                results["manual"] += 1
-                continue
-            if verdict["classification"] == RESUMABLE:
-                # Nothing was published: discard the debris and let a later apply
-                # retry the item cleanly.
-                _clean_temp_files(Path(operation["destination_path"]).parent)
-                self.journal.transition(operation["id"], ArchiveState.PLANNED, fencing_token=token)
-                results["resumed"] += 1
-                continue
-            location = self._location(self._require_plan(operation["plan_id"])["location_id"])
-            try:
-                self._finish(operation, location, token=token, worker_id=worker_id)
-                results["completed"] += 1
-            except PreconditionFailed as error:
-                self._fail(operation, token, error.code, str(error))
-                results["manual"] += 1
-        for plan_id in touched:
-            self._prune_empty_sources(plan_id)
-            self.journal.sync_plan_state(plan_id)
-        return results
-
-    def recovery_status(self) -> dict:
-        verdicts = self.journal.classify_all(direction=ARCHIVE)
-        return {
-            "operations": verdicts,
-            "manual": [v for v in verdicts if v["classification"] == MANUAL],
-            "blocks_mutation": self.journal.blocks_mutation(),
-        }
-
-    # ── helpers ───────────────────────────────────────────────────────────────
-
-    def _fail(self, operation: dict, token: int, code: str, message: str) -> None:
-        self.journal.transition(
-            operation["id"], ArchiveState.FAILED, fencing_token=token, error=(code, message)
-        )
-
-    def _require_plan(self, plan_id: str) -> dict:
-        with self._session_factory() as session:
-            plan = session.get(ArchivePlan, plan_id)
-            if plan is None:
-                raise ArchiveError("unknown_plan", f"unknown archive plan {plan_id!r}")
-            return _plan_dict(plan)
-
-    def _location(self, location_id: str) -> dict:
-        with self._session_factory() as session:
-            location = session.get(ArchiveLocation, location_id)
-            if location is None:
-                raise ArchiveError("unknown_location", f"unknown archive location {location_id!r}")
-            return {"id": location.id, "root": location.root, "media_id": location.media_id}
-
-    def _claim_plan(self, plan_id: str) -> int:
-        """Bump the plan version and use it as this attempt's fencing token, so a
-        worker from a superseded attempt cannot commit."""
-        with self._session_factory() as session:
-            plan = session.get(ArchivePlan, plan_id)
-            plan.version += 1
-            plan.state = "applying"
-            plan.updated_at = _now()
-            token = plan.version
-            session.commit()
-        return token
-
-
-# ── module helpers ───────────────────────────────────────────────────────────
-
-
-def copy_verify_publish(source: Path, destination: Path, expected_sha256: str) -> None:
-    """Copy to a temporary file beside the destination, prove its bytes, then publish
-    it atomically. The source is never touched, so a failure costs nothing.
-
-    Shared by archiving (library → medium) and restoring (medium → library, US06-04):
-    both need the same promise that a published file is either complete and correct
-    or not there at all.
-    """
-    temp = destination.with_name(f"{TEMP_PREFIX}{uuid.uuid4().hex}{TEMP_SUFFIX}")
-    try:
-        with open(source, "rb") as src, open(temp, "wb") as out:
-            shutil.copyfileobj(src, out, 1024 * 1024)
-            out.flush()
-            os.fsync(out.fileno())
-        if sha256_file(temp) != expected_sha256:
-            raise PreconditionFailed("copy_mismatch", f"{source} copied with wrong bytes")
-        if destination.exists():
-            raise PreconditionFailed(
-                "destination_exists", f"{destination} appeared during the transfer"
-            )
-        # ponytail: rename after an exists() check. The archiver lane is single and
-        # local; use O_EXCL/link-based publish if a second writer ever exists.
-        os.rename(temp, destination)
-    finally:
-        temp.unlink(missing_ok=True)
-
-
-def _same_filesystem(source: Path, destination_dir: Path) -> bool:
-    """Proven at run time from the actual devices, never from the plan's preview."""
-    try:
-        return source.stat().st_dev == destination_dir.stat().st_dev
-    except OSError:
-        return False
-
-
-def _fsync_dir(path: Path) -> None:
-    """Make the directory entry itself durable, so the published name survives a
-    power loss and not just the file's data."""
-    fd = os.open(path, os.O_RDONLY)
-    try:
-        os.fsync(fd)
-    except OSError:
-        pass  # some filesystems refuse directory fsync; the data is already synced
-    finally:
-        os.close(fd)
-
-
-def _manifest_entry(operation: dict, location: dict) -> dict:
-    return {
-        "schema_version": MANIFEST_VERSION,
-        "plan_id": operation["plan_id"],
-        "asset_id": operation["asset_id"],
-        "album": operation["album"],
-        "archive_path": operation["archive_path"],
-        "source_path": operation["source_path"],
-        "sha256": operation["expected_sha256"],
-        "byte_size": operation["byte_size"],
-        "media_id": location["media_id"],
-        "archived_at": _now().isoformat(),
-    }
-
-
-def _append_manifest(directory: Path, entry: dict) -> None:
-    """Append one durable manifest line, skipping an entry that is already there.
-
-    The manifest is written before the source is removed, so it is the medium's own
-    record of what it holds even if the database is lost.
-    """
-    path = directory / MANIFEST_NAME
-    # ponytail: rereads the album manifest per file (O(n²) lines for one album).
-    # Keep an in-memory index per plan if an album ever holds enough files to matter.
-    if path.exists():
-        for line in path.read_text(encoding="utf-8").splitlines():
-            try:
-                existing = json.loads(line)
-            except ValueError:
-                continue
-            if (existing.get("asset_id"), existing.get("sha256")) == (
-                entry["asset_id"],
-                entry["sha256"],
-            ):
-                return
-    with open(path, "a", encoding="utf-8") as handle:
-        handle.write(json.dumps(entry, sort_keys=True) + "\n")
-        handle.flush()
-        os.fsync(handle.fileno())
-    _fsync_dir(directory)
-
-
-def read_manifest(directory: Path) -> list[dict]:
-    """Every manifest entry an archived album directory holds."""
-    path = directory / MANIFEST_NAME
-    if not path.exists():
-        return []
-    entries = []
-    for line in path.read_text(encoding="utf-8").splitlines():
-        try:
-            entries.append(json.loads(line))
-        except ValueError:
-            continue
-    return entries
-
-
-def _clean_temp_files(directory: Path) -> None:
-    """Remove this application's own abandoned transfer temporaries — never any
-    other file (concept §17: startup cleans only recognised stale temporaries)."""
-    if not directory.is_dir():
-        return
-    for temp in directory.glob(f"{TEMP_PREFIX}*{TEMP_SUFFIX}"):
-        temp.unlink(missing_ok=True)
-
-
-def _plan_dict(plan: ArchivePlan) -> dict:
-    return {
-        "id": plan.id,
-        "location_id": plan.location_id,
-        "token": plan.token,
-        "direction": plan.direction,
-        "albums": json.loads(plan.albums) if plan.albums else None,
-        "state": plan.state,
-        "schema_version": plan.schema_version,
-        "asset_count": plan.asset_count,
-        "byte_size": plan.byte_size,
-        "version": plan.version,
-        "worker_id": plan.worker_id,
-        "completed_at": plan.completed_at.isoformat() if plan.completed_at else None,
-        "created_at": plan.created_at.isoformat() if plan.created_at else None,
-    }

photo_pipeline/services/archives.py

@@ -24,15 +24,10 @@ Preflight proves, per concept §9 "Archive preflight":
   probed by writing them, not assumed.
 
 Blocker codes: ``no_library_root``, ``location_offline``, ``wrong_volume``,
-``archive_pending``,
 ``unsafe_destination``, ``destination_not_writable``, ``manifest_unwritable``,
 ``insufficient_capacity``, ``backup_unavailable``, ``lock_conflict``,
 ``rename_pending``, ``empty_scope``, ``destination_collision``,
-``upload_unverified``, ``bytes_changed``, ``file_missing``, ``preview_unavailable``.
+``upload_unverified``, ``bytes_changed``, ``file_missing``.
-
-Preflight also *creates* the durable comparison preview of every asset in scope
-(US06-03): it is the evidence duplicate review falls back on once the original is
-on a medium that may be offline, so it has to exist before the original leaves.
 
 Like upload preflight, the confirmation token is *derived* from the report rather
 than stored: any change to the scope, the bytes, the destination, or the blockers
@@ -62,17 +57,14 @@ from photo_pipeline.jobs.domain_handlers import ARCHIVE_LOCK, LIBRARY_WRITE_LOCK
 from photo_pipeline.models import ArchiveLocation, Asset, UploadBatch, UploadItem
 from photo_pipeline.path_policy import PathPolicyError, is_excluded, normalize_root, resolve_within
 from photo_pipeline.services.albums import album_label
-from photo_pipeline.services.archive_journal import ArchiveJournal
-from photo_pipeline.services.availability import MARKER_NAME, read_marker as _read_marker
-from photo_pipeline.services.availability import refresh as refresh_availability
 from photo_pipeline.services.hashing import sha256_file
 from photo_pipeline.services.jobs import JobService
 from photo_pipeline.services.rename_journal import RenameJournal
-from photo_pipeline.services.thumbnails import ThumbnailService
 from photo_pipeline.services.upload_reports import VERIFIED
 
 PREFLIGHT_VERSION = 1
 TOKEN_PREFIX = f"v{PREFLIGHT_VERSION}"
+MARKER_NAME = ".photo-pipeline-archive.json"
 MANIFEST_NAME = "archive-manifest.json"
 
 # Upload outcomes that prove Immich holds these exact bytes. ``skipped``/``failed``/
@@ -169,9 +161,6 @@ class ArchiveService:
                     location.capabilities = json.dumps(probe["capabilities"])
                 reports.append(self._location_report(location, probe=probe))
             session.commit()
-        # A medium that just appeared or vanished changes what is readable, so the
-        # archived assets are re-derived from the same probe (US06-03).
-        refresh_availability(self._session_factory)
             return reports
 
     # ── preflight ─────────────────────────────────────────────────────────────
@@ -208,12 +197,9 @@ class ArchiveService:
             report["blockers"].append(
                 _issue(
                     "insufficient_capacity",
-                    # The free-space number is deliberately left out: it drifts between
-                    # two identical preflights, and the token is a digest of this text,
-                    # so quoting it here would invalidate every approval instantly.
                     f"{report['totals']['bytes']} B plus a "
-                    f"{self._config.archive_free_space_reserve_bytes} B reserve do not fit on "
+                    f"{self._config.archive_free_space_reserve_bytes} B reserve do not fit in "
-                    "the medium",
+                    f"{report['capacity']['free_bytes']} B of free space",
                 )
             )
         report["backup"] = self._backup_probe()
@@ -305,13 +291,6 @@ class ArchiveService:
             blockers.append(
                 _issue("rename_pending", "an unresolved rename must be recovered before archiving")
             )
-        if ArchiveJournal(self._session_factory).blocks_mutation():
-            blockers.append(
-                _issue(
-                    "archive_pending",
-                    "an unresolved archive transfer must be recovered before archiving again",
-                )
-            )
         return blockers
 
     def _capacity(self, required: int, probe: dict) -> dict:
@@ -432,10 +411,7 @@ class ArchiveService:
 
     def _album(self, name: str, rows: list[dict], root: Path, *, reachable: bool) -> dict:
         folder = Path(rows[0]["path"]).parent
-        items = sorted(
+        items = sorted((_item(row) for row in rows), key=lambda item: item["current_path"])
-            (self._with_preview(_item(row)) for row in rows),
-            key=lambda item: item["current_path"],
-        )
         blocked = [item for item in items if item["blockers"]]
         blockers: list[dict] = []
 
@@ -471,30 +447,6 @@ class ArchiveService:
             "assets": items,
         }
 
-    def _with_preview(self, item: dict) -> dict:
-        """Create the durable comparison preview while the original is still here.
-
-        This is the last moment it can be made: once the file is archived and the
-        medium leaves, only the retained preview can answer "is this new photo the
-        same picture?". An original that cannot be decoded at all has no preview to
-        keep — its hashes and metadata stay the evidence — but a preview that fails
-        for any other reason blocks the archive (concept §9).
-        """
-        preview = self._previews().ensure_protected(item["asset_id"])
-        item["preview"] = preview
-        if preview["state"] == "unavailable" and not item["blockers"]:
-            item["blockers"].append(
-                _issue(
-                    "preview_unavailable",
-                    f"a durable comparison preview of {item['current_path']} could not be "
-                    f"created ({preview['error_code']})",
-                )
-            )
-        return item
-
-    def _previews(self) -> ThumbnailService:
-        return ThumbnailService(self._session_factory, self._config)
-
 
 # ── internals ────────────────────────────────────────────────────────────────
 
@@ -551,6 +503,13 @@ def _transfer_method(folder: Path, root: Path) -> str:
     return "copy_verify_remove"
 
 
+def _read_marker(root: Path) -> dict | None:
+    try:
+        return json.loads((root / MARKER_NAME).read_text(encoding="utf-8"))
+    except (OSError, ValueError):
+        return None
+
+
 def _probe_write(path: Path, payload: bytes, *, keep: bool = False) -> str | None:
     """Write ``payload`` to ``path``; return the failure detail or ``None``."""
     try:

photo_pipeline/services/availability.py

@@ -1,124 +0,0 @@
-"""Where an asset's bytes are right now (US06-03).
-
-Archiving removes the original from the active library but never removes the
-asset: its identity, hashes, decisions, and evidence stay. This module is the one
-place that answers "can these bytes be read, and if not, why" so inventory,
-duplicate review, thumbnails, and the archive service all give the same answer.
-
-States (concept §9):
-
-- ``active``             — the original is in the active library;
-- ``archived_online``    — the recorded medium is mounted and holds the file;
-- ``archived_offline``   — archived, but the medium is not available right now;
-- ``missing_unexpected`` — neither an active path nor the recorded archive
-  location explains the absence. This is the state that must never be confused
-  with ``archived_offline``: an unmounted disk is normal, a mounted disk with a
-  hole in it is not.
-
-A medium is identified by its marker file, never by its mountpoint, so a
-different disk mounted at the recorded root is offline rather than accepted.
-"""
-
-from __future__ import annotations
-
-import json
-from collections import Counter
-from datetime import datetime, timezone
-from pathlib import Path
-
-from sqlalchemy import select
-from sqlalchemy.orm import Session, sessionmaker
-
-from photo_pipeline.models import ArchiveLocation, Asset
-
-ACTIVE = "active"
-ARCHIVED_ONLINE = "archived_online"
-ARCHIVED_OFFLINE = "archived_offline"
-MISSING_UNEXPECTED = "missing_unexpected"
-ARCHIVED = (ARCHIVED_ONLINE, ARCHIVED_OFFLINE)
-
-MARKER_NAME = ".photo-pipeline-archive.json"
-
-
-def read_marker(root: Path) -> dict | None:
-    """The medium's identity marker, or ``None`` when it is not readable."""
-    try:
-        return json.loads((root / MARKER_NAME).read_text(encoding="utf-8"))
-    except (OSError, ValueError):
-        return None
-
-
-def location_online(location: ArchiveLocation) -> bool:
-    """True only when the *recorded* medium is mounted at its root."""
-    marker = read_marker(Path(location.root))
-    return bool(marker) and marker.get("media_id") == location.media_id
-
-
-def archive_file(session: Session, asset: Asset) -> Path | None:
-    """The archived file's absolute path, whether or not the medium is mounted."""
-    if not asset.archive_location_id or not asset.archive_path:
-        return None
-    location = session.get(ArchiveLocation, asset.archive_location_id)
-    if location is None:
-        return None
-    return Path(location.root) / asset.archive_path
-
-
-def readable_path(session: Session, asset: Asset) -> Path | None:
-    """A path whose bytes can be read now: the active file, else the archive copy."""
-    if asset.current_path and Path(asset.current_path).exists():
-        return Path(asset.current_path)
-    archived = archive_file(session, asset)
-    if archived is None:
-        return None
-    location = session.get(ArchiveLocation, asset.archive_location_id)
-    if not location_online(location) or not archived.exists():
-        return None
-    return archived
-
-
-def state_of(session: Session, asset: Asset, *, online: dict[str, bool] | None = None) -> str:
-    """The availability this asset's storage actually justifies right now."""
-    if asset.current_path:
-        return ACTIVE if Path(asset.current_path).exists() else MISSING_UNEXPECTED
-    if not asset.archive_location_id:
-        return MISSING_UNEXPECTED if asset.availability_state != ACTIVE else ACTIVE
-    location = session.get(ArchiveLocation, asset.archive_location_id)
-    if location is None:
-        return MISSING_UNEXPECTED
-    reachable = (
-        online[location.id] if online and location.id in online else location_online(location)
-    )
-    if not reachable:
-        return ARCHIVED_OFFLINE
-    archived = archive_file(session, asset)
-    # The medium is mounted and identified: the file is either there, or it is
-    # genuinely gone — that is not "offline", it needs a human.
-    return ARCHIVED_ONLINE if archived and archived.exists() else MISSING_UNEXPECTED
-
-
-def refresh(session_factory: sessionmaker) -> dict[str, int]:
-    """Re-derive availability for every archived asset from the media themselves.
-
-    Only archived assets are probed: whether an *active* file is present is the
-    inventory scan's job and costs one stat per library file. Each medium is
-    probed once, not once per asset.
-    """
-    counts: Counter[str] = Counter()
-    now = datetime.now(timezone.utc)
-    with session_factory() as session:
-        online = {
-            location.id: location_online(location)
-            for location in session.scalars(select(ArchiveLocation))
-        }
-        for asset in session.scalars(
-            select(Asset).where(Asset.archive_location_id.is_not(None))
-        ):
-            state = state_of(session, asset, online=online)
-            counts[state] += 1
-            if state != asset.availability_state:
-                asset.availability_state = state
-                asset.state_version += 1
-                asset.updated_at = now
-        session.commit()
-    return dict(counts)

photo_pipeline/services/duplicates.py

@@ -11,12 +11,6 @@ Detection runs in two categories:
   band (NEAR/SIMILAR). These are review candidates: never decided automatically, and
   negative-linked pairs are suppressed so a rejected pair is not re-suggested.
 
-Archived assets stay in both indexes (US06-03): a new active copy of an archived
-original is recognised through its hashes even while the medium is offline, and
-cluster review falls back to the retained protected preview plus hash evidence.
-An exact/pixel match links straight to the archived canonical; a perceptual match
-is a review candidate that names the medium to mount for a pixel-level decision.
-
 Decisions (``canonical`` / ``not_duplicate`` / ``deferred``) persist with evidence,
 use optimistic version checks, are reversible, and can never form a canonical cycle.
 A new content-identical member of an already-decided cluster inherits the established
@@ -40,14 +34,12 @@ from sqlalchemy import func, select
 from sqlalchemy.orm import sessionmaker
 
 from photo_pipeline.models import (
-    ArchiveLocation,
     Asset,
     DuplicateCluster,
     DuplicateMember,
     DuplicateNegativeLink,
-    Thumbnail,
 )
-from photo_pipeline.services import availability, hashing
+from photo_pipeline.services import hashing
 
 NEAR_MAX = 5
 SIMILAR_MAX = 10
@@ -135,18 +127,18 @@ class DuplicateService:
 
     # ── perceptual hash backfill ───────────────────────────────────────────
     def ensure_phashes(self) -> int:
-        """Hash whatever is readable now — an archived asset keeps the hash it
-        already has, and gains one whenever its medium happens to be mounted."""
         updated = 0
         with self._session_factory() as session:
-            assets = session.execute(select(Asset)).scalars()
+            assets = session.execute(
+                select(Asset).where(
+                    Asset.availability_state == "active",
+                    Asset.current_path.isnot(None),
+                )
+            ).scalars()
             for asset in assets:
                 if asset.phash is not None and asset.phash_version == hashing.PHASH_VERSION:
                     continue
-                source = availability.readable_path(session, asset)
+                value = hashing.safe_phash(asset.current_path)
-                if source is None:
-                    continue
-                value = hashing.safe_phash(str(source))
                 if value is not None:
                     asset.phash = value
                     asset.phash_version = hashing.PHASH_VERSION
@@ -154,40 +146,20 @@ class DuplicateService:
             session.commit()
         return updated
 
-    def ensure_phash(self, asset_id: str, *, source=None) -> str | None:
-        """Backfill one asset's perceptual hash while its bytes are still readable.
-
-        Archiving calls this before the original leaves — passing the archive copy
-        as ``source``, since the database does not point at it yet — because an
-        asset without a pHash silently drops out of the fuzzy index the moment its
-        medium is away.
-        """
-        with self._session_factory() as session:
-            asset = session.get(Asset, asset_id)
-            if asset is None:
-                return None
-            if asset.phash is not None and asset.phash_version == hashing.PHASH_VERSION:
-                return asset.phash
-            source = source or availability.readable_path(session, asset)
-            if source is None:
-                return None
-            value = hashing.safe_phash(str(source))
-            if value is not None:
-                asset.phash = value
-                asset.phash_version = hashing.PHASH_VERSION
-                session.commit()
-            return value
-
     # ── detection ──────────────────────────────────────────────────────────
     def detect(self) -> DetectionReport:
         self.ensure_phashes()
         now = datetime.now(timezone.utc)
         report = DetectionReport()
         with self._session_factory() as session:
-            # Every known asset stays in the indexes, archived or not: a copy of an
+            assets = list(
-            # archived original must be recognised as a duplicate rather than
+                session.execute(
-            # treated as a new photo (concept §9, invariant 12).
+                    select(Asset).where(
-            assets = list(session.execute(select(Asset)).scalars())
+                        Asset.availability_state == "active",
+                        Asset.current_path.isnot(None),
+                    )
+                ).scalars()
+            )
             by_id = {a.id: a for a in assets}
             negatives = {
                 _pair(link.asset_a, link.asset_b)
@@ -459,19 +431,10 @@ class DuplicateService:
 
     @staticmethod
     def _recommend_canonical(ids, by_id) -> str:
-        # ponytail: largest file, then the archived copy, then path as a
+        # ponytail: largest file, path as deterministic tie-break. The concept's
-        # deterministic tie-break. Archived wins ties because it is the reviewed,
+        # richer policy (resolution, least recompression, metadata richness) lands
-        # uploaded original — a fresh active copy must not demote it to a variant.
+        # with the review UI story.
-        # The concept's richer policy (resolution, least recompression, metadata
+        return max(ids, key=lambda i: (by_id[i].byte_size or 0, by_id[i].current_path or ""))
-        # richness) lands with the review UI story.
-        return max(
-            ids,
-            key=lambda i: (
-                by_id[i].byte_size or 0,
-                by_id[i].availability_state in availability.ARCHIVED,
-                by_id[i].current_path or by_id[i].archive_path or "",
-            ),
-        )
 
     def _apply_canonical(self, session, cluster, ids, canonical_id):
         for member in session.execute(
@@ -544,15 +507,9 @@ class DuplicateService:
                         "current_path": asset.current_path if asset else None,
                         "byte_size": asset.byte_size if asset else None,
                         "phash": asset.phash if asset else None,
-                        **self._offline_evidence(session, asset),
                     }
                 )
             members.sort(key=lambda m: m["asset_id"])
-            # A full-resolution comparison of an offline original is impossible; the
-            # UI asks for that named medium instead of guessing (concept §9).
-            mount_required = sorted(
-                {m["archive_location"] for m in members if m["requires_mount"]}
-            )
             return {
                 "id": cluster.id,
                 "method": cluster.method,
@@ -562,54 +519,9 @@ class DuplicateService:
                 "canonical_asset_id": cluster.canonical_asset_id,
                 "version": cluster.version,
                 "requires_confirmation": cluster.method == Method.PERCEPTUAL.value,
-                "mount_required": mount_required,
                 "members": members,
             }
 
-    def _offline_evidence(self, session, asset: Asset | None) -> dict:
-        """What review can still rely on when a member's original is not readable."""
-        if asset is None:
-            return {
-                "availability_state": None,
-                "archive_location": None,
-                "archive_location_id": None,
-                "archive_path": None,
-                "preview": {"state": "missing", "protected": False},
-                "requires_mount": False,
-            }
-        location = (
-            session.get(ArchiveLocation, asset.archive_location_id)
-            if asset.archive_location_id
-            else None
-        )
-        preview = self._preview_evidence(session, asset.id)
-        archived = asset.availability_state in availability.ARCHIVED
-        return {
-            "availability_state": asset.availability_state,
-            "archive_location": location.name if location else None,
-            "archive_location_id": asset.archive_location_id,
-            "archive_path": asset.archive_path,
-            "preview": preview,
-            # Offline archived members can still be compared through their retained
-            # preview and hash evidence; only pixel-level review needs the medium.
-            "requires_mount": archived
-            and asset.availability_state == availability.ARCHIVED_OFFLINE
-            and bool(location),
-        }
-
-    @staticmethod
-    def _preview_evidence(session, asset_id: str) -> dict:
-        rows = list(
-            session.execute(select(Thumbnail).where(Thumbnail.asset_id == asset_id)).scalars()
-        )
-        ready = [r for r in rows if r.state == "ready" and r.path]
-        if ready:
-            best = max(ready, key=lambda r: (bool(r.protected), r.size or 0))
-            return {"state": "ready", "protected": bool(best.protected), "size": best.size}
-        if rows:
-            return {"state": "unsupported", "protected": False, "size": rows[0].size}
-        return {"state": "missing", "protected": False, "size": None}
-
     # ── decisions ────────────────────────────────────────────────────────────
     def decide(
         self,

photo_pipeline/services/exif_checkpoint.py

@@ -1,178 +0,0 @@
-"""The EXIF checkpoint every metadata stage ends with (concept §3, US07-03).
-
-A stage does not own a file's metadata; it owns a few fields in it. So writing is
-never "set these tags" — it is:
-
-    snapshot everything → write only the owned fields → read everything back
-    → prove the owned fields landed → prove nothing else moved
-    → refresh the file hash → record the projection
-
-Non-destructive here means *semantic* preservation: exiftool may rewrite the whole
-container, so the file's bytes, size, and timestamps legitimately change. What may
-not change is any field this stage does not own. When one does, the checkpoint is
-``divergent``: the result is recorded, the stage is **not** marked verified, and
-nothing is silently repaired — a later stage that needs verified metadata (upload)
-therefore stays blocked until a human looks.
-
-``failed`` is the third outcome and is deliberately distinct: exiftool missing, an
-unreadable file, or a write that did not take is not evidence that metadata is fine.
-"""
-
-from __future__ import annotations
-
-import json
-import uuid
-from dataclasses import dataclass
-from datetime import datetime, timezone
-
-from photo_pipeline.faults import EXIF_WRITTEN, maybe_fault
-from photo_pipeline.integrations import exiftool
-from photo_pipeline.models import ExifProjection
-from photo_pipeline.services import hashing
-
-# The tags the safety and analysis stages may change. Matched on the tag name, so
-# every group carries the same rule (IPTC:Keywords, XMP:XMP-dc:Subject, ...).
-OWNED_TAGS = frozenset({"Keywords", "Subject"})
-
-# Not metadata about the picture: filesystem facts, the digest that necessarily
-# moves whenever IPTC does, and the structural tags exiftool has to create the first
-# time it writes an IPTC or XMP block. Comparing these would report every write as
-# divergent and make the signal worthless.
-VOLATILE_PREFIXES = ("File:System:", "ExifTool:")
-VOLATILE_KEYS = frozenset(
-    {
-        "File:CurrentIPTCDigest",
-        "IPTC:ApplicationRecordVersion",
-        "XMP:XMP-x:XMPToolkit",
-        "XMP:XMP-xmp:MetadataDate",
-    }
-)
-
-VERIFIED = "verified"
-DIVERGENT = "divergent"
-FAILED = "failed"
-
-
-@dataclass(frozen=True)
-class CheckpointResult:
-    state: str  # verified | divergent | failed
-    changed_fields: tuple[str, ...] = ()
-    sha256: str | None = None
-    verified_at: datetime | None = None
-    reason: str | None = None
-
-    @property
-    def verified(self) -> bool:
-        return self.state == VERIFIED
-
-
-def _now() -> datetime:
-    return datetime.now(timezone.utc)
-
-
-def is_owned(key: str) -> bool:
-    return key.rsplit(":", 1)[-1] in OWNED_TAGS
-
-
-def is_volatile(key: str) -> bool:
-    return key in VOLATILE_KEYS or key.startswith(VOLATILE_PREFIXES)
-
-
-def compare(before: dict, after: dict) -> tuple[str, ...]:
-    """Fields outside this stage's ownership whose value did not survive the write.
-
-    Additions count: a tag that appears out of nowhere is as much a divergence as a
-    tag that disappeared — both mean the write did more than it was asked to.
-    """
-    keys = set(before) | set(after)
-    return tuple(
-        sorted(
-            key
-            for key in keys
-            if not is_owned(key)
-            and not is_volatile(key)
-            and before.get(key) != after.get(key)
-        )
-    )
-
-
-def owned_values(snapshot: dict) -> set[str]:
-    """Lowercased Keywords/Subject values across every group in the snapshot."""
-    values: set[str] = set()
-    for key, value in snapshot.items():
-        if not is_owned(key):
-            continue
-        items = value if isinstance(value, list) else [value]
-        values.update(str(item).strip().lower() for item in items if item is not None)
-    return values
-
-
-def run(
-    path: str,
-    *,
-    add: tuple[str, ...] = (),
-    remove: tuple[str, ...] = (),
-) -> CheckpointResult:
-    """Write the owned keywords for one asset and verify the whole file around them."""
-    before = exiftool.read_all(path)
-    if before is None:
-        return CheckpointResult(FAILED, reason="metadata_unreadable")
-
-    if not exiftool.apply_keywords(path, add=add, remove=remove):
-        return CheckpointResult(FAILED, reason="write_failed")
-
-    # The file on disk has changed; nothing about it is recorded yet. A crash here
-    # is the worst case for metadata, so it is a fault control point (US07-04).
-    maybe_fault(EXIF_WRITTEN)
-
-    after = exiftool.read_all(path)
-    if after is None:
-        return CheckpointResult(FAILED, reason="readback_unreadable")
-
-    present = owned_values(after)
-    wanted = {value.strip().lower() for value in add}
-    unwanted = {value.strip().lower() for value in remove}
-    if not wanted <= present or (unwanted & present):
-        return CheckpointResult(FAILED, reason="owned_fields_not_written")
-
-    changed = compare(before, after)
-    sha256 = hashing.sha256_file(path)
-    if changed:
-        return CheckpointResult(DIVERGENT, changed_fields=changed, sha256=sha256)
-    return CheckpointResult(VERIFIED, sha256=sha256, verified_at=_now())
-
-
-def record(
-    session_factory,
-    *,
-    asset_id: str,
-    stage: str,
-    result: CheckpointResult,
-    add: tuple[str, ...] = (),
-    remove: tuple[str, ...] = (),
-) -> None:
-    """Persist the projection for ``(asset_id, stage)`` — one current row per pair.
-
-    The row is what makes divergence durable and reviewable rather than a log line
-    that scrolled away.
-    """
-    with session_factory() as session:
-        row = session.get(ExifProjection, (asset_id, stage))
-        if row is None:
-            row = ExifProjection(asset_id=asset_id, stage=stage, id=str(uuid.uuid4()))
-            session.add(row)
-        row.projection_version = (row.projection_version or 0) + 1
-        row.desired_json = json.dumps({"add": list(add), "remove": list(remove)})
-        row.divergent_fields = json.dumps(list(result.changed_fields))
-        row.result_file_sha256 = result.sha256
-        row.state = result.state
-        row.error_code = result.reason
-        row.verified_at = result.verified_at
-        row.updated_at = _now()
-        session.commit()
-
-
-def state_for(session_factory, asset_id: str, stage: str) -> str | None:
-    with session_factory() as session:
-        row = session.get(ExifProjection, (asset_id, stage))
-        return row.state if row else None

photo_pipeline/services/hashing.py

@@ -16,9 +16,7 @@ from __future__ import annotations
 import hashlib
 from pathlib import Path
 
-from PIL import ImageOps
+from PIL import Image, ImageOps
-
-from photo_pipeline import imaging
 
 PIXEL_HASH_VERSION = 1
 PHASH_VERSION = 1
@@ -43,9 +41,7 @@ def _digest_file(path: Path | str, digest) -> str:
 
 
 def pixel_sha256(path: Path | str) -> str:
-    # Bounded decode: a header claiming a billion pixels is refused before it is
+    with Image.open(path) as image:
-    # ever turned into memory (US07-03).
-    with imaging.open_image(path) as image:
         oriented = ImageOps.exif_transpose(image)
         rgb = oriented.convert("RGB")
         header = f"{PIXEL_HASH_VERSION}:{rgb.width}x{rgb.height}:".encode()
@@ -72,9 +68,7 @@ def phash(path: Path | str) -> str:
     import numpy as np
     from scipy.fftpack import dct
 
-    from PIL import Image
+    with Image.open(path) as image:
-
-    with imaging.open_image(path) as image:
         small = image.convert("L").resize((32, 32), Image.LANCZOS)
         matrix = np.asarray(small, dtype=np.float64)
     transformed = dct(dct(matrix, axis=0), axis=1)

photo_pipeline/services/inventory.py

@@ -13,11 +13,8 @@ renames. Every discovered or absent path is classified as one occurrence:
 - ``missing``          — a known active asset whose file is gone (kept, flagged).
 
 Missing files are never pruned (that would break identity); the asset is retained
-with ``missing_at`` set and its availability becomes ``missing_unexpected`` —
+with ``missing_at`` set. Archived assets are left untouched. Rescanning unchanged
-nothing explains where the bytes went. Archived assets are left untouched: their
+input makes no durable change.
-absence from the active roots is expected, and each scan re-derives whether their
-medium is reachable (:mod:`photo_pipeline.services.availability`). Rescanning
-unchanged input makes no durable change.
 
 Extracted from photo_analyzer.discover_photos/reconcile_moved/prune_missing
 (see donor_ledger.yaml: pa-discovery, pa-prune-missing).
@@ -33,12 +30,12 @@ from enum import Enum
 from pathlib import Path
 from typing import Iterable
 
-from sqlalchemy import func, or_, select
+from sqlalchemy import func, select
 from sqlalchemy.orm import Session, sessionmaker
 
 from photo_pipeline import path_policy
 from photo_pipeline.models import Asset, AssetPath
-from photo_pipeline.services import availability, hashing
+from photo_pipeline.services import hashing
 
 
 class Occurrence(str, Enum):
@@ -63,8 +60,6 @@ def _asset_dict(asset: Asset) -> dict:
         "id": asset.id,
         "current_path": asset.current_path,
         "availability_state": asset.availability_state,
-        "archive_location_id": asset.archive_location_id,
-        "archive_path": asset.archive_path,
         "byte_size": asset.byte_size,
         "current_sha256": asset.current_sha256,
         "pixel_sha256": asset.pixel_sha256,
@@ -106,9 +101,7 @@ class InventoryService:
                 result.asset_ids[str(path)] = asset.id
 
             for asset in assets:
-                # Archived assets are explained by their location, not by the active
+                if asset.availability_state != "active" or asset.id in seen_ids:
-                # roots: a scan must never prune or flag them (concept §9).
-                if asset.availability_state in availability.ARCHIVED or asset.id in seen_ids:
                     continue
                 if asset.current_path and asset.current_path not in discovered_paths:
                     if not Path(asset.current_path).exists():
@@ -116,18 +109,10 @@ class InventoryService:
                             asset.missing_at = now
                             asset.state_version += 1
                             asset.updated_at = now
-                        # Nothing explains this absence — it is not an offline medium.
-                        if asset.availability_state != availability.MISSING_UNEXPECTED:
-                            asset.availability_state = availability.MISSING_UNEXPECTED
-                            asset.state_version += 1
-                            asset.updated_at = now
                         result.occurrences[asset.current_path] = Occurrence.MISSING.value
 
             session.commit()
 
-        # Media may have been mounted or removed since the last scan.
-        availability.refresh(self._session_factory)
-
         result.counts = dict(Counter(result.occurrences.values()))
         return result
 
@@ -147,11 +132,7 @@ class InventoryService:
             if availability:
                 stmt = stmt.where(Asset.availability_state == availability)
             if query:
-                like = f"%{query}%"
+                stmt = stmt.where(Asset.current_path.like(f"%{query}%"))
-                # An archived asset has no active path; it is searched where it lives.
-                stmt = stmt.where(
-                    or_(Asset.current_path.like(like), Asset.archive_path.like(like))
-                )
             total = session.scalar(select(func.count()).select_from(stmt.subquery()))
             rows = session.execute(
                 stmt.order_by(Asset.current_path).limit(limit).offset(offset)
@@ -188,7 +169,6 @@ class InventoryService:
                 self._open_path(session, existing.id, path_str, now, occ.value)
             if existing.missing_at is not None:
                 existing.missing_at = None
-                existing.availability_state = availability.ACTIVE
                 existing.state_version += 1
                 existing.updated_at = now
             return existing, occ
@@ -209,7 +189,6 @@ class InventoryService:
             moved_from.current_path = path_str
             moved_from.byte_size = size
             moved_from.missing_at = None
-            moved_from.availability_state = availability.ACTIVE
             moved_from.state_version += 1
             moved_from.updated_at = now
             self._open_path(session, moved_from.id, path_str, now, Occurrence.MOVED.value)

photo_pipeline/services/jobs.py

@@ -236,32 +236,17 @@ class JobService:
                 raise InvalidTransition(f"{job.state} -> {to_state}")
             if worker_id is not None and job.lease_owner not in (None, worker_id):
                 raise JobConflict(f"job {job_id} owned by {job.lease_owner}, not {worker_id}")
-
+            job.state = to_state
-            # Compare-and-set on the version this decision was made against. Without
+            job.version += 1
-            # it a transition validated against a row that has since been claimed,
+            job.updated_at = now
-            # cancelled, or finished would overwrite that newer state (concept §16
-            # database rule 6) — a cancel racing a claim used to un-claim a running
-            # job and leave the worker finalizing a job it no longer owned.
-            values = {
-                "state": to_state,
-                "version": job.version + 1,
-                "updated_at": now,
-            }
             if error:
-                values["error_code"], values["error_message"] = error
+                job.error_code, job.error_message = error
             if to_state in TERMINAL_STATES:
-                values.update(finished_at=now, lease_owner=None, lease_expires_at=None)
+                job.finished_at = now
-            result = session.execute(
+                job.lease_owner = None
-                update(Job).where(Job.id == job_id, Job.version == job.version).values(**values)
+                job.lease_expires_at = None
-            )
-            if result.rowcount != 1:
-                session.rollback()
-                raise JobConflict(
-                    f"job {job_id} changed while transitioning to {to_state}; retry"
-                )
             self._event(session, job_id, f"state:{to_state}", error[1] if error else None)
             session.commit()
-            session.expire_all()  # the core UPDATE bypassed the identity map
             return self._snapshot(session, job_id)
 
     def cancel(self, job_id: str) -> dict:

photo_pipeline/services/legacy_import.py

@@ -1,220 +0,0 @@
-"""Import the last path-keyed CSV state into the database (US07-01).
-
-``nsfwtag`` cached its safety scores in ``nsfw_scores.csv`` next to the library:
-one ``path,nsfw_score`` row per photo, four decimals, unreadable rows dropped. That
-file stops being a source of truth when the CLI is archived, so its scores are
-imported once — as scored-but-unreviewed ``safety_reviews`` rows on the stable
-``assets.id`` each path resolves to — and the CSV is left untouched on disk as its
-own backup.
-
-The import is deliberately conservative, because a score is evidence about a photo
-and a path is not an identity:
-
-- a row whose path matches no known asset is **unmatched**, never a new asset;
-- an asset a human already reviewed is **never** touched: a score is evidence, a
-  decision is a judgement, and an import may not overwrite the second with the
-  first — the difference is reported as a conflict instead;
-- a row for an asset that already carries a score is **skipped** unless
-  ``overwrite`` is asked for, and a differing score is reported either way;
-- an unparsable score or a duplicate path is **rejected** with its reason;
-- rerunning changes nothing that is already imported.
-
-Everything it did — and everything it refused to do — comes back as a
-reconciliation report, which the caller can persist next to the database. The
-donor's own reader is the specification for the format (donor ledger:
-``nt-score-cache``).
-"""
-
-from __future__ import annotations
-
-import csv
-import json
-import uuid
-from collections import Counter
-from dataclasses import asdict, dataclass, field
-from datetime import datetime, timezone
-from pathlib import Path
-
-from sqlalchemy import select
-from sqlalchemy.orm import sessionmaker
-
-from photo_pipeline.models import Asset, AssetPath, SafetyReview
-
-REPORT_VERSION = 1
-# The donor wrote scores as four-decimal strings; anything outside 0..1 was never
-# something it produced, so it is data corruption rather than a score.
-SCORE_RANGE = (0.0, 1.0)
-# Recorded as the reviewer so an imported score is never mistaken for a judgement.
-REVIEWER = "legacy-csv-import"
-
-
-@dataclass
-class ImportReport:
-    source: str
-    rows: int = 0
-    imported: int = 0
-    skipped_existing: int = 0
-    unchanged: int = 0
-    reviewed: int = 0
-    unmatched: list[str] = field(default_factory=list)
-    rejected: list[dict] = field(default_factory=list)
-    conflicts: list[dict] = field(default_factory=list)
-    generated_at: str = ""
-    schema_version: int = REPORT_VERSION
-
-    def to_dict(self) -> dict:
-        return asdict(self)
-
-    @property
-    def counts(self) -> dict[str, int]:
-        return {
-            "rows": self.rows,
-            "imported": self.imported,
-            "skipped_existing": self.skipped_existing,
-            "unchanged": self.unchanged,
-            "reviewed": self.reviewed,
-            "unmatched": len(self.unmatched),
-            "rejected": len(self.rejected),
-            "conflicts": len(self.conflicts),
-        }
-
-
-class LegacyImportService:
-    def __init__(self, session_factory: sessionmaker) -> None:
-        self._session_factory = session_factory
-
-    def import_nsfw_scores(
-        self, csv_path: Path | str, *, overwrite: bool = False, dry_run: bool = False
-    ) -> ImportReport:
-        """Import ``nsfw_scores.csv`` onto asset identity and report what happened."""
-        path = Path(csv_path)
-        report = ImportReport(source=str(path))
-        if not path.is_file():
-            report.rejected.append({"path": str(path), "reason": "csv_missing"})
-            return self._stamp(report)
-
-        with self._session_factory() as session:
-            # Paths are matched against every occurrence an asset ever had, so a
-            # photo scored before a rename is still recognised.
-            by_path = self._path_index(session)
-            latest = self._latest_reviews(session)
-            seen: Counter[str] = Counter()
-
-            for row in self._rows(path, report):
-                report.rows += 1
-                raw_path, raw_score = row
-                seen[raw_path] += 1
-                if seen[raw_path] > 1:
-                    report.rejected.append({"path": raw_path, "reason": "duplicate_path"})
-                    continue
-                score = _parse_score(raw_score)
-                if score is None:
-                    report.rejected.append(
-                        {"path": raw_path, "reason": "unparsable_score", "value": raw_score}
-                    )
-                    continue
-                asset_id = by_path.get(raw_path)
-                if asset_id is None:
-                    report.unmatched.append(raw_path)
-                    continue
-
-                current = latest.get(asset_id)
-                if current is not None and current.decision:
-                    # A human decided this one; the CSV is older evidence.
-                    report.reviewed += 1
-                    if current.score is None or abs(current.score - score) >= 1e-9:
-                        report.conflicts.append(
-                            {
-                                "path": raw_path,
-                                "asset_id": asset_id,
-                                "current": current.score,
-                                "decision": current.decision,
-                                "csv": score,
-                            }
-                        )
-                    continue
-                if current is not None and current.score is not None:
-                    if abs(current.score - score) < 1e-9:
-                        report.unchanged += 1
-                        continue
-                    report.conflicts.append(
-                        {
-                            "path": raw_path,
-                            "asset_id": asset_id,
-                            "current": current.score,
-                            "decision": None,
-                            "csv": score,
-                        }
-                    )
-                    if not overwrite:
-                        report.skipped_existing += 1
-                        continue
-                session.add(
-                    SafetyReview(
-                        id=str(uuid.uuid4()),
-                        asset_id=asset_id,
-                        score=score,
-                        reviewer=REVIEWER,
-                    )
-                )
-                latest[asset_id] = SafetyReview(asset_id=asset_id, score=score)
-                report.imported += 1
-
-            if dry_run:
-                session.rollback()
-            else:
-                session.commit()
-        return self._stamp(report)
-
-    @staticmethod
-    def _rows(path: Path, report: ImportReport):
-        """Yield ``(path, score)`` pairs, tolerating the donor's own sloppiness."""
-        with open(path, newline="", encoding="utf-8", errors="replace") as handle:
-            for row in csv.DictReader(handle):
-                raw_path = (row.get("path") or "").strip()
-                if not raw_path:
-                    report.rejected.append({"path": "", "reason": "missing_path"})
-                    continue
-                yield raw_path, row.get("nsfw_score")
-
-    @staticmethod
-    def _latest_reviews(session) -> dict[str, SafetyReview]:
-        """The current safety row per asset — latest wins, as everywhere else."""
-        latest: dict[str, SafetyReview] = {}
-        for review in session.scalars(select(SafetyReview).order_by(SafetyReview.created_at)):
-            latest[review.asset_id] = review
-        return latest
-
-    @staticmethod
-    def _path_index(session) -> dict[str, str]:
-        """Every path an asset is or was known by → its stable id."""
-        index: dict[str, str] = {}
-        for asset_id, path in session.execute(select(AssetPath.asset_id, AssetPath.path)):
-            index.setdefault(path, asset_id)
-        for asset_id, path in session.execute(select(Asset.id, Asset.current_path)):
-            if path:
-                index[path] = asset_id  # the current path wins over a closed one
-        return index
-
-    @staticmethod
-    def _stamp(report: ImportReport) -> ImportReport:
-        report.generated_at = datetime.now(timezone.utc).isoformat()
-        return report
-
-
-def write_report(report: ImportReport, directory: Path) -> Path:
-    """Persist the reconciliation report; the import is not evidence until it is."""
-    directory.mkdir(parents=True, exist_ok=True)
-    path = directory / "legacy-nsfw-import.json"
-    path.write_text(json.dumps(report.to_dict(), indent=2, sort_keys=True), encoding="utf-8")
-    return path
-
-
-def _parse_score(value) -> float | None:
-    try:
-        score = float(value)
-    except (TypeError, ValueError):
-        return None
-    if not SCORE_RANGE[0] <= score <= SCORE_RANGE[1]:
-        return None
-    return score

photo_pipeline/services/rename_apply.py

@@ -46,11 +46,9 @@ from pathlib import Path
 from sqlalchemy import select
 from sqlalchemy.orm import sessionmaker
 
-from photo_pipeline.faults import maybe_fault
 from photo_pipeline.models import Asset, AssetPath, RenamePlan
 from photo_pipeline.services import hashing
 from photo_pipeline.services.rename_journal import (
-    ALLOWED_TRANSITIONS,
     MANUAL,
     RESUMABLE,
     JournalState,
@@ -85,6 +83,17 @@ def _now() -> datetime:
     return datetime.now(timezone.utc)
 
 
+def _maybe_fault(state: str) -> None:
+    """Test-only crash barrier (concept §18 fault injection).
+
+    When ``PHOTO_PIPELINE_FAULT_AFTER`` names a journal state, the process dies
+    abruptly the moment that state has been persisted — modelling a real kill at
+    exactly that transition. Never set outside tests.
+    """
+    if os.environ.get("PHOTO_PIPELINE_FAULT_AFTER") == state:
+        os._exit(9)
+
+
 class RenameApplyService:
     def __init__(self, session_factory: sessionmaker, *, library_roots: tuple = ()) -> None:
         self._session_factory = session_factory
@@ -133,10 +142,20 @@ class RenameApplyService:
                 self._apply_one(operation, token=token, worker_id=worker_id)
                 applied += 1
             except PreconditionFailed as error:
-                self._record_failure(operation["id"], token, error.code, str(error))
+                self.journal.transition(
+                    operation["id"],
+                    JournalState.FAILED,
+                    fencing_token=token,
+                    error=(error.code, str(error)),
+                )
                 failed += 1
             except Exception as error:  # unexpected: record and stop touching disk
-                self._record_failure(operation["id"], token, "apply_error", str(error))
+                self.journal.transition(
+                    operation["id"],
+                    JournalState.FAILED,
+                    fencing_token=token,
+                    error=("apply_error", str(error)),
+                )
                 failed += 1
         state = self.journal.sync_plan_state(plan_id)
         return {
@@ -147,33 +166,13 @@ class RenameApplyService:
             "state": state,
         }
 
-    def _record_failure(self, operation_id: str, token: int, code: str, message: str) -> None:
-        """Record a failed operation in a state its journal can actually reach.
-
-        ``failed`` only makes sense while nothing has moved. Once the folder is at
-        its destination — a postcondition failure such as bytes edited during the
-        move — the operation is not "failed and forgotten": the disk changed and
-        the database followed, so it becomes ``rollback_required`` and waits for a
-        human (US07-04). Guessing an unreachable transition used to raise out of
-        ``apply`` and lose the record entirely.
-        """
-        current = self.journal.get(operation_id)["journal_state"]
-        target = (
-            JournalState.FAILED
-            if JournalState.FAILED in ALLOWED_TRANSITIONS.get(current, set())
-            else JournalState.ROLLBACK_REQUIRED
-        )
-        self.journal.transition(
-            operation_id, target, fencing_token=token, error=(code, message)
-        )
-
     def _apply_one(self, operation: dict, *, token: int, worker_id: str) -> None:
         source = Path(operation["source_path"])
         destination = Path(operation["destination_path"])
 
         # 1. Intent first — after this point a crash is recoverable from evidence.
         self.journal.begin(operation["id"], worker_id=worker_id, fencing_token=token)
-        maybe_fault(JournalState.MOVING)
+        _maybe_fault(JournalState.MOVING)
 
         # 2. Recheck preconditions immediately before mutating, never trusting the
         #    plan's snapshot: files can change between preview and confirmation.
@@ -188,21 +187,21 @@ class RenameApplyService:
             os.rename(source, destination)
 
         self.journal.transition(operation["id"], JournalState.MOVED, fencing_token=token)
-        maybe_fault(JournalState.MOVED)
+        _maybe_fault(JournalState.MOVED)
 
         # 4. Database: stable IDs keep their identity, paths are re-pointed and the
         #    old occurrence is closed — all in one transaction.
         self._reconcile_paths(operation, source, destination)
         self.journal.transition(operation["id"], JournalState.DATABASE_UPDATED, fencing_token=token)
-        maybe_fault(JournalState.DATABASE_UPDATED)
+        _maybe_fault(JournalState.DATABASE_UPDATED)
 
         # 5. Postconditions: the bytes really are at the new paths.
         self._verify(operation, destination)
         self.journal.transition(operation["id"], JournalState.VERIFIED, fencing_token=token)
-        maybe_fault(JournalState.VERIFIED)
+        _maybe_fault(JournalState.VERIFIED)
 
         self.journal.transition(operation["id"], JournalState.COMPLETE, fencing_token=token)
-        maybe_fault(JournalState.COMPLETE)
+        _maybe_fault(JournalState.COMPLETE)
 
     def _recheck(self, operation: dict, source: Path, destination: Path) -> None:
         if not source.exists():

photo_pipeline/services/rename_journal.py

@@ -81,17 +81,7 @@ ALLOWED_TRANSITIONS = {
 
 TERMINAL_STATES = frozenset({JournalState.COMPLETE, JournalState.ROLLED_BACK})
 # States where the disk may already have been touched by this operation.
-# ``rollback_required`` belongs here too (US07-04): the move happened and someone
+UNSAFE_STATES = frozenset({JournalState.MOVING, JournalState.MOVED, JournalState.DATABASE_UPDATED})
-# has to decide what to do about it, so the library is not in a state another
-# mutation may build on.
-UNSAFE_STATES = frozenset(
-    {
-        JournalState.MOVING,
-        JournalState.MOVED,
-        JournalState.DATABASE_UPDATED,
-        JournalState.ROLLBACK_REQUIRED,
-    }
-)
 
 RESUMABLE = "resumable"
 ROLLBACK_SAFE = "rollback_safe"

photo_pipeline/services/restores.py

@@ -1,651 +0,0 @@
-"""RestoreService — plan and execute safe restores (US06-04).
-
-Restore is archiving read backwards, with one decisive difference: it removes
-nothing. The archived copy stays on its medium, so every failure mode here costs
-at most a discarded temporary file. What restore must never do is *lose identity*
-— the asset that comes back is the same asset, with its duplicate decision, safety
-review, analysis, and upload history intact — or *overwrite* something in the
-active library.
-
-Preflight proves, per concept §9 "Restore":
-
-- the recorded medium is mounted and is the right one (marker ``media_id``);
-- every selected asset is archived, its archive copy exists, and it hashes to
-  exactly the bytes the database recorded — a mismatch is ``divergent`` and is
-  refused, never silently accepted as "the file";
-- the destination lies inside the library, outside ``_IGNORE/``, and is free; a
-  taken path is answered with a collision-free name, never an overwrite;
-- the library filesystem has room for the scope plus the configured reserve;
-- no rename, archive, or restore lease is holding the lane.
-
-Blocker codes: ``no_library_root``, ``location_offline``, ``wrong_volume``,
-``unsafe_destination``, ``library_not_writable``, ``insufficient_capacity``,
-``lock_conflict``, ``rename_pending``, ``archive_pending``, ``empty_scope``,
-``not_archived``, ``archive_missing``, ``bytes_changed``.
-
-Per item the sequence is:
-
-```
-journal.begin (transferring)   ← intent persisted BEFORE any disk change
-recheck: medium, hash, free destination, asset still archived
-copy to a temporary file beside the destination, fsync, hash it back
-atomically publish into the library
-journal → verified
-current_path = destination, availability = active, path occurrence opened
-journal → complete
-```
-
-Like archiving, the confirmation token is derived from the report, so a changed
-scope, a swapped medium, or a destination that filled up invalidates it.
-"""
-
-from __future__ import annotations
-
-import hashlib
-import json
-import os
-import shutil
-import uuid
-from datetime import datetime, timezone
-from pathlib import Path
-
-from sqlalchemy import select
-from sqlalchemy.orm import sessionmaker
-
-from photo_pipeline.config import Config
-from photo_pipeline.faults import maybe_fault
-from photo_pipeline.jobs.domain_handlers import ARCHIVE_LOCK, LIBRARY_WRITE_LOCK, UPLOAD_LOCK
-from photo_pipeline.models import ArchiveLocation, ArchiveOperation, ArchivePlan, Asset, AssetPath
-from photo_pipeline.path_policy import PathPolicyError, is_excluded, normalize_root, resolve_within
-from photo_pipeline.services import availability
-from photo_pipeline.services.archive_journal import (
-    MANUAL,
-    RESTORE,
-    RESUMABLE,
-    ArchiveJournal,
-    ArchiveState,
-)
-from photo_pipeline.services.archive_transfer import (
-    _clean_temp_files,
-    _fsync_dir,
-    _plan_dict,
-    copy_verify_publish,
-)
-from photo_pipeline.services.archives import ArchiveError
-from photo_pipeline.services.hashing import sha256_file
-from photo_pipeline.services.jobs import JobService
-from photo_pipeline.services.rename_apply import PreconditionFailed
-from photo_pipeline.services.rename_journal import RenameJournal
-
-PREFLIGHT_VERSION = 1
-TOKEN_PREFIX = f"r{PREFLIGHT_VERSION}"
-# What a restored file is called when its original name is taken. The suffix is
-# visible on purpose: a restore that quietly reuses a name is indistinguishable
-# from an overwrite.
-RESTORED_SUFFIX = "restored"
-
-LOCKS = (LIBRARY_WRITE_LOCK, UPLOAD_LOCK, ARCHIVE_LOCK)
-
-APPLYABLE_PLAN_STATES = frozenset({"planned", "applying", "failed", "complete"})
-
-
-def _now() -> datetime:
-    return datetime.now(timezone.utc)
-
-
-def _issue(code: str, message: str) -> dict:
-    return {"code": code, "message": message}
-
-
-class RestoreService:
-    def __init__(self, session_factory: sessionmaker, *, config: Config) -> None:
-        self._session_factory = session_factory
-        self._config = config
-        self._roots = tuple(normalize_root(root) for root in config.library_roots)
-        self.journal = ArchiveJournal(session_factory)
-
-    # ── preflight ─────────────────────────────────────────────────────────────
-
-    def preflight(self, location_id: str, asset_ids: list[str] | None = None) -> dict:
-        """Validate a restore scope and issue its token. Nothing is written."""
-        with self._session_factory() as session:
-            location = session.get(ArchiveLocation, location_id)
-            if location is None:
-                raise ArchiveError("unknown_location", f"unknown archive location {location_id!r}")
-            root = Path(location.root)
-            online = availability.location_online(location)
-            marker = availability.read_marker(root)
-            report = {
-                "schema_version": PREFLIGHT_VERSION,
-                "location": {
-                    "id": location.id,
-                    "name": location.name,
-                    "root": str(root),
-                    "media_id": location.media_id,
-                    "state": _location_state(root, marker, location.media_id),
-                },
-                "blockers": [],
-            }
-            items = self._items(session, location, asset_ids, reachable=online)
-
-        report["blockers"] += self._destination_blockers(report["location"]["state"], root)
-        report["blockers"] += self._lock_blockers()
-        report["items"] = items
-        report["totals"] = {
-            "assets": len(items),
-            "blocked": sum(1 for item in items if item["blockers"]),
-            "bytes": sum(item["byte_size"] or 0 for item in items),
-        }
-        report["capacity"] = self._capacity(report["totals"]["bytes"])
-        if not report["capacity"]["sufficient"]:
-            report["blockers"].append(
-                _issue(
-                    "insufficient_capacity",
-                    # No free-space number here: it drifts between two identical
-                    # preflights and the token is a digest of this text (US06-06).
-                    f"{report['totals']['bytes']} B plus a "
-                    f"{self._config.archive_free_space_reserve_bytes} B reserve do not fit in "
-                    "the library",
-                )
-            )
-        if not items:
-            report["blockers"].append(
-                _issue("empty_scope", "no archived assets are in the selected scope")
-            )
-        report["state"] = (
-            "ready"
-            if not report["blockers"] and not report["totals"]["blocked"]
-            else "blocked"
-        )
-        report["token"] = _token(report)
-        report["generated_at"] = _now().isoformat()
-        return report
-
-    def verify_token(self, token: str, location_id: str, asset_ids: list[str] | None = None) -> bool:
-        return bool(token) and token == self.preflight(location_id, asset_ids)["token"]
-
-    def _items(
-        self, session, location: ArchiveLocation, asset_ids: list[str] | None, *, reachable: bool
-    ) -> list[dict]:
-        stmt = select(Asset).where(Asset.archive_location_id == location.id)
-        if asset_ids is None:
-            # A restored asset keeps its archive link; the default scope is only what
-            # is still archived, so restoring twice is an empty scope, not a blocker.
-            stmt = stmt.where(Asset.availability_state.in_(availability.ARCHIVED))
-        else:
-            stmt = stmt.where(Asset.id.in_(asset_ids))
-        assets = list(session.scalars(stmt.order_by(Asset.archive_path)))
-        if asset_ids is not None:
-            unknown = sorted(set(asset_ids) - {asset.id for asset in assets})
-            if unknown:
-                raise ArchiveError(
-                    "unknown_asset", f"not archived at this location: {', '.join(unknown)}"
-                )
-        taken: set[str] = set()
-        return [self._item(asset, location, reachable=reachable, taken=taken) for asset in assets]
-
-    def _item(self, asset: Asset, location: ArchiveLocation, *, reachable: bool, taken: set) -> dict:
-        source = Path(location.root) / (asset.archive_path or "")
-        blockers: list[dict] = []
-        archive_sha256 = None
-
-        if asset.availability_state not in availability.ARCHIVED:
-            blockers.append(
-                _issue("not_archived", f"asset {asset.id} is {asset.availability_state}")
-            )
-        if reachable:
-            if not source.exists():
-                blockers.append(_issue("archive_missing", f"{source} is not on the medium"))
-            else:
-                archive_sha256 = sha256_file(source)
-                if asset.current_sha256 and archive_sha256 != asset.current_sha256:
-                    blockers.append(
-                        _issue(
-                            "bytes_changed",
-                            f"{source} holds bytes that are not the recorded ones; "
-                            "the archived copy is divergent",
-                        )
-                    )
-
-        destination, destination_blockers = self._destination(asset, taken)
-        blockers += destination_blockers
-        if destination is not None:
-            taken.add(str(destination))
-        return {
-            "asset_id": asset.id,
-            "archive_path": asset.archive_path,
-            "source_path": str(source),
-            "destination_path": str(destination) if destination else None,
-            "expected_sha256": asset.current_sha256,
-            "archive_sha256": archive_sha256,
-            "byte_size": asset.byte_size,
-            "availability_state": asset.availability_state,
-            "blockers": blockers,
-        }
-
-    def _destination(self, asset: Asset, taken: set) -> tuple[Path | None, list[dict]]:
-        """A free path inside the library that mirrors the archived layout.
-
-        Restoring onto an existing file is never an option, so a taken name is
-        answered with ``name (restored).ext`` — visible, ordinary, and impossible to
-        confuse with an overwrite.
-        """
-        if not self._roots:
-            return None, [_issue("no_library_root", "no library root is configured")]
-        root = self._roots[0]
-        try:
-            candidate = resolve_within(root, root / (asset.archive_path or ""))
-        except PathPolicyError as error:
-            return None, [_issue("unsafe_destination", str(error))]
-        if is_excluded(candidate):
-            return None, [
-                _issue("unsafe_destination", f"{candidate} is inside an excluded (_IGNORE/) tree")
-            ]
-        return _free_path(candidate, taken), []
-
-    def _destination_blockers(self, state: str, root: Path) -> list[dict]:
-        blockers: list[dict] = []
-        if not self._roots:
-            blockers.append(_issue("no_library_root", "no library root is configured"))
-        elif not os.access(self._roots[0], os.W_OK):
-            blockers.append(
-                _issue("library_not_writable", f"{self._roots[0]} is not writable")
-            )
-        if state == "offline":
-            blockers.append(
-                _issue("location_offline", f"the archive medium is not mounted at {root}")
-            )
-        elif state == "wrong_volume":
-            blockers.append(_issue("wrong_volume", f"{root} holds a different archive medium"))
-        return blockers
-
-    def _lock_blockers(self) -> list[dict]:
-        blockers: list[dict] = []
-        jobs = JobService(self._session_factory)
-        for lock in LOCKS:
-            held = jobs.blockers(lock)
-            if held:
-                blockers.append(
-                    _issue("lock_conflict", f"the {lock} lane is busy: job {held[0]['id']}")
-                )
-        if RenameJournal(self._session_factory).blocks_mutation():
-            blockers.append(
-                _issue("rename_pending", "an unresolved rename must be recovered before restoring")
-            )
-        if self.journal.blocks_mutation():
-            blockers.append(
-                _issue(
-                    "archive_pending",
-                    "an unresolved archive or restore must be recovered before restoring",
-                )
-            )
-        return blockers
-
-    def _capacity(self, required: int) -> dict:
-        reserve = self._config.archive_free_space_reserve_bytes
-        free = shutil.disk_usage(self._roots[0]).free if self._roots else None
-        return {
-            "required_bytes": required,
-            "reserve_bytes": reserve,
-            "free_bytes": free,
-            "sufficient": free is not None and free >= required + reserve,
-        }
-
-    # ── plans ─────────────────────────────────────────────────────────────────
-
-    def create(self, location_id: str, asset_ids: list[str] | None = None, *, token: str) -> dict:
-        preflight = self.preflight(location_id, asset_ids)
-        if not token or token != preflight["token"]:
-            raise ArchiveError("stale_token", "the restore preflight changed since it was approved")
-        if preflight["state"] != "ready":
-            codes = ", ".join(sorted({issue["code"] for issue in preflight["blockers"]})) or "-"
-            blocked = sorted(
-                {issue["code"] for item in preflight["items"] for issue in item["blockers"]}
-            )
-            raise ArchiveError(
-                "blocked", f"the restore scope is blocked: {', '.join(blocked) or codes}"
-            )
-
-        plan_id = str(uuid.uuid4())
-        with self._session_factory() as session:
-            session.add(
-                ArchivePlan(
-                    id=plan_id,
-                    location_id=location_id,
-                    token=token,
-                    albums=json.dumps(asset_ids) if asset_ids is not None else None,
-                    direction=RESTORE,
-                    state="planned",
-                    schema_version=PREFLIGHT_VERSION,
-                    asset_count=preflight["totals"]["assets"],
-                    byte_size=preflight["totals"]["bytes"],
-                )
-            )
-            session.flush()
-            for sequence, item in enumerate(preflight["items"]):
-                session.add(
-                    ArchiveOperation(
-                        id=str(uuid.uuid4()),
-                        plan_id=plan_id,
-                        direction=RESTORE,
-                        sequence=sequence,
-                        album=Path(item["archive_path"]).parent.name or "(root)",
-                        asset_id=item["asset_id"],
-                        source_path=item["source_path"],
-                        destination_path=item["destination_path"],
-                        archive_path=item["archive_path"],
-                        expected_sha256=item["expected_sha256"],
-                        byte_size=item["byte_size"],
-                        journal_state=ArchiveState.PLANNED,
-                    )
-                )
-            session.commit()
-        return self.get(plan_id)
-
-    def get(self, plan_id: str) -> dict | None:
-        with self._session_factory() as session:
-            plan = session.get(ArchivePlan, plan_id)
-            if plan is None or plan.direction != RESTORE:
-                return None
-            report = _plan_dict(plan)
-        report["operations"] = self.journal.operations(plan_id)
-        return report
-
-    def list(self) -> list[dict]:
-        with self._session_factory() as session:
-            rows = session.scalars(
-                select(ArchivePlan)
-                .where(ArchivePlan.direction == RESTORE)
-                .order_by(ArchivePlan.created_at)
-            )
-            return [_plan_dict(row) for row in rows]
-
-    # ── apply ─────────────────────────────────────────────────────────────────
-
-    def apply(
-        self, plan_id: str, *, expected_version: int | None = None, worker_id: str = "restore"
-    ) -> dict:
-        plan = self._require_plan(plan_id)
-        if expected_version is not None and plan["version"] != expected_version:
-            raise ArchiveError(
-                "stale_plan",
-                f"plan {plan_id} is at version {plan['version']}, expected {expected_version}",
-            )
-        if plan["state"] not in APPLYABLE_PLAN_STATES:
-            raise ArchiveError("invalid_state", f"plan {plan_id} is {plan['state']}")
-        blocking = [row for row in self.journal.incomplete() if row["plan_id"] != plan_id]
-        if blocking:
-            raise ArchiveError(
-                "archive_pending",
-                f"another archive operation is unresolved ({blocking[0]['id']}); recover it first",
-            )
-
-        token = self._claim_plan(plan_id)
-        location = self._location(plan["location_id"])
-        restored = failed = skipped = 0
-        for operation in self.journal.operations(plan_id):
-            if operation["journal_state"] == ArchiveState.COMPLETE:
-                skipped += 1
-                continue
-            try:
-                if operation["journal_state"] == ArchiveState.VERIFIED:
-                    self._finish(operation, token=token)
-                else:
-                    self._restore_one(operation, location, token=token, worker_id=worker_id)
-                restored += 1
-            except PreconditionFailed as error:
-                self._fail(operation, token, error.code, str(error))
-                failed += 1
-            except Exception as error:  # unexpected: record and stop touching disk
-                self._fail(operation, token, "restore_error", str(error))
-                failed += 1
-        state = self.journal.sync_plan_state(plan_id)
-        return {
-            "plan_id": plan_id,
-            "restored": restored,
-            "failed": failed,
-            "skipped": skipped,
-            "state": state,
-        }
-
-    def _restore_one(self, operation: dict, location: dict, *, token: int, worker_id: str) -> None:
-        source = Path(operation["source_path"])
-        destination = Path(operation["destination_path"])
-
-        # 1. Intent first; from here a crash is resolvable from journal + disk.
-        self.journal.begin(operation["id"], worker_id=worker_id, fencing_token=token)
-        maybe_fault(ArchiveState.TRANSFERRING)
-
-        # 2. Recheck against the medium and the library as they are right now.
-        self._recheck(operation, source, destination, location)
-        destination.parent.mkdir(parents=True, exist_ok=True)
-
-        # 3. Always copy: the archived original stays on its medium.
-        copy_verify_publish(source, destination, operation["expected_sha256"])
-        _fsync_dir(destination.parent)
-
-        if sha256_file(destination) != operation["expected_sha256"]:
-            raise PreconditionFailed(
-                "restore_mismatch", f"{destination} does not hold the expected bytes"
-            )
-        self.journal.transition(operation["id"], ArchiveState.VERIFIED, fencing_token=token)
-        maybe_fault(ArchiveState.VERIFIED)
-
-        self._finish(self.journal.get(operation["id"]), token=token)
-
-    def _finish(self, operation: dict, *, token: int) -> None:
-        """Publish the restored file to the database. Idempotent, so recovery may
-        replay it after a crash between the copy and the bookkeeping."""
-        destination = Path(operation["destination_path"])
-        if not destination.exists() or sha256_file(destination) != operation["expected_sha256"]:
-            raise PreconditionFailed(
-                "restore_unverified", f"{destination} is not a verified restored copy"
-            )
-        self._record_restored(operation, destination)
-        self.journal.transition(operation["id"], ArchiveState.COMPLETE, fencing_token=token)
-        maybe_fault(ArchiveState.COMPLETE)
-
-    def _recheck(self, operation: dict, source: Path, destination: Path, location: dict) -> None:
-        root = Path(location["root"])
-        if not root.is_dir() or not (root / availability.MARKER_NAME).exists():
-            raise PreconditionFailed("location_offline", f"{root} is not the archive medium")
-        if not source.exists():
-            raise PreconditionFailed("archive_missing", f"{source} is not on the medium")
-        if source.is_symlink() or destination.is_symlink():
-            raise PreconditionFailed("symlink", "refusing to restore through a symlink")
-        if destination.exists():
-            # Never overwrite: the plan's free path was taken since it was made.
-            raise PreconditionFailed(
-                "destination_exists", f"destination {destination} is occupied"
-            )
-        if not self._inside_library(destination):
-            raise PreconditionFailed(
-                "destination_escape", f"{destination} is outside the library roots"
-            )
-        if sha256_file(source) != operation["expected_sha256"]:
-            self._mark_divergent(operation["asset_id"])
-            raise PreconditionFailed(
-                "bytes_changed", f"{source} changed since the plan was approved"
-            )
-        with self._session_factory() as session:
-            asset = session.get(Asset, operation["asset_id"])
-            if asset is None or asset.availability_state not in availability.ARCHIVED:
-                raise PreconditionFailed(
-                    "not_archived", f"asset {operation['asset_id']} is no longer archived"
-                )
-
-    def _inside_library(self, destination: Path) -> bool:
-        for root in self._roots:
-            try:
-                resolve_within(root, destination)
-                return True
-            except PathPolicyError:
-                continue
-        return False
-
-    # ── database ──────────────────────────────────────────────────────────────
-
-    def _record_restored(self, operation: dict, destination: Path) -> None:
-        """The bytes are back in the library: open the new active occurrence and set
-        availability. Identity, decisions, and history are untouched — that is the
-        entire point of restoring rather than re-importing."""
-        now = _now()
-        with self._session_factory() as session:
-            asset = session.get(Asset, operation["asset_id"])
-            if asset is None:
-                raise PreconditionFailed(
-                    "asset_missing", f"asset {operation['asset_id']} no longer exists"
-                )
-            # A restored asset may be returning to a path it once held, so only an
-            # *open* occurrence counts as already registered — that is what keeps
-            # recovery idempotent without collapsing the path history.
-            recorded = session.scalar(
-                select(AssetPath).where(
-                    AssetPath.asset_id == asset.id,
-                    AssetPath.path == str(destination),
-                    AssetPath.valid_until.is_(None),
-                )
-            )
-            if recorded is None:  # idempotent: recovery may replay this
-                session.add(
-                    AssetPath(
-                        asset_id=asset.id,
-                        path=str(destination),
-                        valid_from=now,
-                        reason="restore",
-                    )
-                )
-            asset.current_path = str(destination)
-            asset.availability_state = availability.ACTIVE
-            asset.missing_at = None
-            # The archive copy stays where it is; keeping the link means a restored
-            # asset still knows which medium holds its archived bytes.
-            asset.archive_divergent_at = None
-            asset.state_version += 1
-            asset.updated_at = now
-            session.commit()
-
-    def _mark_divergent(self, asset_id: str) -> None:
-        """Record that the archived copy is not the recorded file. Durable, because
-        the next restore attempt must not rediscover this from scratch."""
-        with self._session_factory() as session:
-            asset = session.get(Asset, asset_id)
-            if asset is None:
-                return
-            asset.archive_divergent_at = _now()
-            asset.state_version += 1
-            session.commit()
-
-    # ── recovery ──────────────────────────────────────────────────────────────
-
-    def recover(self, *, worker_id: str = "restore-recovery") -> dict:
-        """Resolve every incomplete restore from journal + disk evidence.
-
-        A restore never removed anything, so ``resumable`` simply discards the
-        temporary debris and re-plans the item; ``forward`` finishes the bookkeeping
-        for a published file; ``manual`` is left untouched and keeps blocking.
-        """
-        results = {"resumed": 0, "completed": 0, "manual": 0}
-        touched: set[str] = set()
-        for verdict in self.journal.classify_all(direction=RESTORE):
-            operation = self.journal.get(verdict["operation_id"])
-            touched.add(operation["plan_id"])
-            token = (operation["fencing_token"] or 0) + 1
-            if verdict["classification"] == MANUAL:
-                results["manual"] += 1
-                continue
-            if verdict["classification"] == RESUMABLE:
-                _clean_temp_files(Path(operation["destination_path"]).parent)
-                self.journal.transition(operation["id"], ArchiveState.PLANNED, fencing_token=token)
-                results["resumed"] += 1
-                continue
-            try:
-                self._finish(operation, token=token)
-                results["completed"] += 1
-            except PreconditionFailed as error:
-                self._fail(operation, token, error.code, str(error))
-                results["manual"] += 1
-        for plan_id in touched:
-            self.journal.sync_plan_state(plan_id)
-        return results
-
-    def recovery_status(self) -> dict:
-        verdicts = self.journal.classify_all(direction=RESTORE)
-        return {
-            "operations": verdicts,
-            "manual": [v for v in verdicts if v["classification"] == MANUAL],
-            "blocks_mutation": self.journal.blocks_mutation(),
-        }
-
-    # ── helpers ───────────────────────────────────────────────────────────────
-
-    def _fail(self, operation: dict, token: int, code: str, message: str) -> None:
-        self.journal.transition(
-            operation["id"], ArchiveState.FAILED, fencing_token=token, error=(code, message)
-        )
-
-    def _require_plan(self, plan_id: str) -> dict:
-        plan = self.get(plan_id)
-        if plan is None:
-            raise ArchiveError("unknown_plan", f"unknown restore plan {plan_id!r}")
-        return plan
-
-    def _location(self, location_id: str) -> dict:
-        with self._session_factory() as session:
-            location = session.get(ArchiveLocation, location_id)
-            if location is None:
-                raise ArchiveError("unknown_location", f"unknown archive location {location_id!r}")
-            return {"id": location.id, "root": location.root, "media_id": location.media_id}
-
-    def _claim_plan(self, plan_id: str) -> int:
-        with self._session_factory() as session:
-            plan = session.get(ArchivePlan, plan_id)
-            plan.version += 1
-            plan.state = "applying"
-            plan.updated_at = _now()
-            token = plan.version
-            session.commit()
-        return token
-
-
-# ── module helpers ───────────────────────────────────────────────────────────
-
-
-def _location_state(root: Path, marker: dict | None, media_id: str) -> str:
-    if not root.is_dir() or marker is None:
-        return "offline"
-    return "online" if marker.get("media_id") == media_id else "wrong_volume"
-
-
-def _free_path(candidate: Path, taken: set) -> Path:
-    """``a.jpg`` → ``a (restored).jpg`` → ``a (restored 2).jpg`` …
-
-    ``taken`` holds the destinations already claimed by earlier items of the same
-    plan, so two restores in one scope cannot plan the same path.
-    """
-    if not candidate.exists() and str(candidate) not in taken:
-        return candidate
-    stem, suffix = candidate.stem, candidate.suffix
-    attempt = 1
-    while True:
-        label = RESTORED_SUFFIX if attempt == 1 else f"{RESTORED_SUFFIX} {attempt}"
-        alternative = candidate.with_name(f"{stem} ({label}){suffix}")
-        if not alternative.exists() and str(alternative) not in taken:
-            return alternative
-        attempt += 1
-
-
-def _token(report: dict) -> str:
-    """Digest of everything the report asserts about the scope and the medium.
-
-    Free space is excluded: it drifts constantly without changing what a restore
-    would do, and the capacity verdict itself is part of the digest.
-    """
-    payload = {key: value for key, value in report.items() if key not in ("generated_at", "token")}
-    payload["capacity"] = {
-        key: value for key, value in payload["capacity"].items() if key != "free_bytes"
-    }
-    digest = hashlib.sha256(
-        json.dumps(payload, sort_keys=True, ensure_ascii=False, default=str).encode("utf-8")
-    ).hexdigest()
-    return f"{TOKEN_PREFIX}:{digest}"

photo_pipeline/services/safety.py

@@ -108,8 +108,9 @@ from datetime import datetime, timezone
 from sqlalchemy import select
 from sqlalchemy.orm import sessionmaker
 
-from photo_pipeline.models import Asset, ExifProjection, SafetyReview
+from photo_pipeline.integrations import exiftool
-from photo_pipeline.services import exif_checkpoint
+from photo_pipeline.models import Asset, SafetyReview
+from photo_pipeline.services import hashing
 
 DECISIONS = {SFW, NSFW, "deferred"}
 
@@ -173,14 +174,6 @@ class SafetyService:
         with self._session_factory() as session:
             assets = list(session.scalars(_eligible_assets_query().order_by(Asset.current_path)))
             latest = self._latest_by_asset(session)
-            # One query, not one per asset: the reviewer needs to see a divergent
-            # checkpoint, which is neither "verified" nor a plain failure (US07-03).
-            projections = {
-                row.asset_id: row.state
-                for row in session.scalars(
-                    select(ExifProjection).where(ExifProjection.stage == "safety")
-                )
-            }
             rows = []
             for asset in assets:
                 review = latest.get(asset.id)
@@ -196,7 +189,6 @@ class SafetyService:
                         "decision": decision,
                         "suggested": classify(review.score) if review and review.score is not None else None,
                         "exif_verified": bool(review and review.exif_verified_at),
-                        "exif_state": projections.get(asset.id),
                     }
                 )
             return {"total": len(rows), "items": rows[offset : offset + limit]}
@@ -255,24 +247,13 @@ class SafetyService:
         result_sha256 = None
         if write_exif and decision in (SFW, NSFW) and path:
             ops = exif_projection(decision)
-            # The full checkpoint: write the owned keyword, read the whole file back,
+            if exiftool.apply_keywords(path, add=ops["add"], remove=ops["remove"]):
-            # and prove every field this stage does not own survived. A divergent
+                # Read back: the chosen keyword present, the opposite absent.
-            # result is recorded and left alone — it must not count as verified, so
+                keywords = exiftool.read_keyword_sets([path]).get(path, set())
-            # upload stays blocked until a human decides (US07-03).
+                opposite = NSFW if decision == SFW else SFW
-            result = exif_checkpoint.run(
+                if decision in keywords and opposite not in keywords:
-                path, add=tuple(ops["add"]), remove=tuple(ops["remove"])
+                    exif_verified_at = _now()
-            )
+                    result_sha256 = hashing.sha256_file(path)
-            exif_checkpoint.record(
-                self._session_factory,
-                asset_id=asset_id,
-                stage="safety",
-                result=result,
-                add=tuple(ops["add"]),
-                remove=tuple(ops["remove"]),
-            )
-            if result.verified:
-                exif_verified_at = result.verified_at
-                result_sha256 = result.sha256
 
         now = _now()
         with self._session_factory() as session:

photo_pipeline/services/thumbnails.py

@@ -8,11 +8,6 @@ an EXIF-only edit reuses the file while a real pixel change invalidates it; writ
 are atomic and the cache is bounded by an LRU quota. Failures are persisted as typed
 errors so a broken original is not retried on every request.
 
-An archived asset is served from its medium when that medium is mounted, and from
-its *protected* preview when it is not (US06-03). Protected previews are evidence,
-not cache: the quota never evicts them, because the original they describe may be
-unreachable when duplicate review needs it.
-
 Reuses photo_analyzer.prepare_image decode/resize/HEIC handling, adding the missing
 EXIF-orientation step, WebP output, and a managed cache (donor_ledger.yaml:
 pa-imaging).
@@ -20,19 +15,17 @@ pa-imaging).
 
 from __future__ import annotations
 
-import os
 import uuid
 from pathlib import Path
 
-from PIL import Image, ImageOps
+from PIL import Image, ImageOps, UnidentifiedImageError
 from sqlalchemy import select
 from sqlalchemy.exc import IntegrityError
 from sqlalchemy.orm import sessionmaker
 
-from photo_pipeline import imaging, path_policy
+from photo_pipeline import path_policy
 from photo_pipeline.config import Config
 from photo_pipeline.models import Asset, Thumbnail
-from photo_pipeline.services import availability
 
 # Best-effort HEIC support: registered only if the optional decoder is installed.
 try:  # pragma: no cover - depends on an optional native dependency
@@ -43,13 +36,8 @@ except Exception:  # pragma: no cover
     pass
 
 SIZES = (256, 512, 1280)
-# Every in-flight render writes one of these beside its destination; startup
-# cleanup recognises exactly this pattern and nothing else.
-TEMP_SUFFIX = ".tmp"
 THUMB_VERSION = 1
 THUMB_FORMAT = "webp"
-# The size kept as durable comparison evidence for archived assets (concept §9).
-PROTECTED_SIZE = 1280
 
 
 class ThumbnailError(RuntimeError):
@@ -98,12 +86,7 @@ class ThumbnailService:
         self._config = config
         self._cache_dir = config.thumbnail_cache_dir
 
-    def generate(
+    def generate(self, asset_id: str, size: int) -> Path:
-        self, asset_id: str, size: int, *, protected: bool = False, source: Path | None = None
-    ) -> Path:
-        """Render (or reuse) a preview. ``source`` overrides where the bytes are read
-        from — the archiver passes its verified archive copy, which the database does
-        not yet point at while the transfer is still in flight."""
         if size not in SIZES:
             raise InvalidSize(f"size must be one of {SIZES}")
 
@@ -111,7 +94,9 @@ class ThumbnailService:
             asset = session.get(Asset, asset_id)
             if asset is None:
                 raise ThumbnailNotFound(f"unknown asset {asset_id}")
-            archived = asset.availability_state in availability.ARCHIVED
+            if asset.availability_state != "active" or not asset.current_path:
+                raise ThumbnailUnavailable(f"asset {asset_id} has no active file")
+            self._validate_path(asset.current_path)
             cache_key = self._cache_key(asset, size)
 
             row = session.get(Thumbnail, cache_key)
@@ -122,21 +107,9 @@ class ThumbnailService:
                     )
                 if row.path and Path(row.path).exists():
                     _touch(row.path)
-                    if protected and not row.protected:
-                        self._protect(cache_key)
                     return Path(row.path)
 
-            # An archived original is read from its medium; when that medium is not
+            source = asset.current_path
-            # mounted the retained preview above is the only evidence there is.
-            source = source or availability.readable_path(session, asset)
-            if source is None:
-                raise ThumbnailUnavailable(f"asset {asset_id} has no readable file")
-            source = str(source)
-            if source == asset.current_path:
-                # Render the *resolved* path the check approved: revalidating and then
-                # reopening the original name would let a symlink swapped in between
-                # the two steps decide which bytes are served (US07-02).
-                source = str(self._validate_path(source))  # archive roots lie outside
 
         # Rendering happens outside the DB session (no transaction held during I/O).
         try:
@@ -146,72 +119,25 @@ class ThumbnailService:
                 self._record_error(cache_key, asset_id, size, error.code)
             raise
 
-        # Archived assets keep their preview permanently: it is the comparison
+        self._record_ready(cache_key, asset_id, size, rendered)
-        # evidence that survives the original leaving active storage.
-        self._record_ready(cache_key, asset_id, size, rendered, protected=protected or archived)
         self._enforce_quota(keep=rendered["path"])
         return Path(rendered["path"])
 
-    def ensure_protected(self, asset_id: str, *, source: Path | None = None) -> dict:
-        """Produce (or confirm) the durable comparison preview for an asset.
-
-        Returns evidence rather than raising, because the caller — archive
-        preflight and the transfer itself — decides what an unrenderable original
-        means. ``unsupported`` is a recorded property of the file, not a failure of
-        the policy: its hashes and metadata remain the comparison evidence.
-        """
-        try:
-            path = self.generate(asset_id, PROTECTED_SIZE, protected=True, source=source)
-        except tuple(_PERSISTED_ERRORS) as error:
-            return {"state": "unsupported", "error_code": error.code, "path": None}
-        except ThumbnailError as error:
-            return {"state": "unavailable", "error_code": error.code, "path": None}
-        return {"state": "ready", "error_code": None, "path": str(path)}
-
-    def evidence(self, asset_id: str) -> dict:
-        """What durable preview this asset has right now, without rendering."""
-        with self._session_factory() as session:
-            rows = list(
-                session.execute(
-                    select(Thumbnail).where(Thumbnail.asset_id == asset_id)
-                ).scalars()
-            )
-        for row in rows:
-            if row.state == "ready" and row.path and Path(row.path).exists():
-                return {
-                    "state": "ready",
-                    "protected": bool(row.protected),
-                    "size": row.size,
-                    "error_code": None,
-                }
-        for row in rows:
-            if row.state == "error":
-                return {
-                    "state": "unsupported",
-                    "protected": False,
-                    "size": row.size,
-                    "error_code": row.error_code,
-                }
-        return {"state": "missing", "protected": False, "size": None, "error_code": None}
-
-    def _protect(self, cache_key: str) -> None:
-        with self._session_factory() as session:
-            row = session.get(Thumbnail, cache_key)
-            if row is not None:
-                row.protected = True
-                session.commit()
-
     # ── path safety ──────────────────────────────────────────────────────────
-    def _validate_path(self, current_path: str) -> Path:
+    def _validate_path(self, current_path: str) -> None:
-        """The resolved path to read, or ``PathNotAllowed``.
+        path = Path(current_path)
-
+        if path_policy.is_excluded(path):
-        The message names no path: a refusal is returned to the browser, and where
+            raise PathNotAllowed(f"excluded path: {current_path}")
-        the library lives is not the caller's business (US07-02).
+        roots = self._config.library_roots
-        """
+        if not roots:
+            return
+        for root in roots:
             try:
-            return path_policy.resolve_in_roots(self._config.library_roots, current_path)
+                path_policy.resolve_within(Path(root), path)
-        except path_policy.PathPolicyError as error:
+                return
-            raise PathNotAllowed(str(error)) from None
+            except path_policy.PathPolicyError:
+                continue
+        raise PathNotAllowed(f"path outside configured roots: {current_path}")
 
     # ── cache key + rendering ──────────────────────────────────────────────────
     @staticmethod
@@ -226,33 +152,28 @@ class ThumbnailService:
         return self._cache_dir / safe[:2] / f"{safe}.{THUMB_FORMAT}"
 
     def _render(self, source: str, size: int, cache_key: str) -> dict:
-        """Decode bounded, orient, colour-manage, resize, and write atomically.
-
-        The temporary file is removed on every failure path: a decoder that dies
-        halfway through ``save`` would otherwise leave a stray ``.tmp`` in the cache
-        forever (US07-03).
-        """
-        destination = self._cache_path(cache_key)
-        destination.parent.mkdir(parents=True, exist_ok=True)
-        tmp = destination.with_name(f".{destination.name}.{uuid.uuid4().hex}{TEMP_SUFFIX}")
         try:
-            with imaging.open_image(source, max_pixels=self._config.thumbnail_max_pixels) as image:
+            with Image.open(source) as image:
-                imaging.draft(image, size)  # JPEG decodes near the target size
+                width, height = image.size
+                if width * height > self._config.thumbnail_max_pixels:
+                    raise ImageTooLarge(
+                        f"{width}x{height} exceeds {self._config.thumbnail_max_pixels} px"
+                    )
                 oriented = ImageOps.exif_transpose(image)
                 mode = "RGBA" if _has_alpha(oriented) else "RGB"
-                converted = imaging.to_srgb(oriented, mode=mode)
+                converted = oriented.convert(mode)
                 converted.thumbnail((size, size), Image.LANCZOS)
                 out_width, out_height = converted.size
+                destination = self._cache_path(cache_key)
+                destination.parent.mkdir(parents=True, exist_ok=True)
+                tmp = destination.with_name(f".{destination.name}.{uuid.uuid4().hex}.tmp")
                 converted.save(tmp, format="WEBP", quality=82, method=4)
-        except imaging.ImageTooLarge as error:
+        except ImageTooLarge:
-            tmp.unlink(missing_ok=True)
-            raise ImageTooLarge(str(error)) from None
-        except imaging.UndecodableImage as error:
-            tmp.unlink(missing_ok=True)
-            raise UnsupportedImage(str(error)) from None
-        except Exception:
-            tmp.unlink(missing_ok=True)
             raise
+        except (UnidentifiedImageError, OSError, ValueError) as error:
+            raise UnsupportedImage(f"cannot decode {source}: {error}") from error
+
+        import os
 
         os.replace(tmp, destination)
         return {
@@ -262,32 +183,8 @@ class ThumbnailService:
             "format": THUMB_FORMAT,
         }
 
-    def cleanup_temp_files(self) -> int:
-        """Remove leftover render temporaries, and only those.
-
-        Runs at startup, where the concept is explicit: clean *recognized* stale
-        temporary files, never arbitrary ones. The pattern is this service's own —
-        a dot-prefixed name inside the managed cache directory ending in
-        ``.tmp`` — so nothing outside the cache and nothing a user put there can
-        match. Returns how many were removed.
-        """
-        if not self._cache_dir.is_dir():
-            return 0
-        removed = 0
-        for leftover in self._cache_dir.rglob(f".*{TEMP_SUFFIX}"):
-            if not leftover.is_file() or leftover.is_symlink():
-                continue
-            try:
-                leftover.unlink()
-                removed += 1
-            except OSError:  # pragma: no cover - a racing render already won
-                pass
-        return removed
-
     # ── persistence ────────────────────────────────────────────────────────────
-    def _record_ready(
+    def _record_ready(self, cache_key: str, asset_id: str, size: int, rendered: dict) -> None:
-        self, cache_key: str, asset_id: str, size: int, rendered: dict, *, protected: bool = False
-    ) -> None:
         with self._session_factory() as session:
             session.merge(
                 Thumbnail(
@@ -300,7 +197,6 @@ class ThumbnailService:
                     width=rendered["width"],
                     height=rendered["height"],
                     format=rendered["format"],
-                    protected=protected,
                 )
             )
             try:
@@ -334,10 +230,6 @@ class ThumbnailService:
         if total <= quota:
             return
         files.sort(key=lambda f: f.stat().st_mtime)  # least-recently-used first
-        # Protected previews are evidence, not cache: an archived original cannot be
-        # re-rendered once its medium is away, so eviction never touches them.
-        protected = self._protected_paths()
-        files = [f for f in files if str(f) not in protected]
         keep_path = str(Path(keep)) if keep else None
         evicted: list[str] = []
         for f in files:
@@ -355,16 +247,6 @@ class ThumbnailService:
         if evicted:
             self._forget(evicted)
 
-    def _protected_paths(self) -> set[str]:
-        with self._session_factory() as session:
-            return {
-                row.path
-                for row in session.execute(
-                    select(Thumbnail).where(Thumbnail.protected.is_(True))
-                ).scalars()
-                if row.path
-            }
-
     def _forget(self, paths: list[str]) -> None:
         with self._session_factory() as session:
             rows = session.execute(

photo_pipeline/services/upload_batches.py

@@ -37,7 +37,6 @@ from sqlalchemy import select, update
 from sqlalchemy.orm import sessionmaker
 
 from photo_pipeline.config import Config
-from photo_pipeline.faults import UPLOAD_ACCEPTED, maybe_fault
 from photo_pipeline.integrations import immich_go
 from photo_pipeline.models import UploadBatch, UploadItem
 from photo_pipeline.services.hashing import sha1_file
@@ -237,10 +236,6 @@ class UploadBatchService:
             error = ("uploader_failed", f"immich-go exited with {result['exit_code']}")
             item_state = ItemState.FAILED
 
-        # The uploader is done and Immich may already hold every file, but nothing
-        # about that is durable yet — the control point for "accepted, outcome not
-        # recorded" (US07-04). Recovery must answer ``unknown_requires_verification``.
-        maybe_fault(UPLOAD_ACCEPTED)
         self._finish(batch_id, token=token, state=state, error=error, result=result)
         if item_state:
             self._set_items(batch_id, item_state)

pyproject.toml

@@ -24,8 +24,6 @@ test = [
 
 [tool.ruff]
 line-length = 100
-# The frozen CLI archive is evidence, not code under maintenance (US07-01).
-extend-exclude = ["legacy_cli_archive"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]

tests/_artifacts.py

@@ -1,96 +0,0 @@
-"""Failure artifacts for the fault and race suites (US07-04).
-
-A randomized concurrency failure that leaves nothing behind is a failure nobody
-can diagnose: the temporary library is deleted, the database goes with it, and the
-seed that produced the interleaving is gone. So when a test fails, everything
-needed to reproduce and read it is copied out of the temporary directory:
-
-    <artifacts>/<test id>/
-        seeds.json     recorded properties (``race_seed``) and the failing test id
-        manifest.json  every file under the temporary directory: path, size, sha256
-        files/...      the databases (with -wal/-shm), journals, and logs themselves
-
-The manifest covers the whole tree — including files too large or too private to
-copy — so a missing or unexpected file is still visible afterwards. Copying is
-bounded by ``MAX_COPY_BYTES``: artifacts must not turn a failing CI run into a
-disk-full one.
-
-Set ``PHOTO_PIPELINE_TEST_ARTIFACTS`` to choose the destination; the default is
-``.artifacts/`` in the repository root.
-"""
-
-from __future__ import annotations
-
-import hashlib
-import json
-import os
-import re
-import shutil
-from pathlib import Path
-
-REPO = Path(__file__).resolve().parents[1]
-DEFAULT_DIR = REPO / ".artifacts"
-MAX_COPY_BYTES = 25 * 1024 * 1024
-# Databases (and their write-ahead logs), journals exported as files, and logs.
-COPY_SUFFIXES = (".db", ".db-wal", ".db-shm", ".sqlite", ".log", ".json", ".jsonl", ".argv")
-
-
-def artifacts_dir() -> Path:
-    return Path(os.environ.get("PHOTO_PIPELINE_TEST_ARTIFACTS", DEFAULT_DIR))
-
-
-def _slug(test_id: str) -> str:
-    return re.sub(r"[^A-Za-z0-9._-]+", "_", test_id)[:120]
-
-
-def manifest(root: Path) -> list[dict]:
-    """Every file under ``root``: relative path, byte size, and SHA-256.
-
-    The filesystem state at the moment of failure — what was moved, what was left
-    behind, what was half-written.
-    """
-    entries = []
-    for path in sorted(root.rglob("*")):
-        if not path.is_file() or path.is_symlink():
-            continue
-        try:
-            body = path.read_bytes()
-        except OSError as error:
-            entries.append({"path": str(path.relative_to(root)), "error": str(error)})
-            continue
-        entries.append(
-            {
-                "path": str(path.relative_to(root)),
-                "bytes": len(body),
-                "sha256": hashlib.sha256(body).hexdigest(),
-            }
-        )
-    return entries
-
-
-def collect(root: Path, test_id: str, *, properties: dict | None = None) -> Path:
-    """Copy the evidence for one failed test out of ``root``. Returns its directory."""
-    destination = artifacts_dir() / _slug(test_id)
-    files = destination / "files"
-    files.mkdir(parents=True, exist_ok=True)
-
-    entries = manifest(root)
-    (destination / "manifest.json").write_text(json.dumps(entries, indent=1))
-    (destination / "seeds.json").write_text(
-        json.dumps({"test": test_id, "properties": properties or {}}, indent=1)
-    )
-
-    budget = MAX_COPY_BYTES
-    for path in sorted(root.rglob("*")):
-        if not path.is_file() or path.is_symlink():
-            continue
-        if not path.name.endswith(COPY_SUFFIXES):
-            continue
-        size = path.stat().st_size
-        if size > budget:
-            continue  # the manifest still records it; the copy is what is skipped
-        target = files / path.relative_to(root)
-        target.parent.mkdir(parents=True, exist_ok=True)
-        shutil.copy2(path, target)
-        budget -= size
-    return destination

tests/characterization/conftest.py

@@ -14,11 +14,6 @@ import pytest
 from PIL import Image
 
 REPO = Path(__file__).resolve().parents[2]
-# The donors are frozen in the read-only archive (US07-01). Only this suite — and
-# the parity test that compares against them — puts that directory on sys.path;
-# production never does, which tests/unit/test_legacy_archive.py enforces.
-ARCHIVED_SOURCES = REPO / "legacy_cli_archive" / "src"
-sys.path.insert(0, str(ARCHIVED_SOURCES))
 sys.path.insert(0, str(REPO))
 
 EXIFTOOL = shutil.which("exiftool")

tests/characterization/test_donor_ledger.py

@@ -1,20 +1,13 @@
-"""Ledger lint (US01-01, extended by US07-01).
+"""Ledger lint (US01-01): every donor-ledger row must carry a real source
-
+reference, a target location, and either existing characterization test IDs or
-Every donor-ledger row must carry a real source reference — now inside the frozen
+a real pending backlog story."""
-archive — a target location, and either existing characterization test IDs or a
-real pending backlog story. Since archival (US07-01) a row may also be ``resolved``:
-its replacement has shipped, ``parity`` names tests that exist and prove it, and
-``delta`` states every intentional difference. Nothing may quietly become
-"finished" without one of those two."""
 import re
 from pathlib import Path
 
 import yaml
 
 REPO = Path(__file__).resolve().parents[2]
-ARCHIVE = REPO / "legacy_cli_archive"
+LEDGER = REPO / "donor_ledger.yaml"
-LEDGER = ARCHIVE / "donor_ledger.yaml"
-ARCHIVED_SOURCES = ARCHIVE / "src"
 STORIES = REPO / "delivery_backlog" / "stories"
 TESTS_DIR = Path(__file__).resolve().parent
 
@@ -22,7 +15,7 @@ CLASSIFICATIONS = {"reuse", "extract", "refactor", "replace"}
 REQUIRED_AREAS = {"discovery", "hashing", "imaging", "nsfw", "vision", "exif",
                   "database", "ui", "configuration", "logging", "cancellation",
                   "error"}
-STATUSES = {"characterized", "resolved", "pending"}
+STATUSES = {"characterized", "pending"}
 
 
 def load_rows():
@@ -56,11 +49,10 @@ def test_rows_have_required_fields_and_unique_ids():
 
 
 def test_source_references_resolve():
-    """Source paths are relative to the archive: the donors moved there, whole."""
     for r in load_rows():
         src = r["source"]
-        f = ARCHIVED_SOURCES / src["file"]
+        f = REPO / src["file"]
-        assert f.is_file(), f"{r['id']}: source file {src['file']} missing from the archive"
+        assert f.is_file(), f"{r['id']}: source file {src['file']} missing"
         text = f.read_text(encoding="utf-8")
         for sym in src["symbols"]:
             assert sym in text, f"{r['id']}: symbol {sym!r} not found in {src['file']}"
@@ -71,9 +63,7 @@ def test_rows_have_tests_or_pending_story():
     for r in load_rows():
         tests = r.get("tests", [])
         pending = r.get("pending_story")
-        parity = r.get("parity", [])
+        assert tests or pending, f"{r['id']}: neither tests nor pending_story"
-        assert tests or pending or parity, \
-            f"{r['id']}: neither characterization tests, parity tests, nor a pending story"
         for t in tests:
             assert t in known_tests, f"{r['id']}: unknown test id {t}"
         if pending:
@@ -83,58 +73,16 @@ def test_rows_have_tests_or_pending_story():
             assert tests, f"{r['id']}: characterized rows need test ids"
 
 
-def test_resolved_rows_name_their_parity_or_their_delta():
-    """A resolved row is a claim that the behavior is handled. It has to say how:
-    tests that prove the replacement, or a stated difference — usually both."""
-    for r in load_rows():
-        if r["status"] != "resolved":
-            assert "parity" not in r, f"{r['id']}: parity on a non-resolved row"
-            continue
-        parity = r.get("parity", [])
-        delta = r.get("delta")
-        assert parity or delta, f"{r['id']}: resolved without parity tests or a delta"
-        if delta:
-            assert len(str(delta).strip()) >= 20, f"{r['id']}: delta too thin to be a reason"
-        for ref in parity:
-            rel, _, func = ref.partition("::")
-            path = REPO / rel
-            assert path.is_file(), f"{r['id']}: parity test file {rel} missing"
-            assert f"def {func}" in path.read_text(encoding="utf-8"), \
-                f"{r['id']}: parity test {ref} not found"
-
-
-def test_pending_rows_are_the_only_unfinished_work():
-    """The ledger is the honest list of what has not been carried over: a pending
-    row names the story that will, and that story must still be open work."""
-    for r in load_rows():
-        if r["status"] != "pending":
-            continue
-        story = r.get("pending_story")
-        assert story, f"{r['id']}: pending without a story"
-        assert list(STORIES.glob(f"{story}-*.md")), f"{r['id']}: unknown story {story}"
-
-
-def test_every_target_module_exists():
-    """A row is only finished if the thing it points at is really there."""
-    for r in load_rows():
-        if r["status"] == "pending":
-            continue
-        modules = re.findall(r"photo_pipeline/[\w/]+\.py", str(r["target"]))
-        for module in modules:
-            assert (REPO / module).is_file(), f"{r['id']}: target {module} does not exist"
-
-
 def test_all_required_areas_covered():
     covered = {r["area"] for r in load_rows()}
     assert REQUIRED_AREAS <= covered, f"uncovered areas: {REQUIRED_AREAS - covered}"
     assert covered <= REQUIRED_AREAS, f"unknown areas: {covered - REQUIRED_AREAS}"
 
 
-def test_every_donor_is_in_the_archive_and_nowhere_else():
+def test_no_legacy_file_moved():
-    """US01-01 forbade archiving before characterization; US07-01 requires it after.
+    # US01-01 explicitly forbids moving/archiving donors; the ledger's source
-    Each donor exists exactly once — frozen, in the archive."""
+    # files must all still exist at their original locations.
     for donor in ("photo_analyzer.py", "nsfwtag/scoring.py", "nsfwtag/exif.py",
                   "nsfwtag/server.py", "webapp/query.py", "webapp/runner.py",
                   "webapp/server.py"):
-        assert (ARCHIVED_SOURCES / donor).is_file(), f"donor missing from archive: {donor}"
+        assert (REPO / donor).is_file(), f"donor moved: {donor}"
-        assert not (REPO / donor).exists(), f"donor still live at the repo root: {donor}"

tests/conftest.py

@@ -1,16 +1,4 @@
-"""Make the repository root importable for the pipeline test suites, give every
+"""Make the repository root importable for the pipeline test suites."""
-suite the application session the API requires since US07-02, and keep the evidence
-of any failure (US07-04).
-
-The suites drive the API the way the browser does — module-level ``httpx`` calls and
-``TestClient`` — so instead of threading a cookie through several hundred call sites,
-both clients bootstrap the session themselves exactly like ``frontend/js/api.js``:
-fetch ``/api/v1/session`` once, then send the cookie plus the CSRF header, and
-re-bootstrap once on 401 (a restarted server issues a new session).
-
-Security tests deliberately bypass this by constructing their own ``httpx.Client``;
-only the module-level helpers are wrapped.
-"""
 
 import sys
 from pathlib import Path
@@ -18,112 +6,3 @@ from pathlib import Path
 REPO = Path(__file__).resolve().parents[1]
 if str(REPO) not in sys.path:
     sys.path.insert(0, str(REPO))
-
-import httpx
-import httpx._api  # httpx.get/post resolve `request` in this module
-import pytest
-from starlette.testclient import TestClient
-
-from photo_pipeline.api.security import CSRF_HEADER, SESSION_COOKIE
-
-_SESSIONS: dict[str, tuple[str, str]] = {}  # base url -> (session id, csrf token)
-
-
-def _base(url) -> str:
-    parts = httpx.URL(str(url))
-    return f"{parts.scheme}://{parts.netloc.decode()}"
-
-
-def _bootstrap(base: str) -> tuple[str, str]:
-    if base not in _SESSIONS:
-        response = httpx.Client(timeout=10).get(f"{base}/api/v1/session")
-        _SESSIONS[base] = (
-            response.cookies.get(SESSION_COOKIE, ""),
-            response.json().get("csrf_token", ""),
-        )
-    return _SESSIONS[base]
-
-
-def session_client(base_url: str, **kwargs) -> httpx.Client:
-    """An ``httpx.Client`` that has bootstrapped its own application session.
-
-    For suites that keep one client for a whole journey; it holds its own cookie, so
-    a client made after a server restart picks up the new session by construction.
-    """
-    client = httpx.Client(base_url=base_url, **kwargs)
-    client.headers[CSRF_HEADER] = client.get("/api/v1/session").json()["csrf_token"]
-    return client
-
-
-def _authorized(url, headers, cookies):
-    session_id, token = _bootstrap(_base(url))
-    headers = dict(headers or {})
-    headers.setdefault(CSRF_HEADER, token)
-    cookies = dict(cookies or {})
-    cookies.setdefault(SESSION_COOKIE, session_id)
-    return headers, cookies
-
-
-@pytest.fixture(autouse=True, scope="session")
-def _api_session():
-    real_request, real_stream = httpx._api.request, httpx._api.stream
-    real_client_request, real_client_init = TestClient.request, TestClient.__init__
-
-    def request(method, url, *, headers=None, cookies=None, **kwargs):
-        sent, jar = _authorized(url, headers, cookies)
-        response = real_request(method, url, headers=sent, cookies=jar, **kwargs)
-        if response.status_code == 401:
-            _SESSIONS.pop(_base(url), None)
-            sent, jar = _authorized(url, headers, cookies)
-            response = real_request(method, url, headers=sent, cookies=jar, **kwargs)
-        return response
-
-    def stream(method, url, *, headers=None, cookies=None, **kwargs):
-        sent, jar = _authorized(url, headers, cookies)
-        return real_stream(method, url, headers=sent, cookies=jar, **kwargs)
-
-    def client_init(self, app, *args, base_url="http://127.0.0.1", **kwargs):
-        # The default "http://testserver" is not a local host, which is exactly what
-        # the Host check refuses; in-process tests are still same-origin callers.
-        real_client_init(self, app, *args, base_url=base_url, **kwargs)
-
-    def client_request(self, method, url, *, headers=None, **kwargs):
-        if CSRF_HEADER not in self.headers:
-            response = real_client_request(self, "GET", "/api/v1/session")
-            self.headers[CSRF_HEADER] = response.json()["csrf_token"]
-        return real_client_request(self, method, url, headers=headers, **kwargs)
-
-    httpx._api.request, httpx.request = request, request
-    httpx._api.stream, httpx.stream = stream, stream
-    TestClient.request, TestClient.__init__ = client_request, client_init
-    yield
-    httpx._api.request, httpx.request = real_request, real_request
-    httpx._api.stream, httpx.stream = real_stream, real_stream
-    TestClient.request, TestClient.__init__ = real_client_request, real_client_init
-
-
-@pytest.hookimpl(hookwrapper=True)
-def pytest_runtest_makereport(item, call):
-    """On failure, keep the temporary library, database, journals, logs, and seed.
-
-    ``tmp_path`` is deleted a few runs later and its contents are the only record
-    of what a randomized or fault-injected failure actually did, so they are copied
-    out while they still exist (US07-04).
-    """
-    outcome = yield
-    report = outcome.get_result()
-    if report.when != "call" or not report.failed:
-        return
-    root = item.funcargs.get("tmp_path")
-    if root is None or not Path(root).exists():
-        return
-    from tests._artifacts import collect
-
-    try:
-        destination = collect(
-            Path(root), item.nodeid, properties=dict(getattr(report, "user_properties", []))
-        )
-    except OSError as error:  # never let evidence collection mask the real failure
-        report.sections.append(("failure artifacts", f"could not be collected: {error}"))
-    else:
-        report.sections.append(("failure artifacts", str(destination)))

tests/e2e/_pipeline_harness.py

@@ -393,125 +393,6 @@ def mark_upload_ready(seeded: Seeded, *, unverified: tuple[str, ...] = ()) -> No
             session.commit()
 
 
-# ── Phase F: an archivable album and a mountable fake medium ─────────────────
-
-
-def mark_uploaded(seeded: Seeded, *, album: str = "rome") -> None:
-    """Give every seeded photo the verified upload evidence archiving requires.
-
-    Archiving refuses anything Immich is not proven to hold, and that proof is an
-    upload batch — recorded here as fixture state so the archive journeys do not
-    have to re-run an upload they are not testing.
-    """
-    from sqlalchemy import select
-
-    from photo_pipeline.models import Asset, UploadBatch, UploadItem
-    from photo_pipeline.services.hashing import sha256_file
-
-    with session_factory(seeded) as sf:
-        with sf() as session:
-            batch_id = str(uuid.uuid4())
-            session.add(
-                UploadBatch(
-                    id=batch_id,
-                    album=album,
-                    folder=str(seeded.lib / album),
-                    album_name=album,
-                    state="succeeded",
-                    preflight_token="v1:e2e",
-                    outcome_state="verified",
-                    created_at=NOW,
-                )
-            )
-            for asset in session.scalars(select(Asset)):
-                if not asset.current_path:
-                    continue
-                session.add(
-                    UploadItem(
-                        batch_id=batch_id,
-                        asset_id=asset.id,
-                        path=asset.current_path,
-                        sha256=sha256_file(asset.current_path),
-                        sha1="0" * 40,
-                        state="sent",
-                        outcome="uploaded",
-                    )
-                )
-            session.commit()
-
-
-class ArchiveStack:
-    """A seeded, archivable library plus the server, the worker, and a fake medium.
-
-    The medium is an ordinary directory whose marker file makes it identifiable;
-    ``unmount()`` takes that marker away, which is exactly what the application sees
-    when an external disk is unplugged.
-    """
-
-    MARKER = ".photo-pipeline-archive.json"
-
-    def __init__(self, tmp_path: Path, seeded: Seeded) -> None:
-        self.tmp_path = tmp_path
-        self.seeded = seeded
-        self.archive = tmp_path / "archive"
-        self.archive.mkdir(exist_ok=True)
-        self.server: Server | None = None
-        self.worker: subprocess.Popen | None = None
-        self.base = ""
-
-    def start(self, *, worker: bool = True, extra_env: dict[str, str] | None = None) -> "ArchiveStack":
-        env = {"PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": "0", **(extra_env or {})}
-        self.server = Server(self.seeded, extra_env=env).start()
-        self.base = self.server.base
-        if worker:
-            self.worker = start_worker(self.seeded, extra_env=env)
-        return self
-
-    def register(self, name: str = "external") -> dict:
-        response = httpx.post(
-            f"{self.base}/api/v1/archive-locations",
-            json={"name": name, "root": str(self.archive)},
-            timeout=20,
-        )
-        response.raise_for_status()
-        return response.json()
-
-    def unmount(self) -> None:
-        (self.archive / self.MARKER).rename(self.archive / f"{self.MARKER}.away")
-
-    def remount(self) -> None:
-        (self.archive / f"{self.MARKER}.away").rename(self.archive / self.MARKER)
-
-    def plans(self) -> list[dict]:
-        return httpx.get(f"{self.base}/api/v1/archive-plans", timeout=20).json()["plans"]
-
-    def assets(self) -> list[dict]:
-        return httpx.get(
-            f"{self.base}/api/v1/inventory/assets", params={"limit": 200}, timeout=20
-        ).json()["items"]
-
-    def restart_server(self) -> None:
-        self.server.stop()
-        self.server.start()
-
-    def restart_worker(self, *, extra_env: dict[str, str] | None = None) -> None:
-        """Replace the worker — a healthy one after a crashed one, by default."""
-        if self.worker is not None and self.worker.poll() is None:
-            self.worker.kill()
-            self.worker.wait(timeout=10)
-        self.worker = start_worker(
-            self.seeded,
-            extra_env={"PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": "0", **(extra_env or {})},
-        )
-
-    def stop(self) -> None:
-        if self.worker is not None:
-            self.worker.kill()
-            self.worker.wait(timeout=10)
-        if self.server is not None:
-            self.server.stop()
-
-
 class UploadStack:
     """A seeded, upload-ready library plus the server, worker, and fake Immich."""
 

tests/e2e/test_archive_ui.py

@@ -1,302 +0,0 @@
-"""Browser journeys for the archive view (US06-05).
-
-Archiving is the only stage that removes originals, so these journeys check the
-two things a browser must never get wrong about it: that the preview names exactly
-what would leave and where it would go, and that nothing offers an action the
-server would refuse. The medium is a real directory whose marker makes it
-identifiable; unmounting it is what an unplugged disk looks like from here.
-
-Nothing is mocked inside the browser: the transfer runs in the real worker process
-and the assertions read the filesystem afterwards.
-"""
-
-from __future__ import annotations
-
-import pytest
-from playwright.sync_api import expect
-
-from tests.e2e._pipeline_harness import (
-    ArchiveStack,
-    mark_uploaded,
-    seed_album,
-    session_factory,
-    wait_until,
-)
-
-pytestmark = pytest.mark.phase_f  # part of the Phase F acceptance gate (US06-06)
-
-TIMEOUT = 10
-RUN_TIMEOUT = 30_000
-
-
-@pytest.fixture
-def stack(tmp_path):
-    seeded = seed_album(tmp_path)
-    mark_uploaded(seeded)
-    running = ArchiveStack(tmp_path, seeded)
-    try:
-        yield running
-    finally:
-        running.stop()
-
-
-def _open(page, stack) -> None:
-    page.goto(f"{stack.base}/app/#/archive")
-    page.get_by_test_id("archive-locations").wait_for()
-
-
-def _archive(page, stack) -> None:
-    """Confirm the archive and wait for the worker to finish the run."""
-    _open(page, stack)
-    page.get_by_test_id("start-archive").click()
-    expect(page.get_by_test_id("detail-state")).to_have_text("complete", timeout=RUN_TIMEOUT)
-
-
-def _archived_paths(stack) -> list[str]:
-    return sorted(p.name for p in (stack.archive / "rome").glob("*.jpg"))
-
-
-# ── preview and confirmation ─────────────────────────────────────────────────
-
-
-def test_the_preview_names_the_scope_destination_and_reclaimable_bytes(page, stack):
-    errors = []
-    page.on("console", lambda m: errors.append(m.text) if m.type == "error" else None)
-    stack.start(worker=False)
-    location = stack.register()
-    _open(page, stack)
-
-    row = page.get_by_test_id("location-row").first
-    expect(row.get_by_test_id("location-label")).to_have_text("external")
-    expect(row.get_by_test_id("location-media")).to_have_text(location["media_id"])
-    expect(row.get_by_test_id("location-state")).to_have_text("online")
-
-    album = page.get_by_test_id("archive-album-row").first
-    expect(album.get_by_test_id("album-name")).to_have_text("rome")
-    expect(album.get_by_test_id("album-destination")).to_have_text(str(stack.archive / "rome"))
-    expect(album.get_by_test_id("album-assets")).to_have_text("2")
-    expect(album.get_by_test_id("album-state")).to_have_text("ready")
-    # Same filesystem here, so the transfer method is the move path — and it is
-    # named, because copy-verify-remove and move fail differently.
-    expect(album.get_by_test_id("album-method")).to_contain_text("move")
-
-    expect(page.get_by_test_id("destination-identity")).to_contain_text(location["media_id"])
-    expect(page.get_by_test_id("capacity")).to_contain_text("reserve")
-    expect(page.get_by_test_id("start-archive")).to_contain_text("Archive 1 album(s) · reclaim")
-    expect(page.get_by_test_id("start-archive")).to_be_enabled()
-    assert stack.plans() == [], "previewing may not create anything"
-    assert errors == [], f"console errors: {errors}"
-
-
-def test_an_offline_medium_blocks_the_confirmation_and_says_what_to_mount(page, stack):
-    stack.start(worker=False)
-    location = stack.register()
-    stack.unmount()
-    _open(page, stack)
-
-    expect(page.get_by_test_id("location-state")).to_have_text("offline")
-    expect(page.get_by_test_id("preflight-blocker").first).to_have_attribute(
-        "data-code", "location_offline"
-    )
-    instruction = page.get_by_test_id("mount-instruction")
-    expect(instruction).to_contain_text("external")
-    expect(instruction).to_contain_text(location["media_id"])
-    expect(page.get_by_test_id("start-archive")).to_be_disabled()
-
-
-def test_an_unuploaded_album_is_blocked_with_its_reason(page, stack):
-    # No upload evidence at all: archiving would remove the only copy.
-    seeded = stack.seeded
-    with session_factory(seeded) as sf:
-        from sqlalchemy import delete
-
-        from photo_pipeline.models import UploadItem
-
-        with sf() as session:
-            session.execute(delete(UploadItem))
-            session.commit()
-    stack.start(worker=False)
-    stack.register()
-    _open(page, stack)
-
-    expect(page.get_by_test_id("album-state")).to_have_text("blocked")
-    expect(page.get_by_test_id("album-blocker").first).to_have_attribute(
-        "data-code", "partial_scope"
-    )
-    expect(page.get_by_test_id("start-archive")).to_be_disabled()
-
-
-# ── running, progress, reload ────────────────────────────────────────────────
-
-
-def test_a_confirmed_archive_runs_and_separates_transfer_verify_and_removal(page, stack):
-    stack.start()
-    stack.register()
-    _archive(page, stack)
-
-    expect(page.get_by_test_id("count-complete")).to_have_text("complete: 2")
-    expect(page.get_by_test_id("count-failed")).to_have_text("failed: 0")
-    expect(page.get_by_test_id("count-transfer")).to_have_text("transfer: 0")
-    expect(page.get_by_test_id("count-verified")).to_have_text("verified: 0")
-    expect(page.get_by_test_id("count-removing")).to_have_text("removing: 0")
-    expect(page.get_by_test_id("operation-phase").first).to_have_text("complete")
-
-    # The library really lost the originals and the medium really holds them.
-    assert _archived_paths(stack) == ["a.jpg", "b.jpg"]
-    assert not (stack.seeded.lib / "rome" / "a.jpg").exists()
-    assert {a["availability_state"] for a in stack.assets()} == {"archived_online"}
-
-
-def test_the_run_survives_a_reload_because_the_state_is_the_servers(page, stack):
-    stack.start()
-    stack.register()
-    _archive(page, stack)
-
-    page.reload()
-    page.get_by_test_id("plan-detail").wait_for()
-    expect(page.get_by_test_id("detail-state")).to_have_text("complete")
-    expect(page.get_by_test_id("count-complete")).to_have_text("complete: 2")
-    # And the run stays addressable by its own URL.
-    plan_id = stack.plans()[0]["id"]
-    page.goto(f"{stack.base}/app/#/archive?plan={plan_id}")
-    expect(page.get_by_test_id("plan-detail")).to_have_attribute("data-plan", plan_id)
-
-
-# ── recovery ─────────────────────────────────────────────────────────────────
-
-
-def test_an_interrupted_transfer_is_shown_with_evidence_and_a_safe_action(page, stack):
-    stack.start(worker=False)
-    stack.register()
-    _open(page, stack)
-    page.get_by_test_id("start-archive").click()
-    expect(page.get_by_test_id("archive-result")).to_be_visible()
-    # No worker ran, so every item is still planned; leave one mid-transfer as a
-    # crash would: intent recorded, nothing published, source intact.
-    _interrupt(stack)
-
-    page.reload()
-    page.get_by_test_id("archive-recovery").wait_for()
-    row = page.get_by_test_id("recovery-row").first
-    expect(row).to_have_attribute("data-classification", "resumable")
-    expect(row.get_by_test_id("recovery-reason")).to_contain_text("source present")
-    page.get_by_test_id("resolve-recovery").click()
-    expect(page.get_by_test_id("archive-recovery")).to_have_count(0)
-    assert (stack.seeded.lib / "rome" / "a.jpg").exists(), "recovery may not move anything"
-
-
-def test_ambiguous_evidence_offers_no_action_at_all(page, stack):
-    stack.start(worker=False)
-    stack.register()
-    _open(page, stack)
-    page.get_by_test_id("start-archive").click()
-    expect(page.get_by_test_id("archive-result")).to_be_visible()
-    # Journal says the copy is verified, but the medium holds nothing: no evidence
-    # supports either finishing or retrying this item.
-    _interrupt(stack, state="verified")
-
-    page.reload()
-    page.get_by_test_id("archive-recovery").wait_for()
-    expect(page.get_by_test_id("recovery-row").first).to_have_attribute(
-        "data-classification", "manual"
-    )
-    expect(page.get_by_test_id("recovery-manual")).to_be_visible()
-    expect(page.get_by_test_id("resolve-recovery")).to_have_count(0)
-    expect(page.get_by_test_id("no-safe-recovery")).to_be_visible()
-
-
-def _interrupt(stack, *, state: str = "transferring") -> None:
-    """Leave the plan's first operation in a non-terminal journal state."""
-    with session_factory(stack.seeded) as sf:
-        from photo_pipeline.services.archive_journal import ArchiveJournal
-
-        journal = ArchiveJournal(sf)
-        plan_id = stack.plans()[0]["id"]
-        operation = journal.operations(plan_id)[0]
-        journal.begin(operation["id"], worker_id="crashed", fencing_token=1)
-        if state != "transferring":
-            journal.transition(operation["id"], state, fencing_token=1)
-
-
-# ── offline browsing and restore ─────────────────────────────────────────────
-
-
-def test_archived_photos_stay_browsable_while_the_medium_is_away(page, stack):
-    stack.start()
-    stack.register()
-    _archive(page, stack)
-    stack.unmount()
-
-    page.reload()
-    page.get_by_test_id("archived-assets").wait_for()
-    row = page.get_by_test_id("archived-row").first
-    expect(row.get_by_test_id("archived-availability")).to_have_text("archived · medium away")
-    expect(row.get_by_test_id("archived-path")).to_contain_text("rome/")
-    expect(row.get_by_test_id("archived-medium")).to_have_text("external")
-    expect(page.get_by_test_id("mount-instruction")).to_contain_text("external")
-    # The retained protected preview is served even though the original is gone.
-    preview = row.get_by_test_id("archived-preview")
-    assert preview.evaluate("img => img.complete && img.naturalWidth > 0")
-    # Restoring is impossible right now, and says so instead of failing later.
-    expect(page.get_by_test_id("start-restore")).to_be_disabled()
-
-
-def test_restoring_brings_the_photos_back_without_losing_identity(page, stack):
-    stack.start()
-    stack.register()
-    _archive(page, stack)
-    before = {asset["id"] for asset in stack.assets()}
-
-    page.reload()
-    page.get_by_test_id("restore").wait_for()
-    expect(page.get_by_test_id("restore-row").first.get_by_test_id("restore-destination")).to_contain_text(
-        str(stack.seeded.lib / "rome")
-    )
-    page.get_by_test_id("start-restore").click()
-    wait_until(
-        lambda: all(a["availability_state"] == "active" for a in stack.assets()),
-        timeout=30,
-    )
-
-    assert {asset["id"] for asset in stack.assets()} == before  # same identities
-    assert (stack.seeded.lib / "rome" / "a.jpg").exists()
-    assert _archived_paths(stack) == ["a.jpg", "b.jpg"]  # the archive copy stays
-
-
-def test_a_taken_name_is_restored_beside_it_never_over_it(page, stack):
-    stack.start()
-    stack.register()
-    _archive(page, stack)
-    squatter = stack.seeded.lib / "rome" / "a.jpg"
-    squatter.parent.mkdir(parents=True, exist_ok=True)
-    squatter.write_bytes(b"a different photo now lives here")
-
-    page.reload()
-    page.get_by_test_id("restore").wait_for()
-    destinations = page.get_by_test_id("restore-destination").all_inner_texts()
-    assert any("(restored)" in text for text in destinations), destinations
-
-    page.get_by_test_id("start-restore").click()
-    wait_until(
-        lambda: all(a["availability_state"] == "active" for a in stack.assets()),
-        timeout=30,
-    )
-    assert squatter.read_bytes() == b"a different photo now lives here"
-    assert (stack.seeded.lib / "rome" / "a (restored).jpg").exists()
-
-
-# ── keyboard ─────────────────────────────────────────────────────────────────
-
-
-def test_the_whole_archive_can_be_confirmed_from_the_keyboard(page, stack):
-    stack.start()
-    stack.register()
-    _open(page, stack)
-
-    button = page.get_by_test_id("start-archive")
-    button.focus()
-    expect(button).to_be_focused()
-    page.keyboard.press("Enter")
-
-    expect(page.get_by_test_id("detail-state")).to_have_text("complete", timeout=RUN_TIMEOUT)
-    assert _archived_paths(stack) == ["a.jpg", "b.jpg"]

tests/e2e/test_crash_recovery.py

@@ -1,274 +0,0 @@
-"""Process death at the newer control points (US07-04, concept §18).
-
-The rename and archive journals already prove crash safety at each of their
-transitions (tests/integration/test_rename_recovery.py,
-tests/integration/test_archive_recovery.py). The three transitions covered here
-are the remaining ones where a kill leaves the world and the database disagreeing:
-
-- ``exif:written``   — keywords are on disk, nothing about them is recorded;
-- ``upload:accepted``— the uploader finished, no outcome is stored;
-- ``job:item_done``  — one item is durably done, the job is not finished.
-
-Each test kills a real child process at the barrier and then asserts what a
-restart does: resume idempotently, or say plainly that a human has to look. Never
-"assume it worked".
-"""
-
-from __future__ import annotations
-
-import os
-import shutil
-import subprocess
-import sys
-import uuid
-from datetime import datetime, timezone
-from pathlib import Path
-
-import numpy as np
-import pytest
-from PIL import Image
-from sqlalchemy import select
-
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.faults import EXIF_WRITTEN, JOB_ITEM_DONE, UPLOAD_ACCEPTED
-from photo_pipeline.integrations import exiftool
-from photo_pipeline.jobs.worker import Worker
-from photo_pipeline.models import Asset, SafetyReview
-from photo_pipeline.services import exif_checkpoint, hashing
-from photo_pipeline.services.jobs import ItemState, JobService, JobState
-from photo_pipeline.services.safety import SafetyService
-from photo_pipeline.services.upload_batches import BatchState, UploadBatchService
-from photo_pipeline.services.upload_verification import retry_blockers
-from photo_pipeline.services.uploads import UploadService
-from tests.e2e._pipeline_harness import (
-    SILENT_UPLOADER,
-    FakeImmich,
-    fake_uploader,
-    mark_upload_ready,
-    seed_album,
-)
-
-REPO = Path(__file__).resolve().parents[2]
-NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
-
-
-def _child(script: str, *args: str, barrier: str, tmp_path: Path) -> None:
-    """Run ``script`` in a child that dies at ``barrier``; assert it really died."""
-    path = tmp_path / f"child_{barrier.replace(':', '_')}.py"
-    path.write_text(script.format(repo=str(REPO)))
-    env = dict(os.environ)
-    env["PHOTO_PIPELINE_FAULT_AFTER"] = barrier
-    result = subprocess.run(
-        [sys.executable, str(path), *args], env=env, capture_output=True
-    )
-    assert result.returncode in (9, -9), (
-        f"child should have been killed at {barrier}, got {result.returncode}: "
-        f"{result.stderr.decode(errors='replace')[-400:]}"
-    )
-
-
-def _env(tmp_path, **extra):
-    (tmp_path / "data").mkdir(exist_ok=True)
-    lib = tmp_path / "lib"
-    lib.mkdir(exist_ok=True)
-    config = Config.from_env(
-        {
-            "PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data"),
-            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
-            **extra,
-        }
-    )
-    run_migrations(config.database_url)
-    return config, create_session_factory(create_db_engine(config.database_url)), lib
-
-
-def _image(path: Path, seed: int = 3) -> None:
-    path.parent.mkdir(parents=True, exist_ok=True)
-    pixels = np.random.default_rng(seed).integers(0, 256, (64, 96, 3), dtype=np.uint8)
-    Image.fromarray(pixels).save(path, quality=90)
-
-
-def _register(sf, path: Path) -> str:
-    asset_id = str(uuid.uuid4())
-    with sf() as session:
-        session.add(
-            Asset(
-                id=asset_id,
-                original_path=str(path),
-                current_path=str(path),
-                discovered_at=NOW,
-                hash_version=1,
-                byte_size=path.stat().st_size,
-                current_sha256=hashing.sha256_file(path),
-            )
-        )
-        session.commit()
-    return asset_id
-
-
-# ── EXIF written, checkpoint not recorded ────────────────────────────────────
-
-DECIDE_SCRIPT = """
-import sys
-sys.path.insert(0, {repo!r})
-from photo_pipeline.db import create_db_engine, create_session_factory
-from photo_pipeline.services.safety import SafetyService
-
-db_url, asset_id = sys.argv[1], sys.argv[2]
-sf = create_session_factory(create_db_engine(db_url))
-SafetyService(sf).decide(asset_id, "nsfw")
-"""
-
-
-@pytest.mark.skipif(shutil.which("exiftool") is None, reason="exiftool not installed")
-def test_a_crash_after_the_exif_write_leaves_nothing_verified_and_re_runs_cleanly(tmp_path):
-    config, sf, lib = _env(tmp_path)
-    path = lib / "a.jpg"
-    _image(path)
-    asset_id = _register(sf, path)
-    original_sha = hashing.sha256_file(path)
-
-    _child(
-        DECIDE_SCRIPT, config.database_url, asset_id, barrier=EXIF_WRITTEN, tmp_path=tmp_path
-    )
-
-    # The file changed, but the application claims nothing about it: no decision,
-    # no projection, and the stored hash is still the pre-write one.
-    assert "nsfw" in exiftool.read_keyword_sets([str(path)])[str(path)]
-    assert hashing.sha256_file(path) != original_sha
-    with sf() as session:
-        assert session.scalars(select(SafetyReview)).all() == []
-        assert session.get(Asset, asset_id).current_sha256 == original_sha
-    assert exif_checkpoint.state_for(sf, asset_id, "safety") is None
-
-    # Re-running is the recovery: the write is idempotent, so the second attempt
-    # verifies and records what the first one only did to the file.
-    review = SafetyService(sf).decide(asset_id, "nsfw")
-    assert review["exif_verified"] is True
-    assert exif_checkpoint.state_for(sf, asset_id, "safety") == exif_checkpoint.VERIFIED
-    keywords = exiftool.read_keyword_sets([str(path)])[str(path)]
-    assert "nsfw" in keywords and "sfw" not in keywords
-    with sf() as session:
-        asset = session.get(Asset, asset_id)
-        assert asset.current_sha256 == hashing.sha256_file(path)
-
-
-# ── uploader accepted, outcome not persisted ─────────────────────────────────
-
-UPLOAD_SCRIPT = """
-import sys
-sys.path.insert(0, {repo!r})
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory
-from photo_pipeline.services.upload_batches import UploadBatchService
-
-db_url, data_dir, lib, binary, server, batch_id = sys.argv[1:7]
-config = Config.from_env(
-    {{
-        "PHOTO_PIPELINE_DATA_DIR": data_dir,
-        "PHOTO_PIPELINE_LIBRARY_ROOTS": lib,
-        "PHOTO_PIPELINE_IMMICH_GO_BINARY": binary,
-        "PHOTO_PIPELINE_IMMICH_SERVER_URL": server,
-        "PHOTO_PIPELINE_IMMICH_API_KEY": "sentinel",
-    }}
-)
-sf = create_session_factory(create_db_engine(db_url))
-UploadBatchService(sf, config=config).run(batch_id)
-"""
-
-
-def test_a_crash_after_the_uploader_accepted_requires_verification(tmp_path):
-    """immich-go exited cleanly and the server may hold every file, but nothing was
-    written down. Recovery must not guess success — and must not blindly retry."""
-    seeded = seed_album(tmp_path)
-    mark_upload_ready(seeded)
-    immich = FakeImmich()
-    binary = fake_uploader(tmp_path, SILENT_UPLOADER)
-    config = Config.from_env(
-        {
-            "PHOTO_PIPELINE_DATA_DIR": str(seeded.data),
-            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(seeded.lib),
-            "PHOTO_PIPELINE_IMMICH_GO_BINARY": str(binary),
-            "PHOTO_PIPELINE_IMMICH_SERVER_URL": immich.url,
-            "PHOTO_PIPELINE_IMMICH_API_KEY": "sentinel",
-        }
-    )
-    engine = create_db_engine(config.database_url)
-    sf = create_session_factory(engine)
-    service = UploadBatchService(sf, config=config)
-    report = UploadService(sf, config=config).preflight(["rome"])
-    assert report["state"] == "ready", report["blockers"]
-    batch_id = service.create(["rome"], token=report["token"])[0]["id"]
-
-    try:
-        # The batch is claimed by the child, which dies once the uploader has run.
-        _child(
-            UPLOAD_SCRIPT,
-            config.database_url,
-            str(seeded.data),
-            str(seeded.lib),
-            str(binary),
-            immich.url,
-            batch_id,
-            barrier=UPLOAD_ACCEPTED,
-            tmp_path=tmp_path,
-        )
-    finally:
-        immich.stop()
-    assert service.get(batch_id)["state"] == BatchState.RUNNING  # lane still held
-
-    recovered = service.recover()
-
-    assert recovered == {"interrupted": 1}
-    batch = service.get(batch_id)
-    assert batch["state"] == BatchState.UNKNOWN and batch["error_code"] == "interrupted"
-    assert [b["code"] for b in retry_blockers(batch)] == ["requires_verification"]
-    assert all(item["state"] == "pending" for item in batch["items"]), "nothing claimed as sent"
-
-
-# ── one item done, the job unfinished ────────────────────────────────────────
-
-WORKER_SCRIPT = """
-import sys
-sys.path.insert(0, {repo!r})
-from pathlib import Path
-from photo_pipeline.db import create_db_engine, create_session_factory
-from photo_pipeline.jobs.worker import Worker
-
-db_url, log = sys.argv[1], Path(sys.argv[2])
-
-def handler(item_key, ctx):
-    with log.open("a") as handle:
-        handle.write(item_key + "\\n")
-
-sf = create_session_factory(create_db_engine(db_url))
-Worker(sf, {{"scan": handler}}, "killable", lease_seconds=1).run_once()
-"""
-
-
-def test_a_crash_between_items_resumes_without_running_the_done_item_twice(tmp_path):
-    config, sf, lib = _env(tmp_path)
-    service = JobService(sf)
-    job = service.enqueue("scan", items=["a", "b", "c"])
-    log = tmp_path / "handled.log"
-
-    _child(
-        WORKER_SCRIPT, config.database_url, str(log), barrier=JOB_ITEM_DONE, tmp_path=tmp_path
-    )
-
-    crashed = log.read_text().split()
-    assert crashed == ["a"], "the child should have died right after its first item"
-    assert service.get(job["id"])["state"] == JobState.RUNNING
-
-    # A fresh worker takes over once the dead lease expires.
-    import time
-
-    time.sleep(1.1)  # the child's lease is one second long
-    fresh = Worker(sf, {"scan": lambda item, ctx: log.open("a").write(item + "\n")}, "alive")
-    fresh.run_once()
-
-    assert service.get(job["id"])["state"] == JobState.SUCCEEDED
-    handled = log.read_text().split()
-    assert sorted(handled) == ["a", "b", "c"], f"an item ran twice or not at all: {handled}"
-    assert service.progress(job["id"])["by_state"] == {ItemState.SUCCEEDED: 3}

tests/e2e/test_phase_a_pipeline.py

@@ -22,8 +22,6 @@ import numpy as np
 import pytest
 from PIL import Image
 
-from tests.conftest import session_client
-
 REPO = Path(__file__).resolve().parents[2]
 
 
@@ -94,7 +92,7 @@ class ServerController:
                 pytest.fail(f"server exited: {err.decode(errors='replace')}")
             try:
                 if httpx.get(f"{self.base}/api/v1/health/ready", timeout=1).status_code == 200:
-                    self.client = session_client(self.base, timeout=10)
+                    self.client = httpx.Client(base_url=self.base, timeout=10)
                     return
             except httpx.HTTPError:
                 time.sleep(0.2)

tests/e2e/test_phase_f_pipeline.py

@@ -1,505 +0,0 @@
-"""Phase F end-to-end acceptance (US06-06): archive lifecycle, black box.
-
-Every journey drives a real ``photo_pipeline serve`` child process and a real
-durable worker over HTTP — register a medium, preflight it, archive an album,
-crash mid-transfer, recover, unmount the medium, rediscover the photos through
-their hashes while they are unreachable, mount it again, restore, and collide.
-Nothing is reached into: the medium is an ordinary directory whose marker file is
-its identity, and unmounting it means taking that marker away, which is what the
-application sees when a disk is unplugged.
-
-Three invariants are asserted wherever they apply, because they are what make the
-only *removing* stage safe:
-
-- **No source is removed before its archive copy is verified.** Every crash barrier
-  is checked for the file still being somewhere: either in the library, or on the
-  medium hashing exactly as recorded.
-- **Archived is not missing.** An unmounted medium leaves its photos
-  ``archived_offline``, still hashed, still deduplicable, still previewable.
-- **Restoring never overwrites.** A taken name comes back beside its occupant, and
-  the archived copy stays on the medium.
-"""
-
-from __future__ import annotations
-
-import hashlib
-import json
-import shutil
-from pathlib import Path
-
-import httpx
-import pytest
-
-from tests.e2e._pipeline_harness import (
-    ArchiveStack,
-    mark_uploaded,
-    seed_album,
-    session_factory,
-    wait_until,
-)
-
-pytestmark = pytest.mark.phase_f
-
-TIMEOUT = 20
-ALBUM = "rome"
-MANIFEST = "archive-manifest.jsonl"
-# Every persisted transition the transfer can die at, in the order it reaches them.
-BARRIERS = ["transferring", "verified", "removing", "source_removed", "complete"]
-
-
-@pytest.fixture
-def stack(tmp_path):
-    """An analysed album with verified upload evidence — archivable, nothing else."""
-    seeded = seed_album(tmp_path)
-    mark_uploaded(seeded)
-    running = ArchiveStack(tmp_path, seeded)
-    try:
-        yield running
-    finally:
-        running.stop()
-
-
-# ── helpers ──────────────────────────────────────────────────────────────────
-
-
-def _preflight(stack, location_id: str, **body) -> dict:
-    response = httpx.post(
-        f"{stack.base}/api/v1/archive-preflight",
-        json={"location_id": location_id, **body},
-        timeout=TIMEOUT,
-    )
-    response.raise_for_status()
-    return response.json()
-
-
-def _plan(stack, location_id: str, **body) -> dict:
-    report = _preflight(stack, location_id, **body)
-    response = httpx.post(
-        f"{stack.base}/api/v1/archive-plans",
-        json={"location_id": location_id, "token": report["token"], **body},
-        timeout=TIMEOUT,
-    )
-    response.raise_for_status()
-    return response.json()
-
-
-def _apply(stack, plan_id: str) -> httpx.Response:
-    return httpx.post(f"{stack.base}/api/v1/archive-plans/{plan_id}/apply", timeout=TIMEOUT)
-
-
-def _plan_state(stack, plan_id: str) -> dict:
-    return httpx.get(f"{stack.base}/api/v1/archive-plans/{plan_id}", timeout=TIMEOUT).json()
-
-
-def _await_plan(stack, plan_id: str, states=("complete", "failed"), *, timeout: float = 60) -> dict:
-    return wait_until(
-        lambda: (lambda p: p if p.get("state") in states else None)(_plan_state(stack, plan_id)),
-        timeout=timeout,
-    )
-
-
-def _archive_album(stack) -> dict:
-    """Register, preflight, plan, apply, and wait for the run to finish."""
-    location = stack.register()
-    plan = _plan(stack, location["id"])
-    _apply(stack, plan["id"]).raise_for_status()
-    return {"location": location, "plan": _await_plan(stack, plan["id"])}
-
-
-def _assets(stack) -> dict[str, dict]:
-    return {asset["id"]: asset for asset in stack.assets()}
-
-
-def _recovery(stack) -> dict:
-    return httpx.get(f"{stack.base}/api/v1/archive-recovery", timeout=TIMEOUT).json()
-
-
-def _resolve(stack) -> dict:
-    response = httpx.post(f"{stack.base}/api/v1/archive-recovery/resolve", timeout=TIMEOUT)
-    response.raise_for_status()
-    return response.json()
-
-
-def _restore(stack, location_id: str) -> dict:
-    report = httpx.post(
-        f"{stack.base}/api/v1/restore-preflight",
-        json={"location_id": location_id},
-        timeout=TIMEOUT,
-    ).json()
-    plan = httpx.post(
-        f"{stack.base}/api/v1/restore-plans",
-        json={"location_id": location_id, "token": report["token"]},
-        timeout=TIMEOUT,
-    )
-    plan.raise_for_status()
-    plan = plan.json()
-    httpx.post(
-        f"{stack.base}/api/v1/restore-plans/{plan['id']}/apply", timeout=TIMEOUT
-    ).raise_for_status()
-    wait_until(
-        lambda: all(a["availability_state"] == "active" for a in stack.assets()), timeout=60
-    )
-    return {"preflight": report, "plan": plan}
-
-
-def _sha256(path: Path) -> str:
-    return hashlib.sha256(path.read_bytes()).hexdigest()
-
-
-def _contents(*roots: Path) -> set[str]:
-    """Every photo byte-string reachable anywhere, keyed by hash."""
-    return {
-        _sha256(path)
-        for root in roots
-        for path in root.rglob("*.jpg")
-        if path.is_file() and not path.name.startswith(".")
-    }
-
-
-def _manifest(stack) -> list[dict]:
-    path = stack.archive / ALBUM / MANIFEST
-    if not path.exists():
-        return []
-    return [json.loads(line) for line in path.read_text().splitlines() if line.strip()]
-
-
-# ── preflight blockers ───────────────────────────────────────────────────────
-
-
-def test_preflight_refuses_an_offline_medium_a_wrong_volume_and_a_full_disk(stack):
-    stack.start(worker=False)
-    location = stack.register()
-
-    ready = _preflight(stack, location["id"])
-    assert ready["state"] == "ready"
-    assert ready["totals"]["assets"] == 2
-
-    stack.unmount()
-    offline = _preflight(stack, location["id"])
-    assert offline["state"] == "blocked"
-    assert "location_offline" in {issue["code"] for issue in offline["blockers"]}
-
-    # A different disk mounted at the same place is not this medium.
-    (stack.archive / ArchiveStack.MARKER).write_text('{"media_id": "another-disk"}')
-    wrong = _preflight(stack, location["id"])
-    assert "wrong_volume" in {issue["code"] for issue in wrong["blockers"]}
-
-    # Nothing has been created, moved, or removed by any of that.
-    assert stack.plans() == []
-    assert sorted(p.name for p in (stack.seeded.lib / ALBUM).glob("*.jpg")) == ["a.jpg", "b.jpg"]
-
-
-def test_preflight_refuses_capacity_it_cannot_prove_and_bytes_it_cannot_vouch_for(tmp_path):
-    seeded = seed_album(tmp_path)
-    mark_uploaded(seeded)
-    stack = ArchiveStack(tmp_path, seeded)
-    try:
-        # A reserve larger than any disk: the scope cannot be promised room.
-        stack.start(
-            worker=False,
-            extra_env={"PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": str(1 << 62)},
-        )
-        location = stack.register()
-        report = _preflight(stack, location["id"])
-        assert "insufficient_capacity" in {issue["code"] for issue in report["blockers"]}
-
-        # And a plan for a blocked scope is refused rather than half-created.
-        refused = httpx.post(
-            f"{stack.base}/api/v1/archive-plans",
-            json={"location_id": location["id"], "token": report["token"]},
-            timeout=TIMEOUT,
-        )
-        assert refused.status_code == 422
-        assert refused.json()["error"]["code"] == "blocked"
-        assert stack.plans() == []
-    finally:
-        stack.stop()
-
-
-def test_an_album_whose_bytes_changed_after_upload_cannot_be_archived(stack):
-    stack.start(worker=False)
-    location = stack.register()
-    (stack.seeded.lib / ALBUM / "a.jpg").write_bytes(b"edited after the upload")
-
-    report = _preflight(stack, location["id"])
-    album = report["albums"][0]
-    assert album["state"] == "blocked"
-    assert "partial_scope" in {issue["code"] for issue in album["blockers"]}
-    assert "bytes_changed" in {
-        issue["code"] for asset in album["assets"] for issue in asset["blockers"]
-    }
-    assert report["state"] == "blocked"
-
-
-# ── copy, verify, remove ─────────────────────────────────────────────────────
-
-
-def test_an_archived_album_leaves_the_library_only_after_it_is_verified(stack):
-    stack.start()
-    before = _contents(stack.seeded.lib)
-    hashes = {a["id"]: a["current_sha256"] for a in stack.assets()}
-
-    result = _archive_album(stack)
-    plan = result["plan"]
-
-    assert plan["state"] == "complete"
-    assert {op["journal_state"] for op in plan["operations"]} == {"complete"}
-    # Same filesystem here, so the transfer took the atomic-move path — recorded as
-    # what actually happened, not as the plan's guess.
-    assert all(op["same_filesystem"] for op in plan["operations"])
-
-    # The bytes are on the medium, hashing exactly as recorded, and gone from the
-    # library. Not one photo was lost in between.
-    assert _contents(stack.archive) == before
-    assert not list((stack.seeded.lib / ALBUM).glob("*.jpg"))
-    for asset_id, sha256 in hashes.items():
-        asset = _assets(stack)[asset_id]
-        assert asset["current_path"] is None
-        assert asset["availability_state"] == "archived_online"
-        assert _sha256(stack.archive / asset["archive_path"]) == sha256
-
-    # The medium carries its own record of what it holds.
-    manifest = _manifest(stack)
-    assert {entry["asset_id"] for entry in manifest} == set(hashes)
-    assert {entry["sha256"] for entry in manifest} == set(hashes.values())
-    assert {entry["media_id"] for entry in manifest} == {result["location"]["media_id"]}
-
-
-def test_the_archived_state_survives_a_server_restart(stack):
-    stack.start()
-    result = _archive_album(stack)
-    before = _assets(stack)
-    plan_before = _plan_state(stack, result["plan"]["id"])
-
-    stack.restart_server()
-
-    assert _assets(stack) == before
-    assert _plan_state(stack, result["plan"]["id"]) == plan_before
-    assert {a["availability_state"] for a in stack.assets()} == {"archived_online"}
-
-
-# ── interruption at every persisted transition ───────────────────────────────
-
-
-@pytest.mark.parametrize("barrier", BARRIERS)
-def test_a_worker_killed_at_each_transition_loses_nothing_and_recovers(tmp_path, barrier):
-    # One photo, so the crash lands on the only operation and the end state is the
-    # recovery's own doing rather than a mixture with untouched work.
-    seeded = seed_album(tmp_path, names=("a.jpg",))
-    mark_uploaded(seeded)
-    stack = ArchiveStack(tmp_path, seeded)
-    try:
-        # Only the worker carries the crash barrier: it dies the instant this journal
-        # state is persisted, while the server stays up to be asked what happened.
-        stack.start(worker=False)
-        stack.restart_worker(extra_env={"PHOTO_PIPELINE_FAULT_AFTER": barrier})
-        before = _contents(stack.seeded.lib)
-        location = stack.register()
-        plan = _plan(stack, location["id"])
-        _apply(stack, plan["id"]).raise_for_status()
-
-        wait_until(lambda: stack.worker.poll() is not None, timeout=60)
-        assert stack.worker.returncode in (9, -9), "the worker should have been killed"
-
-        # Whatever the crash interrupted, every photo is still readable somewhere.
-        assert before <= _contents(stack.seeded.lib, stack.archive), f"content lost at {barrier}"
-        # And nothing was removed that had not been verified first.
-        for operation in _plan_state(stack, plan["id"])["operations"]:
-            source = Path(operation["source_path"])
-            if not source.exists():
-                archived = Path(operation["destination_path"])
-                assert archived.exists(), f"{barrier}: source removed without an archive copy"
-                assert _sha256(archived) == operation["expected_sha256"]
-
-        # Recovery is offered from evidence, resolves without a worker, and converges.
-        verdicts = _recovery(stack)
-        assert all(v["classification"] != "manual" for v in verdicts["operations"]), verdicts
-        _resolve(stack)
-        assert _recovery(stack)["operations"] == []
-        # Repeating it changes nothing.
-        assert _resolve(stack) == {"resumed": 0, "completed": 0, "manual": 0}
-
-        # Recovery finishes what was durable and re-plans what was not. Before the
-        # archive copy existed, that means putting the item back to `planned` with the
-        # original untouched; from `verified` onwards it means completing it.
-        stack.restart_server()
-        operation = _plan_state(stack, plan["id"])["operations"][0]
-        asset = next(iter(stack.assets()))
-        if barrier == "transferring":
-            assert operation["journal_state"] == "planned"
-            assert Path(operation["source_path"]).exists()
-            assert asset["availability_state"] == "active"
-            assert _contents(stack.seeded.lib) == before
-        else:
-            assert operation["journal_state"] == "complete"
-            assert not Path(operation["source_path"]).exists()
-            assert _contents(stack.archive) == before
-            assert asset["availability_state"] == "archived_online"
-            assert _sha256(stack.archive / asset["archive_path"]) == asset["current_sha256"]
-            assert {entry["sha256"] for entry in _manifest(stack)} == before
-    finally:
-        stack.stop()
-
-
-def test_evidence_that_contradicts_the_journal_is_never_guessed(stack):
-    stack.start(worker=False)
-    location = stack.register()
-    plan = _plan(stack, location["id"])
-    # The journal claims a verified archive copy the medium does not have.
-    with session_factory(stack.seeded) as sf:
-        from photo_pipeline.services.archive_journal import ArchiveJournal
-
-        journal = ArchiveJournal(sf)
-        operation = journal.operations(plan["id"])[0]
-        journal.begin(operation["id"], worker_id="crashed", fencing_token=1)
-        journal.transition(operation["id"], "verified", fencing_token=1)
-
-    verdicts = _recovery(stack)
-    assert [v["classification"] for v in verdicts["operations"]] == ["manual"]
-    assert _resolve(stack)["manual"] == 1
-    # The source is untouched and the unresolved item keeps blocking a new archive.
-    assert (stack.seeded.lib / ALBUM / "a.jpg").exists()
-    blocked = _preflight(stack, location["id"])
-    assert "archive_pending" in {issue["code"] for issue in blocked["blockers"]}
-
-
-# ── offline identity and deduplication ───────────────────────────────────────
-
-
-def test_offline_photos_stay_hashed_previewable_and_deduplicable(stack):
-    stack.start()
-    result = _archive_album(stack)
-    archived = _assets(stack)
-    copy_source = stack.archive / next(iter(archived.values()))["archive_path"]
-    exact = stack.seeded.lib / "inbox" / "again.jpg"
-    exact.parent.mkdir(parents=True, exist_ok=True)
-    shutil.copy2(copy_source, exact)
-    stack.unmount()
-
-    scanned = httpx.post(f"{stack.base}/api/v1/inventory/scan", timeout=60)
-    scanned.raise_for_status()
-    # An unmounted medium is not a missing file.
-    assert scanned.json()["counts"].get("missing") is None
-    offline = {a["id"]: a for a in stack.assets() if a["id"] in archived}
-    assert {a["availability_state"] for a in offline.values()} == {"archived_offline"}
-    assert all(a["current_sha256"] for a in offline.values())
-
-    # The new active copy is recognised as the archived photo, not as a new one.
-    httpx.post(f"{stack.base}/api/v1/duplicates/detect", timeout=60).raise_for_status()
-    clusters = httpx.get(
-        f"{stack.base}/api/v1/duplicates/clusters", params={"limit": 50}, timeout=TIMEOUT
-    ).json()["items"]
-    exact_clusters = [c for c in clusters if c["method"] == "exact"]
-    assert len(exact_clusters) == 1
-    detail = httpx.get(
-        f"{stack.base}/api/v1/duplicates/clusters/{exact_clusters[0]['id']}", timeout=TIMEOUT
-    ).json()
-    canonical = next(m for m in detail["members"] if m["asset_id"] == detail["canonical_asset_id"])
-    assert canonical["availability_state"] == "archived_offline"
-    assert canonical["archive_location"] == result["location"]["name"]
-    assert canonical["preview"]["state"] == "ready" and canonical["preview"]["protected"]
-
-    # And the retained preview really is served while the medium is away.
-    thumbnail = httpx.get(
-        f"{stack.base}/api/v1/assets/{canonical['asset_id']}/thumbnail",
-        params={"size": 1280},
-        timeout=TIMEOUT,
-    )
-    assert thumbnail.status_code == 200
-    assert thumbnail.headers["content-type"] == "image/webp"
-
-
-def test_a_fuzzy_offline_match_asks_for_the_medium_instead_of_guessing(stack):
-    stack.start()
-    result = _archive_album(stack)
-    archived = next(iter(_assets(stack).values()))
-    variant = stack.seeded.lib / "inbox" / "resized.jpg"
-    variant.parent.mkdir(parents=True, exist_ok=True)
-    _resize(stack.archive / archived["archive_path"], variant)
-    stack.unmount()
-
-    httpx.post(f"{stack.base}/api/v1/inventory/scan", timeout=60).raise_for_status()
-    httpx.post(f"{stack.base}/api/v1/duplicates/detect", timeout=60).raise_for_status()
-    clusters = httpx.get(
-        f"{stack.base}/api/v1/duplicates/clusters", params={"limit": 50}, timeout=TIMEOUT
-    ).json()["items"]
-    perceptual = [c for c in clusters if c["method"] == "perceptual"]
-    assert len(perceptual) == 1
-
-    detail = httpx.get(
-        f"{stack.base}/api/v1/duplicates/clusters/{perceptual[0]['id']}", timeout=TIMEOUT
-    ).json()
-    # A fuzzy match is never decided automatically, and full-resolution review names
-    # the medium to mount rather than guessing from the preview.
-    assert detail["state"] == "open" and detail["requires_confirmation"] is True
-    assert detail["mount_required"] == [result["location"]["name"]]
-
-
-def test_mounting_the_medium_again_makes_the_originals_reachable(stack):
-    stack.start()
-    result = _archive_album(stack)
-    stack.unmount()
-    httpx.post(f"{stack.base}/api/v1/inventory/scan", timeout=60).raise_for_status()
-    assert {a["availability_state"] for a in stack.assets()} == {"archived_offline"}
-    # Restoring from a medium that is not there is refused, with its reason.
-    away = httpx.post(
-        f"{stack.base}/api/v1/restore-preflight",
-        json={"location_id": result["location"]["id"]},
-        timeout=TIMEOUT,
-    ).json()
-    assert away["state"] == "blocked"
-    assert "location_offline" in {issue["code"] for issue in away["blockers"]}
-
-    stack.remount()
-    locations = httpx.get(f"{stack.base}/api/v1/archive-locations", timeout=TIMEOUT).json()
-    assert locations["locations"][0]["state"] == "online"
-    assert {a["availability_state"] for a in stack.assets()} == {"archived_online"}
-
-
-# ── restore ──────────────────────────────────────────────────────────────────
-
-
-def test_restoring_returns_the_bytes_without_losing_identity_or_the_archive(stack):
-    stack.start()
-    result = _archive_album(stack)
-    identities = set(_assets(stack))
-    hashes = {a["id"]: a["current_sha256"] for a in stack.assets()}
-
-    _restore(stack, result["location"]["id"])
-
-    restored = _assets(stack)
-    assert set(restored) == identities  # the same photos, not new ones
-    for asset_id, sha256 in hashes.items():
-        asset = restored[asset_id]
-        assert asset["availability_state"] == "active"
-        assert _sha256(Path(asset["current_path"])) == sha256
-        # The archive copy is a copy: restoring empties nothing.
-        assert (stack.archive / asset["archive_path"]).exists()
-
-    stack.restart_server()
-    assert _assets(stack) == restored
-
-
-def test_a_restore_comes_back_beside_an_occupant_never_over_it(stack):
-    stack.start()
-    result = _archive_album(stack)
-    squatter = stack.seeded.lib / ALBUM / "a.jpg"
-    squatter.parent.mkdir(parents=True, exist_ok=True)
-    squatter.write_bytes(b"a different photo lives here now")
-
-    report = _restore(stack, result["location"]["id"])["preflight"]
-    destinations = {item["destination_path"] for item in report["items"]}
-    assert any("(restored)" in destination for destination in destinations)
-
-    assert squatter.read_bytes() == b"a different photo lives here now"
-    assert (stack.seeded.lib / ALBUM / "a (restored).jpg").exists()
-    assert {a["availability_state"] for a in stack.assets()} == {"active"}
-
-
-def _resize(source: Path, destination: Path, scale: float = 0.5) -> None:
-    from PIL import Image
-
-    with Image.open(source) as opened:
-        opened.resize(
-            (int(opened.width * scale), int(opened.height * scale)), Image.LANCZOS
-        ).save(destination, quality=95)

tests/e2e/test_review_ui.py

@@ -17,9 +17,6 @@ import httpx
 import numpy as np
 import pytest
 from PIL import Image
-from playwright.sync_api import expect
-
-from tests.conftest import session_client
 
 REPO = Path(__file__).resolve().parents[2]
 
@@ -114,7 +111,7 @@ def server(tmp_path):
         proc.terminate()
         pytest.fail("server never became ready")
 
-    client = session_client(base, timeout=5)
+    client = httpx.Client(base_url=base, timeout=5)
 
     def cluster_by_method(method):
         clusters = client.get("/api/v1/duplicates/clusters").json()["items"]
@@ -151,9 +148,9 @@ def test_fuzzy_decision_requires_confirmation(page, server):
     page.get_by_test_id("confirm").wait_for()
     assert "open" in page.get_by_test_id("cluster-state").inner_text()
     page.get_by_test_id("confirm-yes").click()
-    # A locator assertion, not wait_for_function: the app's CSP forbids eval, and
+    page.wait_for_function(
-    # Playwright's polling predicate is evaluated as a string in the page (US07-02).
+        "document.querySelector('[data-testid=cluster-state]').innerText.includes('dismissed')"
-    expect(page.get_by_test_id("cluster-state")).to_contain_text("dismissed")
+    )
 
 
 def test_stale_version_shows_conflict(page, server):
@@ -179,9 +176,9 @@ def test_decision_persists_after_reload(page, server):
     page.goto(f"{server.base}/app/#/duplicates/{cluster['id']}")
     page.get_by_test_id("not-duplicate").click()
     page.get_by_test_id("confirm-yes").click()
-    # A locator assertion, not wait_for_function: the app's CSP forbids eval, and
+    page.wait_for_function(
-    # Playwright's polling predicate is evaluated as a string in the page (US07-02).
+        "document.querySelector('[data-testid=cluster-state]').innerText.includes('dismissed')"
-    expect(page.get_by_test_id("cluster-state")).to_contain_text("dismissed")
+    )
     page.reload()
     page.get_by_test_id("cluster-state").wait_for()
     assert "dismissed" in page.get_by_test_id("cluster-state").inner_text()

tests/e2e/test_security.py

@@ -1,280 +0,0 @@
-"""US07-02 black-box security tests: authorization and path boundaries.
-
-Everything here talks to a real ``photo_pipeline serve`` child process over HTTP with
-its own ``httpx.Client``, deliberately outside the session helper the rest of the
-suite uses — an attacker does not get a bootstrapped client.
-
-The threat is a page in the user's browser, not a remote attacker: the app listens on
-127.0.0.1, so any site the user visits can send requests to it and can point an
-``<img>`` at its media endpoints. Each journey below is one of those attempts.
-"""
-
-from __future__ import annotations
-
-import json
-from contextlib import contextmanager
-from pathlib import Path
-
-import httpx
-import pytest
-
-from tests.e2e._pipeline_harness import Server, image, seed_library
-
-TIMEOUT = 20
-SENTINEL_KEY = "immich-sentinel-9f3a2b"
-SESSION_COOKIE = "pp_session"
-CSRF_HEADER = "X-CSRF-Token"
-
-# A GET, a mutation, and a media endpoint: the three shapes the policy must cover.
-PROTECTED = [
-    ("GET", "/api/v1/workflow", None),
-    ("POST", "/api/v1/albums/proposals", {}),
-    ("GET", "/api/v1/assets/{asset}/thumbnail?size=256", None),
-]
-
-
-@pytest.fixture(scope="module")
-def stack(tmp_path_factory):
-    """One server, one album of two photos, and a credential sentinel in its config."""
-    tmp_path = tmp_path_factory.mktemp("security")
-    seeded = seed_library(tmp_path, {"a": 1, "b": 2}, {"a": "sfw", "b": "sfw"})
-    outside = tmp_path / "outside"
-    outside.mkdir()
-    image(outside / "secret.jpg", 99)
-    server = Server(
-        seeded,
-        extra_env={
-            "PHOTO_PIPELINE_IMMICH_API_KEY": SENTINEL_KEY,
-            "PHOTO_PIPELINE_IMMICH_SERVER_URL": "http://127.0.0.1:1",
-        },
-    ).start()
-    yield server, seeded, outside
-    server.stop()
-
-
-@contextmanager
-def anonymous(server):
-    with httpx.Client(base_url=server.base, timeout=TIMEOUT) as client:
-        yield client
-
-
-@contextmanager
-def authenticated(server):
-    """A browser that has loaded the app: session cookie in the jar, token in a header."""
-    with httpx.Client(base_url=server.base, timeout=TIMEOUT) as client:
-        client.headers[CSRF_HEADER] = client.get("/api/v1/session").json()["csrf_token"]
-        yield client
-
-
-def call(client, method, path, body, asset):
-    return client.request(method, path.format(asset=asset), json=body)
-
-
-def test_the_api_refuses_every_caller_without_a_session(stack):
-    server, seeded, _ = stack
-    with anonymous(server) as client:
-        for method, path, body in PROTECTED:
-            response = call(client, method, path, body, seeded.asset_ids["a"])
-            assert response.status_code == 401, path
-            assert response.json()["error"]["code"] == "unauthenticated"
-        # Liveness and readiness stay open: an orchestrator holds no session.
-        assert client.get("/api/v1/health/ready").status_code == 200
-        assert client.get("/api/v1/health/live").status_code == 200
-
-
-def test_a_guessed_session_cookie_is_refused(stack):
-    server, _, _ = stack
-    with anonymous(server) as client:
-        client.cookies.set(SESSION_COOKIE, "guessed", domain="127.0.0.1")
-        response = client.get("/api/v1/workflow")
-        assert response.status_code == 401
-        assert response.json()["error"]["code"] == "unauthenticated"
-
-
-def test_the_bootstrap_issues_a_strict_httponly_cookie(stack):
-    server, _, _ = stack
-    with anonymous(server) as client:
-        response = client.get("/api/v1/session")
-        cookie = response.headers["set-cookie"].lower()
-        assert "httponly" in cookie and "samesite=strict" in cookie and "path=/" in cookie
-        assert response.json()["csrf_token"]
-        # The token is in the body, which no other origin may read: no CORS header
-        # grants access to it.
-        assert "access-control-allow-origin" not in response.headers
-
-
-def test_a_session_without_the_csrf_token_may_read_but_not_mutate(stack):
-    server, _, _ = stack
-    with authenticated(server) as client:
-        del client.headers[CSRF_HEADER]
-        assert client.get("/api/v1/albums/proposals").status_code == 200
-
-        response = client.post("/api/v1/albums/proposals", json={})
-        assert response.status_code == 403
-        assert response.json()["error"]["code"] == "csrf_failed"
-
-        response = client.post(
-            "/api/v1/albums/proposals", json={}, headers={CSRF_HEADER: "guessed"}
-        )
-        assert response.status_code == 403
-    # And nothing was created behind the refusal.
-    with authenticated(server) as client:
-        assert client.get("/api/v1/albums/proposals").json()["items"] == []
-
-
-def test_a_foreign_origin_cannot_mutate_even_with_a_session(stack):
-    server, _, _ = stack
-    with authenticated(server) as client:
-        for origin in ("http://evil.example", "http://127.0.0.1:1", "null"):
-            response = client.post(
-                "/api/v1/albums/proposals", json={}, headers={"Origin": origin}
-            )
-            assert response.status_code == 403, origin
-            assert response.json()["error"]["code"] == "origin_not_allowed"
-        # This app's own origin is accepted, so the check is not simply refusing all.
-        allowed = client.post(
-            "/api/v1/albums/proposals", json={}, headers={"Origin": server.base}
-        )
-        assert allowed.status_code == 200
-
-
-def test_a_rebinding_host_is_refused(stack):
-    """A name that resolves to 127.0.0.1 makes the browser treat the attacker's page
-    as same-origin. The Host header still carries that name, so it is checked."""
-    server, _, _ = stack
-    with authenticated(server) as client:
-        response = client.get("/api/v1/workflow", headers={"Host": "photos.evil.example"})
-        assert response.status_code == 403
-        assert response.json()["error"]["code"] == "host_not_allowed"
-
-
-def test_media_cannot_be_embedded_by_another_page(stack):
-    server, seeded, _ = stack
-    url = f"/api/v1/assets/{seeded.asset_ids['a']}/thumbnail?size=256"
-    with authenticated(server) as client:
-        # What an <img> on another site produces: no Origin, but a cross-site marker.
-        blocked = client.get(url, headers={"Sec-Fetch-Site": "cross-site"})
-        assert blocked.status_code == 403
-        assert blocked.json()["error"]["code"] == "cross_site_blocked"
-
-        served = client.get(url, headers={"Sec-Fetch-Site": "same-origin"})
-        assert served.status_code == 200
-        assert served.headers["content-type"] == "image/webp"
-        assert served.headers["cross-origin-resource-policy"] == "same-origin"
-        # Photos must never land in a shared cache.
-        assert served.headers["cache-control"].startswith("private")
-
-
-def test_every_response_carries_the_default_headers_and_no_cors(stack):
-    server, seeded, _ = stack
-    with authenticated(server) as client:
-        responses = [
-            client.get("/api/v1/workflow"),
-            client.get(f"/api/v1/assets/{seeded.asset_ids['a']}/thumbnail?size=256"),
-            client.get("/app/"),
-            client.get("/api/v1/does-not-exist"),
-        ]
-    for response in responses:
-        assert response.headers["x-content-type-options"] == "nosniff"
-        assert response.headers["x-frame-options"] == "DENY"
-        assert response.headers["referrer-policy"] == "no-referrer"
-        assert "frame-ancestors 'none'" in response.headers["content-security-policy"]
-        assert "access-control-allow-origin" not in response.headers
-        assert "access-control-allow-credentials" not in response.headers
-
-
-def test_a_traversal_attempt_addresses_nothing(stack):
-    """Identifiers are database keys, not paths: traversal has nowhere to land."""
-    server, _, _ = stack
-    with authenticated(server) as client:
-        for path in (
-            "/api/v1/albums/proposals/..%2F..%2F..%2Fetc%2Fpasswd",
-            "/api/v1/albums/proposals/../../../etc/passwd",
-            "/api/v1/assets/..%2F..%2Fetc%2Fpasswd/thumbnail?size=256",
-            "/api/v1/assets/%2Fetc%2Fpasswd/thumbnail?size=256",
-        ):
-            response = client.get(path)
-            assert response.status_code in (404, 422), path
-            assert "root:" not in response.text, path
-
-
-def test_an_oversized_request_is_refused_before_it_is_parsed(stack):
-    server, _, _ = stack
-    with authenticated(server) as client:
-        response = client.post(
-            "/api/v1/albums/proposals",
-            content=json.dumps({"albums": ["x" * 2_000_000]}),
-            headers={"Content-Type": "application/json"},
-        )
-        assert response.status_code == 413
-        assert response.json()["error"]["code"] == "payload_too_large"
-
-
-def test_a_malformed_request_reports_the_field_and_nothing_else(stack):
-    server, _, _ = stack
-    with authenticated(server) as client:
-        for content in ("{", '{"albums": 5}', ""):
-            response = client.post(
-                "/api/v1/albums/proposals",
-                content=content,
-                headers={"Content-Type": "application/json"},
-            )
-            assert response.status_code == 422, content
-            error = response.json()["error"]
-            assert error["code"] == "invalid_request"
-            assert "Traceback" not in response.text and "photo_pipeline/" not in response.text
-
-
-def test_errors_reveal_neither_the_credential_nor_an_internal(stack):
-    """The API key is configured but must appear nowhere; failures additionally say
-    nothing about where the library lives or how the server is built.
-
-    Successful responses are a different matter: path previews are the point of the
-    rename and archive views, and the operator is the one who owns those paths.
-    """
-    server, seeded, _ = stack
-    with authenticated(server) as client:
-        responses = [
-            client.post("/api/v1/upload-preflight", json={}),
-            client.get("/api/v1/workflow"),
-            client.get("/api/v1/upload-batches"),
-            client.get("/api/v1/assets/unknown-asset/thumbnail?size=256"),
-            client.get("/api/v1/archive-plans/unknown-plan"),
-            client.post("/api/v1/archive-locations", json={"name": "x", "root": "/nope"}),
-            client.get("/api/v1/does-not-exist"),
-        ]
-    for response in responses:
-        assert SENTINEL_KEY not in response.text
-        assert "Traceback" not in response.text
-        if response.status_code >= 400:
-            assert str(seeded.lib) not in response.text
-            assert "photo_pipeline/" not in response.text
-            assert "sqlite" not in response.text.lower()
-
-
-def test_a_symlink_swapped_under_an_asset_cannot_be_served(stack):
-    """TOCTOU on the media path: the file the database points at is replaced by a
-    link to something outside the library between the scan and the request."""
-    server, seeded, outside = stack
-    secret = outside / "secret.jpg"
-    original = seeded.lib / "b.jpg"
-    original.unlink()
-    original.symlink_to(secret)
-
-    with authenticated(server) as client:
-        response = client.get(f"/api/v1/assets/{seeded.asset_ids['b']}/thumbnail?size=256")
-    assert response.status_code == 403
-    assert response.json()["error"]["code"] == "path_not_allowed"
-    # The refusal names no filesystem location, and no bytes escaped with it.
-    assert str(outside) not in response.text and str(seeded.lib) not in response.text
-    assert secret.read_bytes()[:16] not in response.content
-
-
-def test_the_frontend_shell_stays_reachable_without_a_session(stack):
-    """It must load before any JavaScript can ask for a session."""
-    server, _, _ = stack
-    with anonymous(server) as client:
-        response = client.get("/app/", headers={"Sec-Fetch-Site": "none"})
-        assert response.status_code == 200
-        assert response.headers["content-type"].startswith("text/html")
-        assert Path("frontend/index.html").exists()

tests/e2e/test_traceability.py

@@ -14,7 +14,6 @@ MAP = json.loads((REPO / "tests" / "story_traceability.json").read_text())["stor
 PHASE_A_STORIES = {f"US01-0{n}" for n in range(1, 8)}
 PHASE_D_STORIES = {f"US04-0{n}" for n in range(1, 7)}
 PHASE_E_STORIES = {f"US05-0{n}" for n in range(1, 7)}
-PHASE_F_STORIES = {f"US06-0{n}" for n in range(1, 7)}
 
 
 def test_all_phase_a_stories_are_mapped():
@@ -33,13 +32,6 @@ def test_all_phase_e_stories_are_mapped():
     assert PHASE_E_STORIES <= set(MAP)
 
 
-def test_all_phase_f_stories_are_mapped():
-    """US06-06 acceptance: every archive-lifecycle story, destination through
-    restore, is tied to automated tests — archiving is the only stage that removes
-    an original."""
-    assert PHASE_F_STORIES <= set(MAP)
-
-
 def test_every_mapped_test_file_exists_and_is_nonempty():
     for story, files in MAP.items():
         assert files, f"{story} maps to no tests"

tests/fixtures/media_corpus.py

@@ -1,325 +0,0 @@
-"""The golden media corpus: every format, orientation, profile, damage, and
-metadata case this application claims to survive (US07-03, concept §18).
-
-``CASES`` is the manifest and the authority. Each entry declares a stable logical
-id (never a path), how the file is generated, and what the pipeline must do with
-it — decode it, or refuse it with one precise error code. Tests parametrize over the
-manifest, so a case that is added here without an expectation, or an expectation
-that stops holding, fails the suite rather than quietly going untested.
-
-Everything is generated, never committed: fixed pixel seeds, fixed EXIF strings, no
-clock, no network, no personal data. Regeneration is byte-stable, which
-``test_media_hardening.py`` proves by building the corpus twice and comparing
-checksums — a golden corpus that drifts is not golden.
-"""
-
-from __future__ import annotations
-
-import struct
-import zlib
-from dataclasses import dataclass, field
-from pathlib import Path
-from typing import Callable
-
-import numpy as np
-from PIL import Image
-
-# Error codes the pipeline may answer with; ``None`` means "must render".
-UNSUPPORTED = "unsupported_image"
-TOO_LARGE = "image_too_large"
-
-# One fixed capture time for every metadata case: the corpus must not depend on when
-# it was generated.
-CAPTURE_TIME = "2019:07:14 10:30:00"
-
-
-@dataclass(frozen=True)
-class MediaCase:
-    id: str
-    filename: str
-    kind: str  # format | orientation | profile | damaged | metadata
-    build: Callable[[Path], None]
-    expect_error: str | None = None
-    # Declared for the metadata cases: exiftool arguments applied after the pixels
-    # are written, and the user fields that must survive every later stage.
-    exif_args: tuple[str, ...] = ()
-    preserved_fields: tuple[str, ...] = ()
-    notes: str = ""
-    tags: tuple[str, ...] = field(default_factory=tuple)
-
-
-# ── generators ────────────────────────────────────────────────────────────────
-
-
-def _pixels(width: int, height: int, seed: int, bands: int = 3) -> np.ndarray:
-    return np.random.default_rng(seed).integers(0, 256, (height, width, bands), dtype=np.uint8)
-
-
-def _save(path: Path, image: Image.Image, **kwargs) -> None:
-    path.parent.mkdir(parents=True, exist_ok=True)
-    image.save(path, **kwargs)
-
-
-def _jpeg(width=320, height=240, seed=1, **kwargs):
-    def build(path: Path) -> None:
-        _save(path, Image.fromarray(_pixels(width, height, seed)), quality=90, **kwargs)
-
-    return build
-
-
-def _oriented_jpeg(orientation: int):
-    def build(path: Path) -> None:
-        image = Image.fromarray(_pixels(400, 200, 5))  # landscape source
-        exif = image.getexif()
-        exif[274] = orientation  # 0x0112 Orientation
-        _save(path, image, exif=exif, quality=90)
-
-    return build
-
-
-def _rotated_pixels(path: Path) -> None:
-    """The same scene rotated in the pixels instead of in a tag."""
-    _save(path, Image.fromarray(_pixels(400, 200, 5)).transpose(Image.ROTATE_90), quality=90)
-
-
-def _png(alpha: bool = False, seed: int = 2):
-    def build(path: Path) -> None:
-        if alpha:
-            _save(path, Image.fromarray(_pixels(120, 90, seed, bands=4), "RGBA"))
-        else:
-            _save(path, Image.fromarray(_pixels(120, 90, seed)))
-
-    return build
-
-
-def _webp(path: Path) -> None:
-    _save(path, Image.fromarray(_pixels(150, 100, 3)), quality=80)
-
-
-def _tiff(path: Path) -> None:
-    _save(path, Image.fromarray(_pixels(140, 110, 4)))
-
-
-def _grayscale(path: Path) -> None:
-    _save(path, Image.fromarray(_pixels(100, 100, 6)).convert("L"), quality=90)
-
-
-def _cmyk(path: Path) -> None:
-    _save(path, Image.fromarray(_pixels(100, 100, 7)).convert("CMYK"), quality=90)
-
-
-def _tiny(path: Path) -> None:
-    _save(path, Image.fromarray(_pixels(1, 1, 8)))
-
-
-def _icc_tagged(path: Path) -> None:
-    """A profile-bearing image: the colour-managed decode path must run."""
-    from PIL import ImageCms
-
-    profile = bytearray(ImageCms.ImageCmsProfile(ImageCms.createProfile("sRGB")).tobytes())
-    # An ICC header carries its creation timestamp at bytes 24..35. Left alone, the
-    # corpus would be a different corpus every time it is generated.
-    profile[24:36] = b"\x00" * 12
-    _save(path, Image.fromarray(_pixels(120, 80, 9)), icc_profile=bytes(profile), quality=90)
-
-
-def _broken_icc(path: Path) -> None:
-    """A profile that is not a profile: a picture is still a picture."""
-    _save(path, Image.fromarray(_pixels(120, 80, 10)), icc_profile=b"not-a-profile", quality=90)
-
-
-def _wide_jpeg(path: Path) -> None:
-    """Large enough that decoding it at full resolution is visible in memory."""
-    _save(path, Image.fromarray(_pixels(4000, 3000, 11)), quality=70)
-
-
-def _zero_byte(path: Path) -> None:
-    path.parent.mkdir(parents=True, exist_ok=True)
-    path.write_bytes(b"")
-
-
-def _truncated_jpeg(path: Path) -> None:
-    image = Image.fromarray(_pixels(400, 300, 12))
-    _save(path, image, quality=90)
-    data = path.read_bytes()
-    path.write_bytes(data[: len(data) // 2])  # header intact, pixels missing
-
-
-def _corrupt_png(path: Path) -> None:
-    image = Image.fromarray(_pixels(120, 90, 13))
-    _save(path, image)
-    data = bytearray(path.read_bytes())
-    data[40:80] = b"\x00" * 40  # shred the compressed stream, keep the header
-    path.write_bytes(bytes(data))
-
-
-def _not_an_image(path: Path) -> None:
-    path.parent.mkdir(parents=True, exist_ok=True)
-    path.write_bytes(b"This is a text file that happens to be named .jpg\n")
-
-
-def _png_declaring(width: int, height: int):
-    """A tiny, structurally valid PNG whose header claims an enormous picture.
-
-    A few hundred bytes on disk, gigapixels on paper: the pipeline must refuse it
-    from the declared dimensions, before a single pixel is allocated. Rewriting the
-    IHDR of a real PNG (rather than hand-rolling a stub) keeps the file openable, so
-    the refusal is proven to come from the size check and not from a parse failure.
-    """
-
-    def build(path: Path) -> None:
-        import io
-
-        buffer = io.BytesIO()
-        Image.fromarray(_pixels(4, 4, 15)).save(buffer, format="PNG")
-        data = bytearray(buffer.getvalue())
-        start = 8 + 4  # PNG signature, then the IHDR length field
-        struct.pack_into(">II", data, start + 4, width, height)
-        ihdr = bytes(data[start : start + 4 + 13])
-        struct.pack_into(">I", data, start + 4 + 13, zlib.crc32(ihdr))
-        path.parent.mkdir(parents=True, exist_ok=True)
-        path.write_bytes(bytes(data))
-
-    return build
-
-
-CASES: tuple[MediaCase, ...] = (
-    # ── formats ───────────────────────────────────────────────────────────────
-    MediaCase("jpeg", "formats/plain.jpg", "format", _jpeg()),
-    MediaCase("jpeg_uppercase_ext", "formats/UPPER.JPG", "format", _jpeg(seed=14)),
-    MediaCase("png", "formats/plain.png", "format", _png()),
-    MediaCase("png_alpha", "formats/alpha.png", "format", _png(alpha=True)),
-    MediaCase("webp", "formats/plain.webp", "format", _webp),
-    MediaCase("tiff", "formats/plain.tiff", "format", _tiff),
-    # ── orientation ───────────────────────────────────────────────────────────
-    *(
-        MediaCase(
-            f"orientation_{value}",
-            f"orientation/exif_{value}.jpg",
-            "orientation",
-            _oriented_jpeg(value),
-            notes="EXIF orientation must be applied before resizing",
-        )
-        for value in range(1, 9)
-    ),
-    MediaCase("rotated_pixels", "orientation/rotated.jpg", "orientation", _rotated_pixels),
-    # ── colour and size profiles ──────────────────────────────────────────────
-    MediaCase("grayscale", "profiles/gray.jpg", "profile", _grayscale),
-    MediaCase("cmyk", "profiles/cmyk.jpg", "profile", _cmyk),
-    MediaCase("tiny", "profiles/tiny.png", "profile", _tiny),
-    MediaCase("icc_tagged", "profiles/icc.jpg", "profile", _icc_tagged),
-    MediaCase(
-        "broken_icc",
-        "profiles/broken-icc.jpg",
-        "profile",
-        _broken_icc,
-        notes="an unusable ICC profile falls back to a plain conversion, never an error",
-    ),
-    MediaCase(
-        "large_jpeg",
-        "profiles/large.jpg",
-        "profile",
-        _wide_jpeg,
-        notes="12 megapixels: the decode must stay near the requested size",
-    ),
-    # ── damaged and hostile inputs ────────────────────────────────────────────
-    MediaCase("zero_byte", "damaged/empty.jpg", "damaged", _zero_byte, UNSUPPORTED),
-    MediaCase("truncated_jpeg", "damaged/truncated.jpg", "damaged", _truncated_jpeg, UNSUPPORTED),
-    MediaCase("corrupt_png", "damaged/corrupt.png", "damaged", _corrupt_png, UNSUPPORTED),
-    MediaCase("text_as_jpeg", "damaged/text.jpg", "damaged", _not_an_image, UNSUPPORTED),
-    MediaCase(
-        "bomb_header",
-        "damaged/bomb.png",
-        "damaged",
-        _png_declaring(60_000, 60_000),
-        TOO_LARGE,
-        notes="3.6 gigapixels declared in the header and nothing else",
-    ),
-    MediaCase(
-        "bomb_warning_band",
-        "damaged/bomb-warning.png",
-        "damaged",
-        _png_declaring(10_000, 10_000),
-        TOO_LARGE,
-        notes="inside Pillow's warn-only band; the warning is promoted to a refusal",
-    ),
-    # ── metadata ──────────────────────────────────────────────────────────────
-    MediaCase("no_exif", "metadata/bare.jpg", "metadata", _jpeg(seed=20)),
-    MediaCase(
-        "user_exif",
-        "metadata/user.jpg",
-        "metadata",
-        _jpeg(seed=21),
-        exif_args=(
-            "-Artist=Ada Lovelace",
-            "-Copyright=(c) Ada",
-            f"-DateTimeOriginal={CAPTURE_TIME}",
-            "-GPSLatitude=48.137",
-            "-GPSLatitudeRef=N",
-            "-Rating=4",
-            "-ImageDescription=A day out",
-        ),
-        preserved_fields=(
-            "EXIF:IFD0:Artist",
-            "EXIF:IFD0:Copyright",
-            "EXIF:ExifIFD:DateTimeOriginal",
-            "EXIF:IFD0:ImageDescription",
-            "XMP:XMP-xmp:Rating",
-        ),
-        notes="user metadata that every stage must leave exactly as it found it",
-    ),
-    MediaCase(
-        "prior_safety_keyword",
-        "metadata/prior-safety.jpg",
-        "metadata",
-        _jpeg(seed=22),
-        exif_args=("-Keywords+=nsfw", "-Subject+=nsfw", "-Artist=Ada Lovelace"),
-        preserved_fields=("EXIF:IFD0:Artist",),
-        notes="a safety decision already written by an earlier run",
-    ),
-    MediaCase(
-        "prior_analysis_keywords",
-        "metadata/prior-analysis.jpg",
-        "metadata",
-        _jpeg(seed=23),
-        exif_args=("-Keywords+=beach", "-Keywords+=sunset", "-Subject+=beach", "-Subject+=sunset"),
-        notes="analysis keywords from an earlier run; a safety write must not drop them",
-    ),
-    MediaCase(
-        "conflicting_safety_keywords",
-        "metadata/conflicting.jpg",
-        "metadata",
-        _jpeg(seed=24),
-        exif_args=("-Keywords+=sfw", "-Keywords+=nsfw", "-Subject+=sfw", "-Subject+=nsfw"),
-        notes="both safety keywords at once: mutually exclusive means one must go",
-    ),
-    MediaCase(
-        "malformed_metadata",
-        "metadata/malformed.jpg",
-        "metadata",
-        _jpeg(seed=25, exif=b"\x00\x01\x02not-a-valid-exif-block"),
-        notes="a broken EXIF block must not stop the picture from being usable",
-    ),
-)
-
-CASES_BY_ID = {case.id: case for case in CASES}
-
-
-def build_corpus(root: Path, *, ids: tuple[str, ...] | None = None) -> dict[str, Path]:
-    """Generate the corpus (or a named subset) under ``root``; return id → path."""
-    import subprocess
-
-    built: dict[str, Path] = {}
-    for case in CASES:
-        if ids is not None and case.id not in ids:
-            continue
-        path = root / case.filename
-        case.build(path)
-        if case.exif_args:
-            subprocess.run(
-                ["exiftool", "-m", "-overwrite_original", *case.exif_args, str(path)],
-                capture_output=True,
-                check=False,
-            )
-        built[case.id] = path
-    return built

tests/integration/test_archive_recovery.py

@@ -1,313 +0,0 @@
-"""Recovering interrupted archive transfers (US06-02).
-
-The crash tests are real: a child process applies an archive plan and is killed by
-the ``PHOTO_PIPELINE_FAULT_AFTER`` barrier at each persisted transition in turn —
-including the moment immediately after an active source has been unlinked. The
-parent then reopens the database and asserts the one invariant archiving exists to
-uphold: **no verified file is ever lost, and no source is removed without a durable,
-byte-identical archive copy.**
-
-Both transfer paths are exercised: the same-filesystem atomic move and (with the
-child forcing the device comparison) the cross-filesystem copy/verify/publish.
-"""
-
-import os
-import subprocess
-import sys
-import uuid
-from datetime import datetime, timezone
-from pathlib import Path
-
-import pytest
-from sqlalchemy import select
-
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.models import Asset, UploadBatch, UploadItem
-from photo_pipeline.services.archive_journal import FORWARD, MANUAL, RESUMABLE, ArchiveState
-from photo_pipeline.services.archive_transfer import (
-    MANIFEST_NAME,
-    ArchiveTransferService,
-    read_manifest,
-)
-from photo_pipeline.services.archives import ArchiveService
-from photo_pipeline.services.hashing import sha256_file
-
-pytestmark = pytest.mark.phase_f
-
-REPO = Path(__file__).resolve().parents[2]
-NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
-
-# Barriers, in the order the transfer persists them. ``source_removed`` is the one
-# that matters most: the original is already gone at that point.
-BARRIERS = [
-    ArchiveState.TRANSFERRING,
-    ArchiveState.VERIFIED,
-    ArchiveState.REMOVING,
-    "source_removed",
-    ArchiveState.COMPLETE,
-]
-
-# A child that applies the plan and dies at the configured barrier. ``force_copy``
-# makes it take the cross-filesystem path without a second real volume.
-APPLY_SCRIPT = """
-import sys
-sys.path.insert(0, {repo!r})
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory
-from photo_pipeline.services import archive_transfer
-
-db_url, data_dir, lib, plan_id, force_copy = sys.argv[1:6]
-if force_copy == "1":
-    archive_transfer._same_filesystem = lambda *args: False
-config = Config.from_env(
-    {{"PHOTO_PIPELINE_DATA_DIR": data_dir, "PHOTO_PIPELINE_LIBRARY_ROOTS": lib}}
-)
-sf = create_session_factory(create_db_engine(db_url))
-archive_transfer.ArchiveTransferService(sf, config=config).apply(plan_id)
-"""
-
-
-# ── environment ──────────────────────────────────────────────────────────────
-
-
-def _env(tmp_path):
-    (tmp_path / "data").mkdir(exist_ok=True)
-    lib = tmp_path / "lib"
-    lib.mkdir(exist_ok=True)
-    archive = tmp_path / "archive"
-    archive.mkdir(exist_ok=True)
-    config = Config.from_env(
-        {
-            "PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data"),
-            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
-            "PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": "0",
-        }
-    )
-    run_migrations(config.database_url)
-    return config, create_session_factory(create_db_engine(config.database_url)), lib, archive
-
-
-def _album(sf, lib, album="rome", names=("a.jpg", "b.jpg")):
-    folder = lib / album
-    folder.mkdir(parents=True, exist_ok=True)
-    with sf() as session:
-        batch_id = str(uuid.uuid4())
-        session.add(
-            UploadBatch(
-                id=batch_id,
-                album=album,
-                folder=str(folder),
-                album_name=album,
-                state="succeeded",
-                preflight_token="v1:test",
-                outcome_state="verified",
-                created_at=NOW,
-            )
-        )
-        for name in names:
-            path = folder / name
-            path.write_bytes(f"{album}/{name} content".encode() * 8)
-            asset_id = str(uuid.uuid4())
-            session.add(
-                Asset(
-                    id=asset_id,
-                    original_path=str(path),
-                    current_path=str(path),
-                    discovered_at=NOW,
-                    hash_version=1,
-                    byte_size=path.stat().st_size,
-                    current_sha256=sha256_file(path),
-                )
-            )
-            session.add(
-                UploadItem(
-                    batch_id=batch_id,
-                    asset_id=asset_id,
-                    path=str(path),
-                    sha256=sha256_file(path),
-                    sha1="0" * 40,
-                    state="sent",
-                    outcome="uploaded",
-                )
-            )
-        session.commit()
-    return folder
-
-
-def _plan(sf, config, archive, albums=None):
-    location = ArchiveService(sf, config=config).register("external", str(archive))
-    token = ArchiveService(sf, config=config).preflight(location["id"], albums)["token"]
-    service = ArchiveTransferService(sf, config=config)
-    return service, service.create(location["id"], albums, token=token)
-
-
-def _crash_during_apply(config, tmp_path, lib, plan, barrier, *, force_copy=False):
-    script = tmp_path / f"apply_{barrier}.py"
-    script.write_text(APPLY_SCRIPT.format(repo=str(REPO)))
-    env = dict(os.environ)
-    env["PHOTO_PIPELINE_FAULT_AFTER"] = barrier
-    result = subprocess.run(
-        [
-            sys.executable,
-            str(script),
-            config.database_url,
-            str(tmp_path / "data"),
-            str(lib),
-            plan["id"],
-            "1" if force_copy else "0",
-        ],
-        env=env,
-        capture_output=True,
-    )
-    assert result.returncode in (9, -9), (
-        f"child should have been killed at {barrier}, got {result.returncode}: "
-        f"{result.stderr.decode(errors='replace')[-400:]}"
-    )
-
-
-def _contents(*roots):
-    return sorted(
-        path.read_bytes()
-        for root in roots
-        for path in root.rglob("*")
-        if path.is_file() and path.name != MANIFEST_NAME and not path.name.startswith(".")
-    )
-
-
-def _reopen(config):
-    return create_session_factory(create_db_engine(config.database_url))
-
-
-# ── fault injection at every persisted transition ────────────────────────────
-
-
-@pytest.mark.parametrize("barrier", BARRIERS)
-@pytest.mark.parametrize("force_copy", [False, True], ids=["move", "copy"])
-def test_a_crash_at_every_transition_loses_nothing_and_recovers(tmp_path, barrier, force_copy):
-    config, sf, lib, archive = _env(tmp_path)
-    _album(sf, lib)
-    service, plan = _plan(sf, config, archive)
-    before = _contents(lib)
-
-    _crash_during_apply(config, tmp_path, lib, plan, barrier, force_copy=force_copy)
-
-    # Every file still exists somewhere: the crash may not have cost a single byte.
-    assert set(before) <= set(_contents(lib, archive)), f"content lost at {barrier}"
-
-    reopened = _reopen(config)
-    recovery = ArchiveTransferService(reopened, config=config)
-    # Whatever the crash interrupted is recoverable from evidence — never ambiguous.
-    # (A crash on the last barrier can land on a terminal item and leave none.)
-    verdicts = recovery.journal.classify_all()
-    assert all(v["classification"] in {RESUMABLE, FORWARD} for v in verdicts), verdicts
-
-    recovery.recover()
-    recovery.apply(plan["id"])  # finish whatever the crash never started
-
-    assert _contents(archive) == before
-    assert _contents(lib) == []
-    assert recovery.journal.blocks_mutation() is False
-    assert recovery.journal.plan_state(plan["id"]) == "complete"
-
-
-@pytest.mark.parametrize("barrier", [ArchiveState.VERIFIED, "source_removed"])
-def test_recovery_is_idempotent_across_repeated_restarts(tmp_path, barrier):
-    config, sf, lib, archive = _env(tmp_path)
-    _album(sf, lib)
-    service, plan = _plan(sf, config, archive)
-    _crash_during_apply(config, tmp_path, lib, plan, barrier)
-
-    reopened = _reopen(config)
-    recovery = ArchiveTransferService(reopened, config=config)
-    recovery.recover()
-    settled = (_contents(lib), _contents(archive), read_manifest(archive / "rome"))
-
-    for _ in range(2):
-        recovery.recover()
-        assert (_contents(lib), _contents(archive), read_manifest(archive / "rome")) == settled
-
-
-def test_a_crash_before_anything_was_published_leaves_the_source_intact(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder = _album(sf, lib, names=("a.jpg",))
-    service, plan = _plan(sf, config, archive)
-
-    _crash_during_apply(config, tmp_path, lib, plan, ArchiveState.TRANSFERRING)
-
-    recovery = ArchiveTransferService(_reopen(config), config=config)
-    verdict = recovery.journal.classify_all()[0]
-    assert verdict["classification"] == RESUMABLE
-    assert (folder / "a.jpg").exists()
-    assert not (archive / "rome" / "a.jpg").exists()
-
-    assert recovery.recover() == {"resumed": 1, "completed": 0, "manual": 0}
-    assert recovery.journal.operations(plan["id"])[0]["journal_state"] == ArchiveState.PLANNED
-
-
-def test_a_crash_after_the_source_was_removed_finishes_the_bookkeeping(tmp_path):
-    """The dangerous window: the original is gone and the database still points at
-    it. Recovery must complete the record, never re-transfer or report loss."""
-    config, sf, lib, archive = _env(tmp_path)
-    folder = _album(sf, lib, names=("a.jpg",))
-    service, plan = _plan(sf, config, archive)
-    expected = service.journal.operations(plan["id"])[0]["expected_sha256"]
-
-    _crash_during_apply(config, tmp_path, lib, plan, "source_removed")
-
-    reopened = _reopen(config)
-    recovery = ArchiveTransferService(reopened, config=config)
-    verdict = recovery.journal.classify_all()[0]
-    assert verdict["classification"] == FORWARD
-    assert not (folder / "a.jpg").exists()
-    assert sha256_file(archive / "rome" / "a.jpg") == expected
-    with reopened() as session:
-        stranded = session.scalar(select(Asset))
-        assert stranded.current_path is not None, "the crash happened before the DB update"
-
-    assert recovery.recover() == {"resumed": 0, "completed": 1, "manual": 0}
-
-    with reopened() as session:
-        asset = session.scalar(select(Asset))
-        assert asset.current_path is None
-        assert asset.availability_state == "archived_online"
-        assert asset.archive_path == "rome/a.jpg"
-    assert recovery.journal.operations(plan["id"])[0]["journal_state"] == ArchiveState.COMPLETE
-    assert len(read_manifest(archive / "rome")) == 1, "the manifest is not duplicated"
-
-
-def test_an_archive_copy_that_changed_after_the_crash_blocks_for_a_human(tmp_path):
-    """Wrong bytes at the destination can never justify deleting the original, and
-    are never overwritten either."""
-    config, sf, lib, archive = _env(tmp_path)
-    folder = _album(sf, lib, names=("a.jpg",))
-    service, plan = _plan(sf, config, archive)
-    _crash_during_apply(config, tmp_path, lib, plan, ArchiveState.VERIFIED, force_copy=True)
-    (archive / "rome" / "a.jpg").write_bytes(b"tampered with while the app was down")
-
-    recovery = ArchiveTransferService(_reopen(config), config=config)
-    verdict = recovery.journal.classify_all()[0]
-
-    assert verdict["classification"] == MANUAL
-    assert recovery.recover() == {"resumed": 0, "completed": 0, "manual": 1}
-    assert (folder / "a.jpg").exists(), "the source is kept while the archive is unproven"
-    assert (archive / "rome" / "a.jpg").read_bytes() == b"tampered with while the app was down"
-    # And the unresolved item keeps blocking further archiving.
-    assert recovery.journal.blocks_mutation() is True
-    report = ArchiveService(_reopen(config), config=config).preflight(plan["location_id"])
-    assert "archive_pending" in {issue["code"] for issue in report["blockers"]}
-
-
-def test_a_verified_archive_whose_copy_vanished_is_never_reported_as_archived(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder = _album(sf, lib, names=("a.jpg",))
-    service, plan = _plan(sf, config, archive)
-    _crash_during_apply(config, tmp_path, lib, plan, ArchiveState.REMOVING, force_copy=True)
-    (archive / "rome" / "a.jpg").unlink()  # the medium lost it
-
-    recovery = ArchiveTransferService(_reopen(config), config=config)
-
-    assert recovery.journal.classify_all()[0]["classification"] == MANUAL
-    assert recovery.recover()["manual"] == 1
-    assert (folder / "a.jpg").exists()
-    assert recovery.journal.operations(plan["id"])[0]["journal_state"] != ArchiveState.COMPLETE

tests/integration/test_archive_transfer.py

@@ -1,480 +0,0 @@
-"""Transferring, verifying, and removing active sources (US06-02).
-
-Every case here asks the same question the service exists to answer: could an
-original leave active storage without a durable, byte-identical archive copy? The
-files are real, the hashes are real, and each failure path asserts that the source
-is still exactly where it was.
-
-Cross-filesystem behaviour is forced by patching the device comparison rather than
-by requiring a second real filesystem in CI — the copy/verify/publish code that runs
-is the production one.
-"""
-
-import json
-import uuid
-from datetime import datetime, timezone
-
-import pytest
-from fastapi.testclient import TestClient
-from sqlalchemy import select
-
-from photo_pipeline.api.app import create_app
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.models import Asset, AssetPath, UploadBatch, UploadItem
-from photo_pipeline.services import archive_transfer as transfer_module
-from photo_pipeline.services.archive_journal import ArchiveState
-from photo_pipeline.services.archive_transfer import (
-    MANIFEST_NAME,
-    ArchiveTransferService,
-    read_manifest,
-)
-from photo_pipeline.services.archives import ArchiveError, ArchiveService
-from photo_pipeline.services.hashing import sha256_file
-
-pytestmark = pytest.mark.phase_f  # part of the Phase F acceptance gate (US06-06)
-
-NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
-
-
-# ── environment ──────────────────────────────────────────────────────────────
-
-
-def _env(tmp_path):
-    (tmp_path / "data").mkdir(exist_ok=True)
-    lib = tmp_path / "lib"
-    lib.mkdir(exist_ok=True)
-    archive = tmp_path / "archive"
-    archive.mkdir(exist_ok=True)
-    config = Config.from_env(
-        {
-            "PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data"),
-            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
-            "PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": "0",
-        }
-    )
-    run_migrations(config.database_url)
-    return config, create_session_factory(create_db_engine(config.database_url)), lib, archive
-
-
-def _album(sf, lib, album="rome", names=("a.jpg", "b.jpg")):
-    """A real album whose assets carry the verified upload evidence archiving needs."""
-    folder = lib / album
-    folder.mkdir(parents=True, exist_ok=True)
-    ids = []
-    with sf() as session:
-        batch_id = str(uuid.uuid4())
-        session.add(
-            UploadBatch(
-                id=batch_id,
-                album=album,
-                folder=str(folder),
-                album_name=album,
-                state="succeeded",
-                preflight_token="v1:test",
-                outcome_state="verified",
-                created_at=NOW,
-            )
-        )
-        for name in names:
-            path = folder / name
-            path.write_bytes(f"{album}/{name} bytes".encode() * 8)
-            asset_id = str(uuid.uuid4())
-            ids.append(asset_id)
-            session.add(
-                Asset(
-                    id=asset_id,
-                    original_path=str(path),
-                    current_path=str(path),
-                    discovered_at=NOW,
-                    hash_version=1,
-                    byte_size=path.stat().st_size,
-                    current_sha256=sha256_file(path),
-                )
-            )
-            session.add(AssetPath(asset_id=asset_id, path=str(path), valid_from=NOW))
-            session.add(
-                UploadItem(
-                    batch_id=batch_id,
-                    asset_id=asset_id,
-                    path=str(path),
-                    sha256=sha256_file(path),
-                    sha1="0" * 40,
-                    state="sent",
-                    outcome="uploaded",
-                )
-            )
-        session.commit()
-    return folder, ids
-
-
-def _location(sf, config, archive, name="external"):
-    return ArchiveService(sf, config=config).register(name, str(archive))
-
-
-def _plan(sf, config, location_id, albums=None):
-    service = ArchiveTransferService(sf, config=config)
-    token = ArchiveService(sf, config=config).preflight(location_id, albums)["token"]
-    return service, service.create(location_id, albums, token=token)
-
-
-def _contents(*roots):
-    """Every file's bytes under the given roots, ignoring our own bookkeeping."""
-    return sorted(
-        path.read_bytes()
-        for root in roots
-        for path in root.rglob("*")
-        if path.is_file() and path.name != MANIFEST_NAME and not path.name.startswith(".")
-    )
-
-
-def _states(service, plan_id):
-    return [row["journal_state"] for row in service.journal.operations(plan_id)]
-
-
-def _asset(sf, asset_id):
-    with sf() as session:
-        return session.get(Asset, asset_id)
-
-
-# ── planning ─────────────────────────────────────────────────────────────────
-
-
-def test_a_plan_records_every_file_with_its_expected_hash_and_moves_nothing(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder, ids = _album(sf, lib)
-    location = _location(sf, config, archive)
-    before = _contents(lib)
-
-    service, plan = _plan(sf, config, location["id"])
-
-    assert plan["state"] == "planned" and plan["asset_count"] == 2
-    assert sorted(op["asset_id"] for op in plan["operations"]) == sorted(ids)
-    for operation in plan["operations"]:
-        source = folder / operation["archive_path"].split("/")[-1]
-        assert operation["expected_sha256"] == sha256_file(source)
-        assert operation["archive_path"].startswith("rome/")
-        assert operation["journal_state"] == ArchiveState.PLANNED
-    assert _contents(lib) == before
-    assert _contents(archive) == []
-
-
-def test_a_stale_token_cannot_create_a_plan(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder, _ = _album(sf, lib)
-    location = _location(sf, config, archive)
-    token = ArchiveService(sf, config=config).preflight(location["id"])["token"]
-
-    (folder / "a.jpg").write_bytes(b"edited after approval")
-
-    with pytest.raises(ArchiveError) as error:
-        ArchiveTransferService(sf, config=config).create(location["id"], token=token)
-    assert error.value.code == "stale_token"
-
-
-def test_a_blocked_scope_cannot_create_a_plan(tmp_path):
-    """No verified upload means Immich may not hold these bytes; archiving would
-    remove the only copy."""
-    config, sf, lib, archive = _env(tmp_path)
-    folder = lib / "rome"
-    folder.mkdir()
-    path = folder / "a.jpg"
-    path.write_bytes(b"never uploaded")
-    with sf() as session:
-        session.add(
-            Asset(
-                id=str(uuid.uuid4()),
-                original_path=str(path),
-                current_path=str(path),
-                discovered_at=NOW,
-                hash_version=1,
-                byte_size=path.stat().st_size,
-                current_sha256=sha256_file(path),
-            )
-        )
-        session.commit()
-    location = _location(sf, config, archive)
-    token = ArchiveService(sf, config=config).preflight(location["id"])["token"]
-
-    with pytest.raises(ArchiveError) as error:
-        ArchiveTransferService(sf, config=config).create(location["id"], token=token)
-
-    assert error.value.code == "blocked"
-    assert path.exists()
-
-
-# ── the transfer ─────────────────────────────────────────────────────────────
-
-
-@pytest.mark.parametrize("same_filesystem", [True, False])
-def test_archiving_verifies_the_copy_before_the_source_is_removed(
-    tmp_path, monkeypatch, same_filesystem
-):
-    config, sf, lib, archive = _env(tmp_path)
-    folder, ids = _album(sf, lib)
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"])
-    expected = {op["asset_id"]: op["expected_sha256"] for op in plan["operations"]}
-    before = _contents(lib)
-    if not same_filesystem:
-        # Force the copy-verify-publish path without needing a second real volume.
-        monkeypatch.setattr(transfer_module, "_same_filesystem", lambda *_: False)
-
-    result = service.apply(plan["id"])
-
-    assert result == {
-        "plan_id": plan["id"],
-        "archived": 2,
-        "failed": 0,
-        "skipped": 0,
-        "state": "complete",
-    }
-    # The bytes moved: nothing is left in the library, everything is in the archive.
-    assert _contents(archive) == before
-    assert _contents(lib) == []
-    assert not folder.exists(), "an emptied album folder is not left behind"
-    for asset_id, digest in expected.items():
-        archived = archive / "rome" / _asset(sf, asset_id).archive_path.split("/")[-1]
-        assert sha256_file(archived) == digest
-    assert _states(service, plan["id"]) == [ArchiveState.COMPLETE] * 2
-    # No transfer temporaries survive either path.
-    assert not list((archive / "rome").glob(".archive-*"))
-
-
-@pytest.mark.parametrize("same_filesystem", [True, False])
-def test_the_manifest_on_the_medium_matches_the_archived_bytes(
-    tmp_path, monkeypatch, same_filesystem
-):
-    """The manifest is the medium's own record: it must be usable to verify the
-    archive with no database at all."""
-    config, sf, lib, archive = _env(tmp_path)
-    _album(sf, lib)
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"])
-    if not same_filesystem:
-        monkeypatch.setattr(transfer_module, "_same_filesystem", lambda *_: False)
-
-    service.apply(plan["id"])
-
-    entries = read_manifest(archive / "rome")
-    assert len(entries) == 2
-    for entry in entries:
-        archived = archive / entry["archive_path"]
-        assert sha256_file(archived) == entry["sha256"]
-        assert entry["byte_size"] == archived.stat().st_size
-        assert entry["media_id"] == location["media_id"]
-        assert entry["plan_id"] == plan["id"] and entry["album"] == "rome"
-
-
-def test_archived_assets_keep_their_identity_and_gain_their_new_location(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder, ids = _album(sf, lib)
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"])
-
-    service.apply(plan["id"])
-
-    with sf() as session:
-        for asset_id in ids:
-            asset = session.get(Asset, asset_id)
-            assert asset is not None, "archiving never deletes the record"
-            assert asset.current_path is None
-            assert asset.availability_state == "archived_online"
-            assert asset.archive_location_id == location["id"]
-            assert (archive / asset.archive_path).exists()
-            occurrences = session.scalars(
-                select(AssetPath).where(AssetPath.asset_id == asset_id)
-            ).all()
-            active = [row for row in occurrences if row.valid_until is None]
-            assert [row.path for row in active] == [str(archive / asset.archive_path)]
-            closed = [row for row in occurrences if row.valid_until is not None]
-            assert [row.path for row in closed] == [str(folder / asset.archive_path.split("/")[-1])]
-
-
-def test_applying_the_same_plan_again_changes_nothing(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    _album(sf, lib)
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"])
-    service.apply(plan["id"])
-    archived = _contents(archive)
-    manifest = read_manifest(archive / "rome")
-
-    again = service.apply(plan["id"])
-
-    assert again["skipped"] == 2 and again["archived"] == 0 and again["failed"] == 0
-    assert _contents(archive) == archived
-    assert read_manifest(archive / "rome") == manifest
-
-
-# ── unexpected changes stop the item ─────────────────────────────────────────
-
-
-def test_an_occupied_destination_is_never_overwritten(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder, _ = _album(sf, lib, names=("a.jpg",))
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"])
-    (archive / "rome").mkdir()
-    (archive / "rome" / "a.jpg").write_bytes(b"someone else's file")
-
-    result = service.apply(plan["id"])
-
-    assert result["failed"] == 1 and result["state"] == "failed"
-    assert (archive / "rome" / "a.jpg").read_bytes() == b"someone else's file"
-    assert (folder / "a.jpg").exists(), "the source must survive a refused transfer"
-    assert service.journal.operations(plan["id"])[0]["error_code"] == "destination_exists"
-
-
-def test_a_source_edited_after_planning_is_neither_archived_nor_removed(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder, _ = _album(sf, lib, names=("a.jpg",))
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"])
-
-    (folder / "a.jpg").write_bytes(b"edited between approval and apply")
-    result = service.apply(plan["id"])
-
-    assert result["failed"] == 1
-    assert (folder / "a.jpg").read_bytes() == b"edited between approval and apply"
-    assert not (archive / "rome" / "a.jpg").exists()
-    assert service.journal.operations(plan["id"])[0]["error_code"] == "source_changed"
-
-
-def test_a_copy_that_lands_with_the_wrong_bytes_is_not_published(tmp_path, monkeypatch):
-    """The read-back hash — not the fact that a write returned — is what proves the
-    archive copy."""
-    config, sf, lib, archive = _env(tmp_path)
-    folder, _ = _album(sf, lib, names=("a.jpg",))
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"])
-    monkeypatch.setattr(transfer_module, "_same_filesystem", lambda *_: False)
-    monkeypatch.setattr(
-        transfer_module.shutil,
-        "copyfileobj",
-        lambda src, dst, length=0: dst.write(b"corrupted in flight"),
-    )
-
-    result = service.apply(plan["id"])
-
-    assert result["failed"] == 1
-    assert service.journal.operations(plan["id"])[0]["error_code"] == "copy_mismatch"
-    assert (folder / "a.jpg").exists()
-    assert not (archive / "rome" / "a.jpg").exists()
-    assert not list((archive / "rome").glob(".archive-*")), "the failed copy is cleaned up"
-
-
-def test_a_source_replaced_by_a_symlink_is_refused(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder, _ = _album(sf, lib, names=("a.jpg",))
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"])
-    elsewhere = tmp_path / "elsewhere.jpg"
-    elsewhere.write_bytes(b"not a library file")
-    (folder / "a.jpg").unlink()
-    (folder / "a.jpg").symlink_to(elsewhere)
-
-    result = service.apply(plan["id"])
-
-    assert result["failed"] == 1
-    assert elsewhere.exists() and (folder / "a.jpg").is_symlink()
-    assert not (archive / "rome" / "a.jpg").exists()
-
-
-def test_one_failed_file_does_not_stop_the_rest_of_the_album(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder, _ = _album(sf, lib, names=("a.jpg", "b.jpg"))
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"])
-    (folder / "a.jpg").write_bytes(b"edited after approval")
-
-    result = service.apply(plan["id"])
-
-    assert (result["archived"], result["failed"]) == (1, 1)
-    assert (folder / "a.jpg").exists() and not (folder / "b.jpg").exists()
-    assert (archive / "rome" / "b.jpg").exists()
-    assert folder.exists(), "a folder that still holds an unarchived file stays"
-
-
-def test_an_unresolved_transfer_blocks_the_next_preflight(tmp_path, monkeypatch):
-    config, sf, lib, archive = _env(tmp_path)
-    _album(sf, lib, "rome", names=("a.jpg",))
-    _album(sf, lib, "paris", names=("c.jpg",))
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"], ["rome"])
-    # Leave one item stuck mid-transfer, as a killed process would.
-    service.journal.begin(
-        service.journal.operations(plan["id"])[0]["id"], worker_id="w", fencing_token=1
-    )
-
-    report = ArchiveService(sf, config=config).preflight(location["id"], ["paris"])
-
-    assert report["state"] == "blocked"
-    assert "archive_pending" in {issue["code"] for issue in report["blockers"]}
-
-
-# ── API surface ──────────────────────────────────────────────────────────────
-
-
-def test_api_creates_a_plan_and_queues_it_on_the_archiver_lane(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    _album(sf, lib)
-
-    with TestClient(create_app(config)) as client:
-        location = client.post(
-            "/api/v1/archive-locations", json={"name": "external", "root": str(archive)}
-        ).json()
-        token = client.post(
-            "/api/v1/archive-preflight", json={"location_id": location["id"]}
-        ).json()["token"]
-        created = client.post(
-            "/api/v1/archive-plans", json={"location_id": location["id"], "token": token}
-        )
-        plan_id = created.json()["id"]
-        fetched = client.get(f"/api/v1/archive-plans/{plan_id}")
-        applied = client.post(f"/api/v1/archive-plans/{plan_id}/apply")
-        listed = client.get("/api/v1/archive-plans")
-
-    assert created.status_code == 201 and created.json()["asset_count"] == 2
-    assert fetched.status_code == 200 and len(fetched.json()["operations"]) == 2
-    assert applied.status_code == 200 and applied.json()["job"]["state"] == "queued"
-    assert applied.json()["job"]["lock_key"] == "archive"
-    assert [row["id"] for row in listed.json()["plans"]] == [plan_id]
-    # Queuing alone must not have touched a single file.
-    assert _contents(lib) and _contents(archive) == []
-
-
-def test_api_refuses_a_stale_token_and_an_unknown_plan(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    _album(sf, lib)
-    location = _location(sf, config, archive)
-
-    with TestClient(create_app(config)) as client:
-        stale = client.post(
-            "/api/v1/archive-plans",
-            json={"location_id": location["id"], "token": "v1:not-the-real-token"},
-        )
-        unknown = client.get("/api/v1/archive-plans/nope")
-        unknown_apply = client.post("/api/v1/archive-plans/nope/apply")
-
-    assert stale.status_code == 409 and stale.json()["error"]["code"] == "stale_token"
-    assert unknown.status_code == 404 and unknown_apply.status_code == 404
-
-
-def test_api_reports_recovery_state_for_an_interrupted_transfer(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    _album(sf, lib, names=("a.jpg",))
-    location = _location(sf, config, archive)
-    service, plan = _plan(sf, config, location["id"])
-    service.journal.begin(
-        service.journal.operations(plan["id"])[0]["id"], worker_id="w", fencing_token=1
-    )
-
-    with TestClient(create_app(config)) as client:
-        status = client.get("/api/v1/archive-recovery").json()
-        resolved = client.post("/api/v1/archive-recovery/resolve").json()
-
-    assert status["blocks_mutation"] is True
-    assert status["operations"][0]["classification"] == "resumable"
-    assert resolved == {"resumed": 1, "completed": 0, "manual": 0}
-    assert json.loads(json.dumps(resolved))  # plain JSON, nothing exotic

tests/integration/test_concurrency_races.py

@@ -1,535 +0,0 @@
-"""Randomized concurrency and race tests (US07-04, concept §16 and §18).
-
-Every test here runs several times with a *recorded* seed: the seed decides the
-jitter injected around each racing operation, it is attached to the test result
-(``race_seed``), and a failing run can be replayed exactly with
-
-    PHOTO_PIPELINE_RACE_SEED=<seed> pytest tests/integration/test_concurrency_races.py
-
-``PHOTO_PIPELINE_RACE_REPEATS`` raises the repeat count for a soak run; the
-default is small enough to belong in the ordinary suite.
-
-The assertions are invariants, not schedules — a race whose interleaving decides
-the *outcome* is fine, one whose interleaving decides whether the database still
-makes sense is not:
-
-- no work is claimed, executed, or completed twice;
-- no commit from a stale fencing token lands;
-- no file is lost, overwritten, or left with foreign content;
-- a decision that changed mid-flight is never overwritten by the older answer;
-- the database passes ``PRAGMA integrity_check`` afterwards.
-"""
-
-from __future__ import annotations
-
-import os
-import random
-import subprocess
-import sys
-import threading
-import time
-import uuid
-from collections import Counter
-from concurrent.futures import ThreadPoolExecutor
-from datetime import datetime, timedelta, timezone
-from pathlib import Path
-
-import numpy as np
-import pytest
-from PIL import Image
-from sqlalchemy import select, text
-
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.faults import JOB_ITEM_DONE
-from photo_pipeline.jobs.worker import Worker
-from photo_pipeline.models import (
-    AlbumProposal,
-    AnalysisResult,
-    Asset,
-    ExifProjection,
-    SafetyReview,
-)
-from photo_pipeline.services import hashing
-from photo_pipeline.services.analysis import AnalysisService
-from photo_pipeline.services.jobs import (
-    InvalidTransition,
-    ItemState,
-    JobConflict,
-    JobService,
-    JobState,
-    _now,
-)
-from photo_pipeline.services.rename_apply import RenameApplyService
-from photo_pipeline.services.renames import RenameService
-from photo_pipeline.services.thumbnails import ThumbnailError, ThumbnailService
-
-REPO = Path(__file__).resolve().parents[2]
-NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
-REPEATS = int(os.environ.get("PHOTO_PIPELINE_RACE_REPEATS", "3"))
-MAX_JITTER_SECONDS = 0.01
-
-
-# ── seeded randomness ────────────────────────────────────────────────────────
-
-
-@pytest.fixture(params=range(REPEATS), ids=lambda run: f"run{run}")
-def rng(request, record_property):
-    """A seeded RNG whose seed is recorded on the test result.
-
-    Without a pinned seed the run is genuinely random — which is the point, a
-    fixed schedule stops finding new interleavings after the first green run —
-    so the seed is reported for replay instead.
-    """
-    pinned = os.environ.get("PHOTO_PIPELINE_RACE_SEED")
-    seed = int(pinned) + request.param if pinned else random.SystemRandom().randrange(2**32)
-    record_property("race_seed", seed)
-    print(f"race seed: {seed}")  # visible with -s and in the failure report
-    return random.Random(seed)
-
-
-def jitter(rng: random.Random) -> None:
-    """Sleep a random sliver so racing threads interleave differently each run."""
-    time.sleep(rng.uniform(0, MAX_JITTER_SECONDS))
-
-
-# ── fixtures ─────────────────────────────────────────────────────────────────
-
-
-def _config(tmp_path) -> tuple[Config, Path]:
-    (tmp_path / "data").mkdir(exist_ok=True)
-    lib = tmp_path / "lib"
-    lib.mkdir(exist_ok=True)
-    config = Config.from_env(
-        {
-            "PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data"),
-            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
-        }
-    )
-    run_migrations(config.database_url)
-    return config, lib
-
-
-@pytest.fixture
-def stack(tmp_path):
-    """Config, library root, and a factory for *independent* session factories —
-    each thread must own its session (concept §16 database rule 1)."""
-    config, lib = _config(tmp_path)
-    engines = []
-
-    def factory():
-        engine = create_db_engine(config.database_url)
-        engines.append(engine)
-        return create_session_factory(engine)
-
-    yield config, lib, factory
-    for engine in engines:
-        engine.dispose()
-
-
-def image(path: Path, seed: int) -> None:
-    path.parent.mkdir(parents=True, exist_ok=True)
-    pixels = np.random.default_rng(seed).integers(0, 256, (64, 96, 3), dtype=np.uint8)
-    Image.fromarray(pixels).save(path, quality=90)
-
-
-def register(sf, path: Path) -> str:
-    asset_id = str(uuid.uuid4())
-    with sf() as session:
-        session.add(
-            Asset(
-                id=asset_id,
-                original_path=str(path),
-                current_path=str(path),
-                discovered_at=NOW,
-                hash_version=1,
-                byte_size=path.stat().st_size,
-                current_sha256=hashing.sha256_file(path),
-            )
-        )
-        session.commit()
-    return asset_id
-
-
-def album(sf, lib: Path, name: str, *, approved_name: str, count: int = 2) -> list[str]:
-    """A real folder of real images with an approved proposal renaming it."""
-    folder = lib / name
-    ids = []
-    for index in range(count):
-        path = folder / f"{name}-{index}.jpg"
-        image(path, seed=abs(hash((name, index))) % 10_000)
-        ids.append(register(sf, path))
-    with sf() as session:
-        session.add(
-            AlbumProposal(
-                id=str(uuid.uuid4()),
-                album=name,
-                proposed_name=approved_name,
-                final_name=approved_name,
-                status="approved",
-                version=2,
-            )
-        )
-        session.commit()
-    return ids
-
-
-def contents(lib: Path) -> Counter:
-    """Every file body under the library — what may never be lost or duplicated."""
-    return Counter(
-        path.read_bytes()
-        for path in lib.rglob("*")
-        if path.is_file() and ".rename-" not in path.name
-    )
-
-
-def integrity_ok(sf) -> bool:
-    with sf() as session:
-        return session.execute(text("PRAGMA integrity_check")).scalar() == "ok"
-
-
-# ── database writer pressure ─────────────────────────────────────────────────
-
-
-def test_many_writers_finish_without_losing_a_row(stack, rng):
-    """Eight lanes committing at once: SQLite has one writer, so this either works
-    through the busy timeout or loses data. Nothing may be lost."""
-    _, _, factory = stack
-    service = JobService(factory())
-    job_ids = [service.enqueue("scan", items=[f"i{n}" for n in range(4)])["id"] for n in range(8)]
-
-    def write(index: int) -> None:
-        own = JobService(factory())
-        job_id = job_ids[index]
-        token = own.claim(["scan"], f"w{index}")
-        if token is None:
-            return
-        for item in [f"i{n}" for n in range(4)]:
-            jitter(rng)
-            own.set_item(token["id"], item, ItemState.RUNNING, fencing_token=token["fencing_token"])
-            own.set_item(
-                token["id"], item, ItemState.SUCCEEDED, fencing_token=token["fencing_token"]
-            )
-        assert job_id  # the claim order is racy; every job is claimed by someone
-
-    with ThreadPoolExecutor(max_workers=8) as pool:
-        list(pool.map(write, range(8)))
-
-    for job_id in job_ids:
-        progress = service.progress(job_id)
-        assert progress["by_state"].get(ItemState.SUCCEEDED) == 4, progress
-    assert integrity_ok(factory())
-
-
-# ── worker claim races ───────────────────────────────────────────────────────
-
-
-def test_a_job_is_claimed_by_exactly_one_worker(stack, rng):
-    _, _, factory = stack
-    service = JobService(factory())
-    job_ids = {service.enqueue("scan", items=["a"])["id"] for _ in range(10)}
-
-    claimed: list[str] = []
-    lock = threading.Lock()
-
-    def claim_all(index: int) -> None:
-        own = JobService(factory())
-        while True:
-            jitter(rng)
-            job = own.claim(["scan"], f"w{index}")
-            if job is None:
-                return
-            with lock:
-                claimed.append(job["id"])
-
-    with ThreadPoolExecutor(max_workers=4) as pool:
-        list(pool.map(claim_all, range(4)))
-
-    assert sorted(claimed) == sorted(job_ids), "every job claimed exactly once"
-    assert len(set(claimed)) == len(claimed)
-
-
-def test_a_late_commit_from_an_expired_lease_is_refused(stack, rng):
-    """Lease expiry then a late write from the old owner: the fencing token, not
-    timing, decides who may commit."""
-    _, _, factory = stack
-    service = JobService(factory())
-    job = service.enqueue("scan", items=["a"])
-    old = service.claim(["scan"], "old")
-    service.recover_stale(now=_now() + timedelta(hours=1))
-    new = service.claim(["scan"], "new")
-
-    jitter(rng)
-    with pytest.raises(JobConflict):
-        service.set_item(job["id"], "a", ItemState.RUNNING, fencing_token=old["fencing_token"])
-    with pytest.raises(JobConflict):
-        service.transition(job["id"], JobState.SUCCEEDED, fencing_token=old["fencing_token"])
-
-    # The current owner finishes the same work without interference.
-    service.set_item(job["id"], "a", ItemState.RUNNING, fencing_token=new["fencing_token"])
-    service.set_item(job["id"], "a", ItemState.SUCCEEDED, fencing_token=new["fencing_token"])
-    service.transition(job["id"], JobState.SUCCEEDED, fencing_token=new["fencing_token"])
-    assert service.get(job["id"])["state"] == JobState.SUCCEEDED
-    assert integrity_ok(factory())
-
-
-def test_cancellation_arriving_at_a_random_moment_leaves_a_consistent_job(stack, rng):
-    _, _, factory = stack
-    service = JobService(factory())
-    job = service.enqueue("scan", items=[f"i{n}" for n in range(6)])
-    handled: list[str] = []
-
-    def handler(item, ctx):
-        jitter(rng)
-        handled.append(item)
-
-    def cancel() -> None:
-        try:
-            service.cancel(job["id"])
-        except (JobConflict, InvalidTransition):
-            # The request lost its race with the worker's own transition; the job
-            # simply finishes. What must never happen is a *silent* overwrite.
-            pass
-
-    canceller = threading.Timer(rng.uniform(0, 0.03), cancel)
-    canceller.start()
-    Worker(factory(), {"scan": handler}, "w1").run_once()
-    canceller.join()
-
-    state = service.get(job["id"])["state"]
-    assert state in {JobState.SUCCEEDED, JobState.CANCELLED, JobState.CANCELLING}
-    by_state = service.progress(job["id"])["by_state"]
-    assert by_state.get(ItemState.RUNNING, 0) == 0, "no item left mid-flight"
-    assert len(handled) == len(set(handled)), "no item ran twice"
-    if state == JobState.CANCELLED:
-        # Whatever did not run is resumable, never silently dropped.
-        assert by_state.get(ItemState.SUCCEEDED, 0) + by_state.get(ItemState.QUEUED, 0) == 6
-    assert integrity_ok(factory())
-
-
-KILLABLE_WORKER = """
-import sys
-sys.path.insert(0, {repo!r})
-from pathlib import Path
-from photo_pipeline.db import create_db_engine, create_session_factory
-from photo_pipeline.jobs.worker import Worker
-
-db_url, log = sys.argv[1], Path(sys.argv[2])
-
-def handler(item_key, ctx):
-    with log.open("a") as handle:
-        handle.write(item_key + "\\n")
-
-sf = create_session_factory(create_db_engine(db_url))
-Worker(sf, {{"scan": handler}}, sys.argv[3], lease_seconds=1).run_once()
-"""
-
-
-def test_a_worker_killed_at_a_random_item_resumes_exactly_once(stack, rng, tmp_path):
-    """Kill a real worker mid-batch, restart it, and assert the invariant across
-    the restart: every item ran exactly once and the job ends succeeded."""
-    config, _, factory = stack
-    service = JobService(factory())
-    items = [f"i{n}" for n in range(rng.randint(2, 5))]
-    job = service.enqueue("scan", items=items)
-    log = tmp_path / "handled.log"
-    script = tmp_path / "killable_worker.py"
-    script.write_text(KILLABLE_WORKER.format(repo=str(REPO)))
-
-    def run_worker(worker_id: str, barrier: str | None) -> None:
-        env = dict(os.environ)
-        env.pop("PHOTO_PIPELINE_FAULT_AFTER", None)
-        if barrier:
-            env["PHOTO_PIPELINE_FAULT_AFTER"] = barrier
-        subprocess.run(
-            [sys.executable, str(script), config.database_url, str(log), worker_id],
-            env=env,
-            capture_output=True,
-        )
-
-    run_worker("killable", JOB_ITEM_DONE)  # dies after its first completed item
-    assert log.read_text().split() == items[:1]
-    time.sleep(1.1)  # let the dead worker's one-second lease expire
-    jitter(rng)
-    run_worker("survivor", None)
-
-    assert service.get(job["id"])["state"] == JobState.SUCCEEDED
-    handled = log.read_text().split()
-    assert sorted(handled) == sorted(items), f"an item ran twice or not at all: {handled}"
-    assert integrity_ok(factory())
-
-
-# ── filesystem races ─────────────────────────────────────────────────────────
-
-
-def test_a_file_changed_during_apply_never_loses_content(stack, rng):
-    """The user edits a photo while its folder is being renamed. The rename may
-    win or be refused, but no body may disappear or be overwritten."""
-    config, lib, factory = stack
-    sf = factory()
-    album(sf, lib, "rome", approved_name="2019 Rome")
-    plan = RenameService(sf, library_roots=(lib,)).build_plan()
-    before = contents(lib)
-
-    edited = b"the user saved over this file"
-
-    def edit() -> None:
-        jitter(rng)
-        target = next((lib / "rome").glob("*.jpg"), None)
-        if target is None:
-            return  # the rename won the race; the folder already moved
-        try:
-            target.write_bytes(edited)
-        except OSError:
-            pass  # ...or it moved between the glob and the write
-
-    thread = threading.Thread(target=edit)
-    thread.start()
-    RenameApplyService(sf, library_roots=(lib,)).apply(plan["id"], expected_version=plan["version"])
-    thread.join()
-
-    after = contents(lib)
-    assert sum(after.values()) == sum(before.values()), "a file was lost or duplicated"
-    survived = (before - Counter({edited: 1})) & after
-    assert sum(survived.values()) >= sum(before.values()) - 1, "unrelated content was destroyed"
-    assert integrity_ok(sf)
-
-
-def test_two_folders_claiming_one_destination_never_merge(stack, rng):
-    """Two approved albums want the same name. The plan must refuse rather than
-    move one folder into the other."""
-    config, lib, factory = stack
-    sf = factory()
-    album(sf, lib, "rome-a", approved_name="2019 Rome")
-    album(sf, lib, "rome-b", approved_name="2019 Rome")
-    jitter(rng)
-
-    plan = RenameService(sf, library_roots=(lib,)).build_plan()
-    codes = {issue["code"] for op in plan["operations"] for issue in op["issues"]}
-
-    assert "duplicate_target" in codes
-    assert plan["applicable"] is False
-    assert (lib / "rome-a").is_dir() and (lib / "rome-b").is_dir()
-
-
-def test_thumbnail_requests_racing_a_rename_never_serve_a_wrong_file(stack, rng):
-    """Previews are keyed by pixels, not paths, so a rename must not make a request
-    fail loudly *or* return another asset's picture."""
-    config, lib, factory = stack
-    sf = factory()
-    asset_ids = album(sf, lib, "rome", approved_name="2019 Rome")
-    plan = RenameService(sf, library_roots=(lib,)).build_plan()
-
-    thumbnails = ThumbnailService(factory(), config)
-    expected = {aid: thumbnails.generate(aid, 256).read_bytes() for aid in asset_ids}
-    served: dict[str, set[bytes]] = {aid: set() for aid in asset_ids}
-    errors: list[str] = []
-    stop = threading.Event()
-
-    def serve() -> None:
-        own = ThumbnailService(factory(), config)
-        while not stop.is_set():
-            for asset_id in asset_ids:
-                jitter(rng)
-                try:
-                    served[asset_id].add(own.generate(asset_id, 256).read_bytes())
-                except ThumbnailError as error:
-                    errors.append(error.code)  # precise, never an unhandled crash
-
-    reader = threading.Thread(target=serve)
-    reader.start()
-    try:
-        RenameApplyService(sf, library_roots=(lib,)).apply(
-            plan["id"], expected_version=plan["version"]
-        )
-    finally:
-        stop.set()
-        reader.join()
-
-    for asset_id, bodies in served.items():
-        assert bodies <= {expected[asset_id]}, "a request served another asset's picture"
-    assert integrity_ok(sf)
-
-
-# ── stage races ──────────────────────────────────────────────────────────────
-
-
-def test_an_analysis_result_racing_a_safety_flip_is_discarded(stack, rng):
-    """Concept §18 scenario 7: the reviewer marks an asset NSFW while the provider
-    call is in flight. The answer that comes back describes an asset that may no
-    longer be analysed, so it is dropped — and no analysis EXIF is written."""
-    _, lib, factory = stack
-    sf = factory()
-    path = lib / "beach.jpg"
-    image(path, seed=7)
-    asset_id = register(sf, path)
-    with sf() as session:
-        session.add(
-            SafetyReview(id=str(uuid.uuid4()), asset_id=asset_id, decision="sfw", created_at=NOW)
-        )
-        session.commit()
-    before_sha = hashing.sha256_file(path)
-
-    class FlippingProvider:
-        """Records the call, then the reviewer's decision lands mid-flight."""
-
-        def __init__(self) -> None:
-            self.calls: list[str] = []
-
-        def analyze(self, path, *, album_hint):
-            self.calls.append(path)
-            jitter(rng)
-            with sf() as session:
-                session.add(
-                    SafetyReview(
-                        id=str(uuid.uuid4()),
-                        asset_id=asset_id,
-                        decision="nsfw",
-                        prior_decision="sfw",
-                        created_at=NOW + timedelta(minutes=1),
-                    )
-                )
-                session.commit()
-            return {"description": "a beach", "tags": ["beach", "sand"]}
-
-    provider = FlippingProvider()
-    result = AnalysisService(sf, provider=provider, library_roots=(lib,)).run([asset_id])
-
-    assert provider.calls == [str(path)], "the call was legitimate when it started"
-    assert result == {"analyzed": 0, "skipped": 1, "errors": 0}
-    with sf() as session:
-        row = session.get(AnalysisResult, asset_id)
-        assert row.status == "skipped_nsfw" and row.description is None
-        assert session.get(ExifProjection, (asset_id, "analysis")) is None
-    assert hashing.sha256_file(path) == before_sha, "the file was written after the flip"
-
-
-def test_a_safety_decision_taken_twice_at_once_keeps_one_history(stack, rng):
-    """Two windows, one asset, the same decision: the audit trail may record both
-    attempts, but the effective state must be a single coherent decision."""
-    _, lib, factory = stack
-    sf = factory()
-    path = lib / "twice.jpg"
-    image(path, seed=11)
-    asset_id = register(sf, path)
-
-    from photo_pipeline.services.safety import SafetyService
-
-    def decide(decision: str) -> None:
-        jitter(rng)
-        SafetyService(factory()).decide(asset_id, decision, write_exif=False)
-
-    with ThreadPoolExecutor(max_workers=2) as pool:
-        list(pool.map(decide, ["sfw", "sfw"]))
-
-    with sf() as session:
-        decisions = [
-            review.decision
-            for review in session.scalars(
-                select(SafetyReview)
-                .where(SafetyReview.asset_id == asset_id)
-                .order_by(SafetyReview.created_at)
-            )
-        ]
-    assert decisions and set(decisions) == {"sfw"}
-    assert integrity_ok(sf)

tests/integration/test_exif_checkpoints.py

@@ -1,297 +0,0 @@
-"""US07-03: EXIF checkpoints, asserted with before/after metadata snapshots.
-
-Every test here reads the complete metadata of a real file through exiftool before
-the stage runs and again afterwards, then compares the two. That is the only way to
-prove the property the concept actually asks for: a stage owns a few fields and must
-leave literally everything else — dates, GPS, camera, artist, rating, other people's
-keywords — exactly as it found them.
-
-The other half is divergence. When something outside the stage's ownership does move,
-the checkpoint must say so, refuse to call itself verified, and change nothing back:
-a silent repair is how a library quietly loses the user's metadata.
-"""
-
-from __future__ import annotations
-
-import json
-import shutil
-import subprocess
-import uuid
-from datetime import datetime, timezone
-from types import SimpleNamespace
-
-import pytest
-
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.integrations import exiftool
-from photo_pipeline.models import Asset, ExifProjection
-from photo_pipeline.services import exif_checkpoint
-from photo_pipeline.services.analysis import AnalysisService
-from photo_pipeline.services.safety import SafetyService
-from tests.fixtures.media_corpus import CASES_BY_ID, build_corpus
-
-pytestmark = pytest.mark.skipif(
-    shutil.which("exiftool") is None, reason="exiftool not installed"
-)
-
-
-@pytest.fixture
-def env(tmp_path):
-    data = tmp_path / "data"
-    data.mkdir()
-    lib = tmp_path / "lib"
-    files = build_corpus(
-        lib,
-        ids=(
-            "user_exif",
-            "prior_safety_keyword",
-            "prior_analysis_keywords",
-            "conflicting_safety_keywords",
-            "malformed_metadata",
-            "no_exif",
-        ),
-    )
-    config = Config.from_env(
-        {"PHOTO_PIPELINE_DATA_DIR": str(data), "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib)}
-    )
-    run_migrations(config.database_url)
-    engine = create_db_engine(config.database_url)
-    sf = create_session_factory(engine)
-    assets = {}
-    with sf() as session:
-        for case_id, path in files.items():
-            asset = Asset(
-                id=str(uuid.uuid4()),
-                original_path=str(path),
-                current_path=str(path),
-                discovered_at=datetime.now(timezone.utc),
-                hash_version=1,
-            )
-            session.add(asset)
-            assets[case_id] = asset.id
-        session.commit()
-    yield SimpleNamespace(config=config, lib=lib, sf=sf, files=files, assets=assets)
-    engine.dispose()
-
-
-class Provider:
-    """A vision provider whose tags are fixed, so the EXIF assertion is the test."""
-
-    def __init__(self, tags):
-        self.tags = list(tags)
-
-    def analyze(self, path, *, album_hint):
-        return {"description": "a photo", "tags": self.tags}
-
-
-def snapshot(path):
-    return exiftool.read_all(str(path))
-
-
-def stage_fields(before, after):
-    """Everything that changed except this application's owned and volatile tags."""
-    return exif_checkpoint.compare(before, after)
-
-
-# ── preservation ──────────────────────────────────────────────────────────────
-
-
-def test_a_safety_decision_preserves_every_user_field(env):
-    case = CASES_BY_ID["user_exif"]
-    path = env.files["user_exif"]
-    before = snapshot(path)
-    assert before, "the fixture must actually carry user metadata"
-
-    result = SafetyService(env.sf).decide(env.assets["user_exif"], "nsfw")
-    after = snapshot(path)
-
-    assert result["exif_verified"] is True
-    assert stage_fields(before, after) == ()
-    for field in case.preserved_fields:
-        assert after[field] == before[field], field
-    assert "nsfw" in exif_checkpoint.owned_values(after)
-    assert "sfw" not in exif_checkpoint.owned_values(after)
-
-
-def test_analysis_keywords_are_additive_and_keep_the_safety_decision(env):
-    """The two stages share the Keywords field; the second must merge, not replace."""
-    asset_id = env.assets["prior_analysis_keywords"]
-    path = env.files["prior_analysis_keywords"]
-    SafetyService(env.sf).decide(asset_id, "sfw")
-    before = snapshot(path)
-
-    AnalysisService(
-        env.sf, provider=Provider(["harbour", "boats"]), library_roots=(env.lib,)
-    ).run([asset_id])
-    after = snapshot(path)
-
-    keywords = exif_checkpoint.owned_values(after)
-    assert {"sfw", "beach", "sunset", "harbour", "boats"} <= keywords
-    assert "nsfw" not in keywords
-    assert stage_fields(before, after) == ()
-
-
-def test_flipping_a_safety_decision_removes_only_the_opposite_keyword(env):
-    asset_id = env.assets["prior_analysis_keywords"]
-    path = env.files["prior_analysis_keywords"]
-    safety = SafetyService(env.sf)
-    safety.decide(asset_id, "nsfw")
-    before = snapshot(path)
-
-    safety.decide(asset_id, "sfw")
-    after = snapshot(path)
-
-    keywords = exif_checkpoint.owned_values(after)
-    assert "sfw" in keywords and "nsfw" not in keywords
-    assert {"beach", "sunset"} <= keywords, "analysis keywords are not safety's to remove"
-    assert stage_fields(before, after) == ()
-
-
-def test_conflicting_safety_keywords_are_resolved_to_one(env):
-    asset_id = env.assets["conflicting_safety_keywords"]
-    SafetyService(env.sf).decide(asset_id, "sfw")
-    keywords = exif_checkpoint.owned_values(snapshot(env.files["conflicting_safety_keywords"]))
-    assert keywords & {"sfw", "nsfw"} == {"sfw"}
-
-
-def test_malformed_metadata_does_not_block_the_checkpoint(env):
-    result = SafetyService(env.sf).decide(env.assets["malformed_metadata"], "nsfw")
-    assert result["exif_verified"] is True
-
-
-# ── verification and the refreshed hash ───────────────────────────────────────
-
-
-def test_a_verified_checkpoint_refreshes_the_recorded_bytes(env):
-    """exiftool rewrites the container, so the stored SHA-256 must be the new one —
-    upload compares against exactly these bytes."""
-    asset_id = env.assets["no_exif"]
-    path = env.files["no_exif"]
-    SafetyService(env.sf).decide(asset_id, "sfw")
-
-    with env.sf() as session:
-        asset = session.get(Asset, asset_id)
-        row = session.get(ExifProjection, (asset_id, "safety"))
-    import hashlib
-
-    on_disk = hashlib.sha256(path.read_bytes()).hexdigest()
-    assert asset.current_sha256 == on_disk
-    assert row.state == "verified" and row.result_file_sha256 == on_disk
-    assert json.loads(row.desired_json) == {"add": ["sfw"], "remove": ["nsfw"]}
-
-
-def test_the_projection_survives_a_restart(env):
-    asset_id = env.assets["user_exif"]
-    SafetyService(env.sf).decide(asset_id, "nsfw")
-
-    engine = create_db_engine(env.config.database_url)  # a fresh connection, as a restart is
-    try:
-        with create_session_factory(engine)() as session:
-            row = session.get(ExifProjection, (asset_id, "safety"))
-            assert row.state == "verified" and row.verified_at is not None
-    finally:
-        engine.dispose()
-
-
-# ── divergence ────────────────────────────────────────────────────────────────
-
-
-def test_a_field_the_stage_does_not_own_changing_is_divergent(env, monkeypatch):
-    """Something rewrote the artist while the safety keyword was being written.
-
-    The stage must not call that verified, must record what moved, and must not put
-    the old value back — the file is now a question for a human.
-    """
-    asset_id = env.assets["user_exif"]
-    path = env.files["user_exif"]
-    real_apply = exiftool.apply_keywords
-
-    def sabotage(target, *, add=(), remove=()):
-        ok = real_apply(target, add=add, remove=remove)
-        subprocess.run(
-            ["exiftool", "-m", "-overwrite_original", "-Artist=Someone Else", str(target)],
-            capture_output=True,
-            check=False,
-        )
-        return ok
-
-    monkeypatch.setattr(exiftool, "apply_keywords", sabotage)
-    result = SafetyService(env.sf).decide(asset_id, "nsfw")
-
-    assert result["exif_verified"] is False, "a divergent checkpoint is not verified"
-    with env.sf() as session:
-        row = session.get(ExifProjection, (asset_id, "safety"))
-        asset = session.get(Asset, asset_id)
-    assert row.state == "divergent"
-    assert "EXIF:IFD0:Artist" in json.loads(row.divergent_fields)
-    assert row.verified_at is None
-    # Not repaired, and not silently accepted as the current verified bytes.
-    assert snapshot(path)["EXIF:IFD0:Artist"] == "Someone Else"
-    assert asset.current_sha256 is None
-
-
-def test_a_divergent_asset_shows_up_in_the_review_queue(env, monkeypatch):
-    asset_id = env.assets["user_exif"]
-    monkeypatch.setattr(
-        exif_checkpoint,
-        "run",
-        lambda *args, **kwargs: exif_checkpoint.CheckpointResult(
-            exif_checkpoint.DIVERGENT, changed_fields=("EXIF:IFD0:Artist",), sha256="abc"
-        ),
-    )
-    SafetyService(env.sf).decide(asset_id, "sfw")
-
-    rows = SafetyService(env.sf).review_queue()["items"]
-    row = next(item for item in rows if item["asset_id"] == asset_id)
-    assert row["exif_state"] == "divergent"
-    assert row["exif_verified"] is False
-
-
-def test_a_write_that_does_not_take_is_a_failure_not_a_verification(env, monkeypatch):
-    monkeypatch.setattr(exiftool, "apply_keywords", lambda *a, **k: False)
-    result = SafetyService(env.sf).decide(env.assets["no_exif"], "sfw")
-    assert result["exif_verified"] is False
-    assert exif_checkpoint.state_for(env.sf, env.assets["no_exif"], "safety") == "failed"
-
-
-def test_unreadable_metadata_is_a_failure_not_an_empty_snapshot(env, monkeypatch):
-    """``None`` from exiftool means "cannot answer"; treating it as "nothing there"
-    would make every field look preserved."""
-    monkeypatch.setattr(exiftool, "read_all", lambda path: None)
-    result = exif_checkpoint.run(str(env.files["no_exif"]), add=("sfw",))
-    assert result.state == "failed" and result.reason == "metadata_unreadable"
-
-
-def test_analysis_records_its_own_projection_separately(env):
-    asset_id = env.assets["user_exif"]
-    SafetyService(env.sf).decide(asset_id, "sfw")
-    AnalysisService(env.sf, provider=Provider(["pier"]), library_roots=(env.lib,)).run([asset_id])
-
-    assert exif_checkpoint.state_for(env.sf, asset_id, "safety") == "verified"
-    assert exif_checkpoint.state_for(env.sf, asset_id, "analysis") == "verified"
-
-
-# ── the comparison rules themselves ───────────────────────────────────────────
-
-
-def test_compare_ignores_owned_and_volatile_fields_only():
-    before = {
-        "EXIF:IFD0:Artist": "Ada",
-        "IPTC:Keywords": ["holiday"],
-        "File:System:FileSize": "3.8 kB",
-        "File:CurrentIPTCDigest": "aaa",
-        "XMP:XMP-x:XMPToolkit": "old",
-    }
-    after = {
-        "EXIF:IFD0:Artist": "Ada",
-        "IPTC:Keywords": ["holiday", "sfw"],
-        "File:System:FileSize": "3.9 kB",
-        "File:CurrentIPTCDigest": "bbb",
-        "XMP:XMP-x:XMPToolkit": "new",
-    }
-    assert exif_checkpoint.compare(before, after) == ()
-
-    after["EXIF:GPS:GPSLatitude"] = "48.1"  # an addition counts as much as a loss
-    del after["EXIF:IFD0:Artist"]
-    assert exif_checkpoint.compare(before, after) == ("EXIF:GPS:GPSLatitude", "EXIF:IFD0:Artist")

tests/integration/test_fault_matrix.py

@@ -1,527 +0,0 @@
-"""The fault matrix (US07-04, concept §18 "crash/fault-injection tests").
-
-Process death at each persisted transition lives in tests/e2e/test_crash_recovery.py
-and the per-stage recovery suites. This file covers the *environmental* faults —
-the ones that are not a crash but are just as good at corrupting a library if the
-code guesses:
-
-    disk full · read-only path · database busy · database corruption ·
-    network failure · malformed provider output · GPU exhaustion ·
-    subprocess hang · missing external tool
-
-Every case asserts the same shape of outcome: the operation fails visibly, the
-failure names what happened, and nothing irreversible was done on the way — no
-source removed, no metadata marked verified, no decision invented.
-"""
-
-from __future__ import annotations
-
-import errno
-import os
-import sqlite3
-import stat
-import threading
-import time
-import uuid
-from contextlib import contextmanager
-from datetime import datetime, timezone
-from pathlib import Path
-
-import numpy as np
-import pytest
-from PIL import Image
-from sqlalchemy import select, text
-from sqlalchemy.exc import DatabaseError, OperationalError
-
-from photo_pipeline import faults
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.jobs.worker import Worker
-from photo_pipeline.models import (
-    AlbumProposal,
-    AnalysisResult,
-    Asset,
-    ExifProjection,
-    SafetyReview,
-    UploadBatch,
-    UploadItem,
-)
-from photo_pipeline.services import archive_transfer, exif_checkpoint, hashing, rename_apply
-from photo_pipeline.services.analysis import AnalysisService
-from photo_pipeline.services.archive_transfer import ArchiveTransferService
-from photo_pipeline.services.archives import ArchiveService
-from photo_pipeline.services.jobs import ItemState, JobService
-from photo_pipeline.services.rename_apply import ApplyError, RenameApplyService
-from photo_pipeline.services.rename_journal import JournalState, RenameJournal
-from photo_pipeline.services.renames import RenameService
-from photo_pipeline.services.safety import SafetyService
-from photo_pipeline.services.uploads import UploadService
-
-NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
-
-
-# ── environment ──────────────────────────────────────────────────────────────
-
-
-def _env(tmp_path, **extra):
-    (tmp_path / "data").mkdir(exist_ok=True)
-    lib = tmp_path / "lib"
-    lib.mkdir(exist_ok=True)
-    config = Config.from_env(
-        {
-            "PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data"),
-            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
-            "PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": "0",
-            **extra,
-        }
-    )
-    run_migrations(config.database_url)
-    return config, create_session_factory(create_db_engine(config.database_url)), lib
-
-
-def image(path: Path, seed: int = 1) -> None:
-    path.parent.mkdir(parents=True, exist_ok=True)
-    pixels = np.random.default_rng(seed).integers(0, 256, (64, 96, 3), dtype=np.uint8)
-    Image.fromarray(pixels).save(path, quality=90)
-
-
-def register(sf, path: Path) -> str:
-    asset_id = str(uuid.uuid4())
-    with sf() as session:
-        session.add(
-            Asset(
-                id=asset_id,
-                original_path=str(path),
-                current_path=str(path),
-                discovered_at=NOW,
-                hash_version=1,
-                byte_size=path.stat().st_size,
-                current_sha256=hashing.sha256_file(path),
-            )
-        )
-        session.commit()
-    return asset_id
-
-
-def uploaded_album(sf, lib, album="rome", names=("a.jpg",)):
-    """An album with verified upload evidence — what archiving requires."""
-    folder = lib / album
-    folder.mkdir(parents=True, exist_ok=True)
-    with sf() as session:
-        batch_id = str(uuid.uuid4())
-        session.add(
-            UploadBatch(
-                id=batch_id,
-                album=album,
-                folder=str(folder),
-                album_name=album,
-                state="succeeded",
-                preflight_token="v1:test",
-                outcome_state="verified",
-                created_at=NOW,
-            )
-        )
-        for name in names:
-            path = folder / name
-            path.write_bytes(f"{album}/{name} content".encode() * 8)
-            asset_id = str(uuid.uuid4())
-            session.add(
-                Asset(
-                    id=asset_id,
-                    original_path=str(path),
-                    current_path=str(path),
-                    discovered_at=NOW,
-                    hash_version=1,
-                    byte_size=path.stat().st_size,
-                    current_sha256=hashing.sha256_file(path),
-                )
-            )
-            session.add(
-                UploadItem(
-                    batch_id=batch_id,
-                    asset_id=asset_id,
-                    path=str(path),
-                    sha256=hashing.sha256_file(path),
-                    sha1="0" * 40,
-                    state="sent",
-                    outcome="uploaded",
-                )
-            )
-        session.commit()
-    return folder
-
-
-def archive_plan(sf, config, archive, albums=None):
-    location = ArchiveService(sf, config=config).register("external", str(archive))
-    token = ArchiveService(sf, config=config).preflight(location["id"], albums)["token"]
-    service = ArchiveTransferService(sf, config=config)
-    return service, service.create(location["id"], albums, token=token)
-
-
-def fake_tool(directory: Path, name: str, body: str) -> Path:
-    """A real executable on a directory a test can put in front of PATH."""
-    directory.mkdir(parents=True, exist_ok=True)
-    path = directory / name
-    path.write_text(f"#!/bin/sh\n{body}")
-    path.chmod(path.stat().st_mode | stat.S_IEXEC | stat.S_IXGRP | stat.S_IXOTH)
-    return path
-
-
-# ── control points ───────────────────────────────────────────────────────────
-
-
-def test_the_fault_barrier_does_nothing_unless_its_variable_names_the_point(monkeypatch):
-    monkeypatch.delenv(faults.ENV_VAR, raising=False)
-    for point in (faults.EXIF_WRITTEN, faults.UPLOAD_ACCEPTED, faults.JOB_ITEM_DONE, "moving"):
-        faults.maybe_fault(point)  # would kill the process if it were armed
-    monkeypatch.setenv(faults.ENV_VAR, faults.EXIF_WRITTEN)
-    faults.maybe_fault(faults.UPLOAD_ACCEPTED)  # a different point stays inert
-
-
-def test_no_route_or_configuration_can_arm_a_fault():
-    """The control points are reachable only through an environment variable read
-    inside ``photo_pipeline.faults`` — never through the API, and never through
-    configuration a browser or a config file could set."""
-    from photo_pipeline.api.app import create_app
-
-    app = create_app()
-    assert not [route for route in app.routes if "fault" in getattr(route, "path", "")]
-    assert not [field for field in Config.model_fields if "fault" in field]
-
-    package = Path(__file__).resolve().parents[2] / "photo_pipeline"
-    sources = {
-        path.relative_to(package.parent)
-        for path in package.rglob("*.py")
-        if faults.ENV_VAR in path.read_text()
-    }
-    assert sources == {Path("photo_pipeline/faults.py")}
-
-
-# ── disk full ────────────────────────────────────────────────────────────────
-
-
-def test_a_full_disk_during_an_archive_never_removes_the_source(tmp_path, monkeypatch):
-    config, sf, lib = _env(tmp_path)
-    archive = tmp_path / "archive"
-    archive.mkdir()
-    folder = uploaded_album(sf, lib)
-    original = {path: path.read_bytes() for path in folder.iterdir()}
-    service, plan = archive_plan(sf, config, archive)
-
-    def no_space(*args, **kwargs):
-        raise OSError(errno.ENOSPC, "No space left on device")
-
-    # Force the cross-filesystem path (a real archive medium) and fill it up.
-    monkeypatch.setattr(archive_transfer, "_same_filesystem", lambda *a: False)
-    monkeypatch.setattr(archive_transfer.shutil, "copyfileobj", no_space)
-
-    result = service.apply(plan["id"])
-
-    assert result["archived"] == 0 and result["failed"] == 1
-    for path, body in original.items():
-        assert path.read_bytes() == body, "the source was touched despite the failure"
-    with sf() as session:
-        assert all(a.availability_state == "active" for a in session.scalars(select(Asset)))
-    assert [p for p in archive.rglob("*") if p.is_file() and not p.name.startswith(".")] == []
-
-
-# ── read-only paths ──────────────────────────────────────────────────────────
-
-
-@pytest.mark.skipif(os.geteuid() == 0, reason="root ignores directory permissions")
-def test_a_read_only_library_refuses_the_rename_and_keeps_the_source(tmp_path):
-    config, sf, lib = _env(tmp_path)
-    folder = lib / "rome"
-    image(folder / "a.jpg")
-    register(sf, folder / "a.jpg")
-    with sf() as session:
-        session.add(
-            AlbumProposal(
-                id=str(uuid.uuid4()),
-                album="rome",
-                proposed_name="2019 Rome",
-                final_name="2019 Rome",
-                status="approved",
-                version=2,
-            )
-        )
-        session.commit()
-    plan = RenameService(sf, library_roots=(lib,)).build_plan()
-
-    mode = lib.stat().st_mode
-    lib.chmod(0o500)  # readable, traversable, not writable
-    try:
-        result = RenameApplyService(sf, library_roots=(lib,)).apply(
-            plan["id"], expected_version=plan["version"]
-        )
-    finally:
-        lib.chmod(mode)
-
-    assert result["applied"] == 0 and result["failed"] == 1
-    assert (folder / "a.jpg").exists() and not (lib / "2019 Rome").exists()
-    with sf() as session:
-        asset = session.scalars(select(Asset)).one()
-        assert asset.current_path == str(folder / "a.jpg")
-
-
-# ── database faults ──────────────────────────────────────────────────────────
-
-
-@contextmanager
-def exclusive_lock(database_url: str):
-    """Hold SQLite's write lock from another thread, the way a second process would.
-
-    A sqlite3 connection belongs to the thread that opened it, so the holder thread
-    opens, locks, waits, and releases entirely on its own.
-    """
-    path = database_url.replace("sqlite:///", "")
-    locked, release = threading.Event(), threading.Event()
-
-    def hold() -> None:
-        connection = sqlite3.connect(path, timeout=10)
-        connection.execute("BEGIN EXCLUSIVE")
-        locked.set()
-        release.wait(30)
-        connection.rollback()
-        connection.close()
-
-    thread = threading.Thread(target=hold)
-    thread.start()
-    assert locked.wait(10), "the holder never acquired the lock"
-    try:
-        yield release.set  # callers may release early; exiting releases anyway
-    finally:
-        release.set()
-        thread.join(10)
-
-
-def test_a_busy_database_waits_rather_than_failing(tmp_path):
-    """SQLite has one writer. A short conflict must resolve through the busy
-    timeout instead of surfacing as an error."""
-    config, sf, lib = _env(tmp_path)
-    service = JobService(sf)
-    job = service.enqueue("scan", items=["a"])
-
-    with exclusive_lock(config.database_url) as release:
-        threading.Timer(0.3, release).start()
-        started = time.monotonic()
-        claimed = service.claim(["scan"], "w1")  # blocks until the lock is gone
-        waited = time.monotonic() - started
-
-    assert claimed["id"] == job["id"] and claimed["state"] == "running"
-    assert waited >= 0.25, "the claim did not actually wait for the writer"
-    with sf() as session:
-        assert session.execute(text("PRAGMA busy_timeout")).scalar() >= 1000
-
-
-def test_a_database_locked_beyond_the_timeout_is_an_error_not_a_silent_skip(tmp_path):
-    config, sf, lib = _env(tmp_path)
-    with sf() as session:
-        session.execute(text("SELECT 1"))  # connect first: the lock comes after
-        with exclusive_lock(config.database_url):
-            session.execute(text("PRAGMA busy_timeout=50"))  # do not wait five seconds
-            with pytest.raises(OperationalError, match="locked"):
-                session.execute(
-                    text("INSERT INTO jobs (id, job_type, state) VALUES ('x','scan','queued')")
-                )
-                session.commit()
-            session.rollback()
-
-    # The refusal left nothing behind, and the database is still sound.
-    with sf() as session:
-        assert session.execute(text("PRAGMA integrity_check")).scalar() == "ok"
-        assert session.execute(text("SELECT count(*) FROM jobs")).scalar() == 0
-
-
-def test_a_corrupt_database_fails_loudly_instead_of_answering_wrongly(tmp_path):
-    config, sf, lib = _env(tmp_path)
-    for index in range(50):  # enough rows to fill several pages
-        JobService(sf).enqueue("scan", items=[f"item-{index}-{n}" for n in range(20)])
-    with sf() as session:
-        session.execute(text("PRAGMA wal_checkpoint(TRUNCATE)"))  # WAL into the file
-        session.commit()
-    source = Path(config.database_url.replace("sqlite:///", ""))
-
-    corrupt = tmp_path / "corrupt.db"
-    body = bytearray(source.read_bytes())
-    body[4096 : 4096 + 2048] = b"\xde\xad\xbe\xef" * 512  # shred pages, keep the header
-    corrupt.write_bytes(bytes(body))
-
-    engine = create_db_engine(f"sqlite:///{corrupt}")
-    factory = create_session_factory(engine)
-    try:
-        with factory() as session:
-            assert session.execute(text("PRAGMA integrity_check")).scalar() != "ok"
-            # Reading the shredded pages must raise, never return half a table.
-            with pytest.raises(DatabaseError):
-                session.execute(text("SELECT * FROM job_items")).all()
-                session.execute(text("SELECT * FROM job_events")).all()
-                session.execute(text("REINDEX")).all()
-    finally:
-        engine.dispose()
-
-
-# ── external services ────────────────────────────────────────────────────────
-
-
-def test_an_unreachable_immich_blocks_upload_instead_of_starting_one(tmp_path):
-    # Port 9 (discard) refuses connections deterministically.
-    config, sf, lib = _env(
-        tmp_path,
-        PHOTO_PIPELINE_IMMICH_SERVER_URL="http://127.0.0.1:9",
-        PHOTO_PIPELINE_IMMICH_API_KEY="sentinel",
-    )
-    report = UploadService(sf, config=config).preflight()
-    codes = {blocker["code"] for blocker in report["blockers"]}
-    assert "server_unreachable" in codes
-    assert report["state"] != "ready"
-
-
-def test_a_missing_uploader_blocks_upload_with_the_binary_named(tmp_path):
-    config, sf, lib = _env(
-        tmp_path,
-        PHOTO_PIPELINE_IMMICH_SERVER_URL="http://127.0.0.1:9",
-        PHOTO_PIPELINE_IMMICH_API_KEY="sentinel",
-        PHOTO_PIPELINE_IMMICH_GO_BINARY=str(tmp_path / "no-such-immich-go"),
-    )
-    report = UploadService(sf, config=config).preflight()
-    assert "immich_go_missing" in {blocker["code"] for blocker in report["blockers"]}
-
-
-def test_a_malformed_provider_answer_is_a_per_asset_error(tmp_path):
-    config, sf, lib = _env(tmp_path)
-    path = lib / "a.jpg"
-    image(path)
-    asset_id = register(sf, path)
-    with sf() as session:
-        session.add(
-            SafetyReview(id=str(uuid.uuid4()), asset_id=asset_id, decision="sfw", created_at=NOW)
-        )
-        session.commit()
-
-    class MalformedProvider:
-        def analyze(self, path, *, album_hint):
-            raise ValueError("Expecting value: line 1 column 1 (char 0)")
-
-    result = AnalysisService(sf, provider=MalformedProvider(), library_roots=(lib,)).run([asset_id])
-
-    assert result == {"analyzed": 0, "skipped": 0, "errors": 1}
-    with sf() as session:
-        row = session.get(AnalysisResult, asset_id)
-        assert row.status == "error" and "Expecting value" in row.error_message
-        assert session.get(ExifProjection, (asset_id, "analysis")) is None
-
-
-def test_gpu_exhaustion_fails_the_item_without_inventing_a_score(tmp_path):
-    config, sf, lib = _env(tmp_path)
-    path = lib / "a.jpg"
-    image(path)
-    asset_id = register(sf, path)
-
-    class ExhaustedModel:
-        def score(self, paths):
-            raise RuntimeError("MPS backend out of memory (MPS allocated: 9.00 GB)")
-
-    service = SafetyService(sf, model=ExhaustedModel())
-    jobs = JobService(sf)
-    job = jobs.enqueue("safety_score", items=[asset_id])
-    Worker(sf, {"safety_score": lambda item, ctx: service.score_assets([item])}, "w1").run_once()
-
-    progress = jobs.progress(job["id"])
-    assert progress["by_state"] == {ItemState.FAILED: 1}
-    with sf() as session:
-        assert session.scalars(select(SafetyReview)).all() == [], "no score was invented"
-
-
-# ── external tools ───────────────────────────────────────────────────────────
-
-
-def test_a_hanging_exiftool_times_out_and_verifies_nothing(tmp_path, monkeypatch):
-    config, sf, lib = _env(tmp_path)
-    path = lib / "a.jpg"
-    image(path)
-    asset_id = register(sf, path)
-    before = hashing.sha256_file(path)
-
-    fake_tool(tmp_path / "bin", "exiftool", "sleep 30\n")
-    monkeypatch.setenv("PATH", f"{tmp_path / 'bin'}:{os.environ['PATH']}")
-    monkeypatch.setenv("PHOTO_PIPELINE_EXIFTOOL_TIMEOUT", "1")
-
-    review = SafetyService(sf).decide(asset_id, "nsfw")
-
-    # The decision is durable; the metadata claim is not made.
-    assert review["decision"] == "nsfw" and review["exif_verified"] is False
-    assert exif_checkpoint.state_for(sf, asset_id, "safety") == exif_checkpoint.FAILED
-    assert hashing.sha256_file(path) == before
-
-
-def test_a_missing_exiftool_is_a_failed_checkpoint_not_a_verified_one(tmp_path, monkeypatch):
-    config, sf, lib = _env(tmp_path)
-    path = lib / "a.jpg"
-    image(path)
-    asset_id = register(sf, path)
-
-    empty_bin = tmp_path / "empty-bin"
-    empty_bin.mkdir()
-    monkeypatch.setenv("PATH", str(empty_bin))  # nothing on PATH at all
-
-    review = SafetyService(sf).decide(asset_id, "sfw")
-
-    assert review["exif_verified"] is False
-    assert exif_checkpoint.state_for(sf, asset_id, "safety") == exif_checkpoint.FAILED
-    with sf() as session:
-        # Upload eligibility depends on a verified checkpoint, so it stays blocked.
-        assert session.scalars(select(SafetyReview)).all()[-1].exif_verified_at is None
-
-
-def test_a_file_edited_during_the_move_is_left_for_a_human(tmp_path, monkeypatch):
-    """The user saves over a photo in the instant between the move and its
-    verification. The move already happened and the database already followed it,
-    so the operation cannot simply be "failed": it becomes ``rollback_required``
-    and blocks further mutation until someone decides (US07-04)."""
-    config, sf, lib = _env(tmp_path)
-    folder = lib / "rome"
-    image(folder / "a.jpg")
-    register(sf, folder / "a.jpg")
-    with sf() as session:
-        session.add(
-            AlbumProposal(
-                id=str(uuid.uuid4()),
-                album="rome",
-                proposed_name="2019 Rome",
-                final_name="2019 Rome",
-                status="approved",
-                version=2,
-            )
-        )
-        session.commit()
-    plan = RenameService(sf, library_roots=(lib,)).build_plan()
-
-    real_rename = rename_apply.os.rename
-
-    def rename_then_edit(source, destination):
-        real_rename(source, destination)
-        for path in Path(destination).glob("*.jpg"):
-            path.write_bytes(b"the user saved over this file")
-
-    monkeypatch.setattr(rename_apply.os, "rename", rename_then_edit)
-
-    result = RenameApplyService(sf, library_roots=(lib,)).apply(
-        plan["id"], expected_version=plan["version"]
-    )
-
-    assert result["applied"] == 0 and result["failed"] == 1
-    journal = RenameJournal(sf)
-    operation = journal.incomplete()[0]
-    assert operation["journal_state"] == JournalState.ROLLBACK_REQUIRED
-    assert operation["error_code"] == "verify_bytes"
-    assert journal.blocks_mutation() is True, "the unresolved rename blocks the library"
-
-    # Recovery offers the rollback the evidence supports, and the rollback itself
-    # refuses the edited bytes rather than putting the user's newer file back as if
-    # it were the old one.
-    service = RenameApplyService(sf, library_roots=(lib,))
-    with pytest.raises(ApplyError, match="manual recovery"):
-        service.rollback_operation(operation["id"])
-    # Nothing was lost: the edited file is at its new home, not deleted.
-    assert (lib / "2019 Rome" / "a.jpg").read_bytes() == b"the user saved over this file"

tests/integration/test_inventory_reconcile.py

@@ -167,8 +167,7 @@ def test_missing_file_is_flagged_not_deleted(
     assets = assets_by_id(make_factory(db_url))
     assert missing_id in assets  # not pruned
     assert assets[missing_id].missing_at is not None
-    # Nothing explains the absence: this is not an offline archive medium (US06-03).
+    assert assets[missing_id].availability_state == "active"
-    assert assets[missing_id].availability_state == "missing_unexpected"
 
 
 def test_reappearing_file_clears_missing(
@@ -187,7 +186,6 @@ def test_reappearing_file_clears_missing(
     inventory.scan(lib)
     assets = assets_by_id(make_factory(db_url))
     assert assets[asset_id].missing_at is None
-    assert assets[asset_id].availability_state == "active"
 
 
 def test_identity_and_state_durable_across_restart(

tests/integration/test_legacy_import.py

@@ -1,238 +0,0 @@
-"""Importing the archived CLI's CSV state into the database (US07-01).
-
-The donor's ``nsfw_scores.csv`` was keyed by path; the database is keyed by a
-stable asset id. Every case here is about that gap: a path that moved, a path that
-matches nothing, a score a human has already overruled, and a file that is simply
-malformed. The import may add evidence and must never invent an asset, overwrite a
-judgement, or fail silently — whatever it does ends up in the report.
-"""
-
-from __future__ import annotations
-
-import json
-import uuid
-from datetime import datetime, timedelta, timezone
-from pathlib import Path
-
-import pytest
-from sqlalchemy import select
-
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.models import Asset, AssetPath, SafetyReview
-from photo_pipeline.services.legacy_import import (
-    REVIEWER,
-    LegacyImportService,
-    write_report,
-)
-
-NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
-
-
-@pytest.fixture
-def factory(tmp_path):
-    url = f"sqlite:///{tmp_path / 'legacy.db'}"
-    run_migrations(url)
-    engine = create_db_engine(url)
-    yield create_session_factory(engine)
-    engine.dispose()
-
-
-def _asset(sf, path: Path, *, previous: Path | None = None) -> str:
-    asset_id = str(uuid.uuid4())
-    with sf() as session:
-        session.add(
-            Asset(
-                id=asset_id,
-                original_path=str(previous or path),
-                current_path=str(path),
-                discovered_at=NOW,
-                hash_version=1,
-            )
-        )
-        session.add(AssetPath(asset_id=asset_id, path=str(path), valid_from=NOW))
-        if previous is not None:
-            session.add(
-                AssetPath(
-                    asset_id=asset_id, path=str(previous), valid_from=NOW, valid_until=NOW
-                )
-            )
-        session.commit()
-    return asset_id
-
-
-def _review(sf, asset_id: str, *, score=None, decision=None, at=NOW) -> None:
-    with sf() as session:
-        session.add(
-            SafetyReview(
-                id=str(uuid.uuid4()),
-                asset_id=asset_id,
-                score=score,
-                decision=decision,
-                reviewer="dom",
-                created_at=at,
-            )
-        )
-        session.commit()
-
-
-def _csv(tmp_path, rows: str) -> Path:
-    path = tmp_path / "nsfw_scores.csv"
-    path.write_text("path,nsfw_score\n" + rows, encoding="utf-8")
-    return path
-
-
-def _scores(sf) -> dict[str, float]:
-    with sf() as session:
-        return {
-            review.asset_id: review.score
-            for review in session.scalars(
-                select(SafetyReview).order_by(SafetyReview.created_at)
-            )
-        }
-
-
-def test_scores_are_imported_onto_asset_identity(tmp_path, factory):
-    a = _asset(factory, tmp_path / "a.jpg")
-    # This one was scored under its old path and has since moved.
-    b = _asset(factory, tmp_path / "moved" / "b.jpg", previous=tmp_path / "b.jpg")
-    csv_path = _csv(
-        tmp_path, f"{tmp_path / 'a.jpg'},0.9123\n{tmp_path / 'b.jpg'},0.0100\n"
-    )
-
-    report = LegacyImportService(factory).import_nsfw_scores(csv_path)
-
-    assert report.counts == {
-        "rows": 2,
-        "imported": 2,
-        "skipped_existing": 0,
-        "unchanged": 0,
-        "reviewed": 0,
-        "unmatched": 0,
-        "rejected": 0,
-        "conflicts": 0,
-    }
-    assert _scores(factory) == {a: 0.9123, b: 0.0100}
-    # The imported rows are evidence, not judgements: no decision is invented.
-    with factory() as session:
-        rows = list(session.scalars(select(SafetyReview)))
-        assert {r.decision for r in rows} == {None}
-        assert {r.reviewer for r in rows} == {REVIEWER}
-    assert csv_path.exists(), "the CSV is left on disk as its own backup"
-
-
-def test_a_reviewed_asset_is_never_overwritten_by_the_csv(tmp_path, factory):
-    asset_id = _asset(factory, tmp_path / "a.jpg")
-    _review(factory, asset_id, score=0.2, decision="sfw")
-    csv_path = _csv(tmp_path, f"{tmp_path / 'a.jpg'},0.9999\n")
-
-    report = LegacyImportService(factory).import_nsfw_scores(csv_path, overwrite=True)
-
-    assert (report.imported, report.reviewed) == (0, 1)
-    assert report.conflicts[0]["decision"] == "sfw"
-    assert report.conflicts[0]["csv"] == 0.9999
-    with factory() as session:
-        rows = list(session.scalars(select(SafetyReview)))
-        assert len(rows) == 1 and rows[0].decision == "sfw"
-
-
-def test_an_unknown_path_is_reported_never_turned_into_an_asset(tmp_path, factory):
-    _asset(factory, tmp_path / "a.jpg")
-    csv_path = _csv(
-        tmp_path, f"{tmp_path / 'a.jpg'},0.5000\n{tmp_path / 'ghost.jpg'},0.5000\n"
-    )
-
-    report = LegacyImportService(factory).import_nsfw_scores(csv_path)
-
-    assert report.unmatched == [str(tmp_path / "ghost.jpg")]
-    with factory() as session:
-        assert session.scalar(select(Asset).where(Asset.current_path.like("%ghost%"))) is None
-
-
-def test_malformed_rows_are_rejected_with_their_reason(tmp_path, factory):
-    _asset(factory, tmp_path / "a.jpg")
-    csv_path = _csv(
-        tmp_path,
-        f"{tmp_path / 'a.jpg'},0.5000\n"
-        f"{tmp_path / 'a.jpg'},0.5000\n"  # the same path twice
-        f"{tmp_path / 'b.jpg'},not-a-number\n"
-        f"{tmp_path / 'c.jpg'},7.5\n"  # outside 0..1: corruption, not a score
-        ",0.5\n",  # no path at all
-    )
-
-    report = LegacyImportService(factory).import_nsfw_scores(csv_path)
-
-    reasons = sorted(r["reason"] for r in report.rejected)
-    assert reasons == ["duplicate_path", "missing_path", "unparsable_score", "unparsable_score"]
-    assert report.imported == 1
-
-
-def test_rerunning_changes_nothing_and_a_differing_score_needs_overwrite(tmp_path, factory):
-    asset_id = _asset(factory, tmp_path / "a.jpg")
-    csv_path = _csv(tmp_path, f"{tmp_path / 'a.jpg'},0.5000\n")
-    service = LegacyImportService(factory)
-    service.import_nsfw_scores(csv_path)
-
-    again = service.import_nsfw_scores(csv_path)
-    assert (again.imported, again.unchanged) == (0, 1)
-    assert len(_scores(factory)) == 1
-
-    changed = _csv(tmp_path, f"{tmp_path / 'a.jpg'},0.8000\n")
-    refused = service.import_nsfw_scores(changed)
-    assert (refused.imported, refused.skipped_existing) == (0, 1)
-    assert refused.conflicts[0]["current"] == 0.5
-
-    forced = service.import_nsfw_scores(changed, overwrite=True)
-    assert forced.imported == 1
-    with factory() as session:
-        latest = list(session.scalars(select(SafetyReview).order_by(SafetyReview.created_at)))[-1]
-        assert (latest.asset_id, latest.score) == (asset_id, 0.8)
-
-
-def test_a_dry_run_reports_without_writing(tmp_path, factory):
-    _asset(factory, tmp_path / "a.jpg")
-    csv_path = _csv(tmp_path, f"{tmp_path / 'a.jpg'},0.5000\n")
-
-    report = LegacyImportService(factory).import_nsfw_scores(csv_path, dry_run=True)
-
-    assert report.imported == 1
-    assert _scores(factory) == {}, "a dry run must leave the database alone"
-
-
-def test_a_missing_csv_is_a_reported_outcome_not_a_crash(tmp_path, factory):
-    report = LegacyImportService(factory).import_nsfw_scores(tmp_path / "nothing.csv")
-    assert report.rejected == [{"path": str(tmp_path / "nothing.csv"), "reason": "csv_missing"}]
-    assert report.rows == 0
-
-
-def test_the_report_is_written_where_it_can_be_audited(tmp_path, factory):
-    _asset(factory, tmp_path / "a.jpg")
-    csv_path = _csv(tmp_path, f"{tmp_path / 'a.jpg'},0.5000\n")
-    report = LegacyImportService(factory).import_nsfw_scores(csv_path)
-
-    written = write_report(report, tmp_path / "data")
-
-    payload = json.loads(written.read_text(encoding="utf-8"))
-    assert payload["imported"] == 1
-    assert payload["source"] == str(csv_path)
-    assert payload["schema_version"] == 1
-    assert payload["generated_at"]
-
-
-def test_a_later_review_still_wins_after_an_import(tmp_path, factory):
-    """The import is a floor, not a ceiling: a human decision made afterwards is
-    the current state, exactly as everywhere else in the app."""
-    asset_id = _asset(factory, tmp_path / "a.jpg")
-    csv_path = _csv(tmp_path, f"{tmp_path / 'a.jpg'},0.9000\n")
-    LegacyImportService(factory).import_nsfw_scores(csv_path)
-    # The import stamps itself with the wall clock, so "afterwards" has to be too.
-    _review(
-        factory,
-        asset_id,
-        score=0.9,
-        decision="sfw",
-        at=datetime.now(timezone.utc) + timedelta(hours=1),
-    )
-
-    with factory() as session:
-        latest = list(session.scalars(select(SafetyReview).order_by(SafetyReview.created_at)))[-1]
-        assert latest.decision == "sfw"

tests/integration/test_media_hardening.py

@@ -1,335 +0,0 @@
-"""US07-03: the golden media corpus, driven through the real decode path.
-
-Every case in ``tests/fixtures/media_corpus.py`` is exercised here. The claims:
-
-* a supported format/orientation/profile renders, with the orientation applied and
-  the decode bounded to roughly the size that was asked for;
-* a damaged, empty, lying, or gigapixel file becomes one precise item error — the
-  scan still finishes, the other assets still render, and a job that meets one keeps
-  running;
-* a failure leaves no temporary file behind, and startup removes only the temporaries
-  this service recognises;
-* cache invalidation follows the pixels, not the metadata.
-"""
-
-from __future__ import annotations
-
-import hashlib
-import shutil
-import time
-import tracemalloc
-from types import SimpleNamespace
-
-import pytest
-from PIL import Image
-
-from photo_pipeline import imaging
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.models import Asset, Thumbnail
-from photo_pipeline.services.inventory import InventoryService
-from photo_pipeline.services.thumbnails import (
-    TEMP_SUFFIX,
-    ThumbnailError,
-    ThumbnailService,
-)
-from tests.fixtures.media_corpus import CASES, CASES_BY_ID, build_corpus
-
-RENDERABLE = [case for case in CASES if case.expect_error is None]
-DAMAGED = [case for case in CASES if case.expect_error is not None]
-
-needs_exiftool = pytest.mark.skipif(
-    shutil.which("exiftool") is None, reason="exiftool not installed"
-)
-
-
-@pytest.fixture(scope="module")
-def corpus(tmp_path_factory):
-    root = tmp_path_factory.mktemp("corpus")
-    return SimpleNamespace(root=root, files=build_corpus(root))
-
-
-@pytest.fixture
-def env(tmp_path, corpus):
-    """A library holding the whole corpus, scanned into a fresh database."""
-    data = tmp_path / "data"
-    data.mkdir()
-    lib = tmp_path / "lib"
-    shutil.copytree(corpus.root, lib)
-    config = Config.from_env(
-        {"PHOTO_PIPELINE_DATA_DIR": str(data), "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib)}
-    )
-    run_migrations(config.database_url)
-    engine = create_db_engine(config.database_url)
-    sf = create_session_factory(engine)
-    scan = InventoryService(sf).scan(lib)
-    by_id = {
-        case.id: next(
-            (aid for path, aid in scan.asset_ids.items() if path.endswith(case.filename)), None
-        )
-        for case in CASES
-    }
-    yield SimpleNamespace(
-        config=config, lib=lib, sf=sf, scan=scan, assets=by_id,
-        thumbs=ThumbnailService(sf, config),
-    )
-    engine.dispose()
-
-
-# ── the manifest itself ───────────────────────────────────────────────────────
-
-
-def test_the_manifest_is_internally_consistent():
-    assert len({case.id for case in CASES}) == len(CASES), "logical ids must be unique"
-    assert len({case.filename for case in CASES}) == len(CASES), "paths must be unique"
-    for case in CASES:
-        assert case.kind in {"format", "orientation", "profile", "damaged", "metadata"}
-        assert case.expect_error in (None, "unsupported_image", "image_too_large")
-
-
-def test_the_corpus_regenerates_byte_for_byte(tmp_path):
-    """A golden corpus that drifts between runs cannot be a golden corpus."""
-    first = build_corpus(tmp_path / "one")
-    second = build_corpus(tmp_path / "two")
-    digests = {
-        case_id: (
-            hashlib.sha256(first[case_id].read_bytes()).hexdigest(),
-            hashlib.sha256(second[case_id].read_bytes()).hexdigest(),
-        )
-        for case_id in first
-    }
-    drifted = [case_id for case_id, (a, b) in digests.items() if a != b]
-    assert not drifted, f"non-deterministic fixtures: {drifted}"
-
-
-# ── bounded decode ────────────────────────────────────────────────────────────
-
-
-@pytest.mark.parametrize("case", RENDERABLE, ids=lambda case: case.id)
-def test_every_supported_case_renders(case, env):
-    asset_id = env.assets[case.id]
-    assert asset_id, f"{case.id} was not discovered by the scan"
-    path = env.thumbs.generate(asset_id, 256)
-    with Image.open(path) as thumb:
-        assert thumb.format == "WEBP"
-        assert max(thumb.size) <= 256
-        assert min(thumb.size) >= 1
-
-
-@pytest.mark.parametrize("orientation", range(1, 9))
-def test_exif_orientation_is_applied_before_resizing(orientation, env):
-    """All eight tags: the 90° ones must come out portrait from a landscape source."""
-    asset_id = env.assets[f"orientation_{orientation}"]
-    with Image.open(env.thumbs.generate(asset_id, 256)) as thumb:
-        rotated = orientation in (5, 6, 7, 8)
-        assert (thumb.height > thumb.width) is rotated
-
-
-def test_transparency_and_grayscale_survive_the_pipeline(env):
-    with Image.open(env.thumbs.generate(env.assets["png_alpha"], 256)) as thumb:
-        assert "A" in thumb.getbands()
-    with Image.open(env.thumbs.generate(env.assets["grayscale"], 256)) as thumb:
-        assert thumb.size == (100, 100)  # smaller than the request: never upscaled
-
-
-def test_a_broken_colour_profile_still_produces_a_picture(env):
-    """An unusable ICC profile is a metadata problem, not a reason to lose the preview."""
-    for case_id in ("icc_tagged", "broken_icc"):
-        with Image.open(env.thumbs.generate(env.assets[case_id], 256)) as thumb:
-            assert thumb.size[0] > 0
-
-
-def test_a_large_jpeg_is_not_decoded_at_full_resolution(env):
-    """12 megapixels would be ~36 MB of pixels; the draft decode keeps it far below."""
-    tracemalloc.start()
-    try:
-        env.thumbs.generate(env.assets["large_jpeg"], 256)
-        _, peak = tracemalloc.get_traced_memory()
-    finally:
-        tracemalloc.stop()
-    assert peak < 12_000_000, f"decode peaked at {peak} bytes"
-
-
-# ── damaged inputs ────────────────────────────────────────────────────────────
-
-
-@pytest.mark.parametrize("case", DAMAGED, ids=lambda case: case.id)
-def test_every_damaged_case_is_a_precise_item_error(case, env):
-    asset_id = env.assets[case.id]
-    assert asset_id, f"{case.id} was not discovered by the scan"
-    with pytest.raises(ThumbnailError) as raised:
-        env.thumbs.generate(asset_id, 256)
-    assert raised.value.code == case.expect_error
-    # Persisted, so a broken original is not re-decoded on every request...
-    with env.sf() as session:
-        rows = [
-            row
-            for row in session.query(Thumbnail).all()
-            if row.asset_id == asset_id and row.state == "error"
-        ]
-    assert rows and rows[0].error_code == case.expect_error
-    # ...and the cached failure is the same precise error, not a generic one.
-    with pytest.raises(ThumbnailError) as again:
-        env.thumbs.generate(asset_id, 256)
-    assert again.value.code == case.expect_error
-
-
-def test_a_gigapixel_header_is_refused_quickly_and_cheaply(env):
-    """The refusal must come from the declared size, not from decoding it."""
-    tracemalloc.start()
-    started = time.monotonic()
-    try:
-        with pytest.raises(ThumbnailError):
-            env.thumbs.generate(env.assets["bomb_header"], 1280)
-        _, peak = tracemalloc.get_traced_memory()
-    finally:
-        tracemalloc.stop()
-    assert time.monotonic() - started < 5
-    assert peak < 5_000_000, f"a 3.6 gigapixel header allocated {peak} bytes"
-
-
-def test_one_broken_file_stops_neither_the_scan_nor_its_neighbours(env):
-    """The whole corpus is in the library, damaged files included."""
-    assert len(env.scan.asset_ids) >= len(CASES) - 1  # the empty file has no pixels to hash
-    for case in RENDERABLE[:5]:
-        assert env.thumbs.generate(env.assets[case.id], 256).exists()
-
-
-def test_a_job_that_meets_a_broken_file_gets_evidence_not_an_exception(env):
-    """``ensure_protected`` is what the archive lane calls; a plan must not die on
-    one unreadable original."""
-    evidence = env.thumbs.ensure_protected(env.assets["corrupt_png"])
-    assert evidence["state"] == "unsupported"
-    assert evidence["error_code"] == "unsupported_image"
-    assert env.thumbs.ensure_protected(env.assets["jpeg"])["state"] == "ready"
-
-
-def test_undecodable_files_do_not_break_hashing(env):
-    """Discovery records what it can: byte identity always, pixel identity when the
-    file has pixels."""
-    with env.sf() as session:
-        assets = {a.current_path: a for a in session.query(Asset).all()}
-    for case in DAMAGED:
-        asset = next((a for path, a in assets.items() if path.endswith(case.filename)), None)
-        if asset is None:
-            continue
-        assert asset.current_sha256, "byte identity is always available"
-        assert asset.pixel_sha256 is None, "undecodable files must not invent pixel identity"
-
-
-# ── cache lifecycle ───────────────────────────────────────────────────────────
-
-
-def test_a_failed_render_leaves_no_temporary_behind(env):
-    with pytest.raises(ThumbnailError):
-        env.thumbs.generate(env.assets["truncated_jpeg"], 256)
-    leftovers = list(env.config.thumbnail_cache_dir.rglob(f"*{TEMP_SUFFIX}"))
-    assert leftovers == []
-
-
-def test_startup_cleanup_removes_only_recognised_temporaries(env):
-    cache = env.config.thumbnail_cache_dir
-    env.thumbs.generate(env.assets["jpeg"], 256)  # creates the cache directory
-    stale = cache / f".abc123{TEMP_SUFFIX}"
-    stale.write_bytes(b"half a thumbnail")
-    innocent = cache / "keep-me.webp"
-    innocent.write_bytes(b"not a temporary")
-
-    assert env.thumbs.cleanup_temp_files() == 1
-    assert not stale.exists()
-    assert innocent.exists()
-    assert list(cache.rglob("*.webp")), "real cache entries survive"
-
-
-def test_metadata_only_change_reuses_the_thumbnail_and_a_pixel_change_does_not(env):
-    """The cache key follows the pixels: an EXIF edit must not cost a re-render."""
-    asset_id = env.assets["jpeg"]
-    first = env.thumbs.generate(asset_id, 256)
-
-    with env.sf() as session:  # what a rescan records after an EXIF-only edit
-        asset = session.get(Asset, asset_id)
-        asset.current_sha256 = "different-bytes"
-        session.commit()
-    assert env.thumbs.generate(asset_id, 256) == first
-
-    with env.sf() as session:  # a genuine pixel change
-        asset = session.get(Asset, asset_id)
-        asset.pixel_sha256 = "different-pixels"
-        session.commit()
-    assert env.thumbs.generate(asset_id, 256) != first
-
-
-def test_a_deleted_cache_file_is_regenerated(env):
-    asset_id = env.assets["png"]
-    path = env.thumbs.generate(asset_id, 256)
-    path.unlink()
-    regenerated = env.thumbs.generate(asset_id, 256)
-    assert regenerated == path and regenerated.exists()
-
-
-# ── the imaging door itself ───────────────────────────────────────────────────
-
-
-def test_open_image_translates_every_decoder_failure(corpus):
-    """Callers must be able to catch two typed errors, never bare ``Exception``."""
-    for case in DAMAGED:
-        expected = imaging.ImageTooLarge if case.expect_error == "image_too_large" else imaging.UndecodableImage
-        with pytest.raises(expected):
-            with imaging.open_image(corpus.files[case.id]) as image:
-                image.load()
-
-
-def test_open_image_refuses_more_pixels_than_the_caller_allowed(corpus):
-    with pytest.raises(imaging.ImageTooLarge):
-        with imaging.open_image(corpus.files["jpeg"], max_pixels=100):
-            pass
-    with imaging.open_image(corpus.files["jpeg"], max_pixels=100_000) as image:
-        assert image.size == (320, 240)
-
-
-@needs_exiftool
-def test_malformed_metadata_does_not_stop_the_picture(env):
-    """A broken EXIF block is a metadata fact, not a decode failure."""
-    assert env.thumbs.generate(env.assets["malformed_metadata"], 256).exists()
-    with env.sf() as session:
-        asset = session.get(Asset, env.assets["malformed_metadata"])
-    assert asset.pixel_sha256, "pixels are still identifiable"
-
-
-def test_scoring_never_relaxes_truncated_image_handling(corpus):
-    """The donor's process-global ``LOAD_TRUNCATED_IMAGES`` is gone for good.
-
-    It is global state: switching it on for the safety model would also switch it on
-    for hashing and preview rendering in the same process, and half a file would
-    silently become a valid picture (donor_ledger: nt-score-model).
-    """
-    import inspect
-
-    from PIL import ImageFile
-
-    from photo_pipeline.integrations import nsfw_model
-
-    code = [
-        line
-        for line in inspect.getsource(nsfw_model).splitlines()
-        if not line.strip().startswith("#")
-    ]
-    assert not any("LOAD_TRUNCATED_IMAGES" in line for line in code)
-
-    # And the door holds even when something else in the process turned it on —
-    # the frozen donor does exactly that when the characterization suite imports it.
-    previous = ImageFile.LOAD_TRUNCATED_IMAGES
-    ImageFile.LOAD_TRUNCATED_IMAGES = True
-    try:
-        with pytest.raises(imaging.UndecodableImage):
-            with imaging.open_image(corpus.files["truncated_jpeg"]) as image:
-                image.load()
-        assert ImageFile.LOAD_TRUNCATED_IMAGES is True, "the caller's setting is restored"
-    finally:
-        ImageFile.LOAD_TRUNCATED_IMAGES = previous
-
-
-def test_every_manifest_case_is_exercised():
-    """The corpus lint: no fixture may sit in the manifest untested."""
-    covered = {case.id for case in RENDERABLE} | {case.id for case in DAMAGED}
-    assert covered == set(CASES_BY_ID)

tests/integration/test_offline_assets.py

@@ -1,377 +0,0 @@
-"""Offline identity and review evidence (US06-03).
-
-An archived photo is not gone: it keeps its identity, its hashes, and enough
-evidence to be recognised in a duplicate cluster while its medium sits in a
-drawer. Every case here archives a *real* album through the real transfer, then
-takes the medium away by removing its marker — the same thing the service sees
-when an external disk is unplugged — and asks whether the application still tells
-the truth about where the bytes are.
-
-The distinction that matters throughout: an unmounted medium is
-``archived_offline`` (expected, harmless), a mounted medium with a hole in it is
-``missing_unexpected`` (needs a human). Confusing the two is how an archive
-quietly loses a photo.
-"""
-
-import shutil
-import uuid
-from datetime import datetime, timezone
-
-import numpy as np
-import pytest
-from fastapi.testclient import TestClient
-from PIL import Image
-from sqlalchemy import select
-
-from photo_pipeline.api.app import create_app
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.models import Asset, Thumbnail, UploadBatch, UploadItem
-from photo_pipeline.services.archive_transfer import ArchiveTransferService
-from photo_pipeline.services.archives import MARKER_NAME, ArchiveService
-from photo_pipeline.services.availability import (
-    ACTIVE,
-    ARCHIVED_OFFLINE,
-    ARCHIVED_ONLINE,
-    MISSING_UNEXPECTED,
-)
-from photo_pipeline.services.duplicates import ClusterState, DuplicateService, Method
-from photo_pipeline.services.hashing import sha256_file
-from photo_pipeline.services.inventory import InventoryService
-from photo_pipeline.services.thumbnails import PROTECTED_SIZE, ThumbnailService, ThumbnailUnavailable
-
-pytestmark = pytest.mark.phase_f  # part of the Phase F acceptance gate (US06-06)
-
-NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
-
-
-# ── environment ──────────────────────────────────────────────────────────────
-
-
-def _env(tmp_path):
-    (tmp_path / "data").mkdir(exist_ok=True)
-    lib = tmp_path / "lib"
-    lib.mkdir(exist_ok=True)
-    archive = tmp_path / "archive"
-    archive.mkdir(exist_ok=True)
-    config = Config.from_env(
-        {
-            "PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data"),
-            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
-            "PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": "0",
-        }
-    )
-    run_migrations(config.database_url)
-    return config, create_session_factory(create_db_engine(config.database_url)), lib, archive
-
-
-def structured(path, seed, size=(256, 192)):
-    """A deterministic, decodable photo — previews and pHashes must be real."""
-    path.parent.mkdir(parents=True, exist_ok=True)
-    rng = np.random.default_rng(seed)
-    w, h = size
-    base = np.zeros((h, w, 3), dtype=np.uint8)
-    for _ in range(6):
-        x0 = int(rng.integers(0, w - 60))
-        y0 = int(rng.integers(0, h - 60))
-        base[y0 : y0 + 60, x0 : x0 + 60] = rng.integers(0, 256, 3)
-    grad = np.linspace(0, 120, w, dtype=np.uint8)
-    base[:, :, 0] = np.clip(base[:, :, 0].astype(int) + grad[None, :], 0, 255)
-    Image.fromarray(base).save(path, quality=95)
-    return path
-
-
-def resized_copy(src, dst, scale=0.5):
-    with Image.open(src) as image:
-        image.resize(
-            (int(image.width * scale), int(image.height * scale)), Image.LANCZOS
-        ).save(dst, quality=95)
-    return dst
-
-
-def _uploaded(sf, lib, album="rome", seeds=(1, 2)):
-    """A scanned album carrying the verified upload evidence archiving requires."""
-    folder = lib / album
-    for index, seed in enumerate(seeds):
-        structured(folder / f"{index}.jpg", seed)
-    result = InventoryService(sf).scan(lib)
-    with sf() as session:
-        batch_id = str(uuid.uuid4())
-        session.add(
-            UploadBatch(
-                id=batch_id,
-                album=album,
-                folder=str(folder),
-                album_name=album,
-                state="succeeded",
-                preflight_token="v1:test",
-                outcome_state="verified",
-                created_at=NOW,
-            )
-        )
-        for path, asset_id in result.asset_ids.items():
-            session.add(
-                UploadItem(
-                    batch_id=batch_id,
-                    asset_id=asset_id,
-                    path=path,
-                    sha256=sha256_file(path),
-                    sha1="0" * 40,
-                    state="sent",
-                    outcome="uploaded",
-                )
-            )
-        session.commit()
-    return folder, result.asset_ids
-
-
-def _archive(sf, config, archive, albums=None):
-    service = ArchiveService(sf, config=config)
-    location = service.register("external", str(archive))
-    token = service.preflight(location["id"], albums)["token"]
-    transfers = ArchiveTransferService(sf, config=config)
-    plan = transfers.create(location["id"], albums, token=token)
-    transfers.apply(plan["id"])
-    return location
-
-
-def _unmount(archive):
-    """Take the medium away the way a real one goes: its marker stops answering."""
-    (archive / MARKER_NAME).rename(archive / f"{MARKER_NAME}.away")
-
-
-def _remount(archive):
-    (archive / f"{MARKER_NAME}.away").rename(archive / MARKER_NAME)
-
-
-def _assets(sf):
-    with sf() as session:
-        return {asset.id: asset for asset in session.scalars(select(Asset))}
-
-
-# ── availability ─────────────────────────────────────────────────────────────
-
-
-def test_archived_assets_report_online_offline_and_missing(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    _, ids = _uploaded(sf, lib)
-    _archive(sf, config, archive)
-    inventory = InventoryService(sf)
-
-    states = {a.availability_state for a in _assets(sf).values()}
-    assert states == {ARCHIVED_ONLINE}
-
-    _unmount(archive)
-    inventory.scan(lib)  # the album folder is gone from the active roots
-    assert {a.availability_state for a in _assets(sf).values()} == {ARCHIVED_OFFLINE}
-    assert all(a.missing_at is None for a in _assets(sf).values())  # not "missing"
-
-    _remount(archive)
-    inventory.scan(lib)
-    assert {a.availability_state for a in _assets(sf).values()} == {ARCHIVED_ONLINE}
-
-    # Mounted medium, absent file: that is not an offline archive, it needs a human.
-    victim = sorted(ids.values())[0]
-    with sf() as session:
-        asset = session.get(Asset, victim)
-        (archive / asset.archive_path).unlink()
-    inventory.scan(lib)
-    assert _assets(sf)[victim].availability_state == MISSING_UNEXPECTED
-
-
-def test_scan_never_prunes_or_flags_offline_assets(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    _, ids = _uploaded(sf, lib)
-    _archive(sf, config, archive)
-    _unmount(archive)
-
-    before = _assets(sf)
-    result = InventoryService(sf).scan(lib)
-
-    assert result.counts.get("missing") is None
-    after = _assets(sf)
-    assert set(after) == set(before) == set(ids.values())
-    for asset in after.values():
-        assert asset.availability_state == ARCHIVED_OFFLINE
-        assert asset.current_sha256 and asset.pixel_sha256  # hashes retained
-        assert asset.archive_location_id and asset.archive_path
-
-
-def test_active_missing_file_is_missing_unexpected_not_offline(tmp_path):
-    config, sf, lib, _archive_root = _env(tmp_path)
-    structured(lib / "loose" / "a.jpg", 7)
-    ids = InventoryService(sf).scan(lib).asset_ids
-    asset_id = next(iter(ids.values()))
-
-    (lib / "loose" / "a.jpg").unlink()
-    InventoryService(sf).scan(lib)
-
-    asset = _assets(sf)[asset_id]
-    assert asset.availability_state == MISSING_UNEXPECTED
-    assert asset.missing_at is not None
-
-
-# ── deduplication against archived originals ─────────────────────────────────
-
-
-def test_exact_copy_of_offline_asset_links_to_archived_canonical(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder, ids = _uploaded(sf, lib, seeds=(1,))
-    archived_id = next(iter(ids.values()))
-    original = next(iter(ids))
-    kept = tmp_path / "kept.jpg"
-    shutil.copy2(original, kept)
-    _archive(sf, config, archive)
-    _unmount(archive)
-
-    # The same photo turns up again in the active library while the disk is away.
-    (lib / "inbox").mkdir(parents=True, exist_ok=True)
-    shutil.copy2(kept, lib / "inbox" / "again.jpg")
-    scan = InventoryService(sf).scan(lib)
-    new_id = scan.asset_ids[str(lib / "inbox" / "again.jpg")]
-    assert scan.occurrences[str(lib / "inbox" / "again.jpg")] == "copied"
-
-    clusters = DuplicateService(sf).detect().clusters
-    exact = [c for c in clusters if c["method"] == Method.EXACT.value]
-    assert len(exact) == 1
-    cluster = exact[0]
-    # Byte-identical: linked directly, and the archived original stays canonical.
-    assert cluster["state"] == ClusterState.DECIDED.value
-    assert cluster["canonical_asset_id"] == archived_id
-    assert _assets(sf)[new_id].canonical_asset_id == archived_id
-
-
-def test_fuzzy_copy_of_offline_asset_requires_review_and_names_the_medium(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    folder, ids = _uploaded(sf, lib, seeds=(1,))
-    archived_id = next(iter(ids.values()))
-    original = next(iter(ids))
-    variant_source = resized_copy(original, tmp_path / "small.jpg")
-    _archive(sf, config, archive)
-    _unmount(archive)
-
-    (lib / "inbox").mkdir(parents=True, exist_ok=True)
-    shutil.copy2(variant_source, lib / "inbox" / "small.jpg")
-    InventoryService(sf).scan(lib)
-
-    duplicates = DuplicateService(sf)
-    clusters = duplicates.detect().clusters
-    perceptual = [c for c in clusters if c["method"] == Method.PERCEPTUAL.value]
-    assert len(perceptual) == 1
-
-    detail = duplicates.get_cluster(perceptual[0]["id"])
-    assert detail["state"] == ClusterState.OPEN.value  # never auto-decided
-    assert detail["requires_confirmation"] is True
-    assert detail["mount_required"] == ["external"]  # full-resolution needs the disk
-    archived = next(m for m in detail["members"] if m["asset_id"] == archived_id)
-    assert archived["availability_state"] == ARCHIVED_OFFLINE
-    assert archived["current_path"] is None
-    assert archived["archive_path"] and archived["evidence"]["phash"]
-    # The retained preview is what makes the offline member reviewable at all.
-    assert archived["preview"] == {"state": "ready", "protected": True, "size": PROTECTED_SIZE}
-
-
-# ── protected review evidence ────────────────────────────────────────────────
-
-
-def test_protected_preview_survives_quota_and_serves_while_offline(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    _, ids = _uploaded(sf, lib, seeds=(1,))
-    asset_id = next(iter(ids.values()))
-    _archive(sf, config, archive)
-    _unmount(archive)
-
-    thumbnails = ThumbnailService(sf, config)
-    served = thumbnails.generate(asset_id, PROTECTED_SIZE)
-    assert served.exists()  # rendered before the original left, not from the medium
-
-    # An aggressive quota may empty the cache, but not this evidence.
-    tight = ThumbnailService(sf, config.model_copy(update={"thumbnail_cache_quota_bytes": 1}))
-    tight._enforce_quota()
-    assert served.exists()
-    with sf() as session:
-        row = session.scalar(select(Thumbnail).where(Thumbnail.asset_id == asset_id))
-        assert row.protected is True
-    assert thumbnails.evidence(asset_id)["state"] == "ready"
-
-
-def test_offline_asset_without_preview_reports_unavailable(tmp_path):
-    """No preview and no medium is an honest 409, never a wrong picture."""
-    config, sf, lib, archive = _env(tmp_path)
-    _, ids = _uploaded(sf, lib, seeds=(1,))
-    asset_id = next(iter(ids.values()))
-    _archive(sf, config, archive)
-    with sf() as session:
-        for row in session.scalars(select(Thumbnail).where(Thumbnail.asset_id == asset_id)):
-            session.delete(row)
-        session.commit()
-    _unmount(archive)
-
-    with pytest.raises(ThumbnailUnavailable):
-        ThumbnailService(sf, config).generate(asset_id, 256)
-
-    _remount(archive)  # mounted again: the archived original is readable
-    assert ThumbnailService(sf, config).generate(asset_id, 256).exists()
-
-
-def test_offline_asset_is_browsable_through_the_api(tmp_path):
-    """The browser sees an archived asset, its medium, and its preview — offline."""
-    config, sf, lib, archive = _env(tmp_path)
-    _, ids = _uploaded(sf, lib, seeds=(1,))
-    asset_id = next(iter(ids.values()))
-    _archive(sf, config, archive)
-    _unmount(archive)
-
-    with TestClient(create_app(config)) as client:
-        client.post("/api/v1/inventory/scan")
-        listed = client.get("/api/v1/inventory/assets", params={"availability": ARCHIVED_OFFLINE})
-        assert listed.status_code == 200
-        item = next(row for row in listed.json()["items"] if row["id"] == asset_id)
-        assert item["current_path"] is None
-        assert item["archive_path"] == "rome/0.jpg"
-        assert item["missing"] is False
-
-        # Searching by the archived path still finds it.
-        found = client.get("/api/v1/inventory/assets", params={"q": "rome"}).json()
-        assert [row["id"] for row in found["items"]] == [asset_id]
-
-        preview = client.get(f"/api/v1/assets/{asset_id}/thumbnail", params={"size": 1280})
-        assert preview.status_code == 200
-        assert preview.headers["content-type"] == "image/webp"
-
-
-def test_offline_state_is_stable_across_restart(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    _, ids = _uploaded(sf, lib, seeds=(1, 2))
-    _archive(sf, config, archive)
-    _unmount(archive)
-    InventoryService(sf).scan(lib)
-    before = {
-        asset_id: (
-            asset.availability_state,
-            asset.archive_path,
-            asset.current_sha256,
-            asset.phash,
-        )
-        for asset_id, asset in _assets(sf).items()
-    }
-
-    # Restart: a fresh engine and session factory against the same database.
-    restarted = create_session_factory(create_db_engine(config.database_url))
-    after = {
-        asset_id: (
-            asset.availability_state,
-            asset.archive_path,
-            asset.current_sha256,
-            asset.phash,
-        )
-        for asset_id, asset in _assets(restarted).items()
-    }
-    assert after == before
-    assert set(after) == set(ids.values())
-
-    # And the medium coming back is picked up by the restarted process.
-    _remount(archive)
-    assert ArchiveService(restarted, config=config).locations()[0]["state"] == "online"
-    assert {a.availability_state for a in _assets(restarted).values()} == {ARCHIVED_ONLINE}
-    assert ACTIVE not in {a.availability_state for a in _assets(restarted).values()}

tests/integration/test_restore.py

@@ -1,417 +0,0 @@
-"""Planning and executing safe restores (US06-04).
-
-Restoring is the one archive operation that can *add* a file to the library, so
-every case here asks two questions: did the right bytes come back under the right
-identity, and did anything already in the library get touched? The media are real
-directories, the hashes are real, and the failure paths assert that the archived
-copy is still exactly where it was — a restore that fails must cost nothing.
-"""
-
-import shutil
-import uuid
-from datetime import datetime, timezone
-
-import numpy as np
-import pytest
-from fastapi.testclient import TestClient
-from PIL import Image
-from sqlalchemy import select
-
-from photo_pipeline.api.app import create_app
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.models import Asset, AssetPath, SafetyReview, UploadBatch, UploadItem
-from photo_pipeline.services import availability
-from photo_pipeline.services.archive_journal import ArchiveState
-from photo_pipeline.services.archive_transfer import ArchiveTransferService
-from photo_pipeline.services.archives import MARKER_NAME, ArchiveError, ArchiveService
-from photo_pipeline.services.hashing import sha256_file
-from photo_pipeline.services.inventory import InventoryService
-from photo_pipeline.services.restores import RestoreService
-
-pytestmark = pytest.mark.phase_f  # part of the Phase F acceptance gate (US06-06)
-
-NOW = datetime(2026, 1, 1, tzinfo=timezone.utc)
-
-
-# ── environment ──────────────────────────────────────────────────────────────
-
-
-def _env(tmp_path):
-    (tmp_path / "data").mkdir(exist_ok=True)
-    lib = tmp_path / "lib"
-    lib.mkdir(exist_ok=True)
-    archive = tmp_path / "archive"
-    archive.mkdir(exist_ok=True)
-    config = Config.from_env(
-        {
-            "PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data"),
-            "PHOTO_PIPELINE_LIBRARY_ROOTS": str(lib),
-            "PHOTO_PIPELINE_ARCHIVE_FREE_SPACE_RESERVE_BYTES": "0",
-        }
-    )
-    run_migrations(config.database_url)
-    return config, create_session_factory(create_db_engine(config.database_url)), lib, archive
-
-
-def structured(path, seed, size=(192, 144)):
-    path.parent.mkdir(parents=True, exist_ok=True)
-    rng = np.random.default_rng(seed)
-    w, h = size
-    base = np.zeros((h, w, 3), dtype=np.uint8)
-    for _ in range(5):
-        x0 = int(rng.integers(0, w - 40))
-        y0 = int(rng.integers(0, h - 40))
-        base[y0 : y0 + 40, x0 : x0 + 40] = rng.integers(0, 256, 3)
-    Image.fromarray(base).save(path, quality=95)
-    return path
-
-
-def _archived(sf, config, lib, archive, album="rome", seeds=(1, 2)):
-    """A real album taken all the way through archiving, ready to be restored."""
-    folder = lib / album
-    for index, seed in enumerate(seeds):
-        structured(folder / f"{index}.jpg", seed)
-    scan = InventoryService(sf).scan(lib)
-    with sf() as session:
-        batch_id = str(uuid.uuid4())
-        session.add(
-            UploadBatch(
-                id=batch_id,
-                album=album,
-                folder=str(folder),
-                album_name=album,
-                state="succeeded",
-                preflight_token="v1:test",
-                outcome_state="verified",
-                created_at=NOW,
-            )
-        )
-        for path, asset_id in scan.asset_ids.items():
-            session.add(
-                UploadItem(
-                    batch_id=batch_id,
-                    asset_id=asset_id,
-                    path=path,
-                    sha256=sha256_file(path),
-                    sha1="0" * 40,
-                    state="sent",
-                    outcome="uploaded",
-                )
-            )
-            # A decision that must survive the whole round trip.
-            session.add(
-                SafetyReview(
-                    id=str(uuid.uuid4()),
-                    asset_id=asset_id,
-                    decision="sfw",
-                    score=0.01,
-                    reviewer="test",
-                )
-            )
-        session.commit()
-
-    service = ArchiveService(sf, config=config)
-    location = service.register("external", str(archive))
-    token = service.preflight(location["id"])["token"]
-    transfers = ArchiveTransferService(sf, config=config)
-    plan = transfers.create(location["id"], None, token=token)
-    transfers.apply(plan["id"])
-    return location, scan.asset_ids
-
-
-def _restore(sf, config, location_id, asset_ids=None):
-    service = RestoreService(sf, config=config)
-    token = service.preflight(location_id, asset_ids)["token"]
-    plan = service.create(location_id, asset_ids, token=token)
-    return service, plan, service.apply(plan["id"])
-
-
-def _unmount(archive):
-    (archive / MARKER_NAME).rename(archive / f"{MARKER_NAME}.away")
-
-
-def _assets(sf):
-    with sf() as session:
-        return {asset.id: asset for asset in session.scalars(select(Asset))}
-
-
-def _codes(report):
-    return {issue["code"] for issue in report["blockers"]} | {
-        issue["code"] for item in report["items"] for issue in item["blockers"]
-    }
-
-
-# ── preflight ────────────────────────────────────────────────────────────────
-
-
-def test_preflight_blocks_offline_medium(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, _ = _archived(sf, config, lib, archive)
-    _unmount(archive)
-
-    report = RestoreService(sf, config=config).preflight(location["id"])
-    assert report["state"] == "blocked"
-    assert "location_offline" in _codes(report)
-
-
-def test_preflight_blocks_wrong_volume(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, _ = _archived(sf, config, lib, archive)
-    (archive / MARKER_NAME).write_text('{"media_id": "someone-elses-disk"}', encoding="utf-8")
-
-    report = RestoreService(sf, config=config).preflight(location["id"])
-    assert report["state"] == "blocked"
-    assert "wrong_volume" in _codes(report)
-
-
-def test_preflight_blocks_changed_archive_bytes(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, ids = _archived(sf, config, lib, archive, seeds=(1,))
-    asset_id = next(iter(ids.values()))
-    with sf() as session:
-        archived_file = archive / session.get(Asset, asset_id).archive_path
-    archived_file.write_bytes(b"not the photo that was archived")
-
-    report = RestoreService(sf, config=config).preflight(location["id"])
-    assert report["state"] == "blocked"
-    assert "bytes_changed" in _codes(report)
-    with pytest.raises(ArchiveError) as error:
-        _restore(sf, config, location["id"])
-    assert error.value.code == "blocked"
-
-
-def test_preflight_blocks_insufficient_capacity(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, _ = _archived(sf, config, lib, archive)
-    greedy = config.model_copy(
-        update={"archive_free_space_reserve_bytes": 1 << 62}  # more than any disk has
-    )
-
-    report = RestoreService(sf, config=greedy).preflight(location["id"])
-    assert report["state"] == "blocked"
-    assert "insufficient_capacity" in _codes(report)
-
-
-def test_token_changes_with_the_scope(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, ids = _archived(sf, config, lib, archive)
-    service = RestoreService(sf, config=config)
-
-    whole = service.preflight(location["id"])["token"]
-    partial = service.preflight(location["id"], [sorted(ids.values())[0]])["token"]
-    assert whole != partial
-    assert service.verify_token(whole, location["id"])
-    assert not service.verify_token(partial, location["id"])
-
-
-# ── restore ──────────────────────────────────────────────────────────────────
-
-
-def test_restore_returns_bytes_identity_and_decisions(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, ids = _archived(sf, config, lib, archive)
-    archived_hashes = {
-        asset_id: asset.current_sha256 for asset_id, asset in _assets(sf).items()
-    }
-
-    service, plan, result = _restore(sf, config, location["id"])
-    assert (result["restored"], result["failed"], result["state"]) == (2, 0, "complete")
-
-    for asset_id, asset in _assets(sf).items():
-        assert asset.availability_state == availability.ACTIVE
-        assert asset.current_path == str(lib / asset.archive_path)
-        assert sha256_file(asset.current_path) == archived_hashes[asset_id]
-        # The archived copy is a copy: restoring never empties the medium.
-        assert (archive / asset.archive_path).exists()
-        assert asset.archive_location_id == location["id"]
-
-    with sf() as session:
-        # Identity and decisions survived: same ids, same reviews, new occurrence.
-        assert set(ids.values()) == {a.id for a in session.scalars(select(Asset))}
-        assert {r.decision for r in session.scalars(select(SafetyReview))} == {"sfw"}
-        occurrences = [
-            row.reason
-            for row in session.scalars(
-                select(AssetPath).where(AssetPath.asset_id == sorted(ids.values())[0])
-            )
-        ]
-        assert "restore" in occurrences
-
-
-def test_restore_never_overwrites_a_collision(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, ids = _archived(sf, config, lib, archive, seeds=(1,))
-    asset_id = next(iter(ids.values()))
-    with sf() as session:
-        archive_path = session.get(Asset, asset_id).archive_path
-    occupied = lib / archive_path
-    occupied.parent.mkdir(parents=True, exist_ok=True)
-    occupied.write_bytes(b"a different photo already lives here")
-    before = occupied.read_bytes()
-
-    service, plan, result = _restore(sf, config, location["id"])
-    assert result["failed"] == 0
-
-    assert occupied.read_bytes() == before  # untouched
-    restored = _assets(sf)[asset_id].current_path
-    assert restored != str(occupied)
-    assert "(restored)" in restored
-    assert sha256_file(restored) == sha256_file(archive / archive_path)
-
-
-def test_apply_refuses_a_destination_taken_after_planning(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, ids = _archived(sf, config, lib, archive, seeds=(1,))
-    service = RestoreService(sf, config=config)
-    token = service.preflight(location["id"])["token"]
-    plan = service.create(location["id"], None, token=token)
-
-    # Someone drops a file exactly where the plan intends to publish.
-    destination = plan["operations"][0]["destination_path"]
-    from pathlib import Path
-
-    Path(destination).parent.mkdir(parents=True, exist_ok=True)
-    Path(destination).write_bytes(b"squatter")
-
-    result = service.apply(plan["id"])
-    assert result["failed"] == 1
-    operation = service.journal.operations(plan["id"])[0]
-    assert operation["journal_state"] == ArchiveState.FAILED
-    assert operation["error_code"] == "destination_exists"
-    assert Path(destination).read_bytes() == b"squatter"
-    assert _assets(sf)[next(iter(ids.values()))].availability_state == availability.ARCHIVED_ONLINE
-
-
-def test_changed_archive_bytes_mark_the_asset_divergent(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, ids = _archived(sf, config, lib, archive, seeds=(1,))
-    asset_id = next(iter(ids.values()))
-    service = RestoreService(sf, config=config)
-    token = service.preflight(location["id"])["token"]
-    plan = service.create(location["id"], None, token=token)
-
-    # The medium's copy is edited after the plan was approved.
-    with sf() as session:
-        archived_file = archive / session.get(Asset, asset_id).archive_path
-    archived_file.write_bytes(b"edited on the shelf")
-
-    result = service.apply(plan["id"])
-    assert result["failed"] == 1
-    operation = service.journal.operations(plan["id"])[0]
-    assert operation["error_code"] == "bytes_changed"
-    asset = _assets(sf)[asset_id]
-    assert asset.archive_divergent_at is not None  # durable divergence
-    assert asset.availability_state == availability.ARCHIVED_ONLINE
-    assert asset.current_path is None  # nothing was published
-
-
-# ── interruption and idempotency ─────────────────────────────────────────────
-
-
-def test_interrupted_before_publishing_is_resumable(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, ids = _archived(sf, config, lib, archive, seeds=(1,))
-    service = RestoreService(sf, config=config)
-    token = service.preflight(location["id"])["token"]
-    plan = service.create(location["id"], None, token=token)
-    operation = service.journal.operations(plan["id"])[0]
-
-    # Model a kill right after the intent was written: nothing published yet.
-    service.journal.begin(operation["id"], worker_id="killed", fencing_token=1)
-
-    status = service.recovery_status()
-    assert status["operations"][0]["classification"] == "resumable"
-    assert service.recover() == {"resumed": 1, "completed": 0, "manual": 0}
-    assert service.journal.operations(plan["id"])[0]["journal_state"] == ArchiveState.PLANNED
-
-    result = service.apply(plan["id"])
-    assert result["failed"] == 0
-    assert _assets(sf)[next(iter(ids.values()))].availability_state == availability.ACTIVE
-
-
-def test_interrupted_after_publishing_is_finished_by_recovery(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, ids = _archived(sf, config, lib, archive, seeds=(1,))
-    asset_id = next(iter(ids.values()))
-    service = RestoreService(sf, config=config)
-    token = service.preflight(location["id"])["token"]
-    plan = service.create(location["id"], None, token=token)
-    operation = service.journal.operations(plan["id"])[0]
-
-    # Model a kill between the published copy and the database update.
-    from pathlib import Path
-
-    destination = Path(operation["destination_path"])
-    destination.parent.mkdir(parents=True, exist_ok=True)
-    shutil.copy2(operation["source_path"], destination)
-    service.journal.begin(operation["id"], worker_id="killed", fencing_token=1)
-    service.journal.transition(operation["id"], ArchiveState.VERIFIED, fencing_token=1)
-
-    assert service.recovery_status()["operations"][0]["classification"] == "forward"
-    assert service.recover()["completed"] == 1
-    asset = _assets(sf)[asset_id]
-    assert asset.availability_state == availability.ACTIVE
-    assert asset.current_path == str(destination)
-
-    # Repeated recovery and a repeated apply converge on the same state.
-    assert service.recover() == {"resumed": 0, "completed": 0, "manual": 0}
-    again = service.apply(plan["id"])
-    assert (again["skipped"], again["failed"]) == (1, 0)
-    assert _assets(sf)[asset_id].current_path == str(destination)
-
-
-def test_restored_state_survives_restart_and_rescan(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, ids = _archived(sf, config, lib, archive)
-    _restore(sf, config, location["id"])
-
-    restarted = create_session_factory(create_db_engine(config.database_url))
-    InventoryService(restarted).scan(lib)
-
-    assets = _assets(restarted)
-    assert set(assets) == set(ids.values())  # no new identities from the rescan
-    for asset in assets.values():
-        assert asset.availability_state == availability.ACTIVE
-        assert asset.missing_at is None
-
-    # Nothing is archived at that location any more, so there is nothing to restore.
-    again = RestoreService(restarted, config=config).preflight(location["id"])
-    assert _codes(again) == {"empty_scope"}
-
-
-def test_restore_api_round_trip(tmp_path):
-    config, sf, lib, archive = _env(tmp_path)
-    location, ids = _archived(sf, config, lib, archive, seeds=(1,))
-
-    with TestClient(create_app(config)) as client:
-        report = client.post(
-            "/api/v1/restore-preflight", json={"location_id": location["id"]}
-        ).json()
-        assert report["state"] == "ready"
-
-        stale = client.post(
-            "/api/v1/restore-plans",
-            json={"location_id": location["id"], "token": "r1:not-the-token"},
-        )
-        assert stale.status_code == 409
-
-        created = client.post(
-            "/api/v1/restore-plans",
-            json={"location_id": location["id"], "token": report["token"]},
-        )
-        assert created.status_code == 201
-        plan_id = created.json()["id"]
-        assert created.json()["direction"] == "restore"
-
-        # The plan is visible and applying it queues work on the archiver lane.
-        assert client.get(f"/api/v1/restore-plans/{plan_id}").status_code == 200
-        queued = client.post(f"/api/v1/restore-plans/{plan_id}/apply")
-        assert queued.status_code == 200
-        assert queued.json()["job"]["job_type"] == "restore_plan"
-        assert queued.json()["job"]["lock_key"] == "archive"
-        assert client.get("/api/v1/restore-recovery").json()["manual"] == []
-
-    assert _assets(sf)[next(iter(ids.values()))].availability_state == (
-        availability.ARCHIVED_ONLINE  # the worker, not the request, does the work
-    )

tests/integration/test_safety_parity.py

@@ -3,8 +3,6 @@ same fixture files (real exiftool-written keywords)."""
 
 import shutil
 import subprocess
-import sys
-from pathlib import Path
 
 import numpy as np
 import pytest
@@ -13,10 +11,6 @@ from PIL import Image
 from photo_pipeline.integrations import exiftool
 from photo_pipeline.services import safety
 
-# The donor is frozen in the read-only archive (US07-01); parity still compares
-# against it, so this suite is the second and last place that may import it.
-ARCHIVED_SOURCES = Path(__file__).resolve().parents[2] / "legacy_cli_archive" / "src"
-
 EXIFTOOL = shutil.which("exiftool")
 pytestmark = pytest.mark.skipif(EXIFTOOL is None, reason="exiftool not installed")
 
@@ -37,7 +31,6 @@ def _tag(path, keyword):
 
 
 def test_extracted_marks_and_partition_match_donor(tmp_path):
-    sys.path.insert(0, str(ARCHIVED_SOURCES))  # the donor lives in the archive now
     donor_exif = pytest.importorskip("nsfwtag.exif")
 
     nsfw = _jpeg(tmp_path / "nsfw.jpg", 1)

tests/integration/test_security_boundaries.py

@@ -1,136 +0,0 @@
-"""US07-02: what an unhandled failure says, and where the library ends.
-
-Two boundaries that only show up below the HTTP surface:
-
-* Exception text is where internals leak — absolute paths, SQL, and occasionally a
-  credential passed to the call that blew up. The first tests drive the real
-  application with a route that raises such an exception (no production route does)
-  and assert the client sees only a code.
-* The path the database recorded is not the path the filesystem will open a moment
-  later. Analysis is the one stage whose bytes leave this machine, so it resolves
-  the source against the library roots immediately before the provider call.
-"""
-
-from __future__ import annotations
-
-import logging
-import uuid
-from datetime import datetime, timezone
-
-import pytest
-from PIL import Image
-from starlette.testclient import TestClient
-
-from photo_pipeline.api.app import create_app
-from photo_pipeline.config import Config
-from photo_pipeline.db import create_db_engine, create_session_factory, run_migrations
-from photo_pipeline.models import AnalysisResult, Asset
-from photo_pipeline.services.analysis import AnalysisService
-from photo_pipeline.services.safety import SafetyService
-
-BOOM = "sqlite:///Users/someone/Pictures/private.db failed with key sk-secret-123"
-
-
-@pytest.fixture
-def client(tmp_path):
-    app = create_app(Config(data_dir=tmp_path / "data"))
-
-    @app.get("/api/v1/boom")
-    def boom():
-        raise RuntimeError(BOOM)
-
-    with TestClient(app, raise_server_exceptions=False) as test_client:
-        yield test_client
-
-
-def test_an_unhandled_error_returns_a_bare_envelope(client, caplog):
-    with caplog.at_level(logging.ERROR):
-        response = client.get("/api/v1/boom")
-
-    assert response.status_code == 500
-    assert response.json() == {"error": {"code": "internal_error", "message": "internal error"}}
-    assert BOOM not in response.text and "Traceback" not in response.text
-    # The operator still gets the whole story, on the server side.
-    assert BOOM in caplog.text
-
-
-def test_a_refusal_response_still_carries_the_default_headers(client):
-    """A 500 escaping the middleware's response path would also escape its headers."""
-    response = client.get("/api/v1/boom")
-    assert response.headers["x-content-type-options"] == "nosniff"
-    assert response.headers["x-frame-options"] == "DENY"
-
-
-# ── the library boundary, revalidated at the moment of use ───────────────────
-
-
-class RecordingProvider:
-    def __init__(self):
-        self.calls = []
-
-    def analyze(self, path, *, album_hint):
-        self.calls.append(path)
-        return {"description": "a photo", "tags": []}
-
-
-def _library(tmp_path):
-    lib = tmp_path / "lib"
-    lib.mkdir()
-    outside = tmp_path / "outside"
-    outside.mkdir()
-    for path in (lib / "inside.jpg", outside / "private.jpg"):
-        Image.new("RGB", (8, 8), "blue").save(path)
-    (tmp_path / "data").mkdir()
-    config = Config.from_env({"PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data")})
-    run_migrations(config.database_url)
-    return lib, outside, create_session_factory(create_db_engine(config.database_url))
-
-
-def _sfw_asset(sf, path):
-    asset = Asset(
-        id=str(uuid.uuid4()),
-        original_path=str(path),
-        current_path=str(path),
-        discovered_at=datetime.now(timezone.utc),
-        hash_version=1,
-    )
-    with sf() as session:
-        session.add(asset)
-        session.commit()
-    SafetyService(sf).decide(asset.id, "sfw", write_exif=False)
-    return asset.id
-
-
-def test_analysis_will_not_send_a_file_that_left_the_library(tmp_path):
-    """A link swapped under an asset after the scan points at something the user
-    never put in the library. Those bytes must not reach the vision provider — it is
-    the one place in the pipeline where content leaves this machine."""
-    lib, outside, sf = _library(tmp_path)
-    inside = lib / "inside.jpg"
-    asset_id = _sfw_asset(sf, inside)
-    inside.unlink()
-    inside.symlink_to(outside / "private.jpg")
-
-    provider = RecordingProvider()
-    result = AnalysisService(sf, provider=provider, library_roots=(lib,)).run([asset_id])
-
-    assert provider.calls == [], "the provider must never have been constructed a request"
-    assert result == {"analyzed": 0, "skipped": 0, "errors": 1}
-    with sf() as session:
-        row = session.get(AnalysisResult, asset_id)
-    # The failure is visible and names no path.
-    assert row.status == "error"
-    assert "outside the configured library roots" in row.error_message
-    assert str(outside) not in row.error_message
-
-
-def test_analysis_still_reads_a_file_that_stayed_inside(tmp_path):
-    """The guard must resolve real paths, not refuse everything."""
-    lib, _, sf = _library(tmp_path)
-    asset_id = _sfw_asset(sf, lib / "inside.jpg")
-
-    provider = RecordingProvider()
-    result = AnalysisService(sf, provider=provider, library_roots=(lib,)).run([asset_id])
-
-    assert result["analyzed"] == 1
-    assert provider.calls == [str((lib / "inside.jpg").resolve())]

tests/story_traceability.json

@@ -126,42 +126,6 @@
     ],
     "US06-01": [
       "tests/integration/test_archive_preflight.py"
-    ],
-    "US06-02": [
-      "tests/unit/test_archive_journal_states.py",
-      "tests/integration/test_archive_transfer.py",
-      "tests/integration/test_archive_recovery.py"
-    ],
-    "US06-03": [
-      "tests/integration/test_offline_assets.py"
-    ],
-    "US06-04": [
-      "tests/integration/test_restore.py"
-    ],
-    "US06-05": [
-      "tests/e2e/test_archive_ui.py"
-    ],
-    "US06-06": [
-      "tests/e2e/test_phase_f_pipeline.py"
-    ],
-    "US07-01": [
-      "tests/unit/test_legacy_archive.py",
-      "tests/integration/test_legacy_import.py"
-    ],
-    "US07-02": [
-      "tests/unit/test_security_policy.py",
-      "tests/integration/test_security_boundaries.py",
-      "tests/e2e/test_security.py"
-    ],
-    "US07-03": [
-      "tests/integration/test_media_hardening.py",
-      "tests/integration/test_exif_checkpoints.py"
-    ],
-    "US07-04": [
-      "tests/unit/test_fault_artifacts.py",
-      "tests/integration/test_concurrency_races.py",
-      "tests/integration/test_fault_matrix.py",
-      "tests/e2e/test_crash_recovery.py"
     ]
   }
 }

tests/unit/test_archive_journal_states.py

@@ -1,105 +0,0 @@
-"""Archive journal state machine and evidence table (US06-02).
-
-The evidence table decides whether an original may be deleted, so every
-combination of journal state and disk reality is asserted here as a pure function —
-no database, no files. A wrong cell in this table is data loss.
-"""
-
-import pytest
-
-from photo_pipeline.services.archive_journal import (
-    ALLOWED_TRANSITIONS,
-    FORWARD,
-    MANUAL,
-    RESUMABLE,
-    TERMINAL_STATES,
-    UNSAFE_STATES,
-    ArchiveState,
-    _classify,
-    can_transition,
-)
-
-pytestmark = pytest.mark.phase_f
-
-
-# ── state machine ────────────────────────────────────────────────────────────
-
-
-def test_the_happy_path_is_the_only_way_forward():
-    assert can_transition(ArchiveState.PLANNED, ArchiveState.TRANSFERRING)
-    assert can_transition(ArchiveState.TRANSFERRING, ArchiveState.VERIFIED)
-    assert can_transition(ArchiveState.VERIFIED, ArchiveState.REMOVING)
-    assert can_transition(ArchiveState.REMOVING, ArchiveState.COMPLETE)
-    # No shortcut may skip verification before a source is removed.
-    assert not can_transition(ArchiveState.TRANSFERRING, ArchiveState.REMOVING)
-    assert not can_transition(ArchiveState.PLANNED, ArchiveState.VERIFIED)
-    assert not can_transition(ArchiveState.VERIFIED, ArchiveState.COMPLETE)
-
-
-def test_removal_never_goes_backwards():
-    """Once the source may be gone, retrying the transfer would archive nothing and
-    could overwrite the copy that is now the only one."""
-    assert ALLOWED_TRANSITIONS[ArchiveState.REMOVING] == {
-        ArchiveState.COMPLETE,
-        ArchiveState.FAILED,
-    }
-    assert not can_transition(ArchiveState.REMOVING, ArchiveState.TRANSFERRING)
-    assert not can_transition(ArchiveState.REMOVING, ArchiveState.PLANNED)
-
-
-def test_complete_is_terminal_and_failed_can_be_retried():
-    assert ALLOWED_TRANSITIONS[ArchiveState.COMPLETE] == set()
-    assert TERMINAL_STATES == {ArchiveState.COMPLETE}
-    assert can_transition(ArchiveState.FAILED, ArchiveState.TRANSFERRING)
-    assert can_transition(ArchiveState.FAILED, ArchiveState.PLANNED)
-
-
-def test_every_state_that_can_touch_the_disk_is_marked_unsafe():
-    assert UNSAFE_STATES == {
-        ArchiveState.TRANSFERRING,
-        ArchiveState.VERIFIED,
-        ArchiveState.REMOVING,
-    }
-    assert ArchiveState.PLANNED not in UNSAFE_STATES
-
-
-# ── evidence table ───────────────────────────────────────────────────────────
-
-
-@pytest.mark.parametrize(
-    "state,source,destination,matches,expected",
-    [
-        # Nothing published yet: the source is still the only copy.
-        (ArchiveState.TRANSFERRING, True, False, False, RESUMABLE),
-        (ArchiveState.FAILED, True, False, False, RESUMABLE),
-        # The archive copy is durable and correct: finish the remaining steps.
-        (ArchiveState.TRANSFERRING, True, True, True, FORWARD),
-        (ArchiveState.TRANSFERRING, False, True, True, FORWARD),
-        (ArchiveState.VERIFIED, True, True, True, FORWARD),
-        (ArchiveState.REMOVING, False, True, True, FORWARD),
-        (ArchiveState.FAILED, True, True, True, FORWARD),
-        # Wrong bytes at the destination: never overwrite, never remove.
-        (ArchiveState.TRANSFERRING, True, True, False, MANUAL),
-        (ArchiveState.VERIFIED, True, True, False, MANUAL),
-        (ArchiveState.REMOVING, False, True, False, MANUAL),
-        # The journal claims an archived copy that is not there.
-        (ArchiveState.VERIFIED, True, False, False, MANUAL),
-        (ArchiveState.REMOVING, False, False, False, MANUAL),
-        # Neither copy exists — never silently accepted as success.
-        (ArchiveState.TRANSFERRING, False, False, False, MANUAL),
-        (ArchiveState.FAILED, False, False, False, MANUAL),
-    ],
-)
-def test_classification_of_every_evidence_combination(
-    state, source, destination, matches, expected
-):
-    classification, reason = _classify(state, source, destination, matches)
-
-    assert classification == expected, reason
-
-
-def test_a_source_that_is_gone_without_an_archive_copy_is_never_called_recoverable():
-    """The one combination that must always stop: the original left active storage
-    and nothing verifiable took its place."""
-    for state in (ArchiveState.TRANSFERRING, ArchiveState.VERIFIED, ArchiveState.REMOVING):
-        assert _classify(state, False, False, False)[0] == MANUAL

tests/unit/test_fault_artifacts.py

@@ -1,69 +0,0 @@
-"""The failure-artifact collector (US07-04).
-
-CI must be able to explain a randomized failure after the temporary library is
-gone, which means the seed, the database, the journals, the logs, and a complete
-filesystem manifest have to leave the temporary directory before pytest cleans it.
-"""
-
-from __future__ import annotations
-
-import hashlib
-import json
-
-from tests._artifacts import collect, manifest
-
-
-def _library(root):
-    (root / "data").mkdir(parents=True)
-    (root / "data" / "photo_pipeline.db").write_bytes(b"database bytes")
-    (root / "data" / "photo_pipeline.db-wal").write_bytes(b"write ahead log")
-    (root / "data" / "uploads").mkdir()
-    (root / "data" / "uploads" / "batch.log").write_text("INFO uploaded a.jpg\n")
-    (root / "lib").mkdir()
-    (root / "lib" / "a.jpg").write_bytes(b"pixels")
-    return root
-
-
-def test_the_manifest_covers_every_file_with_its_hash(tmp_path):
-    root = _library(tmp_path / "run")
-    entries = {entry["path"]: entry for entry in manifest(root)}
-
-    assert set(entries) == {
-        "data/photo_pipeline.db",
-        "data/photo_pipeline.db-wal",
-        "data/uploads/batch.log",
-        "lib/a.jpg",
-    }
-    assert entries["lib/a.jpg"]["sha256"] == hashlib.sha256(b"pixels").hexdigest()
-    assert entries["lib/a.jpg"]["bytes"] == 6
-
-
-def test_collect_keeps_the_database_journals_logs_and_seed(tmp_path, monkeypatch):
-    root = _library(tmp_path / "run")
-    monkeypatch.setenv("PHOTO_PIPELINE_TEST_ARTIFACTS", str(tmp_path / "artifacts"))
-
-    destination = collect(root, "tests/x.py::test_races[run1]", properties={"race_seed": 1234})
-
-    seeds = json.loads((destination / "seeds.json").read_text())
-    assert seeds["properties"]["race_seed"] == 1234
-    assert seeds["test"].endswith("test_races[run1]")
-
-    kept = {str(p.relative_to(destination / "files")) for p in (destination / "files").rglob("*") if p.is_file()}
-    assert kept == {
-        "data/photo_pipeline.db",
-        "data/photo_pipeline.db-wal",
-        "data/uploads/batch.log",
-    }, "databases, write-ahead logs, and logs are the diagnosable evidence"
-    # The photo itself is never copied out of the library — but it is in the
-    # manifest, so a file that went missing is still provable.
-    assert any(entry["path"] == "lib/a.jpg" for entry in json.loads((destination / "manifest.json").read_text()))
-
-
-def test_collecting_twice_for_one_test_is_safe(tmp_path, monkeypatch):
-    root = _library(tmp_path / "run")
-    monkeypatch.setenv("PHOTO_PIPELINE_TEST_ARTIFACTS", str(tmp_path / "artifacts"))
-
-    first = collect(root, "tests/x.py::test_a")
-    second = collect(root, "tests/x.py::test_a")
-
-    assert first == second and (second / "manifest.json").exists()

tests/unit/test_legacy_archive.py

@@ -1,173 +0,0 @@
-"""Archive lint (US07-01): the frozen CLI archive is complete, honest, and inert.
-
-Freezing the donors is only worth something if three things stay true: the archive
-still holds exactly the bytes it claims, it holds no secret, and production cannot
-reach it. Each of those is a one-line mistake away — a helpful `sys.path` insert, a
-copied `.env`, an edited "just this once" source — so each is asserted here.
-"""
-
-from __future__ import annotations
-
-import hashlib
-import re
-import subprocess
-import sys
-from pathlib import Path
-
-import pytest
-import yaml
-
-REPO = Path(__file__).resolve().parents[2]
-ARCHIVE = REPO / "legacy_cli_archive"
-SOURCES = ARCHIVE / "src"
-LEDGER = ARCHIVE / "donor_ledger.yaml"
-CHECKSUMS = ARCHIVE / "CHECKSUMS.sha256"
-
-# Everything the concept requires an archive to carry (§3 "Donor-first CLI
-# migration and archival"): the sources, their docs, a dependency lock, schema
-# notes, a redacted sample configuration, the ledger, and recorded checksums.
-REQUIRED_ARTIFACTS = (
-    "README.md",
-    "donor_ledger.yaml",
-    "CHECKSUMS.sha256",
-    "requirements-lock.txt",
-    "photo_analyzer.env.sample",
-)
-ARCHIVED_MODULES = ("photo_analyzer", "nsfwtag", "webapp", "nsfw_tag", "compare_models")
-# Only the two suites that compare against the donors may put the archive on the
-# import path; every other tree must not name it at all.
-IMPORT_ALLOWED = {
-    REPO / "tests" / "characterization" / "conftest.py",
-    REPO / "tests" / "characterization" / "test_donor_ledger.py",  # lints the ledger there
-    REPO / "tests" / "integration" / "test_safety_parity.py",
-    Path(__file__),
-}
-SECRET_PATTERNS = (
-    re.compile(r"sk-(?!REPLACE_WITH_YOUR_KEY)[A-Za-z0-9_\-]{16,}"),
-    re.compile(r"AIza[0-9A-Za-z_\-]{20,}"),  # Google API keys
-    re.compile(r"(?i)api[_-]?key\s*[=:]\s*['\"][A-Za-z0-9_\-]{16,}['\"]"),
-)
-
-
-def _archived_files() -> list[Path]:
-    return sorted(p for p in SOURCES.rglob("*") if p.is_file() and "__pycache__" not in p.parts)
-
-
-# ── completeness ─────────────────────────────────────────────────────────────
-
-
-def test_the_archive_carries_every_required_artifact():
-    for artifact in REQUIRED_ARTIFACTS:
-        path = ARCHIVE / artifact
-        assert path.is_file(), f"archive is missing {artifact}"
-        assert path.stat().st_size > 0, f"archive artifact {artifact} is empty"
-    assert (SOURCES / "photo_analyzer.py").is_file()
-    assert (SOURCES / "nsfwtag" / "README.md").is_file(), "donor docs must be archived too"
-    assert (SOURCES / "webapp" / "README.md").is_file()
-
-
-def test_every_ledger_source_is_present_in_the_archive():
-    rows = yaml.safe_load(LEDGER.read_text(encoding="utf-8"))["rows"]
-    for row in rows:
-        source = SOURCES / row["source"]["file"]
-        assert source.is_file(), f"{row['id']}: {row['source']['file']} is not archived"
-
-
-def test_the_readme_records_provenance_and_the_no_import_rule():
-    readme = (ARCHIVE / "README.md").read_text(encoding="utf-8")
-    for expected in ("CHECKSUMS.sha256", "requirements-lock.txt", "donor_ledger.yaml"):
-        assert expected in readme, f"README does not point at {expected}"
-    assert "Schema notes" in readme, "the donor's schema must be documented"
-    assert "nsfw_scores.csv" in readme, "the CSV's fate must be documented"
-
-
-# ── integrity ────────────────────────────────────────────────────────────────
-
-
-def test_every_archived_source_matches_its_checksum():
-    """A frozen archive that silently drifts is not evidence of anything."""
-    recorded = {}
-    for line in CHECKSUMS.read_text(encoding="utf-8").splitlines():
-        digest, _, rel = line.partition("  ")
-        if rel:
-            recorded[rel.strip()] = digest
-    actual = {
-        str(path.relative_to(ARCHIVE)): hashlib.sha256(path.read_bytes()).hexdigest()
-        for path in _archived_files()
-    }
-    assert actual == recorded, "archived sources and CHECKSUMS.sha256 disagree"
-
-
-# ── redaction ────────────────────────────────────────────────────────────────
-
-
-def test_the_archive_contains_no_credential():
-    for path in [*_archived_files(), *(ARCHIVE / a for a in REQUIRED_ARTIFACTS)]:
-        if path.suffix in (".png", ".jpg", ".webp"):
-            continue
-        text = path.read_text(encoding="utf-8", errors="replace")
-        for pattern in SECRET_PATTERNS:
-            assert not pattern.search(text), f"possible secret in {path.relative_to(REPO)}"
-
-
-def test_the_sample_configuration_is_a_placeholder_only():
-    sample = (ARCHIVE / "photo_analyzer.env.sample").read_text(encoding="utf-8")
-    assert "REDACTED" in sample
-    # Placeholders are not key-shaped, so neither a scanner nor a reader can
-    # mistake the sample for a credential.
-    assert "LLM_API_KEY=<" in sample
-    # No real env file, database, log, or CSV may ride along in the archive.
-    strays = [
-        p.name
-        for p in ARCHIVE.rglob("*")
-        if p.is_file() and p.suffix in (".env", ".db", ".sqlite", ".sqlite3", ".log", ".csv")
-    ]
-    assert strays == [], f"unexpected runtime files archived: {strays}"
-
-
-# ── inertness ────────────────────────────────────────────────────────────────
-
-
-def test_production_code_never_imports_an_archived_module():
-    for path in (REPO / "photo_pipeline").rglob("*.py"):
-        text = path.read_text(encoding="utf-8")
-        for module in ARCHIVED_MODULES:
-            assert not re.search(rf"^\s*(import|from)\s+{module}\b", text, re.MULTILINE), \
-                f"{path.relative_to(REPO)} imports the archived {module}"
-        assert "legacy_cli_archive" not in text, \
-            f"{path.relative_to(REPO)} names the archive"
-
-
-def test_only_the_parity_suites_put_the_archive_on_the_import_path():
-    for path in (REPO / "tests").rglob("*.py"):
-        if path in IMPORT_ALLOWED or "__pycache__" in path.parts:
-            continue
-        assert "legacy_cli_archive" not in path.read_text(encoding="utf-8"), \
-            f"{path.relative_to(REPO)} reaches into the archive"
-
-
-def test_the_archived_modules_are_unimportable_from_a_clean_interpreter():
-    """The real check: a fresh process with the repo on its path cannot load them."""
-    script = (
-        "import importlib.util, sys; "
-        f"sys.path.insert(0, {str(REPO)!r}); "
-        "print([m for m in "
-        f"{list(ARCHIVED_MODULES)!r}"
-        " if importlib.util.find_spec(m) is not None])"
-    )
-    result = subprocess.run(
-        [sys.executable, "-c", script], capture_output=True, text=True, cwd=str(REPO), check=True
-    )
-    assert result.stdout.strip() == "[]", f"still importable: {result.stdout.strip()}"
-
-
-def test_the_application_starts_without_the_archive(tmp_path):
-    """Nothing in the runtime path may need the frozen sources to exist."""
-    pytest.importorskip("fastapi")
-    from photo_pipeline.api.app import create_app
-    from photo_pipeline.config import Config
-
-    config = Config.from_env(
-        {"PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data"), "PHOTO_PIPELINE_LIBRARY_ROOTS": ""}
-    )
-    assert create_app(config) is not None

tests/unit/test_path_policy.py

@@ -81,42 +81,3 @@ def test_symlink_within_root_is_allowed(tmp_path):
         pytest.skip("cannot create symlink on this platform")
     found = path_policy.discover([root])
     assert target in found and link in found
-
-
-def test_resolve_in_roots_returns_the_path_the_caller_must_use(tmp_path):
-    """The resolved path is the answer, not a yes/no: a caller that revalidates one
-    name and then opens another has an open symlink race (US07-02)."""
-    root = tmp_path / "lib"
-    (root / "sub").mkdir(parents=True)
-    target = root / "sub" / "real.jpg"
-    target.write_bytes(b"x")
-    link = root / "alias.jpg"
-    os.symlink(target, link)
-
-    assert path_policy.resolve_in_roots([root], link) == target.resolve()
-    assert path_policy.resolve_in_roots([root], target) == target.resolve()
-
-
-def test_resolve_in_roots_refuses_escapes_without_naming_them(tmp_path):
-    root = tmp_path / "lib"
-    root.mkdir()
-    outside = tmp_path / "outside.jpg"
-    outside.write_bytes(b"x")
-    link = root / "alias.jpg"
-    os.symlink(outside, link)
-
-    for candidate in (link, outside, root / "_IGNORE" / "a.jpg"):
-        with pytest.raises(path_policy.PathPolicyError) as raised:
-            path_policy.resolve_in_roots([root], candidate)
-        assert str(outside) not in str(raised.value)
-
-
-def test_resolve_in_roots_checks_every_configured_root(tmp_path):
-    first, second = tmp_path / "one", tmp_path / "two"
-    first.mkdir()
-    second.mkdir()
-    photo = second / "b.jpg"
-    photo.write_bytes(b"x")
-    assert path_policy.resolve_in_roots([first, second], photo) == photo.resolve()
-    # No configured boundary means nothing to check against.
-    assert path_policy.resolve_in_roots([], photo) == photo

tests/unit/test_rename_journal_states.py

@@ -77,8 +77,6 @@ def test_unsafe_states_are_the_ones_where_disk_may_have_changed():
         JournalState.MOVING,
         JournalState.MOVED,
         JournalState.DATABASE_UPDATED,
-        # The move happened and a human still has to decide about it (US07-04).
-        JournalState.ROLLBACK_REQUIRED,
     }
     # planned has not touched anything; complete/rolled_back are settled.
     assert JournalState.PLANNED not in UNSAFE_STATES