Compare commits
1 Commits
main
...
us/US09-04
| Author | SHA1 | Date | |
|---|---|---|---|
| 1616703071 |
95
docs/errors.md
Normal file
@@ -0,0 +1,95 @@
|
||||
# Errors and refusals
|
||||
|
||||
[← Documentation index](index.md)
|
||||
|
||||
Every error the API returns carries a code:
|
||||
|
||||
```json
|
||||
{"error": {"code": "lock_held", "message": "a library_write job is already running"}}
|
||||
```
|
||||
|
||||
Most of them are **refusals, not faults**. This application would rather stop and
|
||||
explain than guess about somebody's photographs, so a code below usually means it
|
||||
protected something. Each row says what caused it and what to do.
|
||||
|
||||
## Access and the trust boundary
|
||||
|
||||
| code | cause | what to do |
|
||||
|---|---|---|
|
||||
| `unauthenticated` | no application session | reload the page; the browser bootstraps one |
|
||||
| `access_denied` | wrong or missing access secret | check `PHOTO_PIPELINE_ACCESS_SECRET`. Attempts are logged with the caller's address only |
|
||||
| `too_many_attempts` | more than five failed secret attempts in a minute | wait. This is the rate limit, not a lockout |
|
||||
| `csrf_failed` | a mutation without the session's token | reload; a stale tab has an old token |
|
||||
| `host_not_allowed` | the `Host` header is not a configured name | add the real hostname to `PHOTO_PIPELINE_ALLOWED_HOSTS` |
|
||||
| `origin_not_allowed` | the request came from another origin | not something a browser tab of this app produces |
|
||||
| `cross_site_blocked` | another site triggered the request | expected — this is the protection working |
|
||||
| `payload_too_large` | the body exceeds `PHOTO_PIPELINE_MAX_REQUEST_BYTES` | send less; every endpoint takes small commands |
|
||||
| `path_not_allowed` | a path resolved outside the library roots | usually a symlink. Nothing outside the roots is reachable, by design |
|
||||
|
||||
## The safety gate
|
||||
|
||||
| code | cause | what to do |
|
||||
|---|---|---|
|
||||
| `dry_run_not_approved` | mutation is gated until a dry run is approved | run `dry-run`, read it, then `approve-dry-run` |
|
||||
| `approval_scope_mismatch` | the approval covers different library roots | approve a report for the roots actually configured |
|
||||
| `approval_unreadable` | the approval record cannot be read | re-approve; do not edit it by hand |
|
||||
|
||||
## Concurrency and staleness
|
||||
|
||||
| code | cause | what to do |
|
||||
|---|---|---|
|
||||
| `lock_held` | a mutating job already holds the lane | wait for it. One at a time is what makes a crash recoverable |
|
||||
| `version_conflict` | you acted on a version that changed underneath you | the view re-renders with the server's truth; decide again |
|
||||
| `invalid_transition` | a state change that the machine does not allow | usually a stale tab; reload |
|
||||
| `job_error` | the job service refused the request | the message says why |
|
||||
| `rename_recovery_required` | an interrupted rename is unresolved | resolve it in [Renames](stages/renames.md). Unrelated mutations stay blocked on purpose |
|
||||
|
||||
## Stage refusals
|
||||
|
||||
| code | cause | what to do |
|
||||
|---|---|---|
|
||||
| `nothing_to_score` | no photo is eligible for safety scoring | the queue is already decided |
|
||||
| `nothing_eligible` | no photo is eligible for analysis | resolve safety decisions first |
|
||||
| `nothing_to_plan` | no approved album name to build a plan from | approve a proposal first |
|
||||
| `invalid_decision` | the decision is not one this cluster accepts | reload the cluster |
|
||||
| `invalid_proposal` | the name is empty, reserved, or has forbidden characters | fix the name; the rules are in [Album proposals](stages/albums.md) |
|
||||
| `unknown_album` | the album is not a folder the library knows | rescan |
|
||||
| `cannot_apply` | the plan is invalid, stale, or another plan is unresolved | read the blockers in the preview |
|
||||
| `cannot_rollback` | the operation is complete or its preconditions no longer hold | rollback is recovery, not undo |
|
||||
|
||||
## Upload
|
||||
|
||||
| code | cause | what to do |
|
||||
|---|---|---|
|
||||
| `stale_preflight` | bytes changed between approval and running | re-run preflight; this is the check that stops the wrong bytes being uploaded |
|
||||
| `not_runnable` | the batch is in a state that must not be re-run | an uncertain batch is verified, never retried |
|
||||
| `requires_verification` | the outcome is unknown; the server may hold the files | verify against Immich, then resolve |
|
||||
| `changed_after_upload` | the file changed after a successful upload | uploading again may create or upgrade an asset |
|
||||
|
||||
## Operations
|
||||
|
||||
| code | cause | what to do |
|
||||
|---|---|---|
|
||||
| `backup_failed` | the snapshot could not be taken or verified | check free space and the message |
|
||||
| `invalid_retention` | a `--keep` value that is not a positive count | the newest backup is never pruned |
|
||||
| `not_found` | no such id | usually a stale link |
|
||||
|
||||
## Generic
|
||||
|
||||
| code | cause |
|
||||
|---|---|
|
||||
| `invalid_request` | the request body failed validation. Field names only — never the values, which end up in logs and screenshots |
|
||||
| `http_error` | a plain HTTP-level refusal, with its status |
|
||||
| `internal_error` | an unhandled error. The code is all you get; the traceback is in the server log, because it can carry paths and credentials |
|
||||
|
||||
## Diagnostics warnings
|
||||
|
||||
Not errors — reported by `diagnostics` and worth acting on. They are listed with
|
||||
their meanings in [Diagnostics](stages/diagnostics.md): `disk_low`, `disk_critical`,
|
||||
`cache_over_quota`, `wal_growth`, `tool_version_drift`, `legacy_process_active`, and
|
||||
`no_roots`.
|
||||
|
||||
## Startup refusals
|
||||
|
||||
Exit codes 2 to 5 are covered by the installation manual under
|
||||
[when it refuses to start](installation.md#when-it-refuses-to-start).
|
||||
66
docs/first-pass.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# A guided first pass
|
||||
|
||||
[← Documentation index](index.md)
|
||||
|
||||
One library, from nothing to a verified upload. Each stage links to its own page; this
|
||||
is the order, and — more importantly — where the points of no return are.
|
||||
|
||||

|
||||
|
||||
The workflow page is the home view and the honest summary: each stage shows its state,
|
||||
its counts, why it is blocked if it is, and the one action that moves it forward.
|
||||
|
||||
| # | stage | reversible afterwards? |
|
||||
|---|---|---|
|
||||
| 0 | [Inventory](stages/inventory.md) | nothing was changed |
|
||||
| 1 | [Duplicate review](stages/duplicates.md) | yes — decisions can be changed, no file is deleted |
|
||||
| 2 | [Safety review](stages/safety.md) | the decision, yes; the EXIF keyword is a file write |
|
||||
| 3 | [Analysis](stages/analysis.md) | the result, yes; the caption and keywords are a file write |
|
||||
| 4 | [Album proposals](stages/albums.md) | yes — approving renames nothing |
|
||||
| 5 | [Renames](stages/renames.md) | **your folders move.** Recoverable, journaled, but real |
|
||||
| 6 | [Upload](stages/uploads.md) | **assets exist on the Immich server.** Not undone from here |
|
||||
| 7 | [Archive](stages/archive.md) | **originals leave active storage.** Restorable while the medium is reachable |
|
||||
| — | [Diagnostics](stages/diagnostics.md) | read-only, any time |
|
||||
|
||||
## Before the first run
|
||||
|
||||
Take a backup of the photo library itself. This application is careful — it previews,
|
||||
journals, and verifies — but it is the first time you are pointing it at your
|
||||
photographs, and a backup is cheaper than confidence.
|
||||
|
||||
If the library is irreplaceable, turn on `PHOTO_PIPELINE_REQUIRE_DRY_RUN_APPROVAL`
|
||||
before anything else. Every mutating request is then refused until you have produced
|
||||
a read-only reconciliation report and approved it by name.
|
||||
|
||||
## The pass
|
||||
|
||||
1. **Scan.** Inventory → *Rescan*. Reads only. Check the count, and check that
|
||||
nothing under `_IGNORE/` appears.
|
||||
2. **Resolve duplicates.** Exact byte matches can be accepted as recommended; anything
|
||||
fuzzy gets looked at. Resolving first is what stops you paying to analyse the same
|
||||
photo twice.
|
||||
3. **Decide safety.** Every canonical photo becomes `sfw` or `nsfw`. Nothing reaches
|
||||
the vision provider until it is confirmed SFW — this is the gate the whole design
|
||||
exists around.
|
||||
4. **Analyse.** Only confirmed-SFW photos. Each result is written to the database and
|
||||
projected into EXIF, then read back and verified.
|
||||
5. **Propose album names.** Evidence from what was analysed. Edit anything you disagree
|
||||
with. Approving changes no file.
|
||||
6. **Build the rename plan, read it, apply it.** The preview shows every old → new
|
||||
path. This is where folders move.
|
||||
7. **Rescan.** Paths are reconciled; identities are unchanged.
|
||||
8. **Upload.** Preflight lists every blocker. The command is previewed without the API
|
||||
key. One album at a time.
|
||||
9. **Verify.** An uncertain upload is not a failure and not a success — ask the server.
|
||||
10. **Archive** (optional). Copy, verify, and only then reclaim the space.
|
||||
|
||||
Stop anywhere. Every stage is resumable, and closing the browser cancels nothing.
|
||||
|
||||
## Two rules worth internalising
|
||||
|
||||
**The application refuses more than it warns.** A refusal is not a fault; it is the
|
||||
design working. [Errors and refusals](errors.md) explains each one.
|
||||
|
||||
**One long job at a time.** Browsing stays available while a job runs, but a second
|
||||
mutating job is refused — that is what makes a crash recoverable rather than
|
||||
ambiguous.
|
||||
BIN
docs/images/albums.png
Normal file
|
After Width: | Height: | Size: 64 KiB |
BIN
docs/images/analysis.png
Normal file
|
After Width: | Height: | Size: 31 KiB |
BIN
docs/images/archive.png
Normal file
|
After Width: | Height: | Size: 127 KiB |
BIN
docs/images/duplicates.png
Normal file
|
After Width: | Height: | Size: 24 KiB |
BIN
docs/images/inventory.png
Normal file
|
After Width: | Height: | Size: 56 KiB |
BIN
docs/images/renames.png
Normal file
|
After Width: | Height: | Size: 30 KiB |
BIN
docs/images/safety.png
Normal file
|
After Width: | Height: | Size: 31 KiB |
BIN
docs/images/statistics.png
Normal file
|
After Width: | Height: | Size: 127 KiB |
BIN
docs/images/uploads.png
Normal file
|
After Width: | Height: | Size: 101 KiB |
BIN
docs/images/workflow.png
Normal file
|
After Width: | Height: | Size: 63 KiB |
@@ -15,18 +15,32 @@ repository under `docs/`, on Gitea, and inside the running application under
|
||||
2. [Installation and operations](installation.md) — host and container installation,
|
||||
every setting, the first-run checklist, upgrades, backup and restore, and what
|
||||
each refusal at startup means.
|
||||
3. [Architecture](architecture.md) — the context and runtime diagrams, what each
|
||||
3. [A guided first pass](first-pass.md) — one library from scan to verified upload,
|
||||
with the point of no return named in each stage.
|
||||
4. [Architecture](architecture.md) — the context and runtime diagrams, what each
|
||||
module owns, the three journals a restart reads, and where every invariant is
|
||||
enforced.
|
||||
|
||||
## Being written
|
||||
## The stages, one page each
|
||||
|
||||
The remaining manuals are accepted work, not aspiration; each is a story in
|
||||
[E09](https://git.domverse-berlin.eu/domverse/photoanalyzer/src/branch/main/delivery_backlog/E09-documentation.md)
|
||||
and will appear here as it lands.
|
||||
Every page answers the same four questions: what the stage is for, what you decide,
|
||||
what it changes on disk or on the server, and what it refuses.
|
||||
|
||||
- **User manual** — one page per workflow stage with screenshots of the real
|
||||
application, and a catalogue of every error and refusal (US09-04).
|
||||
1. [Inventory and discovery](stages/inventory.md)
|
||||
2. [Duplicate review](stages/duplicates.md)
|
||||
3. [Safety review](stages/safety.md)
|
||||
4. [Analysis](stages/analysis.md)
|
||||
5. [Album proposals](stages/albums.md)
|
||||
6. [Renames](stages/renames.md)
|
||||
7. [Upload](stages/uploads.md)
|
||||
8. [Archive](stages/archive.md)
|
||||
9. [Diagnostics and library statistics](stages/diagnostics.md)
|
||||
|
||||
## When something goes wrong
|
||||
|
||||
- [Errors and refusals](errors.md) — every error code, what caused it, what to do.
|
||||
- [Recovery](recovery.md) — what the application resolves by itself, and what needs
|
||||
you.
|
||||
|
||||
## Conventions
|
||||
|
||||
|
||||
@@ -72,6 +72,6 @@ python -m photo_pipeline serve # the API and this browser application
|
||||
python -m photo_pipeline worker # the process that does the long work
|
||||
```
|
||||
|
||||
The full procedure, the container path, and every setting belong to the installation
|
||||
manual, which is being written next — see
|
||||
[the documentation index](index.md#being-written).
|
||||
The full procedure, the container path, and every setting are in the
|
||||
[installation manual](installation.md); if you would rather start using it, take
|
||||
[the guided first pass](first-pass.md).
|
||||
|
||||
62
docs/recovery.md
Normal file
@@ -0,0 +1,62 @@
|
||||
# Recovery
|
||||
|
||||
[← Documentation index](index.md)
|
||||
|
||||
What the application resolves by itself, and what needs you. The dividing line is
|
||||
simple: it resumes when the evidence is unambiguous, and it stops and asks when it is
|
||||
not. It never guesses about your files.
|
||||
|
||||
## An interrupted rename
|
||||
|
||||
A rename records its intent **before** touching the disk, so after a crash the
|
||||
journal plus the actual filesystem give a verdict per operation:
|
||||
|
||||
| verdict | means | what happens |
|
||||
|---|---|---|
|
||||
| resumable | the move provably did not happen | reset to planned and run again |
|
||||
| rollback-safe | the move happened; the remaining steps can be completed or undone | drive it forward, or roll it back |
|
||||
| manual | the evidence is contradictory | left alone, and it keeps blocking |
|
||||
|
||||
Open [Renames](stages/renames.md); the recovery panel lists every unresolved
|
||||
operation and offers a resolve action only where one is safe. Until then, unrelated
|
||||
mutations are refused with `rename_recovery_required` — building new work on a
|
||||
half-applied move is how a library becomes unexplainable.
|
||||
|
||||
## An uncertain upload
|
||||
|
||||
`unknown_requires_verification` means the uploader died after Immich may have
|
||||
accepted the files. It is **not** retryable. Verification asks the server whether it
|
||||
holds the recorded SHA-1 and answers present, absent, or inconclusive. An
|
||||
inconclusive answer is resolved by you, with your evidence and name recorded in the
|
||||
batch's history.
|
||||
|
||||
## A failed migration
|
||||
|
||||
A pending schema change is snapshotted before it is applied. If the upgrade fails,
|
||||
the previous database and its `pre-migration` backup are both intact and the error
|
||||
log names the backup directory. Stop everything and run the
|
||||
[restore drill](installation.md#restoring).
|
||||
|
||||
## A restored backup
|
||||
|
||||
After restoring, run a scan. Paths are reconciled against the real library and
|
||||
identities are preserved where hashes match. Anything that does not match becomes a
|
||||
visible `stale` or `divergent` state instead of being quietly accepted.
|
||||
|
||||
Mount every archive medium the manifest names before archiving again: the database
|
||||
records where archived originals live, it does not contain them.
|
||||
|
||||
## A job that will not start
|
||||
|
||||
Check, in this order: is a worker running; is another mutating job holding the lane
|
||||
(`lock_held`); is an interrupted rename blocking everything
|
||||
(`rename_recovery_required`); is the library lock held by a process that is still
|
||||
alive (exit code `2` names the holder).
|
||||
|
||||
## What is never automatic
|
||||
|
||||
No file is deleted, no folder is renamed, no upload is retried, and no archive source
|
||||
is removed without either an explicit confirmation from you or a verification the
|
||||
application performed itself. If you are reading this page because something happened
|
||||
that you did not approve, that is a bug worth reporting — with the diagnostics output
|
||||
and the relevant journal, both of which are safe to share: they carry no secrets.
|
||||
47
docs/stages/albums.md
Normal file
@@ -0,0 +1,47 @@
|
||||
# Album proposals
|
||||
|
||||
[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
|
||||
|
||||

|
||||
|
||||
**What it is for.** Turning a folder of analysed photographs into a name a person
|
||||
would have chosen, before any folder is touched.
|
||||
|
||||
**What you decide.** The final name. The proposal is a starting point with its
|
||||
reasoning attached; you edit it, accept it, or ignore it.
|
||||
|
||||
**What it changes.** A row per album: proposed name, rationale, confidence, and your
|
||||
final name, with a version. **Approving renames nothing.** Not one file moves at this
|
||||
stage — approval only marks a name as agreed, and the [rename stage](renames.md) is
|
||||
where it becomes a plan you have to confirm separately.
|
||||
|
||||
**What it refuses.** A name containing `/ \ : * ? " < > |`, a reserved device name, or
|
||||
one that collides with an existing folder is rejected while you type and again on the
|
||||
server. Approval is refused when the evidence changed since the proposal was
|
||||
generated — the proposal is stale, and regenerating is the honest fix. Editing with a
|
||||
stale version returns a conflict and shows you the server's truth rather than
|
||||
overwriting it.
|
||||
|
||||
## Reading the view
|
||||
|
||||
The left pane lists albums with their state: `none`, `proposed`, `edited`,
|
||||
`approved`, `error`, or `stale`. The right pane shows the evidence the name was built
|
||||
from — date range, dominant tags, locations, counts — then the suggested name, the
|
||||
rationale, the confidence, and an editable final name.
|
||||
|
||||
The default naming shape is predictable and sortable:
|
||||
|
||||
```text
|
||||
YYYY-MM — Place — Event
|
||||
YYYY — Event
|
||||
Place — Event
|
||||
```
|
||||
|
||||
## While a rename is unresolved
|
||||
|
||||
Generating and approving proposals is blocked while an interrupted rename is
|
||||
outstanding, with `409 rename_recovery_required`. Resolve the rename first: building
|
||||
new names on top of a half-applied move is how a library becomes hard to reason
|
||||
about.
|
||||
|
||||
Next: [renames](renames.md).
|
||||
39
docs/stages/analysis.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# Analysis
|
||||
|
||||
[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
|
||||
|
||||

|
||||
|
||||
**What it is for.** Describing what a photograph shows, so albums can be named from
|
||||
evidence and the library can be searched by content.
|
||||
|
||||
**What you decide.** When to run it, and over what scope. The descriptions themselves
|
||||
come from the configured vision provider.
|
||||
|
||||
**What it changes.** For each eligible photo: a stored result — description, tags,
|
||||
approximate year, location hint, model and prompt version — and then an EXIF
|
||||
checkpoint that merges a **managed caption segment** and additive keywords into the
|
||||
file, reads them back, and verifies that nothing else moved. The file's SHA-256 is
|
||||
refreshed after the write.
|
||||
|
||||
**What it refuses.** Anything not confirmed SFW. Anything not canonical. A malformed
|
||||
provider response is quarantined rather than stored. Keywords are only ever added,
|
||||
never removed, because EXIF keywords carry no ownership and deleting a generated one
|
||||
could delete yours.
|
||||
|
||||
## Running it
|
||||
|
||||
The view shows eligible, analysed, pending, and error counts, and the job's live
|
||||
progress. Starting a second mutating job while it runs is refused — that is the
|
||||
one-writer rule, not a queue.
|
||||
|
||||
Cancelling drains safely: completed items stay completed, and resuming continues
|
||||
rather than restarting. A photo may be sent to the provider twice if a crash happens
|
||||
mid-item, but its stored result and its EXIF are written once.
|
||||
|
||||
## Costs
|
||||
|
||||
Each analysed photo is a provider call. Resolving duplicates first is what keeps that
|
||||
number honest, and the counts here are the ones to check before starting a large run.
|
||||
|
||||
Next: [album proposals](albums.md).
|
||||
43
docs/stages/archive.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Archive
|
||||
|
||||
[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
|
||||
|
||||

|
||||
|
||||
**What it is for.** Moving a finished album out of active storage onto another disk,
|
||||
while keeping everything the library knows about it.
|
||||
|
||||
**What you decide.** Which album, which destination, and whether to accept the
|
||||
reclaimed space in exchange for needing that medium mounted to see the originals
|
||||
again.
|
||||
|
||||
**What it changes.** Files are copied to the archive location, verified there, and
|
||||
only then removed from the library. The asset keeps its id, its hashes, its
|
||||
decisions, its analysis, its upload history, and a durable preview thumbnail;
|
||||
`current_path` becomes null and availability becomes `archived_online` or
|
||||
`archived_offline`.
|
||||
|
||||
**What it refuses.** Preflight blocks on: an unverified upload, a checksum that no
|
||||
longer matches the uploaded bytes, an unmounted or unwritable destination,
|
||||
insufficient free space plus the configured reserve, a destination that already
|
||||
holds unexpected paths, a missing durable thumbnail, and any conflicting job. The
|
||||
source is **never** removed before the archived copy is verified — not because Immich
|
||||
reported success, which is an ingest, not a backup.
|
||||
|
||||
## Offline is not missing
|
||||
|
||||
An archived photo whose medium is unplugged is `archived_offline`. It still appears
|
||||
in search, still participates in duplicate detection through its retained hashes and
|
||||
thumbnail, and is never reported as lost. If a full-resolution comparison is needed,
|
||||
the application asks you to mount the named medium rather than guessing.
|
||||
|
||||
## Restore
|
||||
|
||||
Restore is planned and journaled like everything else: the medium must be online, the
|
||||
bytes are copied back and verified into a collision-free destination, a new active
|
||||
path is registered, and a reconciliation scan follows. Existing safety, analysis,
|
||||
EXIF, and upload state stay valid when hashes match — and when they do not, the
|
||||
result is a visible `divergent` state rather than silent acceptance of a different
|
||||
file.
|
||||
|
||||
Next: [diagnostics](diagnostics.md).
|
||||
49
docs/stages/diagnostics.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# Diagnostics and library statistics
|
||||
|
||||
[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
|
||||
|
||||

|
||||
|
||||
**What it is for.** Answering "is this installation healthy, and what is in this
|
||||
library?" — the first thing to look at when a stage behaves unexpectedly.
|
||||
|
||||
**What you decide.** Nothing. Both are read-only.
|
||||
|
||||
**What it changes.** Nothing.
|
||||
|
||||
**What it refuses.** Nothing — both are reads, and both stay available while a job
|
||||
holds the library. That is deliberate: the moment you most need to know what the
|
||||
installation is doing is while it is busy.
|
||||
|
||||
## Statistics
|
||||
|
||||
The Stats view summarises the library itself: how many photos are analysed, the
|
||||
common tags, the distribution across albums and years. It answers questions about
|
||||
your photographs.
|
||||
|
||||
## Diagnostics
|
||||
|
||||
Diagnostics answers questions about the *installation*, and lives at
|
||||
`GET /api/v1/diagnostics` and on the command line:
|
||||
|
||||
```bash
|
||||
python -m photo_pipeline diagnostics
|
||||
```
|
||||
|
||||
It reports the database, write-ahead log, thumbnail cache, uploader reports, backups,
|
||||
and logs as separate sizes, plus free space, the tool versions it found against the
|
||||
ones the image pinned, and which locks are held and by whom.
|
||||
|
||||
Its warnings are the ones worth acting on:
|
||||
|
||||
| warning | means |
|
||||
|---|---|
|
||||
| `disk_low` / `disk_critical` | free space is running out; mutating stages stop before the reserve |
|
||||
| `cache_over_quota` | the thumbnail cache exceeds its configured quota |
|
||||
| `wal_growth` | the write-ahead log is outgrowing its database — usually a long-running reader |
|
||||
| `tool_version_drift` | the installed `exiftool` or `immich-go` is not the version the image pinned |
|
||||
| `legacy_process_active` | the frozen command-line tools are writing this library. Stop them |
|
||||
| `no_roots` | no library root is configured, so there is nothing to work on |
|
||||
|
||||
An empty `warnings` list on a fresh installation is what the
|
||||
[first-run checklist](../installation.md#first-run-checklist) is looking for.
|
||||
36
docs/stages/duplicates.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# Duplicate review
|
||||
|
||||
[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
|
||||
|
||||

|
||||
|
||||
**What it is for.** Deciding which file is *the* copy of a picture, before anything
|
||||
expensive or irreversible happens to the others.
|
||||
|
||||
**What you decide.** For each cluster: keep the recommended canonical, choose a
|
||||
different one, keep everything as variants, declare it not a duplicate, or defer.
|
||||
|
||||
**What it changes.** Only the decision, recorded as an auditable event. **No file is
|
||||
ever deleted.** Non-canonical copies stay exactly where they are; they are simply
|
||||
excluded from analysis and upload.
|
||||
|
||||
**What it refuses.** Exact byte and identical-pixel matches may be accepted on the
|
||||
recommendation. Anything fuzzy — a crop, a re-encode, a burst frame — requires an
|
||||
explicit confirmation, because a perceptual hash is evidence, not proof.
|
||||
|
||||
## Reading a cluster
|
||||
|
||||
Each member shows its path, role, pHash distance, and size, with synchronised zoom
|
||||
across the previews. The recommendation is visible but never pre-applied for fuzzy
|
||||
matches. The confidence band tells you how much the machine is claiming.
|
||||
|
||||
Rejecting a pair records a negative link, so a later rescan does not keep proposing
|
||||
the same wrong match.
|
||||
|
||||
## Why first
|
||||
|
||||
Resolving duplicates before safety and analysis is what stops you reviewing and
|
||||
paying for the same photograph three times. It also means the album evidence is built
|
||||
from one copy of each picture rather than a skewed count.
|
||||
|
||||
Next: [safety review](safety.md).
|
||||
39
docs/stages/inventory.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# Inventory and discovery
|
||||
|
||||
[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
|
||||
|
||||

|
||||
|
||||
**What it is for.** Finding every supported photo in the configured library roots and
|
||||
giving each one a permanent identity, so that everything afterwards can refer to a
|
||||
picture rather than a path.
|
||||
|
||||
**What you decide.** Nothing. This stage is the only one with no judgement in it.
|
||||
|
||||
**What it changes.** On disk, nothing at all — discovery reads. In the database it
|
||||
records each photo's path, size, timestamps, full-file SHA-256, normalised pixel hash,
|
||||
and perceptual hash, and it opens a path-history entry.
|
||||
|
||||
**What it refuses.** Anything under an `_IGNORE/` directory is never traversed,
|
||||
counted, hashed, or previewed. Unsupported extensions and unreadable files are
|
||||
reported rather than skipped silently. Nothing outside the configured roots is
|
||||
reachable, including through a symlink.
|
||||
|
||||
## Reading the view
|
||||
|
||||
Each row is one asset: its current path, availability, whether the file is present,
|
||||
and its size. The filter searches paths; the availability selector separates active
|
||||
photos from archived ones.
|
||||
|
||||
A **rescan** after moving files around outside the application is the normal way to
|
||||
reconcile: a file that moved keeps its identity, because identity is the hash and the
|
||||
id, not the location.
|
||||
|
||||
## What good looks like
|
||||
|
||||
The count matches what you expect, `_IGNORE/` contents are absent, and nothing shows
|
||||
as missing. A photo listed as missing means the file is no longer at its recorded
|
||||
path and no new path matched its hashes — that is a question for you, not something
|
||||
the scanner resolves by guessing.
|
||||
|
||||
Next: [duplicate review](duplicates.md).
|
||||
46
docs/stages/renames.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# Renames
|
||||
|
||||
[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
|
||||
|
||||

|
||||
|
||||
**What it is for.** Applying approved album names to the actual folders — the first
|
||||
stage that changes your filesystem.
|
||||
|
||||
**What you decide.** Whether to apply the plan, after reading it. The confirmation
|
||||
names the plan, its version, and its checksum, so what you approve is what runs.
|
||||
|
||||
**What it changes.** Folders move. For each operation: the intent is journaled
|
||||
*before* the disk is touched, the move is made, `assets.current_path` is updated by
|
||||
id in one transaction, the old path is closed and a new one opened in path history,
|
||||
the result is verified, and only then is the operation complete.
|
||||
|
||||
**What it refuses.** A plan is invalid — not merely warned about — when a source is
|
||||
missing or changed, two operations target the same destination, a destination already
|
||||
exists, a path would leave the library or enter `_IGNORE/`, or a source is a symlink.
|
||||
Case-only renames and cross-filesystem moves are allowed but flagged, because they
|
||||
take a different, staged route. A plan whose checksum no longer matches is refused
|
||||
rather than reinterpreted.
|
||||
|
||||
## Reading the preview
|
||||
|
||||
Every operation shows both full paths, the kind of move (`rename`, `case-only
|
||||
rename`, `cross-filesystem move`), how many photos it affects, its journal state, and
|
||||
any issue with its severity. The plan can be exported as JSON before you commit to it.
|
||||
|
||||
**A run cannot be stopped part-way.** It can be interrupted — by a crash, a power
|
||||
cut, a killed container — and interruption is recoverable, but there is no cancel
|
||||
button once it starts. That is stated on the confirmation, not hidden in a manual.
|
||||
|
||||
## After an interruption
|
||||
|
||||
The recovery panel lists each unresolved operation with a verdict read from the
|
||||
journal *and the disk*: resumable, safely rollbackable, or needing a person. It never
|
||||
guesses from a missing path alone. Until it is resolved, unrelated mutations are
|
||||
refused with `409 rename_recovery_required`.
|
||||
|
||||
Rollback is recovery, not undo. A completed operation is terminal — the button only
|
||||
appears for operations that are actually reversible. See
|
||||
[recovery](../recovery.md).
|
||||
|
||||
Next: [upload](uploads.md).
|
||||
43
docs/stages/safety.md
Normal file
@@ -0,0 +1,43 @@
|
||||
# Safety review
|
||||
|
||||
[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
|
||||
|
||||

|
||||
|
||||
**What it is for.** Deciding which photographs may be sent to a cloud vision
|
||||
provider. This is the privacy gate the rest of the design is built around.
|
||||
|
||||
**What you decide.** `sfw`, `nsfw`, or defer — per photo, or in bulk from the filters.
|
||||
A local model can score first to order the queue, but the score is a suggestion; the
|
||||
decision is yours.
|
||||
|
||||
**What it changes.** The decision is stored in the database *and* projected into the
|
||||
file: one mutually exclusive `sfw` or `nsfw` keyword written into `Keywords` and
|
||||
`Subject`. The write is merged with existing metadata, read back, and verified, and
|
||||
the file's SHA-256 is refreshed afterwards.
|
||||
|
||||
**What it refuses.** An undecided or deferred photo does not reach the vision
|
||||
provider, and does not reach Immich either. If the read-back shows that a field the
|
||||
stage does not own has changed, the checkpoint is marked `divergent`, the asset is not
|
||||
marked verified, and the next mutating stage is blocked rather than proceeding on
|
||||
metadata nobody trusts.
|
||||
|
||||
## The rule that matters
|
||||
|
||||
> Only a confirmed `sfw` photo may enter cloud content analysis. A confirmed `nsfw`
|
||||
> photo skips analysis entirely but remains eligible for upload to Immich once its
|
||||
> keyword is verified.
|
||||
|
||||
That is why marking something NSFW is not a punishment: it routes the photo around an
|
||||
external service while keeping it in your own library workflow.
|
||||
|
||||
The gate is re-checked *after* the provider call as well. If a decision flips to NSFW
|
||||
while an analysis is in flight, the result is discarded rather than stored.
|
||||
|
||||
## Reading the view
|
||||
|
||||
The tabs filter by state. Each row shows the path, the model's score, its suggestion,
|
||||
your decision, and an `✓ exif` badge once the keyword is verified on disk. A row
|
||||
without that badge has a decision the file does not yet carry.
|
||||
|
||||
Next: [analysis](analysis.md).
|
||||
45
docs/stages/uploads.md
Normal file
@@ -0,0 +1,45 @@
|
||||
# Upload
|
||||
|
||||
[← Documentation index](../index.md) · [Guided first pass](../first-pass.md)
|
||||
|
||||

|
||||
|
||||
**What it is for.** Sending an approved album to Immich through `immich-go`, with an
|
||||
audit trail of exactly which bytes went.
|
||||
|
||||
**What you decide.** Which albums, and whether to proceed once the preflight has
|
||||
listed its blockers. Upload is never an automatic consequence of renaming.
|
||||
|
||||
**What it changes.** Nothing locally. On the Immich server, assets appear. Locally a
|
||||
batch is recorded: the album, the asset ids, the redacted command, the uploader
|
||||
version, the pre-upload SHA-256 **and** the SHA-1 Immich matches on, the output log,
|
||||
and the outcome counts.
|
||||
|
||||
**What it refuses.** Preflight blocks on: missing credentials, an unreachable server,
|
||||
a missing `immich-go`, an unresolved rename, an undecided or deferred safety
|
||||
decision, an unverified EXIF checkpoint, incomplete analysis for an SFW photo, a file
|
||||
whose bytes changed since it was checkpointed, and a partial scope you have not
|
||||
explicitly accepted. The approval is then **re-proved immediately before the uploader
|
||||
runs** — bytes edited after you approved fail with `stale_preflight` and the uploader
|
||||
never starts.
|
||||
|
||||
## Reading the view
|
||||
|
||||
The scope shows exactly which albums and how many photos. The command preview is the
|
||||
real command with the API key removed; the key never reaches the browser and never
|
||||
reaches a log.
|
||||
|
||||
## Uncertain is not failed
|
||||
|
||||
If the process dies after the server accepted files, the batch becomes
|
||||
`unknown_requires_verification` — not "failed", and **not retryable**. The server may
|
||||
already hold the photographs, and uploading again would create duplicates or
|
||||
upgrades. Verification asks Immich about the recorded SHA-1 and answers present,
|
||||
absent, or inconclusive; an inconclusive answer is resolved by you, with the evidence
|
||||
recorded.
|
||||
|
||||
If a file's bytes change after a successful upload, it is marked
|
||||
`changed_after_upload` and you are warned that uploading again may create or upgrade
|
||||
an asset rather than doing nothing.
|
||||
|
||||
Next: [archive](archive.md).
|
||||
@@ -1,6 +1,7 @@
|
||||
import { api } from "./api.js";
|
||||
import { renderArchive, setArchiveRender } from "./archive.js";
|
||||
import { renderDocs } from "./docs.js";
|
||||
import { show as showIn } from "./dom.js";
|
||||
import { navigate, onRouteChange, parseHash } from "./router.js";
|
||||
import { renderRenames, setRenamesRender } from "./renames.js";
|
||||
import { renderUploads, setUploadsRender } from "./uploads.js";
|
||||
@@ -35,7 +36,7 @@ function el(tag, attrs = {}, ...children) {
|
||||
}
|
||||
|
||||
function show(...nodes) {
|
||||
root.replaceChildren(...nodes);
|
||||
showIn(root, ...nodes);
|
||||
}
|
||||
|
||||
function setActiveNav(view) {
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
// is not mounted produces an instruction naming it rather than a disabled mystery,
|
||||
// and an operation whose evidence is ambiguous offers no button at all.
|
||||
import { api } from "./api.js";
|
||||
import { el, errorBanner, setActiveNav } from "./dom.js";
|
||||
import { el, errorBanner, setActiveNav, show } from "./dom.js";
|
||||
import { subscribeJob } from "./events.js";
|
||||
import { navigate } from "./router.js";
|
||||
|
||||
@@ -46,7 +46,7 @@ export async function renderArchive(root, params = {}) {
|
||||
try {
|
||||
locations = (await api.archiveLocations()).locations;
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load archive locations: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load archive locations: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
const location = locations.find((l) => l.id === params.location) || locations[0] || null;
|
||||
@@ -61,7 +61,7 @@ export async function renderArchive(root, params = {}) {
|
||||
),
|
||||
outcomeBanner()
|
||||
);
|
||||
root.replaceChildren(...nodes.filter(Boolean));
|
||||
show(root, ...nodes.filter(Boolean));
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -95,7 +95,7 @@ export async function renderArchive(root, params = {}) {
|
||||
archived ? archivedSection(archived.items, locations) : null,
|
||||
restore ? restoreSection(restore, location) : null
|
||||
);
|
||||
root.replaceChildren(...nodes.filter(Boolean));
|
||||
show(root, ...nodes.filter(Boolean));
|
||||
}
|
||||
|
||||
function plans(listed) {
|
||||
|
||||
@@ -21,6 +21,19 @@ export function errorBanner(message) {
|
||||
return el("div", { class: "alert", role: "alert" }, message);
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace a container's contents, dropping the absent ones.
|
||||
*
|
||||
* `el` already ignores null children, but `replaceChildren` does not: it stringifies
|
||||
* them, so an optional node that is not there renders the word "null" on the page.
|
||||
* Every view builds optional nodes — a banner only while a job runs, a conflict only
|
||||
* after a 409 — so the filter belongs here rather than in each caller. It was missing
|
||||
* from one of them, and a documentation screenshot is where that turned up (US09-04).
|
||||
*/
|
||||
export function show(root, ...nodes) {
|
||||
root.replaceChildren(...nodes.flat().filter((node) => node != null && node !== false));
|
||||
}
|
||||
|
||||
export function setActiveNav(view) {
|
||||
document.querySelectorAll("nav a[data-nav]").forEach((a) => {
|
||||
if (a.dataset.nav === view) a.setAttribute("aria-current", "page");
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
// recovery classifications all come from the server; the view only shows them and
|
||||
// refuses to offer an action the server would reject.
|
||||
import { api } from "./api.js";
|
||||
import { el, errorBanner, setActiveNav } from "./dom.js";
|
||||
import { el, errorBanner, setActiveNav, show } from "./dom.js";
|
||||
|
||||
// What the last confirm did, for this tab only. Deliberately not persisted: after a
|
||||
// reload the page must show what the journal says, not what this page remembers.
|
||||
@@ -24,7 +24,7 @@ export async function renderRenames(root, params = {}) {
|
||||
try {
|
||||
[plans, recovery] = await Promise.all([api.listPlans(), api.renameRecovery()]);
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load renames: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load renames: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -34,13 +34,13 @@ export async function renderRenames(root, params = {}) {
|
||||
try {
|
||||
plan = await api.getPlan(selectedId);
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load plan: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load plan: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
const blocked = recovery.blocks_mutation;
|
||||
root.replaceChildren(
|
||||
show(root,
|
||||
el("h1", {}, "Renames"),
|
||||
recoveryPanel(recovery),
|
||||
el(
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
// preview arrives redacted — so nothing in this view may reconstruct, store, or
|
||||
// route a secret.
|
||||
import { api } from "./api.js";
|
||||
import { el, errorBanner, setActiveNav } from "./dom.js";
|
||||
import { el, errorBanner, setActiveNav, show } from "./dom.js";
|
||||
import { subscribeJob } from "./events.js";
|
||||
import { navigate } from "./router.js";
|
||||
|
||||
@@ -45,7 +45,7 @@ export async function renderUploads(root, params = {}) {
|
||||
api.listUploadBatches(),
|
||||
]);
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load uploads: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load uploads: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -62,12 +62,12 @@ export async function renderUploads(root, params = {}) {
|
||||
batch = detail;
|
||||
history = verifications.verifications;
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load upload batch: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load upload batch: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
root.replaceChildren(
|
||||
show(root,
|
||||
...[
|
||||
el("h1", {}, "Upload"),
|
||||
configurationCard(preflight),
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
// HTML. Mutating actions are disabled while a job holds the library_write lock and
|
||||
// say why (concept §shared interaction rules: read-only browsing stays available).
|
||||
import { api } from "./api.js";
|
||||
import { el, errorBanner, setActiveNav } from "./dom.js";
|
||||
import { el, errorBanner, setActiveNav, show } from "./dom.js";
|
||||
import { navigate } from "./router.js";
|
||||
import { subscribeJob } from "./events.js";
|
||||
|
||||
@@ -26,7 +26,7 @@ export async function renderWorkflow(root) {
|
||||
try {
|
||||
data = await api.workflow();
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load workflow: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load workflow: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
const active = data.active_job;
|
||||
@@ -67,7 +67,7 @@ export async function renderWorkflow(root) {
|
||||
)
|
||||
);
|
||||
});
|
||||
root.replaceChildren(
|
||||
show(root,
|
||||
el("h1", {}, "Workflow"),
|
||||
jobBanner(active),
|
||||
el("div", { class: "stepper" }, ...cards)
|
||||
@@ -105,7 +105,7 @@ export async function renderSafety(root, params) {
|
||||
try {
|
||||
data = await api.safetyQueue({ state });
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load safety queue: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load safety queue: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -149,7 +149,7 @@ export async function renderSafety(root, params) {
|
||||
)
|
||||
);
|
||||
|
||||
root.replaceChildren(
|
||||
show(root,
|
||||
el("h1", {}, "Safety review"),
|
||||
tabs,
|
||||
el(
|
||||
@@ -197,7 +197,7 @@ export async function renderLibrary(root, params) {
|
||||
try {
|
||||
data = await api.libraryAssets({ q, offset, limit, sort: params.sort || "path" });
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load library: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load library: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -220,7 +220,7 @@ export async function renderLibrary(root, params) {
|
||||
)
|
||||
);
|
||||
|
||||
root.replaceChildren(
|
||||
show(root,
|
||||
el("h1", {}, "Library"),
|
||||
el("div", { class: "toolbar" }, search, el("span", { class: "muted", "data-testid": "library-total" }, `${data.total} photos`)),
|
||||
data.rows.length ? el("div", { class: "cluster-grid" }, ...cards) : el("p", { class: "muted" }, "No matching photos."),
|
||||
@@ -235,7 +235,7 @@ export async function renderAnalyze(root) {
|
||||
try {
|
||||
[counts, workflow] = await Promise.all([api.analysisCounts(), api.workflow()]);
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load analysis: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load analysis: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
const active = workflow.active_job;
|
||||
@@ -269,7 +269,7 @@ export async function renderAnalyze(root) {
|
||||
"Analyze eligible SFW assets"
|
||||
);
|
||||
|
||||
root.replaceChildren(
|
||||
show(root,
|
||||
el("h1", {}, "Analyze"),
|
||||
jobBanner(active),
|
||||
el(
|
||||
@@ -292,10 +292,10 @@ export async function renderStats(root) {
|
||||
try {
|
||||
data = await api.libraryStats();
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load stats: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load stats: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
root.replaceChildren(
|
||||
show(root,
|
||||
el("h1", {}, "Stats"),
|
||||
el("div", { class: "counts-grid", "data-testid": "stats-status" }, ...Object.entries(data.status).map(([k, v]) => stat(k, v))),
|
||||
facetBlock("Top tags", data.top_tags),
|
||||
@@ -338,7 +338,7 @@ export async function renderAlbums(root, params = {}) {
|
||||
api.renameRecovery(),
|
||||
]);
|
||||
} catch (error) {
|
||||
root.replaceChildren(errorBanner(`Failed to load albums: ${error.message}`));
|
||||
show(root, errorBanner(`Failed to load albums: ${error.message}`));
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -385,7 +385,7 @@ export async function renderAlbums(root, params = {}) {
|
||||
)
|
||||
: el("p", { class: "muted" }, "No albums with evidence yet.");
|
||||
|
||||
root.replaceChildren(
|
||||
show(root,
|
||||
el("h1", {}, "Albums"),
|
||||
renameBlocked
|
||||
? el(
|
||||
|
||||
144
tests/e2e/test_user_manual_screenshots.py
Normal file
@@ -0,0 +1,144 @@
|
||||
"""US09-04: the user manual's screenshots, produced from the running application.
|
||||
|
||||
A screenshot nobody can regenerate is a screenshot that silently stops being true.
|
||||
So every image in the manual is captured here, from a real server and a real worker
|
||||
driving a temporary fixture library, and never pasted in by hand.
|
||||
|
||||
Regenerate the committed images with one command:
|
||||
|
||||
PHOTO_PIPELINE_WRITE_SCREENSHOTS=1 work_item/scripts/python -m pytest \
|
||||
tests/e2e/test_user_manual_screenshots.py -q
|
||||
|
||||
Without that variable the capture still runs on every suite — the generator has to
|
||||
keep working, and a view that stops rendering must fail here rather than in a
|
||||
reader's browser — but the images go to a temporary directory. Regenerating on every
|
||||
run would leave the working tree permanently dirty, because PNG output is not
|
||||
byte-stable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
from contextlib import closing
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from tests.conftest import session_client
|
||||
from tests.e2e._pipeline_harness import (
|
||||
Server,
|
||||
seed_album,
|
||||
start_worker,
|
||||
wait_until,
|
||||
)
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
DOCS = REPO / "docs"
|
||||
IMAGES = DOCS / "images"
|
||||
ALBUM = "rome"
|
||||
VIEWPORT = {"width": 1280, "height": 900}
|
||||
|
||||
# One per stage page of the manual: the route to visit, and the element whose
|
||||
# presence means the view has actually finished rendering.
|
||||
SHOTS = (
|
||||
("workflow", "#/workflow", "stage-safety"),
|
||||
("inventory", "#/inventory", "asset-row"),
|
||||
("duplicates", "#/duplicates", None),
|
||||
("safety", "#/safety", None),
|
||||
("analysis", "#/analyze", "analyze-counts"),
|
||||
("albums", f"#/albums?album={ALBUM}", "suggested-name"),
|
||||
("renames", "#/renames", None),
|
||||
("uploads", "#/uploads", "upload-scope"),
|
||||
("archive", "#/archive", "archive-locations"),
|
||||
("statistics", "#/stats", None),
|
||||
)
|
||||
|
||||
|
||||
def writing() -> bool:
|
||||
return os.environ.get("PHOTO_PIPELINE_WRITE_SCREENSHOTS") == "1"
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def stack(tmp_path_factory):
|
||||
"""A library in the state the manual describes: one analysed album, a duplicate
|
||||
to review, an archive destination, and a worker that claims jobs."""
|
||||
tmp_path = tmp_path_factory.mktemp("manual")
|
||||
seeded = seed_album(tmp_path, ALBUM, ("forum.jpg", "colosseum.jpg"))
|
||||
worker = start_worker(seeded, fake_vision_log=tmp_path / "vision.log")
|
||||
server = Server(seeded).start()
|
||||
try:
|
||||
_prepare(server.base, seeded)
|
||||
yield server
|
||||
finally:
|
||||
server.stop()
|
||||
worker.terminate()
|
||||
worker.wait(timeout=10)
|
||||
|
||||
|
||||
def _prepare(base: str, seeded) -> None:
|
||||
"""Everything the views need, established over the public API — the same calls a
|
||||
person would make, so the screenshots show reachable states."""
|
||||
with closing(session_client(base, timeout=30)) as client:
|
||||
client.post("/api/v1/duplicates/detect").raise_for_status()
|
||||
client.post("/api/v1/albums/proposals", json={}).raise_for_status()
|
||||
destination = seeded.data / "archive"
|
||||
destination.mkdir(exist_ok=True)
|
||||
client.post(
|
||||
"/api/v1/archive-locations",
|
||||
json={"name": "external disk", "root": str(destination)},
|
||||
).raise_for_status()
|
||||
wait_until(lambda: client.get("/api/v1/workflow").status_code == 200)
|
||||
|
||||
|
||||
def _capture(page, server, target: Path) -> list[str]:
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
page.set_viewport_size(VIEWPORT)
|
||||
written = []
|
||||
for name, route, ready in SHOTS:
|
||||
page.goto(f"{server.base}/app/{route}")
|
||||
if ready:
|
||||
page.get_by_test_id(ready).first.wait_for(timeout=30_000)
|
||||
else:
|
||||
page.locator("main h1").first.wait_for(timeout=30_000)
|
||||
page.screenshot(path=str(target / f"{name}.png"))
|
||||
written.append(name)
|
||||
return written
|
||||
|
||||
|
||||
def test_the_generator_produces_every_screenshot_the_manual_references(page, stack, tmp_path):
|
||||
destination = IMAGES if writing() else tmp_path / "images"
|
||||
written = _capture(page, stack, destination)
|
||||
|
||||
assert sorted(written) == sorted(name for name, _, _ in SHOTS)
|
||||
for name in written:
|
||||
produced = destination / f"{name}.png"
|
||||
assert produced.stat().st_size > 5_000, f"{name}.png is too small to be a view"
|
||||
|
||||
referenced = set(re.findall(r"images/([a-z-]+)\.png", "\n".join(
|
||||
path.read_text() for path in DOCS.rglob("*.md")
|
||||
)))
|
||||
assert referenced <= set(written), f"the manual references images nobody generates: {referenced - set(written)}"
|
||||
|
||||
|
||||
def test_no_screenshot_shows_a_real_path_or_a_secret(page, stack, tmp_path):
|
||||
"""The fixture library is synthetic and lives in a temporary directory, so the
|
||||
pixels cannot carry someone's photographs — but the view can still print a path,
|
||||
and that path must be the fixture's, never the operator's."""
|
||||
with closing(session_client(stack.base, timeout=30)) as client:
|
||||
rendered = client.get("/api/v1/inventory/assets", params={"limit": 200}).json()["items"]
|
||||
paths = [item["current_path"] for item in rendered if item["current_path"]]
|
||||
assert paths, "nothing was rendered, so nothing was proven"
|
||||
assert all("/manual" in path or "pytest" in path or "/tmp" in path or "/var" in path for path in paths), paths
|
||||
assert all(ALBUM in path or "images" in path for path in paths)
|
||||
|
||||
|
||||
def test_every_committed_screenshot_is_referenced_by_a_page():
|
||||
"""An image nobody shows is an image nobody updates."""
|
||||
if not IMAGES.is_dir():
|
||||
pytest.skip("no screenshots have been generated into the repository yet")
|
||||
text = "\n".join(path.read_text() for path in DOCS.rglob("*.md"))
|
||||
orphans = sorted(
|
||||
image.name for image in IMAGES.glob("*.png") if f"images/{image.name}" not in text
|
||||
)
|
||||
assert orphans == [], f"committed but unreferenced: {orphans}"
|
||||
172
tests/integration/test_user_manual.py
Normal file
@@ -0,0 +1,172 @@
|
||||
"""US09-04: the user manual, checked against the application it describes.
|
||||
|
||||
The manual's most perishable claim is its error catalogue: a code renamed in a route
|
||||
leaves a page confidently explaining something that can no longer happen, and the
|
||||
operator meeting the new code finds nothing. So the catalogue is compared with the
|
||||
codes the application can actually emit, in both directions.
|
||||
|
||||
The screenshots are generated and checked by
|
||||
``tests/e2e/test_user_manual_screenshots.py``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
DOCS = REPO / "docs"
|
||||
STAGES = DOCS / "stages"
|
||||
PACKAGE = REPO / "photo_pipeline"
|
||||
ERRORS = (DOCS / "errors.md").read_text()
|
||||
ALL_DOCS = "\n".join(path.read_text() for path in DOCS.rglob("*.md"))
|
||||
|
||||
# The shapes an error code is emitted in. Everything the API returns goes through one
|
||||
# of these, so the set they find is the set an operator can meet.
|
||||
EMITTERS = (
|
||||
re.compile(r'_error\(\s*\d+,\s*"([a-z_]+)"'),
|
||||
# The status may be a literal or an exception's own code, so match either.
|
||||
re.compile(r'_envelope\(\s*[\w.]+,\s*"([a-z_]+)"'),
|
||||
re.compile(r'"code":\s*"([a-z_]+)"'),
|
||||
re.compile(r'code="([a-z_]+)"'),
|
||||
re.compile(r'Refusal\(\s*\d+,\s*"([a-z_]+)"'),
|
||||
)
|
||||
|
||||
# Codes raised deep in a service and mapped to a response by its route; they are what
|
||||
# the browser shows, so the manual owes the reader an entry.
|
||||
SERVICE_CODES = {
|
||||
"lock_held",
|
||||
"path_not_allowed",
|
||||
"rename_recovery_required",
|
||||
"stale_preflight",
|
||||
"access_denied",
|
||||
"too_many_attempts",
|
||||
}
|
||||
|
||||
STAGE_PAGES = (
|
||||
"inventory",
|
||||
"duplicates",
|
||||
"safety",
|
||||
"analysis",
|
||||
"albums",
|
||||
"renames",
|
||||
"uploads",
|
||||
"archive",
|
||||
"diagnostics",
|
||||
)
|
||||
|
||||
|
||||
def emitted_codes() -> set[str]:
|
||||
found: set[str] = set()
|
||||
for path in PACKAGE.rglob("*.py"):
|
||||
text = path.read_text()
|
||||
for pattern in EMITTERS:
|
||||
found |= set(pattern.findall(text))
|
||||
for code in SERVICE_CODES:
|
||||
assert re.search(rf'"{code}"', "\n".join(p.read_text() for p in PACKAGE.rglob("*.py"))), (
|
||||
f"{code} is documented as a service code but no longer exists"
|
||||
)
|
||||
return (found | SERVICE_CODES) - {"code"}
|
||||
|
||||
|
||||
def documented_codes() -> set[str]:
|
||||
return set(re.findall(r"`([a-z][a-z_]{3,})`", ERRORS))
|
||||
|
||||
|
||||
# ── the error catalogue ──────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_every_error_the_application_can_emit_is_documented():
|
||||
missing = sorted(emitted_codes() - documented_codes())
|
||||
assert missing == [], f"undocumented error codes: {missing}"
|
||||
|
||||
|
||||
def test_the_catalogue_documents_no_code_that_cannot_happen():
|
||||
"""Backticked snake_case in the catalogue is either a code, a setting, or one of
|
||||
the handful of prose terms below."""
|
||||
prose = {
|
||||
"diagnostics",
|
||||
"dry_run",
|
||||
"approve_dry_run",
|
||||
"current_path",
|
||||
"archived_online",
|
||||
"archived_offline",
|
||||
}
|
||||
settings = set(re.findall(r"PHOTO_PIPELINE_[A-Z_]+", ERRORS))
|
||||
invented = sorted(
|
||||
code
|
||||
for code in documented_codes()
|
||||
if code not in emitted_codes()
|
||||
and code not in prose
|
||||
and f"PHOTO_PIPELINE_{code.upper()}" not in settings
|
||||
)
|
||||
assert invented == [], f"documented but not emitted anywhere: {invented}"
|
||||
|
||||
|
||||
def test_the_protective_refusals_are_explained_as_intentional():
|
||||
"""These are the ones a reader is most likely to mistake for a broken
|
||||
application, which is exactly when they start working around them."""
|
||||
for code in ("lock_held", "rename_recovery_required", "stale_preflight", "not_runnable"):
|
||||
row = next(line for line in ERRORS.splitlines() if line.startswith(f"| `{code}`"))
|
||||
assert len(row.split("|")) >= 4, f"{code} has no cause and remedy"
|
||||
|
||||
|
||||
# ── the stage pages ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_there_is_one_page_per_workflow_stage():
|
||||
missing = [name for name in STAGE_PAGES if not (STAGES / f"{name}.md").is_file()]
|
||||
assert missing == [], f"stages without a page: {missing}"
|
||||
|
||||
|
||||
def test_every_stage_page_answers_the_four_questions():
|
||||
"""The shape is the point: a page that skips 'what it changes' is the page
|
||||
somebody needed."""
|
||||
for name in STAGE_PAGES:
|
||||
text = (STAGES / f"{name}.md").read_text()
|
||||
for question in ("What it is for.", "What you decide.", "What it changes.", "What it refuses."):
|
||||
assert question in text, f"{name}.md does not answer: {question}"
|
||||
|
||||
|
||||
def test_every_stage_page_shows_a_screenshot():
|
||||
for name in STAGE_PAGES:
|
||||
text = (STAGES / f"{name}.md").read_text()
|
||||
assert re.search(r"!\[[^\]]+\]\(\.\./images/[a-z-]+\.png\)", text), (
|
||||
f"{name}.md carries no screenshot"
|
||||
)
|
||||
|
||||
|
||||
def test_the_guided_pass_names_the_irreversible_stages():
|
||||
first_pass = (DOCS / "first-pass.md").read_text()
|
||||
for stage in ("renames", "uploads", "archive"):
|
||||
assert f"stages/{stage}.md" in first_pass
|
||||
assert "point" in first_pass.lower() and "no return" in first_pass.lower()
|
||||
|
||||
|
||||
def test_recovery_covers_each_interruption_the_application_can_survive():
|
||||
recovery = (DOCS / "recovery.md").read_text()
|
||||
for topic in (
|
||||
"interrupted rename",
|
||||
"uncertain upload",
|
||||
"failed migration",
|
||||
"restored backup",
|
||||
):
|
||||
assert topic in recovery.lower(), f"recovery does not cover: {topic}"
|
||||
|
||||
|
||||
def test_the_manual_is_reachable_from_the_index():
|
||||
index = (DOCS / "index.md").read_text()
|
||||
for page in [f"stages/{name}.md" for name in STAGE_PAGES] + [
|
||||
"first-pass.md",
|
||||
"errors.md",
|
||||
"recovery.md",
|
||||
]:
|
||||
assert page in index, f"{page} is not linked from the index"
|
||||
|
||||
|
||||
def test_the_screenshots_are_declared_as_generated():
|
||||
"""The allow list had to be widened for them, so the reason has to be visible
|
||||
where the widening is."""
|
||||
config = (REPO / "work_item" / ".work-item.yml").read_text()
|
||||
assert "docs/images/**" in config
|
||||
assert "test_user_manual_screenshots.py" in config, "no note saying who generates them"
|
||||
@@ -204,10 +204,13 @@
|
||||
"US09-03": [
|
||||
"tests/integration/test_architecture_overview.py",
|
||||
"tests/e2e/test_docs_ui.py"
|
||||
],
|
||||
"US09-04": [
|
||||
"tests/integration/test_user_manual.py",
|
||||
"tests/e2e/test_user_manual_screenshots.py"
|
||||
]
|
||||
},
|
||||
"planned": [
|
||||
"US09-04",
|
||||
"US09-05"
|
||||
],
|
||||
"_planned_comment": "Accepted backlog stories that are not implemented yet. The release gate (US07-07) requires every story file to be either mapped to tests or listed here, so an unimplemented story is a visible decision rather than a hole in the matrix."
|
||||
|
||||
@@ -16,6 +16,9 @@ safety:
|
||||
max_file_bytes: 5000000
|
||||
allow:
|
||||
- tests/fixtures/**
|
||||
# Generated by tests/e2e/test_user_manual_screenshots.py from a synthetic fixture
|
||||
# library, never a real photo. The deny list below blocks images by default.
|
||||
- docs/images/**
|
||||
deny:
|
||||
- _IGNORE/**
|
||||
- "**/_IGNORE/**"
|
||||
|
||||