1924 lines
93 KiB
Markdown
1924 lines
93 KiB
Markdown
# Photo Library Pipeline — Integrated App Concept
|
|
|
|
Unify duplicate detection, safety review, content analysis, album naming, guarded
|
|
renaming, and Immich upload into one local web application.
|
|
|
|
> **North star:** every photo moves through one visible, resume-safe workflow.
|
|
> Destructive or irreversible actions always require a preview and explicit approval.
|
|
> A file path is mutable metadata, never the identity of a photo.
|
|
|
|
---
|
|
|
|
## 1. Product goals
|
|
|
|
The app should:
|
|
|
|
- guide the user through the stages in the correct order;
|
|
- preserve the existing NSFW review and Photo Analyzer UI/UX;
|
|
- prevent NSFW photos from reaching the cloud vision provider while still allowing
|
|
reviewed, EXIF-tagged NSFW photos to be uploaded to Immich;
|
|
- detect duplicates before spending API quota or changing metadata;
|
|
- keep database records valid when files or folders are renamed;
|
|
- create a verified DB↔EXIF consistency checkpoint after every metadata-producing
|
|
stage, always merging with and preserving existing metadata;
|
|
- upload each approved album through `immich-go` with a visible audit trail;
|
|
- optionally archive successfully uploaded albums out of active storage while retaining
|
|
their complete identity, metadata, thumbnails, and deduplication evidence;
|
|
- remain safe to stop and resume at every stage;
|
|
- exclude every `_IGNORE/` directory from discovery, counts, previews, and actions.
|
|
|
|
The application remains local and binds only to `127.0.0.1` by default. It preserves
|
|
the existing visual design while replacing the ad-hoc stdlib servers with a typed,
|
|
versioned application API.
|
|
|
|
---
|
|
|
|
## 2. Canonical workflow
|
|
|
|
```text
|
|
0. Inventory & duplicates
|
|
↓ approved canonical set
|
|
1. Safety score & human review
|
|
├── SFW assets → content analysis
|
|
└── NSFW assets → skip analysis, retain safety EXIF
|
|
2. Content analysis & analysis EXIF for SFW assets
|
|
↓ reviewed, metadata-ready assets
|
|
3. Album suggestions & guarded rename
|
|
↓ approved final folder layout
|
|
4. Immich upload
|
|
↓ verified managed assets
|
|
5. Archive
|
|
↓ removed from active storage, retained in the permanent ledger
|
|
```
|
|
|
|
Stages are sequential **gates**, not merely navigation tabs. Later stages may be
|
|
viewed at any time, but actions are disabled until their prerequisites are met.
|
|
|
|
### Gate rules
|
|
|
|
| Stage | Ready when | Blocks next stage when |
|
|
|---|---|---|
|
|
| Inventory | discovery and hashes are complete | unclassified duplicate clusters or missing files remain |
|
|
| Safety | every canonical asset is `sfw`, `nsfw`, or explicitly deferred | undecided assets remain unless the user accepts a partial run |
|
|
| Analysis | every eligible SFW asset has DB data and verified analysis EXIF | pending/error or EXIF-divergent records remain unless explicitly deferred |
|
|
| Albums | every selected album proposal is applied or dismissed | rename plan is pending, partially applied, or inconsistent |
|
|
| Upload | final file hashes are current and credentials validate | metadata changed after upload preparation |
|
|
| Archive | selected album has a verified upload and validated archive destination | upload is uncertain, files changed, archive target unavailable, or transfer verification failed |
|
|
|
|
The app can support a **partial pipeline** for selected folders, but it must show
|
|
that the library is only partially complete.
|
|
|
|
---
|
|
|
|
## 3. Core architecture
|
|
|
|
### Real application, not CLI scripts with attached pages
|
|
|
|
The target is a cohesive Python application with a documented API, database-backed
|
|
services, persistent jobs, and one frontend. The current CLIs are the primary donor
|
|
implementations for the new services: reuse their proven code, algorithms, prompts,
|
|
format handling, subprocess adapters, and edge-case behavior wherever compatible with
|
|
the target architecture. They are not the final application boundary and will be
|
|
archived only after their donated behavior has automated parity coverage.
|
|
|
|
```text
|
|
Browser
|
|
│ JSON API + Server-Sent Events
|
|
▼
|
|
FastAPI application
|
|
├── inventory / duplicate service
|
|
├── thumbnail service
|
|
├── safety service
|
|
├── analysis service
|
|
├── album and rename service
|
|
├── upload service
|
|
└── job coordinator
|
|
│ claims durable jobs
|
|
▼
|
|
Python worker process ── exiftool / local model / vision API / immich-go
|
|
│
|
|
▼
|
|
SQLite database in WAL mode + managed application cache
|
|
```
|
|
|
|
Recommended backend foundation:
|
|
|
|
- **FastAPI** for typed HTTP APIs, validation, OpenAPI documentation, and streaming;
|
|
- **Pydantic** request/response models so browser input never reaches services raw;
|
|
- **SQLAlchemy 2.x** for explicit transactions and repository boundaries;
|
|
- **Alembic** for versioned, testable database migrations;
|
|
- **SQLite WAL mode** initially, with short write transactions and one job writer;
|
|
- a separate Python **worker process** that claims durable jobs from SQLite;
|
|
- Server-Sent Events for live job/activity updates, with polling as fallback;
|
|
- structured JSON logging carrying `job_id`, `asset_id`, and `operation_id`.
|
|
|
|
SQLite is appropriate for a single-user local application. Repository and service
|
|
interfaces must avoid SQLite-specific business logic so PostgreSQL remains a possible
|
|
future deployment option, not an immediate dependency.
|
|
|
|
### Domain services
|
|
|
|
Current code should be progressively extracted into importable services:
|
|
|
|
- `InventoryService` — discovery, path reconciliation, hashes, duplicate candidates;
|
|
- `ThumbnailService` — safe, cached previews for every supported image format;
|
|
- `DuplicateService` — clusters, recommendations, decisions, canonical membership;
|
|
- `SafetyService` — scoring, review decisions, EXIF safety keywords;
|
|
- `AnalysisService` — vision requests, result validation, DB and EXIF writes;
|
|
- `AlbumService` — folder summaries, AI proposals, naming policy;
|
|
- `RenameService` — plans, validation, journaled apply/recovery;
|
|
- `UploadService` — preflight, batches, `immich-go`, verification;
|
|
- `ArchiveService` — archive plans, transfer verification, offline state, restore;
|
|
- `JobService` — durable job lifecycle, locks, progress, cancellation, events.
|
|
|
|
The CLI entry points may call these same services for backwards compatibility. The
|
|
web API never shells out to `photo_analyzer.py` or `nsfw_tag.py`; only genuine external
|
|
tools such as `exiftool` and `immich-go` remain subprocesses.
|
|
|
|
### Donor-first CLI migration and archival
|
|
|
|
This is an extraction and consolidation project, not a greenfield rewrite. Before
|
|
implementing a replacement service, inspect the corresponding CLI implementation and
|
|
classify each relevant function or behavior as **reuse unchanged**, **extract with a
|
|
thin adapter**, **refactor while preserving behavior**, or **replace with documented
|
|
reason**. Default to extraction; replacement requires a concrete incompatibility such
|
|
as unsafe concurrency, path-keyed identity, or inability to support durable jobs.
|
|
|
|
Create and maintain a donor ledger mapping legacy modules/functions to their target
|
|
service location, characterization tests, migration status, intentional behavior
|
|
changes, and final parity test. Important donor material includes:
|
|
|
|
- file discovery, extension handling, folder grouping, and path exclusion policy;
|
|
- exact/perceptual hashing, duplicate clustering, reconciliation, and resume behavior;
|
|
- image decode, orientation, resize, HEIC conversion, and thumbnail behavior;
|
|
- vision prompts, request construction, JSON validation, retries, and rate limiting;
|
|
- NSFW inference, scoring thresholds, review decisions, and model integration;
|
|
- EXIF field projection, preservation rules, subprocess invocation, and read-back;
|
|
- SQLite schema knowledge, migrations, FTS behavior, status transitions, and stats;
|
|
- existing frontend design tokens, layouts, interactions, keyboard behavior, and
|
|
useful browser-side components;
|
|
- operational logging, cancellation, error reporting, configuration, and CLI flags.
|
|
|
|
For each donor area, first add characterization tests against the current CLI code and
|
|
record representative outputs using the deterministic fixture corpus. Extract the
|
|
smallest coherent implementation into the new service layer, then run the same fixture
|
|
expectations against both implementations. Preserve known behavior unless the concept
|
|
explicitly changes it; every intentional difference needs a rationale and a new
|
|
acceptance test. Avoid copying code into parallel implementations—after extraction,
|
|
the legacy entry point should temporarily import the shared service where practical.
|
|
|
|
Archive a CLI script only when all of its in-scope donor rows are resolved, its
|
|
replacement passes unit, characterization, integration, and end-to-end parity tests,
|
|
database migration/reconciliation is verified, and no supported workflow still
|
|
depends on the old entry point. Archival means moving the original sources and their
|
|
documentation into a read-only `legacy_cli_archive/` area with their final dependency
|
|
lock, schema notes, sample configuration with secrets removed, donor ledger, and a
|
|
recorded final version/checksum. Archived scripts are reference material and rollback
|
|
evidence; production code must not import or execute them.
|
|
|
|
### API boundaries
|
|
|
|
All mutations are commands against stable IDs, never raw filesystem paths. Example
|
|
API groups:
|
|
|
|
| Method | Endpoint | Purpose |
|
|
|---|---|---|
|
|
| GET | `/api/v1/workflow` | stage readiness, blockers, totals, active job |
|
|
| POST | `/api/v1/inventory/scan` | enqueue discovery/reconciliation job |
|
|
| GET | `/api/v1/duplicates/clusters` | paged clusters by confidence/state |
|
|
| GET | `/api/v1/duplicates/clusters/{id}` | members, thumbnails, evidence |
|
|
| POST | `/api/v1/duplicates/clusters/{id}/decision` | canonical/variants/not-duplicate/defer |
|
|
| GET | `/api/v1/assets/{id}` | stable asset record and stage history |
|
|
| GET | `/api/v1/assets/{id}/thumbnail?size=512` | cached, oriented preview |
|
|
| POST | `/api/v1/safety/jobs` | enqueue scoring for eligible assets |
|
|
| POST | `/api/v1/safety/decisions` | persist reviewed decisions |
|
|
| POST | `/api/v1/analysis/jobs` | enqueue content analysis |
|
|
| POST | `/api/v1/albums/proposals` | generate album suggestions |
|
|
| POST | `/api/v1/rename-plans/{id}/apply` | apply a validated plan |
|
|
| POST | `/api/v1/upload-batches/{id}/start` | start approved Immich upload |
|
|
| POST | `/api/v1/archive-plans` | create and validate an archive plan |
|
|
| POST | `/api/v1/archive-plans/{id}/apply` | transfer/remove active folder after verification |
|
|
| POST | `/api/v1/assets/{id}/restore` | restore an archived asset/album to active storage |
|
|
| GET | `/api/v1/jobs/{id}` | durable status and progress |
|
|
| POST | `/api/v1/jobs/{id}/cancel` | cooperative cancellation request |
|
|
| GET | `/api/v1/events` | SSE activity stream |
|
|
|
|
API responses use explicit enums and error codes. Conflicts such as stale hashes,
|
|
running jobs, or changed rename plans return `409 Conflict`; validation errors return
|
|
`422`; unknown IDs return `404`. OpenAPI becomes the contract between backend and UI.
|
|
|
|
### Persistent job model
|
|
|
|
Long operations are database jobs, not in-memory threads:
|
|
|
|
```text
|
|
queued → running → cancelling → cancelled
|
|
↘ succeeded
|
|
↘ failed → retry_queued
|
|
```
|
|
|
|
The worker claims a queued job atomically, records a heartbeat, persists incremental
|
|
progress and events, and checks cancellation between items. On application restart,
|
|
stale `running` jobs are detected from their heartbeat and offered for safe resume.
|
|
Mutating job types declare locks such as `library_write`, `rename`, or `upload`, which
|
|
the coordinator uses to prevent unsafe concurrency.
|
|
|
|
### One durable database
|
|
|
|
Extend the current SQLite database rather than introduce separate score and state
|
|
files. CSV export may remain available, but it is not authoritative.
|
|
|
|
Most importantly, replace path identity with a stable internal asset identifier.
|
|
|
|
```sql
|
|
CREATE TABLE assets (
|
|
id TEXT PRIMARY KEY, -- generated UUID, never changes
|
|
current_path TEXT UNIQUE, -- active-library path; NULL when offline
|
|
original_path TEXT NOT NULL,
|
|
current_sha256 TEXT, -- current file bytes; refreshed after EXIF
|
|
pixel_sha256 TEXT, -- normalized decoded pixels; ignores EXIF
|
|
phash TEXT, -- resilient to resize/recompression
|
|
hash_version INTEGER NOT NULL, -- decoder/normalization algorithm version
|
|
byte_size INTEGER,
|
|
discovered_at TEXT NOT NULL,
|
|
missing_at TEXT,
|
|
|
|
canonical_asset_id TEXT REFERENCES assets(id),
|
|
safety_score REAL,
|
|
availability_state TEXT NOT NULL DEFAULT 'active',
|
|
archive_location_id TEXT,
|
|
state_version INTEGER NOT NULL DEFAULT 1
|
|
);
|
|
|
|
CREATE TABLE asset_paths (
|
|
asset_id TEXT NOT NULL REFERENCES assets(id),
|
|
path TEXT NOT NULL,
|
|
valid_from TEXT NOT NULL,
|
|
valid_until TEXT,
|
|
reason TEXT,
|
|
PRIMARY KEY (asset_id, path, valid_from)
|
|
);
|
|
|
|
CREATE TABLE asset_stage_states (
|
|
asset_id TEXT NOT NULL REFERENCES assets(id),
|
|
stage TEXT NOT NULL,
|
|
state TEXT NOT NULL,
|
|
substate TEXT,
|
|
input_version TEXT,
|
|
attempt_count INTEGER NOT NULL DEFAULT 0,
|
|
active_attempt_id TEXT,
|
|
blocker_code TEXT,
|
|
error_code TEXT,
|
|
error_message TEXT,
|
|
updated_at TEXT NOT NULL,
|
|
completed_at TEXT,
|
|
version INTEGER NOT NULL DEFAULT 1,
|
|
PRIMARY KEY (asset_id, stage)
|
|
);
|
|
|
|
CREATE TABLE asset_stage_attempts (
|
|
id TEXT PRIMARY KEY,
|
|
asset_id TEXT NOT NULL REFERENCES assets(id),
|
|
stage TEXT NOT NULL,
|
|
job_id TEXT NOT NULL REFERENCES jobs(id),
|
|
attempt_number INTEGER NOT NULL,
|
|
input_version TEXT NOT NULL,
|
|
started_at TEXT NOT NULL,
|
|
heartbeat_at TEXT,
|
|
finished_at TEXT,
|
|
outcome TEXT,
|
|
last_substate TEXT,
|
|
error_code TEXT,
|
|
error_message TEXT,
|
|
worker_id TEXT,
|
|
fencing_token INTEGER NOT NULL,
|
|
UNIQUE (asset_id, stage, attempt_number)
|
|
);
|
|
```
|
|
|
|
Analysis fields can remain in `photos` during migration, but `photos` should gain an
|
|
`asset_id` foreign key and eventually use it as its stable relationship. All FTS,
|
|
EXIF, duplicate, rename, and upload records refer to `asset_id`, not path.
|
|
|
|
### Per-asset workflow state
|
|
|
|
Every asset has one durable state row per stage. This is the authoritative answer to
|
|
“what happened to this picture?” The broad workflow totals are derived from these rows,
|
|
not maintained independently.
|
|
|
|
Stages include:
|
|
|
|
```text
|
|
inventory → duplicate_review → safety → analysis → album → upload → archive
|
|
```
|
|
|
|
The common state vocabulary is:
|
|
|
|
| State | Meaning |
|
|
|---|---|
|
|
| `blocked` | prerequisite is incomplete; `blocker_code` explains why |
|
|
| `ready` | all prerequisites satisfied |
|
|
| `queued` | assigned to a durable job but not started |
|
|
| `running` | worker owns a valid lease and attempt |
|
|
| `complete` | stage output and required verification are durable |
|
|
| `deferred` | user intentionally postponed the stage |
|
|
| `skipped` | stage does not apply, with a recorded reason |
|
|
| `failed` | attempt ended with a retryable or terminal error |
|
|
| `cancelled` | user/system cancelled before completion |
|
|
| `interrupted` | worker disappeared or application stopped mid-attempt |
|
|
| `stale` | previously complete output no longer matches current inputs |
|
|
|
|
`substate` identifies the precise point within a running/interrupted stage, for example:
|
|
|
|
```text
|
|
safety: decoding | scoring | awaiting_review | writing_exif | verifying_exif
|
|
analysis: preparing | calling_provider | validating | storing_result |
|
|
writing_exif | verifying_exif
|
|
album: summarizing | generating_name | awaiting_approval | renaming | verifying_paths
|
|
upload: preflight | hashing | uploading | verifying_remote
|
|
archive: preflight | transferring | verifying_archive | removing_active |
|
|
verifying_absence | complete_offline
|
|
```
|
|
|
|
An interruption is not guessed from a browser disconnect. A stage becomes
|
|
`interrupted` only when its active attempt was `running`, the worker heartbeat/lease
|
|
expired or the process shut down uncleanly, and no valid newer attempt owns it. The
|
|
recovery service records the interrupted attempt, inspects its last substate and real
|
|
file/external state, then chooses one of:
|
|
|
|
- resume safely from a verified checkpoint;
|
|
- restart the stage idempotently;
|
|
- mark `unknown_requires_verification` and request user action.
|
|
|
|
The immutable `asset_stage_attempts` table preserves every start, retry, interruption,
|
|
cancellation, error, and successful completion. The current state row is a projection
|
|
for fast UI queries; history is never overwritten.
|
|
|
|
### Dependency and invalidation rules
|
|
|
|
Stage completion is versioned against its inputs. Examples:
|
|
|
|
- a changed file invalidates inventory hashes and may make duplicate review stale;
|
|
- changing the duplicate canonical can skip later stages for one asset and ready them
|
|
for another;
|
|
- changing SFW→NSFW makes analysis `skipped` and invalidates any in-flight result, but
|
|
does not invalidate upload eligibility after safety EXIF verification;
|
|
- changing NSFW→SFW makes analysis `ready` and upload `blocked` until analysis completes;
|
|
- external EXIF changes make the relevant EXIF projection `stale` or `divergent`;
|
|
- a rename changes paths but does not invalidate content analysis or safety state;
|
|
- any byte-changing EXIF checkpoint makes upload `stale` until its exact hashes are
|
|
refreshed.
|
|
- archiving changes availability and path state but does not invalidate duplicate,
|
|
safety, analysis, EXIF, or upload history;
|
|
- restoring an archived asset creates a new active path occurrence and requires a
|
|
reconciliation scan before mutation.
|
|
|
|
State transitions use optimistic `version` checks and are committed with their audit
|
|
event. Workers may only complete the attempt they own with the current fencing token.
|
|
There is no generic “set status” endpoint; services expose valid domain transitions.
|
|
|
|
### Additional tables
|
|
|
|
- `duplicate_clusters` / `duplicate_members` — reviewable cluster decisions.
|
|
- `safety_reviews` — score, final decision, reviewer, timestamp, prior decision.
|
|
- `analysis_runs` — model/config/prompt version, token usage, status, timestamps.
|
|
- `album_proposals` — source folder, proposed name, rationale, confidence, status.
|
|
- `rename_plans` / `rename_operations` — complete preview and crash-safe journal.
|
|
- `upload_batches` / `upload_items` — command, album, verified content checksums,
|
|
result, log.
|
|
- `archive_locations` / `archive_plans` / `archive_operations` — target configuration,
|
|
manifests, journaled transfers, verification, removal, and restore history.
|
|
- `jobs` / `job_events` — shared progress and activity history across all stages.
|
|
- `thumbnails` — cache key, dimensions, format, path, generation state, error.
|
|
|
|
### Database ownership and transactions
|
|
|
|
- API routes call application services; they do not contain SQL.
|
|
- Services use repositories; repositories are the only database access layer.
|
|
- Every state change and corresponding audit event is written in one transaction.
|
|
- Filesystem operations use persisted journals because they cannot participate in a
|
|
database transaction.
|
|
- SQLite foreign keys are enabled, WAL mode is required, and migrations run once at
|
|
startup under an application lock.
|
|
- Database backups use SQLite's online backup API rather than copying a live DB file.
|
|
- Secrets live in environment/keychain-backed configuration, never database events.
|
|
|
|
### EXIF ownership and consistency checkpoints
|
|
|
|
EXIF is not a final export performed only before upload. It is a durable projection of
|
|
completed workflow decisions. Every stage that produces metadata ends with an EXIF
|
|
checkpoint for each affected asset:
|
|
|
|
```text
|
|
persist desired stage result in DB
|
|
→ read current EXIF
|
|
→ merge only the stage-owned fields into current metadata
|
|
→ write through exiftool
|
|
→ read EXIF back
|
|
→ verify desired values and preserved values
|
|
→ refresh current full-file SHA-256
|
|
→ mark checkpoint verified
|
|
```
|
|
|
|
The database stores both the desired projection and the last verified projection.
|
|
Suggested table:
|
|
|
|
```sql
|
|
CREATE TABLE exif_projections (
|
|
asset_id TEXT NOT NULL REFERENCES assets(id),
|
|
stage TEXT NOT NULL, -- safety | analysis | album
|
|
projection_version INTEGER NOT NULL,
|
|
desired_json TEXT NOT NULL,
|
|
verified_json TEXT,
|
|
source_file_sha256 TEXT NOT NULL,
|
|
result_file_sha256 TEXT,
|
|
state TEXT NOT NULL, -- pending | writing | verified | divergent | error
|
|
verified_at TEXT,
|
|
PRIMARY KEY (asset_id, stage)
|
|
);
|
|
```
|
|
|
|
Field ownership is intentionally narrow:
|
|
|
|
| Owner | May modify | Must preserve |
|
|
|---|---|---|
|
|
| User/pre-existing metadata | nothing automatically owns it | all dates, GPS, camera, ratings, people, captions, and keywords |
|
|
| Safety stage | only mutually exclusive `nsfw` / `sfw` entries in `Keywords` and `Subject` | all non-safety keywords and all other fields |
|
|
| Analysis stage | managed AI caption segment plus additive AI keywords | user caption text, safety keywords, existing keywords, and unrelated metadata |
|
|
| Album stage | no EXIF by default; optional managed album field only when enabled | every existing field |
|
|
|
|
Non-destructive means **semantic preservation**, not that the file bytes remain
|
|
unchanged: `exiftool` may rewrite the container. Before writing, retain the relevant
|
|
EXIF snapshot in the operation journal; afterward, compare all fields outside the
|
|
stage's ownership. Any unexpected change marks the asset `divergent`, blocks the next
|
|
mutating stage, and exposes a recovery action.
|
|
|
|
Analysis captions should use a recognizable managed segment so re-analysis can replace
|
|
only its own text, for example an application-owned `AI:` portion, without overwriting
|
|
a user's description. Keywords are merged as a set. Because standard EXIF keywords do
|
|
not record ownership, removing an old AI keyword could also remove a user keyword with
|
|
the same value. The conservative default is therefore additive: never automatically
|
|
remove non-safety keywords during re-analysis. Potentially stale generated keywords
|
|
are reported for optional review rather than silently deleted.
|
|
|
|
Stage consistency rules:
|
|
|
|
- **Inventory/duplicate stage:** read-only; records an EXIF fingerprint and proves it
|
|
did not modify the file.
|
|
- **Safety stage:** complete only after the chosen `sfw` or `nsfw` keyword is written,
|
|
the opposite safety keyword is absent, all unrelated metadata is preserved, and the
|
|
new file SHA-256 is stored.
|
|
- **Analysis stage:** applies only to SFW assets and completes per asset only after its
|
|
DB result, managed caption segment, additive keywords, preservation checks, and new
|
|
file SHA-256 are verified.
|
|
- **Album/rename stage:** renaming does not itself require an EXIF rewrite. If optional
|
|
album metadata is enabled, it receives its own projection/checkpoint after rename.
|
|
- **Upload stage:** writes no metadata. It verifies every required prior projection,
|
|
recalculates the current file SHA-256, and uploads exactly those verified bytes.
|
|
|
|
If a user edits EXIF externally after a checkpoint, the next scan compares the current
|
|
projection/hash with the verified state. User additions are preserved and incorporated;
|
|
missing or conflicting managed values create a visible `divergent` checkpoint requiring
|
|
repair or explicit acceptance. Upload never silently repairs metadata as a side effect.
|
|
|
|
---
|
|
|
|
## 4. Stage 0 — Inventory and duplicate review
|
|
|
|
### Discovery
|
|
|
|
Discovery registers supported photos without modifying them. It records current
|
|
path, size, dates, SHA-256, normalized-pixel hash, and perceptual hash. It always skips `_IGNORE/` and generated
|
|
thumbnail/cache directories.
|
|
|
|
The inventory screen shows:
|
|
|
|
- total files and folders;
|
|
- exact duplicates by SHA-256;
|
|
- perceptual duplicate clusters by phash distance;
|
|
- likely variants such as edited, resized, or motion-photo derivatives;
|
|
- unreadable and unsupported files;
|
|
- estimated API calls saved by resolving duplicates.
|
|
|
|
### Duplicate review
|
|
|
|
Do not immediately mark the largest file canonical without review. Exact byte or
|
|
normalized-pixel matches can be recommended with very high confidence; every fuzzy
|
|
match receives a visual decision surface.
|
|
|
|
### Viable thumbnail pipeline
|
|
|
|
Duplicate review must never depend on the browser being able to render the original.
|
|
The backend generates managed thumbnails for JPEG, PNG, WebP, TIFF, HEIC/HEIF, and
|
|
other supported image types.
|
|
|
|
Thumbnail generation rules:
|
|
|
|
1. read the original through Pillow plus the configured HEIC decoder;
|
|
2. apply EXIF orientation before resizing;
|
|
3. convert safely to sRGB/RGB without modifying the original;
|
|
4. generate at least `256px`, `512px`, and `1280px` long-edge variants;
|
|
5. encode browser-compatible WebP, with JPEG fallback;
|
|
6. write to a temporary file and atomically rename into the cache;
|
|
7. persist success/error state so broken originals do not create retry loops.
|
|
|
|
The cache lives in application data outside the photo library, for example:
|
|
|
|
```text
|
|
data/cache/thumbs/{pixel_hash_prefix}/{pixel_hash}-512-v2.webp
|
|
```
|
|
|
|
The key contains the normalized pixel hash, requested size, and thumbnail algorithm
|
|
version. Therefore:
|
|
|
|
- EXIF-only changes reuse the same thumbnail;
|
|
- folder moves and renames reuse the same thumbnail;
|
|
- visually different files never collide merely because filenames match;
|
|
- changing decoder/color/orientation behavior invalidates cache through the version;
|
|
- cache files can be deleted and regenerated without losing application state.
|
|
|
|
Thumbnail endpoints accept only an `asset_id` and bounded size enum. They resolve the
|
|
current path through the database, set immutable-cache headers for versioned keys, and
|
|
never accept an arbitrary path from the browser.
|
|
|
|
### Fuzzy duplicate comparison UI
|
|
|
|
Show uncertain clusters in a dedicated comparison workspace rather than a small grid:
|
|
|
|
- synchronized side-by-side thumbnails, initially fit-to-window;
|
|
- click/keyboard toggle between candidates;
|
|
- linked zoom and pan using the `1280px` preview;
|
|
- optional blink/overlay mode to reveal framing and crop differences;
|
|
- pixel dimensions, aspect ratio, file size, format, dates, folder, camera, EXIF
|
|
richness, file hash, pixel hash, pHash distance, and confidence explanation;
|
|
- badges for exact bytes, metadata-only difference, resized/recompressed copy,
|
|
probable crop, edited variant, or uncertain similarity;
|
|
- recommended canonical clearly marked, but never silently selected for fuzzy matches;
|
|
- prefetch the next cluster's thumbnails for fast keyboard review.
|
|
|
|
The decision applies to a complete cluster and is stored as an auditable event. If a
|
|
new copy appears later, the cluster is reopened only when it changes the confidence or
|
|
conflicts with the previous decision; otherwise the new member inherits the established
|
|
canonical relationship.
|
|
|
|
The app recommends a canonical item using a visible scoring policy:
|
|
|
|
1. original/highest resolution;
|
|
2. least recompressed format;
|
|
3. richest reliable metadata;
|
|
4. preferred folder rules;
|
|
5. largest file only as a final tie-breaker.
|
|
|
|
Actions:
|
|
|
|
- **Keep recommended**;
|
|
- **Choose another canonical**;
|
|
- **Keep all as variants**;
|
|
- **Not duplicates**;
|
|
- **Defer**.
|
|
|
|
No file is deleted automatically. Non-canonical items are excluded from later stages
|
|
and upload, but remain on disk unless a separate future cleanup feature is approved.
|
|
|
|
---
|
|
|
|
## 5. Stage 1 — Safety identification and EXIF tagging
|
|
|
|
Preserve the current NSFW review experience: folder tree, threshold, score ordering,
|
|
bulk selection, per-image decision control, lightbox, EXIF panel, and activity log.
|
|
|
|
Changes for integration:
|
|
|
|
- run only on canonical, non-ignored assets;
|
|
- store score and decisions in SQLite as well as EXIF;
|
|
- write mutually exclusive `nsfw` or `sfw` keywords to `Keywords` and `Subject`;
|
|
- merge safety keywords with all existing EXIF and verify unrelated metadata remains;
|
|
- refresh `current_sha256` after the verified EXIF checkpoint;
|
|
- replace `nsfw_scores.csv` as the source of truth with database state;
|
|
- make `_IGNORE/` exclusion mandatory in the shared discovery service;
|
|
- offer filters for undecided, SFW, NSFW, deferred, and write errors.
|
|
|
|
Privacy invariant:
|
|
|
|
> Only assets with a confirmed `sfw` decision may enter cloud content analysis.
|
|
> Confirmed `nsfw` assets skip that analyzer but remain eligible for Immich upload.
|
|
|
|
An optional setting may treat unreviewed low-score images as SFW, but it should be
|
|
off by default and clearly marked as reducing safety.
|
|
|
|
---
|
|
|
|
## 6. Stage 2 — Content analysis and analysis EXIF
|
|
|
|
Preserve the current Library, Analyze, and Stats views, but scope the engine to
|
|
confirmed SFW canonical assets.
|
|
|
|
For each eligible asset:
|
|
|
|
1. prepare a resized in-memory image;
|
|
2. call the configured OpenAI-compatible vision model;
|
|
3. validate and store the response by `asset_id`;
|
|
4. read the latest EXIF and merge the managed AI caption segment and additive tags;
|
|
5. write and read back EXIF without removing safety or user metadata;
|
|
6. compare non-owned fields with the pre-write snapshot;
|
|
7. refresh the full-file SHA-256 after verification;
|
|
8. mark the analysis EXIF checkpoint verified and the asset metadata-complete.
|
|
|
|
The UI shows model and prompt version, progress, errors, retries, tokens, estimated
|
|
remaining cost/quota, and EXIF verification state.
|
|
|
|
The stage is complete only when every included database result and on-disk EXIF
|
|
projection agree. Verification is per asset, not merely a sample; the UI may additionally
|
|
offer a human-readable sample review before proceeding.
|
|
|
|
---
|
|
|
|
## 7. Stage 3 — AI album suggestions and guarded renaming
|
|
|
|
This stage operates primarily on **folders**, not individual filenames. The default
|
|
goal is to create useful Immich albums while minimizing filesystem changes.
|
|
|
|
### Inputs for an album proposal
|
|
|
|
For each leaf folder, build a compact summary from:
|
|
|
|
- current folder name and parent path;
|
|
- capture-date range and dominant year;
|
|
- number of photos and videos;
|
|
- common analysis tags, settings, places, and descriptions;
|
|
- GPS/location hints where present;
|
|
- detected events or themes;
|
|
- neighboring folder names and date ranges;
|
|
- existing user-provided naming conventions.
|
|
|
|
Prefer using already-stored analysis data. Do not resend every image. If visual
|
|
sampling is useful, make it an explicit option using a few representative SFW images.
|
|
|
|
### Suggested naming format
|
|
|
|
The default recommendation should be predictable and sortable:
|
|
|
|
```text
|
|
YYYY-MM — Place — Event
|
|
YYYY — Event
|
|
Place — Event
|
|
```
|
|
|
|
Examples:
|
|
|
|
```text
|
|
2019-07 — Rome — Summer Holiday
|
|
2016 — Capgemini Team Event
|
|
Family Portraits — Selected
|
|
```
|
|
|
|
Names should be concise, human-readable, filesystem-safe, and suitable for Immich's
|
|
folder-as-album behavior. The app shows why each name was proposed and flags uncertain
|
|
date/place inferences.
|
|
|
|
### Proposal review UI
|
|
|
|
Use a two-pane Albums view:
|
|
|
|
- left: folder tree with proposal state and confidence;
|
|
- right: current name, proposed name, representative thumbnails, date range, common
|
|
tags, rationale, and editable final name.
|
|
|
|
Bulk actions may accept high-confidence proposals, but never apply them immediately.
|
|
Accepted edits first become a **rename plan**.
|
|
|
|
### Guarded rename plan
|
|
|
|
Before any filesystem operation, validate the entire plan:
|
|
|
|
- source exists and is still the registered folder;
|
|
- destination is inside the configured library;
|
|
- neither source nor destination is inside `_IGNORE/`;
|
|
- destination names are unique on the target filesystem;
|
|
- case-only renames are handled safely on case-insensitive filesystems;
|
|
- no asset is currently being analyzed, EXIF-written, or uploaded;
|
|
- final paths do not exceed configured portability limits;
|
|
- all affected assets are present and hashes match the latest inventory;
|
|
- database and filesystem are writable;
|
|
- no previously uploaded asset would be silently changed.
|
|
|
|
The preview displays every old → new path and supports exporting the plan as JSON.
|
|
Applying requires an explicit confirmation such as **Apply 12 folder renames**.
|
|
|
|
### Crash-safe rename transaction
|
|
|
|
Filesystem and SQLite cannot share a true transaction, so use a journaled state
|
|
machine:
|
|
|
|
```text
|
|
planned → validated → moving → moved → database_updated → verified → complete
|
|
↘ failed / rollback_required
|
|
```
|
|
|
|
For each operation:
|
|
|
|
1. persist the operation and expected hashes;
|
|
2. rename on the same filesystem using an atomic move where possible;
|
|
3. update `assets.current_path` by `asset_id` in one SQLite transaction;
|
|
4. append old/new entries to `asset_paths`;
|
|
5. verify all files at their new paths and refresh folder summaries;
|
|
6. mark the operation complete.
|
|
|
|
On restart, the app inspects the journal and filesystem to resume or offer rollback.
|
|
It never guesses based only on a missing old path.
|
|
|
|
Because records use stable asset IDs, renaming does not invalidate analysis, safety,
|
|
duplicate, or upload history.
|
|
|
|
---
|
|
|
|
## 8. Stage 4 — Immich upload via immich-go
|
|
|
|
Upload is an explicit final stage, never an automatic consequence of rename.
|
|
|
|
### Safety and upload eligibility
|
|
|
|
Safety classification controls **analysis routing**, not whether a reviewed photo may
|
|
exist in Immich:
|
|
|
|
| Safety state | Cloud content analysis | Immich upload |
|
|
|---|---|---|
|
|
| `sfw` | eligible/required by normal workflow | eligible after verified safety and analysis EXIF checkpoints |
|
|
| `nsfw` | forbidden | eligible after verified `nsfw` EXIF tag |
|
|
| `unreviewed` | forbidden | blocked until reviewed |
|
|
| `deferred` | forbidden | blocked unless an explicit future policy permits it |
|
|
|
|
Immich therefore receives both reviewed SFW and reviewed NSFW canonical assets. The
|
|
`nsfw` keyword is written before upload so those assets can be found, filtered, or
|
|
managed in Immich without exposing them to the external content-analysis provider.
|
|
|
|
### Preflight
|
|
|
|
Before enabling upload, verify:
|
|
|
|
- asset is canonical and has a confirmed `sfw` or `nsfw` safety decision;
|
|
- SFW assets have verified safety and analysis EXIF checkpoints, unless explicitly
|
|
waived with a recorded exception;
|
|
- NSFW assets carry their verified `nsfw` EXIF keyword and have not been sent to the
|
|
cloud content analyzer;
|
|
- rename journal is empty and album paths are final;
|
|
- current SHA-256 matches the latest verified EXIF checkpoint and post-rename inventory;
|
|
- Immich server and API key validate;
|
|
- `immich-go` is installed;
|
|
- no External Library workflow is configured for the same source;
|
|
- the album name preview matches the folder-as-album result.
|
|
|
|
### Upload batches
|
|
|
|
Create one batch per selected leaf folder. The UI previews the exact command without
|
|
printing the API key. A batch stores:
|
|
|
|
- album/folder and asset IDs;
|
|
- redacted command and immich-go version;
|
|
- pre-upload SHA-256 plus the SHA-1 used by Immich/immich-go for every item, both
|
|
calculated from the latest verified bytes;
|
|
- stdout/stderr activity stream;
|
|
- new, upgraded, metadata-updated, duplicate, skipped, and failed counts;
|
|
- start/end timestamps and exit status.
|
|
|
|
The Upload view provides **Validate**, **Dry run** where supported, **Upload selected**,
|
|
**Stop after current album**, and **Retry failed** actions.
|
|
|
|
After a successful upload, an asset becomes `uploaded` only when its result is parsed
|
|
or otherwise verified. If the file bytes later change, the app marks it
|
|
`changed_after_upload` and warns that uploading again may create or upgrade an asset.
|
|
|
|
---
|
|
|
|
## 9. Stage 5 — Archive and active-storage reclamation
|
|
|
|
Archive is an optional final lifecycle stage. It moves a completed album out of the
|
|
active library branch to a configured archive location, then marks its assets archived
|
|
and unavailable from active storage while preserving their permanent database records.
|
|
|
|
“Archived” means **no longer present in the active photo library**. The archive may be
|
|
another local disk, external disk, NAS-backed archive workflow, or removable medium.
|
|
When that destination is not mounted, the asset is `archived_offline`, not `missing`.
|
|
|
|
Archive is not deduplication and never deletes database history. It preserves:
|
|
|
|
- stable asset ID and complete path history;
|
|
- exact file SHA-256 and Immich SHA-1 from the archived bytes;
|
|
- normalized pixel hash and perceptual hashes;
|
|
- duplicate cluster and canonical decision;
|
|
- safety decision and verified EXIF projections;
|
|
- content analysis, FTS data, album proposal, rename history, and upload history;
|
|
- a durable browser-compatible review thumbnail, preferably the `1280px` variant;
|
|
- archive location, relative archive path, media/volume identity, and transfer manifest.
|
|
|
|
### Availability states
|
|
|
|
Availability is independent from workflow completion:
|
|
|
|
| State | Meaning |
|
|
|---|---|
|
|
| `active` | original is present in the active library |
|
|
| `archiving` | journaled transfer/removal is in progress |
|
|
| `archived_online` | original exists at a currently reachable archive location |
|
|
| `archived_offline` | original is archived but its medium/location is unavailable |
|
|
| `restoring` | copy back to active storage is in progress |
|
|
| `missing_unexpected` | neither active nor recorded archive location explains absence |
|
|
|
|
The ordinary inventory scanner must not prune archived assets. It scans active roots
|
|
and currently mounted archive roots separately and interprets absence using
|
|
`availability_state` and archive-location identity.
|
|
|
|
### Archive preflight
|
|
|
|
Before allowing an album to be archived:
|
|
|
|
- every selected canonical asset has a confirmed safety decision;
|
|
- every SFW asset has verified analysis EXIF and every NSFW asset has verified safety
|
|
EXIF;
|
|
- the selected upload batch is verified, not merely process-successful;
|
|
- current file checksums match the exact uploaded/checkpointed bytes;
|
|
- no rename, EXIF, analysis, upload, restore, or other archive job holds a conflicting
|
|
lease;
|
|
- the archive destination is configured, mounted, writable, outside `_IGNORE/`, and
|
|
has sufficient free space plus a reserve;
|
|
- the destination does not already contain unexpected paths;
|
|
- the durable comparison thumbnail exists and is verified before the original leaves
|
|
active storage;
|
|
- a database backup and archive manifest can be written successfully.
|
|
|
|
The UI previews source, destination, number of files, total bytes, expected reclaimed
|
|
active space, transfer method, archive medium, and every blocker. Archive requires an
|
|
explicit confirmation such as **Archive 1 album · reclaim 84.3 GB**.
|
|
|
|
### Transfer and removal semantics
|
|
|
|
Archive uses a journaled plan similar to rename, but cross-filesystem movement is
|
|
expected:
|
|
|
|
```text
|
|
planned → validated → transferring → archive_verified → removing_active
|
|
→ active_absence_verified → database_updated → complete
|
|
```
|
|
|
|
For each file:
|
|
|
|
1. persist source, archive destination, expected size and hashes;
|
|
2. copy to a temporary destination file;
|
|
3. flush/close it and calculate its SHA-256;
|
|
4. atomically rename the verified temporary file to its archive destination;
|
|
5. record the verified archive occurrence and manifest entry;
|
|
6. only then remove the active source;
|
|
7. verify the active path is absent and archive path still matches;
|
|
8. set `current_path=NULL`, close the active path-history entry, and set availability
|
|
to `archived_online` or `archived_offline` according to location reachability.
|
|
|
|
If source and archive destination are demonstrably on the same filesystem, an atomic
|
|
move may replace copy-verify-delete, but the operation still verifies the resulting
|
|
file and persists the journal before updating database state.
|
|
|
|
The source is never removed merely because Immich reports success. Immich managed
|
|
storage is an ingest target, not automatically the only backup. A future explicit
|
|
“uploaded-only retention” policy may exist, but it must be a separate high-risk option
|
|
and is not the default Archive behavior.
|
|
|
|
### Future deduplication against archived assets
|
|
|
|
Archived assets remain in every hash index. When a new active photo is discovered:
|
|
|
|
- exact SHA-256 or normalized-pixel matches can link directly to the archived canonical;
|
|
- perceptual matches create/reopen a fuzzy duplicate cluster;
|
|
- the retained thumbnail, metadata, dimensions, and hash evidence appear in duplicate
|
|
review even when the archive disk is offline;
|
|
- if full-resolution comparison is required, the UI requests that the named archive
|
|
location/medium be mounted rather than guessing;
|
|
- choosing the archived asset as canonical keeps the new active copy excluded from
|
|
analysis/upload unless the user deliberately restores or changes the canonical.
|
|
|
|
This is why durable comparison thumbnails are part of archival preflight rather than
|
|
an evictable cache entry. They may use a separate protected archive-preview store with
|
|
its own backup policy.
|
|
|
|
### Restore
|
|
|
|
Restore is also planned and journaled. It requires the archive location online, copies
|
|
and verifies the archived bytes into a collision-free active destination, registers a
|
|
new active path occurrence, changes availability to `active`, and runs reconciliation.
|
|
Existing safety, analysis, EXIF, duplicate, and upload states remain valid when hashes
|
|
match. Any mismatch produces `stale`/`divergent` state instead of silently accepting a
|
|
different file.
|
|
|
|
---
|
|
|
|
## 10. Information architecture and UX
|
|
|
|
Retain the dark OLED design, cards, folder sidebar, lightbox, segmented controls,
|
|
progress bars, status colors, activity drawer, and keyboard navigation. The visual
|
|
language remains familiar even though the frontend is now an API client rather than
|
|
HTML generated by Python.
|
|
|
|
### Frontend architecture
|
|
|
|
Keep the first implementation deliberately lightweight and dependency-free: semantic
|
|
HTML, CSS, and modular browser JavaScript served as separate static assets by FastAPI.
|
|
`index.html` contains only the persistent application shell and references the
|
|
stylesheets and JavaScript entry point; it does not embed the application's CSS,
|
|
JavaScript, cards, or operational data. Organize JavaScript by API client, state/store,
|
|
views, and reusable components rather than putting everything in one HTML file or one
|
|
large script. If the duplicate comparison and workflow state outgrow this approach,
|
|
the same versioned API can support a later TypeScript framework without changing the
|
|
backend.
|
|
|
|
Frontend rules:
|
|
|
|
- no Python string interpolation of cards or operational state;
|
|
- `index.html`, CSS, and JavaScript are separate source files;
|
|
- the initial HTML response is only the application shell; JavaScript fetches all
|
|
operational data from `/api/v1` after the shell loads;
|
|
- `/api/v1` endpoints return JSON (and SSE only for the documented event stream),
|
|
never HTML fragments or the application shell; API errors use a consistent JSON
|
|
error envelope and appropriate HTTP status codes;
|
|
- HTML navigation routes may fall back to `index.html`, but requests under `/api/v1`
|
|
must never use that fallback;
|
|
- one shared API client handles errors, cancellation, and optimistic concurrency;
|
|
- SSE updates the job/activity store; failed streams fall back to polling;
|
|
- URL routes encode the current view, cluster, asset, album, and filters;
|
|
- destructive confirmations include the server-issued plan/version token so stale
|
|
browser state cannot apply changed operations.
|
|
|
|
### Global header
|
|
|
|
```text
|
|
[Photo Pipeline] Library: pictures/ [Workflow] [Review] [Library] [Albums] [Upload] [Archive]
|
|
25,318 files · 21 warnings · no job running [Activity]
|
|
```
|
|
|
|
### Workflow home
|
|
|
|
The default page is a five-step vertical or horizontal stepper:
|
|
|
|
```text
|
|
✓ 0 Inventory 24,901 canonical · 417 duplicates
|
|
! 1 Safety 24,620 SFW · 83 NSFW · 198 undecided
|
|
● 2 Analysis 20,110 complete · 4,510 pending
|
|
○ 3 Albums blocked by analysis · 43 suggestions ready
|
|
○ 4 Upload not ready
|
|
○ 5 Archive blocked by upload · 0 B reclaimable
|
|
```
|
|
|
|
Each stage card contains status, prerequisite warnings, last-run time, primary action,
|
|
and a link to details. Status is communicated with text/icons as well as color.
|
|
|
|
### Shared interaction rules
|
|
|
|
- one long-running mutating job at a time;
|
|
- read-only browsing remains available during jobs;
|
|
- every bulk mutation has preview → validate → confirm → execute → verify;
|
|
- stop requests drain safely and leave a resumable job record;
|
|
- every error links to the affected asset, album, or operation;
|
|
- all actions are scoped to the visible selection and show exact counts;
|
|
- credentials are never returned to the browser or written to logs;
|
|
- all paths served by image/EXIF endpoints are validated by asset ID and current path.
|
|
|
|
---
|
|
|
|
## 11. Consistency rules
|
|
|
|
The integrated app enforces these invariants centrally:
|
|
|
|
1. `_IGNORE/` is excluded by one shared path-policy function used everywhere.
|
|
2. Duplicate resolution precedes safety scoring and paid analysis.
|
|
3. NSFW assets never reach the external vision API, but remain eligible for Immich
|
|
upload after their safety EXIF keyword is verified.
|
|
4. Safety keywords survive content EXIF writes.
|
|
5. Every metadata stage ends with a non-destructive, read-back-verified EXIF checkpoint;
|
|
upload uses the SHA-256 of the latest verified bytes.
|
|
6. Rename plans cannot run concurrently with analysis, EXIF, or upload jobs.
|
|
7. Asset identity is stable across moves and renames.
|
|
8. Upload history is tied to asset ID plus the exact uploaded SHA-256 and Immich SHA-1.
|
|
9. Changing bytes after upload produces a visible stale-upload warning.
|
|
10. No automatic deletion, rename, or upload occurs without explicit approval.
|
|
11. Archiving never removes an active source until the archive bytes and durable
|
|
manifest are verified.
|
|
12. Archived assets remain in deduplication indexes even when their original storage
|
|
is offline.
|
|
|
|
---
|
|
|
|
## 12. Proposed project layout
|
|
|
|
```text
|
|
pipeline_app/
|
|
├── pyproject.toml
|
|
├── alembic.ini
|
|
├── migrations/
|
|
├── src/photo_pipeline/
|
|
│ ├── __main__.py # API/worker management CLI
|
|
│ ├── config.py # typed configuration and secret references
|
|
│ ├── db.py # engine, sessions, WAL/foreign-key setup
|
|
│ ├── models/ # SQLAlchemy persistence models
|
|
│ ├── schemas/ # Pydantic API contracts
|
|
│ ├── repositories/ # database access only
|
|
│ ├── services/
|
|
│ │ ├── inventory.py
|
|
│ │ ├── thumbnails.py
|
|
│ │ ├── duplicates.py
|
|
│ │ ├── safety.py
|
|
│ │ ├── analysis.py
|
|
│ │ ├── albums.py
|
|
│ │ ├── renames.py
|
|
│ │ ├── uploads.py
|
|
│ │ ├── archives.py
|
|
│ │ └── workflow.py
|
|
│ ├── jobs/
|
|
│ │ ├── coordinator.py # enqueue, locks, cancellation, recovery
|
|
│ │ ├── worker.py # durable worker loop
|
|
│ │ └── handlers.py # job-type dispatch
|
|
│ ├── api/
|
|
│ │ ├── app.py # FastAPI factory and lifecycle
|
|
│ │ ├── dependencies.py
|
|
│ │ ├── errors.py
|
|
│ │ └── routes/ # versioned /api/v1 route modules
|
|
│ ├── integrations/
|
|
│ │ ├── exiftool.py
|
|
│ │ ├── vision.py
|
|
│ │ ├── nsfw_model.py
|
|
│ │ └── immich_go.py
|
|
│ └── path_policy.py # library boundary and _IGNORE rules
|
|
├── frontend/
|
|
│ ├── index.html # markup-only application shell
|
|
│ ├── css/
|
|
│ │ └── app.css # shared design tokens, layout, components
|
|
│ └── js/
|
|
│ ├── app.js # module entry point and application bootstrap
|
|
│ ├── api.js
|
|
│ ├── store.js
|
|
│ ├── router.js
|
|
│ ├── components/
|
|
│ └── views/
|
|
├── legacy_cli_archive/ # frozen donor sources after verified parity
|
|
│ ├── README.md # provenance, final versions, and archive rationale
|
|
│ └── donor-ledger.yaml # source → service/test mapping and intentional deltas
|
|
├── tests/
|
|
│ ├── unit/
|
|
│ ├── integration/
|
|
│ └── e2e/
|
|
└── data/ # runtime DB/cache; ignored by source control
|
|
└── cache/thumbs/
|
|
```
|
|
|
|
Existing CLIs remain available during migration. Their proven internals should be
|
|
extracted into the new service layer, after which the CLI entry points should call
|
|
those shared services until the archival gates above pass. Business logic must have
|
|
one implementation shared by API, worker, and transitional CLI. This avoids both a
|
|
needless rewrite and two subtly different discovery, path-exclusion, EXIF, or state
|
|
models surviving in parallel.
|
|
|
|
---
|
|
|
|
## 13. Recommended implementation phases
|
|
|
|
### Automated acceptance policy for every phase
|
|
|
|
Automated tests are part of implementation, not deferred hardening. Every user story
|
|
must have at least one automated acceptance test that exercises its observable
|
|
behavior and failure conditions. Maintain a traceable mapping from story identifier
|
|
to test name/path; a story is incomplete while its mapped tests are missing, skipped,
|
|
or failing.
|
|
|
|
Every phase below must add and pass an end-to-end suite covering the user journeys
|
|
introduced or changed by that phase through the real browser, HTTP/SSE API, worker,
|
|
isolated database, and temporary photo library. External services and executables may
|
|
use controlled fakes at their integration boundaries, but browser-visible behavior
|
|
and backend orchestration must not be mocked. The full end-to-end regression suite for
|
|
all completed phases must also remain green. A phase cannot be declared complete or
|
|
merged merely because its unit or API tests pass.
|
|
|
|
### Phase A — Shared identity and inventory
|
|
|
|
- inventory both CLI codebases into the donor ledger and add characterization tests
|
|
before replacing or relocating any in-scope behavior;
|
|
- extract reusable discovery, hashing, reconciliation, image/EXIF, model, database,
|
|
configuration, and UI donor components into stable modules with provenance noted;
|
|
- scaffold FastAPI, SQLAlchemy, Alembic, configuration, and application lifecycle;
|
|
- add stable asset IDs and path history;
|
|
- centralize discovery and `_IGNORE/` policy;
|
|
- migrate current photo and NSFW state;
|
|
- implement normalized pixel hashes plus exact and perceptual indexes;
|
|
- implement managed thumbnail generation and its asset-ID API;
|
|
- build inventory and visual duplicate-review UI.
|
|
|
|
**End-to-end gate:** automated journeys cover scan/reconciliation, `_IGNORE/`
|
|
exclusion, stable identity across a move, thumbnail loading/orientation, exact and
|
|
fuzzy duplicate clusters, canonical selection, and persistence after reload. Donor
|
|
characterization and parity suites pass for every CLI behavior migrated in this phase.
|
|
|
|
This is the foundation. Do not implement renaming before it is complete.
|
|
|
|
### Phase B — Unified workflow shell
|
|
|
|
- implement durable jobs, worker heartbeats, recovery, locks, and SSE events;
|
|
- create Workflow home and shared job/activity model;
|
|
- port the existing safety review to the shared APIs;
|
|
- port the existing Library/Analyze/Stats views to the shared APIs;
|
|
- enforce gates and one-mutating-job policy.
|
|
|
|
**End-to-end gate:** automated journeys cover the workflow shell loading data from
|
|
JSON APIs, job start/progress/reconnect/cancel/resume, activity SSE with polling
|
|
fallback, safety decisions, Library/Analyze/Stats behavior, prerequisite blockers,
|
|
and rejection of a second concurrent mutating job.
|
|
|
|
### Phase C — Album proposals
|
|
|
|
- aggregate folder metadata from existing DB results;
|
|
- define configurable naming templates and forbidden characters;
|
|
- add AI proposal generation, rationale, confidence, editing, and approval.
|
|
|
|
**End-to-end gate:** automated journeys cover proposal generation, editing, approval,
|
|
invalid names, collisions, provider failures, stale proposal versions, and persistence
|
|
after reload.
|
|
|
|
### Phase D — Guarded renaming
|
|
|
|
- implement plan validation and JSON export;
|
|
- implement journaled rename/apply/verify;
|
|
- test interruption at every state transition;
|
|
- add recovery and rollback UI.
|
|
|
|
**End-to-end gate:** automated journeys cover preview/validate/confirm/apply/verify,
|
|
stale confirmation rejection, collisions and case-only renames, interruption at each
|
|
journal state, restart recovery, rollback, and refusal to overwrite unexpected files.
|
|
|
|
### Phase E — Immich upload
|
|
|
|
- add credential preflight and redacted command preview;
|
|
- run one album at a time through `immich-go`;
|
|
- parse and persist reports;
|
|
- add retry, verification, and changed-after-upload warnings.
|
|
|
|
**End-to-end gate:** automated journeys cover credential preflight without secret
|
|
leakage, album scope and command preview, successful upload, exact duplicate, upgrade,
|
|
retryable failure, uncertain outcome verification, cancellation/resume, and stale-byte
|
|
blocking after EXIF changes.
|
|
|
|
### Phase F — Archive lifecycle
|
|
|
|
- configure archive locations and stable volume/media identities;
|
|
- implement archive preview, capacity checks, and protected thumbnails;
|
|
- implement journaled copy-verify-remove and same-filesystem move paths;
|
|
- retain offline assets in hash indexes and duplicate review;
|
|
- implement mount detection, restore plans, and recovery UI;
|
|
- test interruption before and after source removal.
|
|
|
|
**End-to-end gate:** automated journeys cover archive preview and capacity blockers,
|
|
copy/hash verification, same- and cross-filesystem paths, offline browsing through
|
|
protected thumbnails, interruption before and after source removal, mount changes,
|
|
restart recovery, restore, and collision-safe restoration.
|
|
|
|
### Phase G — Hardening
|
|
|
|
- resolve every donor-ledger row, document intentional deltas, and archive the CLI
|
|
sources only after their replacement parity and end-to-end gates pass;
|
|
- migrate remaining CSV state into SQLite;
|
|
- add end-to-end tests with a temporary library and fake uploader;
|
|
- test API authorization assumptions, path validation, stale versions, and malformed
|
|
requests;
|
|
- test thumbnail orientation, HEIC decoding, corruption, regeneration, and cache
|
|
invalidation;
|
|
- test case-only renames, collisions, interruptions, EXIF failures, and stale hashes;
|
|
- document backup and recovery procedures.
|
|
|
|
**End-to-end gate:** the complete automated story matrix passes as one reproducible
|
|
command against a fresh temporary environment, including fault injection, restart
|
|
recovery, authorization/path attacks, malformed inputs, supported image formats, and
|
|
the full discovery-to-upload-and-archive smoke journey.
|
|
|
|
---
|
|
|
|
## 14. Definition of done
|
|
|
|
The integrated application is successful when a new folder can be taken from discovery
|
|
to verified Immich upload and optional archive without using separate commands, while
|
|
still allowing every stage to be stopped, reviewed, resumed, restored, or rerun safely.
|
|
At every point the UI can
|
|
answer:
|
|
|
|
- What stage is this asset or album in?
|
|
- Why is it blocked?
|
|
- What will the next action change?
|
|
- Can that change be reversed?
|
|
- Which exact bytes were uploaded to Immich?
|
|
- Where is the original now, and how can it be restored?
|
|
|
|
In addition, every user story is linked to a passing automated acceptance test, every
|
|
phase's end-to-end gate passes, and the accumulated end-to-end regression suite runs
|
|
successfully from a clean environment with one documented command. Manual testing is
|
|
exploratory evidence only and never substitutes for these automated checks. Every
|
|
legacy CLI donor-ledger row is resolved, all intentional behavior changes are tested
|
|
and documented, and the frozen CLI archive is sufficient to trace the origin of reused
|
|
behavior without being a runtime dependency.
|
|
|
|
---
|
|
|
|
## 15. Architecture challenge and risk register
|
|
|
|
The design is viable, but it can fail in several predictable ways. These risks should
|
|
be treated as design inputs, not deferred hardening.
|
|
|
|
### Critical risks
|
|
|
|
| Risk | Failure mode | Required mitigation |
|
|
|---|---|---|
|
|
| False duplicate decision | distinct burst/cropped photos are grouped and excluded | auto-resolve only exact byte/pixel matches; fuzzy matches require review; decisions reversible |
|
|
| Identity ambiguity | an old path disappears and several matching new paths appear | preserve the old asset; create occurrences; ask which path is canonical; never infer deletion |
|
|
| Files change during work | user/editor modifies or moves a file after validation | stat/hash before and after; optimistic asset version; fail stale operations instead of continuing |
|
|
| SQLite writer contention | many worker threads produce `database is locked` or long stalls | one serialized DB-write channel, short transactions, bounded queue, busy timeout, backpressure |
|
|
| Crash between filesystem and DB | rename succeeds but DB update does not, or EXIF partly completes | durable operation journal, idempotent recovery, pre/postconditions, never hold DB transaction during tool call |
|
|
| Duplicate job execution | stale heartbeat causes the same item to run twice | leases, idempotency keys, compare-and-set transitions, idempotent handlers; assume at-least-once delivery |
|
|
| Wrong upload state | `immich-go` output changes or process dies after server accepted files | store exact pre-upload hash, raw report, versioned parser; use `unknown_requires_verification`, not success |
|
|
| Archive data loss | active source is removed before archive bytes are durable | copy, close, hash-verify, manifest, then remove; journal every item; never overwrite unexpected destinations |
|
|
| Offline canonical uncertainty | fuzzy match targets an archived original that cannot be opened | retain protected preview/evidence; require archive mount for full-resolution decisions |
|
|
| Resource exhaustion | large images/high concurrency exhaust RAM, file descriptors, GPU, or disk | bounded pools and queues, pixel limits, cache quota, admission control, metrics |
|
|
| External path access | symlink or race escapes the configured library | resolve and validate at operation time, reject symlink escapes, use asset IDs, repeat validation before writes |
|
|
| Local web attack | another page triggers mutations through localhost | random session token, strict Origin/Host checks, SameSite cookie, CSRF protection, no permissive CORS |
|
|
|
|
### Deduplication risks
|
|
|
|
- Perceptual hashes have false positives, particularly for screenshots, blank images,
|
|
graphics, bursts, and similarly framed portraits.
|
|
- They have false negatives for crops, rotations, overlays, strong color edits, or
|
|
decoder differences.
|
|
- Normalized pixel hashes can change when color-management or decoder versions change.
|
|
- Automatically attaching a newly discovered file to an already-reviewed cluster can
|
|
propagate an old mistake.
|
|
- Selecting one canonical can discard useful metadata present only on another copy.
|
|
|
|
Mitigations:
|
|
|
|
- store hash algorithm and decoder versions;
|
|
- keep exact, normalized-pixel, and perceptual evidence separately;
|
|
- define confidence bands rather than a single pHash threshold;
|
|
- require visual review for fuzzy matches and for cluster merges;
|
|
- preserve all files and make canonical decisions reversible;
|
|
- show metadata differences and optionally merge approved metadata before exclusion;
|
|
- reopen a cluster when a new member contradicts its existing dimensions/date/evidence;
|
|
- keep `not_duplicate` negative links so rescans do not repeatedly suggest rejected pairs.
|
|
|
|
### Filesystem and EXIF risks
|
|
|
|
- A file can be moved, replaced, truncated, or edited between discovery and action.
|
|
- `exiftool` can rewrite the whole container, fail after creating a temporary file, or
|
|
behave differently by format.
|
|
- Filesystems may be case-insensitive, normalization-sensitive, remote, or lack atomic
|
|
directory rename guarantees.
|
|
- Symlinks can redirect a previously validated path.
|
|
- File timestamps can change due to metadata writes and cannot identify content.
|
|
- Rename rollback may be impossible if another process creates the old destination.
|
|
|
|
Required behavior:
|
|
|
|
- store a lightweight snapshot `(device, inode where useful, size, mtime_ns, hash,
|
|
asset_version)` when planning;
|
|
- revalidate immediately before mutation and again afterward;
|
|
- perform EXIF writes one asset at a time with a durable item state;
|
|
- verify safety keywords and analysis fields by reading EXIF back;
|
|
- treat cross-filesystem moves as copy-verify-delete and require stronger confirmation;
|
|
- refuse rename rollback when either path no longer matches recorded preconditions;
|
|
- never overwrite an unexpected destination;
|
|
- mark ambiguous outcomes for manual recovery instead of guessing.
|
|
|
|
### External integration risks
|
|
|
|
- Vision providers rate-limit, return malformed JSON, change model behavior, or block
|
|
content despite an SFW review.
|
|
- A model or prompt upgrade makes old and new descriptions inconsistent.
|
|
- NSFW model downloads or GPU libraries may be unavailable.
|
|
- `immich-go` flags and report text can change between versions.
|
|
- Immich may accept an upload even if the local process loses its response.
|
|
|
|
Mitigations:
|
|
|
|
- persist provider/model/prompt/schema versions with every result;
|
|
- validate structured responses and quarantine invalid results;
|
|
- use bounded retries with jitter and a circuit breaker, never infinite retry;
|
|
- pin and record supported `immich-go` versions;
|
|
- keep raw subprocess output and parse through version-specific adapters;
|
|
- represent uncertain external outcomes explicitly and provide verification/retry tools;
|
|
- make every external step restartable at item level.
|
|
|
|
### Migration and operational risks
|
|
|
|
- Importing the current path-keyed DB can accidentally merge records or lose FTS data.
|
|
- Running old CLIs while the new app is active can bypass locks and mutate files.
|
|
- Database corruption or an incorrect migration can strand the whole library state.
|
|
- Logs, raw model responses, and thumbnails can expose private information.
|
|
|
|
Mitigations:
|
|
|
|
- migration begins with a verified online backup and produces a reconciliation report;
|
|
- run migrations transactionally where SQLite permits and test downgrade/recovery;
|
|
- add a library-level application lock understood by both new and migrated CLI paths;
|
|
- warn or refuse when an incompatible legacy process is detected;
|
|
- keep application data owner-readable only and support log/thumbnail retention limits;
|
|
- provide database integrity checks and backup restore drills.
|
|
|
|
---
|
|
|
|
## 16. Concurrency, locking, and race-condition model
|
|
|
|
High internal concurrency is allowed for image decoding, hashing, inference, and API
|
|
calls. It must not translate into uncontrolled database or filesystem concurrency.
|
|
|
|
### Concurrency lanes
|
|
|
|
```text
|
|
Scanner/read pool bounded threads, read-only filesystem work
|
|
CPU image pool bounded processes for decode/hash/thumbnail work
|
|
GPU safety lane normally one process/bounded batch queue
|
|
Vision API pool bounded async requests with rate limiter
|
|
Database writer one serialized writer queue
|
|
Filesystem mutator one library-wide lane for EXIF/rename operations
|
|
Uploader one active album batch by default
|
|
Archiver one active archive plan by default
|
|
```
|
|
|
|
Defaults must derive from resources, not a single global `MAX_WORKERS`. For example,
|
|
thumbnail decoding may allow four workers while GPU inference allows one and network
|
|
analysis allows twenty. Every lane has a bounded queue; when full, producers block or
|
|
pause rather than allocating indefinitely.
|
|
|
|
### Database rules
|
|
|
|
1. SQLAlchemy sessions are never shared across threads or tasks.
|
|
2. Worker threads produce result messages; a dedicated writer persists them in small
|
|
batches or individual short transactions.
|
|
3. No transaction remains open while hashing, decoding, calling a model, running
|
|
`exiftool`, moving files, or uploading.
|
|
4. Reads use separate short-lived sessions. Long UI queries are paged and cannot hold
|
|
a WAL read transaction indefinitely.
|
|
5. Job claiming is an atomic compare-and-set update from `queued` to `running` with a
|
|
lease owner and expiry.
|
|
6. Asset updates include `WHERE id=? AND version=?`; zero updated rows means stale data
|
|
and produces a conflict/retry decision.
|
|
7. The write queue has a maximum depth and exposes pressure metrics.
|
|
8. WAL checkpoints are monitored; repeated checkpoint starvation is an operational
|
|
warning, not silently ignored.
|
|
|
|
SQLite still has one writer. If sustained write pressure or UI latency exceeds agreed
|
|
limits after batching and backpressure, PostgreSQL becomes a planned scale transition.
|
|
Adding more SQLite writer threads is not the solution.
|
|
|
|
### Lock hierarchy
|
|
|
|
Locks have a fixed acquisition order to prevent deadlocks:
|
|
|
|
```text
|
|
library lease → stage/job lease → album/folder lease → asset lease
|
|
```
|
|
|
|
- Never acquire a broader lock while holding a narrower one.
|
|
- Database row/lease ownership is authoritative; in-process mutexes are optimizations.
|
|
- Leases have owner IDs, heartbeats, expiry, and fencing tokens.
|
|
- A recovered worker with an old fencing token cannot commit after a new worker has
|
|
taken ownership.
|
|
- Read-only thumbnail generation may run with analysis but not for an asset currently
|
|
being renamed unless it already has a cache hit.
|
|
- Rename takes an exclusive library mutation lease.
|
|
- EXIF writes take an asset mutation lease and are blocked during rename.
|
|
- Upload takes album leases and validates that no EXIF/rename lease is active.
|
|
- Archive takes an exclusive album mutation lease plus archive-location lease and is
|
|
blocked by analysis, EXIF, rename, upload, restore, or another archive operation.
|
|
|
|
### TOCTOU protection
|
|
|
|
Every mutating operation follows:
|
|
|
|
```text
|
|
load asset + version
|
|
→ acquire lease/fencing token
|
|
→ resolve and validate current path
|
|
→ verify expected snapshot/hash
|
|
→ perform one external/filesystem action
|
|
→ verify result
|
|
→ commit state with version + fencing-token check
|
|
→ release lease
|
|
```
|
|
|
|
If the commit loses its version race, the result is not discarded. It is recorded as
|
|
an ambiguous operation event and reconciled from the actual file state.
|
|
|
|
### Idempotency
|
|
|
|
Exactly-once execution is not achievable across SQLite, filesystem, subprocesses, and
|
|
Immich. The design promises **at-least-once execution with idempotent recovery**.
|
|
|
|
- API mutation requests accept or generate idempotency keys.
|
|
- Job items have unique `(job_type, asset_id, input_version)` constraints.
|
|
- EXIF writes merge desired fields and verify the final value.
|
|
- Thumbnail writes target content/version keys and use atomic replacement.
|
|
- Rename operations persist source, destination, and expected hashes before moving.
|
|
- Upload retries consult both local upload history and Immich/immich-go results.
|
|
|
|
---
|
|
|
|
## 17. Resource allocation and backpressure
|
|
|
|
### Memory
|
|
|
|
Decoded images, not file sizes, determine memory pressure. A compressed 100 MB image
|
|
can require far more RAM when decoded.
|
|
|
|
- reject or specially queue images exceeding configurable pixel/dimension limits;
|
|
- use Pillow decompression-bomb protection and treat warnings as reviewable errors;
|
|
- decode thumbnails directly to bounded target sizes where decoder support permits;
|
|
- avoid retaining full pixel arrays after hashing/thumbnail generation;
|
|
- limit concurrent decoded-pixel budget, not only worker count;
|
|
- isolate crash-prone/large decoders in worker processes so memory is reclaimed.
|
|
|
|
### Disk
|
|
|
|
- thumbnail cache has a configurable quota and LRU eviction;
|
|
- active thumbnails are pinned while a review page references them;
|
|
- temporary files live beside their destination when atomic rename is required;
|
|
- startup cleans only recognized stale temporary files, never arbitrary files;
|
|
- database, WAL, logs, raw responses, backups, and thumbnails expose separate size
|
|
metrics and low-disk warnings;
|
|
- mutating stages stop before the disk reaches a safety reserve.
|
|
|
|
### CPU, GPU, and network
|
|
|
|
- CPU pools default to a conservative fraction of available cores;
|
|
- GPU inference has an explicit single-owner semaphore unless proven safe otherwise;
|
|
- network concurrency and requests-per-minute are separate controls;
|
|
- retry traffic consumes the same rate-limit budget as first attempts;
|
|
- file descriptor use is bounded by closing images/subprocess pipes promptly;
|
|
- subprocess stdout is streamed with a size cap while the complete raw report may be
|
|
stored in a rotated file when required.
|
|
|
|
---
|
|
|
|
## 18. Testing strategy
|
|
|
|
Testing must prove crash safety and concurrency behavior, not only successful API
|
|
responses. Tests use synthetic images and temporary directories; they never access the
|
|
real photo library or `_IGNORE/` contents.
|
|
|
|
### Deterministic fixture photo library
|
|
|
|
Maintain a small, generated, non-private, source-controlled fixture specification from
|
|
which the test harness creates a fresh photo library for each test. Do not depend on
|
|
the developer's photo library, network downloads, current time, random model output,
|
|
filesystem enumeration order, or mutable shared fixture directories. If binary
|
|
fixtures are committed, they must be redistributable and accompanied by their source
|
|
and license; prefer reproducible generation where format support permits it.
|
|
|
|
The fixture library must include enough deliberately distinct files and folders to
|
|
exercise every workflow state:
|
|
|
|
- root-level photos, nested albums, empty folders, similarly named folders, Unicode
|
|
and whitespace in names, long names, mixed-case extensions, and names that collide
|
|
on case-insensitive or Unicode-normalizing filesystems;
|
|
- an exclusion-policy sentinel whose identifier is asserted to be absent from scan
|
|
results, counts, logs, provider calls, thumbnails, and upload requests; tests must
|
|
never open or inspect excluded content to prove exclusion;
|
|
- unique synthetic scenes with known visible properties, including zero, one, and
|
|
multiple people; indoor/outdoor; day/night; readable synthetic signs; portrait and
|
|
landscape orientation; transparency; and simple SFW/NSFW-classification fixtures;
|
|
- exact byte copies, renamed copies, same pixels with different metadata, EXIF-only
|
|
changes, resized/recompressed variants, orientation variants, crops, bursts,
|
|
watermarks, color edits, blank images, and intentionally similar non-duplicates;
|
|
- JPEG, PNG, WebP, TIFF, and HEIC/HEIF where the pinned CI decoder supports them, plus
|
|
uppercase extensions, unsupported extensions, corrupt/truncated images, zero-byte
|
|
files, and decompression-bomb/huge-dimension headers that remain safe to test;
|
|
- files with no EXIF, representative existing user EXIF, previous safety keywords,
|
|
previous analysis metadata, conflicting managed fields, malformed metadata, known
|
|
timestamps, and timezone/offset edge cases;
|
|
- read-only, missing, externally moved, externally copied, replaced-after-scan, and
|
|
symlink-escape cases created by the harness only on platforms that support them;
|
|
- album/upload/archive cases for valid names, invalid names, collisions, offline
|
|
archive media, insufficient capacity, already-uploaded bytes, changed-after-upload
|
|
bytes, and restore destination collisions.
|
|
|
|
A versioned machine-readable manifest is the authority for the corpus. For every
|
|
fixture it records a stable logical ID (never an absolute path), relative path, role,
|
|
generation recipe or source/license, byte SHA-256, decoded dimensions/orientation,
|
|
relevant metadata, duplicate-family membership and confidence class, expected stage
|
|
eligibility, and expected fake-provider outcomes. Expected results are keyed by logical
|
|
ID so tests remain valid after copying the corpus into a random temporary directory.
|
|
Hash and thumbnail goldens also record algorithm, decoder, encoder, and fixture-schema
|
|
versions; intentional changes require an explicit golden update and review.
|
|
|
|
Fixture generation must be deterministic: pin image-library/codec versions in CI;
|
|
use fixed seeds for pixels and generated names; fix clock, timezone, and locale; write
|
|
explicit timestamps instead of using creation time; sort all discovered paths; avoid
|
|
lossy re-encoding unless its exact output is committed and checksummed; and skip a
|
|
format with an explicit platform marker when byte-identical generation cannot be
|
|
guaranteed. Tests compare semantic pixels/metadata where cross-platform encoders are
|
|
not byte-stable.
|
|
|
|
Deterministic fakes map stable fixture IDs to fixed vision JSON, safety scores,
|
|
`exiftool` behavior, and `immich-go` reports. The manifest includes success, delay,
|
|
rate-limit, malformed-response, retry, cancellation, and uncertain-outcome scripts.
|
|
Timing-sensitive tests synchronize on observable events or fault-controller barriers,
|
|
never sleeps. Each test clones or regenerates the minimal named fixture subset, and
|
|
the harness verifies its checksums before startup, resets the database/cache/archive,
|
|
and fails on undeclared files or unexpected provider/uploader calls.
|
|
|
|
Keep the default corpus small enough for every change while retaining a separate
|
|
extended corpus for format/platform and soak coverage. CI must provide one fixture
|
|
validation command that regenerates the corpus twice, compares manifests/checksums,
|
|
validates all expected relationships, and proves that a complete test run leaves the
|
|
source corpus unchanged.
|
|
|
|
#### Concrete fixture-corpus TODO checklist
|
|
|
|
Build the default corpus below as the first implementation target. Counts describe
|
|
logical source images; explicitly listed copies/variants are additional files. All
|
|
people and sensitive-content fixtures must be synthetic, illustrated, generated, or
|
|
properly licensed adult-only material—never personal photos and never minors in any
|
|
sexualized or ambiguous context.
|
|
|
|
**Folder topology**
|
|
|
|
- [ ] Create `00_root/` behavior with 3 supported images placed directly at the
|
|
library root: 1 JPEG, 1 PNG with transparency, and 1 uppercase `.JPG`.
|
|
- [ ] Create 3 ordinary album folders with distinct conditions:
|
|
`01_family_day/` (outdoor/day/people), `02_city_night/`
|
|
(outdoor/night/readable sign), and `03_home_indoor/`
|
|
(indoor/mixed lighting/no location hint), with 5 unique source images each.
|
|
- [ ] Create 3 nested leaf albums under one parent, with 2 unique images per leaf, to
|
|
prove leaf-album counts are not collapsed into their parent.
|
|
- [ ] Create 3 naming-edge folders containing 1 image each: one with spaces, one with
|
|
Unicode characters, and one mixed-case name. Add a pair whose names differ only by
|
|
case and a pair using composed/decomposed Unicode for collision simulation.
|
|
- [ ] Create 1 empty album folder and verify it is handled consistently and never
|
|
invents an asset.
|
|
- [ ] Create the exclusion-policy sentinel directory and assert only from outside the
|
|
boundary that it is never traversed, counted, logged, thumbnailed, sent to a
|
|
provider, or uploaded.
|
|
|
|
**Known visual content**
|
|
|
|
- [ ] Create 5 deterministic NSFW-class fixtures spanning the fake classifier's score
|
|
bands: 2 clearly NSFW, 1 boundary/review-required, and 2 clearly SFW controls that
|
|
are visually similar enough to catch simplistic color/skin heuristics.
|
|
- [ ] Create 5 additional SFW fixtures: 1 landscape with no people, 1 single adult,
|
|
1 group of 3 adults, 1 indoor pet, and 1 document/screenshot with readable synthetic
|
|
text.
|
|
- [ ] Across the ordinary albums, include at least 3 portrait, 3 landscape, and 2
|
|
square images; 3 indoor and 3 outdoor scenes; and morning, afternoon, evening,
|
|
night, and unknown time-of-day expectations.
|
|
- [ ] Include exactly 3 synthetic location-hint cases: 1 readable street sign, 1
|
|
unmistakable generated landmark, and 1 ambiguous sign that must yield no confident
|
|
location.
|
|
|
|
**Duplicate and similarity families**
|
|
|
|
- [ ] Family A: 1 canonical JPEG plus 2 byte-identical copies with different paths.
|
|
- [ ] Family B: 1 canonical image plus 2 files with identical decoded pixels but
|
|
different EXIF/timestamps.
|
|
- [ ] Family C: 1 canonical plus 3 near-duplicates: resized, JPEG-recompressed, and
|
|
WebP-converted.
|
|
- [ ] Family D: 1 canonical plus 2 orientation variants: EXIF rotation and physically
|
|
rotated pixels.
|
|
- [ ] Family E: 1 canonical plus 3 review-required variants: crop, watermark, and
|
|
color edit.
|
|
- [ ] Create 3 intentionally similar non-duplicate pairs: blank/near-blank images,
|
|
burst-like frames with a meaningful subject change, and two screenshots sharing the
|
|
same layout. Expected decisions must be `not_duplicate` or `review_required`, never
|
|
automatic merge.
|
|
|
|
**Formats and damaged inputs**
|
|
|
|
- [ ] Include at least 3 JPEG, 3 PNG, 2 WebP, 2 TIFF, and 2 HEIC/HEIF files in the
|
|
extended corpus; keep one supported example of each CI-stable format in the default
|
|
corpus.
|
|
- [ ] Add 1 unsupported-extension file, 1 zero-byte file, 1 truncated JPEG, 1 corrupt
|
|
PNG, and 1 safe synthetic huge-dimension/decompression-bomb header.
|
|
- [ ] Add 1 grayscale, 1 transparent RGBA, 1 wide-gamut/profile-bearing, and 1 very
|
|
small image to exercise decoding and thumbnail normalization.
|
|
|
|
**Metadata and time cases**
|
|
|
|
- [ ] Create 2 files without EXIF, 2 with ordinary user EXIF that must survive, 2 with
|
|
existing safety keywords, 2 with existing analysis fields, 1 with conflicting
|
|
managed fields, and 1 with malformed metadata.
|
|
- [ ] Create 5 fixed capture-time cases: UTC, positive offset, negative offset,
|
|
daylight-saving boundary, and missing timezone. Freeze filesystem mtimes separately
|
|
so capture-time fallback behavior is explicit.
|
|
- [ ] Create 1 file whose EXIF write changes its byte hash without changing decoded
|
|
pixels, then assert upload uses the post-checkpoint hash.
|
|
|
|
**Runtime mutations produced by the harness**
|
|
|
|
- [ ] From clean source fixtures, create one moved file, one copied file, one renamed
|
|
folder, one file replaced after scan, one file modified after upload, and one file
|
|
removed after scan.
|
|
- [ ] Where supported, create one read-only file, one read-only destination folder,
|
|
one safe in-library symlink, and one symlink escape attempt.
|
|
- [ ] Create 3 rename-plan cases: valid multi-file rename, destination collision, and
|
|
case-only rename; add one stale plan by modifying a source after validation.
|
|
- [ ] Create 4 fake-upload results: new upload, exact duplicate, server upgrade, and
|
|
accepted-but-response-lost uncertainty.
|
|
- [ ] Create 3 archive cases: successful copy/verify, insufficient capacity, and
|
|
interruption after verification but before source removal; add one offline restore
|
|
and one restore-destination collision.
|
|
|
|
**Manifest and acceptance totals**
|
|
|
|
- [ ] Give every file a stable logical ID and manifest entry; declare its expected
|
|
discovery status, safety class, analysis response, duplicate family/decision, EXIF
|
|
projection, album membership, and upload/archive eligibility.
|
|
- [ ] Record the exact expected counts after each workflow stage, including canonical,
|
|
duplicate, review-required, excluded, error, analyzed, EXIF-verified, uploaded, and
|
|
archived totals. Tests must compare the complete count object, not selected fields.
|
|
- [ ] Add a corpus-lint test that fails on missing IDs, duplicate IDs, undeclared
|
|
files, checksum drift, inconsistent family membership, missing expected outcomes,
|
|
or a fixture not exercised by at least one automated test.
|
|
- [ ] Add a reproducibility test that generates the corpus twice in separate temporary
|
|
roots and compares manifests, relative paths, semantic metadata, and checksums where
|
|
byte stability is promised.
|
|
|
|
### Unit tests
|
|
|
|
- path-policy normalization, symlink escape rejection, and `_IGNORE/` exclusion;
|
|
- normalized pixel hashing across EXIF-only changes and orientation variants;
|
|
- pHash distance/confidence classification and negative-link behavior;
|
|
- canonical recommendation scoring;
|
|
- stage-gate evaluation and blocker explanations;
|
|
- state-machine transitions for jobs, renames, uploads, and duplicate decisions;
|
|
- idempotency-key and optimistic-version handling;
|
|
- naming sanitization across macOS/Linux and case-insensitive collision simulation;
|
|
- `immich-go` parser fixtures for every supported version and unknown output.
|
|
|
|
Use property-based tests for path transformations, rename plans, hash grouping, state
|
|
machines, and arbitrary API payloads. Important properties include: paths never escape
|
|
the library, canonical clusters never contain cycles, retries do not duplicate events,
|
|
and applying a valid rename plan preserves the asset set.
|
|
|
|
### Golden-image tests
|
|
|
|
Maintain a small generated/non-private corpus containing:
|
|
|
|
- exact copies with different names;
|
|
- identical pixels with different EXIF;
|
|
- resized and recompressed copies;
|
|
- rotations represented by EXIF and by transformed pixels;
|
|
- crops, watermarks, color edits, screenshots, blank images, and burst-like images;
|
|
- JPEG, PNG, WebP, TIFF, HEIC fixtures where licensing permits;
|
|
- corrupt, truncated, huge-dimension, and unsupported files.
|
|
|
|
Assert hashes, confidence bands, thumbnail orientation/dimensions, and which cases must
|
|
require human review. Do not assert that fuzzy hashing is universally correct.
|
|
|
|
### Repository and API integration tests
|
|
|
|
- run every migration from an empty DB and from anonymized snapshots of each supported
|
|
historical schema;
|
|
- verify foreign keys, FTS rebuilds, WAL settings, online backup, and integrity checks;
|
|
- test API status codes, schemas, pagination, idempotency, stale versions, CSRF/Origin,
|
|
path validation, and event-stream reconnect behavior;
|
|
- replace vision, NSFW, `exiftool`, and `immich-go` integrations with deterministic
|
|
fakes that can succeed, delay, emit malformed output, or fail at chosen points.
|
|
- snapshot representative EXIF before and after every stage; assert only owned fields
|
|
change, previous-stage values survive, read-back matches the desired projection, and
|
|
the stored post-write hashes match the actual file bytes.
|
|
|
|
### Black-box API end-to-end tests
|
|
|
|
API end-to-end tests must not import service or repository internals. They launch the
|
|
real FastAPI server and worker as child processes, create a temporary application-data
|
|
directory and photo library, and communicate only over HTTP/SSE. This validates the
|
|
same process boundaries used in production.
|
|
|
|
Recommended harness:
|
|
|
|
- `pytest` owns a session-scoped temporary directory and allocates free ports;
|
|
- Alembic creates a fresh SQLite database through the production startup path;
|
|
- the API and worker receive an isolated test configuration through environment vars;
|
|
- deterministic fake executables/providers replace `exiftool`, the vision API,
|
|
NSFW inference, and `immich-go`, but are invoked through the real integration layer;
|
|
- readiness waits on `/api/v1/health/ready`, never an arbitrary sleep;
|
|
- tests use an HTTP client and consume SSE events with reconnect support;
|
|
- processes are terminated and checked for leaked children after each test session.
|
|
|
|
Do not add a powerful test-only mutation API to production. Fixtures are prepared
|
|
before process startup through files, migrations, and fake integrations. When runtime
|
|
fault control is required, use a separate fake-provider control server bound to its
|
|
own random localhost port and enabled only by test configuration.
|
|
|
|
Core API scenarios:
|
|
|
|
1. **Fresh workflow** — scan, wait for job completion, review duplicate cluster, mark
|
|
safety decisions, verify the safety EXIF checkpoint, analyze, verify the analysis
|
|
EXIF checkpoint and preservation snapshot, generate/apply rename plan, upload,
|
|
archive, restart, and verify the offline asset remains searchable/deduplicable.
|
|
2. **Move and copy outside the app** — stop jobs, manually move A→B and copy B→C,
|
|
rescan, verify the original asset ID survives and C becomes a duplicate candidate.
|
|
3. **Idempotent commands** — submit the same idempotency key concurrently and assert
|
|
one job/decision plus identical responses.
|
|
4. **Optimistic conflict** — submit a decision with an old asset/cluster version and
|
|
require `409 Conflict` without mutation.
|
|
5. **Cancellation and resume** — cancel each job type mid-item, restart processes,
|
|
and verify a valid resumable/terminal state.
|
|
6. **Lease recovery** — kill the worker, let its lease expire, start another worker,
|
|
then prove the old worker cannot commit with its stale fencing token.
|
|
7. **Safety race** — change an asset to NSFW while an analysis fake is delayed; the
|
|
eventual analysis result must be discarded and must not write analysis EXIF. The
|
|
asset remains upload-eligible after its NSFW EXIF keyword is verified.
|
|
8. **Rename uncertainty** — change a source after plan validation and assert apply is
|
|
rejected without moving any unaffected paths.
|
|
9. **Unknown upload result** — fake server acceptance followed by process failure;
|
|
require `unknown_requires_verification`, never automatic retry/success.
|
|
10. **Privacy boundary** — verify `_IGNORE/` identifiers never appear in provider or
|
|
uploader logs; NSFW identifiers never appear in analysis-provider logs but do
|
|
appear in uploader logs when their album is explicitly uploaded.
|
|
11. **External EXIF edit** — add user metadata after a verified checkpoint, rescan, and
|
|
verify the next stage preserves the addition; remove or conflict with a managed
|
|
value and require `divergent` instead of silent repair/upload.
|
|
12. **Archive interruption** — terminate during transfer, after archive verification,
|
|
and after active-source removal; restart and assert no verified source is lost or
|
|
removed twice.
|
|
13. **Offline deduplication** — unmount the fake archive, discover an exact and a fuzzy
|
|
active copy, and verify hash matching plus protected-thumbnail review without
|
|
treating the archived canonical as unexpectedly missing.
|
|
|
|
For every scenario, assert both API state and durable state after a full server/worker
|
|
restart. An HTTP `200` before restart is not sufficient evidence of durability.
|
|
|
|
API contract tests also validate the generated OpenAPI schema. Breaking response or
|
|
enum changes require an explicit API version/migration decision rather than silently
|
|
breaking the frontend.
|
|
|
|
### Playwright browser end-to-end tests
|
|
|
|
Playwright tests run against the same real API/worker test stack. They validate user
|
|
journeys and visible safety controls, while API tests remain responsible for exhaustive
|
|
state combinations. Browser tests should not duplicate every backend assertion.
|
|
|
|
Use stable accessible locators first (`getByRole`, labels, names) and add
|
|
`data-testid` only for dynamic structures without reliable semantics. Never locate by
|
|
CSS styling classes or incidental text generated by AI fixtures.
|
|
|
|
Critical browser journeys:
|
|
|
|
1. **Workflow overview** — stage counts render, blockers are understandable, and
|
|
disabled actions explain their prerequisite.
|
|
2. **Duplicate comparison** — thumbnails load in the correct orientation; keyboard
|
|
navigation, synchronized zoom, blink/overlay, metadata evidence, and canonical
|
|
selection work; fuzzy matches require explicit confirmation.
|
|
3. **Safety review** — individual and bulk SFW/NSFW decisions persist after reload;
|
|
filters/counts update; an NSFW asset disappears from analysis-eligible totals while
|
|
remaining visible in upload-eligible totals.
|
|
4. **Analysis job** — start, live progress via SSE, reconnect after page reload, cancel,
|
|
resume, and inspect an error without losing completed work.
|
|
5. **Album proposal** — edit a suggestion, detect collision/invalid name, preview the
|
|
complete path plan, and verify stale confirmation is rejected visibly.
|
|
6. **Rename recovery** — simulate a controlled interrupted rename, reopen the UI, and
|
|
confirm the recovery screen prevents unrelated mutations.
|
|
7. **Upload preflight** — blocked reasons render, secrets never appear in DOM/logs,
|
|
exact album scope is shown, and uncertain results require verification.
|
|
8. **Move/copy reconciliation** — after the harness changes files externally, rescan
|
|
from the UI and resolve the newly formed duplicate cluster.
|
|
9. **Archive and restore** — preview reclaimable bytes, inspect blockers, confirm an
|
|
archive plan, observe progress/recovery, browse the offline asset from its retained
|
|
thumbnail, and restore it when the fake archive location is mounted again.
|
|
10. **Accessibility and keyboard** — complete duplicate and safety decisions without a
|
|
mouse; focus remains visible; dialogs trap/restore focus; status is not color-only.
|
|
11. **Responsive layout** — desktop is primary, but tablet/narrow widths retain access
|
|
to blockers, decisions, logs, and confirmations without horizontal action loss.
|
|
|
|
Playwright execution policy:
|
|
|
|
- each test gets a fresh browser context and deterministic seeded library state;
|
|
- tests wait for semantic UI state or API responses, never fixed timeouts;
|
|
- external provider calls are observed at the fake integration boundary, not mocked
|
|
inside the browser, so frontend-to-backend behavior remains real;
|
|
- use Playwright's API request context only for setup when the action is also available
|
|
through the public API and does not bypass the behavior under test;
|
|
- run Chromium on every change; run Firefox/WebKit on scheduled or release builds;
|
|
- collect trace, console, network log, screenshot, and video on first retry/failure;
|
|
- fail on uncaught page errors, unexpected console errors, failed API responses, and
|
|
unhandled dialogs;
|
|
- visual snapshots cover only stable layouts/components, with deterministic fonts,
|
|
viewport, timezone, locale, animation disabling, and generated thumbnails;
|
|
- accessibility scans complement—not replace—keyboard journey assertions.
|
|
|
|
Avoid large all-in-one browser tests. Each journey should establish state through the
|
|
API, perform the behavior under test through the UI, and assert the visible result plus
|
|
one authoritative API outcome. Keep one longer smoke journey for the complete pipeline.
|
|
|
|
### Test-stack topology
|
|
|
|
```text
|
|
pytest / Playwright
|
|
│
|
|
├── browser ───────────────► real FastAPI + static frontend
|
|
│ │
|
|
├── HTTP/SSE client ───────────────┤
|
|
│ ▼
|
|
│ isolated SQLite DB
|
|
│ ▲
|
|
└── fixture/fault controller │
|
|
│ real worker process
|
|
└── fake vision / NSFW / exiftool / immich-go
|
|
```
|
|
|
|
The harness must expose a single command for local and CI use, for example:
|
|
|
|
```text
|
|
pytest tests/e2e/api
|
|
playwright test tests/e2e/web
|
|
pytest tests/e2e/full_pipeline
|
|
```
|
|
|
|
Exact commands depend on the final frontend tooling, but both suites must provision
|
|
the same stack through one reusable test-harness module.
|
|
|
|
### Concurrency and race tests
|
|
|
|
Run repeatedly with randomized timing:
|
|
|
|
- many hash/thumbnail workers completing simultaneously into the DB writer queue;
|
|
- two workers attempting to claim the same job;
|
|
- lease expiry followed by a late commit from the old worker;
|
|
- user moves/replaces a file between validation and EXIF write;
|
|
- thumbnail request races with rename and cache eviction;
|
|
- analysis result races with a safety decision changing to NSFW;
|
|
- upload preparation races with an EXIF update;
|
|
- two rename plans target the same destination;
|
|
- long UI reads create WAL checkpoint pressure;
|
|
- cancellation arrives during decoding, API retry, EXIF write, rename, and upload.
|
|
|
|
Assertions include no deadlocks, bounded queue depth, no duplicate terminal state, no
|
|
lost audit events, no writes from stale fencing tokens, and eventual database integrity.
|
|
|
|
### Crash/fault-injection tests
|
|
|
|
Use real child processes and terminate them at every persisted transition:
|
|
|
|
- after journal write but before rename;
|
|
- after filesystem rename but before DB update;
|
|
- during EXIF subprocess execution;
|
|
- after upload acceptance but before report persistence;
|
|
- while a DB writer batch is committing;
|
|
- while thumbnail temporary files exist.
|
|
|
|
Restart the application and assert it either resumes safely or presents a precise
|
|
manual-recovery state. Never accept silent guessing as a passing result.
|
|
|
|
Fault injection should also cover disk full, read-only directories, permission changes,
|
|
database busy timeouts, corrupt DB copies, network timeouts, malformed model JSON, GPU
|
|
out-of-memory, subprocess hangs, and unavailable binaries.
|
|
|
|
### End-to-end workflow tests
|
|
|
|
Build a temporary library, including an `_IGNORE/` sentinel, then run:
|
|
|
|
```text
|
|
scan → duplicate decisions → safety review → analysis → EXIF verification
|
|
→ album proposal → rename → rescan/reconciliation → fake Immich upload
|
|
→ archive → offline deduplication → restore
|
|
```
|
|
|
|
Repeat after manually moving and copying files between stages. Assert stable asset IDs,
|
|
preserved decisions, duplicate exclusion, verified EXIF checkpoint order, upload
|
|
hashes, archive manifests, offline hash-index membership, successful restore, and that the
|
|
sentinel is never discovered, counted, opened, or modified.
|
|
|
|
### Load and soak tests
|
|
|
|
- synthetic databases at 25k, 100k, and 500k asset rows;
|
|
- duplicate clusters with thousands of members;
|
|
- sustained thumbnail browsing while analysis writes complete;
|
|
- multi-hour worker soak with cancellation/retry and periodic backups;
|
|
- cache eviction under quota pressure;
|
|
- memory/RSS, open-file, DB latency, WAL size, queue depth, and event throughput limits.
|
|
|
|
Define acceptance budgets before implementation, for example:
|
|
|
|
- API list/search p95 under 250 ms on 100k rows;
|
|
- no unbounded queue or cache growth;
|
|
- stable RSS during a multi-hour run;
|
|
- clean restart with no lost completed item after forced termination;
|
|
- zero automatic fuzzy-duplicate exclusions without recorded user approval.
|
|
|
|
### Release gates
|
|
|
|
1. Unit/property/golden suites pass.
|
|
2. Migration test succeeds from a copied current database.
|
|
3. Concurrency suite passes repeatedly with randomized scheduling.
|
|
4. Every rename crash point passes recovery tests.
|
|
5. Black-box API workflow passes through real server and worker processes, including a
|
|
restart after every completed stage.
|
|
6. Critical Playwright journeys pass in Chromium with no unexpected browser console,
|
|
network, accessibility, or page errors.
|
|
7. Scheduled cross-browser Playwright suite passes before a release.
|
|
8. Full end-to-end pipeline passes with fake external services and retained artifacts.
|
|
9. Archive crash/recovery suite proves that active sources are removed only after
|
|
verified durable copies and remain deduplicable while offline.
|
|
10. A read-only dry run against the real library produces an approved reconciliation
|
|
report before any mutation is enabled.
|
|
11. The donor ledger has no unresolved in-scope rows; characterization/parity tests
|
|
pass, intentional deltas are approved, and archived CLI sources are not imported
|
|
or executed by the production application.
|