US07-05: Deliver Backup and Operational Recovery (#92)

This commit was merged in pull request #92.
This commit is contained in:
2026-08-17 21:13:21 +02:00
parent d08d19c03c
commit 9851e112a9
19 changed files with 2057 additions and 20 deletions

View File

@@ -0,0 +1,42 @@
# US08-01 — Make the Trust Boundary Configurable and Authenticated
Epic: [E08](../E08-container-deployment.md)
As an operator, I want to reach the application through my own hostname without
weakening it, so a container behind a reverse proxy is as safe as the loopback
deployment it replaces.
## Context
`photo_pipeline/api/security.py` refuses any request whose `Host` or `Origin` is not
loopback. That check is the current stand-in for authentication: whoever can reach
`127.0.0.1:8000` is the owner. Behind a proxy the hostname is no longer loopback, so
relaxing the check without adding an authentication gate would publish the library.
## Acceptance criteria
- Allowed hosts and origins come from configuration (`PHOTO_PIPELINE_*`), default to
the current loopback set, and an unset configuration behaves exactly as today.
- Whenever a non-loopback host is configured, startup requires an access secret and
refuses to serve without one; loopback-only deployments keep working with no secret.
- The secret is exchanged for the existing session cookie and CSRF token through the
bootstrap endpoint; every protected route keeps its current session and CSRF
requirements unchanged.
- Forwarded headers (`X-Forwarded-Proto`, `X-Forwarded-Host`) are honored only from a
configured trusted proxy and ignored otherwise, so a client cannot forge its origin.
- Cookies are marked `Secure` when the effective external scheme is HTTPS.
- Failed authentication is rate-limited and logged without the secret, the session id,
or any request body.
- Health endpoints stay reachable without the secret; nothing else does.
## Automated tests
- Unit tests for host/origin evaluation across loopback default, configured host,
unconfigured host, forged forwarded headers, and trusted-proxy forwarded headers.
- Integration tests: startup refusal without a secret, successful exchange, wrong
secret, replay of an old session, cross-site request, and unauthenticated access to
every route class.
## Dependencies
- US07-02

View File

@@ -0,0 +1,41 @@
# US08-02 — Build a Reproducible Application Image
Epic: [E08](../E08-container-deployment.md)
As an operator, I want one image that can run either application role, so deployment is
a pull instead of a Python environment I have to reproduce by hand.
## Context
The application shells out to `exiftool` and `immich-go`, writes into the library as a
normal filesystem user, and serves a static frontend from `frontend/`. All three have to
be true inside the image, or the container starts and then fails on the first real
operation.
## Acceptance criteria
- A `Dockerfile` builds from a pinned Python base, installs the project and its runtime
dependencies, and contains no test, playwright, or build-only tooling in the final
layer.
- `exiftool` and `immich-go` are present at pinned versions, and their versions are
recorded in the image and reported by `python -m photo_pipeline diagnostics`.
- The image runs as a non-root user whose UID/GID are build-time arguments, so files
the application renames or writes keep the ownership the host library expects.
- One entrypoint selects the role: `serve` or `worker`, passing through the existing
CLI arguments; no supervisor runs two roles in one container.
- `serve` containers declare a `HEALTHCHECK` against `/api/v1/health/ready`, so an
unmigrated or misconfigured database is not reported healthy.
- The image contains no secrets, no library data, no database, and no `.git`; the build
context is constrained by `.dockerignore`.
- Image build is reproducible from a clean checkout and documented in `README.md`.
## Automated tests
- A build-and-run test asserts the image starts, reports ready, serves the frontend
index, and returns the pinned `exiftool` and `immich-go` versions.
- A test asserts the container refuses to run as UID 0 and that a file created by the
container is owned by the configured UID/GID.
## Dependencies
- US07-05

View File

@@ -0,0 +1,47 @@
# US08-03 — Compose the Runtime and Mount the Library Safely
Epic: [E08](../E08-container-deployment.md)
As an operator, I want a single compose file that runs the API and the worker against my
real library, so a deployment is one command and the safety invariants survive it.
## Context
The library process lock (US07-05) assumes both roles see the same lock file, and SQLite
in WAL mode assumes a real local filesystem. Container path policy is the same problem
as host path policy with a new failure mode: the configured library roots must name the
in-container mount paths, not the host paths.
## Acceptance criteria
- `docker-compose.yml` runs exactly one `serve` and one `worker` container from the same
image and the same data volume, and a second worker is refused by the existing lock
rather than by convention.
- The library is a bind mount; `PHOTO_PIPELINE_LIBRARY_ROOTS` names the container-side
paths, and a mismatch between mounted and configured roots fails at startup with a
clear message instead of at the first write.
- The data volume holds the database, WAL, thumbnail cache, and backups on a local
filesystem; the composition documents that a network mount is unsupported for it.
- Migrations run before `serve` and `worker` accept work, using the existing backup-then-
migrate path, and an upgrade that fails leaves the previous database intact.
- Configuration and secrets come from the environment, never from the image or a
committed file; a `.env.example` lists every `PHOTO_PIPELINE_*` variable with safe
defaults and no values.
- The API port is published to host loopback by default; exposing it publicly requires
the configured hostname and access secret from US08-01.
- Containers restart automatically, and a restart mid-job resumes exactly as a host
restart does today.
- Backup, verify-backup, restore, and diagnostics are documented as container commands
and work against the mounted volumes.
## Automated tests
- An integration test brings the composition up against a temporary fixture library,
runs a job, restarts both containers, and asserts the job resumes and the database is
intact.
- Tests for: second worker refused, library-root mismatch refused at startup, failed
migration leaving the previous database restorable.
## Dependencies
- US08-01, US08-02

View File

@@ -0,0 +1,40 @@
# US08-04 — Publish and Deploy from Gitea Actions
Epic: [E08](../E08-container-deployment.md)
As a release owner, I want `main` to build, publish, and redeploy the image
automatically, so deployment is the same reproducible path every time.
## Context
The workflow is adapted from the `crowdsec-admin` deployment workflow
(`.gitea/workflows/deploy.yml` in that repository): build, log in to the Gitea registry,
push, trigger a Portainer webhook, prune. This project needs the same shape plus a test
gate, because unlike that project it has a required suite that must not be skipped.
## Acceptance criteria
- `.gitea/workflows/` contains a test workflow that runs on pull requests and on `main`,
executing the configured required suites, and a deploy workflow that runs only after
the tests pass on `main` and on manual dispatch.
- The deploy workflow publishes to `git.domverse-berlin.eu` under this project's own
image path, tagged `latest` and the commit SHA, so a rollback is a tag change.
- Registry credentials and the Portainer webhook come from repository secrets; runtime
secrets (vision key, Immich key, access secret) stay in the Portainer stack and never
enter the repository or the image.
- Redeploy is triggered by webhook and the workflow fails when the webhook call fails.
- Dangling images are pruned; published tags are not.
- A concurrency guard prevents two deploys of different commits overlapping.
- `README.md` documents the required secrets, the image path, the rollback procedure,
and that the stack is managed by Portainer from git.
## Automated tests
- Workflow files are validated (syntax and required job/step names) by a repository test
so a rename cannot silently disable the test gate.
- A dry-run job builds and pushes to a scratch tag on manual dispatch without touching
`latest` or triggering a redeploy.
## Dependencies
- US08-02, US08-03

View File

@@ -0,0 +1,30 @@
# US08-05 — Automate Container Deployment Acceptance
Epic: [E08](../E08-container-deployment.md)
As a release owner, I want one automated gate that proves the deployed container, so the
packaged application is verified the same way the host application is.
## Acceptance criteria
- One documented command provisions the composition from the built image against a
temporary fixture library and an isolated data volume, and destroys it afterwards.
- A browser journey against the containerized application covers discovery, duplicate
review, analysis, album proposal, rename, upload preflight, and archive views.
- An upgrade journey runs the previous published image, then the new one, and asserts
migrations, journals, jobs, and the thumbnail cache survive.
- A restart journey kills both containers mid-job and asserts resume without duplicate
side effects.
- Security gates run against the deployed instance: unauthenticated access refused,
forged forwarded headers refused, paths outside the mounted library roots refused, and
no secret in container logs.
- Evidence is retained per run and the gate fails on any skipped required check.
## Automated tests
- The container acceptance suite runs on a `phase_h` marker in CI on `main` and before a
published deploy; earlier epic suites keep running unchanged.
## Dependencies
- US08-01 through US08-04