US09-05: Automate Documentation Acceptance #111
@@ -101,3 +101,39 @@ jobs:
|
||||
with:
|
||||
name: container-gate-${{ gitea.sha }}
|
||||
path: gate-evidence
|
||||
|
||||
# The manuals, checked against the application they describe (US09-05). Runs on
|
||||
# every pull request as well as `main`: documentation drifts by the same commits
|
||||
# that change behaviour, and telling the author while the change is still open is
|
||||
# the only time the fix is cheap.
|
||||
documentation:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Install the application and its test dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
python -m pip install -e '.[test]'
|
||||
python -m playwright install --with-deps chromium
|
||||
|
||||
- name: Documentation gate
|
||||
# One command: every phase_i check, offline and in the browser, with the
|
||||
# evidence retained. It accepts no skip — a check that did not run is a page
|
||||
# nobody compared with the code.
|
||||
env:
|
||||
PHOTO_PIPELINE_DATA_DIR: ${{ gitea.workspace }}/docs-gate-data
|
||||
run: python -m photo_pipeline docs-gate --output docs-evidence
|
||||
|
||||
- name: Keep the evidence
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: docs-gate-${{ gitea.sha }}
|
||||
path: docs-evidence
|
||||
|
||||
18
README.md
@@ -4,6 +4,24 @@ Integrated, restart-safe photo analysis, duplicate review, metadata, upload, and
|
||||
archive workflow. Planning lives in `INTEGRATED_PIPELINE_CONCEPT.md` and
|
||||
`delivery_backlog/`.
|
||||
|
||||
## Documentation
|
||||
|
||||
The manuals live in [`docs/`](docs/index.md) and are the same files the running
|
||||
application serves at `/app/#/docs`:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| [Overview](docs/overview.md) | what it does and the rules it will not break |
|
||||
| [Installation and operations](docs/installation.md) | host and container install, every setting, backup and restore, every refusal |
|
||||
| [A guided first pass](docs/first-pass.md) | one library from scan to verified upload |
|
||||
| [The stages](docs/index.md) | one illustrated page each, from inventory to archive |
|
||||
| [Errors and refusals](docs/errors.md) | every error code, cause, and remedy |
|
||||
| [Architecture](docs/architecture.md) | diagrams, module map, journals, invariants |
|
||||
|
||||
This README stays the repository's own notes — how the code is built, tested, and
|
||||
released. Anything an operator or a user needs belongs in `docs/`, and one gate
|
||||
(`python -m photo_pipeline docs-gate`) keeps those pages honest against the code.
|
||||
|
||||
## Application (`photo_pipeline`)
|
||||
|
||||
The target application lives in `photo_pipeline/` (FastAPI + SQLAlchemy + Alembic).
|
||||
|
||||
|
Before Width: | Height: | Size: 64 KiB After Width: | Height: | Size: 64 KiB |
|
Before Width: | Height: | Size: 127 KiB After Width: | Height: | Size: 128 KiB |
|
Before Width: | Height: | Size: 56 KiB After Width: | Height: | Size: 55 KiB |
|
Before Width: | Height: | Size: 30 KiB After Width: | Height: | Size: 78 KiB |
|
Before Width: | Height: | Size: 127 KiB After Width: | Height: | Size: 33 KiB |
|
Before Width: | Height: | Size: 101 KiB After Width: | Height: | Size: 101 KiB |
@@ -42,6 +42,25 @@ what it changes on disk or on the server, and what it refuses.
|
||||
- [Recovery](recovery.md) — what the application resolves by itself, and what needs
|
||||
you.
|
||||
|
||||
## Keeping these pages true
|
||||
|
||||
Documentation that disagrees with the application is worse than none, because it is
|
||||
trusted. So one command compares them:
|
||||
|
||||
```bash
|
||||
python -m photo_pipeline docs-gate
|
||||
```
|
||||
|
||||
It runs every `phase_i` check — dead links and anchors, pages nobody links to, images
|
||||
nobody shows, settings, commands, exit codes, error codes, and states that no longer
|
||||
exist in the code, the pages rendering in a real browser without a console or policy
|
||||
error, and the screenshots still showing what the application shows. It retains its
|
||||
evidence and **accepts no skipped check**: a check that did not run is a page nobody
|
||||
compared. CI runs it on every pull request.
|
||||
|
||||
The repository's own build, test, and release notes live in the
|
||||
[README](../README.md).
|
||||
|
||||
## Conventions
|
||||
|
||||
A page tells you what a stage **changes on disk or on the server** before it tells
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
"""Application management CLI:
|
||||
``python -m photo_pipeline {serve,migrate,worker,import-legacy-scores,backup,verify-backup,restore,diagnostics,benchmark,release-gate,container-gate,dry-run,approve-dry-run}``.
|
||||
``python -m photo_pipeline {serve,migrate,worker,import-legacy-scores,backup,verify-backup,restore,diagnostics,benchmark,release-gate,container-gate,docs-gate,dry-run,approve-dry-run}``.
|
||||
|
||||
``serve`` and ``worker`` take the library process lock for their role (US07-05):
|
||||
two workers, or the frozen CLI running beside the app, would each be safe on their
|
||||
@@ -78,6 +78,12 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||
container_cmd.add_argument(
|
||||
"--output", help="Evidence directory (default: data/container-gate/<stamp>)"
|
||||
)
|
||||
docs_cmd = commands.add_parser(
|
||||
"docs-gate",
|
||||
help="Check the manuals against the application they describe and keep the "
|
||||
"evidence (US09-05)",
|
||||
)
|
||||
docs_cmd.add_argument("--output", help="Evidence directory (default: data/docs-gate/<stamp>)")
|
||||
dry_cmd = commands.add_parser(
|
||||
"dry-run", help="Read-only reconciliation of the configured library (US07-07)"
|
||||
)
|
||||
@@ -188,6 +194,30 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||
)
|
||||
return 0 if report["ok"] else 1
|
||||
|
||||
if args.command == "docs-gate":
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from photo_pipeline.services import release
|
||||
|
||||
# Documentation that disagrees with the application is worse than none: it is
|
||||
# trusted. So this gate accepts no skip either — a check that did not run is a
|
||||
# page nobody compared with the code.
|
||||
output = args.output or Path(config.data_dir) / "docs-gate" / datetime.now(
|
||||
timezone.utc
|
||||
).strftime("%Y%m%dT%H%M%SZ")
|
||||
report = release.run_gate(
|
||||
config,
|
||||
output=output,
|
||||
stages=release.DOCS_STAGES,
|
||||
allowed_skips=release.DOCS_ALLOWED_SKIP_REASONS,
|
||||
)
|
||||
print(
|
||||
json.dumps(
|
||||
{k: v for k, v in report.items() if k not in ("stages", "matrix")}, indent=2
|
||||
)
|
||||
)
|
||||
return 0 if report["ok"] else 1
|
||||
|
||||
if args.command == "dry-run":
|
||||
from photo_pipeline.services import release
|
||||
|
||||
|
||||
@@ -71,6 +71,18 @@ CONTAINER_STAGES: tuple[tuple[str, tuple[str, ...]], ...] = (
|
||||
# would skip here — no daemon, no compose, no browser — means it was not proven.
|
||||
CONTAINER_ALLOWED_SKIP_REASONS: tuple[str, ...] = ()
|
||||
|
||||
# The documentation gate (US09-05): the manuals, held to the application. Selected by
|
||||
# marker across the whole suite, because each check lives beside what it describes —
|
||||
# the offline contracts under tests/integration, the browser and screenshot journeys
|
||||
# under tests/e2e. Adding a phase_i test is enough to put it in front of a release.
|
||||
DOCS_STAGES: tuple[tuple[str, tuple[str, ...]], ...] = (
|
||||
("documentation", ("tests", "-m", "phase_i")),
|
||||
)
|
||||
|
||||
# Also nothing. Every check here is either offline or needs the browser the rest of
|
||||
# the suite already needs, so a skip means the manuals were not compared with the code.
|
||||
DOCS_ALLOWED_SKIP_REASONS: tuple[str, ...] = ()
|
||||
|
||||
|
||||
class ReleaseError(RuntimeError):
|
||||
pass
|
||||
|
||||
@@ -54,4 +54,5 @@ markers = [
|
||||
"phase_f: Phase F end-to-end acceptance (US06-06) — archive destination, transfer, and restore journeys",
|
||||
"container: builds and runs the container image and its composition (US08-02, US08-03) — needs a Docker daemon, the compose plugin, and the network",
|
||||
"phase_h: Phase H container deployment acceptance (US08-05) — the browser, upgrade, restart, and security journeys against the composed stack",
|
||||
"phase_i: Phase I documentation acceptance (US09-05) — the manuals checked against the application, in the browser and offline",
|
||||
]
|
||||
|
||||
@@ -18,6 +18,9 @@ import pytest
|
||||
|
||||
from tests.e2e._pipeline_harness import Server, seed_library
|
||||
|
||||
# Every check here belongs to the documentation gate (US09-05).
|
||||
pytestmark = pytest.mark.phase_i
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def server(tmp_path_factory):
|
||||
|
||||
@@ -24,10 +24,13 @@ from contextlib import closing
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
from playwright.sync_api import TimeoutError as PlaywrightTimeout
|
||||
|
||||
from tests.conftest import session_client
|
||||
from tests.e2e._pipeline_harness import (
|
||||
Server,
|
||||
approve_album,
|
||||
seed_album,
|
||||
start_worker,
|
||||
wait_until,
|
||||
@@ -39,21 +42,29 @@ IMAGES = DOCS / "images"
|
||||
ALBUM = "rome"
|
||||
VIEWPORT = {"width": 1280, "height": 900}
|
||||
|
||||
# One per stage page of the manual: the route to visit, and the element whose
|
||||
# presence means the view has actually finished rendering.
|
||||
# One per stage page of the manual: the route, the heading that view renders, and the
|
||||
# element whose presence means it has finished.
|
||||
#
|
||||
# The heading matters more than it looks. Every route replaces the same container, so
|
||||
# waiting for "an h1" matches the *previous* view's heading and photographs the screen
|
||||
# you just left — which is how the statistics page first shipped a picture of the
|
||||
# archive view.
|
||||
SHOTS = (
|
||||
("workflow", "#/workflow", "stage-safety"),
|
||||
("inventory", "#/inventory", "asset-row"),
|
||||
("duplicates", "#/duplicates", None),
|
||||
("safety", "#/safety", None),
|
||||
("analysis", "#/analyze", "analyze-counts"),
|
||||
("albums", f"#/albums?album={ALBUM}", "suggested-name"),
|
||||
("renames", "#/renames", None),
|
||||
("uploads", "#/uploads", "upload-scope"),
|
||||
("archive", "#/archive", "archive-locations"),
|
||||
("statistics", "#/stats", None),
|
||||
("workflow", "#/workflow", "Workflow", "stage-safety"),
|
||||
("inventory", "#/inventory", "Inventory", "asset-row"),
|
||||
("duplicates", "#/duplicates", "Duplicate clusters", None),
|
||||
("safety", "#/safety", "Safety review", None),
|
||||
("analysis", "#/analyze", "Analyze", "analyze-counts"),
|
||||
("albums", f"#/albums?album={ALBUM}", "Albums", "suggested-name"),
|
||||
("renames", "#/renames", "Renames", None),
|
||||
("uploads", "#/uploads", "Upload", "upload-scope"),
|
||||
("archive", "#/archive", "Archive", "archive-locations"),
|
||||
("statistics", "#/stats", "Stats", None),
|
||||
)
|
||||
|
||||
# Every check here belongs to the documentation gate (US09-05).
|
||||
pytestmark = pytest.mark.phase_i
|
||||
|
||||
|
||||
def writing() -> bool:
|
||||
return os.environ.get("PHOTO_PIPELINE_WRITE_SCREENSHOTS") == "1"
|
||||
@@ -90,17 +101,23 @@ def _prepare(base: str, seeded) -> None:
|
||||
).raise_for_status()
|
||||
wait_until(lambda: client.get("/api/v1/workflow").status_code == 200)
|
||||
|
||||
# An approved name and a built — deliberately unapplied — plan, so the renames
|
||||
# screenshot shows the preview its page describes rather than an empty state.
|
||||
# Building a plan moves nothing; applying it is what would, and nothing here does.
|
||||
approve_album(base, album=ALBUM, name="2019 — Rome")
|
||||
with closing(session_client(base, timeout=30)) as client:
|
||||
client.post("/api/v1/rename-plans").raise_for_status()
|
||||
|
||||
|
||||
def _capture(page, server, target: Path) -> list[str]:
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
page.set_viewport_size(VIEWPORT)
|
||||
written = []
|
||||
for name, route, ready in SHOTS:
|
||||
for name, route, heading, ready in SHOTS:
|
||||
page.goto(f"{server.base}/app/{route}")
|
||||
page.locator("main h1", has_text=heading).first.wait_for(timeout=30_000)
|
||||
if ready:
|
||||
page.get_by_test_id(ready).first.wait_for(timeout=30_000)
|
||||
else:
|
||||
page.locator("main h1").first.wait_for(timeout=30_000)
|
||||
page.screenshot(path=str(target / f"{name}.png"))
|
||||
written.append(name)
|
||||
return written
|
||||
@@ -110,7 +127,7 @@ def test_the_generator_produces_every_screenshot_the_manual_references(page, sta
|
||||
destination = IMAGES if writing() else tmp_path / "images"
|
||||
written = _capture(page, stack, destination)
|
||||
|
||||
assert sorted(written) == sorted(name for name, _, _ in SHOTS)
|
||||
assert sorted(written) == sorted(name for name, *_ in SHOTS)
|
||||
for name in written:
|
||||
produced = destination / f"{name}.png"
|
||||
assert produced.stat().st_size > 5_000, f"{name}.png is too small to be a view"
|
||||
@@ -133,6 +150,60 @@ def test_no_screenshot_shows_a_real_path_or_a_secret(page, stack, tmp_path):
|
||||
assert all(ALBUM in path or "images" in path for path in paths)
|
||||
|
||||
|
||||
def test_the_committed_screenshots_still_show_what_the_application_shows(page, stack, tmp_path):
|
||||
"""A UI change that invalidates the manual should be a red build, not a discovery
|
||||
months later by somebody following a picture of a screen that no longer exists.
|
||||
|
||||
The tolerance is on *content*, not pixels. Comparing the committed PNGs with a
|
||||
fresh capture was tried first and rejected: PNG output is not reproducible, font
|
||||
rasterisation differs between the machine that generated an image and the machine
|
||||
running the gate, and several views legitimately print the fixture library's
|
||||
absolute path, which is a fresh temporary directory every run. A pixel or
|
||||
perceptual comparison therefore fails for reasons that have nothing to do with
|
||||
the manual being wrong — and a check that cries wolf gets deleted.
|
||||
|
||||
What actually invalidates a screenshot is the view no longer showing what its
|
||||
page says it shows. That is what is compared here: the heading, the elements the
|
||||
page describes, and the fact that the image was captured at the same size.
|
||||
|
||||
ponytail: content comparison, not pixels. A visual-diff service with per-platform
|
||||
baselines is the upgrade if cosmetic regressions ever need catching too.
|
||||
"""
|
||||
if not IMAGES.is_dir():
|
||||
pytest.skip("no screenshots have been generated into the repository yet")
|
||||
|
||||
fresh = tmp_path / "fresh"
|
||||
_capture(page, stack, fresh)
|
||||
|
||||
drifted = []
|
||||
for name, route, heading, ready in SHOTS:
|
||||
committed = IMAGES / f"{name}.png"
|
||||
if not committed.is_file():
|
||||
drifted.append(f"{name}: never committed")
|
||||
continue
|
||||
with Image.open(committed) as image:
|
||||
if image.size != (VIEWPORT["width"], VIEWPORT["height"]):
|
||||
drifted.append(f"{name}: committed at {image.size}, captured at {VIEWPORT}")
|
||||
# The views render asynchronously, so each check waits rather than asking a
|
||||
# question the page has not finished answering.
|
||||
page.goto(f"{stack.base}/app/{route}")
|
||||
try:
|
||||
page.locator("main h1", has_text=heading).first.wait_for(timeout=15_000)
|
||||
except PlaywrightTimeout:
|
||||
drifted.append(f"{name}: the view no longer shows the heading '{heading}'")
|
||||
continue
|
||||
if ready:
|
||||
try:
|
||||
page.get_by_test_id(ready).first.wait_for(timeout=15_000)
|
||||
except PlaywrightTimeout:
|
||||
drifted.append(f"{name}: the view no longer renders '{ready}'")
|
||||
|
||||
assert drifted == [], (
|
||||
f"the manual's screenshots no longer match the application: {drifted}. "
|
||||
"Regenerate them with PHOTO_PIPELINE_WRITE_SCREENSHOTS=1 and review the result."
|
||||
)
|
||||
|
||||
|
||||
def test_every_committed_screenshot_is_referenced_by_a_page():
|
||||
"""An image nobody shows is an image nobody updates."""
|
||||
if not IMAGES.is_dir():
|
||||
|
||||
@@ -14,10 +14,15 @@ from __future__ import annotations
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
import photo_pipeline.models # noqa: F401 (registers every table on Base.metadata)
|
||||
from photo_pipeline.db import Base
|
||||
from photo_pipeline.services import jobs, rename_journal, upload_batches
|
||||
|
||||
# Every check here belongs to the documentation gate (US09-05).
|
||||
pytestmark = pytest.mark.phase_i
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
PACKAGE = REPO / "photo_pipeline"
|
||||
OVERVIEW = REPO / "docs" / "architecture.md"
|
||||
|
||||
196
tests/integration/test_docs_gate.py
Normal file
@@ -0,0 +1,196 @@
|
||||
"""US09-05: the documentation gate's own contract, checked without a browser.
|
||||
|
||||
The gate itself needs a browser and a server and takes minutes; what is checkable
|
||||
offline is what makes it a *gate* rather than a long test run:
|
||||
|
||||
* it selects every documentation check by marker, so adding one is enough to put it
|
||||
in front of a release;
|
||||
* it accepts no skip at all — a check that did not run is a page nobody compared
|
||||
with the code;
|
||||
* it retains checksummed evidence per run;
|
||||
* CI runs it on pull requests, which is when a fix is still cheap.
|
||||
|
||||
The checks it runs are the four suites written by US09-01 through US09-04.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from photo_pipeline.config import Config
|
||||
from photo_pipeline.services import release
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
DOCS = REPO / "docs"
|
||||
TEST_WORKFLOW = yaml.safe_load((REPO / ".gitea" / "workflows" / "test.yml").read_text())
|
||||
|
||||
pytestmark = pytest.mark.phase_i
|
||||
|
||||
# The suites the gate exists to run. Each is mapped to a story in the traceability
|
||||
# matrix, so this list is also what stops one being quietly dropped.
|
||||
DOCUMENTATION_SUITES = (
|
||||
"tests/integration/test_documentation.py",
|
||||
"tests/integration/test_installation_manual.py",
|
||||
"tests/integration/test_architecture_overview.py",
|
||||
"tests/integration/test_user_manual.py",
|
||||
"tests/integration/test_docs_gate.py",
|
||||
"tests/e2e/test_docs_ui.py",
|
||||
"tests/e2e/test_user_manual_screenshots.py",
|
||||
)
|
||||
|
||||
|
||||
def _config(tmp_path) -> Config:
|
||||
(tmp_path / "data").mkdir(exist_ok=True)
|
||||
return Config.from_env({"PHOTO_PIPELINE_DATA_DIR": str(tmp_path / "data")})
|
||||
|
||||
|
||||
# ── what the gate runs ───────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_the_gate_selects_every_documentation_check_by_marker():
|
||||
assert release.DOCS_STAGES == (("documentation", ("tests", "-m", "phase_i")),)
|
||||
declared = [
|
||||
line
|
||||
for line in (REPO / "pyproject.toml").read_text().splitlines()
|
||||
if line.strip().startswith('"phase_i:')
|
||||
]
|
||||
assert declared, "an unregistered marker selects nothing and fails no gate"
|
||||
|
||||
|
||||
def test_every_documentation_suite_carries_the_marker():
|
||||
"""A documentation test without the marker is a check the gate never runs."""
|
||||
unmarked = [
|
||||
suite
|
||||
for suite in DOCUMENTATION_SUITES
|
||||
if "pytest.mark.phase_i" not in (REPO / suite).read_text()
|
||||
]
|
||||
assert unmarked == [], f"suites missing the phase_i marker: {unmarked}"
|
||||
|
||||
|
||||
def test_the_marker_selects_nothing_outside_the_documentation_suites():
|
||||
"""Scope matters: the gate's promise is that it ran *the documentation checks*."""
|
||||
marked = sorted(
|
||||
str(path.relative_to(REPO))
|
||||
for path in (REPO / "tests").rglob("test_*.py")
|
||||
if "pytest.mark.phase_i" in path.read_text()
|
||||
)
|
||||
assert marked == sorted(DOCUMENTATION_SUITES)
|
||||
|
||||
|
||||
# ── no skip is an environment limit here ─────────────────────────────────────
|
||||
|
||||
|
||||
def test_the_gate_accepts_no_skipped_check_at_all():
|
||||
assert release.DOCS_ALLOWED_SKIP_REASONS == ()
|
||||
skipped = release.StageResult(
|
||||
"documentation", [], 0, 0.1, "1 skipped", ["SKIPPED [1] x.py:1: exiftool not installed"]
|
||||
)
|
||||
# The release gate tolerates this one because it describes the machine. The
|
||||
# documentation gate cannot: nothing here depends on the machine.
|
||||
assert release.unexpected_skips([skipped]) == []
|
||||
assert release.unexpected_skips([skipped], release.DOCS_ALLOWED_SKIP_REASONS) == [
|
||||
"SKIPPED [1] x.py:1: exiftool not installed"
|
||||
]
|
||||
|
||||
|
||||
def test_a_skipped_check_fails_the_run_and_the_evidence_says_so(tmp_path):
|
||||
skipping = tmp_path / "test_skipping.py"
|
||||
skipping.write_text(
|
||||
"import pytest\n\ndef test_x():\n pytest.skip('exiftool not installed')\n"
|
||||
)
|
||||
evidence = tmp_path / "evidence"
|
||||
|
||||
report = release.run_gate(
|
||||
_config(tmp_path),
|
||||
output=evidence,
|
||||
stages=(("documentation", (str(skipping),)),),
|
||||
allowed_skips=release.DOCS_ALLOWED_SKIP_REASONS,
|
||||
)
|
||||
|
||||
assert report["ok"] is False
|
||||
assert report["failures"] == [], "the stage passed; the skip is what fails the gate"
|
||||
assert report["unexpected_skips"], report
|
||||
written = json.loads((evidence / release.REPORT_NAME).read_text())
|
||||
assert written["ok"] is False
|
||||
assert (evidence / "logs" / "documentation.log").exists()
|
||||
for line in (evidence / release.CHECKSUMS_NAME).read_text().splitlines():
|
||||
digest, name = line.split(" ", 1)
|
||||
assert release.sha256_file(evidence / name) == digest
|
||||
|
||||
|
||||
def test_a_broken_documentation_check_fails_the_gate(tmp_path):
|
||||
"""The failure mode that matters: a page and the code disagreeing."""
|
||||
failing = tmp_path / "test_disagreement.py"
|
||||
failing.write_text(
|
||||
"def test_the_manual_matches_the_code():\n"
|
||||
" documented = {'lock_held'}\n"
|
||||
" emitted = {'lock_held', 'renamed_since'}\n"
|
||||
" assert emitted <= documented\n"
|
||||
)
|
||||
|
||||
report = release.run_gate(
|
||||
_config(tmp_path),
|
||||
output=tmp_path / "evidence",
|
||||
stages=(("documentation", (str(failing),)),),
|
||||
allowed_skips=release.DOCS_ALLOWED_SKIP_REASONS,
|
||||
)
|
||||
|
||||
assert report["ok"] is False
|
||||
assert report["failures"], report
|
||||
|
||||
|
||||
def test_a_clean_tree_passes(tmp_path):
|
||||
passing = tmp_path / "test_passing.py"
|
||||
passing.write_text("def test_x():\n assert True\n")
|
||||
|
||||
report = release.run_gate(
|
||||
_config(tmp_path),
|
||||
output=tmp_path / "evidence",
|
||||
stages=(("documentation", (str(passing),)),),
|
||||
allowed_skips=release.DOCS_ALLOWED_SKIP_REASONS,
|
||||
)
|
||||
|
||||
assert report["ok"] is True and report["unexpected_skips"] == []
|
||||
assert report["revision"], "the evidence must say which commit it covers"
|
||||
|
||||
|
||||
# ── the command, and CI ──────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_the_command_is_documented_and_wired():
|
||||
assert "docs-gate" in (REPO / "photo_pipeline" / "__main__.py").read_text()
|
||||
assert "docs-gate" in (DOCS / "index.md").read_text()
|
||||
|
||||
|
||||
def test_ci_runs_the_gate_on_pull_requests_and_keeps_its_evidence():
|
||||
job = TEST_WORKFLOW["jobs"]["documentation"]
|
||||
# No `if:` — documentation drifts on the same commits that change behaviour, and
|
||||
# the pull request is when saying so is still cheap.
|
||||
assert "if" not in job
|
||||
assert "pull_request" in TEST_WORKFLOW[True] or "pull_request" in TEST_WORKFLOW.get("on", {})
|
||||
script = "\n".join(step["run"] for step in job["steps"] if "run" in step)
|
||||
assert "photo_pipeline docs-gate" in script
|
||||
evidence = next(step for step in job["steps"] if "upload-artifact" in str(step.get("uses")))
|
||||
assert evidence["if"] == "always()", "a failed gate's evidence is the one worth keeping"
|
||||
|
||||
|
||||
# ── the pages and the repository point at each other ─────────────────────────
|
||||
|
||||
|
||||
def test_the_readme_and_the_documentation_index_link_to_each_other():
|
||||
"""Neither may become the forgotten copy."""
|
||||
readme = (REPO / "README.md").read_text()
|
||||
index = (DOCS / "index.md").read_text()
|
||||
assert "docs/index.md" in readme
|
||||
assert re.search(r"\.\./README\.md", index), "the index does not link back to the README"
|
||||
|
||||
|
||||
def test_every_documentation_story_is_mapped_in_the_traceability_matrix():
|
||||
matrix = json.loads((REPO / "tests" / "story_traceability.json").read_text())["stories"]
|
||||
for story in ("US09-01", "US09-02", "US09-03", "US09-04", "US09-05"):
|
||||
assert matrix.get(story), f"{story} is not mapped to any test"
|
||||
@@ -16,12 +16,17 @@ import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from starlette.testclient import TestClient
|
||||
|
||||
from photo_pipeline.api.app import create_app
|
||||
from photo_pipeline.api.security import DEFAULT_HEADERS
|
||||
from photo_pipeline.config import Config
|
||||
|
||||
# Every check here belongs to the documentation gate (US09-05).
|
||||
pytestmark = pytest.mark.phase_i
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
DOCS = REPO / "docs"
|
||||
VENDOR = REPO / "frontend" / "js" / "vendor"
|
||||
|
||||
@@ -16,8 +16,13 @@ import sys
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from photo_pipeline.config import ENV_PREFIX, LEGACY_ALIASES, Config
|
||||
|
||||
# Every check here belongs to the documentation gate (US09-05).
|
||||
pytestmark = pytest.mark.phase_i
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
MANUAL = REPO / "docs" / "installation.md"
|
||||
MAIN = REPO / "photo_pipeline" / "__main__.py"
|
||||
|
||||
@@ -14,6 +14,8 @@ from __future__ import annotations
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
REPO = Path(__file__).resolve().parents[2]
|
||||
DOCS = REPO / "docs"
|
||||
STAGES = DOCS / "stages"
|
||||
@@ -55,6 +57,9 @@ STAGE_PAGES = (
|
||||
"diagnostics",
|
||||
)
|
||||
|
||||
# Every check here belongs to the documentation gate (US09-05).
|
||||
pytestmark = pytest.mark.phase_i
|
||||
|
||||
|
||||
def emitted_codes() -> set[str]:
|
||||
found: set[str] = set()
|
||||
|
||||
@@ -208,10 +208,11 @@
|
||||
"US09-04": [
|
||||
"tests/integration/test_user_manual.py",
|
||||
"tests/e2e/test_user_manual_screenshots.py"
|
||||
],
|
||||
"US09-05": [
|
||||
"tests/integration/test_docs_gate.py"
|
||||
]
|
||||
},
|
||||
"planned": [
|
||||
"US09-05"
|
||||
],
|
||||
"planned": [],
|
||||
"_planned_comment": "Accepted backlog stories that are not implemented yet. The release gate (US07-07) requires every story file to be either mapped to tests or listed here, so an unimplemented story is a visible decision rather than a hole in the matrix."
|
||||
}
|
||||
|
||||