domverse/photoanalyzer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects 1 Releases Wiki Activity

US07-06: Validate Performance and Resource Bounds (#93)

Browse Source
This commit was merged in pull request #93.
This commit is contained in:
domverse
2026-08-17 21:52:02 +02:00
parent 9851e112a9
commit 05f34cdda2
14 changed files with 1341 additions and 121 deletions
Download Patch File Download Diff File Expand all files Collapse all files

59
README.md
View File

@@ -281,6 +281,65 @@ file in the temporary library are copied to `.artifacts/<test id>/` before pytes
deletes the directory. Point `PHOTO_PIPELINE_TEST_ARTIFACTS` elsewhere to collect
them from CI.
## Performance budgets (US07-06)
Budgets are measured, not asserted in prose. `python -m photo_pipeline benchmark`
builds a synthetic library of a stated size, runs the same scenarios every time,
writes a machine-readable report, and **exits non-zero when a budget is breached**.
```bash
work_item/scripts/python -m photo_pipeline benchmark --profile smoke # ~2 s, runs in CI
work_item/scripts/python -m photo_pipeline benchmark --profile short # 25k assets
work_item/scripts/python -m photo_pipeline benchmark --profile full \
--output data/benchmarks/full.json # 25k + 100k
work_item/scripts/python -m photo_pipeline benchmark --profile huge \
--soak-seconds 3600 --output data/benchmarks/soak.json # 500k + soak
```
| Metric | Budget | Why |
|---|---|---|
| `latency_p95_ms` | 250 ms | a list or search page must feel immediate |
| `latency_max_ms` | 2 000 ms | no single page may stall the review flow |
| `rss_growth_bytes` | 400 MB | a run must not leak the library |
| `open_files` | 256 | file descriptors are a hard operating-system limit |
| `wal_bytes` | 200 MB | a growing write-ahead log means checkpoints are starving |
| `queue_depth` | 1 000 | an unbounded queue is an out-of-memory in waiting |
| `cache_over_quota_bytes` | 0 | the thumbnail cache has to respect its quota |
Measured on the reference machine (Apple Silicon, SQLite WAL), p95 per scenario:
| Scenario | 25k | 100k |
|---|---|---|
| `inventory_page` | 0.5 ms | 0.6 ms |
| `library_search` | 4.8 ms | 17.1 ms |
| `library_stats` | 56.8 ms | 197.4 ms |
| `workflow_readiness` | 52.6 ms | 235.4 ms |
| `duplicate_cluster_list` | 0.6 ms | 0.5 ms |
| `duplicate_cluster_page` | 1.7 ms | 1.7 ms |
CI runs the `smoke` profile through `tests/integration/test_performance_budgets.py`;
the 25k/100k/500k matrix and the multi-hour soak belong to scheduled infrastructure,
because minutes of build time do not belong in the suite that runs on every change.
**Exceptions.** A budget that cannot be met is not a warning to ignore: it goes into
`APPROVED_EXCEPTIONS` in `photo_pipeline/services/benchmarks.py` with its raised
limit, who approved it, why, and a review date. Every report lists the exceptions it
applied, so a release review sees them.
Approved today, both for the 500k `huge` profile only, review by 2027-02-17:
| Scenario | Measured at 500k | Raised limit |
|---|---|---|
| `library_stats` | 1.08 s p95 · 3.2 s max | 1.5 s p95 · 4 s max |
| `workflow_readiness` | 1.40 s p95 · 3.3 s max | 1.8 s p95 · 4 s max |
Both are library-wide aggregates — the current safety decision of every asset, and
the album/tag/year breakdown of every analysis row — and both meet the 250 ms budget
at the 100k rows the concept sets it for. Beyond that they are linear against one
SQLite writer; the fix is denormalized totals or the planned PostgreSQL transition,
not a query tweak. Everything else at 500k is inside budget, and a soak at that size
grows neither resident memory nor the job queue.
## Backup and recovery (US07-05)
Backups go through SQLite's online backup API, never a file copy: with WAL enabled