# Rich Live Dashboard — Upgrade Plan

Replace the scrolling `Progress` context in `photo_analyzer.py` with a persistent
`Live + Layout` that shows **album progress**, a **photo feed**, and the **progress
bar** simultaneously — without touching the DB logic or threading model.

> **Focus: robustness.** The dashboard is a *view layer only* — it reads state,
> never owns it. SQLite stays the source of truth at all times.

---

## Target layout

```
┌─ python photo_analyzer.py --library pictures/ ─────────────────┐
│ ┌─ Album Progress — leaf folder = album ───────────────────┐   │
│ │ Camera Roll        238 / 550  ████████░░░░░░░░░░░  43%    │   │
│ │ Urlaub/Rom          45 /  45  ██████████████████   ✓      │   │
│ │ Urlaub/Venedig      12 /  60  ███░░░░░░░░░░░░░░░░   20%    │   │
│ │ WhatsApp Images      0 / 120  ░░░░░░░░░░░░░░░░░░    0%     │   │
│ └──────────────────────────────────────────────────────────┘   │
│ ┌─ Recent ─────────────────────────────────────────────────┐   │
│ │ ✓  IMG_4821.jpg                                           │   │
│ │      A golden retriever playing fetch on a sunny beach    │   │
│ │ ✗  DSC_0042.jpg — Exhausted 4 retries: connection timeout │   │
│ │ ✓  IMG_4820.jpg                                           │   │
│ │      Two children building a sandcastle at sunset         │   │
│ └──────────────────────────────────────────────────────────┘   │
│ ┌─ Analyzing — Camera Roll ────────────────────────────────┐   │
│ │ ████████░░░░░░░░░░  238/550  43%  ok=235 err=3            │   │
│ │ tokens=284,192  est. remaining ~340,000  ETA 14:23        │   │
│ └──────────────────────────────────────────────────────────┘   │
└────────────────────────────────────────────────────────────────┘
```

---

## Core rule: album = leaf folder

An **album is the leaf folder directly containing the photos**, not a top-level
parent. `Urlaub/Rom` and `Urlaub/Venedig` are two separate albums, never collapsed
into `Urlaub`. Key every stats dict on `album_label(path, library)` (already added
to the script for `--stats`):

```python
def album_label(photo: Path, library: Path = None) -> str:
    """Leaf folder relative to library: Urlaub/Rom/IMG.jpg -> 'Urlaub/Rom'."""
    parent = photo.parent
    if library:
        try:
            rel = parent.relative_to(library)
            return "(root)" if str(rel) == "." else str(rel)
        except ValueError:
            pass
    return parent.name or str(parent)
```

The analysis loop already groups on `Path(p).parent` (the leaf), so each sub-album
completes fully before the next starts. Only the *display* side needs this fix.

---

## Phases

### Phase 01 — Scaffold Live + Layout  *(structural, ~30 min)*

Replace `with Progress(...) as progress:` with a `Live` context wrapping a
`Layout`. **Progress becomes a renderable, not a context manager** — it lives
inside the layout rather than owning the terminal. All DB logic, threading, and
retry logic stays unchanged.

```python
from rich.layout import Layout
from rich.live   import Live

layout = Layout()
layout.split_column(
    Layout(name="stats",    ratio=2),
    Layout(name="feed",     ratio=2),
    Layout(name="progress", size=5),
)
progress = Progress(...)        # same columns as today
task_id  = progress.add_task(...)
layout["progress"].update(progress)

with Live(layout, console=console, refresh_per_second=4, transient=False):
    # ... existing folder loop unchanged ...
```

- **Test:** script starts, layout renders with placeholder panels, progress bar
  advances, Ctrl-C stops cleanly.
- Do not move on until this is stable — later phases build on this scaffold.

---

### Phase 02 — Thread-safe log buffer  *(concurrency, ~20 min)*

Workers must **never write to the terminal directly**. Replace every
`progress.console.print()` call with an append to a shared `deque`. The Live
renderer reads from the deque on each refresh cycle and renders it as the feed
panel. Threads write, the renderer reads — no other direction.

```python
from collections import deque

log_buffer = deque(maxlen=10)   # newest first
log_lock   = threading.Lock()

# in process() — replace progress.console.print(...) with:
with log_lock:
    log_buffer.appendleft(entry)

# in the Live render function:
def render_feed() -> Table:
    with log_lock:
        rows = list(log_buffer)   # snapshot
    # build Rich Table from rows ...
```

- **Never** call `live.console.print()` from a worker thread — always via the buffer.
- `deque(maxlen=10)` auto-evicts old entries; no memory growth.
- The lock protects the *snapshot* read, not the render.

---

### Phase 03 — In-memory album stats  *(performance, ~20 min)*

The stats panel refreshes at 4 Hz — **no DB queries in the render loop**. Key
every dict on `album_label(path, library)`. Compute `folder_total` from the sorted
`pending` list, seed `folder_done` from one upfront DB query, and have workers
increment it in memory after each success.

```python
# album = leaf folder relative to library — reuse the existing helper
folder_total: dict[str, int] = {}
folder_done:  dict[str, int] = {}

for folder, paths in folder_groups:     # folder = abs leaf dir
    album = album_label(Path(paths[0]), library)
    folder_total[album] = len(paths)
    folder_done[album]  = 0

# Seed already-done counts from DB (single query):
for path, status in conn.execute(
    "SELECT path, status FROM photos WHERE status IN ('analyzed','exif_written')"
):
    album = album_label(Path(path), library)
    if album in folder_done:
        folder_done[album] += 1

# In the worker result handler (album computed once per folder):
with token_lock:                        # reuse existing lock
    folder_done[album] = folder_done.get(album, 0) + 1
```

- **Album = leaf folder, not parent.** Nested trees like `Urlaub/*` become one
  row per sub-album.
- Render reads the dicts under the existing `token_lock` — no new lock needed.
- Shows **all** albums (future ones at 0%); long paths left-truncated via `fit_label()`.
- Errors do *not* increment `folder_done` — an album shows ✓ only when every
  photo succeeded.

---

### Phase 04 — Non-TTY fallback  *(safety, ~10 min)*

Rich's `Live` behaves unexpectedly when stdout is piped or redirected (log
capture, CI, `nohup`). **Gate the Live path on `console.is_terminal`.** Extract
the current Progress-based loop into `_run_analysis_plain()` unchanged; the new
Live implementation goes in `_run_analysis_live()`.

```python
def run_analysis(args, conn, client):
    ...  # shared setup (discover, pending, folder_groups)
    if console.is_terminal:
        _run_analysis_live(args, conn, client, pending, folder_groups, ...)
    else:
        _run_analysis_plain(args, conn, client, pending, folder_groups, ...)
```

- Plain mode keeps every existing behavior — no regression for scripted use.
- File logs always write regardless of mode.
- No flag needed — detection is automatic.

---

### Phase 05 — Error containment  *(robustness, ~15 min)*

The Live layer must never take down the pipeline. **DB writes happen in worker
threads, outside the render path** — a renderer crash cannot corrupt the
database. Wrap Live construction in try/except: if Rich fails to start, fall back
to plain mode. SIGINT already works (`_stop` set, executor drains, `with` exits).

```python
try:
    with Live(layout, console=console, refresh_per_second=4):
        _run_folder_loop(executor, folder_groups, ...)
except Exception as e:
    log.warning(f"Live render failed ({e}), falling back to plain mode")
    _run_analysis_plain(args, conn, client, pending, folder_groups, ...)
```

- **Worker exception path:** `mark_error()` runs before anything is appended to
  `log_buffer` — DB is always consistent even if the render crashes mid-frame.
- **Render exceptions:** catch inside the render function, return a plain
  `Text("render error")` panel — never let a bad render kill the loop.
- **Resume safety:** unchanged — `--stats` and restart behavior identical to today.

---

## Robustness constraints

| Constraint | Detail |
|---|---|
| No DB queries in render path | Renderer fires 4×/s. Album counts kept in in-memory dicts, seeded once at startup. SQLite never touched during animation. |
| Workers never touch the terminal | Only interface between workers and display is `log_buffer` (deque + lock). Rich's Live is single-threaded by design. |
| Render errors isolated | Each panel's render function wraps its body in try/except, returns a fallback `Text` on failure. A bad render is cosmetic, not fatal. |
| Ctrl-C still works | `_stop` event and SIGINT handler unchanged. Live context exits when the `with` block ends. In-flight requests complete; queued ones cancel. |
| Non-TTY produces no garbage | Piped output, `nohup`, log capture → plain mode via `console.is_terminal`. No ANSI escapes leak into log files. |
| Resume unchanged | All state lives in SQLite. Kill at any point; restart picks up where it left off. Dashboard reads state, never owns it. |

---

## Status: implemented ✓ (2026-06-22)

All five phases shipped. Old script archived as `photo_analyzer.py.bak-20260622-091208`.

Structure: `run_analysis()` does shared setup, then dispatches to
`_run_analysis_live()` (TTY) or `_run_analysis_plain()` (piped/redirected), both
driving the shared `_run_folder_loop()` engine. Live uses a `_Dashboard.__rich__`
that rebuilds stats/feed/progress panels each tick from in-memory dicts + a deque.

**Extra hardening found during testing (not in original plan):** the shared
`sqlite3.Connection` was committed from multiple worker threads, racing with
"cannot commit - no transaction is active". Real API latency hid it; instant
dry-run exposed it. Fixed with a module-level `_db_lock` serialising all writes
(`mark_analyzed` / `mark_error` / `mark_exif_written` / `upsert_pending`).
Verified: 200 photos × 8 workers, zero spurious errors.

Tested: plain mode (piped), live mode (PTY, 6 and 200 photos), Live-failure
fallback, render-error isolation, album windowing (>14 albums summarised),
leaf-folder album grouping.

---

## Implementation order

1. **Phase 1** — scaffold Live+Layout. Test: layout renders, progress bar
   advances, Ctrl-C exits. *(must pass before continuing)*
2. **Phase 3** — in-memory album stats panel. Test: album rows appear with correct
   counts, numbers increment live as photos complete.
3. **Phase 2** — log buffer + feed panel. Test: photo results appear in the feed,
   errors show in red, old entries scroll off.
4. **Phase 4** — non-TTY fallback. Test: `python photo_analyzer.py ... | cat`
   produces clean plain output.
5. **Phase 5** — error containment. Test: force a render exception, confirm
   pipeline continues; interrupt mid-album, confirm DB is consistent on restart.