domverse/outline-sync
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add compare command for two-instance diff (Phase 1)
All checks were successful
Deploy / deploy (push) Successful in 13s
Details

Browse Source 
Adds read-only `compare` command that fetches both the remote Outline
instance (via Tailscale) and a local instance (outline-web container),
matches documents by canonical path key, and reports in_sync / remote_only /
local_only / conflict status. Also adds PRD for the full two-instance sync
workflow.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
domverse
2026-03-18 23:35:47 +01:00
parent 6352fbb03f
commit 14af83a52a
2 changed files with 432 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

277
TWO_INSTANCE_SYNC_PRD.md Normal file
View File

@@ -0,0 +1,277 @@
# PRD: Two-Instance Outline Sync
## Overview
Extend outline-sync to support bidirectional comparison and synchronization between two live Outline instances: a **remote** instance (accessed via Tailscale) and a **local** instance (accessed directly). The goal is to bring both instances to identical content state after resolving all differences interactively.
---
## Current State
The current tool performs a **one-way pull** from the remote Outline instance into a local Git-backed vault (Obsidian-compatible markdown). This vault acts as a local representation of the remote instance's content.
---
## Goal
Enable a four-step workflow:
1. **Pull** from remote instance into local vault (existing)
2. **Fetch** from local Outline instance and detect differences vs. vault
3. **Review** differences interactively (left = remote, right = local)
4. **Resolve** all differences and push chosen versions to both instances
End state: both Outline instances contain identical content.
---
## Instances
| Side | Label | URL | Token |
|---|---|---|---|
| Left (Remote) | `remote` | `http://outline:3000` (Tailscale) | From `settings.json` |
| Right (Local) | `local` | `http://outline-web:3000` (Docker container) | `ol_api_PBU1EX6aRlUVkXzD995oGpMOmYOXpdXEWeEQll` |
---
## Document Matching Strategy
Since the two Outline instances assign independent document IDs, matching must be **by content path** — the canonical path is `Collection / Parent Chain / Document Title` derived from the navigation tree.
**Match key**: `collection_name / parent_title / ... / document_title` (case-insensitive, normalized)
**Match outcomes per document:**
| Outcome | Condition |
|---|---|
| `in_sync` | Match found, content and `updatedAt` identical |
| `remote_only` | Path exists on remote, not found on local |
| `local_only` | Path exists on local, not found on remote |
| `conflict` | Path exists on both, content differs |
---
## Step 2 — Fetch & Diff
### Process
1. Pull from remote into vault (existing Step 1).
2. Instantiate a second `OutlineSync` client for the local instance.
3. Fetch all collections + document trees from local instance.
4. For each document on either side, compute the match key.
5. Build a **diff report**: a structured list of `DiffEntry` objects.
### DiffEntry Model
```python
@dataclass
class DiffEntry:
match_key: str # Canonical path used for matching
status: Literal["in_sync", "remote_only", "local_only", "conflict"]
remote_doc: DocMeta | None # None if remote_only is False
local_doc: DocMeta | None # None if local_only is False
@dataclass
class DocMeta:
instance: str # "remote" | "local"
doc_id: str
title: str
collection_id: str
collection_name: str
parent_id: str | None
updated_at: str # ISO timestamp
content: str # Full markdown body
```
### CLI Output (Step 2)
```
Fetching remote... ✓ (47 documents in 6 collections)
Fetching local... ✓ (39 documents in 5 collections)
Diff summary:
In sync: 35 documents
Remote only: 8 documents
Local only: 4 documents
Conflicts: 8 documents
─────────────────────────────
Total to resolve: 20
```
---
## Step 3 — Interactive Conflict Resolution
### Interface Options
Two interfaces will be supported:
#### A. Terminal (CLI) — Primary
An interactive terminal UI (using `rich` or plain TTY prompts) presents each non-`in_sync` document one at a time.
**Display per item:**
```
[8/20] CONFLICT — Projekte / outline-sync / Architecture
LEFT (remote) updated 2026-03-17T09:12:00Z
RIGHT (local) updated 2026-03-15T14:30:00Z
--- remote
+++ local
@@ -3,7 +3,9 @@
## Overview
-The sync tool uses Git as a local cache.
+The sync tool uses Git as a local cache and Tailscale for connectivity.
+
+Added section on networking.
Choose action:
[r] Keep remote [l] Keep local [s] Skip [d] Full diff [q] Quit
```
**For `remote_only`:**
```
[3/20] REMOTE ONLY — Bewerbungen / Company XYZ / Cover Letter
Document exists on remote, not on local.
Choose: [r] Copy to local [s] Skip [q] Quit
```
**For `local_only`:**
```
[5/20] LOCAL ONLY — Projekte / New Project / Notes
Document exists on local, not on remote.
Choose: [l] Copy to remote [s] Skip [q] Quit
```
#### B. Web UI — Optional (Phase 2)
A web page with a split-panel diff view (similar to a code review UI) with Accept Left / Accept Right / Edit buttons. Built on the existing FastAPI + HTMX stack.
---
## Step 4 — Resolution & Write-Back
After the user has resolved all items, the tool executes the chosen actions:
| User Choice | Action |
|---|---|
| Keep remote | Push remote content to local Outline via API (`documents.create` or `documents.update`) |
| Keep local | Push local content to remote Outline via API |
| Copy to local | Create document on local instance with remote content |
| Copy to remote | Create document on remote instance with local content |
| Skip | No action for this document |
### Write-Back Rules
- **Collection must exist** on target before creating a document. Auto-create collection if missing.
- **Parent document must exist** on target before creating a child. Create parents recursively (depth-first).
- **Title** and **markdown body** are copied as-is. Frontmatter (`outline_id` etc.) is not transferred — target instance assigns its own IDs.
- **Updated timestamps** are NOT forcibly set — the target instance sets `updatedAt` on create/update.
### Final Report
```
Applying resolutions...
✓ Copied to local: 8 documents
✓ Copied to remote: 4 documents
✓ Pushed to local: 3 documents (conflict: chose remote)
✓ Pushed to remote: 5 documents (conflict: chose local)
- Skipped: 0 documents
Both instances now have 47 documents in 6 collections.
```
---
## Configuration
Add a `local` block to `settings.json`:
```json
{
"source": {
"url": "http://outline:3000",
"token": "ol_api_..."
},
"local": {
"url": "http://outline-web:3000",
"token": "ol_api_PBU1EX6aRlUVkXzD995oGpMOmYOXpdXEWeEQll"
}
}
```
CLI flags override settings:
```bash
./sync.sh compare --local-url http://localhost:3000 --local-token ol_api_...
./sync.sh compare --auto-resolve newer # Auto-pick newer timestamp, no prompts
```
---
## New CLI Commands
| Command | Description |
|---|---|
| `compare` | Fetch both instances and print diff report (no writes) |
| `merge` | Interactive resolution workflow (Steps 2–4) |
| `merge --auto-resolve newer` | Non-interactive: always pick newer `updatedAt` |
| `merge --auto-resolve remote` | Non-interactive: always keep remote version |
| `merge --auto-resolve local` | Non-interactive: always keep local version |
---
## Non-Goals
- **Attachment/image sync** — text content only (consistent with existing NG3)
- **Real-time continuous sync** — this is a manual, on-demand operation
- **Three-way merge** — no character-level content merging; user picks one side
- **Deletion propagation** — documents deleted on one side are not auto-deleted on the other; they appear as `remote_only` or `local_only`
- **Comment/reaction sync** — document body only
- **Preserving Outline metadata** (pinned, starred, views) — content only
---
## Implementation Phases
### Phase 1 — Compare (Step 2 only)
- Add `local` config block
- Implement `LocalOutlineClient` (reuse `OutlineSync` API methods, different credentials)
- Build `DiffReport`: fetch both instances, match by path, classify entries
- `compare` command: print diff summary to terminal
- No writes to either instance
### Phase 2 — Interactive CLI Merge (Steps 3–4)
- `merge` command with TTY prompts
- `--auto-resolve` flag for non-interactive use
- Write-back: `documents.create` / `documents.update` on target
- Recursive parent creation
- Final report
### Phase 3 — Web UI Diff View (Optional)
- Split-panel view in FastAPI + HTMX
- Accept Left / Accept Right buttons per document
- Progress indicator during write-back
---
## Success Criteria
- `compare` correctly identifies all differences between two live Outline instances
- `merge` allows user to resolve every difference and both instances end up with identical document trees
- No data is lost — skipped documents remain on their original instance
- The tool is re-runnable: running `compare` after a successful `merge` reports 0 differences
---
## Decisions
1. **Rename detection**: Treated as `remote_only` + `local_only` (delete + create). No fuzzy matching.
2. **Collection renames**: Same — treated as entirely separate collections with all-new documents.
3. **Local instance**: Docker container `outline-web`, accessible at `http://outline-web:3000`.
4. **Initial scope**: Phase 1 only (`compare` command, read-only). Phases 2–3 follow after matching logic is validated.