760 lines
32 KiB
Markdown
760 lines
32 KiB
Markdown
# PRD: Outline ↔ Obsidian Sync
|
|
**Version:** 0.3
|
|
**Date:** 2026-03-04
|
|
**Status:** Revised — Git + Frontmatter Architecture
|
|
|
|
---
|
|
|
|
## 1. Problem Statement
|
|
|
|
The user maintains a knowledge base in Outline (web) and wants to edit it locally in Obsidian. Changes must flow in both directions. The existing export/import scripts provide one-way, full-replacement operations — insufficient for safe bidirectional sync where both sides accumulate independent changes between syncs.
|
|
|
|
This is **not a shared git repository**. Git is used purely as a local sync engine — for its change detection, three-way merge, and conflict resolution primitives. There is no git remote, no `git push` to a server, no collaboration workflow.
|
|
|
|
---
|
|
|
|
## 2. Goals
|
|
|
|
| # | Goal |
|
|
|---|------|
|
|
| G1 | Changes made in Outline are visible in Obsidian after sync |
|
|
| G2 | Changes made in Obsidian are pushed to Outline without destroying document history |
|
|
| G3 | Conflicts are detected and surfaced — never silently overwritten |
|
|
| G4 | Safe to run unattended via Ofelia cron |
|
|
| G5 | Outline document URLs remain stable across syncs (no delete+recreate) |
|
|
| G6 | Setup is self-contained in a dedicated folder with clear structure |
|
|
|
|
## 3. Non-Goals
|
|
|
|
| # | Non-Goal | Reason |
|
|
|---|----------|--------|
|
|
| NG1 | Real-time sync | Async workflow; polling is sufficient |
|
|
| NG2 | Wikilink ↔ Outline link translation | Lossy and fragile |
|
|
| NG3 | Syncing attachments/images | Separate concern |
|
|
| NG4 | Obsidian-only content (Daily Notes, templates) | No Outline equivalent |
|
|
| NG5 | Multi-user conflict resolution | Single-user use case |
|
|
| NG6 | A shared/remote git repository | Git is local tooling only |
|
|
|
|
---
|
|
|
|
## 4. Folder Setup
|
|
|
|
### 4.1 Directory Structure
|
|
|
|
A dedicated folder is created for the Obsidian vault and sync tooling. It is separate from `outline-tools/` (which holds export/import scripts).
|
|
|
|
```
|
|
/home/crabix/docker_authentik/
|
|
├── outline-tools/ ← existing: export + import scripts (unchanged)
|
|
└── outline-vault/ ← NEW: Obsidian vault + git sync engine
|
|
├── .git/ ← local git repo (never pushed to any remote)
|
|
├── .gitignore
|
|
├── .gitattributes
|
|
├── sync.sh ← main sync entrypoint (bash wrapper)
|
|
├── outline_sync.py ← core sync logic
|
|
├── settings.json ← API token + config (gitignored)
|
|
├── _sync_log.md ← human-readable sync log (readable in Obsidian)
|
|
├── .obsidian/ ← Obsidian config (gitignored)
|
|
├── Bewerbungen/ ← Outline collection → top-level folder
|
|
│ ├── CV.md
|
|
│ └── Tipico/ ← nested document → subfolder
|
|
│ ├── Tipico.md
|
|
│ └── Pitch Tipico.md
|
|
└── Projekte/
|
|
└── ...
|
|
```
|
|
|
|
Obsidian is pointed at `/home/crabix/docker_authentik/outline-vault/` as its vault root.
|
|
|
|
### 4.2 Git Initialization
|
|
|
|
Git is initialized locally with two branches. There is no remote.
|
|
|
|
```bash
|
|
cd /home/crabix/docker_authentik/outline-vault
|
|
git init
|
|
git checkout -b outline # branch: tracks last known Outline state
|
|
git checkout -b main # branch: Obsidian edits live here
|
|
```
|
|
|
|
**Branch semantics:**
|
|
|
|
| Branch | Purpose |
|
|
|--------|---------|
|
|
| `outline` | Snapshot of Outline at last successful sync. Committed by sync script only. Never edited by hand. |
|
|
| `main` | Working branch. Obsidian edits land here. Obsidian Git plugin auto-commits. |
|
|
|
|
The `outline` branch tip is the **common ancestor** for all merges. It replaces any custom state file — no `sync_state.json` needed.
|
|
|
|
### 4.3 `.gitignore`
|
|
|
|
```gitignore
|
|
# Obsidian internals
|
|
.obsidian/
|
|
|
|
# Sync config (contains API token)
|
|
settings.json
|
|
|
|
# Conflict sidecars (resolved manually, not tracked)
|
|
*.conflict.md
|
|
|
|
# OS
|
|
.DS_Store
|
|
Thumbs.db
|
|
```
|
|
|
|
### 4.4 `.gitattributes`
|
|
|
|
```gitattributes
|
|
# Treat all markdown as text, normalize line endings
|
|
*.md text eol=lf
|
|
|
|
# Tell git to always use union merge on the sync log
|
|
# (append-only, never conflicts)
|
|
_sync_log.md merge=union
|
|
```
|
|
|
|
### 4.5 `settings.json` (not committed)
|
|
|
|
```json
|
|
{
|
|
"source": {
|
|
"url": "http://outline:3000",
|
|
"token": "ol_api_YOUR_TOKEN_HERE"
|
|
},
|
|
"sync": {
|
|
"vault_dir": "/home/crabix/docker_authentik/outline-vault",
|
|
"enable_deletions": false,
|
|
"auto_push": false
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 5. Frontmatter as the ID Layer
|
|
|
|
Every synced file carries its Outline identity in YAML frontmatter. This is the bridge between git paths and Outline document IDs.
|
|
|
|
```yaml
|
|
---
|
|
outline_id: abc-123
|
|
outline_collection_id: col-456
|
|
outline_parent_id: def-789 # omitted if root document
|
|
outline_updated_at: 2026-03-03T10:30:00Z
|
|
---
|
|
|
|
# CV
|
|
|
|
Actual document content here...
|
|
```
|
|
|
|
**Rules:**
|
|
- Frontmatter is **stripped** before sending content to Outline API
|
|
- Frontmatter is **written back** after each successful API call with updated `outline_updated_at`
|
|
- Frontmatter travels with the file on rename/move — the Outline ID is never lost
|
|
- New files created in Obsidian have **no frontmatter** until first push
|
|
- The sync script identifies new files by the absence of `outline_id` in frontmatter
|
|
|
|
---
|
|
|
|
## 6. Architecture: Outline as a "Dumb Remote"
|
|
|
|
Git handles change detection, merging, and conflict marking. The sync script is purely an **API translation layer**.
|
|
|
|
```
|
|
git pull equivalent:
|
|
Export Outline → temp dir → convert to frontmatter format
|
|
→ git commit to `outline` branch
|
|
→ git merge outline into main
|
|
|
|
git push equivalent:
|
|
git diff outline..main → list of changed files
|
|
→ for each file: call appropriate Outline API endpoint
|
|
→ write updated frontmatter back
|
|
→ advance outline branch tip
|
|
```
|
|
|
|
### 6.1 Change Detection Matrix
|
|
|
|
`git diff outline..main --name-status --find-renames` produces all cases:
|
|
|
|
| Git status | Outline API call |
|
|
|---|---|
|
|
| `M` modified | `documents.update(id, title, text)` |
|
|
| `A` added (with `outline_id` in frontmatter) | Already synced — skip |
|
|
| `A` added (no `outline_id`) | `documents.create(collectionId, title, text)` → write frontmatter |
|
|
| `D` deleted | `documents.delete(id)` — only if `enable_deletions: true` |
|
|
| `R` renamed/moved | `documents.update(id, title)` + `documents.move(id, newParentId)` if folder changed |
|
|
|
|
### 6.2 New Outline API Calls Required
|
|
|
|
Existing scripts only use `documents.create`. Sync additionally requires:
|
|
|
|
| Endpoint | Used for |
|
|
|---|---|
|
|
| `documents.update` | Core push operation — preserves history |
|
|
| `documents.move` | Reparent when file moved between folders |
|
|
| `documents.delete` | Deletion (flag-gated) |
|
|
| `documents.list` | Detect remote changes during pull |
|
|
| `collections.list` | Detect new/deleted collections during pull |
|
|
| `collections.create` | Create collection for new top-level folder |
|
|
| `collections.delete` | Delete collection (flag-gated) |
|
|
|
|
---
|
|
|
|
## 7. Sync Commands
|
|
|
|
### `sync.sh init`
|
|
First-time setup. Creates vault from current Outline state.
|
|
```bash
|
|
./sync.sh init
|
|
```
|
|
Steps:
|
|
1. Run existing `outline_export_fixed.py` into temp dir
|
|
2. `git init` + create `outline` and `main` branches
|
|
3. Convert export metadata headers → YAML frontmatter in each `.md` file
|
|
4. Commit to `outline` branch: `git commit -m "sync: initial import"`
|
|
5. Checkout `main`, merge `outline`
|
|
6. Write `.gitignore`, `.gitattributes`
|
|
|
|
### `sync.sh pull`
|
|
Pull latest Outline state into Obsidian.
|
|
```bash
|
|
./sync.sh pull [--auto]
|
|
```
|
|
Steps:
|
|
1. Export Outline → temp dir
|
|
2. Convert to frontmatter format
|
|
3. Checkout `outline` branch, apply changes, commit: `git commit -m "sync: outline@TIMESTAMP"`
|
|
4. Checkout `main`
|
|
5. `git merge outline`
|
|
6. If conflict: write `*.conflict.md` sidecars, append to `_sync_log.md`, exit non-zero
|
|
7. If clean: append success to `_sync_log.md`
|
|
|
|
### `sync.sh push`
|
|
Push local Obsidian changes to Outline.
|
|
```bash
|
|
./sync.sh push [--dry-run]
|
|
```
|
|
Steps:
|
|
1. Check for unresolved merge conflicts — abort if any
|
|
2. `git diff outline..main --name-status --find-renames`
|
|
3. For each changed file: call Outline API (topological order for new docs)
|
|
4. Write updated frontmatter, `git add` the file
|
|
5. On full success: `git checkout outline && git merge main`
|
|
6. On partial failure: log errors, do not advance `outline` branch — next push retries failed files
|
|
|
|
### `sync.sh` (bidirectional)
|
|
Pull, then push if pull was clean.
|
|
```bash
|
|
./sync.sh [--dry-run] [--auto]
|
|
```
|
|
|
|
### `sync.sh status`
|
|
Show pending local changes not yet in Outline.
|
|
```bash
|
|
./sync.sh status
|
|
```
|
|
Output: `git diff outline..main --name-status` formatted as human-readable table.
|
|
|
|
### `sync.sh resolve FILE --accept [local|remote]`
|
|
Resolve a merge conflict.
|
|
```bash
|
|
./sync.sh resolve Bewerbungen/CV.md --accept local
|
|
./sync.sh resolve Bewerbungen/CV.md --accept remote
|
|
```
|
|
|
|
---
|
|
|
|
## 8. Conflict Handling
|
|
|
|
A conflict occurs when `git merge outline` produces conflict markers (same document edited on both sides since last sync).
|
|
|
|
```
|
|
Bewerbungen/CV.md ← contains git conflict markers, NOT pushed
|
|
Bewerbungen/CV.conflict.md ← human-readable: what changed locally vs remotely
|
|
```
|
|
|
|
`_sync_log.md` always reflects current sync state (readable in Obsidian):
|
|
```markdown
|
|
## 2026-03-04 08:00
|
|
- pull: 3 updated, 1 CONFLICT (Bewerbungen/CV.md) — push blocked
|
|
- push: blocked pending conflict resolution
|
|
```
|
|
|
|
---
|
|
|
|
## 9. Scheduled Sync via Ofelia
|
|
|
|
```ini
|
|
[job-exec "outline-pull"]
|
|
schedule = @every 1h
|
|
container = outline-sync
|
|
command = /work/sync.sh pull --auto
|
|
|
|
# Push is manual by default. Enable only after P6 (conflict detection) is complete.
|
|
# [job-exec "outline-push"]
|
|
# schedule = @daily
|
|
# container = outline-sync
|
|
# command = /work/sync.sh push --auto
|
|
```
|
|
|
|
`--auto` flag: suppresses interactive prompts, writes to `_sync_log.md`, exits non-zero on conflict so Ofelia can detect failures.
|
|
|
|
---
|
|
|
|
## 10. Implementation Phases + User Stories
|
|
|
|
---
|
|
|
|
### Phase 1 — `sync init`
|
|
|
|
**Scope:** Create vault from current Outline state. Git initialized. Frontmatter injected.
|
|
|
|
#### User Stories
|
|
|
|
**US-1.1** — As a user, I want to run a single command that sets up the vault from scratch, so that I don't need to manually configure git or copy files.
|
|
- Given: `outline-vault/` does not exist (or is empty)
|
|
- When: `./sync.sh init` is run
|
|
- Then: The vault folder is created, git is initialized with `outline` and `main` branches, all Outline documents are exported as `.md` files with frontmatter, and `main` is checked out
|
|
|
|
**US-1.2** — As a user, I want every exported document to have frontmatter with its Outline ID, so that future syncs can identify documents without an external map.
|
|
- Given: A document exists in Outline with id `abc-123`
|
|
- When: `sync init` completes
|
|
- Then: The corresponding `.md` file contains `outline_id: abc-123` in its YAML frontmatter
|
|
|
|
**US-1.3** — As a user, I want the vault folder structure to mirror Outline's collection/document hierarchy, so that navigation in Obsidian feels natural.
|
|
- Given: Outline has collection "Bewerbungen" with nested document "Tipico > Pitch"
|
|
- When: `sync init` completes
|
|
- Then: Files exist at `Bewerbungen/Tipico/Tipico.md` and `Bewerbungen/Tipico/Pitch.md`
|
|
|
|
**US-1.4** — As a user, I want `settings.json` to be gitignored automatically, so that my API token is never accidentally committed.
|
|
- Given: `sync init` runs
|
|
- Then: `.gitignore` is created containing `settings.json` and `*.conflict.md`
|
|
|
|
---
|
|
|
|
### Phase 2 — `sync pull`
|
|
|
|
**Scope:** Export Outline → commit to `outline` branch → merge into `main`.
|
|
|
|
#### User Stories
|
|
|
|
**US-2.1** — As a user, I want to run `sync pull` to get the latest Outline changes into Obsidian, so that edits made on the web are reflected locally.
|
|
- Given: A document was updated in Outline since last sync
|
|
- When: `./sync.sh pull` is run
|
|
- Then: The corresponding `.md` file is updated in the vault with the new content and updated `outline_updated_at` frontmatter
|
|
|
|
**US-2.2** — As a user, I want new Outline documents to appear as new files after pull, so that I don't miss newly created content.
|
|
- Given: A new document was created in Outline since last sync
|
|
- When: `./sync.sh pull` is run
|
|
- Then: A new `.md` file exists with correct frontmatter, in the correct folder
|
|
|
|
**US-2.3** — As a user, I want deleted Outline documents to be removed locally after pull, so that my vault stays in sync with Outline.
|
|
- Given: A document was deleted in Outline since last sync
|
|
- When: `./sync.sh pull` is run
|
|
- Then: The corresponding local `.md` file is removed
|
|
|
|
**US-2.4** — As a user, I want to be informed if a pull produces a conflict (same document edited both locally and in Outline), so that I can resolve it manually.
|
|
- Given: `CV.md` was edited locally AND in Outline since last sync
|
|
- When: `./sync.sh pull` is run
|
|
- Then: A `CV.conflict.md` sidecar is created, `_sync_log.md` records the conflict, the command exits non-zero, and `CV.md` is left untouched
|
|
|
|
**US-2.5** — As a user running pull in `--auto` mode (cron), I want the sync log to be updated even on failure, so that I can check `_sync_log.md` in Obsidian to understand what happened.
|
|
- Given: `./sync.sh pull --auto` runs via Ofelia and encounters a conflict
|
|
- Then: `_sync_log.md` is updated with timestamp and conflict details, exit code is non-zero
|
|
|
|
---
|
|
|
|
### Phase 3 — `sync status`
|
|
|
|
**Scope:** Show pending local changes not yet pushed to Outline.
|
|
|
|
#### User Stories
|
|
|
|
**US-3.1** — As a user, I want to see which local files have changed since last sync, so that I know what will be pushed before running push.
|
|
- Given: `CV.md` was edited locally and not yet pushed
|
|
- When: `./sync.sh status` is run
|
|
- Then: Output shows `M Bewerbungen/CV.md` (modified)
|
|
|
|
**US-3.2** — As a user, I want `status` to also show new files that will be created in Outline on push, so that I can review before committing.
|
|
- Given: `NewNote.md` was created in Obsidian with no frontmatter
|
|
- When: `./sync.sh status` is run
|
|
- Then: Output shows `A Bewerbungen/NewNote.md` (new → will create in Outline)
|
|
|
|
**US-3.3** — As a user, I want `status` to exit cleanly when there are no pending changes, so that cron jobs don't produce false alarms.
|
|
- Given: No local changes since last sync
|
|
- When: `./sync.sh status` is run
|
|
- Then: Output is "Nothing to push. Outline is up to date." and exit code is 0
|
|
|
|
---
|
|
|
|
### Phase 4 — `sync push` (modify + rename)
|
|
|
|
**Scope:** Push modified and renamed/moved files to Outline. No new doc creation yet.
|
|
|
|
#### User Stories
|
|
|
|
**US-4.1** — As a user, I want editing a file in Obsidian and running push to update the document in Outline, so that my local edits appear on the web.
|
|
- Given: `CV.md` was edited in Obsidian (has `outline_id` in frontmatter)
|
|
- When: `./sync.sh push` is run
|
|
- Then: Outline document `abc-123` is updated with new content, document history is preserved (not recreated), `outline_updated_at` in frontmatter is updated
|
|
|
|
**US-4.2** — As a user, I want renaming a file in Obsidian to update the document title in Outline, so that both sides stay consistent.
|
|
- Given: `CV.md` is renamed to `Lebenslauf.md` in Obsidian (frontmatter `outline_id` intact)
|
|
- When: `./sync.sh push` is run
|
|
- Then: Outline document title is updated to "Lebenslauf", document URL is preserved
|
|
|
|
**US-4.3** — As a user, I want moving a file to a different folder in Obsidian to reparent the document in Outline, so that the hierarchy stays consistent.
|
|
- Given: `Bewerbungen/CV.md` is moved to `Projekte/CV.md` in Obsidian
|
|
- When: `./sync.sh push` is run
|
|
- Then: In Outline, the document is moved to the "Projekte" collection with the new parent
|
|
|
|
**US-4.4** — As a user, I want push to be atomic per-document — a failure on one file should not block others.
|
|
- Given: Push is run with 5 changed files, one API call fails
|
|
- When: `./sync.sh push` completes
|
|
- Then: 4 documents are updated in Outline, 1 failure is logged, the `outline` branch is NOT advanced (failed file will retry on next push)
|
|
|
|
**US-4.5** — As a user, I want `--dry-run` to show me exactly what API calls would be made without touching Outline.
|
|
- Given: 3 files modified locally
|
|
- When: `./sync.sh push --dry-run`
|
|
- Then: Output lists each file and the API call that would be made, no changes to Outline or git
|
|
|
|
---
|
|
|
|
### Phase 5 — `sync push` (new documents)
|
|
|
|
**Scope:** Create new Outline documents from new Obsidian files.
|
|
|
|
#### User Stories
|
|
|
|
**US-5.1** — As a user, I want creating a new `.md` file in an existing collection folder to create a new document in Outline on push.
|
|
- Given: `Bewerbungen/NewApplication.md` created in Obsidian, no frontmatter
|
|
- When: `./sync.sh push` is run
|
|
- Then: Document created in Outline under "Bewerbungen" collection, frontmatter written back with new `outline_id`
|
|
|
|
**US-5.2** — As a user, I want creating a file in a subfolder to correctly set the parent document in Outline, preserving hierarchy.
|
|
- Given: `Bewerbungen/Tipico/NewDoc.md` created, `Tipico.md` has `outline_id: def-789`
|
|
- When: `./sync.sh push` is run
|
|
- Then: Outline document created with `parentDocumentId: def-789`
|
|
|
|
**US-5.3** — As a user, I want creating a new top-level folder to create a new Outline collection on push.
|
|
- Given: `NewCollection/FirstDoc.md` created in Obsidian
|
|
- When: `./sync.sh push` is run
|
|
- Then: New collection "NewCollection" is created in Outline, `FirstDoc` is created inside it
|
|
|
|
**US-5.4** — As a user, I want parent documents to always be created before their children, even if I created them in reverse order.
|
|
- Given: `Bewerbungen/Tipico/NewChild.md` created, but `Bewerbungen/Tipico/Tipico.md` is also new
|
|
- When: `./sync.sh push` is run
|
|
- Then: `Tipico.md` is created first, then `NewChild.md` with correct `parentDocumentId`
|
|
|
|
---
|
|
|
|
### Phase 6 — Conflict Detection
|
|
|
|
**Scope:** Block push when unresolved conflicts exist. Generate conflict sidecars.
|
|
|
|
#### User Stories
|
|
|
|
**US-6.1** — As a user, I want push to be blocked if there are unresolved merge conflicts, so that I never push broken content.
|
|
- Given: `CV.md` contains git conflict markers from a previous pull
|
|
- When: `./sync.sh push` is run
|
|
- Then: Push aborts with "Resolve conflicts before pushing", lists conflicted files, exits non-zero
|
|
|
|
**US-6.2** — As a user, I want a human-readable conflict sidecar file, so that I can understand what changed on each side.
|
|
- Given: `CV.md` conflicts during pull
|
|
- When: Pull completes
|
|
- Then: `CV.conflict.md` exists containing: last synced content, local changes, Outline changes, and instructions for resolution
|
|
|
|
**US-6.3** — As a user, I want non-conflicted files to be pushed successfully even when other files have conflicts.
|
|
- Given: `CV.md` has conflict, `Projekte/Roadmap.md` is cleanly modified
|
|
- When: `./sync.sh push` is run (after resolving or acknowledging the conflict situation)
|
|
- Then: Only `Roadmap.md` is pushed; `CV.md` is skipped with a warning
|
|
|
|
---
|
|
|
|
### Phase 7 — `sync resolve`
|
|
|
|
**Scope:** Command to resolve conflicts without needing raw git knowledge.
|
|
|
|
#### User Stories
|
|
|
|
**US-7.1** — As a user, I want to resolve a conflict by accepting the local version with a single command.
|
|
- Given: `CV.md` is in conflict
|
|
- When: `./sync.sh resolve Bewerbungen/CV.md --accept local`
|
|
- Then: Conflict markers removed, local content kept, file staged, `CV.conflict.md` deleted
|
|
|
|
**US-7.2** — As a user, I want to resolve a conflict by accepting the remote (Outline) version with a single command.
|
|
- Given: `CV.md` is in conflict
|
|
- When: `./sync.sh resolve Bewerbungen/CV.md --accept remote`
|
|
- Then: Conflict markers removed, Outline's content kept, file staged, `CV.conflict.md` deleted
|
|
|
|
**US-7.3** — As a user, I want `sync status` to clearly show which files are in conflict, so that I know what needs resolving.
|
|
- Given: `CV.md` and `Notes.md` are conflicted
|
|
- When: `./sync.sh status`
|
|
- Then: Output shows both files marked as `CONFLICT` with instructions to run `sync resolve`
|
|
|
|
---
|
|
|
|
### Phase 8 — Ofelia + `_sync_log.md`
|
|
|
|
**Scope:** Unattended scheduled pull. Human-readable log in vault.
|
|
|
|
#### User Stories
|
|
|
|
**US-8.1** — As a user, I want hourly automatic pulls from Outline so that Obsidian stays current without manual intervention.
|
|
- Given: Ofelia runs `sync.sh pull --auto` every hour
|
|
- When: New documents were created in Outline during the hour
|
|
- Then: New `.md` files appear in vault, `_sync_log.md` records the event
|
|
|
|
**US-8.2** — As a user, I want to check `_sync_log.md` in Obsidian to see the history of syncs, so that I can diagnose issues without accessing a terminal.
|
|
- Given: Multiple sync runs have occurred
|
|
- When: Opening `_sync_log.md` in Obsidian
|
|
- Then: A chronological log of pull/push operations, counts of changes, and any errors or conflicts is visible
|
|
|
|
**US-8.3** — As a user, I want failed syncs to be visible in Ofelia's output so that I can be alerted.
|
|
- Given: A pull encounters a conflict
|
|
- When: Ofelia's job completes
|
|
- Then: Exit code is non-zero, Ofelia marks the job as failed
|
|
|
|
---
|
|
|
|
### Phase 9 — Deletions (flag-gated)
|
|
|
|
**Scope:** Handle deleted files/documents. Off by default.
|
|
|
|
#### User Stories
|
|
|
|
**US-9.1** — As a user, I want to opt into deletion sync so that documents I delete locally are also removed from Outline.
|
|
- Given: `enable_deletions: true` in `settings.json`, `OldNote.md` deleted in Obsidian
|
|
- When: `./sync.sh push`
|
|
- Then: Outline document is deleted, frontmatter entry removed
|
|
|
|
**US-9.2** — As a user, I want deletions to be off by default so that accidental file deletion never cascades to Outline without explicit opt-in.
|
|
- Given: `enable_deletions: false` (default), `OldNote.md` deleted in Obsidian
|
|
- When: `./sync.sh push`
|
|
- Then: Deletion is logged as a warning, Outline document is untouched
|
|
|
|
**US-9.3** — As a user, I want `--dry-run` to show pending deletions so that I can review before enabling.
|
|
- Given: `enable_deletions: true`, 2 files deleted locally
|
|
- When: `./sync.sh push --dry-run`
|
|
- Then: Output shows 2 documents marked for deletion, no Outline changes made
|
|
|
|
---
|
|
|
|
## 11. Automated Testing
|
|
|
|
All tests run inside Docker (same `python:3.11-slim` + `domnet` network as existing scripts). A dedicated test collection `_sync_test_TIMESTAMP` is created in Outline at test start and deleted at teardown.
|
|
|
|
### Test Runner
|
|
|
|
```bash
|
|
./sync_tests.sh # run all tests
|
|
./sync_tests.sh --phase 1 # run tests for specific phase
|
|
./sync_tests.sh --keep # don't delete test collection on failure (debug)
|
|
./sync_tests.sh --verbose # show full API request/response
|
|
```
|
|
|
|
### 11.1 Phase 1 Tests — Init
|
|
|
|
```
|
|
TEST-1.1 init creates vault directory if not exists
|
|
TEST-1.2 init creates git repo with outline and main branches
|
|
TEST-1.3 init exports all Outline collections as top-level folders
|
|
TEST-1.4 init injects frontmatter into every .md file
|
|
TEST-1.5 frontmatter contains: outline_id, outline_collection_id, outline_updated_at
|
|
TEST-1.6 outline_id in frontmatter matches actual Outline document ID (API verified)
|
|
TEST-1.7 nested documents are in correct subfolder matching hierarchy
|
|
TEST-1.8 settings.json is listed in .gitignore
|
|
TEST-1.9 .obsidian/ is listed in .gitignore
|
|
TEST-1.10 outline branch and main branch are at same commit after init
|
|
TEST-1.11 re-running init on existing vault aborts with clear error message
|
|
```
|
|
|
|
### 11.2 Phase 2 Tests — Pull (Outline → Local)
|
|
|
|
**Read direction:**
|
|
```
|
|
TEST-2.1 pull: modified Outline document updates local .md content
|
|
TEST-2.2 pull: modified Outline document updates outline_updated_at in frontmatter
|
|
TEST-2.3 pull: outline_id in frontmatter unchanged after content update
|
|
TEST-2.4 pull: new Outline document appears as new .md file in correct folder
|
|
TEST-2.5 pull: new nested Outline document appears in correct subfolder
|
|
TEST-2.6 pull: new Outline collection appears as new top-level folder
|
|
TEST-2.7 pull: deleted Outline document removes local .md file
|
|
TEST-2.8 pull: deleted Outline collection removes local folder
|
|
TEST-2.9 pull: unchanged documents are not modified (mtime unchanged)
|
|
TEST-2.10 pull: git commits to outline branch, not main
|
|
TEST-2.11 pull: outline branch advances, main branch merges cleanly
|
|
TEST-2.12 pull: _sync_log.md updated with counts and timestamp
|
|
```
|
|
|
|
**Conflict detection:**
|
|
```
|
|
TEST-2.13 pull: same document edited locally + in Outline → conflict detected
|
|
TEST-2.14 pull: conflict produces *.conflict.md sidecar file
|
|
TEST-2.15 pull: conflict sidecar contains local diff and remote diff sections
|
|
TEST-2.16 pull: conflicted file is left untouched (not overwritten)
|
|
TEST-2.17 pull: non-conflicted files merge successfully despite other conflicts
|
|
TEST-2.18 pull: --auto mode exits non-zero on conflict
|
|
TEST-2.19 pull: _sync_log.md records conflict with filename and timestamp
|
|
```
|
|
|
|
### 11.3 Phase 3 Tests — Status
|
|
|
|
```
|
|
TEST-3.1 status: shows M for locally modified file
|
|
TEST-3.2 status: shows A for new local file without outline_id
|
|
TEST-3.3 status: shows D for locally deleted file (even if deletions off)
|
|
TEST-3.4 status: shows R for renamed/moved file
|
|
TEST-3.5 status: shows CONFLICT for files with unresolved merge conflicts
|
|
TEST-3.6 status: shows nothing when vault is clean
|
|
TEST-3.7 status: exit code 0 in all cases (informational only)
|
|
TEST-3.8 status: does not modify any files or git state
|
|
```
|
|
|
|
### 11.4 Phase 4 Tests — Push Modified + Renamed
|
|
|
|
**Update direction:**
|
|
```
|
|
TEST-4.1 push: modified local file calls documents.update (not create)
|
|
TEST-4.2 push: Outline document content matches local file content after push (strip frontmatter)
|
|
TEST-4.3 push: documents.update preserves document history (version count increases)
|
|
TEST-4.4 push: outline_updated_at in frontmatter updated to API response timestamp
|
|
TEST-4.5 push: outline_id unchanged after update
|
|
TEST-4.6 push: Outline document URL unchanged after update (ID preserved)
|
|
TEST-4.7 push: renamed file → Outline document title updated
|
|
TEST-4.8 push: renamed file → Outline document ID unchanged
|
|
TEST-4.9 push: file moved to different folder → documents.move called
|
|
TEST-4.10 push: moved file → document appears under new parent in Outline
|
|
TEST-4.11 push: moved file → document removed from old parent in Outline
|
|
TEST-4.12 push: file moved to different top-level folder → document moved to different collection
|
|
TEST-4.13 push: API failure on one file → other files still pushed
|
|
TEST-4.14 push: failed file → outline branch NOT advanced
|
|
TEST-4.15 push: failed file → retried on next push
|
|
TEST-4.16 push: --dry-run → no Outline changes, no git state changes
|
|
TEST-4.17 push: --dry-run → lists all API calls that would be made
|
|
TEST-4.18 push: frontmatter stripped from content sent to Outline API
|
|
TEST-4.19 push: Outline document does NOT contain frontmatter YAML in its body
|
|
```
|
|
|
|
### 11.5 Phase 5 Tests — Push New Documents
|
|
|
|
```
|
|
TEST-5.1 push: new file in existing collection folder → documents.create called
|
|
TEST-5.2 push: new document appears in correct Outline collection
|
|
TEST-5.3 push: outline_id written back to frontmatter after create
|
|
TEST-5.4 push: outline_collection_id written back to frontmatter after create
|
|
TEST-5.5 push: new file in subfolder → parentDocumentId set correctly
|
|
TEST-5.6 push: parent document outline_id used as parentDocumentId
|
|
TEST-5.7 push: new file in new top-level folder → collection created first
|
|
TEST-5.8 push: new file in new top-level folder → document created after collection
|
|
TEST-5.9 push: two new files parent+child → parent created before child (topological order)
|
|
TEST-5.10 push: three-level new hierarchy → correct creation order preserved
|
|
TEST-5.11 push: new file with no frontmatter → document still created (title from filename)
|
|
TEST-5.12 push: created document is published (not draft)
|
|
```
|
|
|
|
### 11.6 Phase 6 Tests — Conflict Detection
|
|
|
|
```
|
|
TEST-6.1 push blocked when any file contains git conflict markers
|
|
TEST-6.2 push blocked lists all conflicted files
|
|
TEST-6.3 push blocked exits non-zero
|
|
TEST-6.4 non-conflicted files can still be pushed despite other files in conflict
|
|
TEST-6.5 conflict sidecar .conflict.md contains "LOCAL" and "REMOTE" sections
|
|
TEST-6.6 conflict sidecar contains instructions for resolution commands
|
|
TEST-6.7 outline branch not advanced when push is blocked by conflicts
|
|
```
|
|
|
|
### 11.7 Phase 7 Tests — Resolve
|
|
|
|
```
|
|
TEST-7.1 resolve --accept local → file contains local content
|
|
TEST-7.2 resolve --accept local → no conflict markers remain
|
|
TEST-7.3 resolve --accept local → .conflict.md sidecar deleted
|
|
TEST-7.4 resolve --accept local → file is staged (git add)
|
|
TEST-7.5 resolve --accept remote → file contains Outline content
|
|
TEST-7.6 resolve --accept remote → no conflict markers remain
|
|
TEST-7.7 resolve --accept remote → .conflict.md sidecar deleted
|
|
TEST-7.8 resolve --accept remote → file is staged (git add)
|
|
TEST-7.9 resolve on non-conflicted file → error message, no changes
|
|
TEST-7.10 after resolve, push succeeds for that file
|
|
```
|
|
|
|
### 11.8 Phase 8 Tests — Ofelia / Auto mode
|
|
|
|
```
|
|
TEST-8.1 --auto flag: no interactive prompts, completes without stdin
|
|
TEST-8.2 --auto flag: _sync_log.md updated on success
|
|
TEST-8.3 --auto flag: _sync_log.md updated on conflict
|
|
TEST-8.4 --auto flag: exit code 0 on clean pull
|
|
TEST-8.5 --auto flag: exit code non-zero on conflict
|
|
TEST-8.6 _sync_log.md uses union merge strategy (no conflicts on the log itself)
|
|
TEST-8.7 _sync_log.md entries are append-only (history preserved)
|
|
```
|
|
|
|
### 11.9 Phase 9 Tests — Deletions
|
|
|
|
```
|
|
TEST-9.1 enable_deletions false: locally deleted file → no Outline change
|
|
TEST-9.2 enable_deletions false: warning logged in _sync_log.md
|
|
TEST-9.3 enable_deletions true: locally deleted file → documents.delete called
|
|
TEST-9.4 enable_deletions true: Outline document is gone after push
|
|
TEST-9.5 enable_deletions true: deleted folder → collection deleted after all docs deleted
|
|
TEST-9.6 enable_deletions true: --dry-run lists pending deletions without executing
|
|
TEST-9.7 Outline-side deletion during pull: local file removed (regardless of enable_deletions)
|
|
TEST-9.8 Outline-side deletion during pull: deletion logged in _sync_log.md
|
|
```
|
|
|
|
### 11.10 Full Round-Trip Tests
|
|
|
|
These run the complete cycle and verify end-to-end consistency.
|
|
|
|
```
|
|
TEST-RT.1 Create in Outline → pull → verify local file content + frontmatter
|
|
TEST-RT.2 Create in Obsidian → push → verify Outline document content + ID in frontmatter
|
|
TEST-RT.3 Edit in Outline → pull → edit locally → push → verify Outline has latest content
|
|
TEST-RT.4 Edit locally → push → edit in Outline → pull → verify local has Outline content
|
|
TEST-RT.5 Edit same doc in Outline AND locally → pull → verify conflict detected → resolve local → push → verify Outline updated
|
|
TEST-RT.6 Edit same doc in Outline AND locally → pull → verify conflict detected → resolve remote → push → verify Outline unchanged, local matches Outline
|
|
TEST-RT.7 Rename locally → push → pull (clean) → verify no duplicate documents
|
|
TEST-RT.8 Move locally → push → pull (clean) → verify hierarchy correct on both sides
|
|
TEST-RT.9 Delete locally (deletions on) → push → verify gone from Outline
|
|
TEST-RT.10 Delete in Outline → pull → verify gone locally
|
|
TEST-RT.11 Create parent+child locally → push → verify parent-child relationship in Outline
|
|
TEST-RT.12 Full CRUD cycle: create → edit → rename → move → delete, verify Outline matches at each step
|
|
```
|
|
|
|
---
|
|
|
|
## 12. What Is NOT Written from Scratch
|
|
|
|
| Problem | Solution |
|
|
|---|---|
|
|
| Change detection | `git diff` |
|
|
| Three-way merge | `git merge` |
|
|
| Conflict markers | git native |
|
|
| History + rollback | `git log` / `git reset` |
|
|
| Local auto-commit | Obsidian Git plugin |
|
|
| API retry + rate limiting | Reuse `OutlineImporter._api_request()` from `outline_import.py` |
|
|
| Docker network execution | Reuse pattern from `import_to_outline.sh` |
|
|
| Export logic | Reuse `outline_export_fixed.py` unchanged |
|
|
|
|
**New code only:** `outline_sync.py` (~500 lines) + `sync.sh` wrapper (~100 lines).
|
|
|
|
---
|
|
|
|
## 13. Risks
|
|
|
|
| Risk | Mitigation |
|
|
|---|---|
|
|
| `git merge` fails in unattended cron | Pull conflicts block push only; clean files still merged; logged to `_sync_log.md` |
|
|
| Frontmatter stripped incorrectly before push | Unit test the strip/restore; verify Outline body does not contain YAML |
|
|
| `outline_updated_at` clock skew | Use API response timestamp, not local clock; normalize to UTC |
|
|
| New file in unknown folder (not an Outline collection) | Warn and skip; require top-level folder to match existing collection or be brand new |
|
|
| Outline deletes document between pull and push | `documents.update` returns 404 → log as conflict, skip |
|
|
| Obsidian Git plugin commits during sync | Sync script checks for clean working tree before starting; aborts if dirty |
|
|
| `outline` branch diverges from `main` after long conflict | `sync status` shows divergence; `sync pull` always catches up |
|