# Outline Import Script - Product Requirements Document

**Document Version:** 1.0
**Created:** 2026-01-17
**Last Updated:** 2026-01-19
**Status:** Draft

---

## 1. Executive Summary

Create `import_to_outline.sh` - a companion script to the existing export tool that imports markdown files back into Outline. The script restores documents with their full hierarchy using metadata preserved during export, enabling disaster recovery, migration between Outline instances, and content restoration workflows.

---

## 2. Problem Statement

### Current State
- Export functionality exists via `export_with_trees.sh` and `outline_export_fixed.py`
- Exports include markdown content and `_collection_metadata.json` with full hierarchy
- No automated way to restore or migrate exported content back into Outline

### Pain Points
1. **Disaster Recovery**: Manual recreation of collections and documents after data loss
2. **Migration**: No tooling to move content between Outline instances
3. **Restore Workflow**: Cannot selectively restore deleted documents or collections
4. **Testing**: No way to verify export integrity via round-trip import

### Business Impact
- Hours of manual work to rebuild wiki structure after incidents
- Risk of hierarchy/relationship loss during manual restoration
- No confidence in backup validity without restore testing

---

## 3. Goals & Success Criteria

### Primary Goals
1. Restore exported markdown files to Outline with preserved hierarchy
2. Support both full restore and selective import workflows
3. Provide clear feedback on import progress and results

### Success Criteria
| Metric | Target |
|--------|--------|
| Document import success rate | ≥99% |
| Hierarchy accuracy | 100% parent-child relationships preserved |
| Performance | ≥10 documents/second |
| Dry-run accuracy | 100% match between preview and actual import |

### Non-Goals
- Image/attachment import (future enhancement)
- Conflict resolution with existing content (skip or fail)
- Real-time sync between instances
- User/permission migration

---

## 4. User Stories

### US-1: Disaster Recovery
> As an **administrator**, I want to **restore all collections from a backup** so that **I can recover from data loss**.

**Acceptance Criteria:**
- Import all collections from `outline_export/` directory
- Recreate exact hierarchy as shown in metadata
- Report success/failure summary

### US-2: Selective Restoration
> As a **user**, I want to **import a single collection** so that **I can restore specific content without affecting other data**.

**Acceptance Criteria:**
- Specify source directory containing single collection
- Create collection if it doesn't exist
- Skip import if collection already exists (configurable)

### US-3: Migration to New Instance
> As an **administrator**, I want to **import all content into a fresh Outline instance** so that **I can migrate to new infrastructure**.

**Acceptance Criteria:**
- Works against empty Outline instance
- Creates all collections and documents
- Preserves document nesting structure

### US-4: Safe Preview
> As a **user**, I want to **preview what will be imported** so that **I can verify before making changes**.

**Acceptance Criteria:**
- `--dry-run` flag shows all planned operations
- No API calls that modify data during dry run
- Output matches actual import behavior

### US-5: Consolidated Import
> As a **user**, I want to **import multiple collections into a single new collection** so that **I can reorganize content during import**.

**Acceptance Criteria:**
- `--single` mode creates timestamped collection
- Original collection names become top-level documents
- All nested hierarchy preserved under these parents

---

## 5. Functional Requirements

### 5.1 Import Modes

#### Mode 1: Collection-per-Folder (Default)
```
outline_export/
├── Bewerbungen/     → Creates "Bewerbungen" collection
├── Projekte/        → Creates "Projekte" collection
└── Privat/          → Creates "Privat" collection
```

**Behavior:**
- Each subdirectory in source becomes a separate collection
- Collection names match folder names exactly
- If collection exists: skip entire collection (default) or error

#### Mode 2: Single Collection (`--single`)
```
outline_export/
├── Bewerbungen/     → Becomes parent doc "Bewerbungen"
├── Projekte/        → Becomes parent doc "Projekte"
└── Privat/          → Becomes parent doc "Privat"

All imported into: "import_20260119_143052" collection
```

**Behavior:**
- Creates one collection named `import_YYYYMMDD_HHMMSS`
- Each original collection folder becomes a top-level parent document
- Original document hierarchy nested under these parents

### 5.2 Command-Line Interface

```bash
./import_to_outline.sh [OPTIONS]

Options:
  -s, --single           Import all into single timestamped collection
  -n, --dry-run          Preview operations without making changes
  -d, --source DIR       Source directory (default: outline_export)
  -v, --verbose          Increase output verbosity (-vv for debug)
  -f, --force            Overwrite existing collections (instead of skip)
  --settings FILE        Path to settings file (default: settings.json)
  -h, --help             Show help message
```

### 5.3 Document Creation Logic

#### Hierarchy Reconstruction Algorithm
```
1. Load _collection_metadata.json
2. Build document tree from `documents` array (using parent_id)
3. Topological sort: ensure parents created before children
4. For each document in sorted order:
   a. Read markdown content from file
   b. Map old parent_id → new parent_id (from creation responses)
   c. Create document via API with parentDocumentId
   d. Store id mapping: old_id → new_id
5. Verify: created count matches expected count
```

#### ID Mapping Example
```
Export metadata:
  doc_A (id: abc-123, parent_id: null)
  doc_B (id: def-456, parent_id: abc-123)

After creating doc_A:
  id_map = { "abc-123": "new-789" }

Creating doc_B:
  parent_id = id_map["abc-123"] = "new-789"
  API call: create doc_B with parentDocumentId: "new-789"
```

### 5.4 Duplicate Handling

| Scenario | Default Behavior | With `--force` |
|----------|------------------|----------------|
| Collection exists | Skip entire collection | Delete and recreate |
| Document title exists in collection | Skip document | Update document |

### 5.5 Error Handling

| Error Type | Behavior |
|------------|----------|
| API connection failure | Abort with error message |
| Collection creation fails | Abort that collection, continue others |
| Document creation fails | Log error, continue with siblings |
| Missing markdown file | Log warning, skip document |
| Invalid metadata JSON | Abort that collection |
| Parent document not found | Create as root-level document |

---

## 6. Technical Design

### 6.1 Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                    import_to_outline.sh                      │
│  (Bash wrapper - Docker execution, backup, UI)              │
└─────────────────────────┬───────────────────────────────────┘
                          │
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                    outline_import.py                         │
│  (Python core - API calls, hierarchy logic)                 │
└─────────────────────────┬───────────────────────────────────┘
                          │
            ┌─────────────┼─────────────┐
            ▼             ▼             ▼
    ┌──────────────┐ ┌──────────┐ ┌──────────────┐
    │ settings.json│ │ metadata │ │ Outline API  │
    │ (API config) │ │  .json   │ │ (HTTP POST)  │
    └──────────────┘ └──────────┘ └──────────────┘
```

### 6.2 API Endpoints

| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/api/collections.list` | POST | Check existing collections |
| `/api/collections.create` | POST | Create new collection |
| `/api/collections.delete` | POST | Delete collection (--force mode) |
| `/api/documents.create` | POST | Create document with content |
| `/api/documents.list` | POST | Check existing documents |
| `/api/documents.update` | POST | Update document (--force mode) |

### 6.3 API Request Examples

#### Create Collection
```json
POST /api/collections.create
{
  "name": "Bewerbungen",
  "permission": "read_write"
}
```

#### Create Document
```json
POST /api/documents.create
{
  "collectionId": "col-uuid-here",
  "title": "DORA Metrics (Top 4)",
  "text": "# DORA Metrics\n\nContent here...",
  "parentDocumentId": "parent-uuid-or-null",
  "publish": true
}
```

### 6.4 Data Structures

#### Input: `_collection_metadata.json`
```json
{
  "id": "original-collection-uuid",
  "name": "Bewerbungen",
  "directory": "Bewerbungen",
  "expected_count": 11,
  "documents": [
    {
      "id": "doc-uuid",
      "title": "Document Title",
      "filename": "Document Title.md",
      "parent_id": "parent-uuid-or-null",
      "checksum": "sha256-hash",
      "children": [...]
    }
  ]
}
```

#### Runtime: ID Mapping
```python
id_map: Dict[str, str] = {
    "old-uuid-1": "new-uuid-1",
    "old-uuid-2": "new-uuid-2"
}
```

### 6.5 Docker Execution

```bash
docker run --rm --network domnet \
    --user "$(id -u):$(id -g)" \
    -e HOME=/tmp \
    -v "$WORK_DIR:/work" \
    -w /work \
    python:3.11-slim \
    bash -c "pip install -qqq requests 2>/dev/null && \
             python3 outline_import.py $CLI_ARGS"
```

---

## 7. User Interface

### 7.1 Progress Output

```
════════════════════════════════════════════════════════════
  OUTLINE IMPORT
════════════════════════════════════════════════════════════

Source:  outline_export/
Target:  http://outline:3000
Mode:    Collection per folder

Checking API connectivity... ✓

Bewerbungen/ (11 documents)
  Creating collection... ✓ (id: 7f3a...)
  ├── CV.md                              ✓ created
  ├── DORA Metrics (Top 4).md            ✓ created
  ├── Tipico.md                          ✓ created
  │   ├── Pitch Tipico.md                ✓ created
  │   ├── Fragen 3. Runde.md             ✓ created
  │   ├── Tipico 3rd Party.md            ✓ created
  │   └── Tipico Top 10 Functions.md     ✓ created
  └── Ihre PVS.md                        ✓ created
      ├── Mobilepass.md                  ✓ created
      ├── PVS erster Call.md             ✓ created
      └── Fragen Dirk.md                 ✓ created

Projekte/ (8 documents)
  Collection exists, skipping...

════════════════════════════════════════════════════════════
SUMMARY
════════════════════════════════════════════════════════════
  Collections:  1 created, 1 skipped, 0 errors
  Documents:   11 created, 0 skipped, 0 errors
  Duration:     2.3 seconds
════════════════════════════════════════════════════════════
```

### 7.2 Dry-Run Output

```
════════════════════════════════════════════════════════════
  OUTLINE IMPORT (DRY RUN)
════════════════════════════════════════════════════════════

Source:  outline_export/
Target:  http://outline:3000
Mode:    Collection per folder

[DRY RUN] No changes will be made

Bewerbungen/ (11 documents)
  [DRY RUN] Would create collection "Bewerbungen"
  [DRY RUN] Would create 11 documents:
    ├── CV.md
    ├── DORA Metrics (Top 4).md
    ├── Tipico.md
    │   ├── Pitch Tipico.md
    │   └── ...
    └── ...

Projekte/ (8 documents)
  [DRY RUN] Collection exists - would skip

════════════════════════════════════════════════════════════
DRY RUN SUMMARY
════════════════════════════════════════════════════════════
  Would create:  1 collection, 11 documents
  Would skip:    1 collection (exists)
════════════════════════════════════════════════════════════
```

### 7.3 Error Output

```
Bewerbungen/ (11 documents)
  Creating collection... ✓
  ├── CV.md                              ✓ created
  ├── Missing Doc.md                     ✗ file not found
  └── Tipico.md                          ✗ API error: 500
      └── (children skipped due to parent failure)
```

---

## 8. Configuration

### 8.1 settings.json (shared with export)

```json
{
  "source": {
    "url": "http://outline:3000",
    "token": "ol_api_xxxxxxxxxxxx"
  },
  "import": {
    "source_directory": "outline_export",
    "on_collection_exists": "skip",
    "on_document_exists": "skip",
    "default_permission": "read_write"
  },
  "advanced": {
    "request_timeout": 30,
    "retry_attempts": 3,
    "retry_delay": 1.0,
    "rate_limit_delay": 0.1
  }
}
```

### 8.2 Configuration Options

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `import.source_directory` | string | `outline_export` | Default source path |
| `import.on_collection_exists` | enum | `skip` | `skip`, `error`, `merge` |
| `import.on_document_exists` | enum | `skip` | `skip`, `error`, `update` |
| `import.default_permission` | enum | `read_write` | `read`, `read_write` |
| `advanced.request_timeout` | int | 30 | API timeout in seconds |
| `advanced.retry_attempts` | int | 3 | Retries on failure |
| `advanced.rate_limit_delay` | float | 0.1 | Delay between API calls |

---

## 9. Testing Strategy

### 9.1 Test Cases

| ID | Category | Test Case | Expected Result |
|----|----------|-----------|-----------------|
| T1 | Happy Path | Import single collection | Collection + docs created |
| T2 | Happy Path | Import multiple collections | All collections created |
| T3 | Happy Path | Import nested hierarchy (3+ levels) | All parent-child relationships preserved |
| T4 | Duplicate | Collection already exists | Skip collection |
| T5 | Duplicate | Document title exists | Skip document |
| T6 | Error | Missing markdown file | Log warning, continue |
| T7 | Error | Invalid metadata JSON | Abort collection |
| T8 | Error | API unreachable | Abort with clear error |
| T9 | Mode | --single flag | Single timestamped collection |
| T10 | Mode | --dry-run flag | No API mutations |
| T11 | Mode | --force flag | Overwrites existing |
| T12 | Edge | Empty collection | Create empty collection |
| T13 | Edge | Special chars in title | Handled correctly |
| T14 | Edge | Very large document | Imported successfully |

### 9.2 Verification Methods

1. **Round-trip Test**: Export → Import to test instance → Export again → Compare checksums
2. **API Verification**: Query created documents and verify parent relationships
3. **Manual Inspection**: Spot-check imported content in Outline UI

---

## 10. Rollback & Recovery

### Pre-Import Safety
- No automatic backup (user should have export as backup)
- `--dry-run` always available to preview

### Rollback Procedure
If import fails partway through:
1. Note which collections were created (from import log)
2. Manually delete partial collections via Outline UI or API
3. Fix issue and re-run import

### Future Enhancement
- `--backup` flag to export existing content before import
- Transaction-like behavior: delete partially-imported collection on failure

---

## 11. Future Enhancements

| Priority | Enhancement | Description |
|----------|-------------|-------------|
| P1 | Attachment support | Import images and file attachments |
| P1 | Merge mode | Add documents to existing collections |
| P2 | Selective import | Import specific documents by path/pattern |
| P2 | Update mode | Update existing documents with new content |
| P3 | User mapping | Preserve authorship via user email mapping |
| P3 | Permission sync | Restore document-level permissions |
| P3 | Incremental import | Only import new/changed documents |

---

## 12. Implementation Checklist

- [ ] Create `outline_import.py` with core import logic
- [ ] Create `import_to_outline.sh` bash wrapper
- [ ] Implement collection creation
- [ ] Implement document creation with hierarchy
- [ ] Implement ID mapping for parent references
- [ ] Add `--dry-run` mode
- [ ] Add `--single` mode
- [ ] Add `--force` mode
- [ ] Add progress visualization
- [ ] Add error handling and reporting
- [ ] Add retry logic for API failures
- [ ] Update settings.json schema
- [ ] Write tests
- [ ] Update CLAUDE.md documentation

---

## 13. References

- **Export Script**: `export_with_trees.sh`, `outline_export_fixed.py`
- **Outline API Docs**: https://www.getoutline.com/developers
- **Metadata Format**: See `outline_export/*/_collection_metadata.json`
- **Settings Format**: See `settings.json`