# Backup System Setup Progress

## Status: Complete (as of 2025-12-09)

## Completed Steps

### 1. Backblaze B2 Setup ✅
- Account created
- Bucket: `ovh-prod-01-backups` (private, server-side encryption enabled)
- Application key: `synology-hyper-backup` (stored in Bitwarden as "Backblaze | ovh-prod-01-backups key")
- Client-side encryption: Not needed (server-side encryption sufficient for this use case)

### 2. Server-Side Backup Script ✅
- Location: `~/services/backup.sh` on OVH server (15.204.247.153)
- Auto-discovers services in `~/services/*/docker-compose.yml`
- Supports: MariaDB, PostgreSQL, MongoDB, PocketBase
- Output: `~/backups/current/latest.tar.gz`
- Cron: `0 7 * * *` (2 AM EST = 7 AM UTC)
- Tested successfully - created 792MB backup

### 3. Synology SSH Key Setup ✅
- Key generated: `/var/services/homes/jared/.ssh/id_ed25519_ovh_backup`
- Public key added to OVH server's `~/.ssh/authorized_keys`
- Connection tested successfully

### 4. Synology Pull Script ✅
- Directory created: `/volume1/backups/ovh-prod-01/`
- Script created: `/volume1/backups/ovh-prod-01/pull-backup.sh`
- SSH key: `/var/services/homes/jared/.ssh/id_ed25519_ovh_backup`

### 5. Synology Pull Task Scheduled ✅
- Configured via Synology Task Scheduler
- Schedule: 3 AM EST (8 AM UTC)
- Runs as root to avoid permission issues

### 6. Hyper Backup to Backblaze B2 ✅
- Task name: "OVH Server Backup"
- Source: `/volume1/backups/ovh-prod-01/`
- Destination: `ovh-prod-01-backups` bucket on Backblaze B2
- Schedule: 4 AM EST daily (after Synology pull completes)
- Integrity check: Weekly on Saturday
- Rotation: Smart Recycle (256 max versions)
- Settings: Compression enabled, transfer encryption enabled

### 7. Test Full Chain ✅
- Initial backup completed successfully on 2025-12-09

### 8. Disaster Recovery Documentation ✅
- Created `docs/disaster-recovery.md`
- Covers single service recovery and full server rebuild
- Documents backup locations and credential sources

### 9. Update Service Deployment Docs ✅
- Added step 6 to deployment pattern in CLAUDE.md
- Added disaster-recovery.md link to specialized documentation

## Architecture Summary

```
OVH Server (2 AM EST)     Synology (3 AM EST)      Backblaze B2
─────────────────────     ──────────────────       ────────────
~/services/backup.sh  →   rsync pull           →   Hyper Backup
       ↓                        ↓                       ↓
~/backups/current/        /volume1/backups/      ovh-prod-01-backups
  latest.tar.gz           ovh-prod-01/              bucket
```

## Retention Policy
- Server: 2 days (staging only)
- Synology/Backblaze: Smart Recycle (hourly for 24h, daily for ~1 month, weekly beyond)
- Max versions: 256

## Gotchas & Lessons Learned

### Hyper Backup Folder Visibility
- **Issue:** Newly created folders don't appear in Hyper Backup's folder picker
- **Solution:** Close the entire Hyper Backup wizard and restart it; the folder list refreshes on wizard launch

### Synology Permissions
- **Issue:** Pull script had permission problems when run as regular user
- **Solution:** Run the scheduled task as root in Task Scheduler

### B2 Encryption
- **Decision:** Server-side encryption only (no client-side)
- **Rationale:** SSE is sufficient for server backups; client-side adds key management complexity with minimal benefit for this use case