A lightweight backup health monitoring system built on the existing self-hosted n8n instance at `https://n8n.swansoncloud.com`. Each backup script calls a shared Ruby notifier script after it runs. The notifier POSTs a structured result to an n8n webhook. Every Friday morning, a second n8n workflow checks the week's backup history against a defined expected list and sends a health digest email via MailPace.
---
## Architecture & Data Flow
```
backup.sh (or any backup script)
└── calls notify_backup.rb with result args
└── POSTs JSON to n8n webhook (auth header from ENV)
n8n Workflow 1 (Receiver)
├── Webhook trigger (POST /backup-notify)
├── Validate Authorization header
└── Append row to "backup_runs" datatable
n8n Workflow 2 (Reporter)
├── Cron trigger (Friday, 8:00 AM)
├── Read "backup_runs" datatable (last 7 days)
├── Load expected sources from Set node
├── For each expected source: classify status
├── Roll up overall status (GREEN / YELLOW / RED)
└── Send MailPace email to jaredmswanson@gmail.com
- **Sandi Metz OOP:** Small, single-purpose objects. Each class has one reason to change. Dependencies injected, not hardcoded. No class does more than one job.
- **Self-documenting:** The script's header comment explains its purpose, all accepted arguments, exit behavior, and required ENV vars — enough for any future backup process to integrate without reading the implementation.
- **Classes:**
-`BackupNotifier` — orchestrates the flow; the only public entry point
-`PayloadBuilder` — converts raw CLI args into a clean hash (formats bytes, validates status)
-`WebhookClient` — handles the HTTP POST with auth header; knows nothing about backup data
-`ByteFormatter` — single responsibility: raw integer → human-readable string
- CLI argument parsing at the top level (OptionParser), outside of classes
| `notes` | text | free-form restic output or skip reason |
The `machine` + `destination` pair is the identity key matched against the expected list.
---
## Workflow 1 — Receiver
**Trigger:** Webhook, POST `/backup-notify`
**Nodes:**
1.**Webhook** — POST only, path `/backup-notify`
2.**Validate Auth** — If node: checks `Authorization` header equals `Bearer <token>` (stored as n8n credential). On failure: respond 401 and halt.
3.**Append to Datatable** — writes one row to `backup_runs`
4.**Respond 200** — returns `{"ok": true}`
The receiver has no logic beyond auth + write. Malformed payloads are logged by n8n without blocking the caller.
**Auth token:** Stored in n8n as a credential. Must match `N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN` in the sending machine's environment.
---
## Workflow 2 — Reporter
**Trigger:** Cron, every Friday at 8:00 AM
**Nodes:**
1.**Cron** — Friday 8:00 AM
2.**Set: Expected Sources** — hardcoded list of `{machine, destination}` pairs. Edit this node to add/remove sources. Initial list:
-`desktop / synology`
-`desktop / backblaze-b2`
-`vps-ovh-prod-01 / local`
Note: the VPS backup chain is `OVH server → Synology pull → Hyper Backup to B2`. Only the first step (OVH server creating `latest.tar.gz`) is scriptable with `notify_backup.rb`. The Synology pull and Hyper Backup steps are Synology DSM-managed tasks with no easy hook. Monitoring those downstream steps is deferred to a future phase.
3.**Read Datatable** — fetches all `backup_runs` rows with `timestamp` in the past 7 days
4.**Evaluate Each Source** — for each expected `machine/destination` pair:
- At least one `success` this week → **green**
- No `success`, but rows exist (`failure` or `skipped`) → **yellow**
- No rows at all → **red** (silent — never reported in)
5.**Roll Up Status:**
- Any red source → overall **RED**
- No red, any yellow → overall **YELLOW**
- All green → overall **GREEN**
6.**Build Email** — see Email Format below
7.**MailPace** — sends to `jaredmswanson@gmail.com`
---
## Email Format
### GREEN
- **Subject:** `✓ Backups healthy — week of May 4`
- **Body:** One-line confirmation. Table of all sources with their run count and last success timestamp.
### YELLOW
- **Subject:** `⚠ Backup warning — week of May 4`
- **Body:**
- Summary of which sources had no successful run
- For each yellow source: failure/skipped rows with timestamp, status, notes, and log path
- Next steps: check the log at the listed path, verify connectivity (for Synology: check LAN reachability, HDD spin-up), re-run manually if needed
- Note: a single skipped night (e.g. power outage) is expected — yellow just means "worth a glance"
### RED
- **Subject:** `✗ Backup failure — week of May 4`
- **Body:**
- Which sources reported zero runs all week
- Last known good run date for each red source (from datatable history)
- Log path from the most recent row if any exists
- Next steps: verify the backup script is still being called (check systemd timer), confirm notifier script is reachable, check n8n logs
---
## Security
- Webhook protected by `Authorization: Bearer <token>` header
- Token stored in n8n as a credential (not in workflow JSON)
- Token available on sending machines via `N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN` env var, sourced from `~/.credentials`
-`~/.credentials` is excluded from dotfiles git tracking
---
## Adding a New Backup Source
1. Add the `notify_backup.rb` call to the new backup script with appropriate `--machine` and `--destination` values
2. Ensure `N8N_SWANSONCLOUD_URL` and `N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN` are in the machine's environment
3. Add the new `{machine, destination}` pair to the **Set: Expected Sources** node in Workflow 2
