docs(backup-monitor): replace UI-click workflow tasks with JSON file approach
- Add JS code files + Ruby build script + Python validator to file map - Replace Tasks 7-9 (UI clicks) with Tasks 7-10: datatable bootstrap, validator script, build/validate workflow JSON, API import with credentials - Fix MailPace credential header: X-Server-Token (not MailPace-Server-Token) - Renumber old Tasks 10-12 → 11-13 - Update self-review notes with data table uncertainty and node naming Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
jared
parent
85bbd7317f
commit
5fcc7fb515
|
|
@ -16,12 +16,16 @@
|
||||||
|------|--------|----------------|
|
|------|--------|----------------|
|
||||||
| `scripts/notify_backup.rb` | Create | Source-of-record for the notifier script |
|
| `scripts/notify_backup.rb` | Create | Source-of-record for the notifier script |
|
||||||
| `scripts/notify_backup_test.rb` | Create | Minitest suite for all four classes |
|
| `scripts/notify_backup_test.rb` | Create | Minitest suite for all four classes |
|
||||||
|
| `scripts/n8n-evaluate-sources.js` | Create | JS code for the Evaluate Sources Code node |
|
||||||
|
| `scripts/n8n-build-email.js` | Create | JS code for the Build Email Code node |
|
||||||
|
| `scripts/build-n8n-workflows.rb` | Create | Assembles workflow JSON using `JSON.generate` (handles all JS escaping) |
|
||||||
|
| `scripts/n8n-workflow-receiver.json` | Generated | Importable Workflow 1 JSON |
|
||||||
|
| `scripts/n8n-workflow-reporter.json` | Generated | Importable Workflow 2 JSON |
|
||||||
|
| `scripts/validate-n8n-workflow.py` | Create | Structural validator for n8n workflow JSON files |
|
||||||
| `~/.local/bin/notify_backup.rb` | Deploy | Runtime location on desktop |
|
| `~/.local/bin/notify_backup.rb` | Deploy | Runtime location on desktop |
|
||||||
| `~/.local/bin/backup.sh` | Modify | Add notify calls after each restic run |
|
| `~/.local/bin/backup.sh` | Modify | Add notify calls after each restic run |
|
||||||
| `~/services/backup.sh` on OVH | Modify via SSH | Add notify calls after VPS backup run |
|
| `~/services/backup.sh` on OVH | Modify via SSH | Add notify calls after VPS backup run |
|
||||||
|
|
||||||
n8n workflows are configured in the n8n UI — no local files, but each task below gives exact node settings.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Task 1: Script skeleton and test infrastructure
|
## Task 1: Script skeleton and test infrastructure
|
||||||
|
|
@ -588,144 +592,217 @@ echo $? # must print 0
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Task 7: Create `backup_runs` datatable in n8n
|
## Task 7: Bootstrap — create datatable and verify Data Table node JSON shape
|
||||||
|
|
||||||
This is a UI task in n8n. Open https://n8n.swansoncloud.com.
|
Both research agents flagged the Data Table node (`n8n-nodes-base.dataTable`) as a recent addition with uncertain parameter structure. This task pins the exact shape before generating workflow JSON.
|
||||||
|
|
||||||
- [ ] In the left sidebar, click **"Variables"** (or **"Data"** depending on your n8n version). Look for a **"Tables"** section.
|
- [ ] Open https://n8n.swansoncloud.com. In the left sidebar find **"Tables"** (under Variables or Data). Create a new table named exactly `backup_runs`. Add string columns: `timestamp`, `machine`, `destination`, `status`, `bytes_transferred`, `snapshot_id`, `log_path`, `notes`. n8n auto-adds an `id` column.
|
||||||
|
|
||||||
- [ ] Create a new table named exactly: `backup_runs`
|
- [ ] Create a scratch workflow named `_datatable-shape-probe`:
|
||||||
|
- Add a **Manual Trigger** node
|
||||||
|
- Add a **Data Table** node, configure it: operation = **Insert**, table = `backup_runs`, map one dummy field
|
||||||
|
- Connect Manual Trigger → Data Table
|
||||||
|
- Add a second **Data Table** node: operation = **Get Many Rows**, table = `backup_runs`
|
||||||
|
- Connect the Insert node → Get Many Rows
|
||||||
|
- Save (do not activate)
|
||||||
|
|
||||||
- [ ] Add the following columns (use string type for all unless noted):
|
- [ ] Export the scratch workflow: click the `⋮` menu → **Download**. Save as `/tmp/datatable-probe.json`.
|
||||||
|
|
||||||
| Column name | Type |
|
- [ ] Inspect the exported JSON to extract the exact node structure:
|
||||||
|---|---|
|
|
||||||
| `timestamp` | String |
|
|
||||||
| `machine` | String |
|
|
||||||
| `destination` | String |
|
|
||||||
| `status` | String |
|
|
||||||
| `bytes_transferred` | String |
|
|
||||||
| `snapshot_id` | String |
|
|
||||||
| `log_path` | String |
|
|
||||||
| `notes` | String |
|
|
||||||
|
|
||||||
Note: n8n auto-assigns an `id` column. If the UI only lets you create columns at workflow time (not pre-create), skip this step — Workflow 1 will create them automatically on first insert.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Task 8: n8n Workflow 1 — Receiver
|
|
||||||
|
|
||||||
Build this workflow in the n8n UI. Create a new workflow named **"Backup Monitor — Receiver"**.
|
|
||||||
|
|
||||||
- [ ] **Store the auth token as an n8n credential first:**
|
|
||||||
- Go to **Credentials** → **New**
|
|
||||||
- Type: **Header Auth**
|
|
||||||
- Name: `backup-monitor-token`
|
|
||||||
- Name field: `Authorization`
|
|
||||||
- Value field: `Bearer <paste value of N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN from your terminal>`
|
|
||||||
- Save
|
|
||||||
|
|
||||||
- [ ] **Add node 1 — Webhook:**
|
|
||||||
- Node type: **Webhook**
|
|
||||||
- HTTP Method: `POST`
|
|
||||||
- Path: `backup-notify`
|
|
||||||
- Authentication: **Header Auth** → select `backup-monitor-token`
|
|
||||||
- Response Mode: **Using 'Respond to Webhook' Node**
|
|
||||||
- Save
|
|
||||||
|
|
||||||
- [ ] **Add node 2 — Insert to datatable:**
|
|
||||||
- Node type: **n8n** (the built-in n8n node)
|
|
||||||
- Resource: **Database**
|
|
||||||
- Operation: **Insert**
|
|
||||||
- Table: `backup_runs`
|
|
||||||
- Map each field from the incoming JSON body:
|
|
||||||
- `timestamp` → `{{ $json.body.timestamp }}`
|
|
||||||
- `machine` → `{{ $json.body.machine }}`
|
|
||||||
- `destination` → `{{ $json.body.destination }}`
|
|
||||||
- `status` → `{{ $json.body.status }}`
|
|
||||||
- `bytes_transferred` → `{{ $json.body.bytes_transferred }}`
|
|
||||||
- `snapshot_id` → `{{ $json.body.snapshot_id }}`
|
|
||||||
- `log_path` → `{{ $json.body.log_path }}`
|
|
||||||
- `notes` → `{{ $json.body.notes }}`
|
|
||||||
- Connect from Webhook node
|
|
||||||
|
|
||||||
- [ ] **Add node 3 — Respond to Webhook:**
|
|
||||||
- Node type: **Respond to Webhook**
|
|
||||||
- Response Code: `200`
|
|
||||||
- Response Body: `{"ok":true}`
|
|
||||||
- Connect from the Insert node
|
|
||||||
|
|
||||||
- [ ] **Activate the workflow** (toggle in top-right)
|
|
||||||
|
|
||||||
- [ ] **Test with a real webhook call** (ENV vars must be sourced):
|
|
||||||
```bash
|
```bash
|
||||||
source ~/.credentials
|
python3 -c "
|
||||||
notify_backup.rb \
|
import json, sys
|
||||||
--machine desktop \
|
wf = json.load(open('/tmp/datatable-probe.json'))
|
||||||
--destination test \
|
for n in wf['nodes']:
|
||||||
--status success \
|
if 'dataTable' in n.get('type','') or 'dataStore' in n.get('type',''):
|
||||||
--notes "smoke test $(date)"
|
print(json.dumps(n, indent=2))
|
||||||
|
"
|
||||||
```
|
```
|
||||||
Expected: no error output from `notify_backup.rb`
|
Note the `type` string (likely `n8n-nodes-base.dataTable`), `typeVersion`, and the exact keys inside `parameters` (especially the property wrapping the table reference and the filter shape).
|
||||||
|
|
||||||
- [ ] **Verify the row appeared in the datatable:** In n8n, open the `backup_runs` table and confirm a row exists with `machine=desktop`, `destination=test`, `status=success`.
|
- [ ] Also note the datatable ID from the exported JSON (the `value` field inside the table picker). You will need this to replace `REPLACE_WITH_DATATABLE_ID` in Task 9.
|
||||||
|
|
||||||
- [ ] Delete the test row from the datatable before continuing.
|
- [ ] If the `type` string or `typeVersion` differ from `n8n-nodes-base.dataTable` / `1`, update the two constants at the top of `scripts/build-n8n-workflows.rb` (created in Task 9) before running it.
|
||||||
|
|
||||||
|
- [ ] Delete the scratch workflow from n8n.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Task 9: n8n Workflow 2 — Reporter
|
## Task 8: Write JSON validation script
|
||||||
|
|
||||||
Build this workflow in the n8n UI. Create a new workflow named **"Backup Monitor — Weekly Reporter"**.
|
**Files:**
|
||||||
|
- Create: `scripts/validate-n8n-workflow.py`
|
||||||
|
|
||||||
- [ ] **Store MailPace token as a credential:**
|
- [ ] Create `scripts/validate-n8n-workflow.py` with this content (Python stdlib, no pip required):
|
||||||
- Go to **Credentials** → **New**
|
|
||||||
- Type: **Header Auth**
|
|
||||||
- Name: `mailpace-token`
|
|
||||||
- Name field: `MailPace-Server-Token`
|
|
||||||
- Value field: your MailPace server API token (find it at app.mailpace.com → Server Settings)
|
|
||||||
- Save
|
|
||||||
|
|
||||||
- [ ] **Add node 1 — Schedule Trigger:**
|
```python
|
||||||
- Node type: **Schedule Trigger**
|
#!/usr/bin/env python3
|
||||||
- Trigger interval: **Weeks**
|
"""Validate an n8n workflow JSON file for structural correctness.
|
||||||
- Day of week: **Friday**
|
|
||||||
- Hour: `8`
|
|
||||||
- Minute: `0`
|
|
||||||
|
|
||||||
- [ ] **Add node 2 — Set: Expected Sources:**
|
Usage: validate-n8n-workflow.py path/to/workflow.json
|
||||||
- Node type: **Set**
|
Exit codes: 0 = valid, 1 = invalid, 2 = usage error.
|
||||||
- Mode: **Manual**
|
"""
|
||||||
- Add one field:
|
import json
|
||||||
- Name: `sources`
|
import sys
|
||||||
- Type: **Array**
|
from pathlib import Path
|
||||||
- Value:
|
|
||||||
```json
|
KNOWN_MIN_TYPE_VERSIONS = {
|
||||||
[
|
"n8n-nodes-base.code": 2,
|
||||||
{"machine": "desktop", "destination": "synology"},
|
"n8n-nodes-base.httpRequest": 4.2,
|
||||||
{"machine": "desktop", "destination": "backblaze-b2"},
|
"n8n-nodes-base.webhook": 2,
|
||||||
{"machine": "vps-ovh-prod-01", "destination": "local"}
|
"n8n-nodes-base.set": 3.4,
|
||||||
]
|
"n8n-nodes-base.scheduleTrigger": 1.2,
|
||||||
|
}
|
||||||
|
|
||||||
|
REQUIRED_TOP = ["name", "nodes", "connections"]
|
||||||
|
REQUIRED_NODE = ["id", "name", "type", "typeVersion", "position", "parameters"]
|
||||||
|
|
||||||
|
|
||||||
|
def validate(workflow):
|
||||||
|
errors, warnings = [], []
|
||||||
|
if not isinstance(workflow, dict):
|
||||||
|
return ["Root must be a JSON object."], []
|
||||||
|
for key in REQUIRED_TOP:
|
||||||
|
if key not in workflow:
|
||||||
|
errors.append(f"Top-level: missing required field '{key}'.")
|
||||||
|
if errors:
|
||||||
|
return errors, warnings
|
||||||
|
if not isinstance(workflow["nodes"], list):
|
||||||
|
errors.append("Top-level 'nodes' must be an array.")
|
||||||
|
if not isinstance(workflow["connections"], dict):
|
||||||
|
errors.append("Top-level 'connections' must be an object.")
|
||||||
|
if errors:
|
||||||
|
return errors, warnings
|
||||||
|
|
||||||
|
seen_names, seen_ids = set(), set()
|
||||||
|
for i, node in enumerate(workflow["nodes"]):
|
||||||
|
loc = f"nodes[{i}]"
|
||||||
|
if not isinstance(node, dict):
|
||||||
|
errors.append(f"{loc}: must be an object.")
|
||||||
|
continue
|
||||||
|
for key in REQUIRED_NODE:
|
||||||
|
if key not in node:
|
||||||
|
errors.append(f"{loc}: missing required field '{key}'.")
|
||||||
|
name = node.get("name")
|
||||||
|
if name:
|
||||||
|
loc = f"nodes[{i}] '{name}'"
|
||||||
|
if name in seen_names:
|
||||||
|
errors.append(f"{loc}: duplicate node name.")
|
||||||
|
seen_names.add(name)
|
||||||
|
nid = node.get("id")
|
||||||
|
if nid:
|
||||||
|
if nid in seen_ids:
|
||||||
|
errors.append(f"{loc}: duplicate node id '{nid}'.")
|
||||||
|
seen_ids.add(nid)
|
||||||
|
pos = node.get("position")
|
||||||
|
if pos is not None and not (
|
||||||
|
isinstance(pos, list) and len(pos) == 2
|
||||||
|
and all(isinstance(v, (int, float)) for v in pos)
|
||||||
|
):
|
||||||
|
errors.append(f"{loc}: 'position' must be [number, number].")
|
||||||
|
if node.get("parameters") is not None and not isinstance(node["parameters"], dict):
|
||||||
|
errors.append(f"{loc}: 'parameters' must be an object.")
|
||||||
|
creds = node.get("credentials")
|
||||||
|
if isinstance(creds, dict):
|
||||||
|
for ctype, cred in creds.items():
|
||||||
|
if not isinstance(cred, dict) or "name" not in cred:
|
||||||
|
errors.append(f"{loc}: credentials['{ctype}'] must have a 'name' field.")
|
||||||
|
ntype, tv = node.get("type"), node.get("typeVersion")
|
||||||
|
if isinstance(tv, (int, float)) and ntype in KNOWN_MIN_TYPE_VERSIONS:
|
||||||
|
min_tv = KNOWN_MIN_TYPE_VERSIONS[ntype]
|
||||||
|
if tv < min_tv:
|
||||||
|
warnings.append(f"{loc}: typeVersion {tv} for '{ntype}' is below recommended {min_tv}.")
|
||||||
|
|
||||||
|
for src, outputs in workflow["connections"].items():
|
||||||
|
if src not in seen_names:
|
||||||
|
errors.append(f"connections: source '{src}' does not match any node name.")
|
||||||
|
if not isinstance(outputs, dict):
|
||||||
|
errors.append(f"connections['{src}']: must be an object.")
|
||||||
|
continue
|
||||||
|
for out_type, out_list in outputs.items():
|
||||||
|
if not isinstance(out_list, list):
|
||||||
|
continue
|
||||||
|
for oi, branch in enumerate(out_list):
|
||||||
|
if branch is None:
|
||||||
|
continue
|
||||||
|
if not isinstance(branch, list):
|
||||||
|
errors.append(f"connections['{src}']['{out_type}'][{oi}]: must be an array.")
|
||||||
|
continue
|
||||||
|
for ti, target in enumerate(branch):
|
||||||
|
if not isinstance(target, dict):
|
||||||
|
errors.append(f"connections['{src}']['{out_type}'][{oi}][{ti}]: must be an object.")
|
||||||
|
continue
|
||||||
|
for k in ("node", "type", "index"):
|
||||||
|
if k not in target:
|
||||||
|
errors.append(f"connections['{src}']['{out_type}'][{oi}][{ti}]: missing '{k}'.")
|
||||||
|
tnode = target.get("node")
|
||||||
|
if tnode and tnode not in seen_names:
|
||||||
|
errors.append(f"connections['{src}']['{out_type}'][{oi}][{ti}]: target '{tnode}' not found.")
|
||||||
|
return errors, warnings
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
if len(sys.argv) != 2:
|
||||||
|
print("Usage: validate-n8n-workflow.py <workflow.json>", file=sys.stderr)
|
||||||
|
sys.exit(2)
|
||||||
|
path = Path(sys.argv[1])
|
||||||
|
try:
|
||||||
|
workflow = json.loads(path.read_text())
|
||||||
|
except FileNotFoundError:
|
||||||
|
print(f"File not found: {path}", file=sys.stderr)
|
||||||
|
sys.exit(2)
|
||||||
|
except json.JSONDecodeError as e:
|
||||||
|
print(f"Invalid JSON: {e}", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
errors, warnings = validate(workflow)
|
||||||
|
for w in warnings:
|
||||||
|
print(f"WARN: {w}")
|
||||||
|
for e in errors:
|
||||||
|
print(f"ERROR: {e}", file=sys.stderr)
|
||||||
|
if errors:
|
||||||
|
print(f"\n{len(errors)} error(s), {len(warnings)} warning(s).", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
print(f"OK — {len(workflow['nodes'])} nodes, {len(workflow['connections'])} connections, {len(warnings)} warning(s).")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
```
|
```
|
||||||
- Connect from Schedule Trigger
|
|
||||||
|
|
||||||
- [ ] **Add node 3 — Read Datatable:**
|
- [ ] Make it executable and test it on a known-bad payload:
|
||||||
- Node type: **n8n**
|
```bash
|
||||||
- Resource: **Database**
|
chmod +x /home/jared/systems-admin/scripts/validate-n8n-workflow.py
|
||||||
- Operation: **Get Many Rows**
|
echo '{"name":"test","nodes":[],"connections":{"missing-node":{"main":[[]]}}}' \
|
||||||
- Table: `backup_runs`
|
| python3 /home/jared/systems-admin/scripts/validate-n8n-workflow.py /dev/stdin
|
||||||
- Filter: `timestamp` is greater than `{{ new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString() }}`
|
```
|
||||||
- Connect from Set node
|
Expected: `ERROR: connections: source 'missing-node' does not match any node name.`
|
||||||
|
|
||||||
- [ ] **Add node 4 — Evaluate Sources (Code node):**
|
- [ ] Commit:
|
||||||
- Node type: **Code**
|
```bash
|
||||||
- Language: **JavaScript**
|
cd /home/jared/systems-admin
|
||||||
- Connect from Read Datatable node
|
git add scripts/validate-n8n-workflow.py
|
||||||
- Paste this code exactly:
|
git commit -m "feat(backup-monitor): add n8n workflow JSON validator"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 9: Write JS code files, build script, generate and validate workflow JSONs
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `scripts/n8n-evaluate-sources.js`
|
||||||
|
- Create: `scripts/n8n-build-email.js`
|
||||||
|
- Create: `scripts/build-n8n-workflows.rb`
|
||||||
|
- Generated: `scripts/n8n-workflow-receiver.json`
|
||||||
|
- Generated: `scripts/n8n-workflow-reporter.json`
|
||||||
|
|
||||||
|
The build script uses Ruby's `JSON.generate` to handle all escaping automatically — no manual string escaping needed for the Code node JS.
|
||||||
|
|
||||||
|
- [ ] Create `scripts/n8n-evaluate-sources.js`:
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
const sources = $('Set: Expected Sources').first().json.sources;
|
// Runs once for all items. References upstream nodes by exact name.
|
||||||
const runs = $('Read Datatable').all().map(item => item.json);
|
const sources = $('Set sources').first().json.sources;
|
||||||
|
const runs = $('Get backup_runs (last 7d)').all().map(item => item.json);
|
||||||
const weekAgo = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000);
|
const weekAgo = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000);
|
||||||
|
|
||||||
const results = sources.map(source => {
|
const results = sources.map(source => {
|
||||||
|
|
@ -734,124 +811,479 @@ const results = sources.map(source => {
|
||||||
r.destination === source.destination &&
|
r.destination === source.destination &&
|
||||||
new Date(r.timestamp) >= weekAgo
|
new Date(r.timestamp) >= weekAgo
|
||||||
);
|
);
|
||||||
|
|
||||||
const hasSuccess = sourceRuns.some(r => r.status === 'success');
|
const hasSuccess = sourceRuns.some(r => r.status === 'success');
|
||||||
const hasAnyRun = sourceRuns.length > 0;
|
const hasAnyRun = sourceRuns.length > 0;
|
||||||
|
|
||||||
const status = hasSuccess ? 'green' : hasAnyRun ? 'yellow' : 'red';
|
const status = hasSuccess ? 'green' : hasAnyRun ? 'yellow' : 'red';
|
||||||
|
|
||||||
const lastSuccess = sourceRuns
|
const lastSuccess = sourceRuns
|
||||||
.filter(r => r.status === 'success')
|
.filter(r => r.status === 'success')
|
||||||
.sort((a, b) => new Date(b.timestamp) - new Date(a.timestamp))[0] || null;
|
.sort((a, b) => new Date(b.timestamp) - new Date(a.timestamp))[0] || null;
|
||||||
|
|
||||||
return { ...source, status, runs: sourceRuns, lastSuccess };
|
return { ...source, status, runs: sourceRuns, lastSuccess };
|
||||||
});
|
});
|
||||||
|
|
||||||
const overallStatus = results.some(r => r.status === 'red') ? 'red'
|
const overallStatus = results.some(r => r.status === 'red') ? 'red'
|
||||||
: results.some(r => r.status === 'yellow') ? 'yellow'
|
: results.some(r => r.status === 'yellow') ? 'yellow' : 'green';
|
||||||
: 'green';
|
|
||||||
|
|
||||||
return [{ json: { results, overallStatus } }];
|
return [{ json: { results, overallStatus } }];
|
||||||
```
|
```
|
||||||
|
|
||||||
- [ ] **Add node 5 — Build Email (Code node):**
|
- [ ] Create `scripts/n8n-build-email.js`:
|
||||||
- Node type: **Code**
|
|
||||||
- Language: **JavaScript**
|
|
||||||
- Connect from Evaluate Sources node
|
|
||||||
- Paste this code exactly:
|
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
|
// Builds subject + body from Evaluate Sources output.
|
||||||
|
// Uses an array of lines joined at the end to avoid \n escape issues in JSON storage.
|
||||||
const { results, overallStatus } = $input.first().json;
|
const { results, overallStatus } = $input.first().json;
|
||||||
|
|
||||||
const icon = { green: '✓', yellow: '⚠', red: '✗' };
|
const icon = { green: '✓', yellow: '⚠', red: '✗' };
|
||||||
const label = { green: 'Backups healthy', yellow: 'Backup warning', red: 'Backup failure' };
|
const label = { green: 'Backups healthy', yellow: 'Backup warning', red: 'Backup failure' };
|
||||||
|
|
||||||
const weekOf = new Date().toLocaleDateString('en-US', { month: 'short', day: 'numeric' });
|
const weekOf = new Date().toLocaleDateString('en-US', { month: 'short', day: 'numeric' });
|
||||||
const subject = `${icon[overallStatus]} ${label[overallStatus]} — week of ${weekOf}`;
|
const subject = icon[overallStatus] + ' ' + label[overallStatus] + ' — week of ' + weekOf;
|
||||||
|
|
||||||
const fmt = ts => ts ? new Date(ts).toLocaleString('en-US', { month: 'short', day: 'numeric', hour: '2-digit', minute: '2-digit' }) : 'unknown';
|
const fmt = ts => ts
|
||||||
|
? new Date(ts).toLocaleString('en-US', { month: 'short', day: 'numeric', hour: '2-digit', minute: '2-digit' })
|
||||||
|
: 'unknown';
|
||||||
|
|
||||||
let body = '';
|
const L = [];
|
||||||
|
|
||||||
if (overallStatus === 'green') {
|
if (overallStatus === 'green') {
|
||||||
body += `All backup sources ran successfully this week.\n\n`;
|
L.push('All backup sources ran successfully this week.');
|
||||||
body += results.map(r =>
|
L.push('');
|
||||||
`✓ ${r.machine} / ${r.destination} (last success: ${fmt(r.lastSuccess?.timestamp)})`
|
results.forEach(r =>
|
||||||
).join('\n');
|
L.push('✓ ' + r.machine + ' / ' + r.destination +
|
||||||
|
' (last success: ' + fmt(r.lastSuccess && r.lastSuccess.timestamp) + ')')
|
||||||
|
);
|
||||||
} else {
|
} else {
|
||||||
const problems = results.filter(r => r.status !== 'green');
|
const problems = results.filter(r => r.status !== 'green');
|
||||||
const healthy = results.filter(r => r.status === 'green');
|
const healthy = results.filter(r => r.status === 'green');
|
||||||
|
|
||||||
body += `${problems.length} source(s) need attention:\n\n`;
|
L.push(problems.length + ' source(s) need attention:');
|
||||||
|
L.push('');
|
||||||
|
|
||||||
for (const r of problems) {
|
problems.forEach(r => {
|
||||||
body += `${icon[r.status]} ${r.machine} / ${r.destination}\n`;
|
L.push(icon[r.status] + ' ' + r.machine + ' / ' + r.destination);
|
||||||
|
|
||||||
if (r.status === 'yellow') {
|
if (r.status === 'yellow') {
|
||||||
body += ` Ran this week but never succeeded.\n`;
|
L.push(' Ran this week but never succeeded.');
|
||||||
const shown = r.runs.slice(0, 3);
|
r.runs.slice(0, 3).forEach(run => {
|
||||||
for (const run of shown) {
|
var line = ' ' + fmt(run.timestamp) + ' [' + run.status + ']';
|
||||||
body += ` ${fmt(run.timestamp)} [${run.status}]`;
|
if (run.log_path) line += ' log: ' + run.log_path;
|
||||||
if (run.log_path) body += ` log: ${run.log_path}`;
|
L.push(line);
|
||||||
body += '\n';
|
if (run.notes) L.push(' notes: ' + (run.notes || '').slice(0, 300));
|
||||||
if (run.notes) body += ` notes: ${run.notes.slice(0, 300).replace(/\n/g, ' ')}\n`;
|
});
|
||||||
}
|
L.push('');
|
||||||
body += `\n Next steps:\n`;
|
L.push(' Next steps:');
|
||||||
body += ` 1. Check the log at the path shown above\n`;
|
L.push(' 1. Check the log at the path shown above');
|
||||||
body += ` 2. For Synology failures: verify LAN reachability (ssh synology) and give disks 30s to spin up\n`;
|
L.push(' 2. For Synology failures: verify LAN reachability (ssh synology) — disks need ~30s to spin up');
|
||||||
body += ` 3. Re-run manually: bash ~/.local/bin/backup.sh\n`;
|
L.push(' 3. Re-run manually: bash ~/.local/bin/backup.sh');
|
||||||
body += ` 4. One-off skip (power outage, travel) is expected — yellow is a heads-up, not an alarm\n\n`;
|
L.push(' 4. One-off skip (power outage, travel) is expected — yellow is a heads-up, not an alarm');
|
||||||
|
L.push('');
|
||||||
}
|
}
|
||||||
|
|
||||||
if (r.status === 'red') {
|
if (r.status === 'red') {
|
||||||
body += ` No runs reported all week — silent failure.\n`;
|
L.push(' No runs reported all week — silent failure.');
|
||||||
if (r.lastSuccess) {
|
if (r.lastSuccess) {
|
||||||
body += ` Last known good run: ${fmt(r.lastSuccess.timestamp)}\n`;
|
L.push(' Last known good run: ' + fmt(r.lastSuccess.timestamp));
|
||||||
if (r.lastSuccess.log_path) body += ` Last log: ${r.lastSuccess.log_path}\n`;
|
if (r.lastSuccess.log_path) L.push(' Last log: ' + r.lastSuccess.log_path);
|
||||||
} else {
|
} else {
|
||||||
body += ` No history found in the datatable.\n`;
|
L.push(' No history found in the datatable.');
|
||||||
}
|
|
||||||
body += `\n Next steps:\n`;
|
|
||||||
body += ` 1. Check systemd timer: systemctl --user status restic-backup.timer\n`;
|
|
||||||
body += ` 2. Verify backup.sh calls notify_backup.rb at the end\n`;
|
|
||||||
body += ` 3. Check n8n Workflow 1 execution log for webhook errors\n`;
|
|
||||||
body += ` 4. Run manually: bash ~/.local/bin/backup.sh\n\n`;
|
|
||||||
}
|
}
|
||||||
|
L.push('');
|
||||||
|
L.push(' Next steps:');
|
||||||
|
L.push(' 1. Check systemd timer: systemctl --user status restic-backup.timer');
|
||||||
|
L.push(' 2. Verify backup.sh calls notify_backup.rb at the end');
|
||||||
|
L.push(' 3. Check n8n Workflow 1 execution log for webhook errors');
|
||||||
|
L.push(' 4. Run manually: bash ~/.local/bin/backup.sh');
|
||||||
|
L.push('');
|
||||||
}
|
}
|
||||||
|
});
|
||||||
|
|
||||||
if (healthy.length > 0) {
|
if (healthy.length > 0) {
|
||||||
body += `Healthy sources:\n`;
|
L.push('Healthy sources:');
|
||||||
body += healthy.map(r => `✓ ${r.machine} / ${r.destination}`).join('\n');
|
healthy.forEach(r => L.push('✓ ' + r.machine + ' / ' + r.destination));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return [{ json: { subject, body } }];
|
return [{ json: { subject, body: L.join('\n') } }];
|
||||||
```
|
```
|
||||||
|
|
||||||
- [ ] **Add node 6 — Send via MailPace (HTTP Request node):**
|
- [ ] Create `scripts/build-n8n-workflows.rb`:
|
||||||
- Node type: **HTTP Request**
|
|
||||||
- Method: `POST`
|
|
||||||
- URL: `https://app.mailpace.com/api/v1/send`
|
|
||||||
- Authentication: **Predefined Credential Type** → **Header Auth** → `mailpace-token`
|
|
||||||
- Body Content Type: **JSON**
|
|
||||||
- Body parameters:
|
|
||||||
- `from`: `backups@swansoncloud.com` (or whichever sending address your MailPace server is configured for)
|
|
||||||
- `to`: `jaredmswanson@gmail.com`
|
|
||||||
- `subject`: `{{ $json.subject }}`
|
|
||||||
- `text_body`: `{{ $json.body }}`
|
|
||||||
- Connect from Build Email node
|
|
||||||
|
|
||||||
- [ ] **Test the reporter by running it manually:**
|
```ruby
|
||||||
- In n8n, open the workflow and click **"Test workflow"** (or **"Execute Workflow"**)
|
#!/usr/bin/env ruby
|
||||||
- Check that an email arrives at jaredmswanson@gmail.com
|
# frozen_string_literal: true
|
||||||
- Expect RED or YELLOW since no real backup data is in the table yet — that's correct
|
#
|
||||||
|
# Generates importable n8n workflow JSON from component pieces.
|
||||||
|
# Run: ruby scripts/build-n8n-workflows.rb
|
||||||
|
# Output: scripts/n8n-workflow-receiver.json
|
||||||
|
# scripts/n8n-workflow-reporter.json
|
||||||
|
#
|
||||||
|
# Before running:
|
||||||
|
# 1. Complete Task 7 and verify DATATABLE_TYPE / DATATABLE_TYPE_VERSION below.
|
||||||
|
# 2. After Task 10 (credentials set up), replace REPLACE_WITH_*_CREDENTIAL_ID
|
||||||
|
# in the generated JSON and re-run to regenerate clean files before import.
|
||||||
|
|
||||||
- [ ] **Activate the workflow** (toggle in top-right)
|
require 'json'
|
||||||
|
require 'securerandom'
|
||||||
|
|
||||||
|
SCRIPTS_DIR = File.dirname(__FILE__)
|
||||||
|
|
||||||
|
# Verify against the exported scratch-workflow JSON from Task 7.
|
||||||
|
DATATABLE_TYPE = 'n8n-nodes-base.dataTable'
|
||||||
|
DATATABLE_TYPE_VERSION = 1
|
||||||
|
|
||||||
|
evaluate_sources_js = File.read(File.join(SCRIPTS_DIR, 'n8n-evaluate-sources.js'))
|
||||||
|
build_email_js = File.read(File.join(SCRIPTS_DIR, 'n8n-build-email.js'))
|
||||||
|
|
||||||
|
# ── Workflow 1: Receiver ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
receiver = {
|
||||||
|
name: 'Backup Monitor — Receiver',
|
||||||
|
nodes: [
|
||||||
|
{
|
||||||
|
parameters: {
|
||||||
|
httpMethod: 'POST',
|
||||||
|
path: 'backup-notify',
|
||||||
|
authentication: 'headerAuth',
|
||||||
|
responseMode: 'responseNode',
|
||||||
|
options: {}
|
||||||
|
},
|
||||||
|
id: SecureRandom.uuid,
|
||||||
|
name: 'Webhook',
|
||||||
|
type: 'n8n-nodes-base.webhook',
|
||||||
|
typeVersion: 2,
|
||||||
|
position: [240, 300],
|
||||||
|
webhookId: SecureRandom.uuid,
|
||||||
|
credentials: {
|
||||||
|
httpHeaderAuth: {
|
||||||
|
id: 'REPLACE_WITH_BACKUP_CREDENTIAL_ID',
|
||||||
|
name: 'backup-monitor-token'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
parameters: {
|
||||||
|
resource: 'row',
|
||||||
|
operation: 'insert',
|
||||||
|
dataTableId: {
|
||||||
|
'__rl' => true,
|
||||||
|
mode: 'list',
|
||||||
|
value: 'REPLACE_WITH_DATATABLE_ID',
|
||||||
|
cachedResultName: 'backup_runs'
|
||||||
|
},
|
||||||
|
columns: {
|
||||||
|
mappingMode: 'defineBelow',
|
||||||
|
value: {
|
||||||
|
timestamp: '={{ $json.body.timestamp }}',
|
||||||
|
machine: '={{ $json.body.machine }}',
|
||||||
|
destination: '={{ $json.body.destination }}',
|
||||||
|
status: '={{ $json.body.status }}',
|
||||||
|
bytes_transferred: '={{ $json.body.bytes_transferred }}',
|
||||||
|
snapshot_id: '={{ $json.body.snapshot_id }}',
|
||||||
|
log_path: '={{ $json.body.log_path }}',
|
||||||
|
notes: '={{ $json.body.notes }}'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
id: SecureRandom.uuid,
|
||||||
|
name: 'Insert to backup_runs',
|
||||||
|
type: DATATABLE_TYPE,
|
||||||
|
typeVersion: DATATABLE_TYPE_VERSION,
|
||||||
|
position: [520, 300]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
parameters: {
|
||||||
|
respondWith: 'json',
|
||||||
|
responseBody: '={{ JSON.stringify({ ok: true }) }}',
|
||||||
|
options: { responseCode: 200 }
|
||||||
|
},
|
||||||
|
id: SecureRandom.uuid,
|
||||||
|
name: 'Respond 200',
|
||||||
|
type: 'n8n-nodes-base.respondToWebhook',
|
||||||
|
typeVersion: 1.1,
|
||||||
|
position: [800, 300]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
connections: {
|
||||||
|
'Webhook' => { main: [[{ node: 'Insert to backup_runs', type: 'main', index: 0 }]] },
|
||||||
|
'Insert to backup_runs' => { main: [[{ node: 'Respond 200', type: 'main', index: 0 }]] }
|
||||||
|
},
|
||||||
|
active: false,
|
||||||
|
settings: { executionOrder: 'v1' },
|
||||||
|
pinData: {},
|
||||||
|
meta: { templateCredsSetupCompleted: true },
|
||||||
|
tags: []
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── Workflow 2: Reporter ──────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
reporter = {
|
||||||
|
name: 'Backup Monitor — Reporter',
|
||||||
|
nodes: [
|
||||||
|
{
|
||||||
|
parameters: {
|
||||||
|
rule: {
|
||||||
|
interval: [
|
||||||
|
{
|
||||||
|
field: 'weeks',
|
||||||
|
weeksInterval: 1,
|
||||||
|
triggerAtDay: [5],
|
||||||
|
triggerAtHour: 8,
|
||||||
|
triggerAtMinute: 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
id: SecureRandom.uuid,
|
||||||
|
name: 'Every Friday 8 AM',
|
||||||
|
type: 'n8n-nodes-base.scheduleTrigger',
|
||||||
|
typeVersion: 1.2,
|
||||||
|
position: [240, 300]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
parameters: {
|
||||||
|
mode: 'manual',
|
||||||
|
includeOtherFields: false,
|
||||||
|
assignments: {
|
||||||
|
assignments: [
|
||||||
|
{
|
||||||
|
id: SecureRandom.uuid,
|
||||||
|
name: 'sources',
|
||||||
|
value: '={{ [{"machine":"desktop","destination":"synology"},{"machine":"desktop","destination":"backblaze-b2"},{"machine":"vps-ovh-prod-01","destination":"local"}] }}',
|
||||||
|
type: 'arrayValue'
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
options: {}
|
||||||
|
},
|
||||||
|
id: SecureRandom.uuid,
|
||||||
|
name: 'Set sources',
|
||||||
|
type: 'n8n-nodes-base.set',
|
||||||
|
typeVersion: 3.4,
|
||||||
|
position: [460, 300]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
parameters: {
|
||||||
|
resource: 'row',
|
||||||
|
operation: 'getMany',
|
||||||
|
dataTableId: {
|
||||||
|
'__rl' => true,
|
||||||
|
mode: 'list',
|
||||||
|
value: 'REPLACE_WITH_DATATABLE_ID',
|
||||||
|
cachedResultName: 'backup_runs'
|
||||||
|
},
|
||||||
|
filters: {
|
||||||
|
conditions: [
|
||||||
|
{
|
||||||
|
keyName: 'timestamp',
|
||||||
|
condition: 'gt',
|
||||||
|
keyValue: '={{ $now.minus({days: 7}).toISO() }}'
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
options: {}
|
||||||
|
},
|
||||||
|
id: SecureRandom.uuid,
|
||||||
|
name: 'Get backup_runs (last 7d)',
|
||||||
|
type: DATATABLE_TYPE,
|
||||||
|
typeVersion: DATATABLE_TYPE_VERSION,
|
||||||
|
position: [680, 300]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
parameters: {
|
||||||
|
mode: 'runOnceForAllItems',
|
||||||
|
language: 'javaScript',
|
||||||
|
jsCode: evaluate_sources_js
|
||||||
|
},
|
||||||
|
id: SecureRandom.uuid,
|
||||||
|
name: 'Evaluate Sources',
|
||||||
|
type: 'n8n-nodes-base.code',
|
||||||
|
typeVersion: 2,
|
||||||
|
position: [900, 300]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
parameters: {
|
||||||
|
mode: 'runOnceForAllItems',
|
||||||
|
language: 'javaScript',
|
||||||
|
jsCode: build_email_js
|
||||||
|
},
|
||||||
|
id: SecureRandom.uuid,
|
||||||
|
name: 'Build Email',
|
||||||
|
type: 'n8n-nodes-base.code',
|
||||||
|
typeVersion: 2,
|
||||||
|
position: [1120, 300]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
parameters: {
|
||||||
|
method: 'POST',
|
||||||
|
url: 'https://app.mailpace.com/api/v1/send',
|
||||||
|
authentication: 'genericCredentialType',
|
||||||
|
genericAuthType: 'httpHeaderAuth',
|
||||||
|
sendBody: true,
|
||||||
|
specifyBody: 'json',
|
||||||
|
# Expression returning a JS object; n8n serializes it to JSON automatically.
|
||||||
|
# Replace REPLACE_WITH_MAILPACE_FROM_ADDRESS with your verified MailPace sending address.
|
||||||
|
jsonBody: '={{ { from: "REPLACE_WITH_MAILPACE_FROM_ADDRESS", to: "jaredmswanson@gmail.com", subject: $json.subject, text_body: $json.body } }}',
|
||||||
|
options: {}
|
||||||
|
},
|
||||||
|
id: SecureRandom.uuid,
|
||||||
|
name: 'Send via MailPace',
|
||||||
|
type: 'n8n-nodes-base.httpRequest',
|
||||||
|
typeVersion: 4.2,
|
||||||
|
position: [1340, 300],
|
||||||
|
credentials: {
|
||||||
|
httpHeaderAuth: {
|
||||||
|
id: 'REPLACE_WITH_MAILPACE_CREDENTIAL_ID',
|
||||||
|
name: 'mailpace-token'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
connections: {
|
||||||
|
'Every Friday 8 AM' => { main: [[{ node: 'Set sources', type: 'main', index: 0 }]] },
|
||||||
|
'Set sources' => { main: [[{ node: 'Get backup_runs (last 7d)', type: 'main', index: 0 }]] },
|
||||||
|
'Get backup_runs (last 7d)' => { main: [[{ node: 'Evaluate Sources', type: 'main', index: 0 }]] },
|
||||||
|
'Evaluate Sources' => { main: [[{ node: 'Build Email', type: 'main', index: 0 }]] },
|
||||||
|
'Build Email' => { main: [[{ node: 'Send via MailPace', type: 'main', index: 0 }]] }
|
||||||
|
},
|
||||||
|
active: false,
|
||||||
|
settings: { executionOrder: 'v1' },
|
||||||
|
pinData: {},
|
||||||
|
meta: { templateCredsSetupCompleted: true },
|
||||||
|
tags: []
|
||||||
|
}
|
||||||
|
|
||||||
|
File.write(File.join(SCRIPTS_DIR, 'n8n-workflow-receiver.json'), JSON.pretty_generate(receiver))
|
||||||
|
File.write(File.join(SCRIPTS_DIR, 'n8n-workflow-reporter.json'), JSON.pretty_generate(reporter))
|
||||||
|
|
||||||
|
puts 'Generated:'
|
||||||
|
puts " #{SCRIPTS_DIR}/n8n-workflow-receiver.json"
|
||||||
|
puts " #{SCRIPTS_DIR}/n8n-workflow-reporter.json"
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] Run the build script:
|
||||||
|
```bash
|
||||||
|
cd /home/jared/systems-admin
|
||||||
|
ruby scripts/build-n8n-workflows.rb
|
||||||
|
```
|
||||||
|
Expected: two files created with no error output
|
||||||
|
|
||||||
|
- [ ] Validate both generated files:
|
||||||
|
```bash
|
||||||
|
python3 scripts/validate-n8n-workflow.py scripts/n8n-workflow-receiver.json
|
||||||
|
python3 scripts/validate-n8n-workflow.py scripts/n8n-workflow-reporter.json
|
||||||
|
```
|
||||||
|
Expected: both print `OK — N nodes, N connections, 0 warning(s).`
|
||||||
|
|
||||||
|
If warnings appear about `typeVersion`, update the constants in `build-n8n-workflows.rb` to match what Task 7 found, re-run, and re-validate.
|
||||||
|
|
||||||
|
- [ ] Commit:
|
||||||
|
```bash
|
||||||
|
git add scripts/n8n-evaluate-sources.js scripts/n8n-build-email.js \
|
||||||
|
scripts/build-n8n-workflows.rb \
|
||||||
|
scripts/n8n-workflow-receiver.json scripts/n8n-workflow-reporter.json
|
||||||
|
git commit -m "feat(backup-monitor): add workflow JS files, build script, and generated workflow JSON"
|
||||||
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Task 10: Integrate `notify_backup.rb` into desktop `~/.local/bin/backup.sh`
|
## Task 10: Set up credentials and import workflows via n8n API
|
||||||
|
|
||||||
|
- [ ] **Create the backup-monitor credential in n8n UI:**
|
||||||
|
- Credentials → New → **Header Auth**
|
||||||
|
- Name: `backup-monitor-token`
|
||||||
|
- Header Name field: `Authorization`
|
||||||
|
- Header Value field: `Bearer <value of N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN>`
|
||||||
|
- Save
|
||||||
|
|
||||||
|
- [ ] **Create the MailPace credential in n8n UI:**
|
||||||
|
- Credentials → New → **Header Auth**
|
||||||
|
- Name: `mailpace-token`
|
||||||
|
- Header Name field: `X-Server-Token`
|
||||||
|
- Header Value field: your MailPace server API token (find at app.mailpace.com → Server Settings)
|
||||||
|
- Save
|
||||||
|
|
||||||
|
- [ ] **Get credential IDs from the n8n API:**
|
||||||
|
```bash
|
||||||
|
source ~/.credentials
|
||||||
|
curl -s "https://n8n.swansoncloud.com/api/v1/credentials" \
|
||||||
|
-H "X-N8N-API-KEY: $N8N_SWANSONCLOUD_API_KEY" | python3 -m json.tool | grep -E '"id"|"name"'
|
||||||
|
```
|
||||||
|
Note the numeric `id` for `backup-monitor-token` and `mailpace-token`.
|
||||||
|
|
||||||
|
- [ ] **Fill in the credential IDs and datatable ID.** Edit `scripts/build-n8n-workflows.rb`:
|
||||||
|
- Replace `REPLACE_WITH_BACKUP_CREDENTIAL_ID` with the backup-monitor-token ID
|
||||||
|
- Replace `REPLACE_WITH_MAILPACE_CREDENTIAL_ID` with the mailpace-token ID
|
||||||
|
- Replace both `REPLACE_WITH_DATATABLE_ID` occurrences with the datatable ID from Task 7
|
||||||
|
- Replace `REPLACE_WITH_MAILPACE_FROM_ADDRESS` with your verified MailPace sending address
|
||||||
|
|
||||||
|
- [ ] **Regenerate the workflow JSON files:**
|
||||||
|
```bash
|
||||||
|
ruby scripts/build-n8n-workflows.rb
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Validate again after substitution:**
|
||||||
|
```bash
|
||||||
|
python3 scripts/validate-n8n-workflow.py scripts/n8n-workflow-receiver.json
|
||||||
|
python3 scripts/validate-n8n-workflow.py scripts/n8n-workflow-reporter.json
|
||||||
|
```
|
||||||
|
Expected: both `OK`
|
||||||
|
|
||||||
|
- [ ] **Import Workflow 1 via the n8n API:**
|
||||||
|
```bash
|
||||||
|
source ~/.credentials
|
||||||
|
curl -s -X POST "https://n8n.swansoncloud.com/api/v1/workflows" \
|
||||||
|
-H "X-N8N-API-KEY: $N8N_SWANSONCLOUD_API_KEY" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d @scripts/n8n-workflow-receiver.json | python3 -m json.tool | grep '"id"' | head -1
|
||||||
|
```
|
||||||
|
Note the workflow `id` returned (e.g. `42`).
|
||||||
|
|
||||||
|
- [ ] **Activate Workflow 1:**
|
||||||
|
```bash
|
||||||
|
curl -s -X PATCH "https://n8n.swansoncloud.com/api/v1/workflows/WORKFLOW_1_ID" \
|
||||||
|
-H "X-N8N-API-KEY: $N8N_SWANSONCLOUD_API_KEY" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"active": true}' | python3 -m json.tool | grep '"active"'
|
||||||
|
```
|
||||||
|
Expected: `"active": true`
|
||||||
|
|
||||||
|
- [ ] **Import Workflow 2:**
|
||||||
|
```bash
|
||||||
|
curl -s -X POST "https://n8n.swansoncloud.com/api/v1/workflows" \
|
||||||
|
-H "X-N8N-API-KEY: $N8N_SWANSONCLOUD_API_KEY" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d @scripts/n8n-workflow-reporter.json | python3 -m json.tool | grep '"id"' | head -1
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Activate Workflow 2:**
|
||||||
|
```bash
|
||||||
|
curl -s -X PATCH "https://n8n.swansoncloud.com/api/v1/workflows/WORKFLOW_2_ID" \
|
||||||
|
-H "X-N8N-API-KEY: $N8N_SWANSONCLOUD_API_KEY" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"active": true}' | python3 -m json.tool | grep '"active"'
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **Verify both workflows are visible and active** in the n8n UI at https://n8n.swansoncloud.com.
|
||||||
|
|
||||||
|
- [ ] **Test Workflow 1 with a real webhook call:**
|
||||||
|
```bash
|
||||||
|
source ~/.credentials
|
||||||
|
notify_backup.rb \
|
||||||
|
--machine desktop \
|
||||||
|
--destination test \
|
||||||
|
--status success \
|
||||||
|
--notes "smoke test $(date)"
|
||||||
|
```
|
||||||
|
Expected: no error output. Verify a row appears in the `backup_runs` table in n8n, then delete it.
|
||||||
|
|
||||||
|
- [ ] Commit the final build script with real IDs:
|
||||||
|
```bash
|
||||||
|
cd /home/jared/systems-admin
|
||||||
|
git add scripts/build-n8n-workflows.rb scripts/n8n-workflow-receiver.json scripts/n8n-workflow-reporter.json
|
||||||
|
git commit -m "feat(backup-monitor): fill credential/datatable IDs and reimport workflows"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Task 11: Integrate `notify_backup.rb` into desktop `~/.local/bin/backup.sh`
|
||||||
|
|
||||||
The existing `backup.sh` uses `set -euo pipefail`. We add a helper function that temporarily disables `set -e` to capture the exit code of `run_backup`, then always re-enables it, then sends the notification.
|
The existing `backup.sh` uses `set -euo pipefail`. We add a helper function that temporarily disables `set -e` to capture the exit code of `run_backup`, then always re-enables it, then sends the notification.
|
||||||
|
|
||||||
|
|
@ -945,7 +1377,7 @@ git commit -m "feat(backup-monitor): integrate notify_backup.rb into desktop bac
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Task 11: Deploy and integrate on OVH VPS
|
## Task 12: Deploy and integrate on OVH VPS
|
||||||
|
|
||||||
The VPS backup script is at `~/services/backup.sh` on `15.204.247.153`. The notifier script and ENV vars must be deployed there too.
|
The VPS backup script is at `~/services/backup.sh` on `15.204.247.153`. The notifier script and ENV vars must be deployed there too.
|
||||||
|
|
||||||
|
|
@ -1018,7 +1450,7 @@ git commit --allow-empty -m "feat(backup-monitor): deploy notify_backup.rb to OV
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Task 12: End-to-end smoke test
|
## Task 13: End-to-end smoke test
|
||||||
|
|
||||||
- [ ] **Trigger the desktop backup manually** and watch for webhook activity:
|
- [ ] **Trigger the desktop backup manually** and watch for webhook activity:
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -1034,7 +1466,7 @@ Expected: backup runs as normal, no new errors in output
|
||||||
- If all sources have at least one success: subject line starts with `✓`
|
- If all sources have at least one success: subject line starts with `✓`
|
||||||
- If any source has no success: subject starts with `⚠` or `✗` with detail in the body
|
- If any source has no success: subject starts with `⚠` or `✗` with detail in the body
|
||||||
|
|
||||||
- [ ] **Remove any test rows** from `backup_runs` (rows with `destination=test` from Task 8)
|
- [ ] **Remove any test rows** from `backup_runs` (rows with `destination=test` from Task 10)
|
||||||
|
|
||||||
- [ ] Final commit:
|
- [ ] Final commit:
|
||||||
```bash
|
```bash
|
||||||
|
|
@ -1047,7 +1479,9 @@ git commit -m "docs(backup-monitor): implementation complete — smoke test pass
|
||||||
|
|
||||||
## Self-Review Notes
|
## Self-Review Notes
|
||||||
|
|
||||||
- **Spec coverage:** Ruby script (Tasks 1–6) ✓ · Datatable (Task 7) ✓ · Workflow 1 (Task 8) ✓ · Workflow 2 (Task 9) ✓ · Desktop integration (Task 10) ✓ · VPS integration (Task 11) ✓ · Email format green/yellow/red (Task 9) ✓ · Auth token from ENV (Tasks 8, 10) ✓ · `log_path` field (Tasks 2–3, 10) ✓ · `skipped` status (Task 10) ✓
|
- **Spec coverage:** Ruby script (Tasks 1–6) ✓ · Datatable bootstrap (Task 7) ✓ · Validator (Task 8) ✓ · Workflow JSON generation (Task 9) ✓ · Credential setup + import (Task 10) ✓ · Desktop integration (Task 11) ✓ · VPS integration (Task 12) ✓ · Email format green/yellow/red (Task 9) ✓ · Auth token from ENV (Tasks 9–10) ✓ · `log_path` field (Tasks 2–3, 11) ✓ · `skipped` status (Task 11) ✓
|
||||||
- **`backup.sh` variable name:** The VPS script uses `LOG_FILE` not `LOG` — Task 11 calls this out explicitly.
|
- **Data Table node uncertainty:** Both research agents flagged the Data Table node type as uncertain. Task 7 resolves this by exporting a probe workflow. If the type or parameter shape differs from `n8n-nodes-base.dataTable`, update the constants in `build-n8n-workflows.rb` and regenerate before import.
|
||||||
- **MailPace `from` address:** Needs to match a verified sending domain in your MailPace account. Confirm before activating Workflow 2.
|
- **MailPace credential header:** Must be `X-Server-Token` (not `Authorization` or `MailPace-Server-Token`). This is set in Task 10 when creating the `mailpace-token` Header Auth credential.
|
||||||
- **n8n node naming:** The Code node in Task 9 references `$('Set: Expected Sources')` by name — the Set node in step 2 must be named exactly `Set: Expected Sources` or update the reference to match.
|
- **MailPace `from` address:** Must match a verified sending domain in your MailPace server. Confirm at app.mailpace.com before filling in `build-n8n-workflows.rb`.
|
||||||
|
- **`backup.sh` variable name:** The VPS script uses `LOG_FILE` not `LOG` — Task 12 calls this out explicitly.
|
||||||
|
- **Node name consistency:** `n8n-evaluate-sources.js` references `$('Set sources')` and `$('Get backup_runs (last 7d)')` — these must match the `name` fields in `build-n8n-workflows.rb` exactly (they do as written).
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue