server-desktop-01/docs/superpowers/plans/2026-05-07-n8n-backup-monit...

50 KiB
Raw Blame History

n8n Backup Monitor Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Build a webhook-based backup health monitoring system: a reusable Ruby notifier script, two n8n workflows (receiver + weekly reporter), and integrations into the desktop and VPS backup scripts.

Architecture: notify_backup.rb is a standalone Ruby CLI script (no gems, stdlib only) that POSTs a structured JSON payload to an n8n webhook after each backup run. n8n Workflow 1 validates auth and records each run to a datatable. Every Friday at 8 AM, n8n Workflow 2 reads the week's data, compares it against a hardcoded expected-sources list, and emails a green/yellow/red health digest via MailPace.

Tech Stack: Ruby stdlib (net/http, json, optparse, minitest), n8n self-hosted at https://n8n.swansoncloud.com (Webhook, Schedule, Code, HTTP Request nodes, datatables), MailPace REST API, bash


File Map

File Action Responsibility
scripts/notify_backup.rb Create Source-of-record for the notifier script
scripts/notify_backup_test.rb Create Minitest suite for all four classes
scripts/n8n-evaluate-sources.js Create JS code for the Evaluate Sources Code node
scripts/n8n-build-email.js Create JS code for the Build Email Code node
scripts/build-n8n-workflows.rb Create Assembles workflow JSON using JSON.generate (handles all JS escaping)
scripts/n8n-workflow-receiver.json Generated Importable Workflow 1 JSON
scripts/n8n-workflow-reporter.json Generated Importable Workflow 2 JSON
scripts/validate-n8n-workflow.py Create Structural validator for n8n workflow JSON files
~/.local/bin/notify_backup.rb Deploy Runtime location on desktop
~/.local/bin/backup.sh Modify Add notify calls after each restic run
~/services/backup.sh on OVH Modify via SSH Add notify calls after VPS backup run

Task 1: Script skeleton and test infrastructure

Files:

  • Create: scripts/notify_backup.rb

  • Create: scripts/notify_backup_test.rb

  • Create the scripts/ directory:

mkdir -p /home/jared/systems-admin/scripts
  • Create scripts/notify_backup.rb with the full header comment and empty class stubs. The if __FILE__ == $PROGRAM_NAME guard keeps the CLI from running during tests:
#!/usr/bin/env ruby
# frozen_string_literal: true

# notify_backup — send a backup run result to the n8n backup monitor
#
# USAGE
#   notify_backup.rb [options]
#
# REQUIRED OPTIONS
#   --machine NAME        Identifier for this host (e.g. desktop, vps-ovh-prod-01)
#   --destination NAME    Backup destination (e.g. synology, backblaze-b2, local)
#   --status STATUS       Result: success | failure | skipped
#
# OPTIONAL OPTIONS
#   --bytes N             Bytes transferred (raw integer; formatted to human-readable before sending)
#   --snapshot ID         Restic snapshot ID or equivalent
#   --log-path PATH       Path to the backup log file for this run
#   --notes TEXT          Free-form output (restic tail, skip reason, error message, etc.)
#
# REQUIRED ENV VARS
#   N8N_SWANSONCLOUD_URL                Base URL of the n8n instance (no trailing slash)
#                                       e.g. https://n8n.swansoncloud.com
#   N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN  Bearer token for webhook authentication
#
# EXIT BEHAVIOR
#   Always exits 0. A notification failure is logged to stderr but never causes
#   the calling backup process to fail — the backup ran; the log is the source of truth.
#
# ADDING A NEW BACKUP SOURCE
#   1. Ensure the two ENV vars above are available on the new host
#   2. Call this script at the end of the backup process with --machine, --destination, --status
#   3. Add the {machine, destination} pair to the Set: Expected Sources node in n8n Workflow 2
#
# CLASSES
#   ByteFormatter    raw integer bytes → human-readable string (e.g. "4.50 GB")
#   PayloadBuilder   converts CLI option hash into a clean JSON-ready hash
#   WebhookClient    sends HTTP POST with auth header; knows nothing about backup data
#   BackupNotifier   orchestrates: build payload → send; the only public entry point

require 'net/http'
require 'json'
require 'optparse'

class ByteFormatter
end

class PayloadBuilder
end

class WebhookClient
end

class BackupNotifier
end

if __FILE__ == $PROGRAM_NAME
  # CLI entry point — implemented in Task 5
end
  • Create scripts/notify_backup_test.rb:
#!/usr/bin/env ruby
# frozen_string_literal: true

require 'minitest/autorun'
require_relative 'notify_backup'
  • Run the test file to confirm the skeleton loads without errors:
ruby /home/jared/systems-admin/scripts/notify_backup_test.rb

Expected output: 0 runs, 0 assertions, 0 failures, 0 errors

  • Commit:
cd /home/jared/systems-admin
git add scripts/
git commit -m "feat(backup-monitor): add notify_backup script skeleton and test infrastructure"

Task 2: ByteFormatter

Files:

  • Modify: scripts/notify_backup.rb — implement ByteFormatter

  • Modify: scripts/notify_backup_test.rb — add ByteFormatterTest

  • Add the failing tests to scripts/notify_backup_test.rb:

class ByteFormatterTest < Minitest::Test
  def test_gigabytes
    assert_equal '4.50 GB', ByteFormatter.format(4_831_838_208)
  end

  def test_megabytes
    assert_equal '823.00 MB', ByteFormatter.format(863_076_352)
  end

  def test_kilobytes
    assert_equal '12.00 KB', ByteFormatter.format(12_288)
  end

  def test_bytes_below_one_kb
    assert_equal '512 B', ByteFormatter.format(512)
  end

  def test_nil_returns_nil
    assert_nil ByteFormatter.format(nil)
  end

  def test_string_integer_is_coerced
    assert_equal '1.00 GB', ByteFormatter.format('1073741824')
  end
end
  • Run to confirm 6 failures:
ruby /home/jared/systems-admin/scripts/notify_backup_test.rb

Expected: 6 failures (NoMethodError on ByteFormatter.format)

  • Replace the empty ByteFormatter class stub in scripts/notify_backup.rb with:
class ByteFormatter
  UNITS = [['GB', 1_073_741_824], ['MB', 1_048_576], ['KB', 1_024]].freeze

  def self.format(bytes)
    return nil if bytes.nil?

    bytes = bytes.to_i
    unit, divisor = UNITS.find { |_, d| bytes >= d }
    return "#{bytes} B" unless unit

    '%.2f %s' % [bytes.to_f / divisor, unit]
  end
end
  • Run tests to confirm they pass:
ruby /home/jared/systems-admin/scripts/notify_backup_test.rb

Expected: 6 runs, 6 assertions, 0 failures, 0 errors

  • Commit:
cd /home/jared/systems-admin
git add scripts/
git commit -m "feat(backup-monitor): implement ByteFormatter with tests"

Task 3: PayloadBuilder

Files:

  • Modify: scripts/notify_backup.rb — implement PayloadBuilder

  • Modify: scripts/notify_backup_test.rb — add PayloadBuilderTest

  • Add the failing tests to scripts/notify_backup_test.rb:

class PayloadBuilderTest < Minitest::Test
  def base_options
    { machine: 'desktop', destination: 'synology', status: 'success' }
  end

  def test_builds_required_fields
    payload = PayloadBuilder.new(base_options).build
    assert_equal 'desktop', payload[:machine]
    assert_equal 'synology', payload[:destination]
    assert_equal 'success', payload[:status]
    assert_match(/\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z\z/, payload[:timestamp])
  end

  def test_formats_bytes_to_human_readable
    payload = PayloadBuilder.new(base_options.merge(bytes: 1_073_741_824)).build
    assert_equal '1.00 GB', payload[:bytes_transferred]
  end

  def test_omits_nil_optional_fields
    payload = PayloadBuilder.new(base_options).build
    refute payload.key?(:bytes_transferred)
    refute payload.key?(:snapshot_id)
    refute payload.key?(:log_path)
    refute payload.key?(:notes)
  end

  def test_includes_all_optional_fields_when_present
    opts = base_options.merge(
      bytes: 1_048_576,
      snapshot: 'abc12345',
      log_path: '/home/jared/.local/log/restic/2026-05-07.log',
      notes: 'snapshot abc12345 saved'
    )
    payload = PayloadBuilder.new(opts).build
    assert_equal '1.00 MB',    payload[:bytes_transferred]
    assert_equal 'abc12345',   payload[:snapshot_id]
    assert_equal '/home/jared/.local/log/restic/2026-05-07.log', payload[:log_path]
    assert_equal 'snapshot abc12345 saved', payload[:notes]
  end

  def test_raises_on_invalid_status
    assert_raises(ArgumentError) { PayloadBuilder.new(base_options.merge(status: 'unknown')).build }
  end

  def test_all_valid_statuses_accepted
    %w[success failure skipped].each do |s|
      assert PayloadBuilder.new(base_options.merge(status: s)).build
    end
  end
end
  • Run to confirm failures:
ruby /home/jared/systems-admin/scripts/notify_backup_test.rb

Expected: PayloadBuilderTest failures, ByteFormatterTest still passing

  • Replace the empty PayloadBuilder stub in scripts/notify_backup.rb with:
class PayloadBuilder
  VALID_STATUSES = %w[success failure skipped].freeze

  def initialize(options)
    @options = options
  end

  def build
    validate!
    {
      timestamp:         Time.now.utc.strftime('%Y-%m-%dT%H:%M:%SZ'),
      machine:           @options[:machine],
      destination:       @options[:destination],
      status:            @options[:status],
      bytes_transferred: ByteFormatter.format(@options[:bytes]),
      snapshot_id:       @options[:snapshot],
      log_path:          @options[:log_path],
      notes:             @options[:notes]
    }.compact
  end

  private

  def validate!
    return if VALID_STATUSES.include?(@options[:status])

    raise ArgumentError, "status must be one of: #{VALID_STATUSES.join(', ')}"
  end
end
  • Run all tests:
ruby /home/jared/systems-admin/scripts/notify_backup_test.rb

Expected: 12 runs, 12+ assertions, 0 failures, 0 errors

  • Commit:
cd /home/jared/systems-admin
git add scripts/
git commit -m "feat(backup-monitor): implement PayloadBuilder with tests"

Task 4: WebhookClient

Dependencies are injected so the HTTP layer can be stubbed in tests without making real network calls.

Files:

  • Modify: scripts/notify_backup.rb — implement WebhookClient

  • Modify: scripts/notify_backup_test.rb — add WebhookClientTest + two stub helpers

  • Add the failing tests and helpers to scripts/notify_backup_test.rb:

# Stub HTTP adapter for WebhookClient tests — replaces Net::HTTP entirely.
# Sandi Metz: test the interface, inject the dependency, don't touch globals.
class FakeHttpAdapter
  def initialize(response)
    @response = response
  end

  def start(_host, _port, use_ssl: false)
    yield FakeHttpSession.new(@response)
  end
end

class FakeHttpSession
  def initialize(response)
    @response = response
  end

  def request(_req)
    @response
  end
end

class WebhookClientTest < Minitest::Test
  def make_success_response
    res = Net::HTTPSuccess.new('1.1', '200', 'OK')
    res.instance_variable_set(:@read, true)
    res
  end

  def make_error_response
    res = Net::HTTPServerError.new('1.1', '500', 'Internal Server Error')
    res.instance_variable_set(:@body, 'something broke')
    res.instance_variable_set(:@read, true)
    res
  end

  def make_client(response)
    WebhookClient.new(
      base_url:     'https://n8n.example.com',
      auth_token:   'test-token',
      http_adapter: FakeHttpAdapter.new(response)
    )
  end

  def test_post_succeeds_on_200
    client = make_client(make_success_response)
    client.post({ machine: 'desktop', status: 'success' })  # no error raised
  end

  def test_post_raises_on_500
    client = make_client(make_error_response)
    err = assert_raises(RuntimeError) { client.post({}) }
    assert_match(/HTTP 500/, err.message)
  end
end
  • Run to confirm failures:
ruby /home/jared/systems-admin/scripts/notify_backup_test.rb

Expected: WebhookClientTest failures

  • Replace the empty WebhookClient stub in scripts/notify_backup.rb with:
class WebhookClient
  def initialize(base_url:, auth_token:, http_adapter: Net::HTTP)
    @base_url     = base_url
    @auth_token   = auth_token
    @http_adapter = http_adapter
  end

  def post(payload)
    uri     = URI("#{@base_url}/webhook/backup-notify")
    request = build_request(uri, payload)

    response = @http_adapter.start(uri.hostname, uri.port, use_ssl: uri.scheme == 'https') do |http|
      http.request(request)
    end

    raise "HTTP #{response.code}: #{response.body}" unless response.is_a?(Net::HTTPSuccess)
  end

  private

  def build_request(uri, payload)
    req = Net::HTTP::Post.new(uri)
    req['Content-Type']  = 'application/json'
    req['Authorization'] = "Bearer #{@auth_token}"
    req.body             = JSON.generate(payload)
    req
  end
end
  • Run all tests:
ruby /home/jared/systems-admin/scripts/notify_backup_test.rb

Expected: all tests pass, 0 failures

  • Commit:
cd /home/jared/systems-admin
git add scripts/
git commit -m "feat(backup-monitor): implement WebhookClient with injected HTTP adapter and tests"

Task 5: BackupNotifier and CLI entry point

Files:

  • Modify: scripts/notify_backup.rb — implement BackupNotifier + CLI block

  • Modify: scripts/notify_backup_test.rb — add BackupNotifierTest

  • Add the failing tests to scripts/notify_backup_test.rb:

class BackupNotifierTest < Minitest::Test
  def base_options
    { machine: 'desktop', destination: 'synology', status: 'success' }
  end

  def test_notify_calls_client_with_a_hash_payload
    fake_client = Minitest::Mock.new
    fake_client.expect(:post, nil, [Hash])

    BackupNotifier.new(options: base_options, client: fake_client).notify

    fake_client.verify
  end

  def test_from_env_returns_a_backup_notifier
    ENV['N8N_SWANSONCLOUD_URL']                = 'https://n8n.example.com'
    ENV['N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN']  = 'test-token'

    notifier = BackupNotifier.from_env(base_options)
    assert_instance_of BackupNotifier, notifier
  ensure
    ENV.delete('N8N_SWANSONCLOUD_URL')
    ENV.delete('N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN')
  end

  def test_from_env_raises_when_env_var_missing
    ENV.delete('N8N_SWANSONCLOUD_URL')
    assert_raises(KeyError) { BackupNotifier.from_env(base_options) }
  end
end
  • Run to confirm failures:
ruby /home/jared/systems-admin/scripts/notify_backup_test.rb

Expected: BackupNotifierTest failures

  • Replace the empty BackupNotifier stub in scripts/notify_backup.rb with:
class BackupNotifier
  def initialize(options:, client:)
    @options = options
    @client  = client
  end

  def self.from_env(options)
    url   = ENV.fetch('N8N_SWANSONCLOUD_URL')
    token = ENV.fetch('N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN')
    new(options: options, client: WebhookClient.new(base_url: url, auth_token: token))
  end

  def notify
    payload = PayloadBuilder.new(@options).build
    @client.post(payload)
  end
end
  • Replace the empty CLI block at the bottom of scripts/notify_backup.rb with:
if __FILE__ == $PROGRAM_NAME
  options = {}

  OptionParser.new do |opts|
    opts.banner = 'Usage: notify_backup.rb [options]'
    opts.separator ''
    opts.separator 'Required:'
    opts.on('--machine NAME',     'Host identifier (e.g. desktop, vps-ovh-prod-01)')   { |v| options[:machine] = v }
    opts.on('--destination NAME', 'Backup destination (e.g. synology, backblaze-b2)')  { |v| options[:destination] = v }
    opts.on('--status STATUS',    'success | failure | skipped')                        { |v| options[:status] = v }
    opts.separator ''
    opts.separator 'Optional:'
    opts.on('--bytes N',     Integer, 'Raw bytes transferred (formatted before sending)')  { |v| options[:bytes] = v }
    opts.on('--snapshot ID',         'Snapshot ID (e.g. restic snapshot hash)')            { |v| options[:snapshot] = v }
    opts.on('--log-path PATH',       'Path to the backup log for this run')                { |v| options[:log_path] = v }
    opts.on('--notes TEXT',          'Free-form output: restic tail, skip reason, etc.')   { |v| options[:notes] = v }
  end.parse!

  %i[machine destination status].each do |required|
    abort "Error: --#{required} is required. Run with --help for usage." unless options[required]
  end

  begin
    BackupNotifier.from_env(options).notify
  rescue => e
    warn "notify_backup: failed to send webhook: #{e.message}"
    exit 0
  end
end
  • Run all tests:
ruby /home/jared/systems-admin/scripts/notify_backup_test.rb

Expected: all tests pass (roughly 16+ assertions, 0 failures)

  • Commit:
cd /home/jared/systems-admin
git add scripts/
git commit -m "feat(backup-monitor): implement BackupNotifier and CLI entry point"

Task 6: Deploy to ~/.local/bin/ and verify locally

Files:

  • Deploy: ~/.local/bin/notify_backup.rb

  • Copy and make executable:

cp /home/jared/systems-admin/scripts/notify_backup.rb ~/.local/bin/notify_backup.rb
chmod +x ~/.local/bin/notify_backup.rb
  • Verify --help output shows usage cleanly:
notify_backup.rb --help

Expected: usage banner with Required/Optional sections visible

  • Verify missing required arg exits with a clear message:
notify_backup.rb --machine desktop --destination synology

Expected: Error: --status is required. Run with --help for usage.

  • Verify missing ENV var exits 0 with warning (ENV vars not yet set in this test):
env -u N8N_SWANSONCLOUD_URL notify_backup.rb --machine desktop --destination synology --status success

Expected: notify_backup: failed to send webhook: key not found: "N8N_SWANSONCLOUD_URL" and exit code 0:

echo $?   # must print 0

Task 7: Bootstrap — create datatable and verify Data Table node JSON shape

Both research agents flagged the Data Table node (n8n-nodes-base.dataTable) as a recent addition with uncertain parameter structure. This task pins the exact shape before generating workflow JSON.

  • Open https://n8n.swansoncloud.com. In the left sidebar find "Tables" (under Variables or Data). Create a new table named exactly backup_runs. Add string columns: timestamp, machine, destination, status, bytes_transferred, snapshot_id, log_path, notes. n8n auto-adds an id column.

  • Create a scratch workflow named _datatable-shape-probe:

    • Add a Manual Trigger node
    • Add a Data Table node, configure it: operation = Insert, table = backup_runs, map one dummy field
    • Connect Manual Trigger → Data Table
    • Add a second Data Table node: operation = Get Many Rows, table = backup_runs
    • Connect the Insert node → Get Many Rows
    • Save (do not activate)
  • Export the scratch workflow: click the menu → Download. Save as /tmp/datatable-probe.json.

  • Inspect the exported JSON to extract the exact node structure:

python3 -c "
import json, sys
wf = json.load(open('/tmp/datatable-probe.json'))
for n in wf['nodes']:
    if 'dataTable' in n.get('type','') or 'dataStore' in n.get('type',''):
        print(json.dumps(n, indent=2))
"

Note the type string (likely n8n-nodes-base.dataTable), typeVersion, and the exact keys inside parameters (especially the property wrapping the table reference and the filter shape).

  • Also note the datatable ID from the exported JSON (the value field inside the table picker). You will need this to replace REPLACE_WITH_DATATABLE_ID in Task 9.

  • If the type string or typeVersion differ from n8n-nodes-base.dataTable / 1, update the two constants at the top of scripts/build-n8n-workflows.rb (created in Task 9) before running it.

  • Delete the scratch workflow from n8n.


Task 8: Write JSON validation script

Files:

  • Create: scripts/validate-n8n-workflow.py

  • Create scripts/validate-n8n-workflow.py with this content (Python stdlib, no pip required):

#!/usr/bin/env python3
"""Validate an n8n workflow JSON file for structural correctness.

Usage: validate-n8n-workflow.py path/to/workflow.json
Exit codes: 0 = valid, 1 = invalid, 2 = usage error.
"""
import json
import sys
from pathlib import Path

KNOWN_MIN_TYPE_VERSIONS = {
    "n8n-nodes-base.code": 2,
    "n8n-nodes-base.httpRequest": 4.2,
    "n8n-nodes-base.webhook": 2,
    "n8n-nodes-base.set": 3.4,
    "n8n-nodes-base.scheduleTrigger": 1.2,
}

REQUIRED_TOP  = ["name", "nodes", "connections"]
REQUIRED_NODE = ["id", "name", "type", "typeVersion", "position", "parameters"]


def validate(workflow):
    errors, warnings = [], []
    if not isinstance(workflow, dict):
        return ["Root must be a JSON object."], []
    for key in REQUIRED_TOP:
        if key not in workflow:
            errors.append(f"Top-level: missing required field '{key}'.")
    if errors:
        return errors, warnings
    if not isinstance(workflow["nodes"], list):
        errors.append("Top-level 'nodes' must be an array.")
    if not isinstance(workflow["connections"], dict):
        errors.append("Top-level 'connections' must be an object.")
    if errors:
        return errors, warnings

    seen_names, seen_ids = set(), set()
    for i, node in enumerate(workflow["nodes"]):
        loc = f"nodes[{i}]"
        if not isinstance(node, dict):
            errors.append(f"{loc}: must be an object.")
            continue
        for key in REQUIRED_NODE:
            if key not in node:
                errors.append(f"{loc}: missing required field '{key}'.")
        name = node.get("name")
        if name:
            loc = f"nodes[{i}] '{name}'"
            if name in seen_names:
                errors.append(f"{loc}: duplicate node name.")
            seen_names.add(name)
        nid = node.get("id")
        if nid:
            if nid in seen_ids:
                errors.append(f"{loc}: duplicate node id '{nid}'.")
            seen_ids.add(nid)
        pos = node.get("position")
        if pos is not None and not (
            isinstance(pos, list) and len(pos) == 2
            and all(isinstance(v, (int, float)) for v in pos)
        ):
            errors.append(f"{loc}: 'position' must be [number, number].")
        if node.get("parameters") is not None and not isinstance(node["parameters"], dict):
            errors.append(f"{loc}: 'parameters' must be an object.")
        creds = node.get("credentials")
        if isinstance(creds, dict):
            for ctype, cred in creds.items():
                if not isinstance(cred, dict) or "name" not in cred:
                    errors.append(f"{loc}: credentials['{ctype}'] must have a 'name' field.")
        ntype, tv = node.get("type"), node.get("typeVersion")
        if isinstance(tv, (int, float)) and ntype in KNOWN_MIN_TYPE_VERSIONS:
            min_tv = KNOWN_MIN_TYPE_VERSIONS[ntype]
            if tv < min_tv:
                warnings.append(f"{loc}: typeVersion {tv} for '{ntype}' is below recommended {min_tv}.")

    for src, outputs in workflow["connections"].items():
        if src not in seen_names:
            errors.append(f"connections: source '{src}' does not match any node name.")
        if not isinstance(outputs, dict):
            errors.append(f"connections['{src}']: must be an object.")
            continue
        for out_type, out_list in outputs.items():
            if not isinstance(out_list, list):
                continue
            for oi, branch in enumerate(out_list):
                if branch is None:
                    continue
                if not isinstance(branch, list):
                    errors.append(f"connections['{src}']['{out_type}'][{oi}]: must be an array.")
                    continue
                for ti, target in enumerate(branch):
                    if not isinstance(target, dict):
                        errors.append(f"connections['{src}']['{out_type}'][{oi}][{ti}]: must be an object.")
                        continue
                    for k in ("node", "type", "index"):
                        if k not in target:
                            errors.append(f"connections['{src}']['{out_type}'][{oi}][{ti}]: missing '{k}'.")
                    tnode = target.get("node")
                    if tnode and tnode not in seen_names:
                        errors.append(f"connections['{src}']['{out_type}'][{oi}][{ti}]: target '{tnode}' not found.")
    return errors, warnings


def main():
    if len(sys.argv) != 2:
        print("Usage: validate-n8n-workflow.py <workflow.json>", file=sys.stderr)
        sys.exit(2)
    path = Path(sys.argv[1])
    try:
        workflow = json.loads(path.read_text())
    except FileNotFoundError:
        print(f"File not found: {path}", file=sys.stderr)
        sys.exit(2)
    except json.JSONDecodeError as e:
        print(f"Invalid JSON: {e}", file=sys.stderr)
        sys.exit(1)
    errors, warnings = validate(workflow)
    for w in warnings:
        print(f"WARN: {w}")
    for e in errors:
        print(f"ERROR: {e}", file=sys.stderr)
    if errors:
        print(f"\n{len(errors)} error(s), {len(warnings)} warning(s).", file=sys.stderr)
        sys.exit(1)
    print(f"OK — {len(workflow['nodes'])} nodes, {len(workflow['connections'])} connections, {len(warnings)} warning(s).")


if __name__ == "__main__":
    main()
  • Make it executable and test it on a known-bad payload:
chmod +x /home/jared/systems-admin/scripts/validate-n8n-workflow.py
echo '{"name":"test","nodes":[],"connections":{"missing-node":{"main":[[]]}}}' \
  | python3 /home/jared/systems-admin/scripts/validate-n8n-workflow.py /dev/stdin

Expected: ERROR: connections: source 'missing-node' does not match any node name.

  • Commit:
cd /home/jared/systems-admin
git add scripts/validate-n8n-workflow.py
git commit -m "feat(backup-monitor): add n8n workflow JSON validator"

Task 9: Write JS code files, build script, generate and validate workflow JSONs

Files:

  • Create: scripts/n8n-evaluate-sources.js
  • Create: scripts/n8n-build-email.js
  • Create: scripts/build-n8n-workflows.rb
  • Generated: scripts/n8n-workflow-receiver.json
  • Generated: scripts/n8n-workflow-reporter.json

The build script uses Ruby's JSON.generate to handle all escaping automatically — no manual string escaping needed for the Code node JS.

  • Create scripts/n8n-evaluate-sources.js:
// Runs once for all items. References upstream nodes by exact name.
const sources = $('Set sources').first().json.sources;
const runs = $('Get backup_runs (last 7d)').all().map(item => item.json);
const weekAgo = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000);

const results = sources.map(source => {
  const sourceRuns = runs.filter(r =>
    r.machine === source.machine &&
    r.destination === source.destination &&
    new Date(r.timestamp) >= weekAgo
  );
  const hasSuccess = sourceRuns.some(r => r.status === 'success');
  const hasAnyRun = sourceRuns.length > 0;
  const status = hasSuccess ? 'green' : hasAnyRun ? 'yellow' : 'red';
  const lastSuccess = sourceRuns
    .filter(r => r.status === 'success')
    .sort((a, b) => new Date(b.timestamp) - new Date(a.timestamp))[0] || null;
  return { ...source, status, runs: sourceRuns, lastSuccess };
});

const overallStatus = results.some(r => r.status === 'red') ? 'red'
  : results.some(r => r.status === 'yellow') ? 'yellow' : 'green';

return [{ json: { results, overallStatus } }];
  • Create scripts/n8n-build-email.js:
// Builds subject + body from Evaluate Sources output.
// Uses an array of lines joined at the end to avoid \n escape issues in JSON storage.
const { results, overallStatus } = $input.first().json;
const icon  = { green: '✓', yellow: '⚠', red: '✗' };
const label = { green: 'Backups healthy', yellow: 'Backup warning', red: 'Backup failure' };
const weekOf = new Date().toLocaleDateString('en-US', { month: 'short', day: 'numeric' });
const subject = icon[overallStatus] + ' ' + label[overallStatus] + ' — week of ' + weekOf;

const fmt = ts => ts
  ? new Date(ts).toLocaleString('en-US', { month: 'short', day: 'numeric', hour: '2-digit', minute: '2-digit' })
  : 'unknown';

const L = [];

if (overallStatus === 'green') {
  L.push('All backup sources ran successfully this week.');
  L.push('');
  results.forEach(r =>
    L.push('✓  ' + r.machine + ' / ' + r.destination +
      '  (last success: ' + fmt(r.lastSuccess && r.lastSuccess.timestamp) + ')')
  );
} else {
  const problems = results.filter(r => r.status !== 'green');
  const healthy  = results.filter(r => r.status === 'green');

  L.push(problems.length + ' source(s) need attention:');
  L.push('');

  problems.forEach(r => {
    L.push(icon[r.status] + '  ' + r.machine + ' / ' + r.destination);

    if (r.status === 'yellow') {
      L.push('   Ran this week but never succeeded.');
      r.runs.slice(0, 3).forEach(run => {
        var line = '   ' + fmt(run.timestamp) + '  [' + run.status + ']';
        if (run.log_path) line += '  log: ' + run.log_path;
        L.push(line);
        if (run.notes) L.push('   notes: ' + (run.notes || '').slice(0, 300));
      });
      L.push('');
      L.push('   Next steps:');
      L.push('   1. Check the log at the path shown above');
      L.push('   2. For Synology failures: verify LAN reachability (ssh synology) — disks need ~30s to spin up');
      L.push('   3. Re-run manually:  bash ~/.local/bin/backup.sh');
      L.push('   4. One-off skip (power outage, travel) is expected — yellow is a heads-up, not an alarm');
      L.push('');
    }

    if (r.status === 'red') {
      L.push('   No runs reported all week — silent failure.');
      if (r.lastSuccess) {
        L.push('   Last known good run: ' + fmt(r.lastSuccess.timestamp));
        if (r.lastSuccess.log_path) L.push('   Last log: ' + r.lastSuccess.log_path);
      } else {
        L.push('   No history found in the datatable.');
      }
      L.push('');
      L.push('   Next steps:');
      L.push('   1. Check systemd timer:  systemctl --user status restic-backup.timer');
      L.push('   2. Verify backup.sh calls notify_backup.rb at the end');
      L.push('   3. Check n8n Workflow 1 execution log for webhook errors');
      L.push('   4. Run manually:  bash ~/.local/bin/backup.sh');
      L.push('');
    }
  });

  if (healthy.length > 0) {
    L.push('Healthy sources:');
    healthy.forEach(r => L.push('✓  ' + r.machine + ' / ' + r.destination));
  }
}

return [{ json: { subject, body: L.join('\n') } }];
  • Create scripts/build-n8n-workflows.rb:
#!/usr/bin/env ruby
# frozen_string_literal: true
#
# Generates importable n8n workflow JSON from component pieces.
# Run: ruby scripts/build-n8n-workflows.rb
# Output: scripts/n8n-workflow-receiver.json
#         scripts/n8n-workflow-reporter.json
#
# Before running:
#   1. Complete Task 7 and verify DATATABLE_TYPE / DATATABLE_TYPE_VERSION below.
#   2. After Task 10 (credentials set up), replace REPLACE_WITH_*_CREDENTIAL_ID
#      in the generated JSON and re-run to regenerate clean files before import.

require 'json'
require 'securerandom'

SCRIPTS_DIR = File.dirname(__FILE__)

# Verify against the exported scratch-workflow JSON from Task 7.
DATATABLE_TYPE         = 'n8n-nodes-base.dataTable'
DATATABLE_TYPE_VERSION = 1

evaluate_sources_js = File.read(File.join(SCRIPTS_DIR, 'n8n-evaluate-sources.js'))
build_email_js      = File.read(File.join(SCRIPTS_DIR, 'n8n-build-email.js'))

# ── Workflow 1: Receiver ──────────────────────────────────────────────────────

receiver = {
  name: 'Backup Monitor — Receiver',
  nodes: [
    {
      parameters: {
        httpMethod: 'POST',
        path: 'backup-notify',
        authentication: 'headerAuth',
        responseMode: 'responseNode',
        options: {}
      },
      id: SecureRandom.uuid,
      name: 'Webhook',
      type: 'n8n-nodes-base.webhook',
      typeVersion: 2,
      position: [240, 300],
      webhookId: SecureRandom.uuid,
      credentials: {
        httpHeaderAuth: {
          id: 'REPLACE_WITH_BACKUP_CREDENTIAL_ID',
          name: 'backup-monitor-token'
        }
      }
    },
    {
      parameters: {
        resource: 'row',
        operation: 'insert',
        dataTableId: {
          '__rl' => true,
          mode: 'list',
          value: 'REPLACE_WITH_DATATABLE_ID',
          cachedResultName: 'backup_runs'
        },
        columns: {
          mappingMode: 'defineBelow',
          value: {
            timestamp:         '={{ $json.body.timestamp }}',
            machine:           '={{ $json.body.machine }}',
            destination:       '={{ $json.body.destination }}',
            status:            '={{ $json.body.status }}',
            bytes_transferred: '={{ $json.body.bytes_transferred }}',
            snapshot_id:       '={{ $json.body.snapshot_id }}',
            log_path:          '={{ $json.body.log_path }}',
            notes:             '={{ $json.body.notes }}'
          }
        }
      },
      id: SecureRandom.uuid,
      name: 'Insert to backup_runs',
      type: DATATABLE_TYPE,
      typeVersion: DATATABLE_TYPE_VERSION,
      position: [520, 300]
    },
    {
      parameters: {
        respondWith: 'json',
        responseBody: '={{ JSON.stringify({ ok: true }) }}',
        options: { responseCode: 200 }
      },
      id: SecureRandom.uuid,
      name: 'Respond 200',
      type: 'n8n-nodes-base.respondToWebhook',
      typeVersion: 1.1,
      position: [800, 300]
    }
  ],
  connections: {
    'Webhook'               => { main: [[{ node: 'Insert to backup_runs', type: 'main', index: 0 }]] },
    'Insert to backup_runs' => { main: [[{ node: 'Respond 200',           type: 'main', index: 0 }]] }
  },
  active: false,
  settings: { executionOrder: 'v1' },
  pinData: {},
  meta: { templateCredsSetupCompleted: true },
  tags: []
}

# ── Workflow 2: Reporter ──────────────────────────────────────────────────────

reporter = {
  name: 'Backup Monitor — Reporter',
  nodes: [
    {
      parameters: {
        rule: {
          interval: [
            {
              field: 'weeks',
              weeksInterval: 1,
              triggerAtDay: [5],
              triggerAtHour: 8,
              triggerAtMinute: 0
            }
          ]
        }
      },
      id: SecureRandom.uuid,
      name: 'Every Friday 8 AM',
      type: 'n8n-nodes-base.scheduleTrigger',
      typeVersion: 1.2,
      position: [240, 300]
    },
    {
      parameters: {
        mode: 'manual',
        includeOtherFields: false,
        assignments: {
          assignments: [
            {
              id: SecureRandom.uuid,
              name: 'sources',
              value: '={{ [{"machine":"desktop","destination":"synology"},{"machine":"desktop","destination":"backblaze-b2"},{"machine":"vps-ovh-prod-01","destination":"local"}] }}',
              type: 'arrayValue'
            }
          ]
        },
        options: {}
      },
      id: SecureRandom.uuid,
      name: 'Set sources',
      type: 'n8n-nodes-base.set',
      typeVersion: 3.4,
      position: [460, 300]
    },
    {
      parameters: {
        resource: 'row',
        operation: 'getMany',
        dataTableId: {
          '__rl' => true,
          mode: 'list',
          value: 'REPLACE_WITH_DATATABLE_ID',
          cachedResultName: 'backup_runs'
        },
        filters: {
          conditions: [
            {
              keyName: 'timestamp',
              condition: 'gt',
              keyValue: '={{ $now.minus({days: 7}).toISO() }}'
            }
          ]
        },
        options: {}
      },
      id: SecureRandom.uuid,
      name: 'Get backup_runs (last 7d)',
      type: DATATABLE_TYPE,
      typeVersion: DATATABLE_TYPE_VERSION,
      position: [680, 300]
    },
    {
      parameters: {
        mode: 'runOnceForAllItems',
        language: 'javaScript',
        jsCode: evaluate_sources_js
      },
      id: SecureRandom.uuid,
      name: 'Evaluate Sources',
      type: 'n8n-nodes-base.code',
      typeVersion: 2,
      position: [900, 300]
    },
    {
      parameters: {
        mode: 'runOnceForAllItems',
        language: 'javaScript',
        jsCode: build_email_js
      },
      id: SecureRandom.uuid,
      name: 'Build Email',
      type: 'n8n-nodes-base.code',
      typeVersion: 2,
      position: [1120, 300]
    },
    {
      parameters: {
        method: 'POST',
        url: 'https://app.mailpace.com/api/v1/send',
        authentication: 'genericCredentialType',
        genericAuthType: 'httpHeaderAuth',
        sendBody: true,
        specifyBody: 'json',
        # Expression returning a JS object; n8n serializes it to JSON automatically.
        # Replace REPLACE_WITH_MAILPACE_FROM_ADDRESS with your verified MailPace sending address.
        jsonBody: '={{ { from: "REPLACE_WITH_MAILPACE_FROM_ADDRESS", to: "jaredmswanson@gmail.com", subject: $json.subject, text_body: $json.body } }}',
        options: {}
      },
      id: SecureRandom.uuid,
      name: 'Send via MailPace',
      type: 'n8n-nodes-base.httpRequest',
      typeVersion: 4.2,
      position: [1340, 300],
      credentials: {
        httpHeaderAuth: {
          id: 'REPLACE_WITH_MAILPACE_CREDENTIAL_ID',
          name: 'mailpace-token'
        }
      }
    }
  ],
  connections: {
    'Every Friday 8 AM'         => { main: [[{ node: 'Set sources',               type: 'main', index: 0 }]] },
    'Set sources'               => { main: [[{ node: 'Get backup_runs (last 7d)', type: 'main', index: 0 }]] },
    'Get backup_runs (last 7d)' => { main: [[{ node: 'Evaluate Sources',          type: 'main', index: 0 }]] },
    'Evaluate Sources'          => { main: [[{ node: 'Build Email',               type: 'main', index: 0 }]] },
    'Build Email'               => { main: [[{ node: 'Send via MailPace',         type: 'main', index: 0 }]] }
  },
  active: false,
  settings: { executionOrder: 'v1' },
  pinData: {},
  meta: { templateCredsSetupCompleted: true },
  tags: []
}

File.write(File.join(SCRIPTS_DIR, 'n8n-workflow-receiver.json'), JSON.pretty_generate(receiver))
File.write(File.join(SCRIPTS_DIR, 'n8n-workflow-reporter.json'), JSON.pretty_generate(reporter))

puts 'Generated:'
puts "  #{SCRIPTS_DIR}/n8n-workflow-receiver.json"
puts "  #{SCRIPTS_DIR}/n8n-workflow-reporter.json"
  • Run the build script:
cd /home/jared/systems-admin
ruby scripts/build-n8n-workflows.rb

Expected: two files created with no error output

  • Validate both generated files:
python3 scripts/validate-n8n-workflow.py scripts/n8n-workflow-receiver.json
python3 scripts/validate-n8n-workflow.py scripts/n8n-workflow-reporter.json

Expected: both print OK — N nodes, N connections, 0 warning(s).

If warnings appear about typeVersion, update the constants in build-n8n-workflows.rb to match what Task 7 found, re-run, and re-validate.

  • Commit:
git add scripts/n8n-evaluate-sources.js scripts/n8n-build-email.js \
        scripts/build-n8n-workflows.rb \
        scripts/n8n-workflow-receiver.json scripts/n8n-workflow-reporter.json
git commit -m "feat(backup-monitor): add workflow JS files, build script, and generated workflow JSON"

Task 10: Set up credentials and import workflows via n8n API

  • Create the backup-monitor credential in n8n UI:

    • Credentials → New → Header Auth
    • Name: backup-monitor-token
    • Header Name field: Authorization
    • Header Value field: Bearer <value of N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN>
    • Save
  • Create the MailPace credential in n8n UI:

    • Credentials → New → Header Auth
    • Name: mailpace-token
    • Header Name field: X-Server-Token
    • Header Value field: your MailPace server API token (find at app.mailpace.com → Server Settings)
    • Save
  • Get credential IDs from the n8n API:

source ~/.credentials
curl -s "https://n8n.swansoncloud.com/api/v1/credentials" \
  -H "X-N8N-API-KEY: $N8N_SWANSONCLOUD_API_KEY" | python3 -m json.tool | grep -E '"id"|"name"'

Note the numeric id for backup-monitor-token and mailpace-token.

  • Fill in the credential IDs and datatable ID. Edit scripts/build-n8n-workflows.rb:

    • Replace REPLACE_WITH_BACKUP_CREDENTIAL_ID with the backup-monitor-token ID
    • Replace REPLACE_WITH_MAILPACE_CREDENTIAL_ID with the mailpace-token ID
    • Replace both REPLACE_WITH_DATATABLE_ID occurrences with the datatable ID from Task 7
    • Replace REPLACE_WITH_MAILPACE_FROM_ADDRESS with your verified MailPace sending address
  • Regenerate the workflow JSON files:

ruby scripts/build-n8n-workflows.rb
  • Validate again after substitution:
python3 scripts/validate-n8n-workflow.py scripts/n8n-workflow-receiver.json
python3 scripts/validate-n8n-workflow.py scripts/n8n-workflow-reporter.json

Expected: both OK

  • Import Workflow 1 via the n8n API:
source ~/.credentials
curl -s -X POST "https://n8n.swansoncloud.com/api/v1/workflows" \
  -H "X-N8N-API-KEY: $N8N_SWANSONCLOUD_API_KEY" \
  -H "Content-Type: application/json" \
  -d @scripts/n8n-workflow-receiver.json | python3 -m json.tool | grep '"id"' | head -1

Note the workflow id returned (e.g. 42).

  • Activate Workflow 1:
curl -s -X PATCH "https://n8n.swansoncloud.com/api/v1/workflows/WORKFLOW_1_ID" \
  -H "X-N8N-API-KEY: $N8N_SWANSONCLOUD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"active": true}' | python3 -m json.tool | grep '"active"'

Expected: "active": true

  • Import Workflow 2:
curl -s -X POST "https://n8n.swansoncloud.com/api/v1/workflows" \
  -H "X-N8N-API-KEY: $N8N_SWANSONCLOUD_API_KEY" \
  -H "Content-Type: application/json" \
  -d @scripts/n8n-workflow-reporter.json | python3 -m json.tool | grep '"id"' | head -1
  • Activate Workflow 2:
curl -s -X PATCH "https://n8n.swansoncloud.com/api/v1/workflows/WORKFLOW_2_ID" \
  -H "X-N8N-API-KEY: $N8N_SWANSONCLOUD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"active": true}' | python3 -m json.tool | grep '"active"'
source ~/.credentials
notify_backup.rb \
  --machine desktop \
  --destination test \
  --status success \
  --notes "smoke test $(date)"

Expected: no error output. Verify a row appears in the backup_runs table in n8n, then delete it.

  • Commit the final build script with real IDs:
cd /home/jared/systems-admin
git add scripts/build-n8n-workflows.rb scripts/n8n-workflow-receiver.json scripts/n8n-workflow-reporter.json
git commit -m "feat(backup-monitor): fill credential/datatable IDs and reimport workflows"

Task 11: Integrate notify_backup.rb into desktop ~/.local/bin/backup.sh

The existing backup.sh uses set -euo pipefail. We add a helper function that temporarily disables set -e to capture the exit code of run_backup, then always re-enables it, then sends the notification.

File: ~/.local/bin/backup.sh

Current structure for reference (key lines):

  • Line 3: set -euo pipefail

  • Line 16: run_backup() function definition

  • Line 35: Synology conditional block

  • Line 4345: B2 unlock + run_backup

  • Add the run_and_notify helper function to ~/.local/bin/backup.sh immediately after the existing run_backup function (after line ~32):

run_and_notify() {
  local repo="$1"
  local label="$2"
  local machine="$3"
  local destination="$4"

  set +e
  run_backup "$repo" "$label"
  local exit_code=$?
  set -e

  local status="failure"
  [[ $exit_code -eq 0 ]] && status="success"

  local snap
  snap=$(grep -oE 'snapshot [a-f0-9]+' "$LOG" 2>/dev/null | tail -1 | awk '{print $2}')

  notify_backup.rb \
    --machine "$machine" \
    --destination "$destination" \
    --status "$status" \
    ${snap:+--snapshot "$snap"} \
    --log-path "$LOG" \
    --notes "$(tail -10 "$LOG")" || true

  return $exit_code
}
  • Replace the Synology block (lines ~3540) with:
# Synology: attempt only if reachable (SSH key auth, 30s timeout for HDD spin-up)
if ssh -o ConnectTimeout=30 -o BatchMode=yes synology true 2>/dev/null; then
  run_and_notify "sftp:${SYNOLOGY_HOST}:${SYNOLOGY_PATH}" "Synology" "desktop" "synology"
else
  echo "Synology unreachable — skipping SFTP backup" | tee -a "$LOG"
  notify_backup.rb \
    --machine desktop \
    --destination synology \
    --status skipped \
    --log-path "$LOG" \
    --notes "Synology unreachable at $(date)" || true
fi
  • Replace the B2 lines (~4345) with:
# B2: clear any stale lock before attempting (safe no-op if no lock exists)
restic -r "b2:${B2_BUCKET}:" unlock --remove-all 2>&1 | tee -a "$LOG"
run_and_notify "b2:${B2_BUCKET}:" "Backblaze B2" "desktop" "backblaze-b2"
  • Verify the final file looks correct (no duplicate lines, indentation intact):
cat -n ~/.local/bin/backup.sh
  • Run a dry-run to confirm the script is syntactically valid:
bash -n ~/.local/bin/backup.sh

Expected: no output (no syntax errors)

  • Copy the updated backup.sh back to the source in this repo so it stays tracked:
cp ~/.local/bin/backup.sh /home/jared/systems-admin/scripts/backup.sh
  • Commit:
cd /home/jared/systems-admin
git add scripts/backup.sh
git commit -m "feat(backup-monitor): integrate notify_backup.rb into desktop backup.sh"

Task 12: Deploy and integrate on OVH VPS

The VPS backup script is at ~/services/backup.sh on 15.204.247.153. The notifier script and ENV vars must be deployed there too.

  • Check if Ruby is available on the VPS:
ssh jared@15.204.247.153 'ruby --version'

If Ruby is not installed:

ssh jared@15.204.247.153 'sudo apt-get install -y ruby'
  • Copy notify_backup.rb to the VPS:
scp ~/.local/bin/notify_backup.rb jared@15.204.247.153:~/.local/bin/notify_backup.rb
ssh jared@15.204.247.153 'chmod +x ~/.local/bin/notify_backup.rb'
  • Add the ENV vars to the VPS credentials file. SSH to the server and edit ~/.credentials:
ssh jared@15.204.247.153

Then in the SSH session, append to ~/.credentials:

export N8N_SWANSONCLOUD_URL="https://n8n.swansoncloud.com"
export N8N_SWANSONCLOUD_BACKUP_AUTH_TOKEN="<same token value as on desktop>"

Then source it:

source ~/.credentials
  • Verify the notifier works from the VPS (still in the SSH session):
notify_backup.rb --machine vps-ovh-prod-01 --destination local --status success --notes "manual test $(date)"

Expected: no error output

  • Verify the test row appeared in the n8n backup_runs datatable, then delete it.

  • Add the notify call to ~/services/backup.sh on the VPS. Open the file:

nano ~/services/backup.sh

Find the final success line (likely an echo at the end). Before it, add:

source "$HOME/.credentials"
SNAP=$(grep -oE 'snapshot [a-f0-9]+' "${LOG_FILE}" 2>/dev/null | tail -1 | awk '{print $2}')
notify_backup.rb \
  --machine vps-ovh-prod-01 \
  --destination local \
  --status success \
  --log-path "${LOG_FILE}" \
  ${SNAP:+--snapshot "$SNAP"} \
  --notes "$(tail -10 "${LOG_FILE}")" || true

Note: The VPS backup.sh uses LOG_FILE (not LOG) — confirm the variable name before saving.

  • Verify syntax:
bash -n ~/services/backup.sh

Expected: no output

  • Exit the SSH session. Back on desktop, commit the documentation of this change:
cd /home/jared/systems-admin
git commit --allow-empty -m "feat(backup-monitor): deploy notify_backup.rb to OVH VPS and integrate into backup.sh"

Task 13: End-to-end smoke test — COMPLETE (2026-05-07)

  • Triggered desktop backup — Synology succeeded (snapshot f5f25c53), B2 had stale lock from 2026-04-13 (PID 43407) causing restic forget to fail (exit 11). Both runs notified; rows recorded in datatable.
  • Datatable confirmed — 3 rows visible in n8n UI after backup run.
  • Reporter triggered manually — MailPace queued (id: 35621228, status: queued). Email sent to jaredmswanson@gmail.com.
  • Key bug fixed: MailPace body field must be textbody (not text_body). Required explicit Accept: application/json and Content-Type: application/json headers in n8n HTTP Request node.

Remaining known issues (not blocking):

  • Stale restic locks on Synology SFTP and B2 repos (from 2026-04-13) — restic unlock returns 0 but lock persists; restic forget fails, causing all runs to report status: failure even when snapshots save successfully. Run restic -r <repo> unlock manually or investigate lock file directly.
  • MailPace credential header: working config uses MailPace-Server-Token (not X-Server-Token as noted in Self-Review). Credential named mailpace-token in n8n (ID: tC1vdhAHGVwAZxcg).

Self-Review Notes

  • Spec coverage: Ruby script (Tasks 16) ✓ · Datatable bootstrap (Task 7) ✓ · Validator (Task 8) ✓ · Workflow JSON generation (Task 9) ✓ · Credential setup + import (Task 10) ✓ · Desktop integration (Task 11) ✓ · VPS integration (Task 12) ✓ · Email format green/yellow/red (Task 9) ✓ · Auth token from ENV (Tasks 910) ✓ · log_path field (Tasks 23, 11) ✓ · skipped status (Task 11) ✓
  • Data Table node uncertainty: Both research agents flagged the Data Table node type as uncertain. Task 7 resolves this by exporting a probe workflow. If the type or parameter shape differs from n8n-nodes-base.dataTable, update the constants in build-n8n-workflows.rb and regenerate before import.
  • MailPace credential header: Must be X-Server-Token (not Authorization or MailPace-Server-Token). This is set in Task 10 when creating the mailpace-token Header Auth credential.
  • MailPace from address: Must match a verified sending domain in your MailPace server. Confirm at app.mailpace.com before filling in build-n8n-workflows.rb.
  • backup.sh variable name: The VPS script uses LOG_FILE not LOG — Task 12 calls this out explicitly.
  • Node name consistency: n8n-evaluate-sources.js references $('Set sources') and $('Get backup_runs (last 7d)') — these must match the name fields in build-n8n-workflows.rb exactly (they do as written).