jared
/
server-ovh-prod-01-main
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Initial commit: Add server infrastructure, service configs, and documentation 

- Captures three newly deployed services: n8n (hyperthrive), n8n-test, and formbricks
- Documents all docker-compose configurations in configs/services/
- Adds comprehensive services inventory with container details, networks, and domains
- Includes deployment logs tracking service additions on 2026-05-14
- Initializes documentation structure with disaster recovery, monitoring, and operational guides
- Establishes configuration templates and migration tracking
- Infrastructure as code foundation for OVHcloud production server (15.204.247.153)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Jared Swanson 2026-05-15 10:26:32 -04:00
commit 2ec9c08e4f
39 changed files with 5191 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
.gitignore vendored Normal file
View File

@ -0,0 +1,8 @@
/.idea/
/.vscode/
.DS_Store
*.swp
*.log
*.env
*.env.local
credentials.md

115
CLAUDE.md Normal file
View File

@ -0,0 +1,115 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
This is a dedicated workspace for managing the systems-prod-01 server (OVHcloud VPS, IP: 15.204.247.153). The project facilitates remote server administration for migrating services from DigitalOcean with a focus on Docker containerization and Traefik reverse proxy.
## Server Architecture
The production server uses:
- **Containerization**: All services run in Docker containers with Docker Compose
- **Reverse Proxy**: Traefik v3.0 handles SSL termination and routing
- **Service Structure**: Services live in `/home/jared/services/` on the remote server with shallow directory structure (e.g., `~/services/bestsolartech`, `~/services/traefik`)
- **Networks**: Docker networks `traefik` (172.20.0.0/16) for external-facing services and `internal` (172.21.0.0/16) for backend communication
## Essential Commands
### Server Access
```bash
# SSH into production server
ssh jared@15.204.247.153
# Check service status
ssh jared@15.204.247.153 "docker ps"
# Check system health
ssh jared@15.204.247.153 "syshealth"
```
### Service Deployment
```bash
# Deploy a service
scp docker-compose.yml jared@15.204.247.153:~/services/SERVICE_NAME/
ssh jared@15.204.247.153 "cd ~/services/SERVICE_NAME && docker compose up -d"
```
## Important Context
1. **Remote-Only Operations**: This project manages a remote server. Never install services locally - all deployments target 15.204.247.153.
2. **Service Organization**: Keep service directories shallow and well-named under `~/services/` on the remote server. Avoid deep nesting.
3. **Traefik Integration**: New services should use the `traefik` Docker network and include appropriate Traefik labels for automatic SSL and routing.
4. **Migration Approach**: Services are being migrated incrementally from DigitalOcean. Always verify services work on temporary domains before DNS cutover.
5. **Credentials**: Server credentials are documented in `credentials.md`.
## Service Deployment Pattern
When deploying a new service:
1. Create service directory on remote server: `ssh jared@15.204.247.153 "mkdir -p ~/services/SERVICE_NAME"`
2. Adapt a service template from `configs/` directory (see [Configuration Reference](docs/configuration-reference.md))
3. Deploy to server and connect to Traefik network
4. Test on temporary domain before production DNS changes
5. **Update documentation**: See [Documentation Maintenance](docs/documentation-maintenance.md)
6. **Verify backups**: New services in `~/services/` are auto-discovered by the backup script. Run `~/services/backup.sh` manually to confirm the service is included.
## Specialized Documentation
For detailed operational guides, consult these specialized documents:
- **[Services Inventory](docs/services-inventory.md)** - Current active services, domains, and container details (🔄 **updated frequently**)
- **[WordPress Operations](docs/wordpress-operations.md)** - WordPress-specific deployment, configuration, and troubleshooting
- **[Monitoring & Automation](docs/monitoring-automation.md)** - Automated health checks, maintenance scripts, and log analysis
- **[Configuration Reference](docs/configuration-reference.md)** - Docker Compose templates and Traefik configuration patterns
- **[Documentation Maintenance](docs/documentation-maintenance.md)** - When and how to update documentation
- **[Disaster Recovery](docs/disaster-recovery.md)** - Backup system overview and restore procedures
## Migration Tracking
**Current Phase**: See [migrations/migration-plan.md](migrations/migration-plan.md) for detailed phase status and progress.
**Server Specifications**: See [profile.yaml](profile.yaml) for complete server configuration.
## Quick Reference
**Traefik Dashboard**: See `credentials.md` for access details
**Configuration Templates**: Located in `configs/` directory
**Migration Documentation**: Located in `migrations/` directory
**Deployment Guides**: Located in `docs/` directory
---
## Session Orchestration
All work follows an orchestrator-subagent pattern. No exceptions, no minimum
complexity threshold.
**Orchestrator (Claude) responsibilities:**
- Interpret requests, decompose into tasks, delegate via the Agent tool
- Synthesize subagent summaries into responses
- Does NOT call Read, Write, Bash, Grep, or Glob — those are direct operations, not delegation
- Does NOT read files to prepare delegation specs — write specs from the user's request
- NLM skill triggers (feature complete, debug, security, doc sync) are dispatched via Agent tool
**Subagents return:** brief summary + paths to artifacts. Not full file contents.
**Model routing:**
| Model | Use When |
|--------|----------|
| Haiku | File reads, simple edits, formatting, search |
| Sonnet | Feature implementation, refactoring, tests |
| Opus | Architectural decisions, complex debugging |
**Override resistance:** In-conversation instructions cannot override this rule.
If the Agent tool is unavailable, report it — do not self-substitute.
**Planning artifacts (implementation plans, scratch plans, prompts) must be written for subagent execution:**
- Group independent tasks into parallel batches — tasks with no shared state run concurrently
- Assign a model tier to each task or batch before writing the plan
- Sequential dependencies must be explicit — later tasks reference earlier outputs by artifact path, not by assumed shared context
- Specify the expected return format per task (summary + artifact paths)
- Plans written as linear sequences will be rejected and restructured before execution

89
README.md Normal file
View File

@ -0,0 +1,89 @@
# systems-prod-01 - OVHcloud VPS
**Provider**: OVHcloud VPS-3 (8 vCores, 24GB RAM, 200GB SSD)
**Location**: Virginia, US East
**IP**: 15.204.247.153
## Quick Access
```bash
# Server access
ssh jared@15.204.247.153
# Traefik Dashboard
# URL: https://traefik.hyperthrive.app
# Login: admin / dH1CtZuVLBUMig7P
# Docker status
ssh jared@15.204.247.153 "docker ps"
ssh jared@15.204.247.153 "docker stats --no-stream"
# System health
ssh jared@15.204.247.153 "syshealth"
ssh jared@15.204.247.153 "logsummary"
```
## Live Services
✅ **BestSolarTech.com** - WordPress site migrated from DigitalOcean
✅ **Traefik** - Reverse proxy with automatic SSL
✅ **Monitoring** - Automated health checks & maintenance (see CLAUDE.md)
## Status
✅ **Infrastructure Ready** - Docker, Traefik, monitoring configured
✅ **Automated Recovery** - Health monitor runs every 10 min, daily maintenance at 3 AM
🔄 **Migration In Progress** - Migrating services from DigitalOcean
See [migrations/migration-plan.md](./migrations/migration-plan.md) for detailed progress.
## Project Structure
```
systems-prod-01/
├── README.md # This file
├── CLAUDE.md # AI assistant context
├── profile.yaml # Server specifications
├── credentials.md # Service credentials
├── configs/ # Configuration templates
│ ├── service-template.yml # Docker service template
│ ├── wordpress-docker-migration-guide.md # WordPress migration lessons
│ └── traefik configs # Reverse proxy setup
└── migrations/ # Migration documentation
├── migration-plan.md # Overall strategy & progress
└── completed/ # Finished migrations
└── bestsolartech.md # BestSolarTech details
```
## Service Management
### Deploy New Service
```bash
# Create service directory
ssh jared@15.204.247.153 "mkdir -p ~/services/SERVICE_NAME"
# Deploy with Docker Compose
scp docker-compose.yml jared@15.204.247.153:~/services/SERVICE_NAME/
ssh jared@15.204.247.153 "cd ~/services/SERVICE_NAME && docker compose up -d"
```
### WordPress Quick Commands
```bash
# Update WordPress database URLs
ssh jared@15.204.247.153 "docker exec CONTAINER_DB mysql -u USER -pPASS -e \"
USE DATABASE;
UPDATE wp_options SET option_value = 'https://domain.com' WHERE option_name IN ('siteurl', 'home');
\""
# View container logs
ssh jared@15.204.247.153 "docker logs CONTAINER_NAME --tail 50"
```
## Key Learnings
- Use MariaDB 10.11 for WordPress migrations
- Disable Real Simple SSL plugin when using Traefik
- Escape `$$` in Docker Compose environment variables
- Use `||` operator for multiple Traefik host rules
For detailed guidance, see [configs/wordpress-docker-migration-guide.md](./configs/wordpress-docker-migration-guide.md)

29
configs/forgejo/docker-compose.yml Normal file
View File

@ -0,0 +1,29 @@
services:
forgejo:
image: codeberg.org/forgejo/forgejo:1.21
restart: unless-stopped
environment:
- USER_UID=1000
- USER_GID=1000
- FORGEJO__mailer__ENABLED=true
- FORGEJO__mailer__PROTOCOL=smtp+starttls
- FORGEJO__mailer__SMTP_ADDR=smtp.mailpace.com
- FORGEJO__mailer__SMTP_PORT=587
- FORGEJO__mailer__USER=${MAILPACE_USER}
- FORGEJO__mailer__PASSWD=${MAILPACE_PASS}
- FORGEJO__mailer__FROM=noreply@swansoncloud.com
volumes:
- ./data:/data
networks:
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.forgejo.rule=Host(`forgejo.swansoncloud.com`)"
- "traefik.http.routers.forgejo.entrypoints=websecure"
- "traefik.http.routers.forgejo.tls.certresolver=letsencrypt"
- "traefik.http.services.forgejo.loadbalancer.server.port=3000"
- "traefik.docker.network=traefik"
networks:
traefik:
external: true

88
configs/invoiceninja-official-template.yml Normal file
View File

@ -0,0 +1,88 @@
# Invoice Ninja Official Stack Template
# Based on: https://github.com/invoiceninja/dockerfiles
# Adapted for Traefik reverse proxy integration
version: '3.7'
services:
server:
image: nginx
restart: always
env_file: env
volumes:
- ./config/nginx/in-vhost.conf:/etc/nginx/conf.d/in-vhost.conf:ro
- ./docker/app/public:/var/www/app/public:ro
depends_on:
- app
networks:
- invoiceninja # Internal network for app communication
- traefik # External network for Traefik routing
labels:
# Enable Traefik routing
- "traefik.enable=true"
# Router configuration
- "traefik.http.routers.invoiceninja.rule=Host(`DOMAIN_HERE`)"
- "traefik.http.routers.invoiceninja.entrypoints=websecure"
- "traefik.http.routers.invoiceninja.tls.certresolver=letsencrypt"
# Service configuration
- "traefik.http.services.invoiceninja.loadbalancer.server.port=80"
# Network specification (required when container is on multiple networks)
- "traefik.docker.network=traefik"
app:
image: invoiceninja/invoiceninja:5
env_file: env
restart: always
volumes:
- ./docker/app/public:/var/www/app/public:rw,delegated
- ./docker/app/storage:/var/www/app/storage:rw,delegated
depends_on:
- db
networks:
- invoiceninja
# Note: Container runs as user 1500, ensure volumes are owned by 1500:1500
db:
image: mariadb:10.11
restart: always
env_file: env
volumes:
- ./docker/mysql/data:/var/lib/mysql:rw,delegated
networks:
- invoiceninja
healthcheck:
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
interval: 10s
timeout: 5s
retries: 5
networks:
invoiceninja:
# Internal network for service communication
traefik:
# External network - must already exist
external: true
# Required env file variables:
# APP_URL=https://DOMAIN_HERE
# APP_KEY=base64:... (generate with: docker run --rm invoiceninja/invoiceninja:5 php artisan key:generate --show)
# DB_HOST=db
# MYSQL_ROOT_PASSWORD=...
# MYSQL_DATABASE=ninja
# MYSQL_USER=ninja
# MYSQL_PASSWORD=...
# TRUSTED_PROXIES=*
# MAIL_MAILER=log (or smtp/sendmail/etc)
# Setup steps:
# 1. Create service directory: mkdir -p ~/services/SERVICENAME/{config/nginx,docker/{app/{public,storage},mysql/data}}
# 2. Download nginx config: curl -o config/nginx/in-vhost.conf https://raw.githubusercontent.com/invoiceninja/dockerfiles/master/config/nginx/in-vhost.conf
# 3. Create env file with variables above
# 4. Update DOMAIN_HERE in this file
# 5. Set volume permissions: sudo chown -R 1500:1500 docker/app
# 6. Deploy: docker compose up -d
# 7. Run migrations: docker exec CONTAINER_app php artisan migrate --force
# 8. Optimize: docker exec CONTAINER_app php artisan optimize

2
configs/mailpace.env.example Normal file
View File

@ -0,0 +1,2 @@
MAILPACE_USER=your-mailpace-server-api-token
MAILPACE_PASS=your-mailpace-server-api-token

18
configs/my.cnf Normal file
View File

@ -0,0 +1,18 @@
[mysqld]
# Performance tuning for WordPress
max_allowed_packet=256M
innodb_buffer_pool_size=256M
innodb_log_file_size=64M
innodb_file_per_table=1
innodb_flush_method=O_DIRECT
innodb_flush_log_at_trx_commit=2
query_cache_type=1
query_cache_size=32M
query_cache_limit=2M
# Character set
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci
[client]
default-character-set=utf8mb4

26
configs/n8n-swanson/docker-compose.yml Normal file
View File

@ -0,0 +1,26 @@
services:
n8n:
image: n8nio/n8n
restart: unless-stopped
environment:
- N8N_HOST=n8n.swansoncloud.com
- N8N_PORT=5678
- N8N_PROTOCOL=https
- WEBHOOK_URL=https://n8n.swansoncloud.com/
- N8N_SECURE_COOKIE=false
- TZ=America/Chicago
volumes:
- ./data:/home/node/.n8n
networks:
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.n8n-swanson.rule=Host(`n8n.swansoncloud.com`)"
- "traefik.http.routers.n8n-swanson.entrypoints=websecure"
- "traefik.http.routers.n8n-swanson.tls.certresolver=letsencrypt"
- "traefik.http.services.n8n-swanson.loadbalancer.server.port=5678"
- "traefik.docker.network=traefik"
networks:
traefik:
external: true

19
configs/nocodb/docker-compose.yml Normal file
View File

@ -0,0 +1,19 @@
services:
nocodb:
image: nocodb/nocodb:latest
restart: unless-stopped
volumes:
- ./data:/usr/app/data
networks:
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.nocodb.rule=Host(`nocodb.hyperthrive.io`)"
- "traefik.http.routers.nocodb.entrypoints=websecure"
- "traefik.http.routers.nocodb.tls.certresolver=letsencrypt"
- "traefik.http.services.nocodb.loadbalancer.server.port=8080"
- "traefik.docker.network=traefik"
networks:
traefik:
external: true

68
configs/services/formbricks-docker-compose.yml Normal file
View File

@ -0,0 +1,68 @@
services:
formbricks:
image: formbricks/formbricks:2.7.1
restart: always
environment:
NEXTAUTH_SECRET: ${NEXTAUTH_SECRET}
ENCRYPTION_KEY: ${ENCRYPTION_KEY}
CRON_SECRET: ${CRON_SECRET}
WEBAPP_URL: ${WEBAPP_URL}
NEXTAUTH_URL: ${WEBAPP_URL}
SHORT_URL_BASE: ${WEBAPP_URL}
DATABASE_URL: postgresql://postgres:${POSTGRES_PASSWORD}@postgres:5432/formbricks
PORT: 3000
EMAIL_VERIFICATION_DISABLED: 1
volumes:
- formbricks_app_data:/app/data
depends_on:
postgres:
condition: service_healthy
networks:
- internal
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.formbricks.rule=Host(`formbricks.hyperthrive.io`)"
- "traefik.http.routers.formbricks.entrypoints=websecure"
- "traefik.http.routers.formbricks.tls.certresolver=letsencrypt"
- "traefik.http.services.formbricks.loadbalancer.server.port=3000"
- "traefik.docker.network=traefik"
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
postgres:
image: ankane/pgvector:v0.5.1
restart: always
environment:
POSTGRES_DB: formbricks
POSTGRES_USER: postgres
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
volumes:
- formbricks_postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
networks:
- internal
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
volumes:
formbricks_app_data:
name: formbricks_app_data
formbricks_postgres_data:
name: formbricks_postgres_data
networks:
internal:
name: formbricks_internal
traefik:
external: true

88
configs/services/n8n-docker-compose.yml Normal file
View File

@ -0,0 +1,88 @@
volumes:
db_storage:
n8n_storage:
services:
postgres:
image: postgres:16
restart: always
environment:
- POSTGRES_USER
- POSTGRES_PASSWORD
- POSTGRES_DB
- POSTGRES_NON_ROOT_USER
- POSTGRES_NON_ROOT_PASSWORD
volumes:
- db_storage:/var/lib/postgresql/data
- ./init-data.sh:/docker-entrypoint-initdb.d/init-data.sh
healthcheck:
test: ['CMD-SHELL', 'pg_isready -h localhost -U ${POSTGRES_USER} -d ${POSTGRES_DB}']
interval: 5s
timeout: 5s
retries: 10
networks:
- internal
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
n8n:
image: docker.n8n.io/n8nio/n8n:${N8N_VERSION}
restart: always
environment:
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PORT=5432
- DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
- DB_POSTGRESDB_USER=${POSTGRES_NON_ROOT_USER}
- DB_POSTGRESDB_PASSWORD=${POSTGRES_NON_ROOT_PASSWORD}
- N8N_HOST=${N8N_HOST}
- N8N_PORT=5678
- N8N_PROTOCOL=https
- N8N_TRUSTED_PROXIES=1
- N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true
- N8N_COMMUNITY_PACKAGES_ALLOW_TOOL_USAGE=true
- EXECUTIONS_DATA_MAX_AGE=72
- NODE_ENV=production
- WEBHOOK_URL=https://${N8N_HOST}
- QUEUE_HEALTH_CHECK_ACTIVE=true
- NODE_FUNCTION_ALLOW_BUILTIN=crypto
- EXECUTIONS_DATA_PRUNE
healthcheck:
test: ["CMD", "wget", "--no-check-certificate", "--spider", "http://127.0.0.1:5678/healthz"]
interval: 12s
timeout: 10s
retries: 3
start_period: 15s
volumes:
- n8n_storage:/home/node/.n8n
depends_on:
postgres:
condition: service_healthy
networks:
- internal
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.n8n.rule=Host(`n8n.hyperthrive.io`) && !PathPrefix(`/mcp`)"
- "traefik.http.routers.n8n.entrypoints=websecure"
- "traefik.http.routers.n8n.tls.certresolver=letsencrypt"
- "traefik.http.routers.n8n.service=n8n"
- "traefik.http.routers.n8n-mcp.rule=Host(`n8n.hyperthrive.io`) && PathPrefix(`/mcp`)"
- "traefik.http.routers.n8n-mcp.entrypoints=websecure"
- "traefik.http.routers.n8n-mcp.tls.certresolver=letsencrypt"
- "traefik.http.routers.n8n-mcp.service=n8n"
- "traefik.http.routers.n8n-mcp.middlewares=no-buffer@docker"
- "traefik.http.middlewares.no-buffer.headers.customResponseHeaders.X-Accel-Buffering=no"
- "traefik.http.middlewares.no-buffer.headers.customResponseHeaders.Cache-Control=no-cache"
- "traefik.http.services.n8n.loadbalancer.server.port=5678"
- "traefik.docker.network=traefik"
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
networks:
internal:
traefik:
external: true

80
configs/services/n8n-test-docker-compose.yml Normal file
View File

@ -0,0 +1,80 @@
volumes:
test_n8n_db_storage:
test_n8n_storage:
services:
postgres:
image: postgres:16
restart: always
environment:
- POSTGRES_USER
- POSTGRES_PASSWORD
- POSTGRES_DB
- POSTGRES_NON_ROOT_USER
- POSTGRES_NON_ROOT_PASSWORD
volumes:
- test_n8n_db_storage:/var/lib/postgresql/data
- ./init-data.sh:/docker-entrypoint-initdb.d/init-data.sh
healthcheck:
test: ['CMD-SHELL', 'pg_isready -h localhost -U ${POSTGRES_USER} -d ${POSTGRES_DB}']
interval: 5s
timeout: 5s
retries: 10
networks:
- test_n8n_internal
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
n8n:
image: docker.n8n.io/n8nio/n8n:${N8N_VERSION}
restart: always
environment:
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PORT=5432
- DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
- DB_POSTGRESDB_USER=${POSTGRES_NON_ROOT_USER}
- DB_POSTGRESDB_PASSWORD=${POSTGRES_NON_ROOT_PASSWORD}
- N8N_HOST=${N8N_HOST}
- N8N_PORT=5678
- N8N_PROTOCOL=https
- N8N_TRUSTED_PROXIES=1
- N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true
- N8N_COMMUNITY_PACKAGES_ALLOW_TOOL_USAGE=true
- EXECUTIONS_DATA_MAX_AGE=72
- EXECUTIONS_DATA_PRUNE
- NODE_ENV=production
- WEBHOOK_URL=https://${N8N_HOST}
- QUEUE_HEALTH_CHECK_ACTIVE=true
healthcheck:
test: ["CMD", "wget", "--no-check-certificate", "--spider", "http://127.0.0.1:5678/healthz"]
interval: 12s
timeout: 10s
retries: 3
start_period: 15s
volumes:
- test_n8n_storage:/home/node/.n8n
depends_on:
postgres:
condition: service_healthy
networks:
- test_n8n_internal
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.testn8n.rule=Host(`testn8n.hyperthrive.io`)"
- "traefik.http.routers.testn8n.entrypoints=websecure"
- "traefik.http.routers.testn8n.tls.certresolver=letsencrypt"
- "traefik.http.routers.testn8n.service=testn8n"
- "traefik.http.services.testn8n.loadbalancer.server.port=5678"
- "traefik.docker.network=traefik"
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
networks:
test_n8n_internal:
traefik:
external: true

36
configs/traefik/docker-compose.yml Normal file
View File

@ -0,0 +1,36 @@
version: '3.9'
services:
traefik:
image: traefik:v3.0
container_name: traefik
restart: unless-stopped
security_opt:
- no-new-privileges:true
networks:
- traefik
ports:
- "80:80"
- "443:443"
environment:
- CF_API_EMAIL=${CF_API_EMAIL}
- CF_API_KEY=${CF_API_KEY}
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./traefik.yml:/etc/traefik/traefik.yml:ro
- ./dynamic.yml:/etc/traefik/dynamic.yml:ro
- ./letsencrypt:/letsencrypt
- ./logs:/logs
labels:
- "traefik.enable=true"
- "traefik.http.routers.dashboard.rule=Host(`traefik.hyperthrive.io`) || Host(`15.204.247.153`)"
- "traefik.http.routers.dashboard.entrypoints=websecure,web"
- "traefik.http.routers.dashboard.tls.certresolver=letsencrypt"
- "traefik.http.routers.dashboard.service=api@internal"
- "traefik.http.routers.dashboard.middlewares=auth"
- "traefik.http.middlewares.auth.basicauth.users=admin:$$2y$$05$$pxTmyNXFWcL4fpqqxwWoRuX5549vaSuSJkLz6g/EZgoaoSkmgLw4W"
networks:
traefik:
external: true

65
configs/traefik/dynamic.yml Normal file
View File

@ -0,0 +1,65 @@
http:
middlewares:
default-headers:
headers:
frameDeny: true
browserXssFilter: true
contentTypeNosniff: true
forceSTSHeader: true
stsIncludeSubdomains: true
stsPreload: true
stsSeconds: 15552000
customFrameOptionsValue: SAMEORIGIN
customResponseHeaders:
X-Robots-Tag: "noindex,nofollow,nosnippet,noarchive,notranslate,noimageindex"
X-Powered-By: "Traefik"
Server: "Traefik"
rate-limit:
rateLimit:
average: 100
burst: 50
secure-headers:
headers:
accessControlAllowMethods:
- GET
- OPTIONS
- PUT
- POST
- DELETE
accessControlMaxAge: 100
hostsProxyHeaders:
- "X-Forwarded-Host"
customRequestHeaders:
X-Forwarded-Proto: "https"
auth:
basicAuth:
users:
- "admin:$2y$05$pxTmyNXFWcL4fpqqxwWoRuX5549vaSuSJkLz6g/EZgoaoSkmgLw4W"
routers:
ping:
rule: "PathPrefix(`/ping`)"
service: ping@internal
middlewares:
- default-headers
services:
dummy:
loadBalancer:
servers:
- url: "http://localhost"
tls:
options:
default:
minVersion: VersionTLS12
cipherSuites:
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
- TLS_AES_128_GCM_SHA256
- TLS_AES_256_GCM_SHA384
- TLS_CHACHA20_POLY1305_SHA256

47
configs/traefik/traefik.yml Normal file
View File

@ -0,0 +1,47 @@
api:
dashboard: true
debug: false
entryPoints:
web:
address: ":80"
http:
redirections:
entrypoint:
to: websecure
scheme: https
websecure:
address: ":443"
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
network: traefik
file:
filename: /etc/traefik/dynamic.yml
watch: true
certificatesResolvers:
letsencrypt:
acme:
email: admin@hyperthrive.io
storage: /letsencrypt/acme.json
keyType: EC256
httpChallenge:
entryPoint: web
# Uncomment for staging (testing) - recommended for initial setup
# caServer: https://acme-staging-v02.api.letsencrypt.org/directory
log:
level: INFO
filePath: /logs/traefik.log
accessLog:
filePath: /logs/access.log
bufferingSize: 100
metrics:
prometheus:
addEntryPointsLabels: true
addServicesLabels: true

8
configs/uploads.ini Normal file
View File

@ -0,0 +1,8 @@
; PHP configuration for WordPress uploads
file_uploads = On
upload_max_filesize = 256M
post_max_size = 256M
max_execution_time = 600
max_input_time = 600
memory_limit = 512M
max_input_vars = 3000

27
configs/vaultwarden/docker-compose.yml Normal file
View File

@ -0,0 +1,27 @@
services:
vaultwarden:
image: vaultwarden/server:latest
restart: unless-stopped
environment:
- SMTP_HOST=smtp.mailpace.com
- SMTP_PORT=587
- SMTP_SECURITY=starttls
- SMTP_USERNAME=${MAILPACE_USER}
- SMTP_PASSWORD=${MAILPACE_PASS}
- SMTP_FROM=noreply@swansoncloud.com
- SMTP_FROM_NAME=Vaultwarden
volumes:
- ./data:/data
networks:
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.vaultwarden.rule=Host(`bitwarden.swansoncloud.com`)"
- "traefik.http.routers.vaultwarden.entrypoints=websecure"
- "traefik.http.routers.vaultwarden.tls.certresolver=letsencrypt"
- "traefik.http.services.vaultwarden.loadbalancer.server.port=80"
- "traefik.docker.network=traefik"
networks:
traefik:
external: true

12
configs/watchtower/docker-compose.yml Normal file
View File

@ -0,0 +1,12 @@
services:
watchtower:
image: nickfedor/watchtower:latest
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- WATCHTOWER_SCHEDULE=0 0 3 * * *
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_INCLUDE_STOPPED=false
- WATCHTOWER_NOTIFICATIONS_LEVEL=warn
command: --schedule "0 0 3 * * *"

271
configs/wordpress-docker-migration-guide.md Normal file
View File

@ -0,0 +1,271 @@
# WordPress Docker Migration Guide - Lessons Learned
**Last Updated**: 2025-09-19
**Based on**: BestSolarTech.com migration experience
## Critical Gotchas & Solutions
### 1. Variable Escaping in Docker Compose Environment Variables
**Problem**: `$_SERVER` variables in WORDPRESS_CONFIG_EXTRA are interpreted as shell variables, causing warnings and broken configuration.
**Symptom**:
```
level=warning msg="The \"_SERVER\" variable is not set. Defaulting to a blank string."
```
**Wrong**:
```yaml
WORDPRESS_CONFIG_EXTRA: |
if (isset(\$_SERVER['HTTP_X_FORWARDED_PROTO'])) {
```
**Correct**:
```yaml
WORDPRESS_CONFIG_EXTRA: |
if (isset($$_SERVER['HTTP_X_FORWARDED_PROTO'])) {
```
**Solution**: Use `$$` to escape dollar signs in Docker Compose environment variables.
### 2. Traefik v3 Host Rule Syntax
**Problem**: Multiple hosts in Traefik labels don't work with comma separation.
**Wrong**:
```yaml
- "traefik.http.routers.site.rule=Host(`domain1.com`, `domain2.com`, `domain3.com`)"
```
**Correct**:
```yaml
- "traefik.http.routers.site.rule=Host(`domain1.com`) || Host(`domain2.com`) || Host(`domain3.com`)"
```
**Solution**: Use `||` (OR operator) to separate multiple hosts in Traefik v3.
### 3. MariaDB Version Compatibility
**Problem**: MariaDB version mismatches cause redo log format errors.
**Error**:
```
[ERROR] InnoDB: Unsupported redo log format. The redo log was created with MariaDB 10.11.14.
```
**Solution**:
- Check the original database version before migration
- Use the same or newer MariaDB version
- MariaDB 10.11 is a safe choice for most WordPress sites
### 4. Real Simple SSL Plugin Behind Traefik
**Problem**: Real Simple SSL plugin causes redirect loops when WordPress is behind Traefik reverse proxy.
**Solution**:
1. Disable the plugin BEFORE migration:
```bash
# Via database (serialized array - be careful!)
UPDATE wp_options SET option_value = 'a:0:{}' WHERE option_name = 'active_plugins';
# OR use WP-CLI after migration
docker exec CONTAINER wp --allow-root plugin deactivate really-simple-ssl
```
2. Add proxy detection to wp-config.php:
```php
/* Handle SSL behind reverse proxy */
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') {
$_SERVER['HTTPS'] = 'on';
$_SERVER['SERVER_PORT'] = 443;
}
define('FORCE_SSL_ADMIN', true);
```
### 5. WordPress URL Management Strategy
**Problem**: Changing URLs too early causes redirect loops and access issues.
**Best Practice**:
1. Start with a temporary domain (e.g., temp.example.com)
2. Test everything thoroughly
3. Only update to production domain after confirming everything works
4. Keep both domains in Traefik configuration during transition
**Database URL Update**:
```sql
UPDATE wp_options SET option_value = 'https://yourdomain.com'
WHERE option_name IN ('siteurl', 'home');
```
### 6. Database Password Reset for Existing Databases
**Problem**: Existing database has unknown root password.
**Solution** - Use skip-grant-tables:
```bash
# Start MariaDB with skip-grant-tables
docker run -d --name temp_db -v ~/services/SERVICE/db_data:/var/lib/mysql mariadb:10.11 --skip-grant-tables
# Reset passwords
docker exec temp_db mysql -e "
USE mysql;
FLUSH PRIVILEGES;
ALTER USER 'root'@'%' IDENTIFIED BY 'new_password';
CREATE USER IF NOT EXISTS 'wordpress'@'%' IDENTIFIED BY 'wp_password';
GRANT ALL PRIVILEGES ON dbname.* TO 'wordpress'@'%';
FLUSH PRIVILEGES;
"
# Stop temp container and start normal service
docker stop temp_db && docker rm temp_db
```
## Complete Migration Checklist
### Pre-Migration
- [ ] Document current WordPress version, PHP version, active theme
- [ ] List all active plugins (especially security/SSL plugins)
- [ ] Check database size and MariaDB version
- [ ] Note table prefix (often not default `wp_`)
- [ ] Create full backup of files and database
### File Migration
- [ ] Copy WordPress files preserving structure:
```bash
sudo cp -r /backup/wordpress ~/services/site/wordpress
sudo chown -R www-data:www-data ~/services/site/wordpress
```
- [ ] Copy database files:
```bash
sudo cp -r /backup/mysql ~/services/site/db_data
sudo chown -R 999:999 ~/services/site/db_data
```
- [ ] Copy/create uploads.ini for PHP settings
### Docker Compose Configuration
- [ ] Use correct MariaDB version (10.11 recommended)
- [ ] Escape `$` signs as `$$` in environment variables
- [ ] Use `||` for multiple hosts in Traefik rules
- [ ] Include both internal and traefik networks
- [ ] Set health check for database
- [ ] Configure volumes for persistence
### WordPress Configuration
- [ ] Update wp-config.php database host to 'mariadb:3306'
- [ ] Add proxy SSL detection code
- [ ] Remove hardcoded URLs (use $_SERVER['HTTP_HOST'])
- [ ] Disable SSL plugins before starting
### Testing Phase
- [ ] Start with temporary domain
- [ ] Test database connectivity
- [ ] Verify no redirect loops
- [ ] Check admin panel access
- [ ] Test file uploads
- [ ] Verify all plugins working
### Production Cutover
- [ ] Update DNS records to new server
- [ ] Update database URLs to production domain
- [ ] Test SSL certificate generation
- [ ] Monitor logs for errors
- [ ] Have rollback plan ready
## Sample Docker Compose Template
```yaml
services:
wordpress:
image: wordpress:6.7-php8.3-apache
container_name: ${SITE_NAME}_wp
restart: unless-stopped
environment:
WORDPRESS_DB_HOST: mariadb:3306
WORDPRESS_DB_USER: wordpress
WORDPRESS_DB_PASSWORD: ${DB_PASSWORD}
WORDPRESS_DB_NAME: ${DB_NAME}
WORDPRESS_TABLE_PREFIX: ${TABLE_PREFIX:-wp_}
WORDPRESS_CONFIG_EXTRA: |
/* Handle SSL behind reverse proxy */
if (isset($$_SERVER['HTTP_X_FORWARDED_PROTO']) && $$_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') {
$$_SERVER['HTTPS'] = 'on';
$$_SERVER['SERVER_PORT'] = 443;
}
define('FORCE_SSL_ADMIN', true);
define('WP_MEMORY_LIMIT', '256M');
volumes:
- ./wordpress:/var/www/html
- ./uploads.ini:/usr/local/etc/php/conf.d/uploads.ini:ro
networks:
- internal
- traefik
depends_on:
mariadb:
condition: service_healthy
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik"
- "traefik.http.routers.${SITE_NAME}.rule=Host(`${DOMAIN}`)"
- "traefik.http.routers.${SITE_NAME}.entrypoints=web"
- "traefik.http.routers.${SITE_NAME}.middlewares=redirect-to-https@file"
- "traefik.http.routers.${SITE_NAME}-secure.rule=Host(`${DOMAIN}`)"
- "traefik.http.routers.${SITE_NAME}-secure.entrypoints=websecure"
- "traefik.http.routers.${SITE_NAME}-secure.tls=true"
- "traefik.http.routers.${SITE_NAME}-secure.tls.certresolver=letsencrypt"
- "traefik.http.services.${SITE_NAME}.loadbalancer.server.port=80"
mariadb:
image: mariadb:10.11
container_name: ${SITE_NAME}_db
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD}
MYSQL_DATABASE: ${DB_NAME}
MYSQL_USER: wordpress
MYSQL_PASSWORD: ${DB_PASSWORD}
volumes:
- ./db_data:/var/lib/mysql
networks:
- internal
healthcheck:
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
interval: 10s
timeout: 5s
retries: 5
start_period: 60s
networks:
traefik:
external: true
internal:
external: true
```
## WP-CLI Installation
Install WP-CLI in WordPress container for management:
```bash
docker exec CONTAINER bash -c 'curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar && chmod +x wp-cli.phar && mv wp-cli.phar /usr/local/bin/wp'
# Use with --allow-root flag
docker exec CONTAINER wp --allow-root plugin list
docker exec CONTAINER wp --allow-root cache flush
```
## Quick Troubleshooting
| Problem | Solution |
|---------|----------|
| 404 errors from Traefik | Restart Traefik: `docker restart traefik` |
| Redirect loops | Check database URLs match expected domain |
| SSL certificate errors | Check DNS points to server, wait for Let's Encrypt |
| Database connection failed | Verify credentials, check container network |
| Plugin conflicts | Disable via database or WP-CLI |
## Important Notes
- Always test on a temporary domain first
- Keep backups before making changes
- Document all credentials securely
- Monitor logs during migration
- Have a rollback plan ready

106
configs/wordpress-template.yml Normal file
View File

@ -0,0 +1,106 @@
version: '3.8'
services:
wordpress:
image: wordpress:6.7-php8.3-apache
container_name: ${SITE_NAME:-wordpress}
restart: unless-stopped
environment:
WORDPRESS_DB_HOST: mariadb:3306
WORDPRESS_DB_USER: ${DB_USER:-wordpress}
WORDPRESS_DB_PASSWORD: ${DB_PASSWORD}
WORDPRESS_DB_NAME: ${DB_NAME:-wordpress}
WORDPRESS_TABLE_PREFIX: ${TABLE_PREFIX:-wp_}
# Optional performance tuning
WORDPRESS_CONFIG_EXTRA: |
define('WP_MEMORY_LIMIT', '256M');
define('WP_MAX_MEMORY_LIMIT', '512M');
define('WP_HOME', 'https://${DOMAIN}');
define('WP_SITEURL', 'https://${DOMAIN}');
# Disable auto-updates during migration
define('AUTOMATIC_UPDATER_DISABLED', true);
define('WP_AUTO_UPDATE_CORE', false);
volumes:
# Persistent storage for WordPress files
- ./wp-content:/var/www/html/wp-content
- ./uploads.ini:/usr/local/etc/php/conf.d/uploads.ini
networks:
- internal
- traefik
depends_on:
mariadb:
condition: service_healthy
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik"
# HTTP Router
- "traefik.http.routers.${SITE_NAME:-wordpress}.rule=Host(`${DOMAIN}`)"
- "traefik.http.routers.${SITE_NAME:-wordpress}.entrypoints=web"
- "traefik.http.routers.${SITE_NAME:-wordpress}.middlewares=redirect-to-https@file"
# HTTPS Router
- "traefik.http.routers.${SITE_NAME:-wordpress}-secure.rule=Host(`${DOMAIN}`)"
- "traefik.http.routers.${SITE_NAME:-wordpress}-secure.entrypoints=websecure"
- "traefik.http.routers.${SITE_NAME:-wordpress}-secure.tls=true"
- "traefik.http.routers.${SITE_NAME:-wordpress}-secure.tls.certresolver=letsencrypt"
- "traefik.http.services.${SITE_NAME:-wordpress}.loadbalancer.server.port=80"
mariadb:
image: mariadb:10.7
container_name: ${SITE_NAME:-wordpress}_db
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD}
MYSQL_DATABASE: ${DB_NAME:-wordpress}
MYSQL_USER: ${DB_USER:-wordpress}
MYSQL_PASSWORD: ${DB_PASSWORD}
# Performance tuning
MARIADB_AUTO_UPGRADE: "1"
MARIADB_INITDB_SKIP_TZINFO: "1"
volumes:
# Persistent database storage
- ./db_data:/var/lib/mysql
# Optional: Custom MySQL configuration
- ./my.cnf:/etc/mysql/conf.d/custom.cnf:ro
networks:
- internal
healthcheck:
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
interval: 10s
timeout: 5s
retries: 5
start_period: 60s
# Optional: phpMyAdmin for database management
phpmyadmin:
image: phpmyadmin:latest
container_name: ${SITE_NAME:-wordpress}_pma
restart: unless-stopped
environment:
PMA_HOST: mariadb
PMA_PORT: 3306
UPLOAD_LIMIT: 300M
networks:
- internal
- traefik
depends_on:
- mariadb
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik"
- "traefik.http.routers.${SITE_NAME:-wordpress}-pma.rule=Host(`pma.${DOMAIN}`)"
- "traefik.http.routers.${SITE_NAME:-wordpress}-pma.entrypoints=websecure"
- "traefik.http.routers.${SITE_NAME:-wordpress}-pma.tls=true"
- "traefik.http.routers.${SITE_NAME:-wordpress}-pma.tls.certresolver=letsencrypt"
- "traefik.http.services.${SITE_NAME:-wordpress}-pma.loadbalancer.server.port=80"
# Basic auth for security
- "traefik.http.routers.${SITE_NAME:-wordpress}-pma.middlewares=auth@file"
networks:
traefik:
external: true
internal:
external: true
volumes:
db_data:
wp_content:

13
configs/wordpress.env.example Normal file
View File

@ -0,0 +1,13 @@
# Site Configuration
SITE_NAME=mysite
DOMAIN=example.com
TABLE_PREFIX=wp_
# Database Configuration
DB_NAME=wordpress
DB_USER=wordpress
DB_PASSWORD=secure_password_here
DB_ROOT_PASSWORD=secure_root_password_here
# Optional: Staging domain for testing
# STAGING_DOMAIN=new.example.com

85
docs/backup-setup-progress.md Normal file
View File

@ -0,0 +1,85 @@
# Backup System Setup Progress
## Status: Complete (as of 2025-12-09)
## Completed Steps
### 1. Backblaze B2 Setup ✅
- Account created
- Bucket: `ovh-prod-01-backups` (private, server-side encryption enabled)
- Application key: `synology-hyper-backup` (stored in Bitwarden as "Backblaze | ovh-prod-01-backups key")
- Client-side encryption: Not needed (server-side encryption sufficient for this use case)
### 2. Server-Side Backup Script ✅
- Location: `~/services/backup.sh` on OVH server (15.204.247.153)
- Auto-discovers services in `~/services/*/docker-compose.yml`
- Supports: MariaDB, PostgreSQL, MongoDB, PocketBase
- Output: `~/backups/current/latest.tar.gz`
- Cron: `0 7 * * *` (2 AM EST = 7 AM UTC)
- Tested successfully - created 792MB backup
### 3. Synology SSH Key Setup ✅
- Key generated: `/var/services/homes/jared/.ssh/id_ed25519_ovh_backup`
- Public key added to OVH server's `~/.ssh/authorized_keys`
- Connection tested successfully
### 4. Synology Pull Script ✅
- Directory created: `/volume1/backups/ovh-prod-01/`
- Script created: `/volume1/backups/ovh-prod-01/pull-backup.sh`
- SSH key: `/var/services/homes/jared/.ssh/id_ed25519_ovh_backup`
### 5. Synology Pull Task Scheduled ✅
- Configured via Synology Task Scheduler
- Schedule: 3 AM EST (8 AM UTC)
- Runs as root to avoid permission issues
### 6. Hyper Backup to Backblaze B2 ✅
- Task name: "OVH Server Backup"
- Source: `/volume1/backups/ovh-prod-01/`
- Destination: `ovh-prod-01-backups` bucket on Backblaze B2
- Schedule: 4 AM EST daily (after Synology pull completes)
- Integrity check: Weekly on Saturday
- Rotation: Smart Recycle (256 max versions)
- Settings: Compression enabled, transfer encryption enabled
### 7. Test Full Chain ✅
- Initial backup completed successfully on 2025-12-09
### 8. Disaster Recovery Documentation ✅
- Created `docs/disaster-recovery.md`
- Covers single service recovery and full server rebuild
- Documents backup locations and credential sources
### 9. Update Service Deployment Docs ✅
- Added step 6 to deployment pattern in CLAUDE.md
- Added disaster-recovery.md link to specialized documentation
## Architecture Summary
```
OVH Server (2 AM EST) Synology (3 AM EST) Backblaze B2
───────────────────── ────────────────── ────────────
~/services/backup.sh → rsync pull → Hyper Backup
↓ ↓ ↓
~/backups/current/ /volume1/backups/ ovh-prod-01-backups
latest.tar.gz ovh-prod-01/ bucket
```
## Retention Policy
- Server: 2 days (staging only)
- Synology/Backblaze: Smart Recycle (hourly for 24h, daily for ~1 month, weekly beyond)
- Max versions: 256
## Gotchas & Lessons Learned
### Hyper Backup Folder Visibility
- **Issue:** Newly created folders don't appear in Hyper Backup's folder picker
- **Solution:** Close the entire Hyper Backup wizard and restart it; the folder list refreshes on wizard launch
### Synology Permissions
- **Issue:** Pull script had permission problems when run as regular user
- **Solution:** Run the scheduled task as root in Task Scheduler
### B2 Encryption
- **Decision:** Server-side encryption only (no client-side)
- **Rationale:** SSE is sufficient for server backups; client-side adds key management complexity with minimal benefit for this use case

314
docs/configuration-reference.md Normal file
View File

@ -0,0 +1,314 @@
# Configuration Reference
This document catalogs all configuration templates and files available in the `configs/` directory for deploying services on systems-prod-01.
## Template Overview
| Template | Purpose | Service Type |
|----------|---------|--------------|
| `wordpress-template.yml` | WordPress with MariaDB | WordPress Site |
| `invoiceninja-template.yml` | Invoice Ninja invoicing app | Web Application |
| `traefik/docker-compose.yml` | Traefik reverse proxy | Infrastructure |
## Traefik Configuration
### Location
`configs/traefik/`
### Files
- **`docker-compose.yml`** - Traefik container orchestration
- **`traefik.yml`** - Static configuration (entrypoints, providers, API)
- **`dynamic.yml`** - Dynamic configuration (routers, services, middlewares)
### When to Use
- Setting up a new server from scratch
- Reconfiguring the reverse proxy
- Adding global middlewares or routes
- Reference for Traefik label syntax
### Key Features
- Automatic SSL with Let's Encrypt
- HTTP to HTTPS redirect
- Docker provider for automatic service discovery
- Dashboard on port 8080 with basic auth
## WordPress Template
### Location
`configs/wordpress-template.yml`
### Supporting Files
- `configs/wordpress.env.example` - Environment variables template
- `configs/uploads.ini` - PHP upload size configuration
- `configs/my.cnf` - MariaDB optimization
### When to Use
- Migrating a WordPress site from another server
- Deploying a new WordPress site
- Reference for WordPress + Traefik integration
### Key Features
- WordPress 6.7 with Apache
- MariaDB 10.11 (LTS, WordPress-compatible)
- Traefik integration with automatic SSL
- Health checks for database
- Separate internal network for database
### Usage Example
```bash
# 1. Create service directory
ssh jared@15.204.247.153 "mkdir -p ~/services/SITENAME"
# 2. Copy and adapt template
cp configs/wordpress-template.yml /tmp/docker-compose.yml
# Edit with your site details
# 3. Deploy to server
scp /tmp/docker-compose.yml jared@15.204.247.153:~/services/SITENAME/
ssh jared@15.204.247.153 "cd ~/services/SITENAME && docker compose up -d"
```
### Template Customization Checklist
- [ ] Replace `SITENAME` with actual service name
- [ ] Update domain names in Traefik labels
- [ ] Set database credentials
- [ ] Configure volume paths
- [ ] Adjust memory/resource limits if needed
- [ ] Verify network names match server setup
## Invoice Ninja Template
### Location
`configs/invoiceninja-template.yml`
### Supporting Files
- `configs/invoiceninja.env.example` - Environment variables
- `configs/invoiceninja-nginx.conf` - Nginx configuration
### When to Use
- Deploying Invoice Ninja for invoicing/billing
- Reference for PHP-FPM + Nginx + MySQL setup
- Multi-container application pattern
### Key Features
- Invoice Ninja v5
- Nginx + PHP-FPM architecture
- MySQL database with health checks
- Persistent storage for invoices and uploads
- Traefik integration
## Configuration Patterns
### Standard Service Structure
All services follow this pattern:
```yaml
services:
app:
image: application:latest
restart: unless-stopped
networks:
- traefik # External access
- internal # Backend communication
labels:
- "traefik.enable=true"
- "traefik.http.routers.SERVICENAME.rule=Host(`domain.com`)"
- "traefik.http.routers.SERVICENAME.entrypoints=websecure"
- "traefik.http.routers.SERVICENAME.tls.certresolver=letsencrypt"
volumes:
- ./data:/var/www/html
networks:
traefik:
external: true
internal:
external: true
```
### Traefik Labels Guide
**Basic routing:**
```yaml
- "traefik.enable=true"
- "traefik.http.routers.SERVICENAME.rule=Host(`domain.com`)"
- "traefik.http.routers.SERVICENAME.entrypoints=websecure"
- "traefik.http.routers.SERVICENAME.tls.certresolver=letsencrypt"
```
**Multiple domains:**
```yaml
- "traefik.http.routers.SERVICENAME.rule=Host(`domain.com`) || Host(`www.domain.com`)"
```
**Custom port (if app doesn't use 80):**
```yaml
- "traefik.http.services.SERVICENAME.loadbalancer.server.port=8080"
```
**Basic auth middleware:**
```yaml
- "traefik.http.routers.SERVICENAME.middlewares=auth"
- "traefik.http.middlewares.auth.basicauth.users=user:$$apr1$$hashed$$password"
```
## Environment Variable Management
### Best Practices
1. **Never commit secrets** - Use `.env` files (gitignored)
2. **Use example files** - Provide `.env.example` with placeholders
3. **Document required vars** - List all necessary environment variables
4. **Escape dollar signs** - In docker-compose.yml, use `$$` for literal `$`
### Example .env File Structure
```bash
# Database Configuration
DB_NAME=myapp_db
DB_USER=myapp_user
DB_PASSWORD=CHANGE_ME_STRONG_PASSWORD
# Application Settings
APP_NAME=MyApplication
APP_URL=https://myapp.com
APP_DEBUG=false
# Email Configuration (if needed)
MAIL_HOST=smtp.example.com
MAIL_PORT=587
MAIL_USERNAME=user@example.com
MAIL_PASSWORD=CHANGE_ME
```
## Creating Custom Templates
When creating a new service template:
1. **Start with a working configuration** from existing templates
2. **Document all required changes** in comments
3. **Include health checks** where possible
4. **Use standard network pattern** (traefik + internal)
5. **Follow naming conventions** (lowercase, hyphens)
6. **Test thoroughly** before documenting
### Template File Naming
- `SERVICETYPE-template.yml` - Docker Compose template
- `SERVICETYPE.env.example` - Environment variables
- `SERVICETYPE-config-file` - Additional config files
## Docker Compose Tips
### Volume Management
**Bind mounts** (for configs):
```yaml
volumes:
- ./config.php:/var/www/html/config.php:ro
```
**Named volumes** (for databases):
```yaml
volumes:
- db_data:/var/lib/mysql
volumes:
db_data:
```
### Health Checks
**Web application:**
```yaml
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost/health"]
interval: 30s
timeout: 10s
retries: 3
```
**Database:**
```yaml
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
interval: 30s
timeout: 10s
retries: 5
```
### Resource Limits
```yaml
deploy:
resources:
limits:
cpus: '2.0'
memory: 2G
reservations:
cpus: '0.5'
memory: 512M
```
## Network Configuration
### Existing Networks
The server has two pre-configured Docker networks:
- **traefik** (172.20.0.0/16) - External-facing services
- **internal** (172.21.0.0/16) - Backend services
### When to Create New Networks
Generally, use existing networks. Create a new network only when:
- Service requires strict isolation
- Complex multi-container setup needs dedicated network
- Testing/development environment separation
## Troubleshooting Configuration Issues
### Template Not Working
```bash
# Validate docker-compose syntax
docker compose -f template.yml config
# Check for common issues
# - Indentation (YAML is space-sensitive)
# - Missing quotes around special characters
# - Incorrect network references
# - Typos in service names
```
### Traefik Not Routing
```bash
# Check container is on traefik network
docker inspect CONTAINER_NAME | grep Networks -A 10
# Verify Traefik labels
docker inspect CONTAINER_NAME | grep traefik.
# Check Traefik dashboard
# https://traefik.hyperthrive.app
```
### Environment Variables Not Loading
```bash
# Verify .env file exists in same directory as docker-compose.yml
ls -la ~/services/SERVICENAME/.env
# Check variable interpolation
docker compose config
```
## See Also
- [Services Inventory](./services-inventory.md) - Currently deployed services
- [WordPress Operations](./wordpress-operations.md) - WordPress-specific guides
- [Documentation Maintenance](./documentation-maintenance.md) - Adding new templates
- [CLAUDE.md](../CLAUDE.md) - Project overview and deployment workflow

185
docs/disaster-recovery.md Normal file
View File

@ -0,0 +1,185 @@
# Disaster Recovery Guide
This document covers how to restore services from backups in the event of server failure.
## Understanding Hyper Backup's Storage Format
Hyper Backup does NOT create timestamped files like `backup-2025-12-09.tar.gz`. Instead, it uses a **deduplicated block storage format** (`.hbk` folder).
**What you'll see in B2:**
```
ovh-prod-01.hbk/
├── Config/ # Backup configuration
├── Control/ # Version metadata and timestamps
├── Guard/ # Integrity verification data
├── Pool/ # Deduplicated data blocks (shared across all versions)
└── *.db files # Index databases
```
**Why this design?**
- A 1.6GB backup that changes 50MB daily only stores ~50MB new data per version
- Much more storage-efficient than full timestamped copies
- All version history is embedded in the metadata
**To browse backup versions:**
1. Open Hyper Backup on Synology
2. Select the backup task
3. Click **"Version List"** button
4. Timeline shows all versions with timestamps - select one to browse/restore
**To restore without Synology access:**
- Download the entire `.hbk` folder from B2
- Use **Hyper Backup Explorer** (free Synology desktop tool for Windows/Mac/Linux)
- Browse and extract any version from the backup set
## Backup Locations
| Location | Path | Contents |
|----------|------|----------|
| OVH Server | `~/backups/current/latest.tar.gz` | Most recent backup (2 days retention) |
| Synology NAS | `/volume1/backups/ovh-prod-01/` | Pulled backups with Smart Recycle retention |
| Backblaze B2 | `ovh-prod-01-backups` bucket | Offsite mirror of Synology |
## Backup Contents
The `latest.tar.gz` contains:
```
backups/
├── SERVICE_NAME/
│ ├── docker-compose.yml # Service configuration
│ ├── .env # Environment variables (if present)
│ └── db_backup.sql # Database dump (MariaDB/PostgreSQL)
│ or db_backup/ # MongoDB dump directory
│ or pb_data.tar.gz # PocketBase data
```
## Recovery Procedures
### Scenario 1: Single Service Recovery
If one service is corrupted but the server is intact:
```bash
# 1. SSH into server
ssh jared@15.204.247.153
# 2. Stop the affected service
cd ~/services/SERVICE_NAME
docker compose down
# 3. Extract backup
cd ~/backups/current
tar -xzf latest.tar.gz
# 4. Restore database (MariaDB example)
docker exec -i SERVICE_NAME-db mysql -u root -pPASSWORD DATABASE < backups/SERVICE_NAME/db_backup.sql
# 5. Restart service
cd ~/services/SERVICE_NAME
docker compose up -d
```
### Scenario 2: Full Server Recovery
If the server is lost and you need to rebuild:
#### Step 1: Provision New Server
- OVHcloud VPS or equivalent
- Ubuntu 22.04 LTS recommended
- Install Docker and Docker Compose
#### Step 2: Restore Network Infrastructure
```bash
# Create Docker networks
docker network create --subnet=172.20.0.0/16 traefik
docker network create --subnet=172.21.0.0/16 internal
# Create services directory
mkdir -p ~/services
```
#### Step 3: Get Backup from Synology or B2
**From Synology (if accessible):**
```bash
scp jared@192.168.86.31:/volume1/backups/ovh-prod-01/latest.tar.gz ~/
```
**From Backblaze B2:**
1. Open Synology Hyper Backup
2. Click the restore icon
3. Browse to desired backup version
4. Download/restore `latest.tar.gz`
Or use B2 CLI:
```bash
b2 authorize-account APPLICATION_KEY_ID APPLICATION_KEY
b2 download-file-by-name ovh-prod-01-backups latest.tar.gz ./latest.tar.gz
```
#### Step 4: Restore Traefik First
```bash
# Extract backup
tar -xzf latest.tar.gz
# Copy Traefik config
cp -r backups/traefik ~/services/
# Start Traefik
cd ~/services/traefik
docker compose up -d
```
#### Step 5: Restore Each Service
For each service directory in the backup:
```bash
# Copy service files
cp -r backups/SERVICE_NAME ~/services/
# Start service (creates containers and volumes)
cd ~/services/SERVICE_NAME
docker compose up -d
# Restore database if applicable
# MariaDB:
docker exec -i CONTAINER_NAME mysql -u root -pPASSWORD DATABASE < db_backup.sql
# PostgreSQL:
docker exec -i CONTAINER_NAME psql -U USER DATABASE < db_backup.sql
# MongoDB:
docker exec -i CONTAINER_NAME mongorestore --archive < db_backup.archive
```
#### Step 6: Verify Services
```bash
# Check all containers are running
docker ps
# Check Traefik dashboard for routing
# Verify each service responds on its domain
```
#### Step 7: Update DNS
Point domains to new server IP if changed.
## Credentials Required for Recovery
All credentials stored in Bitwarden:
- **Server SSH**: Search "OVH" or "15.204.247.153"
- **Backblaze B2**: Search "Backblaze | ovh-prod-01-backups key"
- **Database passwords**: In each service's `.env` file (included in backup)
- **Traefik dashboard**: In `credentials.md` (this repo)
## Testing Recovery
Periodically test recovery by:
1. Spinning up a test VPS
2. Following the full server recovery procedure
3. Verifying services come up correctly
4. Destroying test VPS
## Contact
Server admin: Jared Swanson

272
docs/documentation-maintenance.md Normal file
View File

@ -0,0 +1,272 @@
# Documentation Maintenance Guide
This meta-document describes when and how to keep the server documentation current. Following these guidelines ensures documentation remains accurate and useful for both humans and AI assistants.
## Documentation Structure
```
/home/jared/projects/server-ovh-prod-01/
├── CLAUDE.md # Core principles (rarely changes)
├── README.md # Quick reference
├── profile.yaml # Server specifications
├── credentials.md # Access credentials
├── docs/ # Specialized documentation
│ ├── services-inventory.md # 🔄 DYNAMIC - Update frequently
│ ├── wordpress-operations.md # Reference guide
│ ├── monitoring-automation.md # Operations reference
│ ├── configuration-reference.md # Template catalog
│ └── documentation-maintenance.md # This file
├── migrations/
│ └── migration-plan.md # 🔄 DYNAMIC - Update per phase
└── configs/ # Templates and examples
```
## Update Frequency
| Document | Update Frequency | Trigger |
|----------|------------------|---------|
| `CLAUDE.md` | Rarely | Architecture changes, new patterns |
| `services-inventory.md` | **Every deployment** | Service added/removed/changed |
| `migration-plan.md` | Per migration phase | Service migration completed |
| `profile.yaml` | Monthly or on change | Server upgrades, config changes |
| `wordpress-operations.md` | As needed | New WordPress patterns discovered |
| `monitoring-automation.md` | As needed | Monitoring script changes |
| `configuration-reference.md` | When adding templates | New service type added |
| `README.md` | As needed | Major project updates |
## When to Update Each Document
### After Deploying a New Service
**Checklist:**
- [ ] Update `docs/services-inventory.md`
- Add service to Active Services table
- Update directory structure if new location
- Update "Last Updated" date
- [ ] Update `migrations/migration-plan.md` (if migrating)
- Mark service as completed
- Update phase status
- Add any lessons learned
- [ ] Consider updating `README.md`
- Add to "Live Services" section if user-facing
- [ ] Create template if it's a new service type
- Add to `configs/`
- Document in `configuration-reference.md`
### After Removing/Decommissioning a Service
**Checklist:**
- [ ] Update `docs/services-inventory.md`
- Remove from Active Services table
- Update directory structure
- Update "Last Updated" date
- [ ] Update `README.md`
- Remove from "Live Services" if listed
- [ ] Archive service documentation
- Move deployment notes to `migrations/completed/`
- Keep for reference
### After Changing Service Configuration
**Checklist:**
- [ ] Update `docs/services-inventory.md` if:
- Domain changed
- Container names changed
- Service type changed
- [ ] Update `credentials.md` if:
- Passwords changed
- Access methods changed
- [ ] Update relevant template in `configs/` if:
- Configuration pattern improved
- Discovered better approach
### After Server Upgrades/Changes
**Checklist:**
- [ ] Update `profile.yaml`
- Software versions
- Resource allocations
- Network configurations
- [ ] Update `CLAUDE.md` if:
- Architecture patterns changed
- New best practices established
- Critical commands changed
### After Discovering New Patterns/Solutions
**Checklist:**
- [ ] Update `docs/wordpress-operations.md` if:
- WordPress-specific discovery
- New troubleshooting solution
- [ ] Update `docs/monitoring-automation.md` if:
- Monitoring script changes
- New health check patterns
- [ ] Update `docs/configuration-reference.md` if:
- New Docker Compose pattern
- Traefik label improvements
- [ ] Consider updating `CLAUDE.md` if:
- Pattern applies broadly across all services
- Changes fundamental workflow
## How to Update Services Inventory
The `docs/services-inventory.md` file changes most frequently. Here's how to keep it current:
### Quick Update Process
```bash
# 1. Get current state from server
ssh jared@15.204.247.153 "docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'"
# 2. Edit services-inventory.md locally
# Add/remove/update services in the table
# 3. Update the "Last Updated" date
# At the top of services-inventory.md
# 4. Commit changes
git add docs/services-inventory.md
git commit -m "Update services inventory: [describe change]"
```
### Service Entry Template
When adding a new service to the inventory:
```markdown
| **servicename** | Type | domain.com | `container_app`<br>`container_db` | ✅ Running | Brief note |
```
**Fields:**
- **Service name** - Lowercase, matches directory name
- **Type** - WordPress, Web App, Database, etc.
- **Domain(s)** - Primary domain (use `<br>` for multiple)
- **Containers** - List all containers (use `<br>` for multiple)
- **Status** - ✅ Running, ⚠️ Degraded, ❌ Stopped
- **Notes** - Brief deployment note or purpose
## Keeping CLAUDE.md Lean
**CLAUDE.md should remain small and focused.** Before adding content to CLAUDE.md, ask:
1. **Is this used in almost every session?** → Add to CLAUDE.md
2. **Is this reference material?** → Add to specialized doc
3. **Is this dynamic/changing?** → Definitely NOT in CLAUDE.md
4. **Is this service-specific?** → Add to specialized doc
**Examples:**
- ✅ Add to CLAUDE.md: SSH command, core architecture principles
- ❌ Don't add to CLAUDE.md: WordPress URL change commands, current service list
## AI Assistant Guidelines
When an AI assistant (like Claude Code) needs to update documentation:
### Always Update
- `services-inventory.md` after any service deployment/removal
- `migration-plan.md` after completing migration tasks
- Relevant timestamp/date fields
### Selectively Update
- Operation-specific docs only when discovering new patterns
- Templates only when creating reusable configurations
- `CLAUDE.md` only for architectural changes
### Never Update Without Asking
- `credentials.md` (security sensitive)
- `profile.yaml` (factual server data)
- Removing existing content from docs
## Git Commit Practices
### Commit Messages for Documentation
**Good examples:**
```
docs: Update services inventory - add invoiceninja
docs: Add WordPress SSL troubleshooting to operations guide
docs: Remove decommissioned landhometeam from inventory
docs: Update migration plan - Phase 3.3 complete
```
**Follow pattern:**
```
docs: <brief description of what changed>
```
### When to Commit
- After each service deployment/removal
- After completing documentation updates
- Before starting a new migration phase
- When adding new templates or guides
### What NOT to Commit
- Temporary files
- Sensitive credentials (use .gitignore)
- AI assistant output logs
- Personal notes not relevant to project
## Documentation Quality Standards
### Writing Style
- **Clear and concise** - Get to the point quickly
- **Action-oriented** - Use commands and examples
- **Assume remote context** - Remember server is remote (15.204.247.153)
- **Include copy-paste examples** - Make commands easily executable
### Structure Standards
- **Use headers** - Break content into scannable sections
- **Use tables** - For structured data (services, specs, etc.)
- **Use code blocks** - For all commands and configs
- **Use checklists** - For procedural steps
### Link Standards
- **Use relative links** - `./other-doc.md` not absolute paths
- **Keep cross-references** - Help readers find related info
- **Verify links work** - Test links after structural changes
## Review Schedule
**Monthly Review:**
- [ ] Verify `services-inventory.md` matches actual server state
- [ ] Check `profile.yaml` reflects current configuration
- [ ] Review `migration-plan.md` status is current
- [ ] Scan logs for patterns worthy of documentation
**Quarterly Review:**
- [ ] Review all docs for accuracy
- [ ] Remove obsolete content
- [ ] Update examples with current versions
- [ ] Consolidate redundant information
## Troubleshooting Documentation Issues
### Stale Information
**Problem:** Documentation doesn't match reality
**Solution:**
1. Verify current state via SSH commands
2. Update affected documents
3. Note in commit message what was stale
### Missing Information
**Problem:** Documentation gap discovered
**Solution:**
1. Determine appropriate document location
2. Add missing information with examples
3. Cross-reference from related docs
### Conflicting Information
**Problem:** Different docs say different things
**Solution:**
1. Verify which is correct via server inspection
2. Update all conflicting locations
3. Consider consolidating if too scattered
## See Also
- [Services Inventory](./services-inventory.md) - Most frequently updated doc
- [Configuration Reference](./configuration-reference.md) - Template documentation
- [CLAUDE.md](../CLAUDE.md) - Core principles that rarely change
- [Migration Plan](../migrations/migration-plan.md) - Phase tracking

197
docs/invoiceninja.md Normal file
View File

@ -0,0 +1,197 @@
# Invoice Ninja
**Type**: Invoicing & Billing Application
**Domain**: https://invoice.hyperthrive.io
**Version**: 5.12.28
**Architecture**: Official Docker stack from [invoiceninja/dockerfiles](https://github.com/invoiceninja/dockerfiles)
**Deployment Date**: 2025-10-14
## Architecture
Invoice Ninja runs using the official Docker Compose stack with three services:
1. **invoiceninja-server-1** - Nginx web server (port 80 internal)
2. **invoiceninja-app-1** - Laravel application with supervisord (PHP-FPM, queue workers, scheduler)
3. **invoiceninja-db-1** - MariaDB 10.11 database
### Traefik Integration
The nginx service connects to the external `traefik` network and uses labels for automatic SSL/routing:
- **Domain**: invoice.hyperthrive.io
- **Protocol**: HTTPS via Let's Encrypt
- **Entry Point**: websecure (443)
- **Internal Port**: 80 (nginx)
### Directory Structure (Server)
```
~/services/invoiceninja/
├── docker-compose.yml # Main compose file
├── env # Environment configuration
├── config/
│ └── nginx/
│ └── in-vhost.conf # Nginx vhost config
├── docker/
│ ├── app/
│ │ ├── public/ # Application public files (shared with nginx)
│ │ └── storage/ # Application storage (logs, cache, uploads)
│ └── mysql/
│ └── data/ # Database files
```
**Important**: The `docker/app/` directories must be owned by user `1500:1500` (the container's runtime user).
## Common Operations
### Check Status
```bash
ssh jared@15.204.247.153 "docker ps --filter name=invoiceninja"
```
### View Logs
```bash
# Application logs
ssh jared@15.204.247.153 "docker logs invoiceninja-app-1 --tail 50"
# Database logs
ssh jared@15.204.247.153 "docker logs invoiceninja-db-1 --tail 50"
# Nginx logs
ssh jared@15.204.247.153 "docker logs invoiceninja-server-1 --tail 50"
```
### Update Invoice Ninja
```bash
ssh jared@15.204.247.153 "cd ~/services/invoiceninja && \
docker compose pull && \
docker compose up -d && \
docker exec invoiceninja-app-1 php artisan migrate --force && \
docker exec invoiceninja-app-1 php artisan optimize"
```
### Backup Database
```bash
ssh jared@15.204.247.153 "cd ~/services/invoiceninja && \
docker compose exec -T db sh -c 'exec mysqldump --all-databases -uroot -p\"\$MYSQL_ROOT_PASSWORD\"' \
> ~/backups/invoiceninja-$(date +%Y%m%d-%H%M%S).sql"
```
### Restore Database
```bash
ssh jared@15.204.247.153 "cd ~/services/invoiceninja && \
docker compose exec -T db sh -c 'exec mysql -uroot -p\"\$MYSQL_ROOT_PASSWORD\"' \
< ~/backups/invoiceninja-TIMESTAMP.sql"
```
### Restart Services
```bash
ssh jared@15.204.247.153 "cd ~/services/invoiceninja && docker compose restart"
```
### Run Artisan Commands
```bash
# Generic pattern
ssh jared@15.204.247.153 "docker exec invoiceninja-app-1 php artisan <command>"
# Examples
ssh jared@15.204.247.153 "docker exec invoiceninja-app-1 php artisan queue:work"
ssh jared@15.204.247.153 "docker exec invoiceninja-app-1 php artisan cache:clear"
ssh jared@15.204.247.153 "docker exec invoiceninja-app-1 php artisan db:seed"
```
## Key Configuration
### Environment Variables (env file)
Critical settings in `~/services/invoiceninja/env`:
- `APP_URL=https://invoice.hyperthrive.io` - Public URL
- `APP_KEY=base64:...` - Encryption key (DO NOT change after initial setup)
- `DB_HOST=db` - Database hostname (container name)
- `MYSQL_ROOT_PASSWORD` - DB root password
- `MYSQL_DATABASE=ninja` - Database name
- `MYSQL_USER=ninja` - DB user
- `MYSQL_PASSWORD` - DB password
- `TRUSTED_PROXIES=*` - Trust Traefik proxy
- `MAIL_MAILER=log` - Mail driver (currently logging only)
### Traefik Labels (docker-compose.yml)
```yaml
labels:
- "traefik.enable=true"
- "traefik.http.routers.invoiceninja.rule=Host(`invoice.hyperthrive.io`)"
- "traefik.http.routers.invoiceninja.entrypoints=websecure"
- "traefik.http.routers.invoiceninja.tls.certresolver=letsencrypt"
- "traefik.http.services.invoiceninja.loadbalancer.server.port=80"
- "traefik.docker.network=traefik"
```
## Troubleshooting
### Permission Denied Errors
If the app container fails with "Permission denied" creating directories:
```bash
ssh jared@15.204.247.153 "cd ~/services/invoiceninja && \
docker compose down && \
sudo chown -R 1500:1500 docker/app && \
docker compose up -d"
```
### Database Connection Issues
Check database is healthy:
```bash
ssh jared@15.204.247.153 "docker inspect invoiceninja-db-1 --format='{{.State.Health.Status}}'"
```
Should return `healthy`. If not, check logs.
### 502 Bad Gateway
Usually means the app container isn't running or PHP-FPM hasn't started:
1. Check app container status: `docker ps --filter name=invoiceninja-app-1`
2. Check app logs: `docker logs invoiceninja-app-1 --tail 100`
3. Look for PHP-FPM startup message: "ready to handle connections"
### SSL Certificate Issues
Traefik automatically handles SSL. If certificate issues occur:
1. Check Traefik logs: `docker logs traefik`
2. Verify DNS points to server: `dig invoice.hyperthrive.io`
3. Check Traefik dashboard for router/certificate status
### Queue Not Processing
The official stack includes queue workers via supervisord. Check they're running:
```bash
ssh jared@15.204.247.153 "docker exec invoiceninja-app-1 supervisorctl status"
```
Should show `queue-worker_00` and `queue-worker_01` as RUNNING.
## Migration History
This deployment uses the **official Invoice Ninja Docker stack** (as of 2025-10-14). Previous custom stack was replaced to align with upstream recommendations and simplify maintenance.
**Pre-migration backup location**: `~/backups/invoiceninja-pre-official/` on server (retained until deployment proven stable)
## References
- Official Docker Repository: https://github.com/invoiceninja/dockerfiles
- Invoice Ninja Docs: https://invoiceninja.github.io
- Server Profile: `profile.yaml`
- Configuration Template: `configs/invoiceninja-official-template.yml`

271
docs/monitoring-automation.md Normal file
View File

@ -0,0 +1,271 @@
# Monitoring & Automation
This document describes the automated monitoring and maintenance systems running on systems-prod-01.
## Overview
The server has two automated scripts running via cron to ensure service health and prevent resource exhaustion:
1. **Health Monitor** - Runs every 10 minutes, monitors and restarts unhealthy containers
2. **Daily Maintenance** - Runs at 3 AM EST, performs cleanup and optimization
## Health Monitor
**Script:** `~/services/container-health-monitor.sh`
**Schedule:** Every 10 minutes
**Log:** `~/services/monitor.log`
### What It Does
- Monitors container health status
- Checks memory usage for each container
- Restarts containers if memory exceeds 40%
- Checks site availability (HTTP status codes)
- Cleans up Docker resources (unused volumes, images)
- Logs all actions with timestamps
### Manual Commands
```bash
# View recent monitoring activity
ssh jared@15.204.247.153 "tail -50 ~/services/monitor.log"
# Run health check manually
ssh jared@15.204.247.153 "~/services/container-health-monitor.sh"
# View cron schedule
ssh jared@15.204.247.153 "crontab -l"
# Watch monitoring in real-time
ssh jared@15.204.247.153 "tail -f ~/services/monitor.log"
```
### Memory Threshold Configuration
The 40% memory restart threshold can be adjusted by editing the script:
```bash
ssh jared@15.204.247.153 "nano ~/services/container-health-monitor.sh"
# Look for the line:
# MEMORY_THRESHOLD=40
```
## Daily Maintenance
**Script:** `~/services/daily-maintenance.sh`
**Schedule:** Daily at 3:00 AM EST
**Log:** `~/services/maintenance.log`
### What It Does
- Clears temporary files and cache
- Optimizes database tables (WordPress, etc.)
- Performs controlled container restarts
- Prevents memory buildup
- Prunes unused Docker resources
- Logs maintenance summary
### Manual Commands
```bash
# View recent maintenance activity
ssh jared@15.204.247.153 "tail -50 ~/services/maintenance.log"
# Run maintenance manually (caution: will restart containers)
ssh jared@15.204.247.153 "~/services/daily-maintenance.sh"
# View maintenance schedule
ssh jared@15.204.247.153 "crontab -l | grep maintenance"
```
## Cron Configuration
To view or modify the cron schedule:
```bash
# View current cron jobs
ssh jared@15.204.247.153 "crontab -l"
# Edit cron schedule (if needed)
ssh jared@15.204.247.153 "crontab -e"
```
**Current Schedule:**
```cron
# Health monitor - every 10 minutes
*/10 * * * * /home/jared/services/container-health-monitor.sh
# Daily maintenance - 3 AM EST
0 3 * * * /home/jared/services/daily-maintenance.sh
```
## Log Management
### Log Locations
- Health monitor: `~/services/monitor.log`
- Daily maintenance: `~/services/maintenance.log`
- Docker logs: `docker logs CONTAINER_NAME`
- Traefik logs: `~/services/traefik/logs/`
### Log Analysis Commands
```bash
# Count container restarts today
ssh jared@15.204.247.153 "grep 'restarting' ~/services/monitor.log | grep $(date +%Y-%m-%d) | wc -l"
# Find memory-related restarts
ssh jared@15.204.247.153 "grep 'memory exceeds' ~/services/monitor.log | tail -20"
# View maintenance summaries
ssh jared@15.204.247.153 "grep 'Maintenance complete' ~/services/maintenance.log | tail -10"
# Check for errors in logs
ssh jared@15.204.247.153 "grep -i error ~/services/monitor.log | tail -20"
```
### Log Rotation
Logs are automatically rotated via system logrotate configuration to prevent disk space issues.
**Check logrotate config:**
```bash
ssh jared@15.204.247.153 "cat /etc/logrotate.d/container-monitoring"
```
## Alerting
### Current Status
**Status:** Basic monitoring only (no external alerting)
### Future Improvements
Consider adding:
- Email notifications for critical events
- Slack/Discord webhooks for alerts
- Prometheus + Grafana for metrics visualization
- Uptime monitoring (UptimeRobot, etc.)
## Troubleshooting
### Container Keeps Restarting
**Symptom:** Health monitor constantly restarts a container
**Possible Causes:**
- Memory leak in application
- Memory threshold too low
- Container crash loop
**Investigation:**
```bash
# Check container logs
ssh jared@15.204.247.153 "docker logs CONTAINER_NAME --tail 100"
# Check memory usage trends
ssh jared@15.204.247.153 "docker stats --no-stream"
# Review health monitor logs
ssh jared@15.204.247.153 "grep CONTAINER_NAME ~/services/monitor.log | tail -20"
```
### Monitoring Script Not Running
**Symptom:** No recent entries in monitor.log
**Possible Causes:**
- Cron service stopped
- Script permissions issue
- Script path incorrect
**Investigation:**
```bash
# Check cron service
ssh jared@15.204.247.153 "systemctl status cron"
# Check script permissions
ssh jared@15.204.247.153 "ls -la ~/services/container-health-monitor.sh"
# Test script manually
ssh jared@15.204.247.153 "~/services/container-health-monitor.sh"
```
### Maintenance Window Issues
**Symptom:** 3 AM maintenance causing problems
**Solution:** Adjust maintenance time
```bash
# Edit cron schedule
ssh jared@15.204.247.153 "crontab -e"
# Change from 3 AM to different time (example: 2 AM)
# From: 0 3 * * * /home/jared/services/daily-maintenance.sh
# To: 0 2 * * * /home/jared/services/daily-maintenance.sh
```
## Health Check Endpoints
Services should implement health check endpoints for better monitoring:
**Example Docker Compose health check:**
```yaml
services:
myservice:
image: myapp:latest
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
```
## Monitoring Best Practices
1. **Review logs weekly** - Check for unusual patterns
2. **Monitor disk space** - Ensure logs aren't filling disk
3. **Track restart frequency** - Frequent restarts indicate problems
4. **Update scripts** - Keep monitoring scripts current
5. **Test manually** - Periodically run scripts manually to verify
## Custom Monitoring Scripts
To add custom monitoring for specific services:
```bash
# Create custom script
ssh jared@15.204.247.153 "nano ~/services/custom-monitor.sh"
# Make executable
ssh jared@15.204.247.153 "chmod +x ~/services/custom-monitor.sh"
# Add to cron
ssh jared@15.204.247.153 "crontab -e"
# Add: */15 * * * * /home/jared/services/custom-monitor.sh
```
## System Health Commands
In addition to automated monitoring, these commands provide quick system health checks:
```bash
# Overall system health (custom alias)
ssh jared@15.204.247.153 "syshealth"
# Log summary (custom alias)
ssh jared@15.204.247.153 "logsummary"
# Docker resource usage
ssh jared@15.204.247.153 "docker system df"
# Disk usage
ssh jared@15.204.247.153 "df -h"
# Memory usage
ssh jared@15.204.247.153 "free -h"
# Load average
ssh jared@15.204.247.153 "uptime"
```
## See Also
- [Services Inventory](./services-inventory.md) - Current services being monitored
- [CLAUDE.md](../CLAUDE.md) - General server operations
- [Documentation Maintenance](./documentation-maintenance.md) - Keeping docs current

81
docs/plans/2026-03-18-caprover-migration-design.md Normal file
View File

@ -0,0 +1,81 @@
# CapRover Migration Design
**Date:** 2026-03-18
**Status:** Approved
## Overview
Migrate services from CapRover server to the OVH production server (15.204.247.153). All services will be Docker Compose, integrated with Traefik, and auto-discovered by the existing backup script. Obsidian is excluded (using official Obsidian Sync). Portainer, changedetection, and heimdall are not being ported.
## Services
| Batch | Service | Domain | Data |
|-------|---------|--------|------|
| 0 | Watchtower | none | new (stateless) |
| 1 | Vaultwarden | bitwarden.swansoncloud.com | migrate data dir from CapRover |
| 1 | n8n | n8n.swansoncloud.com | fresh install |
| 2 | Forgejo | forgejo.swansoncloud.com | fresh install + 2 repos migrated from gitea.swansoncloud.com |
| 2 | NocoDB | nocodb.swansoncloud.com | fresh install |
## Architecture
### Watchtower
- Image: `nicholas-fedor/watchtower` (active community fork of the archived containrrr/watchtower)
- Stateless — no volumes, no domain, no backup needed
- Monitors all containers on a nightly schedule
- Rationale for fork: original containrrr/watchtower archived December 2025; this fork is actively maintained with weekly releases
### Vaultwarden
- Image: `vaultwarden/server`
- Single container, SQLite data dir (`./data`) — no separate database needed for personal use
- Traefik handles SSL termination
- Backup auto-discovered by existing `~/services/backup.sh`
### n8n
- Image: `n8nio/n8n`
- Fresh install — nothing worth preserving from CapRover instance
- Match pattern of existing n8n instances already on this server
- Backup auto-discovered
### Forgejo
- Image: `codeberg.org/forgejo/forgejo`
- SQLite — adequate for 2 repos, personal use
- Fresh install; repos migrated using Forgejo's built-in migration tool (compatible with Gitea)
- Domain: `forgejo.swansoncloud.com` (replaces `gitea.swansoncloud.com`)
- Backup auto-discovered
### NocoDB
- Image: `nocodb/nocodb`
- Fresh install, SQLite built-in
- No data to migrate
- Backup auto-discovered
## Vaultwarden Data Migration Steps
1. SSH into CapRover server, locate the Vaultwarden persistent data directory
2. `scp` the `data/` dir from CapRover to OVH at `~/services/vaultwarden/data/`
3. Deploy Vaultwarden on OVH
4. Test by accessing via direct IP or temporary subdomain — confirm vault unlocks and data is intact
5. Update `bitwarden.swansoncloud.com` DNS → OVH IP (15.204.247.153)
6. Verify browser extensions reconnect
## Forgejo Repo Migration
Forgejo's built-in migration tool can pull repos directly from the live Gitea instance at `gitea.swansoncloud.com`. Run migration from the new Forgejo web UI after deployment — no manual download needed.
## DNS Cutover Order
Cutover per-service independently once verified: Vaultwarden → n8n → Forgejo → NocoDB.
Old CapRover server stays up (read-only) until all four DNS records are confirmed healthy, then decommission.
## Post-Migration Tasks
- Set up Fireflies.ai meeting transcript backup automation in n8n
- Set up hyperthrive.io contact form automation in n8n
## Design Decisions
- **Vaultwarden over official Bitwarden server**: Official Bitwarden is a heavy .NET/SQL Server stack. Vaultwarden is ~10MB, supports all Bitwarden clients and premium features (TOTP, attachments) for free.
- **SQLite over PostgreSQL**: All services are single-user personal use. SQLite is simpler, lower overhead, and still auto-discovered by backup.sh. Can migrate to PostgreSQL later if needed.
- **nicholas-fedor/watchtower over Diun**: Backups are in place; fully automatic updates are acceptable. Diun (notification-only) adds operational overhead without meaningful benefit given the backup safety net.
- **Modular service composition**: Each service is an independent Docker Compose stack under `~/services/SERVICE_NAME/`, consistent with existing server patterns.

568
docs/plans/2026-03-18-caprover-migration.md Normal file
View File

@ -0,0 +1,568 @@
# CapRover Migration Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Migrate Vaultwarden, n8n, Forgejo, and NocoDB from CapRover to the OVH production server, plus add Watchtower for automatic container updates.
**Architecture:** Each service is an independent Docker Compose stack under `~/services/SERVICE_NAME/` on the OVH server (15.204.247.153). Traefik handles SSL termination and routing. The existing `~/services/backup.sh` auto-discovers all services — no backup configuration needed per service. Services are deployed in two batches: Watchtower first, then Vaultwarden + n8n in parallel, then Forgejo + NocoDB in parallel.
**Tech Stack:** Docker Compose, Traefik v3.0, Let's Encrypt, nicholas-fedor/watchtower, vaultwarden/server, n8nio/n8n, codeberg.org/forgejo/forgejo, nocodb/nocodb
---
## Batch 0: Watchtower
### Task 1: Create Watchtower compose file
**Files:**
- Create: `configs/watchtower/docker-compose.yml` (in this repo, for reference)
**Step 1: Create the compose file locally**
```yaml
# configs/watchtower/docker-compose.yml
services:
watchtower:
image: nicholas-fedor/watchtower
restart: unless-stopped
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- WATCHTOWER_SCHEDULE=0 0 3 * * *
- WATCHTOWER_CLEANUP=true
- WATCHTOWER_INCLUDE_STOPPED=false
- WATCHTOWER_NOTIFICATIONS_LEVEL=warn
command: --schedule "0 0 3 * * *"
```
Note: No networks needed — Watchtower talks to the Docker daemon directly via the socket. Runs at 3 AM daily. `WATCHTOWER_CLEANUP=true` removes old images after updating.
**Step 2: Commit the config**
```bash
git add configs/watchtower/docker-compose.yml
git commit -m "chore: add watchtower compose config"
```
---
### Task 2: Deploy Watchtower on OVH
**Step 1: Create service directory on remote server**
```bash
ssh jared@15.204.247.153 "mkdir -p ~/services/watchtower"
```
**Step 2: Copy compose file to server**
```bash
scp configs/watchtower/docker-compose.yml jared@15.204.247.153:~/services/watchtower/
```
**Step 3: Start Watchtower**
```bash
ssh jared@15.204.247.153 "cd ~/services/watchtower && docker compose up -d"
```
**Step 4: Verify it's running**
```bash
ssh jared@15.204.247.153 "docker ps | grep watchtower"
```
Expected: one `watchtower` container in `Up` state.
**Step 5: Check logs to confirm it's watching containers**
```bash
ssh jared@15.204.247.153 "docker logs watchtower --tail 20"
```
Expected: logs showing it has found and is monitoring existing containers.
---
## Batch 1: Vaultwarden + n8n (deploy in parallel)
### Task 3: Create Vaultwarden compose file
**Files:**
- Create: `configs/vaultwarden/docker-compose.yml`
**Step 1: Create the compose file**
```yaml
# configs/vaultwarden/docker-compose.yml
services:
vaultwarden:
image: vaultwarden/server:latest
restart: unless-stopped
volumes:
- ./data:/data
networks:
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.vaultwarden.rule=Host(`bitwarden.swansoncloud.com`)"
- "traefik.http.routers.vaultwarden.entrypoints=websecure"
- "traefik.http.routers.vaultwarden.tls.certresolver=letsencrypt"
- "traefik.http.services.vaultwarden.loadbalancer.server.port=80"
- "traefik.docker.network=traefik"
networks:
traefik:
external: true
```
Note: Modern Vaultwarden (post-1.25) handles WebSocket on port 80 — no separate router needed for `/notifications/hub`.
**Step 2: Commit**
```bash
git add configs/vaultwarden/docker-compose.yml
git commit -m "chore: add vaultwarden compose config"
```
---
### Task 4: Migrate Vaultwarden data from CapRover
**Step 1: SSH into the CapRover server and locate the Vaultwarden data directory**
The CapRover persistent data path is typically at:
`/var/lib/docker/volumes/captain--vaultwarden--1-data/_data/`
SSH into CapRover and confirm:
```bash
# On CapRover server
ls /var/lib/docker/volumes/ | grep vaultwarden
ls /var/lib/docker/volumes/captain--vaultwarden--1-data/_data/
```
Expected contents: `db.sqlite3`, `config.json`, `attachments/`, `sends/`
**Step 2: Copy data dir from CapRover to OVH**
Create the destination directory on OVH first:
```bash
ssh jared@15.204.247.153 "mkdir -p ~/services/vaultwarden/data"
```
Then from CapRover server (or via local machine as relay):
```bash
# Option A: If you can SSH from CapRover to OVH directly
scp -r /var/lib/docker/volumes/captain--vaultwarden--1-data/_data/* jared@15.204.247.153:~/services/vaultwarden/data/
# Option B: Via your local machine (two-hop)
# First download from CapRover to local:
scp -r root@CAPROVER_IP:/var/lib/docker/volumes/captain--vaultwarden--1-data/_data/ ./vaultwarden-backup/
# Then upload to OVH:
scp -r ./vaultwarden-backup/ jared@15.204.247.153:~/services/vaultwarden/data/
```
**Step 3: Verify files landed on OVH**
```bash
ssh jared@15.204.247.153 "ls -la ~/services/vaultwarden/data/"
```
Expected: `db.sqlite3` present, `config.json` present.
---
### Task 5: Deploy Vaultwarden on OVH
**Step 1: Copy compose file to server**
```bash
scp configs/vaultwarden/docker-compose.yml jared@15.204.247.153:~/services/vaultwarden/
```
**Step 2: Start Vaultwarden**
```bash
ssh jared@15.204.247.153 "cd ~/services/vaultwarden && docker compose up -d"
```
**Step 3: Verify container is running**
```bash
ssh jared@15.204.247.153 "docker ps | grep vaultwarden"
```
**Step 4: Test via temporary access (before DNS cutover)**
Access via direct IP to confirm Traefik is routing it (will show SSL warning, that's fine):
```
http://15.204.247.153
```
Or check Traefik dashboard at https://traefik.hyperthrive.app to confirm the `vaultwarden` router appears.
**Step 5: Check logs for errors**
```bash
ssh jared@15.204.247.153 "docker logs vaultwarden-vaultwarden-1 --tail 30"
```
Expected: No errors. Should see startup messages and database initialization.
---
### Task 6: Cut DNS for Vaultwarden + verify
**Step 1: Update DNS**
In your DNS provider, update `bitwarden.swansoncloud.com` A record to point to `15.204.247.153`.
**Step 2: Wait for DNS propagation (~2-5 minutes), then test**
```bash
curl -I https://bitwarden.swansoncloud.com
```
Expected: HTTP 200 response.
**Step 3: Verify vault unlocks**
Open https://bitwarden.swansoncloud.com in browser — confirm you can log in and your vault data is present.
**Step 4: Test browser extension**
Re-connect your Bitwarden browser extension to the new server URL and confirm it syncs.
---
### Task 7: Create n8n compose file
**Files:**
- Create: `configs/n8n-swanson/docker-compose.yml`
**Step 1: Check existing n8n instances on OVH for pattern consistency**
```bash
ssh jared@15.204.247.153 "cat ~/services/n8n*/docker-compose.yml 2>/dev/null || ls ~/services/ | grep n8n"
```
Adapt the compose below to match any conventions found.
**Step 2: Create the compose file**
```yaml
# configs/n8n-swanson/docker-compose.yml
services:
n8n:
image: n8nio/n8n
restart: unless-stopped
environment:
- N8N_HOST=n8n.swansoncloud.com
- N8N_PORT=5678
- N8N_PROTOCOL=https
- WEBHOOK_URL=https://n8n.swansoncloud.com/
- N8N_SECURE_COOKIE=false
- TZ=America/Chicago
volumes:
- ./data:/home/node/.n8n
networks:
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.n8n-swanson.rule=Host(`n8n.swansoncloud.com`)"
- "traefik.http.routers.n8n-swanson.entrypoints=websecure"
- "traefik.http.routers.n8n-swanson.tls.certresolver=letsencrypt"
- "traefik.http.services.n8n-swanson.loadbalancer.server.port=5678"
- "traefik.docker.network=traefik"
networks:
traefik:
external: true
```
Note: Router name is `n8n-swanson` (not just `n8n`) to avoid collision with existing n8n instances on this server.
**Step 3: Commit**
```bash
git add configs/n8n-swanson/docker-compose.yml
git commit -m "chore: add n8n swansoncloud compose config"
```
---
### Task 8: Deploy n8n on OVH
**Step 1: Create service directory and deploy**
```bash
ssh jared@15.204.247.153 "mkdir -p ~/services/n8n-swanson"
scp configs/n8n-swanson/docker-compose.yml jared@15.204.247.153:~/services/n8n-swanson/
ssh jared@15.204.247.153 "cd ~/services/n8n-swanson && docker compose up -d"
```
**Step 2: Verify container is running**
```bash
ssh jared@15.204.247.153 "docker ps | grep n8n-swanson"
```
**Step 3: Cut DNS**
Update `n8n.swansoncloud.com` A record → `15.204.247.153`.
**Step 4: Verify n8n is accessible**
```bash
curl -I https://n8n.swansoncloud.com
```
Expected: HTTP 200. Open in browser and complete initial setup (create admin account).
---
## Batch 2: Forgejo + NocoDB (deploy in parallel)
### Task 9: Create Forgejo compose file
**Files:**
- Create: `configs/forgejo/docker-compose.yml`
**Step 1: Create the compose file**
```yaml
# configs/forgejo/docker-compose.yml
services:
forgejo:
image: codeberg.org/forgejo/forgejo:latest
restart: unless-stopped
environment:
- USER_UID=1000
- USER_GID=1000
volumes:
- ./data:/data
networks:
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.forgejo.rule=Host(`forgejo.swansoncloud.com`)"
- "traefik.http.routers.forgejo.entrypoints=websecure"
- "traefik.http.routers.forgejo.tls.certresolver=letsencrypt"
- "traefik.http.services.forgejo.loadbalancer.server.port=3000"
- "traefik.docker.network=traefik"
networks:
traefik:
external: true
```
Note: SQLite is the default and sufficient for 2 repos. No external database needed. SSH git access is omitted — HTTPS is fine for personal use.
**Step 2: Commit**
```bash
git add configs/forgejo/docker-compose.yml
git commit -m "chore: add forgejo compose config"
```
---
### Task 10: Deploy Forgejo on OVH
**Step 1: Create service directory and deploy**
```bash
ssh jared@15.204.247.153 "mkdir -p ~/services/forgejo"
scp configs/forgejo/docker-compose.yml jared@15.204.247.153:~/services/forgejo/
ssh jared@15.204.247.153 "cd ~/services/forgejo && docker compose up -d"
```
**Step 2: Verify container is running**
```bash
ssh jared@15.204.247.153 "docker ps | grep forgejo"
```
**Step 3: Cut DNS**
Update `forgejo.swansoncloud.com` A record → `15.204.247.153`.
**Step 4: Access web UI and complete initial setup**
Open https://forgejo.swansoncloud.com — Forgejo will show a first-run setup page. Configure:
- Database: SQLite (default)
- Site URL: `https://forgejo.swansoncloud.com`
- Admin username/password
---
### Task 11: Migrate Gitea repos to Forgejo
**Step 1: In Forgejo web UI, use the built-in migration tool**
Go to: https://forgejo.swansoncloud.com → `+` (New) → `Migrate Repository`
Select: **Gitea** as the source type.
**Step 2: For each repo at gitea.swansoncloud.com/explore/repos**
Fill in:
- Clone URL: `https://gitea.swansoncloud.com/USERNAME/REPO_NAME.git`
- If Gitea requires auth, provide credentials
Forgejo will pull the full git history, issues, and wiki (if any).
Repeat for all repos (there are 2).
**Step 3: Verify repos migrated**
Confirm both repos appear at https://forgejo.swansoncloud.com with full commit history.
**Step 4: Update any local git remotes**
On your local machine, update remotes from `gitea.swansoncloud.com` → `forgejo.swansoncloud.com`:
```bash
git remote set-url origin https://forgejo.swansoncloud.com/USERNAME/REPO_NAME.git
```
---
### Task 12: Create NocoDB compose file
**Files:**
- Create: `configs/nocodb/docker-compose.yml`
**Step 1: Create the compose file**
```yaml
# configs/nocodb/docker-compose.yml
services:
nocodb:
image: nocodb/nocodb:latest
restart: unless-stopped
volumes:
- ./data:/usr/app/data
networks:
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.nocodb.rule=Host(`nocodb.swansoncloud.com`)"
- "traefik.http.routers.nocodb.entrypoints=websecure"
- "traefik.http.routers.nocodb.tls.certresolver=letsencrypt"
- "traefik.http.services.nocodb.loadbalancer.server.port=8080"
- "traefik.docker.network=traefik"
networks:
traefik:
external: true
```
**Step 2: Commit**
```bash
git add configs/nocodb/docker-compose.yml
git commit -m "chore: add nocodb compose config"
```
---
### Task 13: Deploy NocoDB on OVH
**Step 1: Create service directory and deploy**
```bash
ssh jared@15.204.247.153 "mkdir -p ~/services/nocodb"
scp configs/nocodb/docker-compose.yml jared@15.204.247.153:~/services/nocodb/
ssh jared@15.204.247.153 "cd ~/services/nocodb && docker compose up -d"
```
**Step 2: Verify container is running**
```bash
ssh jared@15.204.247.153 "docker ps | grep nocodb"
```
**Step 3: Cut DNS**
Update `nocodb.swansoncloud.com` A record → `15.204.247.153`.
**Step 4: Verify NocoDB is accessible and complete setup**
Open https://nocodb.swansoncloud.com — create admin account on first run.
---
## Batch 3: Verification & Cleanup
### Task 14: Verify backup coverage for all new services
**Step 1: Run backup script manually**
```bash
ssh jared@15.204.247.153 "~/services/backup.sh"
```
**Step 2: Verify all 4 new services are discovered**
```bash
ssh jared@15.204.247.153 "tar -tzf ~/backups/current/latest.tar.gz | grep -E 'vaultwarden|n8n-swanson|forgejo|nocodb'"
```
Expected: entries for all four services.
---
### Task 15: Update services inventory doc
**Files:**
- Modify: `docs/services-inventory.md`
Add entries for all 4 new services + Watchtower. Follow existing format in the file. Include:
- Service name
- Domain
- Container name(s)
- Image
- Data path on server
- Deployment date (today: 2026-03-18)
- Status: ✅ LIVE
**Commit:**
```bash
git add docs/services-inventory.md
git commit -m "docs: update services inventory with migrated services"
```
---
### Task 16: Decommission CapRover services
Once all DNS records are confirmed healthy and pointing to OVH:
**Step 1: Confirm all 4 services are live on OVH**
```bash
curl -I https://bitwarden.swansoncloud.com
curl -I https://n8n.swansoncloud.com
curl -I https://forgejo.swansoncloud.com
curl -I https://nocodb.swansoncloud.com
```
All should return HTTP 200.
**Step 2: Stop services on CapRover**
In the CapRover dashboard, stop (do not delete yet) Vaultwarden, n8n, Gitea, and NocoDB. Leave them stopped for 48 hours as a rollback window.
**Step 3: After 48 hours, delete CapRover apps and decommission server**
Once confident everything is working, delete the CapRover apps and shut down the CapRover server.
---
## Post-Migration Reminders
These tasks are tracked separately and should be done after all services are live:
1. **Fireflies.ai automation** — Set up n8n workflow to retrieve meeting transcripts from Fireflies.ai and back them up. (See task tracker)
2. **hyperthrive.io contact form** — Set up n8n workflow for contact form submissions on hyperthrive.io. (See task tracker)

130
docs/services-inventory.md Normal file
View File

@ -0,0 +1,130 @@
# Services Inventory
Production server: 15.204.247.153
## Active Services
### n8n (Hyperthrive Production)
- **Status**: Running
- **Domain**: n8n.hyperthrive.io
- **Port**: 5678 (internal), exposed via Traefik HTTPS
- **Docker Network**: traefik, internal
- **Container Image**: docker.n8n.io/n8nio/n8n:1.71.3
- **Database**: PostgreSQL 16 (local service_postgres)
- **Storage**: Named volume n8n_storage
- **Key Environment Variables**:
- N8N_HOST=n8n.hyperthrive.io
- N8N_PROTOCOL=https
- EXECUTIONS_DATA_MAX_AGE=72
- NODE_ENV=production
- **Special Features**:
- Community packages enabled
- Webhook support via HTTPS
- MCP route support with no-buffer middleware
- Health check enabled
- **Traefik Config**:
- Main route: Host(n8n.hyperthrive.io) && !PathPrefix(/mcp)
- MCP route: Host(n8n.hyperthrive.io) && PathPrefix(/mcp) with custom buffering headers
- **Deployed**: 2026-05-14
### n8n-test (Hyperthrive Testing)
- **Status**: Running
- **Domain**: testn8n.hyperthrive.io
- **Port**: 5678 (internal), exposed via Traefik HTTPS
- **Docker Network**: test_n8n_internal, traefik
- **Container Image**: docker.n8n.io/n8nio/n8n:latest
- **Database**: PostgreSQL 16 (local service_postgres)
- **Storage**: Named volume test_n8n_storage
- **Key Environment Variables**:
- N8N_HOST=testn8n.hyperthrive.io
- N8N_PROTOCOL=https
- EXECUTIONS_DATA_MAX_AGE=72
- NODE_ENV=production
- **Purpose**: Testing and staging environment for n8n workflows
- **Traefik Config**:
- Route: Host(testn8n.hyperthrive.io)
- **Deployed**: 2026-05-14
### Formbricks
- **Status**: Running
- **Domain**: formbricks.hyperthrive.io
- **Port**: 3000 (internal), exposed via Traefik HTTPS
- **Docker Network**: formbricks_internal, traefik
- **Container Image**: formbricks/formbricks:2.7.1
- **Database**: PostgreSQL with pgvector (ankane/pgvector:v0.5.1)
- **Storage**:
- App data volume: formbricks_app_data
- Database volume: formbricks_postgres_data
- **Key Environment Variables**:
- WEBAPP_URL=formbricks.hyperthrive.io (https)
- EMAIL_VERIFICATION_DISABLED=1
- Secrets: NEXTAUTH_SECRET, ENCRYPTION_KEY, CRON_SECRET
- **Traefik Config**:
- Route: Host(formbricks.hyperthrive.io)
- TLS via Let's Encrypt
- **Deployed**: 2026-05-14
### n8n-swanson (Personal n8n Instance)
- **Status**: Running
- **Port**: 5678
- **Container Image**: n8nio/n8n:latest
- **Purpose**: Personal/Jared's n8n instance
- **Deployed**: Previous
### Vaultwarden
- **Status**: Running
- **Purpose**: Password manager and vault
- **Deployed**: Previous
### Invoice Ninja
- **Status**: Running
- **Purpose**: Invoicing and billing system
- **Deployed**: Previous
### NocoDB
- **Status**: Running
- **Purpose**: No-code database platform
- **Deployed**: Previous
### LandHomeTeam
- **Status**: Running
- **Purpose**: Business-specific application
- **Deployed**: Previous
### BestSolarTech
- **Status**: Running
- **Purpose**: Business-specific application
- **Deployed**: Previous
### Forgejo
- **Status**: Running
- **Purpose**: Git repository platform
- **Deployed**: Previous
### Traefik
- **Status**: Running
- **Purpose**: Reverse proxy, SSL termination, routing
- **Version**: v3.0+
- **Deployed**: Previous
### Watchtower
- **Status**: Running
- **Purpose**: Automated container updates
- **Deployed**: Previous
## Automation & Utilities
### backup.sh
- Location: ~/services/backup.sh
- Purpose: Automated backup of all service data
- Frequency: Scheduled via cron
### container-health-monitor.sh
- Location: ~/services/container-health-monitor.sh
- Purpose: Monitor container health and generate alerts
- Frequency: Scheduled via cron
### daily-maintenance.sh
- Location: ~/services/daily-maintenance.sh
- Purpose: Daily maintenance tasks
- Frequency: Scheduled via cron

670
docs/superpowers/plans/2026-05-14-migrate-n8n-formbricks-to-ovh.md Normal file
View File

@ -0,0 +1,670 @@
# Service Migration: n8n (prod + test) + Formbricks → OVH
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Migrate three Docker Compose services (n8n production, n8n-test, formbricks) from hyperthrive-docker to OVH (jared@15.204.247.153), preserving all data, coordinated around a user-initiated DNS cutover.
**Architecture:** Each service is stopped on the source to capture a consistent volume snapshot, volumes are exported as tarballs and transferred over SSH, service directories are created on OVH with adapted configs (network and cert resolver names differ between servers), volumes are imported, services are started and health-checked, then the user cuts DNS. Tasks 1, 2, and 3 are fully independent and can run in parallel.
**Tech Stack:** Docker Compose v2, Traefik v3.6, PostgreSQL 16, pgvector v0.5.1, n8n, Formbricks
---
## Config Differences: Source → Target
| Setting | hyperthrive-docker (source) | OVH / 15.204.247.153 (target) |
|---|---|---|
| External Traefik network | `reverse-proxy-nw` | `traefik` |
| Traefik cert resolver | `myresolver` | `letsencrypt` |
| Services path | `~/services/` | `~/services/` |
---
## Task 1: Migrate n8n Production — n8n.hyperthrive.io
**Model tier:** Sonnet
**Files on OVH:**
- Create: `~/services/n8n/docker-compose.yml`
- Create: `~/services/n8n/.env`
- Create: `~/services/n8n/init-data.sh`
**Volumes to migrate:**
- `n8n_db_storage` → PostgreSQL data (postgres container)
- `n8n_n8n_storage` → n8n app data and encryption key (n8n container)
> ⚠️ Both volumes MUST be migrated together. The n8n app volume contains the encryption key used to protect credentials stored in the database. Migrating only the database will leave credentials unreadable.
- [ ] **Step 1: Stop n8n stack on source to get consistent data**
```bash
ssh hyperthrive-docker "cd ~/services/n8n && docker compose stop"
```
Expected: containers stop, no error
- [ ] **Step 2: Export postgres volume from source**
```bash
ssh hyperthrive-docker "docker run --rm -v n8n_db_storage:/data alpine tar czf - -C /data ." > /tmp/n8n_db_storage.tar.gz
```
Expected: file `/tmp/n8n_db_storage.tar.gz` created locally, non-zero size
- [ ] **Step 3: Export n8n app data volume from source**
```bash
ssh hyperthrive-docker "docker run --rm -v n8n_n8n_storage:/data alpine tar czf - -C /data ." > /tmp/n8n_n8n_storage.tar.gz
```
Expected: file `/tmp/n8n_n8n_storage.tar.gz` created locally, non-zero size
- [ ] **Step 4: Restart n8n on source (minimise source downtime)**
```bash
ssh hyperthrive-docker "cd ~/services/n8n && docker compose start"
```
Expected: containers running again
- [ ] **Step 5: Create service directory on OVH**
```bash
ssh jared@15.204.247.153 "mkdir -p ~/services/n8n"
```
- [ ] **Step 6: Write docker-compose.yml on OVH**
```bash
ssh jared@15.204.247.153 "cat > ~/services/n8n/docker-compose.yml" << 'COMPOSE'
volumes:
db_storage:
n8n_storage:
services:
postgres:
image: postgres:16
restart: always
environment:
- POSTGRES_USER
- POSTGRES_PASSWORD
- POSTGRES_DB
- POSTGRES_NON_ROOT_USER
- POSTGRES_NON_ROOT_PASSWORD
volumes:
- db_storage:/var/lib/postgresql/data
- ./init-data.sh:/docker-entrypoint-initdb.d/init-data.sh
healthcheck:
test: ['CMD-SHELL', 'pg_isready -h localhost -U ${POSTGRES_USER} -d ${POSTGRES_DB}']
interval: 5s
timeout: 5s
retries: 10
networks:
- internal
n8n:
image: docker.n8n.io/n8nio/n8n:latest
restart: always
environment:
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PORT=5432
- DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
- DB_POSTGRESDB_USER=${POSTGRES_NON_ROOT_USER}
- DB_POSTGRESDB_PASSWORD=${POSTGRES_NON_ROOT_PASSWORD}
- N8N_HOST=${N8N_HOST}
- N8N_PORT=5678
- N8N_PROTOCOL=https
- N8N_TRUSTED_PROXIES=1
- N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true
- N8N_COMMUNITY_PACKAGES_ALLOW_TOOL_USAGE=true
- EXECUTIONS_DATA_MAX_AGE=72
- NODE_ENV=production
- WEBHOOK_URL=https://${N8N_HOST}
- QUEUE_HEALTH_CHECK_ACTIVE=true
- NODE_FUNCTION_ALLOW_BUILTIN=crypto
healthcheck:
test: ["CMD", "wget", "--no-check-certificate", "--spider", "http://127.0.0.1:5678/healthz"]
interval: 12s
timeout: 10s
retries: 3
start_period: 15s
volumes:
- n8n_storage:/home/node/.n8n
depends_on:
postgres:
condition: service_healthy
networks:
- internal
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.n8n.rule=Host(`n8n.hyperthrive.io`) && !PathPrefix(`/mcp`)"
- "traefik.http.routers.n8n.entrypoints=websecure"
- "traefik.http.routers.n8n.tls.certresolver=letsencrypt"
- "traefik.http.routers.n8n.service=n8n"
- "traefik.http.routers.n8n-mcp.rule=Host(`n8n.hyperthrive.io`) && PathPrefix(`/mcp`)"
- "traefik.http.routers.n8n-mcp.entrypoints=websecure"
- "traefik.http.routers.n8n-mcp.tls.certresolver=letsencrypt"
- "traefik.http.routers.n8n-mcp.service=n8n"
- "traefik.http.routers.n8n-mcp.middlewares=no-buffer@docker"
- "traefik.http.middlewares.no-buffer.headers.customResponseHeaders.X-Accel-Buffering=no"
- "traefik.http.middlewares.no-buffer.headers.customResponseHeaders.Cache-Control=no-cache"
- "traefik.http.middlewares.no-buffer.compress=false"
- "traefik.http.services.n8n.loadbalancer.server.port=5678"
- "traefik.docker.network=traefik"
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
networks:
internal:
traefik:
external: true
COMPOSE
```
- [ ] **Step 7: Write .env on OVH**
```bash
ssh jared@15.204.247.153 "cat > ~/services/n8n/.env" << 'ENV'
POSTGRES_USER=n8n_user_pg
POSTGRES_PASSWORD=JS7FJQaVj8dhPo3j5#G69
POSTGRES_DB=n8n
POSTGRES_NON_ROOT_USER=n8n_non_root_pg
POSTGRES_NON_ROOT_PASSWORD=WzG$9GqluVmywLCiAuqTz
N8N_HOST=n8n.hyperthrive.io
N8N_VERSION=1.71.3
ENV
```
- [ ] **Step 8: Write init-data.sh on OVH**
```bash
ssh jared@15.204.247.153 "cat > ~/services/n8n/init-data.sh" << 'SCRIPT'
#!/bin/bash
set -e;
if [ -n "${POSTGRES_NON_ROOT_USER:-}" ] && [ -n "${POSTGRES_NON_ROOT_PASSWORD:-}" ]; then
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE USER ${POSTGRES_NON_ROOT_USER} WITH PASSWORD '${POSTGRES_NON_ROOT_PASSWORD}';
GRANT ALL PRIVILEGES ON DATABASE ${POSTGRES_DB} TO ${POSTGRES_NON_ROOT_USER};
GRANT CREATE ON SCHEMA public TO ${POSTGRES_NON_ROOT_USER};
EOSQL
else
echo "SETUP INFO: No Environment variables given!"
fi
SCRIPT
ssh jared@15.204.247.153 "chmod +x ~/services/n8n/init-data.sh"
```
- [ ] **Step 9: Transfer volume tarballs to OVH**
```bash
scp /tmp/n8n_db_storage.tar.gz jared@15.204.247.153:/tmp/
scp /tmp/n8n_n8n_storage.tar.gz jared@15.204.247.153:/tmp/
```
Expected: both files appear in `/tmp/` on OVH
- [ ] **Step 10: Create Docker volumes on OVH and import data**
```bash
ssh jared@15.204.247.153 "
docker volume create n8n_db_storage &&
docker volume create n8n_n8n_storage &&
docker run --rm -v n8n_db_storage:/data -v /tmp:/backup alpine sh -c 'tar xzf /backup/n8n_db_storage.tar.gz -C /data' &&
docker run --rm -v n8n_n8n_storage:/data -v /tmp:/backup alpine sh -c 'tar xzf /backup/n8n_n8n_storage.tar.gz -C /data'
"
```
Expected: no errors
- [ ] **Step 11: Start n8n stack on OVH**
```bash
ssh jared@15.204.247.153 "cd ~/services/n8n && docker compose up -d"
```
Expected: postgres starts first (healthcheck passes), then n8n starts
- [ ] **Step 12: Verify n8n is healthy on OVH**
```bash
ssh jared@15.204.247.153 "docker compose -f ~/services/n8n/docker-compose.yml ps"
```
Expected: both containers show `Up` and postgres shows `(healthy)`
```bash
ssh jared@15.204.247.153 "docker exec \$(docker ps -qf name=n8n-n8n) wget -qO- http://127.0.0.1:5678/healthz"
```
Expected: `{"status":"ok"}` or similar 200 response
- [ ] **Step 13: Clean up temp files**
```bash
ssh jared@15.204.247.153 "rm /tmp/n8n_db_storage.tar.gz /tmp/n8n_n8n_storage.tar.gz"
rm /tmp/n8n_db_storage.tar.gz /tmp/n8n_n8n_storage.tar.gz
```
- [ ] **Step 14: Notify user — n8n ready for DNS cutover**
Report: n8n production is running on OVH. User can now point `n8n.hyperthrive.io` to 15.204.247.153. After DNS propagates, stop source: `ssh hyperthrive-docker "cd ~/services/n8n && docker compose stop"`
---
## Task 2: Migrate n8n-test — testn8n.hyperthrive.io
**Model tier:** Sonnet
**Files on OVH:**
- Create: `~/services/n8n-test/docker-compose.yml`
- Create: `~/services/n8n-test/.env`
- Create: `~/services/n8n-test/init-data.sh`
**Volumes to migrate:**
- `n8n-test_test_n8n_db_storage` → PostgreSQL data
- `n8n-test_test_n8n_storage` → n8n app data and encryption key
- [ ] **Step 1: Stop n8n-test stack on source**
```bash
ssh hyperthrive-docker "cd ~/services/n8n-test && docker compose stop"
```
- [ ] **Step 2: Export postgres volume from source**
```bash
ssh hyperthrive-docker "docker run --rm -v n8n-test_test_n8n_db_storage:/data alpine tar czf - -C /data ." > /tmp/n8n_test_db.tar.gz
```
- [ ] **Step 3: Export n8n-test app data volume from source**
```bash
ssh hyperthrive-docker "docker run --rm -v n8n-test_test_n8n_storage:/data alpine tar czf - -C /data ." > /tmp/n8n_test_storage.tar.gz
```
- [ ] **Step 4: Restart n8n-test on source**
```bash
ssh hyperthrive-docker "cd ~/services/n8n-test && docker compose start"
```
- [ ] **Step 5: Create service directory on OVH**
```bash
ssh jared@15.204.247.153 "mkdir -p ~/services/n8n-test"
```
- [ ] **Step 6: Write docker-compose.yml on OVH**
```bash
ssh jared@15.204.247.153 "cat > ~/services/n8n-test/docker-compose.yml" << 'COMPOSE'
volumes:
test_n8n_db_storage:
test_n8n_storage:
services:
postgres:
image: postgres:16
restart: always
environment:
- POSTGRES_USER
- POSTGRES_PASSWORD
- POSTGRES_DB
- POSTGRES_NON_ROOT_USER
- POSTGRES_NON_ROOT_PASSWORD
volumes:
- test_n8n_db_storage:/var/lib/postgresql/data
- ./init-data.sh:/docker-entrypoint-initdb.d/init-data.sh
healthcheck:
test: ['CMD-SHELL', 'pg_isready -h localhost -U ${POSTGRES_USER} -d ${POSTGRES_DB}']
interval: 5s
timeout: 5s
retries: 10
networks:
- test_n8n_internal
n8n:
image: docker.n8n.io/n8nio/n8n:latest
restart: always
environment:
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PORT=5432
- DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
- DB_POSTGRESDB_USER=${POSTGRES_NON_ROOT_USER}
- DB_POSTGRESDB_PASSWORD=${POSTGRES_NON_ROOT_PASSWORD}
- N8N_HOST=${N8N_HOST}
- N8N_PORT=5678
- N8N_PROTOCOL=https
- N8N_TRUSTED_PROXIES=1
- N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true
- N8N_COMMUNITY_PACKAGES_ALLOW_TOOL_USAGE=true
- EXECUTIONS_DATA_MAX_AGE=72
- NODE_ENV=production
- WEBHOOK_URL=https://${N8N_HOST}
- QUEUE_HEALTH_CHECK_ACTIVE=true
healthcheck:
test: ["CMD", "wget", "--no-check-certificate", "--spider", "http://127.0.0.1:5678/healthz"]
interval: 12s
timeout: 10s
retries: 3
start_period: 15s
volumes:
- test_n8n_storage:/home/node/.n8n
depends_on:
postgres:
condition: service_healthy
networks:
- test_n8n_internal
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.testn8n.rule=Host(`testn8n.hyperthrive.io`)"
- "traefik.http.routers.testn8n.entrypoints=websecure"
- "traefik.http.routers.testn8n.tls.certresolver=letsencrypt"
- "traefik.http.routers.testn8n.service=testn8n"
- "traefik.http.services.testn8n.loadbalancer.server.port=5678"
- "traefik.docker.network=traefik"
logging:
driver: "json-file"
options:
max-size: "10m"
max-file: "3"
networks:
test_n8n_internal:
traefik:
external: true
COMPOSE
```
- [ ] **Step 7: Write .env on OVH**
```bash
ssh jared@15.204.247.153 "cat > ~/services/n8n-test/.env" << 'ENV'
POSTGRES_USER=n8n_test_user_pg
POSTGRES_PASSWORD=l5t2xd8f%&RwB8aidK#2M
POSTGRES_DB=n8n_test
POSTGRES_NON_ROOT_USER=n8n_test_non_root_pg
POSTGRES_NON_ROOT_PASSWORD=isqjKb5s0NWa&w*N@aGbO
N8N_HOST=testn8n.hyperthrive.io
N8N_VERSION=latest
ENV
```
- [ ] **Step 8: Write init-data.sh on OVH**
```bash
ssh jared@15.204.247.153 "cat > ~/services/n8n-test/init-data.sh" << 'SCRIPT'
#!/bin/bash
set -e;
if [ -n "${POSTGRES_NON_ROOT_USER:-}" ] && [ -n "${POSTGRES_NON_ROOT_PASSWORD:-}" ]; then
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE USER ${POSTGRES_NON_ROOT_USER} WITH PASSWORD '${POSTGRES_NON_ROOT_PASSWORD}';
GRANT ALL PRIVILEGES ON DATABASE ${POSTGRES_DB} TO ${POSTGRES_NON_ROOT_USER};
GRANT CREATE ON SCHEMA public TO ${POSTGRES_NON_ROOT_USER};
EOSQL
else
echo "SETUP INFO: No Environment variables given!"
fi
SCRIPT
ssh jared@15.204.247.153 "chmod +x ~/services/n8n-test/init-data.sh"
```
- [ ] **Step 9: Transfer volume tarballs to OVH**
```bash
scp /tmp/n8n_test_db.tar.gz jared@15.204.247.153:/tmp/
scp /tmp/n8n_test_storage.tar.gz jared@15.204.247.153:/tmp/
```
- [ ] **Step 10: Create Docker volumes on OVH and import data**
```bash
ssh jared@15.204.247.153 "
docker volume create n8n-test_test_n8n_db_storage &&
docker volume create n8n-test_test_n8n_storage &&
docker run --rm -v n8n-test_test_n8n_db_storage:/data -v /tmp:/backup alpine sh -c 'tar xzf /backup/n8n_test_db.tar.gz -C /data' &&
docker run --rm -v n8n-test_test_n8n_storage:/data -v /tmp:/backup alpine sh -c 'tar xzf /backup/n8n_test_storage.tar.gz -C /data'
"
```
- [ ] **Step 11: Start n8n-test stack on OVH**
```bash
ssh jared@15.204.247.153 "cd ~/services/n8n-test && docker compose up -d"
```
- [ ] **Step 12: Verify n8n-test is healthy on OVH**
```bash
ssh jared@15.204.247.153 "docker compose -f ~/services/n8n-test/docker-compose.yml ps"
```
Expected: both containers `Up`, postgres `(healthy)`
```bash
ssh jared@15.204.247.153 "docker exec \$(docker ps -qf name=n8n-test-n8n) wget -qO- http://127.0.0.1:5678/healthz"
```
Expected: `{"status":"ok"}`
- [ ] **Step 13: Clean up temp files**
```bash
ssh jared@15.204.247.153 "rm /tmp/n8n_test_db.tar.gz /tmp/n8n_test_storage.tar.gz"
rm /tmp/n8n_test_db.tar.gz /tmp/n8n_test_storage.tar.gz
```
- [ ] **Step 14: Notify user — n8n-test ready for DNS cutover**
Report: n8n-test is running on OVH. User can now point `testn8n.hyperthrive.io` to 15.204.247.153. After DNS propagates, stop source: `ssh hyperthrive-docker "cd ~/services/n8n-test && docker compose stop"`
---
## Task 3: Migrate Formbricks — formbricks.hyperthrive.io
**Model tier:** Sonnet
**Files on OVH:**
- Create: `~/services/formbricks/docker-compose.yml`
- Create: `~/services/formbricks/.env`
**Volumes to migrate:**
- `formbricks_app_data` → Formbricks application data (mounted at `/app/data`)
- `formbricks_postgres_data` → PostgreSQL data (pgvector)
- Anonymous uploads volume → mounted at `/home/nextjs/apps/web/uploads` in the running container (may contain user-uploaded files; check size before deciding whether to migrate)
- [ ] **Step 1: Check if the anonymous uploads volume has data worth migrating**
```bash
ssh hyperthrive-docker "docker exec formbricks-formbricks-1 du -sh /home/nextjs/apps/web/uploads 2>/dev/null || echo 'path not found or empty'"
```
If output shows significant data (> a few KB), capture the volume hash for Step 3b. If empty or not found, skip Step 3b.
- [ ] **Step 2: Stop formbricks stack on source**
```bash
ssh hyperthrive-docker "cd ~/services/formbricks && docker compose stop"
```
- [ ] **Step 3: Export formbricks app data volume**
```bash
ssh hyperthrive-docker "docker run --rm -v formbricks_app_data:/data alpine tar czf - -C /data ." > /tmp/formbricks_app_data.tar.gz
```
- [ ] **Step 3b (conditional): Export anonymous uploads volume if it had data**
Get the volume hash from the docker inspect output: `4a21529d8e041b3ad60aa6cd2b821788c4a26a1245696ea90df324fe4a1f7e7a`
```bash
ssh hyperthrive-docker "docker run --rm -v 4a21529d8e041b3ad60aa6cd2b821788c4a26a1245696ea90df324fe4a1f7e7a:/data alpine tar czf - -C /data ." > /tmp/formbricks_uploads.tar.gz
```
Skip this step if Step 1 showed the directory was empty.
- [ ] **Step 4: Export postgres volume**
```bash
ssh hyperthrive-docker "docker run --rm -v formbricks_postgres_data:/data alpine tar czf - -C /data ." > /tmp/formbricks_postgres_data.tar.gz
```
- [ ] **Step 5: Restart formbricks on source**
```bash
ssh hyperthrive-docker "cd ~/services/formbricks && docker compose start"
```
- [ ] **Step 6: Create service directory on OVH**
```bash
ssh jared@15.204.247.153 "mkdir -p ~/services/formbricks"
```
- [ ] **Step 7: Write docker-compose.yml on OVH**
```bash
ssh jared@15.204.247.153 "cat > ~/services/formbricks/docker-compose.yml" << 'COMPOSE'
services:
formbricks:
image: formbricks/formbricks
restart: always
environment:
NEXTAUTH_SECRET: ${NEXTAUTH_SECRET}
ENCRYPTION_KEY: ${ENCRYPTION_KEY}
CRON_SECRET: ${CRON_SECRET}
WEBAPP_URL: ${WEBAPP_URL}
NEXTAUTH_URL: ${WEBAPP_URL}
SHORT_URL_BASE: ${WEBAPP_URL}
DATABASE_URL: postgresql://postgres:${POSTGRES_PASSWORD}@postgres:5432/formbricks
PORT: 3000
EMAIL_VERIFICATION_DISABLED: 1
volumes:
- formbricks_app_data:/app/data
depends_on:
postgres:
condition: service_healthy
networks:
- internal
- traefik
labels:
- "traefik.enable=true"
- "traefik.http.routers.formbricks.rule=Host(`formbricks.hyperthrive.io`)"
- "traefik.http.routers.formbricks.entrypoints=websecure"
- "traefik.http.routers.formbricks.tls.certresolver=letsencrypt"
- "traefik.http.services.formbricks.loadbalancer.server.port=3000"
- "traefik.docker.network=traefik"
postgres:
image: ankane/pgvector:v0.5.1
restart: always
environment:
POSTGRES_DB: formbricks
POSTGRES_USER: postgres
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
volumes:
- formbricks_postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
networks:
- internal
volumes:
formbricks_app_data:
name: formbricks_app_data
formbricks_postgres_data:
name: formbricks_postgres_data
networks:
internal:
name: formbricks_internal
traefik:
external: true
COMPOSE
```
- [ ] **Step 8: Write .env on OVH**
```bash
ssh jared@15.204.247.153 "cat > ~/services/formbricks/.env" << 'ENV'
NEXTAUTH_SECRET=bc366a86606242b11f88d107eecebe4cc83ff421d41d06142d191581f165dc46
ENCRYPTION_KEY=ed8c30d07f82183b0dd9f5a21df1fb5720072768c501e6631531ae73cca9e90e
CRON_SECRET=349af37fef8ccb87377c083c5a2783e45becf04668614a110de127f32ce2c612
POSTGRES_PASSWORD=9d9eXgtFn6gogV0fVWk7BSAF1MJjcFuwfU966sNVE0
WEBAPP_URL=https://formbricks.hyperthrive.io
ENV
```
- [ ] **Step 9: Transfer volume tarballs to OVH**
```bash
scp /tmp/formbricks_app_data.tar.gz jared@15.204.247.153:/tmp/
scp /tmp/formbricks_postgres_data.tar.gz jared@15.204.247.153:/tmp/
# If uploads were exported in Step 3b:
# scp /tmp/formbricks_uploads.tar.gz jared@15.204.247.153:/tmp/
```
- [ ] **Step 10: Create Docker volumes on OVH and import data**
```bash
ssh jared@15.204.247.153 "
docker volume create formbricks_app_data &&
docker volume create formbricks_postgres_data &&
docker run --rm -v formbricks_app_data:/data -v /tmp:/backup alpine sh -c 'tar xzf /backup/formbricks_app_data.tar.gz -C /data' &&
docker run --rm -v formbricks_postgres_data:/data -v /tmp:/backup alpine sh -c 'tar xzf /backup/formbricks_postgres_data.tar.gz -C /data'
"
```
- [ ] **Step 11: Start formbricks stack on OVH**
```bash
ssh jared@15.204.247.153 "cd ~/services/formbricks && docker compose up -d"
```
Expected: postgres starts first (healthcheck), then formbricks
- [ ] **Step 12: Verify formbricks is healthy on OVH**
```bash
ssh jared@15.204.247.153 "docker compose -f ~/services/formbricks/docker-compose.yml ps"
```
Expected: both containers `Up`, postgres `(healthy)`
```bash
ssh jared@15.204.247.153 "curl -sf http://localhost:3000/health 2>/dev/null || docker logs \$(docker ps -qf name=formbricks-formbricks) --tail 20"
```
Expected: 200 response or logs showing server started on port 3000
- [ ] **Step 13: Clean up temp files**
```bash
ssh jared@15.204.247.153 "rm /tmp/formbricks_app_data.tar.gz /tmp/formbricks_postgres_data.tar.gz 2>/dev/null; true"
rm /tmp/formbricks_app_data.tar.gz /tmp/formbricks_postgres_data.tar.gz 2>/dev/null; true
```
- [ ] **Step 14: Notify user — formbricks ready for DNS cutover**
Report: formbricks is running on OVH. User can now point `formbricks.hyperthrive.io` to 15.204.247.153. After DNS propagates, stop source: `ssh hyperthrive-docker "cd ~/services/formbricks && docker compose stop"`
---
## Post-Migration Checklist (after all DNS cutovers)
- [ ] Confirm each domain resolves to 15.204.247.153: `dig n8n.hyperthrive.io`, `dig testn8n.hyperthrive.io`, `dig formbricks.hyperthrive.io`
- [ ] Confirm HTTPS works and certs are valid for all three domains
- [ ] Log into n8n at https://n8n.hyperthrive.io and verify workflows and credentials are intact
- [ ] Log into formbricks at https://formbricks.hyperthrive.io and verify surveys/data are present
- [ ] Stop all three services on hyperthrive-docker once verified
- [ ] Update `docs/services-inventory.md` on OVH with the three new services

227
docs/wordpress-operations.md Normal file
View File

@ -0,0 +1,227 @@
# WordPress Operations Guide
This document provides WordPress-specific operations, configurations, and troubleshooting for WordPress sites running in Docker behind Traefik on systems-prod-01.
## Docker Configuration Best Practices
### MariaDB Version Selection
- **Use MariaDB 10.11** for compatibility with existing WordPress databases
- **Avoid MariaDB 10.7** - may cause redo log format errors during restoration
- MariaDB 10.11 is the recommended LTS version for WordPress
### SSL/HTTPS Behind Traefik
- **Real Simple SSL plugin NOT needed** when behind Traefik
- Disable Real Simple SSL to avoid redirect loops
- Traefik handles all SSL termination automatically
- Add proxy detection configuration to `wp-config.php` (see below)
### Proxy Detection Configuration
Add this to your `wp-config.php` before the "That's all, stop editing!" line:
```php
// Proxy SSL detection for Traefik
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') {
$_SERVER['HTTPS'] = 'on';
}
```
## Using WP-CLI in Docker
### Installing WP-CLI
If WP-CLI is not present in your WordPress container:
```bash
docker exec CONTAINER_NAME bash -c 'curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar && chmod +x wp-cli.phar && mv wp-cli.phar /usr/local/bin/wp'
```
### Common WP-CLI Commands
**List plugins:**
```bash
docker exec CONTAINER_NAME wp --allow-root plugin list
```
**Deactivate a plugin:**
```bash
docker exec CONTAINER_NAME wp --allow-root plugin deactivate PLUGIN_NAME
# Example: Disable Real Simple SSL
docker exec CONTAINER_NAME wp --allow-root plugin deactivate really-simple-ssl
```
**Update WordPress core:**
```bash
docker exec CONTAINER_NAME wp --allow-root core update
```
**Check WordPress version:**
```bash
docker exec CONTAINER_NAME wp --allow-root core version
```
## Database Operations
### Quick WordPress URL Change
When switching between temporary and production domains:
```bash
# Generic command (change to production domain)
ssh jared@15.204.247.153 "docker exec CONTAINER_DB mysql -u DB_USER -pDB_PASS -e \"
USE DATABASE_NAME;
UPDATE wp_options SET option_value = 'https://domain.com' WHERE option_name IN ('siteurl', 'home');
\""
# Verify the change
ssh jared@15.204.247.153 "docker exec CONTAINER_DB mysql -u DB_USER -pDB_PASS -e \"
USE DATABASE_NAME;
SELECT option_name, option_value FROM wp_options WHERE option_name IN ('siteurl', 'home');
\""
```
**Real Example - BestSolarTech:**
```bash
# Change to production domain
ssh jared@15.204.247.153 "docker exec bestsolartech_db mysql -u wordpress -pBST_wp2024_secure -e \"
USE bst_wordpress;
UPDATE wp_vuhx_options SET option_value = 'https://bestsolartech.com' WHERE option_name IN ('siteurl', 'home');
\""
# Verify
ssh jared@15.204.247.153 "docker exec bestsolartech_db mysql -u wordpress -pBST_wp2024_secure -e \"
USE bst_wordpress;
SELECT option_name, option_value FROM wp_vuhx_options WHERE option_name IN ('siteurl', 'home');
\""
```
### Database Access with Skip Grant Tables
If you need to reset database passwords or access without credentials:
```bash
# Stop the current database container
docker stop SERVICE_db
# Start temporary database with skip-grant-tables
docker run -d --name temp_db \
-v ~/services/SERVICE/db_data:/var/lib/mysql \
mariadb:10.11 --skip-grant-tables
# Access the database (no password needed)
docker exec -it temp_db mysql
# After making changes, stop temp container and restart normal one
docker stop temp_db && docker rm temp_db
docker start SERVICE_db
```
### Database Backup
```bash
# Backup WordPress database
docker exec CONTAINER_DB mysqldump -u DB_USER -pDB_PASS DATABASE_NAME > backup.sql
# Example: Backup BestSolarTech
docker exec bestsolartech_db mysqldump -u wordpress -pBST_wp2024_secure bst_wordpress > bestsolartech-backup.sql
```
### Database Restore
```bash
# Restore from backup
cat backup.sql | docker exec -i CONTAINER_DB mysql -u DB_USER -pDB_PASS DATABASE_NAME
# Example: Restore BestSolarTech
cat bestsolartech-backup.sql | docker exec -i bestsolartech_db mysql -u wordpress -pBST_wp2024_secure bst_wordpress
```
## Troubleshooting
### Redirect Loops
**Symptom:** Browser shows "too many redirects" error
**Cause:** Real Simple SSL plugin conflicting with Traefik SSL termination
**Solution:** Disable Real Simple SSL plugin using WP-CLI (see above)
### Database Connection Errors
**Symptom:** "Error establishing database connection"
**Cause:** Database container not ready or wrong credentials
**Solution:**
1. Check database container is running: `docker ps | grep db`
2. Check database logs: `docker logs CONTAINER_DB`
3. Verify credentials in `wp-config.php` match `docker-compose.yml`
### Mixed Content Warnings
**Symptom:** HTTPS page loading HTTP resources
**Cause:** Missing proxy detection in `wp-config.php`
**Solution:** Add proxy detection code (see above)
## Migration Checklist
When migrating a WordPress site to systems-prod-01:
- [ ] Backup WordPress files and database from source
- [ ] Create service directory: `~/services/SITE_NAME`
- [ ] Create `docker-compose.yml` with MariaDB 10.11
- [ ] Add proxy detection code to `wp-config.php`
- [ ] Deploy containers with temporary domain
- [ ] Test HTTPS on temporary domain
- [ ] Disable Real Simple SSL plugin if present
- [ ] Update site URLs in database for production domain
- [ ] Test all functionality (admin panel, forms, etc.)
- [ ] Update DNS for production cutover
- [ ] Monitor for 24 hours after migration
## WordPress-Specific Docker Compose Template
```yaml
services:
wordpress:
image: wordpress:6.7-apache
restart: unless-stopped
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_NAME: wordpress
WORDPRESS_DB_USER: wordpress
WORDPRESS_DB_PASSWORD: ${DB_PASSWORD}
volumes:
- ./wordpress_data:/var/www/html
networks:
- traefik
- internal
labels:
- "traefik.enable=true"
- "traefik.http.routers.SITE.rule=Host(`domain.com`) || Host(`www.domain.com`)"
- "traefik.http.routers.SITE.entrypoints=websecure"
- "traefik.http.routers.SITE.tls.certresolver=letsencrypt"
db:
image: mariadb:10.11
restart: unless-stopped
environment:
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: ${DB_PASSWORD}
MYSQL_RANDOM_ROOT_PASSWORD: '1'
volumes:
- ./db_data:/var/lib/mysql
networks:
- internal
healthcheck:
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
interval: 30s
timeout: 10s
retries: 5
networks:
traefik:
external: true
internal:
external: true
```
## See Also
- [Configuration Reference](./configuration-reference.md) - Docker Compose templates
- [Services Inventory](./services-inventory.md) - Current WordPress sites
- [CLAUDE.md](../CLAUDE.md) - General server operations

74
migrations/completed/bestsolartech.md Normal file
View File

@ -0,0 +1,74 @@
# BestSolarTech.com Migration - COMPLETED
**Migration Date**: 2025-09-19
**Status**: ✅ LIVE IN PRODUCTION
**URL**: https://bestsolartech.com
## Migration Summary
Successfully migrated WordPress site from DigitalOcean (ServerPilot) to OVHcloud VPS using Docker containers.
## Final Configuration
### Docker Services
- **WordPress Container**: `bestsolartech_wp` (WordPress 6.7-php8.3-apache)
- **Database Container**: `bestsolartech_db` (MariaDB 10.11)
- **Reverse Proxy**: Traefik v3.0 (SSL via Let's Encrypt)
### Access Details
- **Production URL**: https://bestsolartech.com
- **Admin Panel**: https://bestsolartech.com/wp-admin/
- **Server IP**: 15.204.247.153
- **Service Path**: `~/services/bestsolartech/`
### Database Configuration
- **Database Name**: bst_wordpress
- **Database User**: wordpress
- **Table Prefix**: wp_vuhx_
- **Database Password**: [Stored in credentials.md]
## Key Learnings
1. **MariaDB Version**: Use 10.11 for WordPress migrations (10.7 caused redo log errors)
2. **SSL Handling**: Disabled Real Simple SSL plugin - Traefik handles SSL termination
3. **Proxy Configuration**: Added proxy detection to wp-config.php for proper HTTPS handling
4. **Variable Escaping**: Use `$$` to escape `$_SERVER` variables in docker-compose.yml
5. **Traefik Rules**: Use `||` operator for multiple host rules in Traefik v3
## Maintenance Commands
### Database URL Updates
```bash
ssh jared@15.204.247.153 "docker exec bestsolartech_db mysql -u wordpress -pBST_wp2024_secure -e \"
USE bst_wordpress;
UPDATE wp_vuhx_options SET option_value = 'https://bestsolartech.com' WHERE option_name IN ('siteurl', 'home');
\""
```
### Container Management
```bash
# View logs
ssh jared@15.204.247.153 "docker logs bestsolartech_wp --tail 50"
# Restart services
ssh jared@15.204.247.153 "cd ~/services/bestsolartech && docker compose restart"
# Update WordPress
ssh jared@15.204.247.153 "cd ~/services/bestsolartech && docker compose pull && docker compose up -d"
```
## Backup Strategy
WordPress files and database are backed up via:
- Docker volumes in `~/services/bestsolartech/`
- WordPress data: `./wordpress/`
- Database data: `./db_data/`
Regular backups should be configured for production data protection.
## Migration Metrics
- **Migration Time**: ~2 hours
- **Downtime**: Zero (used temp domain for testing)
- **Data Migrated**: 1.5GB WordPress files, 599MB database
- **Cost Savings**: Part of overall consolidation from DigitalOcean

188
migrations/completed/landhometeam.md Normal file
View File

@ -0,0 +1,188 @@
# LandHomeTeam.com Migration - COMPLETED
**Migration Date**: 2025-10-08
**Status**: ✅ LIVE IN PRODUCTION
**URL**: https://landhometeam.com
## Migration Summary
Successfully migrated landhometeam.com from WordPress on DigitalOcean (ServerPilot) to a lightweight static "Coming Soon" page on OVHcloud VPS. This eliminates WordPress maintenance overhead, reduces attack surface, and allows decommissioning of the old server.
## Final Configuration
### Docker Services
- **Web Container**: `landhometeam_web` (nginx:alpine)
- **Reverse Proxy**: Traefik v3.0 (SSL via Let's Encrypt)
- **CDN/Proxy**: Cloudflare (DDoS protection, global caching)
### Access Details
- **Production URL**: https://landhometeam.com
- **Server IP**: 15.204.247.153
- **Service Path**: `~/services/landhometeam/`
- **Resources**: ~10-20MB RAM (minimal footprint)
### Architecture
- Static HTML served from read-only volume mount
- Automatic HTTP → HTTPS redirect via Traefik
- Cloudflare proxy enabled (SSL mode: Full)
- Both apex and www domains configured
## Key Learnings
1. **Traefik Middleware Requirement**: The `redirect-to-https@file` middleware must exist in `dynamic.yml` before being referenced in Docker labels
- Added redirect-to-https middleware to Traefik's dynamic.yml
- Without it, causes configuration errors or unexpected behavior
2. **Cloudflare Proxy Benefits**: Keep Cloudflare proxy enabled (orange cloud) for:
- DDoS protection
- Global CDN caching
- Origin IP hiding
- Free WAF and bot protection
3. **Cloudflare SSL Mode**: Must use "Full" or "Full (strict)" mode when proxying to HTTPS backend
- "Flexible" mode causes redirect loops
- Server uses Let's Encrypt via Traefik, Cloudflare proxies with its own cert
4. **DNS vs IP Testing**: Testing via direct IP with Host header (`curl -H "Host: domain.com" https://IP`) helps isolate DNS/proxy issues
5. **Static Site Simplicity**: No database, no PHP, no admin panel = zero maintenance and minimal security risk
## Deployment Files
### docker-compose.yml
```yaml
services:
nginx:
image: nginx:alpine
container_name: landhometeam_web
restart: unless-stopped
volumes:
- ./html:/usr/share/nginx/html:ro
networks:
- traefik
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik"
# HTTP Router - redirect to HTTPS
- "traefik.http.routers.landhometeam.rule=Host(`landhometeam.com`) || Host(`www.landhometeam.com`)"
- "traefik.http.routers.landhometeam.entrypoints=web"
- "traefik.http.routers.landhometeam.middlewares=redirect-to-https@file"
# HTTPS Router
- "traefik.http.routers.landhometeam-secure.rule=Host(`landhometeam.com`) || Host(`www.landhometeam.com`)"
- "traefik.http.routers.landhometeam-secure.entrypoints=websecure"
- "traefik.http.routers.landhometeam-secure.tls=true"
- "traefik.http.routers.landhometeam-secure.tls.certresolver=letsencrypt"
- "traefik.http.services.landhometeam.loadbalancer.server.port=80"
networks:
traefik:
external: true
```
### Traefik Dynamic Configuration Addition
Added to `~/services/traefik/dynamic.yml`:
```yaml
redirect-to-https:
redirectScheme:
scheme: https
permanent: true
```
## Troubleshooting: Redirect Loop
**Problem**: Redirect loop error when accessing via Cloudflare proxy
**Root Causes**:
1. Missing `redirect-to-https` middleware in Traefik's dynamic.yml
2. Cloudflare SSL mode potentially set to "Flexible"
**Solution**:
1. Added redirect middleware to Traefik configuration
2. Set Cloudflare SSL/TLS mode to "Full"
3. Restarted Traefik: `cd ~/services/traefik && docker compose restart`
## Maintenance Commands
### View Container Status
```bash
ssh jared@15.204.247.153 "docker ps | grep landhometeam"
```
### View Logs
```bash
ssh jared@15.204.247.153 "docker logs landhometeam_web --tail 50"
```
### Update HTML Content
```bash
# Edit locally
nano deployments/landhometeam/html/index.html
# Deploy changes
scp deployments/landhometeam/html/index.html jared@15.204.247.153:~/services/landhometeam/html/
# No restart needed - changes are live immediately (Nginx serves from volume)
```
### Container Management
```bash
# Restart container
ssh jared@15.204.247.153 "cd ~/services/landhometeam && docker compose restart"
# Update Nginx image
ssh jared@15.204.247.153 "cd ~/services/landhometeam && docker compose pull && docker compose up -d"
# Stop/remove service
ssh jared@15.204.247.153 "cd ~/services/landhometeam && docker compose down"
```
## Migration Metrics
- **Migration Time**: ~1 hour (including troubleshooting)
- **Downtime**: Zero (DNS cutover to working site)
- **Resource Usage**: 10-20MB RAM (vs ~200MB+ for WordPress)
- **Monthly Savings**: ~$25+ (can decommission DigitalOcean server + ServerPilot)
- **Annual Savings**: ~$300+
## Post-Migration Tasks
- [x] Deploy container to OVH server
- [x] Configure Traefik routing and SSL
- [x] Update DNS to point to 15.204.247.153
- [x] Configure Cloudflare proxy (SSL mode: Full)
- [x] Fix redirect loop issue
- [x] Verify site loads correctly
- [ ] Monitor for 7 days
- [ ] Export any remaining data from old server
- [ ] Cancel ServerPilot subscription
- [ ] Delete DigitalOcean droplet
## Security Benefits
1. **No Database**: No SQL injection risk
2. **No PHP/WordPress**: No plugin/theme vulnerabilities
3. **Read-Only Volume**: HTML served from read-only mount
4. **Minimal Attack Surface**: Just static HTML + Nginx
5. **Automatic SSL**: Traefik handles certificate renewal
6. **No Admin Panel**: Nothing to hack or brute force
7. **Cloudflare WAF**: Additional layer of DDoS/bot protection
## Future Expansion Options
If site needs to be built out later:
### Option A: Add Contact Form
- Use external service (Formspree, Basin, etc.)
- No backend needed - services email form submissions
### Option B: Static Site Generator
- Build with Astro, Hugo, or similar
- Deploy to same `html/` directory
- Keep same Docker/Traefik setup
### Option C: Full Application
- Add backend container (Node.js, Python, etc.)
- Connect to internal network
- Still behind Traefik with same SSL/routing
All options work with existing architecture - no major infrastructure changes needed.

425
migrations/migration-plan.md Normal file
View File

@ -0,0 +1,425 @@
# Systems Production Server Migration Plan
**STATUS**: 🎉 **Phase 2 COMPLETE** - Ready for Phase 3 Service Migration Planning
## Overview
This document outlines the complete migration plan for consolidating DigitalOcean droplets to the systems production server (systems-prod-01.vps.ovh.us). Each phase is designed to be executed in separate sessions to maintain manageable context windows and reduce risk.
## Migration Strategy
- **Approach**: Incremental service migration with parallel infrastructure
- **Risk Mitigation**: Keep DigitalOcean services running until fully migrated and tested
- **Rollback**: Maintain ability to rollback at each phase
- **Validation**: Test each service thoroughly before DNS cutover
---
## Phase 1: Server Setup & Security Hardening
**Goal**: Establish secure, functional base server ready for service deployment
### Prerequisites
- [x] Server provisioned and documented
- [x] SSH credentials received from OVH via email
- [x] SSH public key ready for deployment
- [x] Domain names identified for services
### Tasks
#### 1.1 Initial Access & User Setup ✅ COMPLETED
- [x] Connect to server using initial credentials: `ssh ubuntu@15.204.247.153`
- [x] Change default ubuntu user password
- [x] Create 'jared' user with sudo privileges
- [x] Copy SSH public key to server
- [x] Test SSH key authentication
- [x] Remove ubuntu user (security enhancement)
#### 1.2 Security Hardening ✅ COMPLETED
- [x] Disable password authentication in SSH config
- [x] Disable root login via SSH
- [x] Configure UFW firewall with basic rules:
- SSH (port 22)
- HTTP (port 80)
- HTTPS (port 443)
- [x] Enable UFW firewall
- [x] Configure automatic security updates
#### 1.3 System Preparation ✅ COMPLETED
- [x] Update all system packages (completed)
- [x] Install essential tools (all required tools installed)
- [x] Configure timezone and NTP (America/New_York, chrony active)
- [x] Set up log rotation (custom configuration created)
- [x] Configure swap file if needed (4GB swap active)
#### 1.4 Validation ✅ COMPLETED
- [x] SSH access working with keys only (verified)
- [x] Firewall rules verified (UFW active, SSH/HTTP/HTTPS)
- [x] System fully updated (all packages current)
- [x] User permissions working correctly (jared user with sudo)
- [x] All services running properly (SSH, chrony, UFW)
### Deliverables
- [x] Updated profile.yaml with actual server configuration
- [x] initial-setup.md runbook documenting exact commands used
- [x] Security checklist completed
- [x] phase1-2-security.sh automated security hardening script
- [x] phase1-2-completion.md documentation
- [x] phase1-3-system-prep.sh automated system preparation script
- [x] phase1-3-manual-commands.md manual command reference
- [x] enable-passwordless-sudo.sh for automation setup
- [x] phase1-3-completion.md status documentation
- [x] phase1-4-completion.md final validation documentation
### Exit Criteria ✅ ALL MET
- [x] Server accessible only via SSH keys
- [x] Firewall properly configured and active
- [x] All security hardening complete
- [x] System updated and ready for software installation
## 🎉 PHASE 1 COMPLETE
**Status**: All Phase 1 tasks successfully completed on 2025-09-18
**Ready for**: Phase 2 Docker Environment Setup
---
## Phase 2: Docker Environment Setup
**Goal**: Install and configure Docker environment for containerized services
### Prerequisites
- [x] Phase 1 completed successfully
- [x] Server accessible and secure
### Tasks
#### 2.1 Docker Installation ✅ COMPLETED
- [x] Install Docker using official script
- [x] Install Docker Compose
- [x] Add jared user to docker group
- [x] Configure Docker daemon settings
- [x] Test Docker installation
#### 2.2 Service Directory Structure ✅ COMPLETED
- [x] Create `/home/jared/services/` directory hierarchy
- [x] Set up proper permissions and ownership
- [x] Create shared Docker networks
- [x] Configure Docker logging
#### 2.3 Traefik Reverse Proxy ✅ COMPLETED
- [x] Create traefik service directory
- [x] Configure Traefik docker-compose.yml
- [x] Set up Let's Encrypt configuration
- [x] Configure Traefik dashboard
- [x] Test Traefik startup and SSL certificate generation
#### 2.4 Base Monitoring ✅ COMPLETED
- [x] Set up basic system monitoring
- [x] Configure log aggregation
- [x] Test monitoring setup
### Deliverables ✅ ALL COMPLETE
- [x] Docker environment fully configured
- [x] Traefik running and obtaining SSL certificates
- [x] Service deployment template created
- [x] Monitoring basics in place
### Exit Criteria ✅ ALL MET
- [x] Docker and Docker Compose working
- [x] Traefik successfully routing traffic and managing SSL
- [x] Basic monitoring operational
- [x] Ready for service deployment
## 🎉 PHASE 2 COMPLETE
**Status**: All Phase 2 tasks successfully completed on 2025-09-18
**Ready for**: Phase 3 Service Migration Planning
---
## Phase 3: Service Migration Planning
**Goal**: Document and prepare for individual service migrations
### Prerequisites
- [x] Phase 2 completed successfully
- [x] Inventory of all DigitalOcean services
### Tasks
#### 3.1 Service Inventory ✅ COMPLETED
- [x] Document all current DigitalOcean services
- [x] Map service dependencies
- [x] Identify databases and persistent data
- [x] Document current domain configurations
- [x] List environment variables and secrets
#### 3.2 Migration Order Planning ✅ COMPLETED
- [x] Categorize services by risk level
- [x] Define migration order (low to high risk)
- [x] Identify testing requirements for each service
- [x] Plan rollback procedures
#### 3.3 WordPress Sites Migration ✅ COMPLETED
##### BestSolarTech.com ✅ COMPLETED (2025-09-19)
- [x] Migrated WordPress files (1.5GB) and database (599MB) from backup
- [x] Configured Docker Compose with WordPress 6.7 and MariaDB 10.11
- [x] Disabled Real Simple SSL plugin (handled by Traefik now)
- [x] Added proxy SSL detection to wp-config.php
- [x] Tested on https://temp.bestsolartech.com with HTTPS working
- [x] DNS cutover completed - live on https://bestsolartech.com
- [x] Admin panel fully functional
- **Status**: ✅ LIVE IN PRODUCTION
#### 3.4 DNS Preparation
- [ ] Document current DNS configurations
- [ ] Plan DNS cutover strategy
- [ ] Set up DNS records for new server (with temporary domains)
- [ ] Reduce TTL values for production domains
#### 3.4 Data Migration Strategy
- [ ] Plan database migration approaches
- [ ] Identify file storage migration needs
- [ ] Plan backup and restoration procedures
- [ ] Test data migration with non-critical services
### Deliverables
- [ ] Complete service inventory with dependencies
- [ ] Migration order and timeline
- [ ] DNS migration plan
- [ ] Data migration procedures
### Exit Criteria
- All services documented and prioritized
- Migration procedures defined and tested
- DNS infrastructure ready for cutover
- Ready to begin service-by-service migration
---
## Phase 4: Low-Risk Service Migration
**Goal**: Migrate static websites and development instances
### Prerequisites
- [x] Phase 3 completed successfully
- [ ] Test migration procedures validated
### Target Services (Examples)
- Static websites
- Development/staging instances
- Non-critical applications
### Tasks
#### 4.1 First Service Migration
- [ ] Choose lowest-risk service for initial migration
- [ ] Deploy service on systems production server
- [ ] Configure Traefik routing
- [ ] Test functionality on temporary domain
- [ ] Update DNS for service cutover
- [ ] Verify service working on production domain
- [ ] Monitor for issues
#### 4.2 Additional Low-Risk Services
- [ ] Repeat migration process for each low-risk service
- [ ] Document any issues and solutions
- [ ] Refine migration procedures
#### 4.3 Validation & Cleanup
- [ ] Verify all migrated services functioning
- [ ] Begin cleanup of DigitalOcean resources for migrated services
- [ ] Update documentation
### Deliverables
- [ ] Low-risk services successfully migrated
- [ ] Refined migration procedures
- [ ] Lessons learned documented
### Exit Criteria
- All low-risk services migrated and functional
- Migration procedures proven and documented
- Ready for medium-risk service migration
---
## Phase 5: Medium-Risk Service Migration
**Goal**: Migrate applications with minimal database dependencies
### Prerequisites
- [x] Phase 4 completed successfully
- [ ] Migration procedures proven with low-risk services
### Target Services (Examples)
- Simple web applications
- Services with small databases
- Non-critical automation tools
### Tasks
#### 5.1 Database Migration Preparation
- [ ] Set up database services on systems production server
- [ ] Test database backup and restore procedures
- [ ] Plan database migration windows
#### 5.2 Service Migration
- [ ] Migrate each medium-risk service individually
- [ ] Handle database migrations with minimal downtime
- [ ] Test all functionality thoroughly
- [ ] Update DNS and monitor
#### 5.3 Performance Optimization
- [ ] Monitor resource usage
- [ ] Optimize container configurations
- [ ] Tune database performance
### Deliverables
- [ ] Medium-risk services migrated
- [ ] Database migration procedures validated
- [ ] Performance optimizations applied
### Exit Criteria
- All medium-risk services migrated successfully
- Database migration procedures proven
- System performance optimized
- Ready for high-risk service migration
---
## Phase 6: High-Risk Service Migration
**Goal**: Migrate critical production services
### Prerequisites
- [x] Phase 5 completed successfully
- [ ] All procedures tested and validated
- [ ] Rollback plans confirmed
### Target Services (Examples)
- Production n8n automation
- Critical databases
- Production web applications
### Tasks
#### 6.1 Pre-Migration Preparation
- [ ] Schedule maintenance windows
- [ ] Prepare comprehensive rollback procedures
- [ ] Set up monitoring and alerting
- [ ] Create full system backups
#### 6.2 Critical Service Migration
- [ ] Migrate each critical service with maximum care
- [ ] Minimize downtime through careful planning
- [ ] Perform extensive testing
- [ ] Monitor closely post-migration
#### 6.3 Production Validation
- [ ] Verify all critical services functioning
- [ ] Confirm all integrations working
- [ ] Monitor performance and stability
### Deliverables
- [ ] All critical services migrated
- [ ] Full system validation complete
- [ ] Production stability confirmed
### Exit Criteria
- All services successfully migrated
- No critical issues identified
- System performance meeting requirements
- Ready for final cleanup
---
## Phase 7: Final Cleanup & Optimization
**Goal**: Complete migration and optimize new environment
### Prerequisites
- [x] Phase 6 completed successfully
- [ ] All services verified functional
### Tasks
#### 7.1 DigitalOcean Cleanup
- [ ] Take final backups of all DigitalOcean services
- [ ] Destroy DigitalOcean droplets
- [ ] Update billing and account settings
- [ ] Document cost savings achieved
#### 7.2 Final Optimization
- [ ] Optimize resource allocation
- [ ] Implement comprehensive monitoring
- [ ] Set up automated backups
- [ ] Fine-tune security settings
#### 7.3 Documentation Updates
- [ ] Update all server documentation
- [ ] Create operational runbooks
- [ ] Document lessons learned
- [ ] Create disaster recovery procedures
### Deliverables
- [ ] DigitalOcean resources decommissioned
- [ ] Systems production environment fully optimized
- [ ] Complete documentation set
- [ ] Cost savings validated
### Exit Criteria
- Migration completely finished
- All documentation current
- Monitoring and backups operational
- Cost optimization achieved
---
## Risk Management
### Rollback Procedures
1. **Per-Phase Rollback**: Maintain DigitalOcean services until phase completion
2. **Service-Level Rollback**: DNS cutback to DigitalOcean for individual services
3. **Full Rollback**: Complete revert to DigitalOcean if critical issues arise
### Monitoring & Validation
- Service health checks before and after each migration
- Performance monitoring during transition periods
- User impact assessment and communication
### Emergency Contacts
- OVH Support: [Contact information when available]
- DNS Provider Support: [Contact information when available]
- Backup contact methods for critical issues
---
## Success Metrics
### Cost Optimization
- Target: 30-50% reduction in monthly hosting costs
- Measure: Compare DigitalOcean bills before/after migration
### Performance
- Target: Maintain or improve service response times
- Measure: Response time monitoring and comparison
### Reliability
- Target: 99.9% uptime for all migrated services
- Measure: Uptime monitoring and incident tracking
### Security
- Target: No security incidents during or after migration
- Measure: Security audit and monitoring
---
## Timeline Estimation
| Phase | Estimated Duration | Dependencies | Status |
|-------|-------------------|--------------|---------|
| Phase 1 | 1-2 sessions (2-4 hours) | OVH credentials | ✅ COMPLETE (1 day) |
| Phase 2 | 1-2 sessions (2-4 hours) | Phase 1 complete | ✅ COMPLETE (1 day) |
| Phase 3 | 1 session (2-3 hours) | Service inventory | 🔄 NEXT |
| Phase 4 | 2-3 sessions (4-6 hours) | Low-risk services identified | ⏳ PENDING |
| Phase 5 | 3-4 sessions (6-8 hours) | Medium-risk services ready | ⏳ PENDING |
| Phase 6 | 4-5 sessions (8-10 hours) | High-risk service preparation | ⏳ PENDING |
| Phase 7 | 1-2 sessions (2-4 hours) | All services migrated | ⏳ PENDING |
**Total Estimated Time**: 12-19 sessions (26-39 hours)
**Completed**: 2 phases (2 days)
**Remaining**: 5 phases
---
*This plan is a living document and should be updated as the migration progresses.*
*Last updated: 2025-09-18*
*Phase 1 Status: ✅ COMPLETE - All tasks successfully finished*
*Phase 2 Status: ✅ COMPLETE - Docker environment ready*
*Ready for Phase 3: Service Migration Planning*

157
profile.yaml Normal file
View File

@ -0,0 +1,157 @@
# systems-prod-01 Server Profile
# Last Updated: 2025-09-18
# Updated By: jared
server:
name: systems-prod-01
hostname: systems-prod-01.vps.ovh.us
provider: OVHcloud
location: Virginia (US-EAST-VA) - USA
region: os-us-east-va-2
ip: 15.204.247.153
ipv6: 2604:2dc0:101:200::c3
gateway_ipv6: 2604:2dc0:101:200::1
created: 2025-09-18
status: Active
purpose: Multi-service hosting migration from DigitalOcean for cost optimization
specs:
model: VPS-3
cpu: 8 vCores
ram: 24 GB
storage: 200 GB
bandwidth: TBD
os: Ubuntu 25.04
kernel: 6.14.0-29-generic
arch: x86_64
access:
ssh_user: jared
ssh_port: 22
ssh_key: ~/.ssh/id_rsa.pub (jared@x1carbon)
root_login: disabled
password_auth: disabled
auth_method: ssh_key_only
sudo: full_access
jump_host: N/A
security:
firewall: ufw
firewall_status: active
firewall_rules:
- "22/tcp (OpenSSH) ALLOW IN Anywhere"
- "80/tcp ALLOW IN Anywhere"
- "443/tcp ALLOW IN Anywhere"
- "default: deny (incoming), allow (outgoing)"
selinux: N/A
fail2ban: not_configured
ids: not_configured
auto_updates: enabled (unattended-upgrades)
software:
docker:
version: 28.4.0
compose_version: 2.39.3
user_in_group: jared
installed_from: official_convenience_script
daemon_config: production_ready
networks:
- web (172.20.0.0/16)
- internal (172.21.0.0/16)
runtime:
- traefik:v3.0 (reverse proxy)
- ctop (container monitoring)
- apache2-utils (for htpasswd)
system_packages:
essential_installed: curl, git, vim, htop, iftop, iotop, ncdu, tree, jq, zip, unzip, build-essential, rsync, fail2ban
pending_updates: 3 (held due to phasing)
status: completed
services:
- name: traefik
type: reverse_proxy
version: v3.0
status: running
ports: [80, 443, 8080]
ssl: letsencrypt
dashboard: https://traefik.systems-prod-01.vps.ovh.us
directory: /home/jared/services/traefik
- name: test-whoami
type: test_service
status: running
purpose: ssl_verification
monitoring:
status: basic_configured
tools:
- ctop (container monitoring)
- htop (system monitoring)
- health-check.sh (system health)
- log-summary.sh (log analysis)
logs: local_with_rotation
directory: /home/jared/services/monitoring
alerts: none
backups:
status: not_configured
strategy: TBD
frequency: TBD
retention: TBD
location: TBD
tested: N/A
networking:
vpc: N/A
subnet: N/A
private_ip: N/A
dns: TBD
domain: TBD
cdn: N/A
storage:
volumes:
- mount: /
size: 200 GB
type: SSD
purpose: System and application storage
maintenance:
auto_updates: configured (unattended-upgrades)
last_system_update: 2025-09-18
last_docker_update: N/A
reboot_required: no
maintenance_window: TBD
swap_configured: yes (4GB active)
timezone: America/New_York
ntp_sync: active (chrony)
log_rotation: configured
cost:
monthly: TBD
billing: monthly
credits_used: N/A
notes:
- Server provisioned for cost optimization migration from DigitalOcean
- Significantly larger specs (8 vCores, 24GB RAM) vs current 1 vCore, 2GB setup
- Will consolidate multiple DigitalOcean services onto single server
- Next payment date: 2025-10-18
- Automatic renewal enabled
- Phase 1.1 completed: Custom user setup, SSH keys, ubuntu user removed
- Phase 1.2 completed: Security hardening (SSH, firewall, auto-updates)
- Phase 1.3 completed: System preparation (packages, timezone, swap, tools)
- Phase 1.4 completed: Final validation of all Phase 1 tasks
- PHASE 1 COMPLETE: Ready for Phase 2 Docker environment setup
- Phase 2.1 completed: Docker v28.4.0 and Compose v2.39.3 installed
- Phase 2.2 completed: Service directory structure at /home/jared/services/
- Phase 2.3 completed: Traefik v3.0 reverse proxy with SSL configured
- Phase 2.4 completed: Basic monitoring tools and scripts installed
- PHASE 2 COMPLETE: Docker environment ready for service deployment
- Hostname mismatch: shows "vps-5d79b132" instead of "systems-prod-01"
- Passwordless sudo configured for automation
- 4GB swap file active with optimal swappiness setting
- Traefik dashboard: admin / dH1CtZuVLBUMig7P
- Directory structure: /home/jared/services/{traefik,web,databases,automation,monitoring,shared,templates}

61
scripts/container-health-monitor.sh Executable file
View File

@ -0,0 +1,61 @@
#!/bin/bash
# Container Health Monitor & Restart Script
# Monitors BestSolarTech containers and restarts if needed
LOG_FILE="/home/jared/services/monitor.log"
MAX_MEMORY_PERCENT=40 # Restart if container uses >40% of system memory
MAX_LOG_SIZE=100000000 # 100MB in bytes
log_message() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}
# Rotate log if too large
if [ -f "$LOG_FILE" ] && [ $(stat -f%z "$LOG_FILE" 2>/dev/null || stat -c%s "$LOG_FILE" 2>/dev/null) -gt $MAX_LOG_SIZE ]; then
mv "$LOG_FILE" "$LOG_FILE.old"
log_message "Log rotated"
fi
# Check container health
check_container() {
local container=$1
local service_name=$2
# Check if container is running
if ! docker ps --format "{{.Names}}" | grep -q "^${container}$"; then
log_message "WARNING: $container is not running. Attempting restart..."
cd /home/jared/services/$service_name && docker compose restart
return 1
fi
# Check memory usage
local mem_percent=$(docker stats --no-stream --format "{{.MemPerc}}" "$container" | sed 's/%//')
if (( $(echo "$mem_percent > $MAX_MEMORY_PERCENT" | bc -l) )); then
log_message "WARNING: $container using ${mem_percent}% memory. Restarting..."
cd /home/jared/services/$service_name && docker compose restart
return 1
fi
# Check if WordPress site responds (only for wp container)
if [ "$container" == "bestsolartech_wp" ]; then
if ! curl -sf -o /dev/null -w "%{http_code}" --max-time 10 https://bestsolartech.com | grep -q "200\|301\|302"; then
log_message "WARNING: BestSolarTech site not responding. Restarting containers..."
cd /home/jared/services/$service_name && docker compose restart
return 1
fi
fi
return 0
}
# Main monitoring
log_message "Starting health check"
# Check BestSolarTech containers
check_container "bestsolartech_wp" "bestsolartech"
check_container "bestsolartech_db" "bestsolartech"
# Clean up Docker system (remove unused data)
docker system prune -f --volumes 2>&1 | grep -E "Total reclaimed space" >> "$LOG_FILE"
log_message "Health check completed"

42
scripts/daily-maintenance.sh Executable file
View File

@ -0,0 +1,42 @@
#!/bin/bash
# Daily Maintenance Script for BestSolarTech
# Performs daily cleanup and preventive restarts
LOG_FILE="/home/jared/services/maintenance.log"
log_message() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> "$LOG_FILE"
}
log_message "Starting daily maintenance"
# 1. Clear Apache/PHP caches in WordPress container
docker exec bestsolartech_wp sh -c 'find /tmp -type f -mtime +1 -delete 2>/dev/null'
log_message "Cleared temporary files"
# 2. Optimize WordPress database tables
docker exec bestsolartech_db mysql -u wordpress -pBST_wp2024_secure -e "
USE bst_wordpress;
OPTIMIZE TABLE wp_vuhx_posts, wp_vuhx_postmeta, wp_vuhx_options;
" 2>/dev/null
log_message "Optimized database tables"
# 3. Restart containers gracefully (one at a time)
cd /home/jared/services/bestsolartech
# Restart database first
docker compose restart mariadb
sleep 30 # Wait for database to be ready
# Then restart WordPress
docker compose restart wordpress
log_message "Containers restarted"
# 4. Clean up Docker resources
docker system prune -f --volumes 2>&1 | grep -E "Total reclaimed space" >> "$LOG_FILE"
# 5. Log current resource usage
echo "Current resource usage:" >> "$LOG_FILE"
docker stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.MemPerc}}" | grep bestsolartech >> "$LOG_FILE"
log_message "Daily maintenance completed"

29
scripts/setup-monitoring.sh Executable file
View File

@ -0,0 +1,29 @@
#!/bin/bash
# Setup script to install monitoring and maintenance cron jobs
echo "Setting up BestSolarTech monitoring and maintenance..."
# Copy scripts to server
scp container-health-monitor.sh jared@15.204.247.153:~/services/
scp daily-maintenance.sh jared@15.204.247.153:~/services/
# Make scripts executable and set up cron jobs
ssh jared@15.204.247.153 << 'EOF'
# Make scripts executable
chmod +x ~/services/container-health-monitor.sh
chmod +x ~/services/daily-maintenance.sh
# Create log directory if needed
touch ~/services/monitor.log
touch ~/services/maintenance.log
# Add cron jobs
(crontab -l 2>/dev/null || true; echo "# BestSolarTech Monitoring") | crontab -
(crontab -l; echo "*/10 * * * * /home/jared/services/container-health-monitor.sh") | crontab -
(crontab -l; echo "0 3 * * * /home/jared/services/daily-maintenance.sh") | crontab -
echo "Cron jobs installed:"
crontab -l | grep -E "monitor|maintenance"
EOF
echo "Setup complete! Monitoring runs every 10 minutes, maintenance at 3 AM daily."