server-ovh-prod-01-main/migrations/migration-plan.md

Systems Production Server Migration Plan

STATUS: ✅ MIGRATION COMPLETE (2026-06-24) — All services migrated from DigitalOcean to OVH and verified live. The source DigitalOcean VPS (hyperthrive-docker-services, 161.35.12.75) was deleted on 2026-06-24. This document is retained as a historical record. The phase markers below reflect the original incremental plan; all phases are now done.

Overview

This document outlines the complete migration plan for consolidating DigitalOcean droplets to the systems production server (systems-prod-01.vps.ovh.us). Each phase is designed to be executed in separate sessions to maintain manageable context windows and reduce risk.

Migration Strategy

• Approach: Incremental service migration with parallel infrastructure
• Risk Mitigation: Keep DigitalOcean services running until fully migrated and tested
• Rollback: Maintain ability to rollback at each phase
• Validation: Test each service thoroughly before DNS cutover

---

Phase 1: Server Setup & Security Hardening

Goal: Establish secure, functional base server ready for service deployment

Prerequisites

• Server provisioned and documented
• SSH credentials received from OVH via email
• SSH public key ready for deployment
• Domain names identified for services

Tasks

1.1 Initial Access & User Setup ✅ COMPLETED

• Connect to server using initial credentials: ssh ubuntu@15.204.247.153
• Change default ubuntu user password
• Create 'jared' user with sudo privileges
• Copy SSH public key to server
• Test SSH key authentication
• Remove ubuntu user (security enhancement)

1.2 Security Hardening ✅ COMPLETED

• Disable password authentication in SSH config
• Disable root login via SSH
• Configure UFW firewall with basic rules:
  • SSH (port 22)
  • HTTP (port 80)
  • HTTPS (port 443)
• Enable UFW firewall
• Configure automatic security updates

1.3 System Preparation ✅ COMPLETED

• Update all system packages (completed)
• Install essential tools (all required tools installed)
• Configure timezone and NTP (America/New_York, chrony active)
• Set up log rotation (custom configuration created)
• Configure swap file if needed (4GB swap active)

1.4 Validation ✅ COMPLETED

• SSH access working with keys only (verified)
• Firewall rules verified (UFW active, SSH/HTTP/HTTPS)
• System fully updated (all packages current)
• User permissions working correctly (jared user with sudo)
• All services running properly (SSH, chrony, UFW)

Deliverables

• Updated profile.yaml with actual server configuration
• initial-setup.md runbook documenting exact commands used
• Security checklist completed
• phase1-2-security.sh automated security hardening script
• phase1-2-completion.md documentation
• phase1-3-system-prep.sh automated system preparation script
• phase1-3-manual-commands.md manual command reference
• enable-passwordless-sudo.sh for automation setup
• phase1-3-completion.md status documentation
• phase1-4-completion.md final validation documentation

Exit Criteria ✅ ALL MET

• Server accessible only via SSH keys
• Firewall properly configured and active
• All security hardening complete
• System updated and ready for software installation

🎉 PHASE 1 COMPLETE

Status: All Phase 1 tasks successfully completed on 2025-09-18 Ready for: Phase 2 Docker Environment Setup

---

Phase 2: Docker Environment Setup

Goal: Install and configure Docker environment for containerized services

Prerequisites

• Phase 1 completed successfully
• Server accessible and secure

Tasks

2.1 Docker Installation ✅ COMPLETED

• Install Docker using official script
• Install Docker Compose
• Add jared user to docker group
• Configure Docker daemon settings
• Test Docker installation

2.2 Service Directory Structure ✅ COMPLETED

• Create /home/jared/services/ directory hierarchy
• Set up proper permissions and ownership
• Create shared Docker networks
• Configure Docker logging

2.3 Traefik Reverse Proxy ✅ COMPLETED

• Create traefik service directory
• Configure Traefik docker-compose.yml
• Set up Let's Encrypt configuration
• Configure Traefik dashboard
• Test Traefik startup and SSL certificate generation

2.4 Base Monitoring ✅ COMPLETED

• Set up basic system monitoring
• Configure log aggregation
• Test monitoring setup

Deliverables ✅ ALL COMPLETE

• Docker environment fully configured
• Traefik running and obtaining SSL certificates
• Service deployment template created
• Monitoring basics in place

Exit Criteria ✅ ALL MET

• Docker and Docker Compose working
• Traefik successfully routing traffic and managing SSL
• Basic monitoring operational
• Ready for service deployment

🎉 PHASE 2 COMPLETE

Status: All Phase 2 tasks successfully completed on 2025-09-18 Ready for: Phase 3 Service Migration Planning

---

Phase 3: Service Migration Planning

Goal: Document and prepare for individual service migrations

Prerequisites

• Phase 2 completed successfully
• Inventory of all DigitalOcean services

Tasks

3.1 Service Inventory ✅ COMPLETED

• Document all current DigitalOcean services
• Map service dependencies
• Identify databases and persistent data
• Document current domain configurations
• List environment variables and secrets

3.2 Migration Order Planning ✅ COMPLETED

• Categorize services by risk level
• Define migration order (low to high risk)
• Identify testing requirements for each service
• Plan rollback procedures

3.3 WordPress Sites Migration ✅ COMPLETED

BestSolarTech.com ✅ COMPLETED (2025-09-19)

• Migrated WordPress files (1.5GB) and database (599MB) from backup
• Configured Docker Compose with WordPress 6.7 and MariaDB 10.11
• Disabled Real Simple SSL plugin (handled by Traefik now)
• Added proxy SSL detection to wp-config.php
• Tested on https://temp.bestsolartech.com with HTTPS working
• DNS cutover completed - live on https://bestsolartech.com
• Admin panel fully functional
• Status: ✅ LIVE IN PRODUCTION

3.4 DNS Preparation

• Document current DNS configurations
• Plan DNS cutover strategy
• Set up DNS records for new server (with temporary domains)
• Reduce TTL values for production domains

3.4 Data Migration Strategy

• Plan database migration approaches
• Identify file storage migration needs
• Plan backup and restoration procedures
• Test data migration with non-critical services

Deliverables

• Complete service inventory with dependencies
• Migration order and timeline
• DNS migration plan
• Data migration procedures

Exit Criteria

• All services documented and prioritized
• Migration procedures defined and tested
• DNS infrastructure ready for cutover
• Ready to begin service-by-service migration

---

Historical / Superseded — Phases 4–7

Note (2026-06-25): The migration is complete. Phases 4–7 below were the original incremental plan template. They were never executed as discrete phases — services were migrated individually as needed, culminating in the remetrics addendum below. These sections are preserved as a historical record but do not reflect actual execution steps.

---

Phase 4: Low-Risk Service Migration

Goal: Migrate static websites and development instances

Prerequisites

• Phase 3 completed successfully
• Test migration procedures validated

Target Services (Examples)

• Static websites
• Development/staging instances
• Non-critical applications

Tasks

4.1 First Service Migration

• Choose lowest-risk service for initial migration
• Deploy service on systems production server
• Configure Traefik routing
• Test functionality on temporary domain
• Update DNS for service cutover
• Verify service working on production domain
• Monitor for issues

4.2 Additional Low-Risk Services

• Repeat migration process for each low-risk service
• Document any issues and solutions
• Refine migration procedures

4.3 Validation & Cleanup

• Verify all migrated services functioning
• Begin cleanup of DigitalOcean resources for migrated services
• Update documentation

Deliverables

• Low-risk services successfully migrated
• Refined migration procedures
• Lessons learned documented

Exit Criteria

• All low-risk services migrated and functional
• Migration procedures proven and documented
• Ready for medium-risk service migration

---

Phase 5: Medium-Risk Service Migration

Goal: Migrate applications with minimal database dependencies

Prerequisites

• Phase 4 completed successfully
• Migration procedures proven with low-risk services

Target Services (Examples)

• Simple web applications
• Services with small databases
• Non-critical automation tools

Tasks

5.1 Database Migration Preparation

• Set up database services on systems production server
• Test database backup and restore procedures
• Plan database migration windows

5.2 Service Migration

• Migrate each medium-risk service individually
• Handle database migrations with minimal downtime
• Test all functionality thoroughly
• Update DNS and monitor

5.3 Performance Optimization

• Monitor resource usage
• Optimize container configurations
• Tune database performance

Deliverables

• Medium-risk services migrated
• Database migration procedures validated
• Performance optimizations applied

Exit Criteria

• All medium-risk services migrated successfully
• Database migration procedures proven
• System performance optimized
• Ready for high-risk service migration

---

Phase 6: High-Risk Service Migration

Goal: Migrate critical production services

Prerequisites

• Phase 5 completed successfully
• All procedures tested and validated
• Rollback plans confirmed

Target Services (Examples)

• Production n8n automation
• Critical databases
• Production web applications

Tasks

6.1 Pre-Migration Preparation

• Schedule maintenance windows
• Prepare comprehensive rollback procedures
• Set up monitoring and alerting
• Create full system backups

6.2 Critical Service Migration

• Migrate each critical service with maximum care
• Minimize downtime through careful planning
• Perform extensive testing
• Monitor closely post-migration

6.3 Production Validation

• Verify all critical services functioning
• Confirm all integrations working
• Monitor performance and stability

Deliverables

• All critical services migrated
• Full system validation complete
• Production stability confirmed

Exit Criteria

• All services successfully migrated
• No critical issues identified
• System performance meeting requirements
• Ready for final cleanup

---

Phase 7: Final Cleanup & Optimization

Goal: Complete migration and optimize new environment

Prerequisites

• Phase 6 completed successfully
• All services verified functional

Tasks

7.1 DigitalOcean Cleanup

• Take final backups of all DigitalOcean services
• Destroy DigitalOcean droplets
• Update billing and account settings
• Document cost savings achieved

7.2 Final Optimization

• Optimize resource allocation
• Implement comprehensive monitoring
• Set up automated backups
• Fine-tune security settings

7.3 Documentation Updates

• Update all server documentation
• Create operational runbooks
• Document lessons learned
• Create disaster recovery procedures

Deliverables

• DigitalOcean resources decommissioned
• Systems production environment fully optimized
• Complete documentation set
• Cost savings validated

Exit Criteria

• Migration completely finished
• All documentation current
• Monitoring and backups operational
• Cost optimization achieved

---

Risk Management

Rollback Procedures

1. Per-Phase Rollback: Maintain DigitalOcean services until phase completion
2. Service-Level Rollback: DNS cutback to DigitalOcean for individual services
3. Full Rollback: Complete revert to DigitalOcean if critical issues arise

Monitoring & Validation

• Service health checks before and after each migration
• Performance monitoring during transition periods
• User impact assessment and communication

Emergency Contacts

• OVH Support: [Contact information when available]
• DNS Provider Support: [Contact information when available]
• Backup contact methods for critical issues

---

Success Metrics

Cost Optimization

• Target: 30-50% reduction in monthly hosting costs
• Measure: Compare DigitalOcean bills before/after migration

Performance

• Target: Maintain or improve service response times
• Measure: Response time monitoring and comparison

Reliability

• Target: 99.9% uptime for all migrated services
• Measure: Uptime monitoring and incident tracking

Security

• Target: No security incidents during or after migration
• Measure: Security audit and monitoring

---

Timeline Estimation

Phase	Estimated Duration	Dependencies	Status
Phase 1	1-2 sessions (2-4 hours)	OVH credentials	✅ COMPLETE (1 day)
Phase 2	1-2 sessions (2-4 hours)	Phase 1 complete	✅ COMPLETE (1 day)
Phase 3	1 session (2-3 hours)	Service inventory	🔄 NEXT
Phase 4	2-3 sessions (4-6 hours)	Low-risk services identified	⏳ PENDING
Phase 5	3-4 sessions (6-8 hours)	Medium-risk services ready	⏳ PENDING
Phase 6	4-5 sessions (8-10 hours)	High-risk service preparation	⏳ PENDING
Phase 7	1-2 sessions (2-4 hours)	All services migrated	⏳ PENDING

Total Estimated Time: 12-19 sessions (26-39 hours) Completed: 2 phases (2 days) Remaining: 5 phases

---

This plan is a living document and should be updated as the migration progresses.

Last updated: 2025-09-18 Phase 1 Status: ✅ COMPLETE - All tasks successfully finished Phase 2 Status: ✅ COMPLETE - Docker environment ready Ready for Phase 3: Service Migration Planning

---

Addendum: remetrics.io Migration ✅ COMPLETE (2026-06-24)

Note: This was a separate migration from a different DigitalOcean box than the Phases above. Phases 1-7 cover the hyperthrive-docker-services VPS (161.35.12.75, deleted 2026-06-24). remetrics lived on a standalone DigitalOcean web droplet (147.182.131.217), unrelated to the hyperthrive box. It is recorded here as an addendum rather than retrofitted into a phase.

Outcome: remetrics.io (Rails app + Strapi CMS) was migrated from the DigitalOcean web droplet (147.182.131.217) to the OVH server (15.204.247.153) on 2026-06-24.

• Services on OVH:
  • ~/services/remetrics/ — Rails web (remetrics-remetrics-1), Sidekiq worker (remetrics-worker-1), Postgres (remetrics-db-1, db remetrics_production), redis. Web/worker images baked GitHub-independent (no boot-time bundle/git).
  • ~/services/strapi-remetrics/ — Strapi (strapi-remetrics), SQLite, media on GCS bucket remetrics-strapi.
• DNS (Cloudflare zone remetrics.io, grey-cloud A-records → 15.204.247.153): remetrics.io, www.remetrics.io, cms.remetrics.io. temp.remetrics.io and the dead/unused strapi.remetrics.io intentionally remain on the old DO box.
• Cutover: ~4m40s frozen window, zero data loss (480 users migrated exactly; a 503 "freeze" on the old box + final pg_dump captured a signup that landed seconds before cutover).
• TLS: Added a second Traefik cert-resolver cloudflare (ACME DNS-01, storage /letsencrypt/acme-cloudflare.json) alongside the existing letsencrypt (HTTP-01). remetrics/cms/www routers use certresolver=cloudflare. Certs were pre-issued via DNS-01 before the DNS flip for a zero-cert-window cutover.
• Backups: ~/services/backup.sh auto-discovers both new services (remetrics Postgres via pg_dumpall, Strapi SQLite via service-files tar); confirmed in nightly backup.
• CMS coupling: The Rails app fetches CMS from a HARDCODED https://cms.remetrics.io (in strapi/article.rb and help_page.rb) — not env-configurable.

Rollback status: The DigitalOcean remetrics web droplet (147.182.131.217) is currently FROZEN (apex returns 503, sidekiq worker stopped) and kept as a short-term rollback. It is NOT yet deleted — decommission pending.