425 lines
14 KiB
Markdown
425 lines
14 KiB
Markdown
# Systems Production Server Migration Plan
|
|
|
|
**STATUS**: 🎉 **Phase 2 COMPLETE** - Ready for Phase 3 Service Migration Planning
|
|
|
|
## Overview
|
|
This document outlines the complete migration plan for consolidating DigitalOcean droplets to the systems production server (systems-prod-01.vps.ovh.us). Each phase is designed to be executed in separate sessions to maintain manageable context windows and reduce risk.
|
|
|
|
## Migration Strategy
|
|
- **Approach**: Incremental service migration with parallel infrastructure
|
|
- **Risk Mitigation**: Keep DigitalOcean services running until fully migrated and tested
|
|
- **Rollback**: Maintain ability to rollback at each phase
|
|
- **Validation**: Test each service thoroughly before DNS cutover
|
|
|
|
---
|
|
|
|
## Phase 1: Server Setup & Security Hardening
|
|
**Goal**: Establish secure, functional base server ready for service deployment
|
|
|
|
### Prerequisites
|
|
- [x] Server provisioned and documented
|
|
- [x] SSH credentials received from OVH via email
|
|
- [x] SSH public key ready for deployment
|
|
- [x] Domain names identified for services
|
|
|
|
### Tasks
|
|
#### 1.1 Initial Access & User Setup ✅ COMPLETED
|
|
- [x] Connect to server using initial credentials: `ssh ubuntu@15.204.247.153`
|
|
- [x] Change default ubuntu user password
|
|
- [x] Create 'jared' user with sudo privileges
|
|
- [x] Copy SSH public key to server
|
|
- [x] Test SSH key authentication
|
|
- [x] Remove ubuntu user (security enhancement)
|
|
|
|
#### 1.2 Security Hardening ✅ COMPLETED
|
|
- [x] Disable password authentication in SSH config
|
|
- [x] Disable root login via SSH
|
|
- [x] Configure UFW firewall with basic rules:
|
|
- SSH (port 22)
|
|
- HTTP (port 80)
|
|
- HTTPS (port 443)
|
|
- [x] Enable UFW firewall
|
|
- [x] Configure automatic security updates
|
|
|
|
#### 1.3 System Preparation ✅ COMPLETED
|
|
- [x] Update all system packages (completed)
|
|
- [x] Install essential tools (all required tools installed)
|
|
- [x] Configure timezone and NTP (America/New_York, chrony active)
|
|
- [x] Set up log rotation (custom configuration created)
|
|
- [x] Configure swap file if needed (4GB swap active)
|
|
|
|
#### 1.4 Validation ✅ COMPLETED
|
|
- [x] SSH access working with keys only (verified)
|
|
- [x] Firewall rules verified (UFW active, SSH/HTTP/HTTPS)
|
|
- [x] System fully updated (all packages current)
|
|
- [x] User permissions working correctly (jared user with sudo)
|
|
- [x] All services running properly (SSH, chrony, UFW)
|
|
|
|
### Deliverables
|
|
- [x] Updated profile.yaml with actual server configuration
|
|
- [x] initial-setup.md runbook documenting exact commands used
|
|
- [x] Security checklist completed
|
|
- [x] phase1-2-security.sh automated security hardening script
|
|
- [x] phase1-2-completion.md documentation
|
|
- [x] phase1-3-system-prep.sh automated system preparation script
|
|
- [x] phase1-3-manual-commands.md manual command reference
|
|
- [x] enable-passwordless-sudo.sh for automation setup
|
|
- [x] phase1-3-completion.md status documentation
|
|
- [x] phase1-4-completion.md final validation documentation
|
|
|
|
### Exit Criteria ✅ ALL MET
|
|
- [x] Server accessible only via SSH keys
|
|
- [x] Firewall properly configured and active
|
|
- [x] All security hardening complete
|
|
- [x] System updated and ready for software installation
|
|
|
|
## 🎉 PHASE 1 COMPLETE
|
|
|
|
**Status**: All Phase 1 tasks successfully completed on 2025-09-18
|
|
**Ready for**: Phase 2 Docker Environment Setup
|
|
|
|
---
|
|
|
|
## Phase 2: Docker Environment Setup
|
|
**Goal**: Install and configure Docker environment for containerized services
|
|
|
|
### Prerequisites
|
|
- [x] Phase 1 completed successfully
|
|
- [x] Server accessible and secure
|
|
|
|
### Tasks
|
|
#### 2.1 Docker Installation ✅ COMPLETED
|
|
- [x] Install Docker using official script
|
|
- [x] Install Docker Compose
|
|
- [x] Add jared user to docker group
|
|
- [x] Configure Docker daemon settings
|
|
- [x] Test Docker installation
|
|
|
|
#### 2.2 Service Directory Structure ✅ COMPLETED
|
|
- [x] Create `/home/jared/services/` directory hierarchy
|
|
- [x] Set up proper permissions and ownership
|
|
- [x] Create shared Docker networks
|
|
- [x] Configure Docker logging
|
|
|
|
#### 2.3 Traefik Reverse Proxy ✅ COMPLETED
|
|
- [x] Create traefik service directory
|
|
- [x] Configure Traefik docker-compose.yml
|
|
- [x] Set up Let's Encrypt configuration
|
|
- [x] Configure Traefik dashboard
|
|
- [x] Test Traefik startup and SSL certificate generation
|
|
|
|
#### 2.4 Base Monitoring ✅ COMPLETED
|
|
- [x] Set up basic system monitoring
|
|
- [x] Configure log aggregation
|
|
- [x] Test monitoring setup
|
|
|
|
### Deliverables ✅ ALL COMPLETE
|
|
- [x] Docker environment fully configured
|
|
- [x] Traefik running and obtaining SSL certificates
|
|
- [x] Service deployment template created
|
|
- [x] Monitoring basics in place
|
|
|
|
### Exit Criteria ✅ ALL MET
|
|
- [x] Docker and Docker Compose working
|
|
- [x] Traefik successfully routing traffic and managing SSL
|
|
- [x] Basic monitoring operational
|
|
- [x] Ready for service deployment
|
|
|
|
## 🎉 PHASE 2 COMPLETE
|
|
|
|
**Status**: All Phase 2 tasks successfully completed on 2025-09-18
|
|
**Ready for**: Phase 3 Service Migration Planning
|
|
|
|
---
|
|
|
|
## Phase 3: Service Migration Planning
|
|
**Goal**: Document and prepare for individual service migrations
|
|
|
|
### Prerequisites
|
|
- [x] Phase 2 completed successfully
|
|
- [x] Inventory of all DigitalOcean services
|
|
|
|
### Tasks
|
|
#### 3.1 Service Inventory ✅ COMPLETED
|
|
- [x] Document all current DigitalOcean services
|
|
- [x] Map service dependencies
|
|
- [x] Identify databases and persistent data
|
|
- [x] Document current domain configurations
|
|
- [x] List environment variables and secrets
|
|
|
|
#### 3.2 Migration Order Planning ✅ COMPLETED
|
|
- [x] Categorize services by risk level
|
|
- [x] Define migration order (low to high risk)
|
|
- [x] Identify testing requirements for each service
|
|
- [x] Plan rollback procedures
|
|
|
|
#### 3.3 WordPress Sites Migration ✅ COMPLETED
|
|
##### BestSolarTech.com ✅ COMPLETED (2025-09-19)
|
|
- [x] Migrated WordPress files (1.5GB) and database (599MB) from backup
|
|
- [x] Configured Docker Compose with WordPress 6.7 and MariaDB 10.11
|
|
- [x] Disabled Real Simple SSL plugin (handled by Traefik now)
|
|
- [x] Added proxy SSL detection to wp-config.php
|
|
- [x] Tested on https://temp.bestsolartech.com with HTTPS working
|
|
- [x] DNS cutover completed - live on https://bestsolartech.com
|
|
- [x] Admin panel fully functional
|
|
- **Status**: ✅ LIVE IN PRODUCTION
|
|
|
|
#### 3.4 DNS Preparation
|
|
- [ ] Document current DNS configurations
|
|
- [ ] Plan DNS cutover strategy
|
|
- [ ] Set up DNS records for new server (with temporary domains)
|
|
- [ ] Reduce TTL values for production domains
|
|
|
|
#### 3.4 Data Migration Strategy
|
|
- [ ] Plan database migration approaches
|
|
- [ ] Identify file storage migration needs
|
|
- [ ] Plan backup and restoration procedures
|
|
- [ ] Test data migration with non-critical services
|
|
|
|
### Deliverables
|
|
- [ ] Complete service inventory with dependencies
|
|
- [ ] Migration order and timeline
|
|
- [ ] DNS migration plan
|
|
- [ ] Data migration procedures
|
|
|
|
### Exit Criteria
|
|
- All services documented and prioritized
|
|
- Migration procedures defined and tested
|
|
- DNS infrastructure ready for cutover
|
|
- Ready to begin service-by-service migration
|
|
|
|
---
|
|
|
|
## Phase 4: Low-Risk Service Migration
|
|
**Goal**: Migrate static websites and development instances
|
|
|
|
### Prerequisites
|
|
- [x] Phase 3 completed successfully
|
|
- [ ] Test migration procedures validated
|
|
|
|
### Target Services (Examples)
|
|
- Static websites
|
|
- Development/staging instances
|
|
- Non-critical applications
|
|
|
|
### Tasks
|
|
#### 4.1 First Service Migration
|
|
- [ ] Choose lowest-risk service for initial migration
|
|
- [ ] Deploy service on systems production server
|
|
- [ ] Configure Traefik routing
|
|
- [ ] Test functionality on temporary domain
|
|
- [ ] Update DNS for service cutover
|
|
- [ ] Verify service working on production domain
|
|
- [ ] Monitor for issues
|
|
|
|
#### 4.2 Additional Low-Risk Services
|
|
- [ ] Repeat migration process for each low-risk service
|
|
- [ ] Document any issues and solutions
|
|
- [ ] Refine migration procedures
|
|
|
|
#### 4.3 Validation & Cleanup
|
|
- [ ] Verify all migrated services functioning
|
|
- [ ] Begin cleanup of DigitalOcean resources for migrated services
|
|
- [ ] Update documentation
|
|
|
|
### Deliverables
|
|
- [ ] Low-risk services successfully migrated
|
|
- [ ] Refined migration procedures
|
|
- [ ] Lessons learned documented
|
|
|
|
### Exit Criteria
|
|
- All low-risk services migrated and functional
|
|
- Migration procedures proven and documented
|
|
- Ready for medium-risk service migration
|
|
|
|
---
|
|
|
|
## Phase 5: Medium-Risk Service Migration
|
|
**Goal**: Migrate applications with minimal database dependencies
|
|
|
|
### Prerequisites
|
|
- [x] Phase 4 completed successfully
|
|
- [ ] Migration procedures proven with low-risk services
|
|
|
|
### Target Services (Examples)
|
|
- Simple web applications
|
|
- Services with small databases
|
|
- Non-critical automation tools
|
|
|
|
### Tasks
|
|
#### 5.1 Database Migration Preparation
|
|
- [ ] Set up database services on systems production server
|
|
- [ ] Test database backup and restore procedures
|
|
- [ ] Plan database migration windows
|
|
|
|
#### 5.2 Service Migration
|
|
- [ ] Migrate each medium-risk service individually
|
|
- [ ] Handle database migrations with minimal downtime
|
|
- [ ] Test all functionality thoroughly
|
|
- [ ] Update DNS and monitor
|
|
|
|
#### 5.3 Performance Optimization
|
|
- [ ] Monitor resource usage
|
|
- [ ] Optimize container configurations
|
|
- [ ] Tune database performance
|
|
|
|
### Deliverables
|
|
- [ ] Medium-risk services migrated
|
|
- [ ] Database migration procedures validated
|
|
- [ ] Performance optimizations applied
|
|
|
|
### Exit Criteria
|
|
- All medium-risk services migrated successfully
|
|
- Database migration procedures proven
|
|
- System performance optimized
|
|
- Ready for high-risk service migration
|
|
|
|
---
|
|
|
|
## Phase 6: High-Risk Service Migration
|
|
**Goal**: Migrate critical production services
|
|
|
|
### Prerequisites
|
|
- [x] Phase 5 completed successfully
|
|
- [ ] All procedures tested and validated
|
|
- [ ] Rollback plans confirmed
|
|
|
|
### Target Services (Examples)
|
|
- Production n8n automation
|
|
- Critical databases
|
|
- Production web applications
|
|
|
|
### Tasks
|
|
#### 6.1 Pre-Migration Preparation
|
|
- [ ] Schedule maintenance windows
|
|
- [ ] Prepare comprehensive rollback procedures
|
|
- [ ] Set up monitoring and alerting
|
|
- [ ] Create full system backups
|
|
|
|
#### 6.2 Critical Service Migration
|
|
- [ ] Migrate each critical service with maximum care
|
|
- [ ] Minimize downtime through careful planning
|
|
- [ ] Perform extensive testing
|
|
- [ ] Monitor closely post-migration
|
|
|
|
#### 6.3 Production Validation
|
|
- [ ] Verify all critical services functioning
|
|
- [ ] Confirm all integrations working
|
|
- [ ] Monitor performance and stability
|
|
|
|
### Deliverables
|
|
- [ ] All critical services migrated
|
|
- [ ] Full system validation complete
|
|
- [ ] Production stability confirmed
|
|
|
|
### Exit Criteria
|
|
- All services successfully migrated
|
|
- No critical issues identified
|
|
- System performance meeting requirements
|
|
- Ready for final cleanup
|
|
|
|
---
|
|
|
|
## Phase 7: Final Cleanup & Optimization
|
|
**Goal**: Complete migration and optimize new environment
|
|
|
|
### Prerequisites
|
|
- [x] Phase 6 completed successfully
|
|
- [ ] All services verified functional
|
|
|
|
### Tasks
|
|
#### 7.1 DigitalOcean Cleanup
|
|
- [ ] Take final backups of all DigitalOcean services
|
|
- [ ] Destroy DigitalOcean droplets
|
|
- [ ] Update billing and account settings
|
|
- [ ] Document cost savings achieved
|
|
|
|
#### 7.2 Final Optimization
|
|
- [ ] Optimize resource allocation
|
|
- [ ] Implement comprehensive monitoring
|
|
- [ ] Set up automated backups
|
|
- [ ] Fine-tune security settings
|
|
|
|
#### 7.3 Documentation Updates
|
|
- [ ] Update all server documentation
|
|
- [ ] Create operational runbooks
|
|
- [ ] Document lessons learned
|
|
- [ ] Create disaster recovery procedures
|
|
|
|
### Deliverables
|
|
- [ ] DigitalOcean resources decommissioned
|
|
- [ ] Systems production environment fully optimized
|
|
- [ ] Complete documentation set
|
|
- [ ] Cost savings validated
|
|
|
|
### Exit Criteria
|
|
- Migration completely finished
|
|
- All documentation current
|
|
- Monitoring and backups operational
|
|
- Cost optimization achieved
|
|
|
|
---
|
|
|
|
## Risk Management
|
|
|
|
### Rollback Procedures
|
|
1. **Per-Phase Rollback**: Maintain DigitalOcean services until phase completion
|
|
2. **Service-Level Rollback**: DNS cutback to DigitalOcean for individual services
|
|
3. **Full Rollback**: Complete revert to DigitalOcean if critical issues arise
|
|
|
|
### Monitoring & Validation
|
|
- Service health checks before and after each migration
|
|
- Performance monitoring during transition periods
|
|
- User impact assessment and communication
|
|
|
|
### Emergency Contacts
|
|
- OVH Support: [Contact information when available]
|
|
- DNS Provider Support: [Contact information when available]
|
|
- Backup contact methods for critical issues
|
|
|
|
---
|
|
|
|
## Success Metrics
|
|
|
|
### Cost Optimization
|
|
- Target: 30-50% reduction in monthly hosting costs
|
|
- Measure: Compare DigitalOcean bills before/after migration
|
|
|
|
### Performance
|
|
- Target: Maintain or improve service response times
|
|
- Measure: Response time monitoring and comparison
|
|
|
|
### Reliability
|
|
- Target: 99.9% uptime for all migrated services
|
|
- Measure: Uptime monitoring and incident tracking
|
|
|
|
### Security
|
|
- Target: No security incidents during or after migration
|
|
- Measure: Security audit and monitoring
|
|
|
|
---
|
|
|
|
## Timeline Estimation
|
|
|
|
| Phase | Estimated Duration | Dependencies | Status |
|
|
|-------|-------------------|--------------|---------|
|
|
| Phase 1 | 1-2 sessions (2-4 hours) | OVH credentials | ✅ COMPLETE (1 day) |
|
|
| Phase 2 | 1-2 sessions (2-4 hours) | Phase 1 complete | ✅ COMPLETE (1 day) |
|
|
| Phase 3 | 1 session (2-3 hours) | Service inventory | 🔄 NEXT |
|
|
| Phase 4 | 2-3 sessions (4-6 hours) | Low-risk services identified | ⏳ PENDING |
|
|
| Phase 5 | 3-4 sessions (6-8 hours) | Medium-risk services ready | ⏳ PENDING |
|
|
| Phase 6 | 4-5 sessions (8-10 hours) | High-risk service preparation | ⏳ PENDING |
|
|
| Phase 7 | 1-2 sessions (2-4 hours) | All services migrated | ⏳ PENDING |
|
|
|
|
**Total Estimated Time**: 12-19 sessions (26-39 hours)
|
|
**Completed**: 2 phases (2 days)
|
|
**Remaining**: 5 phases
|
|
|
|
---
|
|
|
|
*This plan is a living document and should be updated as the migration progresses.*
|
|
|
|
*Last updated: 2025-09-18*
|
|
*Phase 1 Status: ✅ COMPLETE - All tasks successfully finished*
|
|
*Phase 2 Status: ✅ COMPLETE - Docker environment ready*
|
|
*Ready for Phase 3: Service Migration Planning* |