Cloud Server Migration: Industry Standard Comprehensive Guide and Checklist

Recently, I have been told by my cloud provider that they are shutting down my current cloud package, but they offered me an alternative package for free. The catch? I had to migrate my server on my own. I was mildly annoyed by the thought that now I am gonna have to spend a weekend migrating and testing! However, I also took it as a challenge, as it had been quite a while I migrated a server. So I decided to document the process in this blog post just in case if anyone out there needed any help and also for my future reference.

I have a GitHub repo explaining server setup from scratch right here if you like to directly jump on the technicality.

For anyone else, in this post, I will try to cover industry standards, the best security practices and other important steps for a successful server migration.

Important notes about tech stack:

Before I dive in the deep end, the OS and tools for this migrations are:

• Ubuntu 22.04 LTS – server
• ufw, NGINX & gUnicorn web servers (not Apache) – firewall, proxy, scalability & load balancer
• Letsenript – SSL certificates
• Git and GitHub – version control & backup
• Python for server-side scripting and applications
• FileZilla – FTP client

I am only using free and open sourced tools.

A little chat about server migration: what is it?

In summary, a server migration is the process of moving data, applications, and workloads from one server environment to another. This can involve transferring from physical servers to virtual ones, from one data centre to another, or from on-premises servers to cloud-based environments, or in this case from cloud-to-cloud migration. Most often the goal is to improve performance, scalability, security, and cost-efficiency. It involves careful planning, data backup, application testing, and ensuring minimal downtime and data integrity during the transition.

Although, the standards varies from company to company and their needs and scope of the migration, I have my own guideline and checklist at a personal level project like this one. I call it, “The Eclipse Method” (yeah sorry, I gave it a name lol)

The Eclipse Method

Imagine a cloud-to-cloud server migration as a solar eclipse (from Earth’s point of view). In a solar eclipse, three celestial bodies— the sun, the moon, and the Earth—align perfectly. Similarly, in this “Eclipse Method” of server migration, three key elements need to align: the old server, the new server, and the backup data. Just as the moon gradually moves to cover the sun, the migration process involves carefully integrating these three components. The old server represents your current environment, the new server is your target destination, and the backup data ensures that nothing is lost in the transition. By meticulously merging these elements step-by-step, we can achieve a seamless and unified server environment, much like the awesome moment when the eclipse reaches its totality.

So what are the steps? I am gonna talk about them next. But please keep an open mind and tweak them as you need.

Step by step migration checklists

Pre-Migration Preparation

Note: This step may vary depending on the scope of the migration

1. Assessment and Planning
   • Define the scope of the migration.
   • Identify and document all applications, data, and dependencies.
   • Determine the target cloud environment and its requirements.
   • Plan for downtime.
2. Inventory and Backup
   • Create an inventory of all servers, applications, and databases.
   • Perform a full backup of all data and configurations.
   • Verify the integrity of the backups.
3. Security and Compliance
   • Review and document security policies and compliance requirements.
   • Ensure the target environment meets all security and compliance standards.
4. Performance and Capacity Planning
   • Assess the current performance and capacity requirements.
   • Plan for scalability in the new environment.
   • Ensure the target cloud provider can meet performance needs.

Migration Execution

Note: At this step, we are starting with configuring the new server. The idea is to make a carbon copy of the old server. So all our dependencies are correct and we don’t face any nasty surprises. If you like to tweak any features for your new server, but you would like to keep the old applications to run in this server, then you have to be very very precise about it. In other words, you have to know what you are doing, otherwise Lord have mercy..

5. Environment Setup
   • Set up the target cloud environment (networking, storage, compute resources).
   • I have a detailed post on how to configure an Ubuntu server for Python apps
   • Configure security groups, firewalls, and access controls.
   • Install necessary software and dependencies in the new environment.
   • Test, test and test the server.
6. Data Migration (from backup)
   • Choose a migration method (online, offline, hybrid).
   • Set up a new database server if necessary.
   • Transfer data to the new environment
   • Verify data integrity post-migration.
7. Application Migration (from backup)
   • Migrate applications and services to the new environment.
   • Update DNS settings and endpoint configurations. (can delay this step until all the checks and testing are done!)
   • Test applications in the new environment to ensure functionality.
8. Testing and Validation
   • Perform functional testing of all applications and services.
   • Conduct performance testing to ensure the new environment meets requirements.
   • Validate data integrity and consistency.

Post-Migration

9. Optimisation and Tuning
   • Optimise resource allocation and configuration.
   • Tune performance settings and parameters.
   • Implement auto-scaling and monitoring.
10. Security and Compliance Checks
    • Re-assess security policies and configurations.
    • Ensure compliance with all regulatory requirements. Like GDPR etc
11. Documentation
    • Update documentation with new configurations and procedures.
12. Monitoring
    • Implement monitoring and alerting for the new environment.
    • Establish a plan for ongoing maintenance and troubleshooting.
    • Monitor the environment closely for any issues in the initial days.

Final Review

13. Post-Migration Review
    • Conduct a review and update documentation for future reference.
    • Document lessons learned and areas for improvement.
    • Ensure all goals and objectives of the migration are met.

So are those steps absolutely necessary? Simply, No! Nothing is absolutely necessary. There are probably God level engineers out there who has all mapped out in their brain and can do all these blindfolded while engaging their prefrontal cortex to count how much heap memory their application is consuming. But for a mere mortal like me, I find a careful planning and a checklist to follow can be a fine line between happy days and pulling my hair out.

TBC..