New Server Data Migration

These methods will most likely seem basic to a seasoned web host but until I was forced to switch web hosts I never had to seamlessly move sites between servers.

Goals

1. Almost all clients used numerous mySQL databases that had content being accessed 24/7 (forums and purchase requests to name two). Any entries that were added on the old server had to be placed on the new server in real time.
2. The files from the old server had to be moved to the new one.
3. A number of clients had processes running (mainly as cron jobs) that batch processed data up until a given time. These processes had to be preserved and data processing could not be postponed.

1 - Databases

The first challenged I tackled were the databases. This went down in three steps, the databases were copied to the new server, the new server’s databases were validated against the old servers’, and finally all scripts needing database access on the old server were pointed to the new server.

To begin, the old server’s IP was added to the allowed-hosts table on the new server and vice-versa so scripts on either server could access the others’ databases. Then all the current databases were copied over to the new server. When this was complete, there were some records that had been added/removed/edited during the copy process (there were about 40 since the copy only took about 5 minutes but on a larger site there could be many, many more).

To fix these missing records, I wrote one PHP script for each server (I may post these at some point). The script on the old server had an array storing the paths to all files which needed to be modified in order to connect to the new servers’ databases (mainly config.php type files for forums, order forms, and the like).

The script on the new server used the INFORMATION_SCHEMA database to gather all the database names and tables names which it used to then, in real time, check for differences between the old server and the new one. It then made the necessary changes to the new servers’ database and, immediately following, called the script on the old server to make the aforementioned file change.

Once the last hurdle was completed, all queries made on the old server were made to the new server’s databases , completely cutting off the old servers’ databases. This allowed me to only have to deal with the copy process once rather than mess around with trying to sync the two servers.

2 - Files

This step was very basic. I used the shell command wget to pull all the files from the old server to the new one.

3 - Batch Processes

This was the part that caused the most trouble. Some of the people I host have tasks that run on a cron job which had to be moved over to the new server. I am going to use the most complicated example of these to explain the migration process: One of my clients keeps logs of all transactions users make. I won’t go into what these ‘transactions’ are but they basically do two things, update a database table and move files around.

This cron job was run every 24 hours which gave me a nice window where I was sure the process wouldn’t run and try to access half-missing data.

This was the approach: I let the process run for the last time on the old server. Keep in mind, any database changes would already be made on the new server due to Step 1. All files pertaining to this process were copied to the new server. These files on the old server were replaced with links to the new servers’ equivalent files. The cron job on the old server was completely stopped and then restarted on the new server. I then manually invoked the job to make sure it functioned.

Finally, a cron job was set to copy only changed files from the old server to the new server every 5 minutes.

Done!

After all of this, it was just a matter of changing the old name servers to the new ones. It didn’t matter how long it took for them to propagate through the internet nor did it matter if a user accessed the old server or the new server; the content was served from the new server. After 72 hours (just to be sure), the copy cron job was stopped, and the old server was completely abandoned, left for a month or so just to be 100% sure.

I’m sure there are many better ways to do this but, for a small move, this worked great. After the scripts are made, the entire process only takes about 1 hour to complete. And the best part: the scripts can be re-used so I’m ready for next time.