Portal Home > Knowledgebase > Articles Database > Moving large site
Moving large site
| Posted by NeonHQ, 05-23-2007, 12:48 PM |
| Hi,
i need to move a site to another server at another datacentre that have approx 100 GB of media files.
Typically for small sites i am able to use server to server ftp to move files.
However, when moving large site like this, ftp might be disconnected and lost track of which files are not transferred.
How can i transfer/move the files (server to server) and resume if disconnected?
Please advise.
Thanks.
|
| Posted by SparkSupport, 05-23-2007, 01:17 PM |
| If you have shell access, you can use "rsync" to tranfer your files.
|
| Posted by x86brandon, 05-24-2007, 11:36 AM |
| For 100 GB, you could just ship a drive out.
|
| Posted by Scott.Mc, 05-24-2007, 12:09 PM |
| rsync/scp are going to be the most effective ways to transfer that amount of data.
You could always try to compress it aswell but it would be far more effective just to rsync/scp and have it over and done with quickly.
-Scott
|
| Posted by Russ Foster, 05-24-2007, 12:10 PM |
| If its media I think compressing isn't going to gain much but definitely rsync/scp
|
| Posted by Scott.Mc, 05-24-2007, 01:15 PM |
| Now that would depend on what kind of media
|
| Posted by NeonHQ, 05-24-2007, 02:08 PM |
| thanks for the advise.
those media files are usually avi/flv files.
i think scp does not support resume.
is there anyway to run file transfer something like in background mode? because as soon as i close my ssh, they will stop transferring.
i suspect it could probably take 1-2 days to transfer all the files.
|
| Posted by Scott.Mc, 05-24-2007, 02:21 PM |
| scp and rsync do support resume.
Try this,
screen
rsync -av -e 'ssh' /path/to/media/ root@newserver:/path/to/media/
(ctrl+d)
and it will be running in the background.
|
| Posted by SparkSupport, 05-24-2007, 02:23 PM |
| yes, you can run any commands in background by putting an '&' at the end.
If you are using rsync, the command should be like this
#rsync -e ssh -az $source_folder/ root@IP:/$destination_folder/ &
|
| Posted by Scott.Mc, 05-24-2007, 02:25 PM |
| That would not work without using a key or using the read password from file switch because otherwise you would need to input the password which would not be interprited by the shell when it's inputed as a background process.
|
| Posted by ryan1918, 05-24-2007, 05:40 PM |
| What about like passive FTP, like from your current server to another, I think this would be pretty good if both servers had pretty fast speeds, but if you get disconnected then it would stop the transfers but at least you would know which files haven't been transfered yet.
But like another person said it might be easier to just transfer the harddrive if all else fails.
|
| Posted by NeonHQ, 05-25-2007, 07:54 AM |
| thanks for the tips.
i will probably use rsync
|
| Posted by Techbrace, 05-25-2007, 10:32 AM |
| Rsync with ssh is the best option. You may also update your DC that you are going to transfer so much of data so that they're aware of it. Otherwise, they may turn your server down after noticing unusual outbound traffic from your server
|
Add to Favourites 
Print this Article
Also Read
