• View
• Edit
• History
• Print

RemoteBackup

Creating a remote backup

The aim of this section is to automatically backup your files from your home directory or another local backup (say created from Rsync) automatically. To do this, your computer will use a program called rsync to copy changes between the local copy of your files (the Source) and the backup repository copy of the files (the Destination). The two computers communicate via a program called ssh. To make the running of this easier we will create a small bash script to automate the issusing of instructions to the programs. Finally the program cron automates this process by running the script you have created everyday.

Setting up ssh

In order for the two computers to talk we need to use ssh. This allows your computer to talk to the backup server. You may be familiar with this process however you usually have to enter in a password in order to authenticate yourself on the server. By creating a key-pair and a ssh passphrase, as well as an alias for the backup server (in this case sub-master) it is possible for your computer to autonomously interact with the server without the need for you to enter a password everytime you wish to connect to it. The instructions for this are here: PwdLessSSH. You should test the connection now and create a directory in /g/gibson/username/ called backup. This is where we will copy the files to.

rsync

This is the program that we will use to copy changes on your computer to the backup server. It needs to know the source directory and the target directory on the destination server. Once again we have to set up a script for cron to run on a regular basis. This file can be saved in your home directory. In order to test the connection and the rsync program we issue the following command:

rsync -a --dry-run -e ssh /home/username backup:/g/gibson/username/backup/

You should replace username with your username. Take care that the username in the first argument refers to your username on your machine and the username in the last argument refers to your unix username. The backup server is called using the alias you set up earlier - in this case backup. After issuing this command you will see a list of files and it should terminate like this:

sent 502256 bytes  received 9446 bytes  29240.11 bytes/sec
total size is 863401461  speedup is 1687.31
Completed

The rsync section of the wiki (Rsync) will provide an explanation of how to exclude certain directories or files from the backups and explain some of the other advanced usage of rsync, but that isn't required to set up a basic back up.

Create a script to do this

Now you have tested the connection to the backup server and have shown how to copy files to the remote server. You should now create a small script in your home directory naming it something like .backup.sh. We create three variables at the top of the script to make keeping the script manageable. We set the user variable to the username on the local machine and use this again since the username on the unix machines is the same. If not, its probably easier to delete the $USER variable and just explicitly type the $SOURCE and $TARGET variables. Note on mac's the $SOURCE should be /Users/$USER and not home. Here is a sample of my script:

#! /bin/bash
USER="username"
SOURCE="/home/$USER"
TARGET="backup:/g/gibson/$USER/backup/"

rsync -CHavz -e ssh $SOURCE $TARGET

exit 1;

Cron job - automating the script

Next we want to run this script everyday to back up everything. You can now add this to the crontab by typing crontab -e and adding the following line:

0 6 * * * /home/username/.backup.sh

And now if you list crontab by typing crontab -l you should see the list of scripts that you run every day

username@compy:~> crontab -l
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.XXXXbMIFTO installed on Wed Feb 28 14:29:05 2007)
# (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
0 5 * * * /home/username/.backup.sh
0 5 * * * /home/username/.namazu/.index.sh
0 6 * * * /home/username/.remote_backup.sh

Finally you can check the output of the results by running the following command: cat /var/spool/mail/username

This is a copy of mails that you have effectively sent to yourself after each backup. Check these files as they will contain important information about any errors in the backups.

Two way back up with unison.

This program is capable of syncing changes in two directions. This is useful for say maintaining a remote directory with journal articles if more than one person is adding files to that directory. You can add files to the remote dir from your local copy and receive changes to the remote directory to your local directory. All you need to specify is the location of the two directories.

/usr/local/bin/unison /foo/bar/ ssh://user@examples.com//remote/foo/bar/

The only tricky syntax is the double forward slash after the remote location. This will prompt you to enter your password on the remote system unless you have passwordless ssh configured (PwdLessSSH).