Data backup with rsync

     Rsync is a file synchronization and transfer utility.  It can be used to transfer files between two local or remote locations.  It is fast, supports compression and encryption ( through ssh ).  Its usage is simple and can be used as a backup utility.

Syntax:

rsync <options> <source> <destination>

     Both the source and destination can be local directories, or the source can be a remote location ( pull ) or the destination can be remote ( push ).  For remote file transfer rsync can make use of either ssh/rsh shell as a transport or use rysnc as a daemon.  Here we will look at the local copy and ssh based remote copy.

     You can run rsync without any options, but using the following options will be good when you are backing up data.

-a : Archive - preserves symbolic links,permissions,timestamp,user/group
-z : Compression - compress data while transferring - good for remote date transfer

     If you are backing up a directory tree, then you should be using "-r" option to recursively go through all sub directories and files.  But if you are using "-a" option this is not required as -a does recursive archiving.

     Let us see how we can do a local file transfer ( This is the way I backup my home directory to external usb hard disk )

rsync -az /home/safeer  /media/USB_HD_01/Backup_02_2011

     After the first run, I periodically run the same command  to keep the backup in sync with my home directory.
In the subsequent runs, rsync only copies the files that has changed in my home directory since the last run.  This saves a lot of time.

     Any time you want to do a restore (say after an OS reinstall ), all you need to do is reverse the source and destination and run the rsync command.

rsync -az /media/USB_HD_01/Backup_02_2011 /home/safeer

Note: When you provide the source directory name, if it terminates with "/" only the contents under the directory will be transferred.  But if you omit the ending "/" the same directory will be created at the destination and the contents will be copied over.

     Now backing up into a remote server

rsync -az -e ssh /home/safeer safeer@backup.safeer.in:/home/safeer/Backup_02_2011

"-e ssh" specifies the shell to be used for remote transport.

     Now let us backup remote content locally.

rsync -az -e ssh safeer@web.safeer.in:/var/www/html /media/USB_HD_01/Web_02_2011

     The syntax for a remote location is always USER@REMOTE_SERVER:REMOTE_DIRECTORY.  If USER is omitted the local username will be used, if REMOTE_DIRECTORY is omitted users home directory will be used.

     We will check a few useful options

To monitor the transfer:

-v : Verbose - provides a summary of what is happening
--progress : Displays the progress of data transfer.
--stats : Displays statistics abut the file transfer

To select and deselect files/folders:

--include PATTERN : Include files that match the given pattern
--exclude PATTERN : Exclude files that match the given pattern
--include-from FILE and --exclude-from FILE - to include and exclude files based on patterns in the file
--files-from FILE : Transfer only files listed in FILE
--filter RULE : Filter files based on RULE

     All these options are allowed multiple times.

Deletion/Update

-u : Do not transfter/overwrite if destination file is newer than source.
--delete : Delete file that exist in destination but not in source
--delete-before/--delete-during/--delete-delay/--delete-after/--delete-excluded are some options worth checking

For more detailed options and explanations, checkout the rsync man page.