Simple howtos

This collection of mini howtos is a dynamic copy of the Unix Toolbox. This page extracts the XML content directly from the original XHTML DOM and displays only the requested node. Source code here (use "save as").
Unix Toolbox revision 14.4
Copyright (c) 2007-2012 Colin Barschel. Some rights reserved under Creative Commons [Attribution - Share Alike]
File System
VPN with SSH
Encrypt Files
Encrypt Partitions
SSL Certificates
Useful Commands
Install Software
Convert Media
Disk Quota
Online Help


Rsync can almost completely replace cp and scp, furthermore interrupted transfers are efficiently restarted. A trailing slash (and the absence thereof) has different meanings, the man page is good... Here some examples:
Copy the directories with full content:
# rsync -a /home/colin/ /backup/colin/                # "archive" mode. e.g keep the same
# rsync -a /var/ /var_bak/
# rsync -aR --delete-during /home/user/ /backup/      # use relative (see below)
# /opt/local/bin/rsync -azv --iconv=UTF-8-MAC,UTF-8 ~/Music/flac/ me@server:/dst/
                           # convert filenames OSX UTF8 to Windows UTF8
Same as before but over the network and with compression. Rsync uses SSH for the transport per default and will use the ssh key if they are set. Use ":" as with SCP. A typical remote copy:
# rsync -axSRzv /home/user/ user@server:/backup/user/ # Copy to remote
# rsync -a 'user@server:My\ Documents' My\ Documents  # Quote AND escape spaces for the remote shell
Exclude any directory tmp within /home/user/ and keep the relative folders hierarchy, that is the remote directory will have the structure /backup/home/user/. This is typically used for backups.
# rsync -azR --exclude=tmp/ /home/user/ user@server:/backup/
Use port 20022 for the ssh connection:
# rsync -az -e 'ssh -p 20022' /home/colin/ user@server:/backup/colin/
Using the rsync daemon (used with "::") is much faster, but not encrypted over ssh. The location of /backup is defined by the configuration in /etc/rsyncd.conf. The variable RSYNC_PASSWORD can be set to avoid the need to enter the password manually.
# rsync -axSRz /home/ ruser@hostname::rmodule/backup/
# rsync -axSRz ruser@hostname::rmodule/backup/ /home/    # To copy back
Some important options:

Rsync on Windows

Rsync is available for Windows through cygwin or as stand-alone packaged in cwrsync This is very convenient for automated backups. Install one of them (not both) and add the path to the Windows system variables: # Control Panel -> System -> tab Advanced, button Environment Variables. Edit the "Path" system variable and add the full path to the installed rsync, e.g. C:\Program Files\cwRsync\bin or C:\cygwin\bin. This way the commands rsync and ssh are available in a Windows command shell.

Public key authentication

Rsync is automatically tunneled over SSH and thus uses the SSH authentication on the server. Automatic backups have to avoid a user interaction, for this the SSH public key authentication can be used and the rsync command will run without a password.
All the following commands are executed within a Windows console. In a console (Start -> Run -> cmd) create and upload the key as described in SSH, change "user" and "server" as appropriate. If the file authorized_keys2 does not exist yet, simply copy to authorized_keys2 and upload it.
# ssh-keygen -t dsa -N ''                   # Creates a public and a private key
# rsync user@server:.ssh/authorized_keys2 . # Copy the file locally from the server
# cat >> authorized_keys2        # Or use an editor to add the key
# rsync authorized_keys2 user@server:.ssh/  # Copy the file back to the server
# del authorized_keys2                      # Remove the local copy
Now test it with (in one line):
rsync -rv "/cygdrive/c/Documents and Settings/%USERNAME%/My Documents/" \
'user@server:My\ Documents/'

Automatic backup

Use a batch file to automate the backup and add the file in the scheduled tasks (Programs -> Accessories -> System Tools -> Scheduled Tasks). For example create the file backup.bat and replace user@server.
REM rsync the directory My Documents
SET CYGWIN=nontsec
REM uncomment the next line when using cygwin
echo Press Control-C to abort
rsync -av "/cygdrive/c/Documents and Settings/%USERNAME%/My Documents/" \
'user@server:My\ Documents/'