Rsync
From OpenFSG
Contents |
Installing and using the ipkg application Rsync
Introduction
Rsync is a great tool for synchronising files from disk to disk, or across a network connection. Some smart coding means that if you have two almost similar files then only the changes are copied over.
Rsync uses the "rsync algorithm" which provides a very fast method for bringing remote files into sync. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present at one of the ends of the link beforehand.
Some features of rsync include
- can update whole directory trees and filesystems
- optionally preserves symbolic links, hard links, file ownership, permissions, devices and times
- internal pipelining reduces latency for multiple files
- can use rsh, ssh or direct sockets as the transport
Installation
- Install ipkg
- Make sure you have an internet connection
- Log on and su to root
/ # ipkg install rsync Installing rsync (2.6.8-1) to root... Downloading http://ipkg.nslu2-linux.org/feeds/optware/ds101/cross/stable/rsync_2.6.8-1_armeb.ipk Configuring rsync /opt/etc/init.d/S57rsyncd: 5: pidof: not found Successfully terminated.
The pidof error is since FSG does not include the utility pidof. The rsync daemon will also not run after the first reboot since the startup file appears to be the shutdown script which is installed at /opt/etc/init.d/S57rsyncd which therefore won't be touched (since it should be in /etc/rc/). This isn't really a problem though since you shouldn't need the daemon for anything since it's safer to tunnel it over SSH instead.
(The pidof tool is contained in the package busybox-links)
Usage
To copy from disk to disk
Simply enter:
rsync -auv ''source'' ''destination''
e.g.
rsync -auv /home/LaCie_Hard_Drive_USB/Music/* /home/Music/
This will copy the entire directory tree from source to destination, synchronising it if some (or all) of it is already there.
To copy across the network
Both sides must have ssh and rsync - for a PC running windows you probably want to look at Cygwin and install and set up an ssh server.
If you want to log in without passwords using a pre-shared secret key then look at this advanced tutorial, however remember this is a backdoor that someone else might use to get your data!
You use rsync the same way except you tell it to use ssh as transport.
rsync -auvz --rsh=ssh ''source'' ''destination''
Either (or both) of the source and destination are in SSH syntax like:
user@host:/directory or user@host:/directory/file
e.g.
rsync -auvz --rsh=ssh Administrator@windowsPC:/cygdrive/c/music/* /home/Music/
Options
Very useful ones are:
- The v option will produce logging of every file moved
- The --progress will do a percentage bar for each file copied
- The z option will compress the data before sending it - most useful on a network.
If you type rsync you will get the help page
rsync version 2.6.8 protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.
<http://rsync.samba.org/>
Capabilities: 64-bit files, no socketpairs, hard links, symlinks, batchfiles,
inplace, IPv6, 64-bit system inums, 64-bit internal inums
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
rsync is a file transfer program capable of efficient remote update
via a fast differencing algorithm.
Usage: rsync [OPTION]... SRC [SRC]... DEST
or rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
or rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST
or rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST
or rsync [OPTION]... [USER@]HOST:SRC [DEST]
or rsync [OPTION]... [USER@]HOST::SRC [DEST]
or rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]
The ':' usages connect via remote shell, while '::' & 'rsync://' usages connect
to an rsync daemon, and require SRC or DEST to start with a module name.
Options
-v, --verbose increase verbosity
-q, --quiet suppress non-error messages
-c, --checksum skip based on checksum, not mod-time & size
-a, --archive archive mode; same as -rlptgoD (no -H)
--no-OPTION turn off an implied OPTION (e.g. --no-D)
-r, --recursive recurse into directories
-R, --relative use relative path names
--no-implied-dirs don't send implied dirs with --relative
-b, --backup make backups (see --suffix & --backup-dir)
--backup-dir=DIR make backups into hierarchy based in DIR
--suffix=SUFFIX set backup suffix (default ~ w/o --backup-dir)
-u, --update skip files that are newer on the receiver
--inplace update destination files in-place (SEE MAN PAGE)
--append append data onto shorter files
-d, --dirs transfer directories without recursing
-l, --links copy symlinks as symlinks
-L, --copy-links transform symlink into referent file/dir
--copy-unsafe-links only "unsafe" symlinks are transformed
--safe-links ignore symlinks that point outside the source tree
-k, --copy-dirlinks transform symlink to a dir into referent dir
-K, --keep-dirlinks treat symlinked dir on receiver as dir
-H, --hard-links preserve hard links
-p, --perms preserve permissions
-E, --executability preserve the file's executability
--chmod=CHMOD change destination permissions
-o, --owner preserve owner (super-user only)
-g, --group preserve group
--devices preserve device files (super-user only)
--specials preserve special files
-D same as --devices --specials
-t, --times preserve times
-O, --omit-dir-times omit directories when preserving times
--super receiver attempts super-user activities
-S, --sparse handle sparse files efficiently
-n, --dry-run show what would have been transferred
-W, --whole-file copy files whole (without rsync algorithm)
-x, --one-file-system don't cross filesystem boundaries
-B, --block-size=SIZE force a fixed checksum block-size
-e, --rsh=COMMAND specify the remote shell to use
--rsync-path=PROGRAM specify the rsync to run on the remote machine
--existing ignore non-existing files on receiving side
--ignore-existing ignore files that already exist on receiving side
--remove-sent-files sent files/symlinks are removed from sending side
--del an alias for --delete-during
--delete delete files that don't exist on the sending side
--delete-before receiver deletes before transfer (default)
--delete-during receiver deletes during transfer, not before
--delete-after receiver deletes after transfer, not before
--delete-excluded also delete excluded files on the receiving side
--ignore-errors delete even if there are I/O errors
--force force deletion of directories even if not empty
--max-delete=NUM don't delete more than NUM files
--max-size=SIZE don't transfer any file larger than SIZE
--min-size=SIZE don't transfer any file smaller than SIZE
--partial keep partially transferred files
--partial-dir=DIR put a partially transferred file into DIR
--delay-updates put all updated files into place at transfer's end
-m, --prune-empty-dirs prune empty directory chains from the file-list
--numeric-ids don't map uid/gid values by user/group name
--timeout=TIME set I/O timeout in seconds
-I, --ignore-times don't skip files that match in size and mod-time
--size-only skip files that match in size
--modify-window=NUM compare mod-times with reduced accuracy
-T, --temp-dir=DIR create temporary files in directory DIR
-y, --fuzzy find similar file for basis if no dest file
--compare-dest=DIR also compare destination files relative to DIR
--copy-dest=DIR ... and include copies of unchanged files
--link-dest=DIR hardlink to files in DIR when unchanged
-z, --compress compress file data during the transfer
--compress-level=NUM explicitly set compression level
-C, --cvs-exclude auto-ignore files the same way CVS does
-f, --filter=RULE add a file-filtering RULE
-F same as --filter='dir-merge /.rsync-filter'
repeated: --filter='- .rsync-filter'
--exclude=PATTERN exclude files matching PATTERN
--exclude-from=FILE read exclude patterns from FILE
--include=PATTERN don't exclude files matching PATTERN
--include-from=FILE read include patterns from FILE
--files-from=FILE read list of source-file names from FILE
-0, --from0 all *-from/filter files are delimited by 0s
--address=ADDRESS bind address for outgoing socket to daemon
--port=PORT specify double-colon alternate port number
--sockopts=OPTIONS specify custom TCP options
--blocking-io use blocking I/O for the remote shell
--stats give some file-transfer stats
-8, --8-bit-output leave high-bit chars unescaped in output
-h, --human-readable output numbers in a human-readable format
--progress show progress during transfer
-P same as --partial --progress
-i, --itemize-changes output a change-summary for all updates
--log-format=FORMAT output filenames using the specified format
--password-file=FILE read password from FILE
--list-only list the files instead of copying them
--bwlimit=KBPS limit I/O bandwidth; KBytes per second
--write-batch=FILE write a batched update to FILE
--only-write-batch=FILE like --write-batch but w/o updating destination
--read-batch=FILE read a batched update from FILE
--protocol=NUM force an older protocol version to be used
-4, --ipv4 prefer IPv4
-6, --ipv6 prefer IPv6
--version print version number
(-h) --help show this help (-h works with no other options)
Use "rsync --daemon --help" to see the daemon-mode command-line options.
Please see the rsync(1) and rsyncd.conf(5) man pages for full documentation.
See http://rsync.samba.org/ for updates, bug reports, and answers
rsync error: syntax or usage error (code 1) at main.c(1216) [client=2.6.8]
How to use it to backup from windows with Cygwin to your FSG
- Install Cygwin on your Windows PC.
- (optional) Setup an SSH pre-shared key so that it won't ask you for the password
- If you want it to backup to an external disk then it's easier to create a link. Log into the root account on FSG and type the commands:
mkdir -p /home/External_blah/backup-dir ln -s /home/External_blah/backup-dir /home/backup
Replace External_blah with the name your FSG has for your external drive and replace backup-dir with what you want the backup directory to be called on the external drive.
- Create two files
File: backup.cmd on your desktop (or where you want it to run - startup or shutdown folders are also useful)
@echo off c:\cygwin\bin\bash --login -i -c "backup.sh"
File: backup.sh in c:\cygwin\bin\
$ rsync -auzv --rsh=ssh /cygdrive/c/Documents\ and\ Settings/Matt/My\ Documents/My\ Pictures/ 'admin@192.168.1.2:/home/backup/Matt/Freya/My\ Documents/My\ Pictures'
Replace the windows username ("Matt") and the computer name ("Freya") with your own.
Note: Cygwin is very picky about those quotes and backslashes - test them out first on the cygwin command line with a :
ls /cygdrive/c/Documents\ and\ Settings/Matt/My\ Documents/My\ Pictures/
