README

rsback — Back up file trees in rotating archives on Unix-based hosts

Synopsis

To start one or more backup tasks:

rsback [options] list-of-tasks

To display help:

rsback -h

Description

rsback creates rotating backups using the rsync program and standard file utilities on Unix-based backup hosts. Its purpose is to mirror certain file trees from a remote host or from the local system and to store them as rotating archives in backup repositories on the local backup host. The file structure, permissions, ownerships, and timestamps of the mirrored data are identical to those of the original sources.

rsback is a Perl-based front end to rsync that allows a system administrator to configure and execute backups of different file trees located on remote or local hosts (e.g. tasks for hourly, daily, weekly, or monthly backups).

When rsback is executed at regular intervals — preferably via cron jobs — it maintains rotating backup archives. Restoring files from the backup repository requires no special restore procedure: to recover files or directories, simply copy them from the archive tree back to their original location or to any other destination.

The combination of rsync's powerful capabilities and the extensive use of hard links for copying archives within the local file system results in a fast and disk-space-efficient backup technique.

Prerequisites

rsback runs on Unix-based hosts. It has been tested on various Linux distributions. It should also work on other Unix-based systems provided the following programs and utilities are installed:

  • rsync: The most recent version is recommended.

  • Perl 5.8.1 or later

  • Standard file utilities: cp, rm, mv, and mkdir

  • cron or a similar scheduler (recommended)

  • A working knowledge of rsync

  • Root privileges on the backup host

  • To mirror file trees from remote Windows machines, Cygwin is required.

Motivation

rsback was created to address the need for a backup solution suitable for a workgroup server where project folders (10–20 GB) have to be mirrored daily.

After trying several backup techniques without finding a satisfying solution, the author discovered Mike Rubel's examples of rotating rsync snapshots, which appeared to solve the problem. To manage configurations for multiple backup tasks more comfortably, a front end was built on top of Mike's sample scripts — and rsback (short for RSync BACKup) was the result.

How It Works

Example

The following explanations refer to a typical example:

We want to maintain a rotating backup repository of a file tree that resides on a remote host workbox. The remote host runs rsync in daemon mode on TCP port 873. The file tree on workbox consists of all subdirectories and files under /var/projects, accessible via workbox::work. The corresponding entry in the rsyncd configuration file /etc/rsyncd.conf on workbox might look like this:

[work]
path = /var/projects
comment = project directories

Backup Steps

The backup concept of rsback is based on two steps:

  1. Rotation
  2. Backup

The repeated combination of rotation and backup produces backup archives whose content is comparable to classic combinations of full and incremental backups.

Task: work-daily

All files and directories under workbox::work (i.e. workbox:/var/projects) should be saved every weekday night to the local machine backbox. The five most recent daily backup sets should be kept in the backup repository on backbox.

Task: work-weekly

In addition, a weekly backup of the most recent local archive should be made on Saturdays. The four most recent weekly backup sets should also be retained in the repository on backbox.

This way the repository holds the data from the last five working days and weekly snapshots from the last four weeks.

Remark

Tasks are not restricted to daily or weekly intervals. The backup frequency and the number of archives to retain are entirely up to you.

Backup Repositories

Assuming that the local host backbox has a large disk mounted at /backup, that directory will hold all local backup repositories.

Archive Structure

The backup repository on backbox in our example looks like this:

/backup
 +--/work
 |   history.work-daily     history of task work-daily
 |   history.work-weekly    history of task work-weekly
 |   +--/daily.0            most recent daily archive
 |   +--/daily.1            \
 |   +--/daily.2             |
 |   +--/daily.3             |- previous daily archives
 |   +--/daily.4             |
 |   +--/daily.5            /
 |   +--/weekly.0           most recent weekly archive
 |   +--/weekly.1           \
 |   +--/weekly.2            |- previous weekly archives
 |   +--/weekly.3            |
 |   +--/weekly.4           /

The directories daily.0 through daily.5 contain copies of the original data from the most recent daily backup run (daily.0), the run one day before (daily.1), and so on up to five days ago (daily.5). The directories weekly.0 through weekly.4 hold the most recent and previous weekly archives, respectively.

History File

A history file for each backup task records the timestamps of the archives. It consists of a two-column, tab-separated table. Each row contains the backup number in column one and the date and time in ISO 8601 format in column two:

# rsback-0.4.0 (hjb -- 2002-07-16)
0    2002-07-17 22:24:05
1    2002-07-16 22:24:13
2    2002-07-15 22:24:30
3    2002-07-12 22:25:28
4    2002-07-11 22:24:20
5    2002-07-10 22:24:16
6    2002-07-09 20:15:37

The history file is read before a backup task is processed. If no history file exists, one is created from the timestamps of the existing archive tree. After the backup task finishes, the updated history is written back to the file.

Daily Rotation

When a backup task runs, the existing archives are first rotated. In our example:

rm -rf daily.5
mv daily.4 daily.5
mv daily.3 daily.4
mv daily.2 daily.3
mv daily.1 daily.2

The backup set daily.1 is then replaced by a hard-link copy of the most recent backup set daily.0:

cp -al daily.0 daily.1

Daily Backup

Using rsync, the source tree is mirrored from a remote or local file system to the local backup repository. By default, only files and directories that differ from their counterparts in the repository are copied. A file or directory is considered different if its size, timestamp, or ownership has changed since the last backup, or if it does not yet exist in the repository. Items present in the repository but absent from the source tree are removed from the repository.

This is achieved by invoking rsync roughly as follows:

rsync -al --delete ...

Weekly Rotation

The weekly rotation works in the same way as the daily rotation, except that the archives weekly.0 through weekly.4 are rotated.

Weekly Backup

The goal is to take a snapshot of the most recent daily backup archive. Since both source and destination are local directories, this backup is performed by hard-linking daily.0 to weekly.0:

rm -rf weekly.0
cp -alf daily.0 weekly.0

Installation

To install rsback on a backup host, log in as root and proceed as follows.

Copy the downloaded archive rsback-x.y.z.tar.gz (where x.y.z is the version number) to an installation directory, e.g. /usr/local/src. Change to that directory and unpack the archive:

# cd /usr/local/src
# tar zxvf rsback-x.y.z.tar.gz

Copy rsback to a directory in root's PATH, e.g.:

# cp rsback-x.y.z/rsback /usr/local/sbin

Make sure that rsback is executable only by root:

# chmod 700 /usr/local/sbin/rsback

Create a configuration directory and copy the sample configuration files into it:

# mkdir /etc/rsback
# cp rsback-x.y.z/*.conf /etc/rsback

Ensure that only root can read the configuration file:

# chown root:root /etc/rsback/rsback.conf
# chmod 600 /etc/rsback/rsback.conf

You may then remove the archive:

# rm rsback-x.y.z.tar.gz

Configuration

Some configuration parameters are passed directly to rsync as options. It is therefore strongly recommended that you consult the rsync documentation and man pages (rsync(1), rsyncd.conf(5)) if you are unsure what rsback will do. Before running your configuration with production data, test it with dummy data first and compare the results carefully with your expectations.

If your machines are accessible to more than one person, consider the following security precautions:

  • Do not allow data transfers to or from remote machines without authentication or other access controls.
  • Do not transfer passwords in clear text.
  • Do not transfer sensitive data unencrypted.
  • Do not grant write access to backup repositories to anyone other than root@backupbox.
  • Do not grant read access to backup repositories to anyone other than the owner of the original data.
  • Do not grant read or write access to rsback.conf to anyone other than root@backupbox:

    chown root:root /etc/rsback/rsback.conf

    chmod 600 /etc/rsback/rsback.conf

Configuration File

Edit rsback.conf to customise rsback and define your backup tasks.

Default Location

The default configuration file is /etc/rsback/rsback.conf. To use a different location, either edit the variable $rsback_conf in the rsback script, or use the -c option at runtime (see the Usage section below).

File Format

The file format is similar to that of rsyncd.conf(5).

The file is line-based: each line represents a comment, a section name, or a parameter. Lines beginning with # or ;, and lines containing only whitespace, are ignored. The file consists of sections and parameters. A section begins with its name in square brackets and continues until the next section starts. Sections contain parameters of the form name = value.

Global Section

The [global] section defines general configuration parameters. All parameters are mandatory unless marked as optional.

rsback needs to know where to find certain programs. Set the following paths according to your system:

parameter: rsync_cmd

rsync_cmd = /usr/bin/rsync

parameter: cp_cmd

cp_cmd = /bin/cp

parameter: mv_cmd

mv_cmd = /bin/mv

parameter: rm_cmd

rm_cmd = /bin/rm

parameter: mkdir_cmd

mkdir_cmd = /bin/mkdir

parameter: tasks

A list of all backup tasks to be executed. A backup task is simply an arbitrary name identifying a specific backup job. Each task listed here must have a corresponding task section (see below).

parameter: exclude_file (optional)

Path to a file containing global exclude patterns for rsync. These patterns are applied to all backup tasks that use mode = rsync. The value is passed to rsync via the --exclude-from option. Refer to the rsync documentation for the pattern syntax.

parameter: rsync_options

Additional options to pass to rsync for all tasks. For example:

rsync_options = --stats

Task-specific options can be defined within individual task sections and take precedence over this global setting.

parameter: lock_dir

The directory where lock files are created. Specify an absolute path without a trailing slash.

parameter: if_locked_retry (optional)

If the current task is locked by another task, this parameter controls how many times rsback should retry and how long to wait between attempts. The default delay is 10 minutes.

Example — retry three times with a delay of 10 minutes:

if_locked_retry = 3 10m

parameter: ignore_rsync_errors (optional)

A list of rsync exit codes (see rsync EXIT VALUES) that should be treated as non-fatal.

Example — ignore exit codes 12 and 24:

ignore_rsync_errors = 12 24

parameter: if_error_continue (optional)

Controls what happens when an internal command used by rsback fails. If set to no (the default), subsequent tasks on the command line are skipped. If set to yes, only the failing task is aborted and remaining tasks continue normally.

parameter: if_error_undo (optional)

Whether to undo (remove) a defective or incomplete backup set in case of an error. Accepts yes or no (default is yes).

parameter: use_link_dest (optional)

Whether to use rsync's --link-dest option. Accepts yes or no (default is yes).

Task Sections

Parameters specific to individual backup tasks are declared in their own sections. There must be one task section for each task listed in the global tasks parameter. For example, if you have declared:

[global]
tasks = work-daily work-weekly misc

then the following task sections must be present:

[work-daily]
...

[work-weekly]
...

[misc]
...

parameter: mode

Controls the backup mode for this task.

  • mode = rsync — mirror the source tree from a remote or local host using rsync.
  • mode = link — create a hard-link copy of a local source directory. This is only useful when source and destination reside on the same physical partition.

parameter: source

The location of the source data. The format depends on the backup mode and the location of the source files. This value is passed directly to rsync (when mode = rsync) or to cp (when mode = link). Consult rsync(1) and cp(1) for the appropriate syntax.

Example for a remote rsync daemon source:

source = workbox::work/

Example for a local source (as used in the work-weekly task):

source = /backup/work/daily.0

parameter: destination

The local directory that serves as the backup repository root. The actual archive directories will be created as subdirectories of this path, named according to the rotate parameter.

Example:

destination = /backup/work

parameter: rotate

Two values: an arbitrary name for the archive directories, and a positive integer specifying how many backup sets to retain.

Example:

rotate = daily 5

parameter: rsync_options (optional)

Same as the global rsync_options parameter, but applies to this task only and takes precedence over the global setting.

parameter: exclude_file (optional)

Same as the global exclude_file parameter, but applies to this task only. Global and task-specific exclude files are cumulative — both sets of patterns are applied when the task runs.

Example:

exclude_file = /etc/rsback/work-daily.exclude

parameter: suspend_file (optional)

If this file exists, the task is skipped. Useful for temporarily suspending a task without modifying the configuration.

Example:

suspend_file = /var/work/_dont_backup_now_

parameter: trigger_file (optional)

If this file exists, the task runs; otherwise it is skipped. After a successful backup, the trigger file is removed. The file must be writable by rsback.

Example:

trigger_file = /home/fritz/_backup_please_

parameter: use_link_dest (optional)

Same as the global use_link_dest parameter, but applies to this task only.

parameter: update (optional, mode = rsync only)

Performs a simple rsync update of the most recent backup set of the referenced task, without rotating the archives. All other parameters are inherited from the referenced task section.

This is useful for minimising downtime when backing up live systems such as databases or virtual machines. A typical procedure:

  1. Run a backup task of the running system (live backup). Use ignore_rsync_errors = 24 to tolerate vanished source files.
  2. Shut down the system.
  3. Run a task with update = live-backup to synchronise only the changes that occurred since step 1.
  4. Restart the system.

The downtime in step 3 is considerably shorter than a full backup because only the files changed in the meantime need to be transferred.

parameter: pre_hook (optional)

parameter: post_hook (optional)

Run a system command before (pre_hook) or after (post_hook) a task executes.

Example — mount and unmount a Proxmox container before and after backup:

pre_hook  = pct mount 113
post_hook = pct unmount 113

Exclude Files

Patterns for excluding files or directories from rsync are placed in separate files referenced by the exclude_file parameter. Because these files are passed directly to rsync via --exclude-from, they must follow rsync's pattern syntax. Refer to the EXCLUDE PATTERNS section in rsync(1) for details.

Global and task-specific exclude files are cumulative: patterns from both files are applied when a task runs.

Usage

To start one or more backup tasks:

# rsback [options] task-list

where task-list is a space-separated list of backup task names as defined in the configuration file.

Available options:

-h                Display help and exit

-v                Verbose output

-d                Debug mode: runs rsync with --dry-run (simulation only;
                  nothing is actually copied or deleted)

-i                Initialise the backup repositories for the specified tasks.
                  This is rarely needed, as rsback creates missing directories
                  automatically when a task runs.

-c config-file    Use the specified configuration file instead of the default.

Example:

rsback -vc /etc/rsback/test.conf work-daily misc

Scheduling Backup Tasks

rsback is designed to be invoked by cron at regular intervals. Example crontab entries for our scenario:

0 22 * * 1-5  /usr/local/sbin/rsback -v work-daily  >> /var/log/rsback/work-daily.log
0 22 * * 6    /usr/local/sbin/rsback -v work-weekly >> /var/log/rsback/work-weekly.log

The work-daily task runs every weekday night at 22:00; work-weekly runs on Saturday night.

Changes

See CHANGELOG.

Availability

https://www.pollux.franken.de/projekte/rsback

Author

Copyright © 2002–2026 Hans-Jürgen Beie

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.