All tools
ChainToolsBackups Protect

Rsync Backup Tool

Pull timestamped rsync backups of common and host-specific folders from many hosts

View on GitHub
Best forbacking up a fleet of servers

Overview

Rsync Backup Tool is a Python utility that automates rsync backups for multiple hosts. A single YAML file defines the hosts, the folders to back up and where backups are stored; the tool then synchronises those folders from each remote host to a local destination.

Every run lands in its own timestamped directory, so earlier backups are never overwritten.

Key Features

  • Multi-host support — Back up folders from many remote hosts in a single run.
  • Common and host-specific folders — Define folders backed up from every host, plus extra folders per host.
  • Whole folder or top level only — pull_entire_folder syncs a folder recursively, or only the files directly in it.
  • Timestamped backups — Each run creates a new YYYYMMDDHHMM directory under the destination.
  • Logging — Tool log plus a separate rsync log for debugging and monitoring.

Installation

Requirements

  • Python 3.7 or newer
  • PyYAML
  • rsync and an SSH client
git clone https://github.com/qf3l3k/rsync_backup_tool.git
cd rsync_backup_tool
pip install pyyaml

# rsync and ssh must be available
sudo apt-get install rsync openssh-client

Configuration

config.yaml defines the destination, logging and the folders to back up:

destination: /path/to/backup/destination
log_directory: .logs
log_file: rsync_backup_tool.log
log_level: INFO

common_folders:
  - path: /var/log
    pull_entire_folder: true
  - path: /etc
    pull_entire_folder: false

hosts:
  host1:
    specific_folders:
      - path: /home/user1
        pull_entire_folder: true
  host2:
    specific_folders:
      - path: /opt/app
        pull_entire_folder: false
KeyDescription
destinationBase directory where backups are stored
log_directory / log_file / log_levelWhere and how much to log
common_foldersFolders backed up from every host
hosts.<name>.specific_foldersExtra folders for one host
pathFolder on the remote host
pull_entire_foldertrue: sync the folder recursively; false: only files directly in it

Usage

With the configuration above, a run syncs /var/log and /etc from both hosts, /home/user1 from host1 and /opt/app from host2, into /path/to/backup/destination/YYYYMMDDHHMM/. Logs go to the log directory: rsync_backup_tool.log for the tool and rsync.log for rsync output.

python -m rsync_backup_tool --config /path/to/config.yaml

Related Resources

Need something custom?

We build tooling and automation for node operators and chain teams, and can help you deploy or adapt this tool to your setup.