Rsync Backup Tool
Pull timestamped rsync backups of common and host-specific folders from many hosts
Overview
Rsync Backup Tool is a Python utility that automates rsync backups for multiple hosts. A single YAML file defines the hosts, the folders to back up and where backups are stored; the tool then synchronises those folders from each remote host to a local destination.
Every run lands in its own timestamped directory, so earlier backups are never overwritten.
Key Features
- Multi-host support — Back up folders from many remote hosts in a single run.
- Common and host-specific folders — Define folders backed up from every host, plus extra folders per host.
- Whole folder or top level only — pull_entire_folder syncs a folder recursively, or only the files directly in it.
- Timestamped backups — Each run creates a new YYYYMMDDHHMM directory under the destination.
- Logging — Tool log plus a separate rsync log for debugging and monitoring.
Installation
Requirements
- Python 3.7 or newer
- PyYAML
- rsync and an SSH client
git clone https://github.com/qf3l3k/rsync_backup_tool.git
cd rsync_backup_tool
pip install pyyaml
# rsync and ssh must be available
sudo apt-get install rsync openssh-clientConfiguration
config.yaml defines the destination, logging and the folders to back up:
destination: /path/to/backup/destination
log_directory: .logs
log_file: rsync_backup_tool.log
log_level: INFO
common_folders:
- path: /var/log
pull_entire_folder: true
- path: /etc
pull_entire_folder: false
hosts:
host1:
specific_folders:
- path: /home/user1
pull_entire_folder: true
host2:
specific_folders:
- path: /opt/app
pull_entire_folder: false| Key | Description |
|---|---|
destination | Base directory where backups are stored |
log_directory / log_file / log_level | Where and how much to log |
common_folders | Folders backed up from every host |
hosts.<name>.specific_folders | Extra folders for one host |
path | Folder on the remote host |
pull_entire_folder | true: sync the folder recursively; false: only files directly in it |
Usage
With the configuration above, a run syncs /var/log and /etc from both hosts, /home/user1 from host1 and /opt/app from host2, into /path/to/backup/destination/YYYYMMDDHHMM/. Logs go to the log directory: rsync_backup_tool.log for the tool and rsync.log for rsync output.
python -m rsync_backup_tool --config /path/to/config.yamlRelated Resources
- Full documentation — Requirements, installation and usage on docs.chaintools.tech
- rsync manual — Official rsync documentation
- cron — Scheduling regular backup runs
Need something custom?
We build tooling and automation for node operators and chain teams, and can help you deploy or adapt this tool to your setup.
