How to Mirror Large Directories with Rclone Sync to Storage VPS
If you are working with large amounts of data, having a reliable, exact backup is essential for protecting your files and getting them back quickly after a disaster. Setting up an rclone sync to storage VPS provides a secure, efficient, and automated way to achieve this.
By using a high-capacity Storage VPS via an encrypted SFTP connection, you can securely mirror your directories without needing to install any extra server-side software on the destination.
This guide covers the exact steps to install, configure, execute, and automate this process.
Table of Contents
Prerequisites for Setting up an Rclone Sync to Storage VPS
Before setting up an Rclone Sync to Storage VPS, make sure you have the following:
- A Linux VPS that contains the directory you want to sync.
- A Storage VPS as the destination server where your mirrored files will be stored.
- SSH access to both servers.
- The IP address, SSH port, username, and password or SSH key for the destination server.
- Root privileges or a sudo-enabled user on the source VPS.
- A stable internet connection between the source VPS and the storage server.
- Enough free disk space on the destination server to hold the full copy of your data.
If you are using PerLod Hosting, you can use your main VPS as the source server and your PerLod Hosting Storage VPS as the remote destination for the sync job.
Once you are done with these prerequisites, proceed to the following steps to set up an rclone sync to storage VPS.
Install Rclone on the Source Server
The first step is to install Rclone on the server that currently holds your files. You only need to install it on the source server; your destination Storage VPS doesn’t require any extra software. The destination only requires standard SSH and SFTP access.
To install the latest stable version of Rclone on Linux, you can run the official installation script:
sudo -v ; curl https://rclone.org/install.sh | sudo bash
This downloads the rclone binary, places it in the correct system directory (/usr/bin/), and sets the required execution permissions.
Connect Rclone to the Storage VPS
At this point, you need to tell your source server how to access your destination server. You can use the rclone built-in configuration menu to securely link your main server to your Storage VPS using SFTP.
Launch the rclone interactive configuration menu with the command below:
rclone config
- Press n to create a New remote and give it a name like remote_storage.
- For the storage type, type sftp and press Enter.
- Enter your connection details when prompted:
- Host: The IP address or domain of your remote server.
- User: Your SSH username.
- Port: The SSH port. Press Enter to use the default 22, or specify your custom port.
- Password / Key: Choose y to type in your SSH password, or provide the absolute path to your SSH private key.
- Leave the advanced configuration blank by pressing Enter, then press y to confirm and q to quit the config menu.
Mirror Data to the Storage VPS using Rclone Sync Command
Now you are ready to transfer your data. Unlike the standard copy command, rclone sync makes the destination directory exactly match the source. This means it will securely delete files on the destination if they no longer exist on the source.
Before running a real sync, you can simulate the transfer to see exactly what will be copied or deleted:
rclone sync /path/to/local/dir remote_storage:/path/to/remote/dir --dry-run -P
The –dry-run runs a safe test to see what will happen, without actually copying or deleting any files.
Once you have verified the simulation, run the actual rclone mirroring command:
rclone sync /path/to/local/dir remote_storage:/path/to/remote/dir -P --transfers 8 --checkers 8
Explanation of the command:
- sync: Makes the destination exactly match the source.
- /path/to/local/dir: The folder on your current server that you want to copy.
- remote_storage:/path/to/remote/dir: Your configured remote name and the destination folder path.
- -P: Shows live transfer progress, speed, and ETA.
- –transfers 8: Copies 8 files at the same time to speed up the transfer.
- –checkers 8: Scans 8 files at once to quickly find differences between the servers.
Run Rclone Sync Automatically
Running the sync manually every day is time-consuming. To keep your data safely mirrored without any extra effort, you can schedule Rclone to run automatically in the background using a Linux cron job.
Open the cron editor with the command below:
crontab -e
Add the following line to run the sync every night at 2:00 AM, logging the output for monitoring:
0 2 * * * rclone sync /path/to/local/dir remote_storage:/path/to/remote/dir --transfers 8 >> /var/log/rclone-sync.log 2>&1
Note: Skip the -P flag in cron jobs so your log files stay clean.
That’s it, you are done with setting up an rclone sync to storage VPS.
FAQs
What is the difference between rclone sync and rclone copy?
rclone copy only adds files. rclone sync creates an exact mirror and deletes destination files that are missing from the source.
Can I limit the internet bandwidth Rclone uses?
Yes, use the –bwlimit flag. For example, adding –bwlimit 10M to your command will limit the upload speed to 10 Megabytes per second.
How do I exclude specific folders or file types from Rclone syncing?
You can use the –exclude flag. For example, if you don’t want to copy log files, you can add –exclude “*.log” to your sync command.
Conclusion
Using Rclone is a simple and reliable way to mirror your files to a Storage VPS. Once everything is configured, you can run manual syncs or automate the process with a cron job to keep your backup up to date.
We hope you enjoy this guide on setting up an rclone sync to storage VPS. Subscribe to our X and Facebook channels to get the latest updates.
For further reading:
Configure Secure NFS File Shares for Backups and Media