This post is a review of the different options that we have when deciding on our backup strategy. As more features have appeared over time, this information has become scattered around this blog, so it is nice to have a review in a single article.
It goes without saying that we need to backup our sensitive data in case a data loss event occurs, such as server failure, hard drive failure or a house fire. Many people think about this only too late, and it can be devastating.
Ideally we would have at least two copies, better three, of all our data. This will cover us from hardware failure. If possible, one of them should be in a different location, in case an event such as somebody breaking in our home, or a house fire, and it also would ideally be encrypted.
Now if we are running NCP on a low end hardware, options such as encryption, or RAID can have a prohibitive computational cost. With this in mind, the following are the backup features that have been developed for this scenario.
Periodic backups
This one is the most basic one. Backup our Nextcloud files and database at regular intervals, in case our system breaks or our database becomes corrupted. As we add more and more Gigabytes of data, this can take really long and take up a lot of space.
The data is better off at an external USB drive than at the SD card, not only because it is cheaper per Megabyte, but also because they are more reliable.
Pros:
- Doesn’t require any advanced filesystem such as BTRFS.
- Self contained. We can copy the backup to another NCP instance and restore easily.
Cons:
- Can take up a lot of space, even if compressed.
- They can be slow.
- Our cloud will be inaccessible in maintenance mode during the process.
It is recommended to use a second hard drive to save the backups to, in case the first one fails (instructions). In order to do this just plug in a second hard drive and set it as the destination of backups in nc-backup-auto. Remember that if we are using more than one drive, we shoud reference each one by label in the NCP options as explained here.
Periodic dataless backups
To alleviate some of these issues, we can do dataless backups. These are typically a couple hundred megabytes big and the downtime will be small. The problem with this is that we need to find another way of duplicating the data itself.
Pros:
- Doesn’t require any advanced filesystem such as BTRFS
- Almost self contained. We can copy the backup to another NCP instance and restore our accounts, apps and so on easily.
- The downtime quite small
- Don’t take much additional space
Cons:
- We need to backup our data separatedly
- Restoring can be more complicated to include data
After restoring, we have to edit config.php and point our instance to the path where our data is, then we need to run nc-scan to make Nextcloud aware of the new files.
Periodic BTRFS snapshots
We have dedicated several posts already to BTRFS and its features. NCP will default to format our USB drive to BTRFS, and our datadir will automatically be in a subvolume. Just activate nc-snapshots to get hourly snapshots of your data with btrfs-snp
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
|
$ ls -1 /media/myCloudDrive/ncp-snapshots/
daily_2018-01-30_211703 daily_2018-01-31_221703 daily_2018-02-01_231702 daily_2018-02-02_231703 daily_2018-02-03_231702 daily_2018-02-04_231702 daily_2018-02-05_231702 hourly_2018-02-05_221701 hourly_2018-02-05_231701 hourly_2018-02-06_001701 hourly_2018-02-06_011701 hourly_2018-02-06_021701 hourly_2018-02-06_031701 hourly_2018-02-06_041701 hourly_2018-02-06_051701 hourly_2018-02-06_061701 hourly_2018-02-06_071701 hourly_2018-02-06_081701 hourly_2018-02-06_091701 hourly_2018-02-06_101702 hourly_2018-02-06_111701 hourly_2018-02-06_121701 hourly_2018-02-06_131701 hourly_2018-02-06_141701 hourly_2018-02-06_151701 hourly_2018-02-06_161701 hourly_2018-02-06_171701 hourly_2018-02-06_181702 hourly_2018-02-06_201701 hourly_2018-02-06_211701 hourly_2018-02-06_221701 manual_2017-12-28_113633 monthly_2017-12-28_101702 monthly_2018-01-27_101703 weekly_2018-01-11_111701 weekly_2018-01-18_111702 weekly_2018-01-25_111702 weekly_2018-02-01_111702 |
By virtue of BTRFS being a COW system, the new clones will only take as much additional space as new files we have added since the last snapshot, being space efficient.
Also, the snapshots can be sent incrementally very efficiently to another BTRFS filesystem using btrfs-sync. We can sync to another hard drive on the same machine
, or a BTRFS filesystem on another machine, in which case the transfer is encrypted through SSH.
In a low bandwidth situation, btrfs-sync can send the deltas compressed, but that will take a big toll on the CPU so it is not recommended on low end boards.
Pros:
- Virtually instant, no matter how many Terabytes we have
- No downtime
- Space efficient
- Can be synced efficiently to another hard drive or another machine through SSH by using btrfs-sync
Cons:
- Need to run on BTRFS, which is the default filesystem that nc-format-USB uses
- If we want to sync it to another USB drive it also needs to use BTRFS
- If we want to sync it remotely we neend a BTRFS subvolume on the other end and setting up SSH credentials
If we only care about our data this can mean zero downtime and an efficient means of prevent accidental deletion. In fact, I have the Trash and Versions apps disabled. I recommend combining nc-snapshots and nc-snapshot-sync with dataless backups to get the best of both worlds.
Rsync
Another option to save our data remotely is to sync it through rsync. This is also quite efficient but compared to BTRFS snapshots we won’t retain a history of the datadir, just the latest version.
Pros:
- Doesn’t require a particular filesystem on either end
- Efficient delta sync, only copies the new files
Cons:
- Need to setup SSH credentials
- Not able to sync snapshots, it will only mirror the latest version of our datadir.
This option also allows us to keep our data safe if something happens in our house, and is more flexible as it doesn’t require a BTRFS filesystem in the other end.
Make sure you keep your data with you, we can never be too safe!
Source