I have a load-bearing raspberry pi on my network - it runs a DNS server, zigbee2mqtt, unifi controller, and a restic rest server. This raspberry pi, as is tradition, boots from a microSD card. As we all know, microSD cards suck a little bit and die pretty often; I’ve personally had this happen not all that long ago.
I’d like to keep a reasonably up-to-date hot spare ready, so when it does give up the ghost I can just swap them out and move on with my life. I can think of a few ways to accomplish this, but I’m not really sure what’s the best:
- The simplest is probably cron + dd, but I’m worried about filesystem corruption from imaging a running system and could this also wear out the spare card?
- recreate partition structure, create an fstab with new UUIDs, rsync everything else. Backups are incremental and we won’t get filesystem corruption, but we still aren’t taking a point-in-time backup which means data files could be inconsistent with each other. (honestly unlikely with the services I’m running.)
- Migrate to BTRFS or ZFS, send/receive snapshots. This would be annoying to set up because I’d need to switch the rpi’s filesystem, but once done I think this might be the best option? We get incremental updates, point-in-time backups, and even rollback on the original card if I want it.
I’m thinking out loud a little bit here, but do y’all have any thoughts? I think I’m leaning towards ZFS or BTRFS.
Perhaps the best answer by far is ZFS but I don’t know how much pain it is to set it up to boot from on a Pi. The easiest to setup is probably LVM.
With ZFS you can trivially keep a hot spare even over the network. Just tell syncoid where to replicate.
ZFS isn’t going to perform well on a Raspberry Pi
I used it on a Pi 4 in 2019 for an USB-connected mirror and it worked well. Unencrypted throughput was upwards from 200MB/s. Encrypted throughput dropped down to under 100MB/s due to insufficient compute. The Pi 4 is a powerful computer and the Pi 5 even more so. Pi 3 and older, not so much.
It’s a 4gb pi4, think it could boot from ZFS?
You could boot from from an SD and then store data on a external drive. I would go Btrfs over ZFS unless you have at least 3 or more disks
I would try it. My only issue is I have no idea how to set it up on root on a Pi. Perhaps there’s docs somewhere. If had to setup a new Pi with Pi OS/Debian/Ubuntu today I’d definitely try it. Most of my Pis are running OpenWrt though.
Performance is all but irrelevant in this case
You say that but when the system starts to lock up you might change your tune
Why would it lock up? ZFS will use as much RAM as you give it and it doesn’t seem CPU-bound unless you turn on encryption. It’s not a cluster FS like Ceph. Why do you expect ZFS to lock up and Btrfs not to?