Recently read a comment here: “Think you have a backup? Delete your drives now. If you’re hesitant, you don’t really have a backup.”
As true as that is: I’d be in the “hesitant” category because I only back up my data / containers, not my bare metal system and its config. Do you guys? (Proxmox users will probably do, but what about the rest?)
I could do that, wouldn’t lose a bit, but I hesitate anyways because downloading a few TB over a 300 Mbit/s Internet connection isn’t fun at all and takes a shitload of time
The whole config is in git. The data gets backed up nightly by restic and pushed to a local backup site, and then to 2 separate offsite backup locations, fulfilling 3-2-1.
Every month on the first day there’s a restore test.
I backup “everything” from my raspberries, and just \home for laptops but for an actual disaster recovery I’d just do a clean install, get an IP address and then reinstall everything via Ansible, then drag my home data back off the NAS
But… I like that statement. I’ll use that at work.
Is your NAS on site? If it is not backed up: I used to follow a 3-2-1 strategy, 3 separate copies, 2 mediums, 1 off site. I have sorta feel to a 2-2-1, I am wanting to get to a 2-2-2ish.
I was following a 2-2 strategy but then 3 houses went went up in flames down the street.
The NAS is on site, yes. It’s backed up to Hetzner - inc. it’s own /etc, /home, etc. so the NAS itself is kinda a 2-1 (2 backups, 1 offsite), but it would be a lot of rework to restore and that’s the device I’d need to recover everything else in case of a total disaster (like a house fire).
(🤔 better check the Hetzner encryption is noted in my keypass file)
I do need to get it’s config setup in Ansible though so it can be rebuilt automatically, but Ansible for single devices can be a little overkill and difficult to maintain.
Similar for my hypervisor box… damn… there goes some more evenings…
I actually did that. Or well, I was planning to upgrade the Debian version and instead of doing it in place I replaced the drive (kept the old around for a while just in case) and did a clean install. I didn’t make any new backups or moved configs or anything outside what I already had, the system was back to operations faily quickly. The main reason, I have a markdown file describing every configuration and recovery step needed to get a fresh system to a fully working state in case of a disaster, and my test showed that it worked great. It does take a bit of effort to keep it up-to-date, but as shown, really worth it, and it douples as a small wiki for accumulated knowledge. I do wish to play with Ansible at some point if I have the time, but this works for now.
whenever the self hosting mood strikes, i try so much off the wall shit that it’d all be irrevocably broken if i wasn’t able to scrap the whole day and reload last nights back up!
i keep each group of services in their own lxc so at least only a part goes down if so!
Recently read a comment here: “Think you have a backup? Delete your drives now. If you’re hesitant, you don’t really have a backup.”
As true as that is: I’d be in the “hesitant” category because I only back up my data / containers, not my bare metal system and its config. Do you guys? (Proxmox users will probably do, but what about the rest?)
I could do that, wouldn’t lose a bit, but I hesitate anyways because downloading a few TB over a 300 Mbit/s Internet connection isn’t fun at all and takes a shitload of time
The whole config is in git. The data gets backed up nightly by restic and pushed to a local backup site, and then to 2 separate offsite backup locations, fulfilling 3-2-1.
Every month on the first day there’s a restore test.
Nice philosophy.
I’m kinda in the middle ground here.
I backup “everything” from my raspberries, and just \home for laptops but for an actual disaster recovery I’d just do a clean install, get an IP address and then reinstall everything via Ansible, then drag my home data back off the NAS
But… I like that statement. I’ll use that at work.
Is your NAS on site? If it is not backed up: I used to follow a 3-2-1 strategy, 3 separate copies, 2 mediums, 1 off site. I have sorta feel to a 2-2-1, I am wanting to get to a 2-2-2ish.
I was following a 2-2 strategy but then 3 houses went went up in flames down the street.
The NAS is on site, yes. It’s backed up to Hetzner - inc. it’s own
/etc,/home, etc. so the NAS itself is kinda a 2-1 (2 backups, 1 offsite), but it would be a lot of rework to restore and that’s the device I’d need to recover everything else in case of a total disaster (like a house fire).(🤔 better check the Hetzner encryption is noted in my keypass file)
I do need to get it’s config setup in Ansible though so it can be rebuilt automatically, but Ansible for single devices can be a little overkill and difficult to maintain.
Similar for my hypervisor box… damn… there goes some more evenings…
I actually did that. Or well, I was planning to upgrade the Debian version and instead of doing it in place I replaced the drive (kept the old around for a while just in case) and did a clean install. I didn’t make any new backups or moved configs or anything outside what I already had, the system was back to operations faily quickly. The main reason, I have a markdown file describing every configuration and recovery step needed to get a fresh system to a fully working state in case of a disaster, and my test showed that it worked great. It does take a bit of effort to keep it up-to-date, but as shown, really worth it, and it douples as a small wiki for accumulated knowledge. I do wish to play with Ansible at some point if I have the time, but this works for now.
whenever the self hosting mood strikes, i try so much off the wall shit that it’d all be irrevocably broken if i wasn’t able to scrap the whole day and reload last nights back up!
i keep each group of services in their own lxc so at least only a part goes down if so!