| View previous topic :: View next topic |
| Author |
Message |
Massimo B.
Veteran
Joined: 09 Feb 2005
Posts: 1816
Location: PB, Germany
|
 Posted: Fri Jul 14, 2017 6:54 am Post subject: Gentoo on BTRFS-on-LUKS |
 |
|
BTRFS was the best change I did in the last 2 or 3 years
Quote from my discussion on #gentoo with someone that asked to get convinced to use btrfs...
| Quote: | Yes, I had exactly one crash with btrfs under heavy testing. Yes, that could also break other filesystems but not that bad and unrecoverable like the crashed btrfs: I had 1000+ snapshots, I was deleting all snapshots, the fs was filled up to 99%, I was halting the system, and as it was freezing I did a hard shutdown via Sysreq, as this did not help I switched off power.
But now to the good parts: btrfs changed my life of backups. I have a "time machine" as Apple calls it, I can snapshot everything everyday using btrbk, without much disk space. I send those snapshots to a remote btrfs, without much IO, very fast.
I have rotating btrfs snapshots of all subvolumes for root, home, data.. I have portage creating snapshots for every package INSTALL phase, that means packages breaking the system I have a snapshot for the time before installation. I can boot into different root snapshots, which I never do as I can also chroot there, quickpkg, and use the tarball to repair my running system.
I have only one big LUKS and use btrfs-on-luks, no need for LVM or different LUKS and partitions. Having no partitions but only subs does scale very well. I have btrfs compression enabled everywhere, zlib on slow hdd, and lzo on fast hdd or ssd.
btrbk runs by portage triggers and by daily cronjob.
It's backgrounded and transparent. I don't think about snapshots anymore, I just have them when I need them. |
Here is my setup:
| Code: | # eix -Ic btr
[I] app-backup/btrbk (0.25.0@21.03.2017): Tool for creating snapshots and remote backups of btrfs subvolumes
[I] sys-fs/btrfs-progs (4.11@30.05.2017): Btrfs filesystem utilities |
This is sys-kernel/liquorix-sources: | Code: | # uname -a
Linux moralin71 4.11.8-zen #3 ZEN SMP PREEMPT Mon Jul 3 14:54:43 CEST 2017 x86_64 Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz GenuineIntel GNU/Linux |
| Code: | # grep btrfs /etc/fstab
LABEL=gentoo / btrfs defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=root,commit=100 0 1
LABEL=gentoo /mnt/btrfs-top-lvl btrfs defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=/ 0 1
LABEL=gentoo /home btrfs defaults,noatime,compress=lzo,autodefrag,subvol=home 0 1
LABEL=gentoo /usr/portage btrfs defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=portage 0 1
LABEL=gentoo /var/cache btrfs defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=var_cache 0 1
LABEL=gentoo /.snapshots btrfs defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=snapshots 0 1
LABEL=gentoo /mnt/data btrfs defaults,noatime,nodiratime,compress=lzo,autodefrag,subvol=data 0 1 |
| Code: | # grep btrfs /etc/autofs/auto.usb
data -fstype=btrfs,defaults,users,noauto,noatime,nodiratime,compress=zlib,autodefrag LABEL=usb_data
|
All further LUKS devices are opened by a key stored on the encrypted root fs:
| Code: | # grep -v "^#" /etc/conf.d/dmcrypt
dmcrypt_key_timeout=1
dmcrypt_retries=5
swap=swap_crypt_1
source='PARTUUID=5e974344-05'
options='--cipher aes-xts-plain64 --key-size 512 --key-file /dev/urandom'
pre_mount='mkswap -f ${dev} -L swap_crypt_1'
target=ssd_crypt
source='/dev/disk/by-id/ata-SAMSUNG_MZ7PA256HMDR-010H1_S0NVNEAB900378-part6'
key='/etc/secret/key'
target=usbdata_crypt
source='/dev/disk/by-id/usb-ST375064_0A_D-0:0'
key='/etc/secret/key' |
| Code: | # grep -v "^#" /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="keymap=de splash=silent,theme:livecd-2007.0 console=tty1 quiet crypt_root=UUID=e606e66c-419f-4677-a2fe-872290730e6f root=LABEL=gentoo dobtrfs" |
| Code: | # cat /etc/portage/bashrc
# (...)
if [[ ${EBUILD_PHASE} == "preinst" || ${EBUILD_PHASE} == "prerm" ]]; then
einfo "Creating btrfs snapshot..."
btrbk -c /etc/btrbk/btrbk_portage.conf run |grep "^\(===\|+++\|---\|\*\*\*\|>>>\)"
fi |
As portage is running a different retention policy, I have 2 different btrbk configurations:
| Code: | # cat /etc/btrbk/btrbk.conf
transaction_log /var/log/btrbk.log
timestamp_format long-iso
archive_preserve_min latest
archive_preserve 12m 10y
volume /mnt/btrfs-top-lvl/
snapshot_create onchange
subvolume root
group root
snapshot_dir snapshots/root
snapshot_preserve_min 5d
snapshot_preserve 2d 1w 1m
target send-receive /mnt/usb/data/snapshots/root
target_preserve_min latest
target_preserve 1d 4w 2m 1y
subvolume home
group home
snapshot_dir snapshots/home
snapshot_preserve_min 3d
snapshot_preserve 4d 4w 2m 1y
target send-receive /mnt/usb/data/snapshots/home
target_preserve_min 3d
target_preserve 6d 5w 4m 3y
subvolume data
group home
snapshot_dir snapshots/data
snapshot_preserve_min latest
snapshot_preserve no
target send-receive /mnt/usb/data/snapshots/data
target_preserve_min latest
target_preserve 2m |
| Code: | # cat /etc/btrbk/btrbk_portage.conf
transaction_log /var/log/btrbk.log
timestamp_format long-iso
archive_preserve_min latest
archive_preserve 12m 10y
volume /mnt/btrfs-top-lvl/
snapshot_create onchange
subvolume root
group root
snapshot_dir snapshots/root
snapshot_preserve_min 5d
snapshot_preserve 2d 1w 1m |
Beside portage triggers, I'm running btrbk either manually or as daily cronjob. As cron daemon I'm using sys-process/fcron-3.2.1-r1 for load and uptime related scheduling. There I do the btrbk backups
| Code: | # fcrontab -u root -l
2017-07-14 08:38:11 INFO listing root's fcrontab
# vim: set shiftwidth=8 softtabstop=8 noexpandtab: # vim-modline, don't delete
!mail(false)
#--------------------------------------------------------------------------------------
# ENTRIES BASED ON ELAPSED SYSTEM UP TIME
#@options,FirstExec,LoadAvg1m,5m,15m,timeout eq. command
# HEAVY
%daily,first(10),lavgand,lavg(4,4,4),noticenotrun,nice(15) * * ionice -c 3 /usr/sbin/btrbk -c /etc/btrbk/btrbk.conf run && btrfs subvolume delete -C /.snapshots/data/*
@first(6h6),lavgand,lavg(4,4,4),noticenotrun,nice(15) 2w /sbin/btrfs scrub start -c3 -B -d /
@first(7h7),lavgand,lavg(4,4,4),noticenotrun,nice(15) 2w /sbin/btrfs scrub start -c3 -B -d /mnt/usb/data
@first(8h8),lavgand,lavg(4,4,4),noticenotrun,nice(15) 1w ionice -c 3 schedtool -D -e duperemove --hashfile=/root/.duperemove/hashfile -dr /home/mb/
# Light
@first(20),lavg1(4),noticenotrun,nice(15) 1d ionice -c 3 /sbin/fstrim -av |
And emulating the vixie cron behaviour:
| Code: | # fcrontab -u systab -l
2017-07-14 08:41:29 INFO listing systab's fcrontab
# vim: set shiftwidth=8 softtabstop=8 noexpandtab: # vim-modline, don't delete
!mail(false)
#--------------------------------------------------------------------------------------
# *LY KEYWORDS min hours days command
%hourly,first(5),lavg1(4),until(30),nolog * run-parts /etc/cron.hourly
%daily,first(1h15),lavgand,lavg(4,4,4),until(6h) * * run-parts /etc/cron.daily
%weekly,first(2h30),lavgand,lavg(4,4,4),until(3d) * * run-parts /etc/cron.weekly
%monthly,first(3h45),lavgand,lavg(4,4,4),until(5d) * * * run-parts /etc/cron.monthly |
If you wonder about the job doing btrfs subvolume delete -C /.snapshots/data/*:
I had issues with btrfs used as location for VirtualBox machines. Especially the linux guests were running fine but windows guests got bluescreens, probably due to IO timeouts which is very weird... Anyway I like to avoid snapshots on those subvolumes reducing the fragmentation. However I wanted to have a snapshot on the remote btrfs only, as backup. For the running system, having a snapshots leads to fragmentation as soon as changes are happening. As btrbk does not provide a method to create a read-only snapshot, send that and drop it locally, I delete the local ones myself after every btrbk run. The feature request for btrbk was filed as https://github.com/digint/btrbk/issues/151.
| Code: | # tree -L 2 /.snapshots/
/.snapshots/
├── data
│ └── data.20170714T063834+0200
├── home
│ ├── home.20170103
│ ├── home.20170502T073537+0200
│ ├── home.20170607T071118+0200
│ ├── home.20170612T071617+0200
│ ├── home.20170620T063932+0200
│ ├── home.20170626T071700+0200
│ ├── home.20170703T070612+0200
│ ├── home.20170710T085342+0200
│ ├── home.20170711T065304+0200
│ ├── home.20170712T071711+0200
│ ├── home.20170713T064816+0200
│ └── home.20170714T063834+0200
└── root
├── root.20170607T071118+0200
├── root.20170703T070612+0200
├── root.20170710T085342+0200
├── root.20170710T093949+0200
├── root.20170710T094132+0200
├── root.20170710T094627+0200
├── root.20170710T094729+0200
# (..) and ~100 more
|
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770 |
|
| Back to top |
|
 |
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3765
Location: Rasi, Finland
|
| Posted: Sat Jul 22, 2017 11:26 am Post subject: |
|
|
Hi.
I can also confirm that btrfs has been great (for me) and I've never lost any data because it.
When I had some problems with my new MB and I was trying to figure out where the problem was, I accidentaly pulled out one of my SSDs while the system was running. I meant to first remove it from the btrfs pool but forgot to do that. All data was intact and I could easily insert the cold"pulled" SSD back to the btrfs pool. After scrubbing and balancing the data, everything was like before. Also many sudden systemd freezings and power outakes - btrfs has managed to repair itself.
It's a good choice for easy multi device storage.
_________________
..: Zucca :..
| My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
kakarot
n00b
Joined: 02 Feb 2004
Posts: 10
|
| Posted: Fri Dec 29, 2017 9:32 pm Post subject: |
|
|
Dear Massimo,
Could you please provide a step by step guide?
Thank you a priori. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
