Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Any advice on reflinks... or XFS in general?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
psycho
Guru
Guru


Joined: 22 Jun 2007
Posts: 548
Location: New Zealand

PostPosted: Thu Feb 27, 2025 1:05 am    Post subject: Any advice on reflinks... or XFS in general? Reply with quote

I was reading the (Gentoo website) article about the new ready-to-run Gentoo images and noticed that it gave (right at the bottom) the use of reflinks as a reason for the decision to go with XFS rather than ext4. Last time I looked at the performance comparisons between ext4 and XFS I decided to stick with ext4, but I'm curious about the possible advantages of XFS and whether reflinks are some kind of life-changing bonus that I ought to be enjoying. From a quick read through various explanations of reflinks they sound useful. Any advice on whether to use or avoid XFS?
Back to top
View user's profile Send private message
szatox
Advocate
Advocate


Joined: 27 Aug 2013
Posts: 3527

PostPosted: Thu Feb 27, 2025 10:49 am    Post subject: Reply with quote

Quote:
Any advice on whether to use or avoid XFS
Without a specified workload: no advice.
All somewhat popular filesystems satisfy basic requirements, with FAT being a notable exception (doesn't have links and permissions, but is simple enough to fit in bios).
You can just roll a die and be happy with the result. If you want to choose the best fs for you, you basically have to know what you're optimizing for. In most use cases it doesn't matter anyway though.

I remember XFS as the go-to fs for storing shitloads of small files. Like ceph (up until v13 or something) and moosefs backends.
_________________
Make Computing Fun Again
Back to top
View user's profile Send private message
psycho
Guru
Guru


Joined: 22 Jun 2007
Posts: 548
Location: New Zealand

PostPosted: Fri Feb 28, 2025 2:48 am    Post subject: Reply with quote

What interests me about reflinks is that they sound like they could save space... but from a quick read through some explanations of how they work, I'm a bit suspicious that I'd be trading some safety for that. At the moment I have lots of long histories of file changes (mostly text, but sometimes including large files like videos... in fact even some huge ones like archives and system backups and so on) that are documented in the most crude and inefficient way possible: I just keep all the copies, from ancient to current, without using any of the clever versioning tricks I've read that you can do with BTRFS or whatever.

The reason for this is partly that I don't like change (I got used to ext2 before newfangled ext3 came along, and so when ext4 also seems to perform better overall than other filesystems based on published benchmarks etc., I just haven't been motivated to move), but also a bit of paranoia around the idea of not having multiple copies as actual physical structures on disks. In other words, the idea of 100 "copies" that are all just diffs of a single shared file sounds like a disaster waiting to happen (i.e. my false sense of security from having hundreds of "copies" is trashed when an error trashes the one file upon which they all depend). This paranoia is probably based on ignorant misunderstandings of how these tricks really work, but I just feel safer doing things the old-fashioned way.

The downside is that I'm forever having to upgrade the drives in various boxes as they run out of space, and SSD storage is still very expensive for 2TB+ drives, so a filesystem change that would magically shrink my data to a fraction of its current size is a tempting idea. I just have no idea if it's a realistic one, or whether reflinks are even relevant to this use case, and if so whether they're as safe as old-fashioned space-hogging copies. To be honest it's hard to pin down a specific workload because my workloads are quite varied, from Moodle servers to laptops, but I prefer to keep things consistent across devices just for my own sanity, so would probably switch from ext4 to XFS right across the board, if it provided a significant benefit anywhere at all without a significant downside.
Back to top
View user's profile Send private message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 23144

PostPosted: Fri Feb 28, 2025 3:08 am    Post subject: Reply with quote

As I understand it, reflinks extend the idea of hard links down to the per-block layer. With hard links, you have two or more names for the same inode, and since the inode owns the data blocks, all the data blocks are implicitly shared between the two names. With reflinks, you direct the kernel to use the same disk block to back multiple files, and enjoy the savings of keeping only one copy of that block. Yes, corruption of that one block will extend to all the files that share it. That is why you keep disk backups, preferably at least one stored in a physically separate location. If your backups of a file are all on the same filesystem, you are very vulnerable to data loss from hardware failure or filesystem metadata corruption.
Back to top
View user's profile Send private message
chaostheo
n00b
n00b


Joined: 12 Feb 2025
Posts: 2

PostPosted: Fri Feb 28, 2025 7:20 pm    Post subject: Reply with quote

The curious advantages of XFS are beneath the endless possible usage of reflinks as kind of snapshots,
the ability to use a dedicated metadata device (eg 2/3 nvme in raid1) next to slower device for data (eg of hdd's/ssd's/nvme's in raid6),
dynamic inode generation instead of ext4 static value and in todays need nearly endless filesystem size.
See https://discussion.fedoraproject.org/t/xfs-with-external-disk-for-journal-metadata/109516 with howto for hybrid xfs on raid1+raid6 for splitting metadata and data,
benchmark tool and results - compare with your own.
Back to top
View user's profile Send private message
psycho
Guru
Guru


Joined: 22 Jun 2007
Posts: 548
Location: New Zealand

PostPosted: Fri Feb 28, 2025 9:39 pm    Post subject: Reply with quote

Thanks folks. I think I'll stick with ext4 and its clumsy old-fashioned copies-are-actually-copies approach. The older I get, the more I feel like my favourite filesystem is actually books on shelves.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum