| View previous topic :: View next topic |
| Author |
Message |
mkoniarz
n00b
Joined: 10 Nov 2008
Posts: 13
Location: Zakopane
|
 Posted: Sat Jun 02, 2018 11:17 am Post subject: bcachefs |
 |
|
I'm curious if anyone was testing
https://bcachefs.org/
??
Would anyone be nice enough to prepare ebuild ? (bcachefs-tools)
Or where to start. maybe I could done by myself and share here? |
|
| Back to top |
|
 |
Naib
Watchman
Joined: 21 May 2004
Posts: 6067
Location: Removed by Neddy
|
| Posted: Sat Jun 02, 2018 1:51 pm Post subject: |
|
|
I am looking forward to this.
zfs is out of kernel (been burnt too many times... if my system cannot boot with a stock kernel I am not interested). btrfs seems to have messed up and redhat have their userland ontop of xfs pipedream....
I have just created a 1st pass ebuild
| Code: | # Copyright 1999-2015 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
EAPI=5
EGIT_NONSHALLOW=true
inherit git-r3 toolchain-funcs udev
DESCRIPTION="Tools for bcachefs"
HOMEPAGE="http://bcachefs.evilpiepirate.org/"
SRC_URI=""
EGIT_REPO_URI="https://evilpiepirate.org/git/bcachefs-tools.git"
SLOT="0"
LICENSE="GPL-2"
KEYWORDS="~amd64"
IUSE=""
RDEPEND=""
DEPEND="${RDEPEND}
sys-apps/attr
sys-apps/util-linux
app-crypt/libscrypt
dev-libs/libsodium
sys-apps/keyutils
dev-libs/userspace-rcu
dev-util/pkgconfig
sys-libs/zlib
app-arch/zstd
"
|
--edit--
bug submitted https://bugs.gentoo.org/657120
_________________
| Quote: | | Removed by Chiitoo |
|
|
| Back to top |
|
|
msst
Apprentice
Joined: 07 Jun 2011
Posts: 259
|
| Posted: Sat Jun 02, 2018 9:37 pm Post subject: |
|
|
| Quote: | | btrfs seems to have messed up |
Maybe you have other info than I do, but it works stable and without problems since quite a while now. It is also in the kernel and not going to vanish there. Some distrios have even made it their default filesystem. RedHat decided in the opposite direction, but that won't make it vanish. |
|
| Back to top |
|
|
Naib
Watchman
Joined: 21 May 2004
Posts: 6067
Location: Removed by Neddy
|
| Posted: Sat Jun 02, 2018 9:57 pm Post subject: |
|
|
| msst wrote: | | Quote: | | btrfs seems to have messed up |
Maybe you have other info than I do, but it works stable and without problems since quite a while now. It is also in the kernel and not going to vanish there. Some distrios have even made it their default filesystem. RedHat decided in the opposite direction, but that won't make it vanish. |
https://www.patreon.com/bcachefs
| Quote: | | btrfs, which was supposed to be Linux's next generation COW filesystem - Linux's answer to zfs. Unfortunately, too much code was written too quickly without focusing on getting the core design correct first, and now it has too many design mistakes baked into the on disk format and an enormous, messy codebase - bigger that xfs. It's taken far too long to stabilize as well - poisoning the well for future filesystems because too many people were burned on btrfs, repeatedly (e.g. Fedora's tried to switch to btrfs multiple times and had to switch at the last minute, and server vendors who years ago hoped to one day roll out btrfs are now quietly migrating to xfs instead). |
sure this is the author of bcachefs so it probably has some bias, but outright slander wouldn't last long
_________________
| Quote: | | Removed by Chiitoo |
|
|
| Back to top |
|
|
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5937
|
|
| Back to top |
|
|
Naib
Watchman
Joined: 21 May 2004
Posts: 6067
Location: Removed by Neddy
|
| Posted: Sun Jun 03, 2018 8:25 am Post subject: |
|
|
that's bcache not bcache.
_________________
| Quote: | | Removed by Chiitoo |
|
|
| Back to top |
|
|
msst
Apprentice
Joined: 07 Jun 2011
Posts: 259
|
| Posted: Sun Jun 03, 2018 9:04 am Post subject: |
|
|
| Quote: | | sure this is the author of bcachefs so it probably has some bias, but outright slander wouldn't last long |
It is clearly an opinion, he seems to dislike btrfs coding design (cannot coment on this) and what is correct IMHO is that the early releases were quite alpha and produced problems.They were not tagged as stable, but people tried them and expected something else they got. And yes, development was slow as is true for many open source projects.
Since more than 3 years I am now using btrfs and it has not yet produced any problem including raid 0/10. Facebook seems to be using it in production as well. One can consider it stable with the exception of raid 5/6 (which may never get stable as these raid levels are becoming obsolete quickly).
| Quote: | | Also, isn't stratis just lvm/dm+xfs? |
Red Hats decision seems somewhat political and I have no idea what stratis will be. If they make it just a repackaging of lvm+dm+xfs then it won't be able to compete. Who knows what the future will bring, but brtrfs will not go away from the kernel for many years, so that is no reason to avoid it.
And in the moment btrfs is much more established, adopted and tried than bcachefs. Kent will have to do some catch up work. Making new filesystems is certainly a daunting job. |
|
| Back to top |
|
|
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5937
|
| Posted: Sun Jun 03, 2018 9:25 am Post subject: |
|
|
| msst wrote: | | Who knows what the future will bring, but brtrfs will not go away from the kernel for many years, so that is no reason to avoid it. |
I'm quite happy with openzfs and I don't care whether it hits mainline or not. I also don't care if Oracle eventually ports their v28-v43 closed source zfs to their "unbreakable" redhat clone.
Call me biased but I'm quite happy with my storage situation right now. 
But good for them, there is always more room for competition.
_________________
| Neddyseagoon wrote: | | The problem with leaving is that you can only do it once and it reduces your influence. |
banned from #gentoo since sept 2017 |
|
| Back to top |
|
|
Naib
Watchman
Joined: 21 May 2004
Posts: 6067
Location: Removed by Neddy
|
| Posted: Sun Jun 03, 2018 10:48 am Post subject: |
|
|
| msst wrote: | | Quote: | | sure this is the author of bcachefs so it probably has some bias, but outright slander wouldn't last long |
It is clearly an opinion, he seems to dislike btrfs coding design (cannot coment on this) and what is correct IMHO is that the early releases were quite alpha and produced problems.They were not tagged as stable, but people tried them and expected something else they got. And yes, development was slow as is true for many open source projects.
Since more than 3 years I am now using btrfs and it has not yet produced any problem including raid 0/10. Facebook seems to be using it in production as well. One can consider it stable with the exception of raid 5/6 (which may never get stable as these raid levels are becoming obsolete quickly). |
I agree his statements are opinionated, especially as he has a "competitor" ... but this isn't the 1st time I have come across statement with respect to poor direction with btrfs. RH dropping it was more than likely politically driven BUT equally something had to aggravate them, ie BTRFS not producing what was needed...
| Quote: |
| Quote: | | Also, isn't stratis just lvm/dm+xfs? |
Red Hats decision seems somewhat political and I have no idea what stratis will be. If they make it just a repackaging of lvm+dm+xfs then it won't be able to compete. Who knows what the future will bring, but brtrfs will not go away from the kernel for many years, so that is no reason to avoid it.
And in the moment btrfs is much more established, adopted and tried than bcachefs. Kent will have to do some catch up work. Making new filesystems is certainly a daunting job. |
My concerns with Stratis is it will hook deeply into systemd as the aim appears to use a proven filesystem and bolt onto it "next gen" features in userland via daemons...
So essentially the next set of filesystems are...zfs,btrfs,stratis,bcachefs
ZFS would be the preferred option but the license issue stops it being mainlined (there is talks though...). For some this isn't an issue. HOWEVER I have been burnt in the past (ReiserFS on Mandrake, FakeRAID on Gentoo) and as such anything needed to boot my system must be in the vanilla source.
BTRFS looks like a possible alternative to ZFS but I keep coming across concerns ( straight from there wiki... WARNING: kernels 4.14.25 - 4.14.27 and 4.15.8 - 4.15.9 on big endian machines will damage a filesystem after first umount. https://btrfs.wiki.kernel.org/index.php/Main_Page) Its thinks like that that concerns me "don't use those kernels" ... and when the next issue comes along?
ZFS and BTRFS being developed by Oracle is also oO seems a waste of resource (I know btrfs was started by oracle and they then acquired zfs via sun but still...)
STRATIS... political and I have fears of hooks into systemd. Whether it is an open threat to get BTRFS or ZFS-GPL who knows...
BCACHEFS its intriguing I do like how they have targeted stability and core FS before jumping onto shinies. That was always my concern with BTRFS... they kept pushing the latest buzword stuff but then breaking diskformat or breaking the format 
_________________
| Quote: | | Removed by Chiitoo |
|
|
| Back to top |
|
|
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5937
|
| Posted: Sun Jun 03, 2018 11:47 am Post subject: |
|
|
Not to derail the thread, but openzfs is actually maintained quite well in gentoo. I talk to several gentoo developers (+ foundation members) and users on a daily basis through freenode #zfsonlinux.
Also, openzfs hasn't been maintained by Sun/Oracle since they closed their source in 2005, it's maintained by former Sun employees, illumos (formerly known as OpenSolaris) developers, freebsd developers, linux developers, mac developers, and most importantly, the US Government. (Not to mention countless randoms with no programming experience.)
(I guess technically speaking, Oracle never maintained it, one of the first things they did was to close it, so we shouldn't even be using the O word.  )
That said, if you ever do play around with it and have questions, you know where to find me.
_________________
| Neddyseagoon wrote: | | The problem with leaving is that you can only do it once and it reduces your influence. |
banned from #gentoo since sept 2017 |
|
| Back to top |
|
|
happosai
n00b
Joined: 17 Aug 2005
Posts: 43
|
| Posted: Mon Jun 11, 2018 8:54 pm Post subject: |
|
|
| msst wrote: |
Since more than 3 years I am now using btrfs and it has not yet produced any problem including raid 0/10. Facebook seems to be using it in production as well. One can consider it stable with the exception of raid 5/6 (which may never get stable as these raid levels are becoming obsolete quickly).
| Quote: | | Also, isn't stratis just lvm/dm+xfs? |
Red Hats decision seems somewhat political and I have no idea what stratis will be. If they make it just a repackaging of lvm+dm+xfs then it won't be able to compete. Who knows what the future will bring, but brtrfs will not go away from the kernel for many years, so that is no reason to avoid it.
And in the moment btrfs is much more established, adopted and tried than bcachefs. Kent will have to do some catch up work. Making new filesystems is certainly a daunting job. |
Btrfs has been in development for over ten years now, and while the basic stuff is being considered stable, many of the more advanced features are in development, not really finalized yet and some are even unstable - check this out: https://btrfs.wiki.kernel.org/index.php/Status. If you want an even more sceptic analysis of its status, look here: https://github.com/mosteo/btrfs-status
The show stopper for enterprise use is in many cases that Btrfs doesn't support triple parity, while ZFS has been offering this feature since ages. Another show stopper for using btrfs is that it just plainly sucks with VM images and databases, in fact the official recommendation for both is to switch COW off then, because otherwise your file might fragment in literelly ten thousand of pieces or more. But what's the point in using VM images/database storage on a COW file system, when you cannot use the COW features then? For both usage types you are way better of with XFS, or ext4 - in that order.
Also another problem with the still unstable state of btrfs is that putting it on a server means that you should be able to roll out the latest and greatest kernel version every release cycle, because it may contain important bug fixes and performance optimizations. The thing is if your preferred Linux distribution doesn't do backports, than choosing Btrfs might not be such an exactly brillant idea, because when you are running into issues the developers will tell you to switch to the latest and greatest kernel.
Facebook by the way has been testing it in production, but mostly for the deployment and management of system partitions, and not much more. Btrfs on Red Hat has never been a first class citizen, but yes, they lost faith in it completely.
If you want to get real insights what using btrfs in production use meaned, just look at CoreOS. They rolled it out as their default file system in 2014, and the complaints and problems about it skyrocketed to such a boiling point, that they ditched it entirely back then and switched to ext4 + overlayfs early in 2015. The most typical complaints were out of disk space errors, metadata rebalancing problems, and generally slow performance compared to other file systems (https://lwn.net/Articles/627232/). Rebalanciing problems are even an issue in 2018: https://www.spinics.net/lists/linux-btrfs/msg74892.html
Russel Coker from Australia is writing in his personal blog about his experience with Btrfs on his own machines, which he started bloggin about in 2012 and you might also want to check here: https://etbe.coker.com.au/ In the first few years he had a major hiccup almost every six months or so, were he had to ask the mailing list for help, but then it became more reliable and it slowed down.
I also used it several times on my own, and it always ran okay for some weeks then, when problems started to occur and the fun stuff happened. And RAID1 is now nothing fancy in particular. I always had a backup of my data, just in case, and I always needed it.
Having said that, Btrfs is fine to play around with in my opinion if you are using it with throwaway data only, or you get a regular backup of your data. I would never, ever use it on a production level system.
Synology though does use it since over two years as standard file systems, and they seem do be quite happy with it.
To make it short: if you really need a COW file system on Linux, then take ZFS. If you have issues with it on a Linux machine due to CCDL vs GPL, then switch over to FreeBSD. Btrfs has been a nice idea, but is still to much construction area, and a convoluted mess in reality which I would not trust my data with. And BcacheFS might be what could become the new king, but give it at least two or three years, because they are still lacking some basic features like scrubbing. The thing though is that many people, just like myself, have lost their faith in Btreefs completely, so time is not such a pressing issue for BcacheFS at the moment as some might think.
And by the way Oracle recently in February this year completely unexpected relicensed another cron jewel from Sun, dtrace, under the GPL. Maybe ZFS might follow, who knows? |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Tue Jun 12, 2018 12:41 pm Post subject: |
|
|
I like the sound of it; I've always wondered why people don't talk about the bcache more, since it underpins everything, and has been around since the beginning of UNIX. (cf: Lions, Bach.)
It's the original "page cache", afaic (and I don't think you really need another one in-kernel; just memory areas or "extents".) |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Tue Jun 12, 2018 6:32 pm Post subject: |
|
|
| Bah. Still waiting for Tux3. |
|
| Back to top |
|
|
Naib
Watchman
Joined: 21 May 2004
Posts: 6067
Location: Removed by Neddy
|
|
| Back to top |
|
|
Naib
Watchman
Joined: 21 May 2004
Posts: 6067
Location: Removed by Neddy
|
| Posted: Mon Apr 22, 2019 12:11 pm Post subject: |
|
|
update:
https://www.patreon.com/bcachefs
| Quote: | Apr 20 at 4:39am
Fully persistent allocation info is finally done
Finally! It was a huge effort, but it's done and pushed out.
This means that when mounting a filesystem - even after an unclean shutdown - we don't have to walk all the metadata anymore, because it's always updated in a transactional manner and kept fully consistent in the b-tree.
There may be a performance regression for now on multithreaded write workloads, due to lock contention on the alloc btree. But, that will go away when I implement the new btree key cache code (it'll generalize the current deferred btree updates code, used for inode updates).
If anyone does notice a performance regression, post about it so I'll know to prioritize that.
And, this is the last checkbox - besides catching up on the bug reports - on my list before pushing bcachefs upstream. So, expect to see movement on that front over the coming months. |
getting there
_________________
| Quote: | | Removed by Chiitoo |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3736
Location: Rasi, Finland
|
| Posted: Mon Apr 22, 2019 4:18 pm Post subject: |
|
|
Is bcachefs as painless as btrfs with multi-device storage pools?
I mean with btrfs I can simply add drives to the storage pool, and btrfs spreads the data so that I lose storage space as little as possible while also maintaining redundancy.
_________________
..: Zucca :..
| My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
depontius
Advocate
Joined: 05 May 2004
Posts: 3522
|
| Posted: Tue Apr 23, 2019 12:33 pm Post subject: |
|
|
| Zucca wrote: | | Is bcachefs as painless as btrfs with multi-device storage pools? |
Another way of putting it, does bcachefs engage in layer-breaking like btrfs. The way one would normally do the multi-device stuff is with lvm2 at a layer below the filesystem. With btrfs they break that layering model, taking advantage of that break. So you're really asking if bcachefs sits on lvm2, without layer-breaking, or if it does layer-breaking as btrfs does. I'm not meaning to condemn with the term "layer-breaking", just describe.
_________________
.sigs waste space and bandwidth |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20488
|
| Posted: Tue Apr 23, 2019 10:20 pm Post subject: |
|
|
| depontius wrote: | | Zucca wrote: | | Is bcachefs as painless as btrfs with multi-device storage pools? |
Another way of putting it, does bcachefs engage in layer-breaking like btrfs. The way one would normally do the multi-device stuff is with lvm2 at a layer below the filesystem. With btrfs they break that layering model, taking advantage of that break. So you're really asking if bcachefs sits on lvm2, without layer-breaking, or if it does layer-breaking as btrfs does. I'm not meaning to condemn with the term "layer-breaking", just describe. |
I think what he is asking is how well bcachefs handles growing or shrinking a storage pool. The issue of layer-breaking isn't really relevant unless someone has an arbitrary requirement to use LVM.
That said, the website mentions that it handles multiple devices, but does not explain how it will handle modifying storage pools. I'm pretty sure it mentioned handling drive replacement. So it would seem that like btrfs and ZFS, bcacehfs does not rely on LVM. A different question might be whether or not bcachefs also supports "legacy solutions*" such as LVM. (* After having used ZFS, I can't think of a more accurate way to think of LVM... I'm not aware of any efforts to improve it's capabilities to close the gap with btrfs/ZFS/bcachefs?).
It is important to keep in mind that this is not ready for use. From what I've read, it seems like the intent is for bcachefs to be able to grow and shrink a storage pool. I may have read that in one of the emails about it, or I may be just hopeful. But I seem to recall thinking that it was going to be "better" than btrfs and also avoid ZFS' storage pool modification shortcomings.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Naib
Watchman
Joined: 21 May 2004
Posts: 6067
Location: Removed by Neddy
|
| Posted: Tue Apr 23, 2019 11:22 pm Post subject: |
|
|
totally, this is not production ready but for all intent and purposes it appears stable. It appears to be built on an extremely sound and stable foundation
I'm tempted to build my /var/tmp drive with this to see how it fairs... that does mean I need to fix the ebuild I knocked together for the bcachefs-tools ( https://bugs.gentoo.org/657120 )
as to adding/removing drives, its one of its aims, just don't know the state of it. The goals of BcacheFS seem sound and the development of it seems well thought out. When you look at the so called "next gen FS" for linux, this appears to be ticking alot of boxes after btrfs fizzled out and the relicensing of zfs is nonexistent
_________________
| Quote: | | Removed by Chiitoo |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3736
Location: Rasi, Finland
|
| Posted: Wed Apr 24, 2019 2:49 pm Post subject: |
|
|
| Naib wrote: | | When you look at the so called "next gen FS" for linux, this appears to be ticking alot of boxes after btrfs fizzled out and the relicensing of zfs is nonexistent |
XFS+lvm2 also ticks many boxes too. lvm2 can be very flexible and can even do raid (via underlying mdraid). But as to how simple it is to use... I have no experience.
_________________
..: Zucca :..
| My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20488
|
| Posted: Wed Apr 24, 2019 4:00 pm Post subject: |
|
|
| Naib wrote: | | When you look at the so called "next gen FS" for linux, this appears to be ticking alot of boxes after btrfs fizzled out and the relicensing of zfs is nonexistent |
I didn't expect much from the teaser about the ZFS license change, but I wonder how stable the ZFS ecosystem would be. The versioning scheme for features sounds like a good idea for development flexibility, but also seems like it could lead to some problems. If not for having to maintain patches, I'd probably use it.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3736
Location: Rasi, Finland
|
|
| Back to top |
|
|
Naib
Watchman
Joined: 21 May 2004
Posts: 6067
Location: Removed by Neddy
|
| Posted: Sun Jun 16, 2019 10:24 am Post subject: |
|
|
yup I saw that. Linus was highly critical of what was provided and also the state it was provided. it is constructive criticism and now its a case of tidying up the code for acceptance. this will take some time BUT its happening
_________________
| Quote: | | Removed by Chiitoo |
|
|
| Back to top |
|
|
aidanjt
Veteran
Joined: 20 Feb 2005
Posts: 1118
Location: Rep. of Ireland
|
| Posted: Sun Jun 16, 2019 11:20 am Post subject: |
|
|
| Zucca wrote: | | XFS+lvm2 also ticks many boxes too. lvm2 can be very flexible and can even do raid (via underlying mdraid). But as to how simple it is to use... I have no experience. |
I wouldn't call it simple to use, but it does make managing logical volumes easier than not having it. It has caching support as well, now.
_________________
| juniper wrote: | | you experience political reality dilation when travelling at american political speeds. it's in einstein's formulas. it's not their fault. |
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3736
Location: Rasi, Finland
|
|
| Back to top |
|
|
|
Gentoo Chat
