Gentoo Forums :: View topic - Why is the latest kernel supported by ZFS removed shortly...

Why is the latest kernel supported by ZFS removed shortly...

View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3, 4 Next

Gentoo Forums Forum Index

Kernel & Hardware

View previous topic :: View next topic

Author

Message

Yamakuzure
Advocate

Joined: 21 Jun 2006
Posts: 2293
Location: Adendorf, Germany

Posted: Mon Aug 12, 2024 11:53 am Post subject: Why is the latest kernel supported by ZFS removed shortly...

...after zfs-kmod gets updated?

This now happened (at least) twice!

zfs-kmod update released, now supporting kernel 6.8.*
A few days later Gentoo removes gentoo-sources-6.8.* entirely
zfs-kmod update released, now supporting kernel 6.9.*
A few days later Gentoo removes gentoo-sources-6.9.* entirely

I was able to run on 6.8.12 by simply not letting portage remove it. But this time I did not pay enough attention and

Code:

emerge --depclean

removed gentoo-sources-6.9.12, meaning I have to revert to an ancient 6.6 kernel with its own bunch of problems now, or I can not build vmware modules. Both 6.8.12 and 6.9.12 worked just fine.

Is there a reason for this scheme? Are there maintainers hating ZFS users or something like that? This scheme produces massive problems and huge annoyances, just to let you know.
_________________
Edited 220,176 times by Yamakuzure

logrusx
Advocate

Joined: 22 Feb 2018
Posts: 2373

Posted: Mon Aug 12, 2024 12:41 pm Post subject:

Yamakuzure wrote:

...after zfs-kmod gets updated?

This now happened (at least) twice!

zfs-kmod update released, now supporting kernel 6.8.*
A few days later Gentoo removes gentoo-sources-6.8.* entirely
zfs-kmod update released, now supporting kernel 6.9.*
A few days later Gentoo removes gentoo-sources-6.9.* entirely

Neither of those is stable. They come and go and 6.9 is EOL already.

Yamakuzure wrote:

ancient 6.6 kernel

Ancient?! The current stable long term support kernel is 6.6. The next stable is 6.10 which should be already in tree and I guess we'll see stabilized very soon.

Best Regards,
Georgi

Koyan
n00b
n00b

Joined: 07 Nov 2014
Posts: 32

Posted: Mon Aug 12, 2024 1:03 pm Post subject:

logrusx wrote:

Neither of those is stable. They come and go and 6.9 is EOL already.

6.9.12 may be EOL, but on kernel.org it is also still available marked as stable. For a ZFS user it is indeed very annoying when it's removed prematurely from the portage tree.

logrusx
Advocate

Joined: 22 Feb 2018
Posts: 2373

Posted: Mon Aug 12, 2024 1:31 pm Post subject:

Koyan wrote:

logrusx wrote:

Neither of those is stable. They come and go and 6.9 is EOL already.

6.9.12 may be EOL, but on kernel.org it is also still available marked as stable. For a ZFS user it is indeed very annoying when it's removed prematurely from the portage tree.

Those will never be stable on Gentoo and as unstable will just come and go. If you don't like it, you can step up and become a package maintainer for those.

Or you can browse gitweb/GitHub for an old ebuild to put in your local ebuild repository. As I said, 6.10 is already available. It's also your choice to use such an FS, so you bear the burden of such a decision, not the distribution.

Best Regards,
Georgi

Last edited by logrusx on Mon Aug 12, 2024 1:34 pm; edited 1 time in total

Hu
Administrator

Joined: 06 Mar 2007
Posts: 22585

Posted: Mon Aug 12, 2024 1:34 pm Post subject:

That sounds to me like the ZFS maintainers need to either get their code merged to mainline so it stops breaking, or get faster at issuing updates. Running a kernel with multiple sources of out-of-tree modules has always risked these types of problems, and the core kernel developers have made no secret of this risk.

Yamakuzure
Advocate

Joined: 21 Jun 2006
Posts: 2293
Location: Adendorf, Germany

Posted: Mon Aug 12, 2024 3:41 pm Post subject:

Koyan wrote:

logrusx wrote:

Neither of those is stable. They come and go and 6.9 is EOL already.

6.9.12 may be EOL, but on kernel.org it is also still available marked as stable. For a ZFS user it is indeed very annoying when it's removed prematurely from the portage tree.

Exactly.

It would would be nice to have a bit more grace time in the portage tree.

Hu wrote:

That sounds to me like the ZFS maintainers need to either get their code merged to mainline so it stops breaking, or get faster at issuing updates.

Running a kernel with multiple sources of out-of-tree modules has always risked these types of problems, and the core kernel developers have made no secret of this risk.

Yes, absolutely, faster updates would be great!
The adaption to newer kernels has always been ... erm... careful.
_________________
Edited 220,176 times by Yamakuzure

logrusx
Advocate

Joined: 22 Feb 2018
Posts: 2373

Posted: Mon Aug 12, 2024 3:49 pm Post subject:

Yamakuzure wrote:

It would would be nice to have a bit more grace time in the portage tree.

They were in tree for quite a while. Since the beginning of their lines. They just got cleaned up altogether.

Once again, those were not and never will be stable on Gentoo. If they were, they would have stayed. Also they don't get bug fixes, nothing gets backported and so on. You don't want to use those kernels. The fact that you're using ZFS does not change that. The decision to use ZFS is yours and you bare the responsibility.

Once again, if you need it so badly, go browse the repo and get one of those into your local overlay. I've linked the wiki page of how to do that above.

Best Regards,
Georgi

Koyan
n00b
n00b

Joined: 07 Nov 2014
Posts: 32

Posted: Mon Aug 12, 2024 6:00 pm Post subject:

logrusx wrote:

Those will never be stable on Gentoo and as unstable will just come and go. If you don't like it, you can step up and become a package maintainer for those.

I said marked as stable on kernel.org, not stable on Gentoo. And since I'm running several ~arch systems, I know to keep ebuilds like that in my local overlay. It just seems to me that it would not be a huge burden to keep all the current (mainline, stable, longterm) kernels from kernel.org in the portage tree, even if they're not marked stable on Gentoo.

Yamakuzure wrote:

Hu wrote:

That sounds to me like the ZFS maintainers need to either get their code merged to mainline so it stops breaking, or get faster at issuing updates.

Running a kernel with multiple sources of out-of-tree modules has always risked these types of problems, and the core kernel developers have made no secret of this risk.

Yes, absolutely, faster updates would be great!
The adaption to newer kernels has always been ... erm... careful.

Yes, I have to say I was disappointed to see that OpenZFS 2.2.5 didn't support 6.10 even though 6.10.3 was already out when it was released...

pingtoo
Veteran
Veteran

Joined: 10 Sep 2021
Posts: 1217
Location: Richmond Hill, Canada

Posted: Mon Aug 12, 2024 6:24 pm Post subject:

Koyan wrote:

...
It just seems to me that it would not be a huge burden to keep all the current (mainline, stable, longterm) kernels from kernel.org in the portage tree, even if they're not marked stable on Gentoo.

I agree the concept of have few extra ebuilds in Gentoo repo tree is not a big burden.

And since you know (or have already done) keep those kernel ebuild in your local repo,

Now the question,

Do you willing to share (not necessary publish to internet) those ebuild and accept request from other for update? :lol:

This is why I think using Gentoo as Linux distro is wrong approch. If you using Gentoo as tool set that help you design your own distro than you control the progress of your environment.

Zucca
Moderator

Joined: 14 Jun 2007
Posts: 3679
Location: Rasi, Finland

Posted: Mon Aug 12, 2024 7:23 pm Post subject:

I've been toying with the idea of creating snapshots of some ebuild directories on ::gentoo into a local overlay. That could be executed from presync.d to make it automatic.
But then there are dependencies and eclasses... Safest bet would be to snapshot the whole repo.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--

Quote:

I am NaN! I am a man!

stefantalpalaru
n00b
n00b

Joined: 11 Jan 2009
Posts: 64
Location: Italy

Posted: Tue Aug 13, 2024 2:39 am Post subject:

Look into "zen-sources" from the "kzd" overlay: https://gitlab.com/kzdixon/kzd-ebuilds/-/tree/master/sys-kernel/zen-sources?ref_type=heads

logrusx
Advocate

Joined: 22 Feb 2018
Posts: 2373

Posted: Tue Aug 13, 2024 5:20 am Post subject:

Hu wrote:

That sounds to me like the ZFS maintainers need to either get their code merged to mainline.

This poorly documented thing is not going into mainline anytime soon. I personally wouldn't have chosen it for that same reason never in a million years.

Best Regards,
Georgi

Zucca
Moderator

Joined: 14 Jun 2007
Posts: 3679
Location: Rasi, Finland

Posted: Tue Aug 13, 2024 5:55 am Post subject:

Wasn't it a licensing conflict?
Although I'm not so sure about the current state of things nowdays, since there is zfs and all the others... open-zfs? zfs-on-linux (ZoL)?
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--

Quote:

I am NaN! I am a man!

lars_the_bear
Guru
Guru

Joined: 05 Jun 2024
Posts: 512

Posted: Tue Aug 13, 2024 7:14 am Post subject:

logrusx wrote:

This poorly documented thing is not going into mainline anytime soon.

There's a ton of documentation for ZFS -- the problem is that we can't see it. I imagine it's on tapes in an Oracle vault somewhere.

Twenty years ago, ZFS was a colossal engineering effort. It was better than anything else that was currently available, to the extent that a laser cutter is better than a stone axe. I'm not sure the rest of the Linux world has caught up with it, even now.

But Linus Torvalds poisoned the well, because he didn't like the fact that ZFS was mostly maintained by Oracle/Sun, and he didn't like that the ZFS maintainers refused to back the GPL. I remember really bitter rows about that licence issue at the time, like we had with systemd later. ZFS has often been a pain to use in Linux, which is a shame, because it has some great features.

BR, Lars.

logrusx
Advocate

Joined: 22 Feb 2018
Posts: 2373

Posted: Tue Aug 13, 2024 7:49 am Post subject:

Zucca wrote:

Wasn't it a licensing conflict?

I don't know is it's a licensing conflict, but at first glance it's more fundamental than that.

lars_the_bear wrote:

But Linus Torvalds poisoned the well, because he didn't like the fact that ZFS was mostly maintained by Oracle/Sun, and he didn't like that the ZFS maintainers refused to back the GPL.

It never ceases to amaze me how lightly some people handle matter they don't understand. Do you have any idea what licensing complications will arise from accepting something into the Linux kernel that's not GPL compatible?

It's not Linus who "poisoned the well" (completely inappropriately used term as well), but Oracle who doesn't want to license that piece of crap under GPL.

Best Regards,
Georgi

mrbassie
l33t
l33t

Joined: 31 May 2013
Posts: 821
Location: over here

Posted: Tue Aug 13, 2024 8:06 am Post subject:

Zucca wrote:

Wasn't it a licensing conflict?
Although I'm not so sure about the current state of things nowdays, since there is zfs and all the others... open-zfs? zfs-on-linux (ZoL)?

iiuc openzfs was originally a repo where illumos, freebsd and linux could submit their changes/improvements/innovations from which each implementation could cherrypick. It didn't gain traction and most of the new kewl stuff was happening on linux. Native encryption seemed to be the game changer after which freebsd decided to report zfs from the linux branch since porting the encryption changes required porting loads of earlier changes and it was apparently quite messy.
Along with this, delphix had already decided to rebase their product on linux as they had difficulty sourcing staff with illumos experience. There's a few of the original developers from sun at delphix so they carry disproportionate weight.
openzfs on freebsd is now hosted in the zol repo which is now renamed openzfs (hence the change from 0.whatever releases to 2.0).

Oracle zfs I believe was refactored to no longer be a general purpose file system, it is iiuc now tailored to an appliance which is part of the underpinnings of their 'cloud'.
_________________
I spent a christmas in Vienna twenty something years ago. It was a beautiful city. Everyone was so friendly.

lars_the_bear
Guru
Guru

Joined: 05 Jun 2024
Posts: 512

Posted: Tue Aug 13, 2024 8:18 am Post subject:

logrusx wrote:

It's not Linus who "poisoned the well" (completely inappropriately used term as well), but Oracle who doesn't want to license that piece of crap under GPL.

Yup. Exactly the same argument that was hurled around in 2004. It's good to see that nothing ever changes

Well, one thing has changed: these days the arguments are just online. In 2004 people were literally hurling things at each other across a desk. I've rarely seen anything so unedifying in the software industry.

If ZFS had been a 'piece of crap', there would never have been a problem. Nobody wants a piece of crap. Linux did want ZFS in 2004 -- they just didn't want to budge on their licensing conditions. Perfectly understandable, but pretty unsatisfactory result.

BR, Lars.

PS. Despite what you and Mr Torvalds say, this has nothing to do with Oracle. Oracle just inherited a situation that was already problematic, long before Oracle bought Sun. I don't think Oracle gave a dam about ZFS, or anything that came from Solaris.

logrusx
Advocate

Joined: 22 Feb 2018
Posts: 2373

Posted: Tue Aug 13, 2024 9:39 am Post subject:

lars_the_bear wrote:

logrusx wrote:

It's not Linus who "poisoned the well" (completely inappropriately used term as well), but Oracle who doesn't want to license that piece of crap under GPL.

Yup. Exactly the same argument that was hurled around in 2004. It's good to see that nothing ever changes ;)

Thank you for pointing that out. It's good to see ignorance about legal matter doesn't change it.

I don't know what happened in 2004 I only worked that out from what you said. I just pointed out the flows in your reasoning (or lack there of).

Best Regards,
Georgi

p.s. it's not me and Linus. It's about legal matter. Please stop that ad hominem.

p.s.2 because I usually check my claims:

CDDL Wikipedia page wrote:

The Free Software Foundation (FSF) considers it a free software license, but one which is incompatible with the GNU General Public License (GPL).[1]

lars_the_bear
Guru
Guru

Joined: 05 Jun 2024
Posts: 512

Posted: Tue Aug 13, 2024 11:05 am Post subject:

logrusx wrote:

p.s. it's not me and Linus. It's about legal matter. Please stop that ad hominem.

Blaming "Oracle" for this situation is not a legal matter at all. There is a legal complication here, but that arose long before Oracle was involved in any way. Linux Torvalds blames Oracle for this ZFS mess, and I noticed you doing it, too. I can't imagine you're alone in this, but you both (and everybody else) are wrong.

The CDDL, under which ZFS was originally released as open source, was published in 2004, at around the same time as OpenSolaris was released. Linux and BSD folks took the ZFS code, with its CDDL licence, from OpenSolaris between 2005 and 2010. This (IIUC) is what become OpenZFS. Oracle still has a proprietary version of ZFS, which I believe they use in storage appliances.

By the time Oracle bought Sun in 2010, ZFS was already in widespread use outside Solaris. However, it was the Sun management which originally refused to budge on CDDL. I was at the meetings where this was originally discussed. Nobody could agree on the legal compatibility with GPL at that time, and I think that's still the case. Lots of opinions have been written, but the position has never been tested in court (as far as I know) in any jurisdiction. Either side could have compromised. Neither did.

Oracle just inherited this mess in 2010. Oracle paid lip-service to the idea of open source as, frankly, Sun had done before. These corporations could see the way the industry was going, but they didn't understand how they could adapt their business model to it. Sun had embraced the idea in principle -- I worked occasionally with a senior executive whose official job title was (I kid you not) "open source maven". But the middle management never really seemed to get behind it; and, in any case, Sun collapsed before open source became the force it is now.

I don't know whether Oracle could have fixed the CDDL/GPL mess if they had wanted to, but Oracle didn't care about Solaris or anything related to it. Oracle just wanted to get control of Java. The issue did arise again, several times, after Oracle had bought Sun. Oracle could have released their version of ZFS as GPL (maybe), but why on Earth would they have done that? Oracle is a business who main income comes from sales of proprietary software.

So, while there are many things we can legitimately blame Oracle for, I don't think CDDL is one of them. It's not clear to me that Oracle could have fixed the problem if they wanted to, although I've read many angry complaints that they could. Sun could have fixed the problem, but they didn't see it was their job to. Mr Torvalds and his collaborators could have relaxed their insistence on GPL, but they chose not to. Both sides thought they were right, and were prepared to argue their rightness, sometimes almost to the point of violence. What a sad situation.

BR, Lars.

logrusx
Advocate

Joined: 22 Feb 2018
Posts: 2373

Posted: Tue Aug 13, 2024 1:29 pm Post subject:

lars_the_bear wrote:

Mr Torvalds and his collaborators could have relaxed their insistence on GPL, but they chose not to.

You still refuse to acknowledge the matter you don't understand like it doesn't exist.

Why on earth would they do that? Why would they create a legal mess for everyone else just to suit a bunch of enthusiasts who don't even seem to care about issues of legal matter?

Blaming the previous owner for the current one not fixing a problem is ludicrous too. Only the holder of the rights have the right to change the licensing. You cannot bring code licensed under a conflicting license in GPL code. That's legal nonsense.

Now back to the topic, whoever is crazy enough to go out of their way to use a piece of crap that causes problems just by its existence, should be maintaining a local overlay with a bunch of kernels that are not supported neither officially nor unoffcially instead of complaining about package maintainers doing their job. It shouldn't be a problem to download the sources from kernel.org either. I was doing exactly that with a few release candidates and it worked.

And whoever brought that argument up that kernel.org still provides the sources and that's why Gentoo should too, kernel.org provides sources for as early as version 2.0 if I remember correctly, does that mean Gentoo should provide ebuilds for them too?

Best Regards,
Georgi

lars_the_bear
Guru
Guru

Joined: 05 Jun 2024
Posts: 512

Posted: Tue Aug 13, 2024 2:13 pm Post subject:

logrusx wrote:

lars_the_bear wrote:

Mr Torvalds and his collaborators could have relaxed their insistence on GPL, but they chose not to.

You still refuse to acknowledge the matter you don't understand like it doesn't exist.

I was there when these decisions about ZFS were made twenty-some years ago. I was involved in making them. I understand the situation perfectly well.

When ZFS was a new thing in 2001, it was Linux that was run by enthusiasts. Red Hat, SuSE, etc., existed, but they were tiny players in a Unix industry dominated by Sun. It wasn't until 2006 that Red Hat could stand alongside Sun as an equal, and by that time Sun was starting to decline.

I can't comment on OpenZFS because I've never used it. The original Sun ZFS was rock-solid, and every enterprise customer I worked with, that used the Solaris platform (and in 2004 that was everybody) was using it. It was hailed as a major step forward in filesystem design by almost everybody, at that time.

Why should Sun, a dominating player in the IT industry, change its open-source licensing terms to suit a bunch of hobbyists? That's exactly how Linux was seen at the time. Linux had everything to gain by including ZFS which, let me repeat, was the undisputed leader in Unix filesystems at that time. In fact, Sun wanted Linux to have ZFS; but Sun didn't want to change its licence terms to suit what was at that time a small player.

With the current dominance of Linux in the enterprise IT space, it's easy to lose sight of how different things were twenty-odd years ago. These days, if you want to get something in the Linux kernel, you have to play by Linus's rules. Twenty-odd years ago, if you wanted something from Sun, you had to play by Sun's rules. The Linux folks didn't want to play by anybody's rules but their own. Turns out they were right, but I doubt anybody would have predicted that in 2004.

As for the current situation with OpenZFS, I know nothing and care as little. My objection is simply to your blaming Oracle for the legal problems. Oracle did not become involved until it was probably too late to change anything.

BR, Lars.

pietinger
Moderator

Joined: 17 Oct 2006
Posts: 5041
Location: Bavaria

Posted: Tue Aug 13, 2024 2:28 pm Post subject:

logrusx wrote:

[...] Please stop that ad hominem.

...

logrusx wrote:

[...] You still refuse to acknowledge the matter you don't understand like it doesn't exist. [...]

...

It is very difficult to diagnose from a distance whether someone has not understood something or simply has a different opinion.

Moderation is watching this thread
_________________
https://wiki.gentoo.org/wiki/User:Pietinger

logrusx
Advocate

Joined: 22 Feb 2018
Posts: 2373

Posted: Tue Aug 13, 2024 3:23 pm Post subject:

lars_the_bear wrote:

As for the current situation with OpenZFS, I know nothing and care as little. My objection is simply to your blaming Oracle for the legal problems. Oracle did not become involved until it was probably too late to change anything.

Oracle can change everything. Immediately. Only Oracle can do it. That's what I've been saying all the time and explaining why and you seem to always be avoiding those arguments and instead bringing up arguments of your own that are not relevant to the real issue and belittling the ones that really matter. Where you were 20 years ago is totally irrelevant. Whether you were there or not doesn't change anything. There's the legal side which you're trying to totally deny. You can go to as many meetings as you like and you may make as many decisions as you like, that still won't change the licensing issue and will still not make the compromise from the kernel side right.

p.s. if somebody implements ZFS completely from scratch, that might solve the licensing issue. I don't know if there are patents involved however. That might be a bigger problem.

Best Regards,
Georgi

Last edited by logrusx on Tue Aug 13, 2024 3:41 pm; edited 1 time in total

logrusx
Advocate

Joined: 22 Feb 2018
Posts: 2373

Posted: Tue Aug 13, 2024 3:34 pm Post subject:

pietinger wrote:

logrusx wrote:

[...] Please stop that ad hominem.

...

logrusx wrote:

[...] You still refuse to acknowledge the matter you don't understand like it doesn't exist. [...]

...

It is very difficult to diagnose from a distance whether someone has not understood something or simply has a different opinion.

Moderation is watching this thread

I'm not diagnosing anybody. It's not a matter of opinion. If it was, it would be easy. Just persuade somebody to change their opinion but it's not that simple. As I already said it's a legal matter and some of the participants here refuse to accept that fact.

Ad hominem is a type of logical fallacy that may not pertain to a particular individual and it has all kinds of subtle forms. It's funny how people are used to think ad hominem is something directed against the ones calling it out. When a statement brings up somebody and his collaborators or an unnamed group of (anonymous) people as the reason why something is bad, this still makes it ad hominem and thus a logical fallacy that should be disregarded. It still tries to refute a claim based on personal traits rather than the contents of the claim itself.

Best Regards,
Georgi

p.s. will it be possible this part of the thread be split in Gentoo Chat?

pjp
Administrator

Joined: 16 Apr 2002
Posts: 20476

Posted: Tue Aug 13, 2024 4:17 pm Post subject:

lars_the_bear wrote:

Mr Torvalds and his collaborators could have relaxed their insistence on GPL, but they chose not to.

The CDDL was created to be specifically incompatible with the GPL. What would be the justification to relax the Linux license to make that accommodation?

logrusx wrote:

You still refuse to acknowledge the matter you don't understand like it doesn't exist.

The legal matter is the license. The only other issue I'm aware of was the NetApp lawsuit which was settled in 2010. Linus mainly doesn't want to be sued because of the license:
https://www.zdnet.com/article/linus-torvalds-avoid-oracles-zfs-kernel-code-on-linux-until-litigious-larry-signs-off/

logrusx wrote:

Why would they create a legal mess for everyone else just to suit a bunch of enthusiasts

It isn't "just a bunch of enthusiasts." it also seems like a relatively easy solution. If sued, remove it from the kernel. I'd guess the bigger issue is the hassle and / or even the dilution of the kernel's license.

logrusx wrote:

whoever is crazy enough to go out of their way to use a piece of crap

I could be mistaken, but it sounds like you've never used ZFS. You're in good company though, Linus doesn't understand it either.
https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-statements-arent-right-heres-the-straight-dope/

And given that the ars link mentions it, the API issue seems to make this much more of a political issue than any real legal or licensing concern. Otherwise there isn't a credible reason to try and pretend an API is somehow "off limits".

https://en.m.wikipedia.org/wiki/Google_LLC_v._Oracle_America,_Inc.#Decision

the Court ruled that Google's use of the Java APIs was within the bounds of fair use

Using those APIs for ZFS on Linux or even Nvidia seems in line with that legal decision.
_________________
Quis separabit? Quo animo?

Display posts from previous:

	Gentoo Forums Forum Index Kernel & Hardware	All times are GMT Goto page 1, 2, 3, 4 Next
Page 1 of 4

Jump to:

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Copyright 2001-2024 Gentoo Foundation, Inc. Designed by Kyle Manna © 2003; Style derived from original subSilver theme. | Hosting by Gossamer Threads Inc. © | Powered by phpBB 2.0.23-gentoo-p11 © 2001, 2002 phpBB Group
Privacy Policy