Unofficial Request for Comments: Gentoo Filesystem Hierarchy

marcion

New Elements:

/portage

/portage/distfiles - automatically downloaded files
/portage/fetch - manually downloaded files
/portage/tmp - portage working directory
/portage/tree - main portage tree

Deprecated Elements:

/usr/portage
/var/tmp/portage

Reasoning:

I was looking at Mac OS X and I noticed that instead of slavishly following the 15 year old 'Filesystem Hierarchy Standard' based on a 30 year old Unix, they have tweaked the layout to represent a 21st Century networked operating system.

I was also thinking about Gentoo, and I think we should be willing to consider having a filesystem layout that represents how Gentoo works rather than how System V once worked.

Here I am not actually arguing against the Filesystem Hierarchy Standard, only the Gentoo-specific extensions to it. The Gentoo default directories have been dumped onto the structure without enough due care and attention.

/usr is supposed to be for more static data such as program files. Therefore you can put the /usr directory on a partition or dedicated disk, safely away from frequently written parts of the disk such as /tmp or the swap partition.

So there is a contradiction within Gentoo's default filesystem as the portage tree, which changes quite a bit everytime that I sync, really wants to be away from my compiled programs.

Next problem is /var/tmp/portage. Ideally, at least on a production system, /tmp and /var/tmp should be isolated and restricted, such as using dedicated partitions with the noexec bit set. In normal use, it is very rare to envision a situation where you need 6GBs of space in /var/tmp. However, in Gentoo to some programs to compile you need a massive amount of space for /var/tmp (for the most extreme example, think OpenOffice).

In my opinion, the simpliest solution is to keep to the main portage specific directory within a new top-level directory called /portage

This simplification would allow partitioning to be more sensible, backups to be more focused and have a positive side effect of being easier for new users.

What do you think?

kopp · Posted: Tue Dec 05, 2006 1:48 pm Post subject:

I think this thread could be moved to the Userreps Forum where it would fit well.

runningwithscissors · Posted: Tue Dec 05, 2006 2:21 pm Post subject:

This has been debated to death. The filesystem hierarchy is the way it is for very good reasons.

I do however, think that the portage tree ought to be moved to /var/portage and distfiles to /var/portage/distfiles.
The scripts belong in /usr/lib. The tree kind of feels awkward in there.

Of course, I couldn't care less if it remains in /usr, I just think /var is a more logical choice.

mark_alec · Posted: Tue Dec 05, 2006 8:50 pm Post subject:

Moved from Gentoo Chat to Gentoo User Representatives.
_________________
www.gentoo.org.au || #gentoo-au

antarus

timeBandit

marcion · Posted: Tue Dec 05, 2006 11:01 pm Post subject:

Insanity5902 · Posted: Tue Dec 05, 2006 11:41 pm Post subject:

I agree, the portage directories need to be re-aligned, most of this is already in place to happen with the variables PORTDIR, DISTDIR, PORTAGE_TMPDIR.

What is not covered in this is the stuff in /var/lib/portage (should it stay here), and what about the different symlinks. They don't use the variables (the one, and only one, I know about is make.profile ... which is kind of important.

I don't see how upgrading would be hard at all, a simple script could copy the information over, and have it happen with the next stable version of portage (2.2 or 2.1.3, etc). Like it has been pointed out before, backwards compatibility is not something gentoo has concerned it self with too much before, when it needs to make a change, it is well thought out, documented, tested and then executed.
_________________
Join the adopt an unanswered post initiative today

1clue · Advocate Joined: 05 Feb 2006 Posts: 2569

Even so, you could just draw a line in the sand. Any existing system has its portage directories in the current spots, and any new installation gets the new places.

This is not hard to implement given the existence of the portage-specific variables already mentioned. The next version of portage sets the variables explicitly to the old locations. Following that, the upgrades only set it if it has not already been set on this system. At that point, the directories on new systems go to the newer places, the old systems are not altered.

marcion · Posted: Wed Dec 06, 2006 12:41 am Post subject:

richfish · Posted: Wed Dec 06, 2006 2:02 am Post subject:

I actually couldn't care less whether the portage tree lives at /portage, /usr/portage, /var/portage, or /home/portage (my personal favorite, since portage is user after all!). As long as it can be changed to suite a particular users' tastes, I don't think it matters much. But specifically regarding your proposed layout:

Missing:
- location for binary packages.
- location for layman overlays.

I Like:
- that distfiles have moved out of the tree hierarchy. Much better for "grep -r" usage!!

Dislikes:
- Don't see a point for having separate dirs for "automatically downloaded" vs "manually downloaded". Regardless of how they got on the system, the content is the same.
- I predict lots of users would fail to create a separate filesystem for /portage/tmp, and would end up filling up their root filesystems during a openoffice build+merge. Better to leave this in /var/tmp IMO. Most people who don't make one big filesystem on their box know they should keep /var separate, and I'd much rather see them run out of space on /var than /.

Also for my case, I prefer having the distfiles and binary packages in the same heirarchy, so I can easily make a single filesystem for both. So rather than .../packages and .../distfiles, I'd like .../pkgs and .../pkgs/src/, with binary packages in pkgs/, and "distfiles" in pkgs/src/.

Of course, thanks to the power of symlinks, I already have something similar on my system :-)

Conan · Guru Joined: 02 Nov 2004 Posts: 360

I guess what I don't understand is why you feel it is necessary for the defaults to change knowing that is going to cause a gigantic headache for supporting portage. All of the locations are already configurable, if you take offense to directories being where they are, then move them. Asking for the default to be changed would result in the irc support being overloaded for a few days, the forums being overloaded forever, and a lot of explaining why that doesn't really need to happen.

With reguard to the java/php upgrades, the comparisson isn't even close. Java/php are used by only a small amount of users, and the support for that stretched months. Portage is (obviously) used by all users and the support for that could stretch for a long time.

Hell, we still have people come into irc that havn't switched over to cascading profiles, and thats at least a few years old.

Insanity5902 · Posted: Wed Dec 06, 2006 2:50 am Post subject:

Another idea for the portage dir, instead of putting the tree -in ./tree, do something like ./tree-base, and then for you local portdirs (you could have many, I know I have at least 2 - 3 right now), is to do a ./tree-local. This way a simple ./tree-* will give you then entire portage, instead of putting them in to completely different locations.

Putting the tree in /home is also a perfectly good idea, since Portage is a user.

Another thing about the source downloads, is that what /usr/src has been for? Technically there is a good reason to put several parts of portage in several different places, all of which have pretty good reasons.

@Conan, it isn't just taking offense to the location of the different portage dirs, it is creating a default schema that not only provides better organization, but that allows for easier administration of a machine or several machines. I don't want to have to remember to setup each server to allow for correct partitioning (setting noexec, read-only, etc) of my disk. Your are right though, for a home user there is no reason as to why this matters, as most follow the gentoo guide of 3 MAYBE 4 partition of /boot swap / AND MABYE /home, so they wouldn't be effected, but those of us that run a tighter ship on our desktops, or run a server to minimize any damage done by a rogue program/virus/intruder or just a dumb user, will want to find a way that makes more sense and makes life easier to administer.

Even if the answer is just to create a simple HOW-TO in the docs, and create an ebuild or script that takes care of everything would be easier than having to mainly re-setup portage to take of all of this.

While yes the initial and probably well after the initial change will create a flood of post and help request, the good thing is everyone who browsing all the sections will easily spot the answer and inform the user. And if this is the only reason for not doing it, then I really think it should be done, because this is a none issue to me (and yes, I do help a lot of the forums) and it brings plenty of benefits.
_________________
Join the adopt an unanswered post initiative today

Genone

marcion · Posted: Wed Dec 06, 2006 10:35 am Post subject:

antarus

i92guboj · Posted: Wed Dec 06, 2006 5:48 pm Post subject:

I personally would like to see something like this by default, though, as someone else already declared: I couldn't care less what the defaults are as long as I can change them.

I would say "yes" to /var/portage (or even /home/portage, though I would continue using /var/portage, for a number of reasons that I will resume later). I would definitely say "no" to ${PORTDIR}/tmp, there is already a temporal location for ALL the temporal stuff, and there is NO REASON WHY a) portage should have its own, and b) [...and there is NO REASON WHY] portage needs to be always writable.

My reasons to vote for a /var/portage and not an /usr/portage are those mainly:

1.- The already mentioned FHS, I still can't understand what is the hard thing to understand in this:

marcion · Posted: Wed Dec 06, 2006 7:12 pm Post subject:

bmichaelsen · Posted: Thu Dec 07, 2006 6:21 pm Post subject:

another /me-too voting for the portage tree in /var from an long time gentoo user ...

zerojay · Veteran Joined: 09 Aug 2003 Posts: 1033

Yes, /var please.

antarus · Posted: Sun Dec 10, 2006 9:54 pm Post subject:

bmichaelsen · Posted: Mon Dec 11, 2006 10:31 am Post subject:

steveL · Posted: Mon Dec 11, 2006 12:59 pm Post subject:

Lloeki · Guru Joined: 14 Jun 2006 Posts: 437 Location: France

I second /var/somewhere as a portage tree location.

madchaz · Posted: Mon Dec 11, 2006 4:30 pm Post subject:

Personally, if we want to go thought a significant upgrade, moving the tree wouldn't really be the way to go I'd like to see.

Right now, using a FS tree is becoming more and more of an issue. It creates a structure with over 100 000 ebuilds in it and makes for a lot of load when it comes to updating.

What I'd like to see is a move away from the FS tree entirely and on to something saner, like a database. This could make searching a LOT faster and reduce the load for updating portage.

When you think about it, there is little need to even have the actual ebuild on the machine, unless it gets installed. It would be easy to only keep the metadata and dependencies in a database and when someone tries to build, portage gets the proper ebuild.

In my view, this would weild a lot of benifits for everyone. The users would need less time to sync and use less bandwidth to do so. Infra would have to deal with less load and it would make searching much faster.

Edit: It would also be easy to keep the list of installed packages in the database. This would also make uninstalling a lot easier as it would be easy to cross reference dependencies to only remove those that are truly un-needed.
_________________
Someone asked me once if I suffered from mental illness. I told him I enjoyed every second of it.
www.madchaz.com A small candle of a website. As my lab specs on it.