[RFC] New metastructure proposal

[Nattfodd]

For those who don't read -dev, -core nor planet gentoo, here's your last chance not to miss my prose (NB: if possible, answer on -dev).



Hi everyone,

as everyone probably noticed, there is a current atmosphere of sinking ship,
with quite a lot of people leaving and many agreeing that gentoo is no fun
working on anymore. Before it's too late, I'd like to propose a big reformation
that would help solve some of the issues we are currently having and,
hopefully, bring back some of the fun we all had developing and using this
distribution.


The basic idea is to make gentoo a lot more meta than it already is. Something
that is said quite often is that gentoo lacks a direction. Some people want it
to be a business-oriented distribution, some want it to be bleeding-edge, some
a multimedia, development platform, you name it. Obviously, arbitrarily
choosing one of those directions would mean losing a lot of developers, and
this is something we can't afford to do. The solution, of course, is to go
meta: provide a set of tools that allow people to build the distribution of
their dreams.
This is something that we are already trying to do, but my opinion is that we
are not going far enough. For one thing, we target users as the one who should
build and customize their distribution, while it would be pretty interesting
to also target ourselves as the ones who should be doing this "instantiation"
work. Stage 4's were going in this direction, but they were too isolated and, as
far as I know, they are dead now.



The idea is pretty simple: modularization. There is a core part, with a couple
hundred packages that are absolutely necessary to a system. Then we have a
hierarchy of overlays with additional ebuilds for people's need. Top-level
projects could look like: desktop, dev, business, embedded, misc. Then we
would have subprojects, e.g. multimedia, DE, games for desktop, multimedia
being itself subdivided in audio and video, and so on. This would get us a
tree of arbitrary depth, with development happening mostly in leaves. The
hierarchy would mostly serve as a classification tool, and projects would not
necessarily share resources, including devs, with their subprojects, neither
should they have decision power over them.

This structure has many advantages, one of those being that since projects are
autonomous, they handle their own recruitment, which helps the dev/user
distinction fade away. It concentrates development in small areas where people
will know each other well and be able to interact efficiently. By
decentralizing, we remove the need for a big authority and give everyone the
freedom they want. The current devrel authority is reduced to only the core
project, though people could still ask its wisdom whenever conflicts pop up.
Basically, the only job involving red tape and a central authority is deciding
who gets the nice "gentoo official project" stamp, and the resources infra can
then provide. Of course, QA would be the main, if not only, criterion in this
choice.

By having everything as modular as possible, we also allow an easy fork of a
single project, for whatever reason. So if enough people think that mozilla is
being badly maintained by the current project and that people in it don't want
or can't apply their fixes, they can easily provide their own overlay with
better ebuilds. And if their version is indeed better, over time it will get
the official status and have superseeded smoothly the first project. Think of
paludis and pkgcore vs portage.



So far, I've come up with two main disadvantages to this reformation. The first
is that dependencies between different projects could become a problem if not
handled carefully. For one thing, they would require a change to ebuild
syntax, along the lines of DEPEND="dev.perl.libs:libfoo", and of course
support from package managers. Pulling single ebuilds when required instead of
a full overlay would be a nice thing to have as well. Another idea to simplify
this would be to have a central DB with every known ebuild in it (including
non-official, bad QA ones) and the names of repositories/projects providing
it. This would give enough information to package managers for them to make
intelligent choices about how they should behave.

The other big problem is that a migration to this structure would require a
lot of effort from everyone. The good news is that we could use the
opportunity to migrate to a nicer scm (and given what gentoo would look like,
a distributed one like mercurial or git would be a natural choice) and also
that we would get a good idea of what is maintained and what isn't. Leaving
out stuff that no-one wants would then be very easy. Also, I believe that by
having a clear goal, everyone should get a huge motivation boost and I believe
that things could go quite smoothly, and even quickly.



Of course, many details have been left out that should be discussed and solved
before any decision is taken. Among those is the status of arch teams, which
is a bit unclear. An idea would be to have the main three or four most-used
arches (x86, amd64, ppc would be my guess) in KEYWORDS, as usual, along with a
list of repositories that given person is allowed to keyword in, keeping arch
teams organized as they currently are. Other arches would be projects of their
own, with some kind of meta-overlay that specifies which ebuild from which
overlay has been tested, etc. Or we could make no distinction between popular
and less popular arches, I don't really have any opinion on the matter. Also,
what to do about stuff that isn't specific to any project, even though it
wouldn't happen so often? For instance, deciding whether we should participate
in SoC, or this kind of things. We could use a council as currently, or ask
for representatives of all official projects to punctually decide, or organize a
general vote, or maybe even something else.



To end this proposal, let me say that without a doubt, there are many holes
and hidden problems in it. I don't claim it's perfect, nor that it will
magically solve our problems. But I think it is a better structure than the
one we currently have, and that switching would reduce, perhaps even stop, the
dev bleeding, bring back freedom as well as fun, and help ease the tensions.

Please criticize this with everything constructive you can think of.


Thanks,
/Alexandre
_________________
Don't believe the red stuff, I'm retired.

[theMerge]

Well... I like the idea of keeping the "metaness" of gentoo as the focus. I started using Gentoo about 5 years ago and at the time the "meta" nature of Gentoo seemed to be it's main selling point. Sure the speedup due to optimized builds was mentioned, and the customizable nature of the distro was highlighted, but the main thrust was that Gentoo was what you wanted it to be with fairly few restrictions.

As a user not a dev I don't know if the structure you propose would or would not work, but I do like the idea of keeping the "meta" nature of Gentoo at the center of the distro's focus.
_________________
[Insert favorite Ghandi quote here]

[Drysh]

metaness ++

About making Gentoo less centralized: Great iff we have a strong central team to set the standards. If we fail to have that, we will have a much worse situation: fights of team A versus team B. When it is a few developers, it is possible to find others; but if we have a whole team leaving Gentoo, we are in trouble.

But with a central team to set the standards, and solve problems between two teams, then it is perfect. For this central team, more than great coders, we will need people who can deal with great coders. They will need to be good coders, but more important, they will need to be the kind of person that is able to remain calm when everyone is insulting you, your code and your ideas.

[bssteph]

[bssteph]

Follow up. I read the replies on gentoo-dev and I felt I should echo something genone said.

[theMerge]

I also agree that a central repo is a good idea.

However, something Gentoo has communicated better in the past is a sense of identity. The distro was (and is) about choice. While that general direction has been maintained in it's development decisions, the message is less clear to me than it was. I think that the message needs to be presented in an updated fashion.

I submit to all of you that a bit of marketing would not hurt Gentoo one bit. I've heard multiple times that a new site is going to be implemented, and I hope it comes soon. Just a cosmetic change such as that can create a bit of momentum. Mozilla's site used to be clunky, dull, and UGLY until a few years ago. Now it really looks good, and I think that has helped maintain the sustained growth of it's products.

Gentoo does need some restructuring, some more civility, and some new ideas. However, I think all those things will go unnoticed if Gentoo's message is not presented in a pleasing way. So here's my idea for a plan of action.

1. The developers need to put aside differences, remember this "meta message", and then set a workable development plan into motion.
2. The users need to be reminded that Gentoo is a distro about empowering their choice so they can get what they want as an alternative to the normal constraints release cycle distros impose.
3. A team of creative Gentoo artists and web designers need to work with some backend guys to create a top quality web presence.
4. The Gentoo community needs to continue to be the central glue that holds it all together.
_________________
[Insert favorite Ghandi quote here]

[tboloo]

[theMerge]

I'd just like to stop here and point out what INCREDIBLE nerds we are. Feels good don't it.
_________________
[Insert favorite Ghandi quote here]

[Hypnos]

++ strong central leadership and standards

It could cause a serious problem if, for example, the kernel team wants to be conservative to serve their own agenda, and the compiler teams wants to be bleeding edge. Gentoo should provide a way to manage this chaos and let the users set the policy for their own machine -- which was the original vision, no?

(BTW, I'm one of those GNUstep people )
_________________
Personal overlay | Simple backup scheme

[erm67]

[Naib]

DAmb!!! I came up with an idea like that but didn't bother posting

you could extend it by inheriting ebuilds. Quite alot of ebuilds are exactly the same as the ebuild for the version before it (or minor changes).
_________________

[aidanjt]

[bssteph]

[theMerge]