Binary host and USEs

[Pipeng]

Hi all, I have a server ,which I want to run a binary host, the idea is instead of build on my laptop just do it all there.

The current binary host project can be found here: https://wiki.gentoo.org/wiki/Binary_package_guide

Which is very complete, but lacks of some details, in particular is not very clear about how USEs works in a binary host....

A binary host will store the binaries with the configured USEs, this causes the next:

If the client request other USEs portage will complains (I asked about this part), so I need go to the server, do the same changes of the client on the server, compile and then install on the client, basically we need to have the same USEs configuration on both of them.

If we add a second client and we perform this, we would need to upgrade the USEs for Client 2, if we also need a new USE in it then if we need update the USEs for the Host, to then upgrade again the USEs on the client 1.... at this point I think is easy to understand the issue here.

So going back, the Host is like a full system, it stores everything that can be installed, so, why we need to keep in the Clinets the track of the USEs? in first place, if we need a new USE, we would need to change the Host, sounds more practical to request the World packages of the Clients to the host with the config the Host have, if we need any USE change the ideal is update the Host and reinstall on the clients looking on changed packages (think equery u foo would show the host uses).

All this can also be applied to any package configuration like accept_keywords.

Maybe I didn't understand well some parts of the binary host, but this issue makes hard to use a binary host, is there any way to simplify it?

Thx!

[Hu]

[Pipeng]

Thx for the option!

Just I still don't see it very practical, VIDEO_CARDS, we set it for a client, but if two clients needs different cards, you still need to enable both cards on the host, and both clients will have the needed packages based on that.

Lets pick as example mesa, two clients with different cards, both clients will get the same mesa binary with both use flags enabled, with all deps it means.

So I don't see the difference.....

But I agree is good the client be able to check if their needs match with the server, but with a binhost and several clients, the logic used for this will be different, like "to this packages have at least this uses", "at least this version for this packages".

Seems more complex this than I thought....

[pjp]

Keep in mind that a binhost need not use the binaries it creates for clients.

The host OS could use a non-GUI configuration, such as:
"headless server" A) USE="-X -qt -gtk", etc.

The binhost could additionally offer two different sets of binaries. For example:
"gtk desktop" B) USE="X gtk -qt" and
"qt desktop" C) USE="X qt -gtk"

One client could use gtk binaries, the other client could use qt binaries, and the binhost itself uses the binaries it built in A. You can proie as many unique sets of binaries as you can manage to configure, store on disk, and provide sufficient compiling time to produce. Of course, it's easier to provide a single set of "generic" binaries that most clients can use.

In the guide you linked, see "Advanced topics -> Chrooting":
https://wiki.gentoo.org/wiki/Binary_package_guide#Chrooting

To paraphrase NeddySeagoon:
Don't think of Gentoo as a distro. Think of Gentoo as a toolkit you use to design, create, and install your own distro.
_________________
Quis separabit? Quo animo?

[Pipeng]

How is it? lets follow that, we have two clients, one want GTK and the other one QT.

Now lets imagine there is a package that has both options as USE, GTK and QT.

How we want to both clients be enable, means the Host will have GTK and QT enabled.

And now, if the client of GTK want that package, it will look on the Host, but both USEs are enabled, so the GTK client needs to have QT enabled for that package too.
This would also happens with the QT one.

So, both clients needs a minimum of compatibility.

Oks, maybe if we just enable GTK and QT globally on the clients, there will be some packages that will not retrieved from the Host for dependencies, but still need to match at this extent all the clients.. is really painfull....

[pjp]

[mega_flow]

portage support multi use flag in binpkg dir but if u use eclean-pkg it will clean any package not build with te use flag. don't use that app to clean te dir
u need to have multi chroot/machinectl/virtual machine . if you want to install different tool kits.
U can create a overlay per build machine for various hosts
For managen the different use flags i use the overlay

my-overlay
├── metadata/
│ └── layout.conf
└── profiles/
└── repo_name/parent ( to set the profile example gentoo:default/linux/amd64/23.0/no-multilib/systemd )
└── repo_name/make.defaults ( for make.conf )
└── repo_name/packages ( what package to install )
└── repo_name/package.use ( for use flags per package set )
└── repo_name/package.accept_keywords ( for unmasking unstable keywords )

this is not the official way to manage your settings. but make it a lot easier. I then sync this overlay to the host, so now i don't have to manual manage all setting on the host


For the VIDEO_CARDS i use package.use to set

x11-base/xorg-drivers -video_cards_nvidia -video_cards_radeonsi
x11-libs/libdrm video_cards_radeon video_cards_intel video_cards_amdgpu
media-libs/mesa video_cards_radeon video_cards_radeonsi video_cards_intel

and make.defaults

VIDEO_CARDS="-* radeonsi nvidia"
For manage various gpu hardware

This make it so i don't install nvidia-drivers on a amd gpu machine
_________________
default/linux/amd64/23.0/no-multilib/llvm/gnome/systemd gnome:46

[Pipeng]

@pip the pain is keep sync the host with all the clients! they all must match USEs, accept_keywords, etc, etc. And one change on the host means all clients must change it!
Still all clinets must have the same USEs as the Host, so is not important the Client USEs, change them to the host, or you will need to build locally, you don't want to build locally, then must use the same USEs as the Host..

@mega_flow your idea is great! solve like almost everything about it!

How works the example you post? if a client want a nvidia one, would request x11-base/xorg-drivers nvidia USE... so I don't get it veyr well.

I have no problem on have several chroot ones :3 I learned there is machinectl which seems a great option!

And what do you means with "support multi use flag in binpkg"?

[pjp]

[mega_flow]

@Pipeng portage can keep a package build with difference use flags on the same location . portage is smart enough to pick the right one for your host

the nvidia-driver is only pickup with xorg-drivers. So enable nvidia on example mutter wont pull the driver ( not sure on ffmpeg )
U can set this with the video_cards_nvidia or video_cards_(driver) to enable or disable the flags in package.use
( xorg-driver only install x11 driver ,wayland one ar install with mesa )
If u set VIDEO_CARDS or L10N make.defaults make sure u start with "-*" or portage enable all flags

in the local overlay the file packages wil set each application in the @world ( enable with a * example *sys-kernel/gentoo-kernel )
Any others package like nvidia-drivers i set in hostname file in the local overlay and linkt to /etc/portage/sets and emerge @hostname so i can control only host necessary packages.
_________________
default/linux/amd64/23.0/no-multilib/llvm/gnome/systemd gnome:46

[Goverp]

I exploit the current behaviour. My binpkg host (in a chroot) omits certain entries from /etc/portage/make.conf 'CPU_FLAGS_X86' since they are supported on only one of my two machines.
This means that when I try to install one or two packages that are sensitive to (and presumably exploit) the relevant flags, portage spurns the binpkg and compiles an host-specific version (and -march=native as well) - notably this applies to ffmpeg, and my little laptop needs all the help it can get with that.
_________________
Greybeard

[Pipeng]

thx for all the asnwers! taken me some time and I was able to implement the overlay one! well, close, I'm almost be able to do it.

For some reason, the "packages" file, which should set the packages to install, is not being recognized.... all the other parts are working fine.

I tested emerge -a --depclean and always try to remove everything.

the location is: myoverlay/profiles/profile_name/packages

Thx!