| View previous topic :: View next topic |
| Author |
Message |
techtruth
n00b
Joined: 01 Jan 2025
Posts: 6
|
 Posted: Sun Jan 05, 2025 10:12 pm Post subject: Binhost inside docker, no chroot |
 |
|
I'm not sure what I want is possible, but it seems possible to me. I'd like to see what the other people think.
My high level goal is to have my laptop do no compilation, and have all packages sourced from a binhost. That seems possible. The part that I'm struggling with is compiling all the binary packages that would be served by that binhost, in a way that is isolate from the dockerized root system.
I'd like to install packages and compile against /binhost instead of /
This way the docker container's root ( / ) is not used when compiling and installing packages in /binhost.
I understand there is a bootstrapping piece where the docker containers root is used to compile the first toolsets like gcc and coreutils for /binhost, but after that I think there would be a way to say "ok, use the binhost tooling and libraries to make packages".
This is sortof like cross-compiling for the same architecture, if that makes sense to you all out there.
This is a piece of my dockerfile that I am using to construct the binhost. Note the use of --root, --config-root, and --sysroot.
| Code: |
# Emerge step1 using system build tools
# Prepare host system with dependencies needed to build toolchain on /binhost
# and build the starting toolchain on /binhost
RUN USE="build" emerge --quiet --root /binhost --config-root /binhost --root-deps @toolchain
# Set environment to use libraries and programs and packages from /binhost
ENV PATH="/binhost/bin:/binhost/usr/bin"
ENV LD_LIBRARY_PATH="/binhost/lib:/binhost/lib64"
ENV PKG_CONFIG_PATH="/binhost/lib/pkgconfig:/binhost/lib64/pkgconfig:/binhost/usr/lib/pkgconfig:/binhost/usr/lib64/pkgconfig:/binhost/usr/share/pkgconfig"
ENV CPPFLAGS="-I/binhost/usr/include"
ENV LDFLAGS="-L/binhost/usr/lib64"
# Emerge step2 using binhost build tools to build out binhost packages
# After this, binhost should have compiled itself, and not be connected to host system toolchain
# Nothing from host system should be used...
RUN emerge --quiet --buildpkg --root /binhost --config-root /binhost --sysroot /binhost --root-deps @world
# Emerge step3 Installs and builds the user-selected packages, using all binhost programs and libs
RUN emerge --quiet --buildpkgonly --root /binhost --config-root /binhost --sysroot /binhost @my_package_set
|
I'd rather avoid chroot, as I feel like that hides linking/environment problems rather than addressing them.
Is is possible to achieve an isolated /binhost like this? I'm happy to post follow up info, but don't want to overload the initial forum post...
I have seen the prefix project but it doesn't seem to do what I want, at least as I understand it.
Bonus question (where I am currently stuck):
How do I tell emerge to use /binhost instead of / as the gnuconfig path? I've set things like CONFIG_SITE=/binhost/path/to/gnnuconfig but it still seem to run the packages configure with /usr/share/gnuconfig as the config directory. Ideally this would be something I can set with command line flags to emerge or environment variables. I've also tried CONFIG_SITE, CONFIG_SUB, CONFIG_GUESS, and EXTRA_ECONF="--config-site=/path/to/custom/gnuconfig" without success. |
|
| Back to top |
|
 |
zen_desu
Tux's lil' helper
Joined: 25 Oct 2024
Posts: 94
|
| Posted: Sun Jan 05, 2025 11:25 pm Post subject: |
|
|
https://github.com/desultory/genTree/tree/main
I'm actively working on this, genTree-server runs a webserver that serves the pkgdir from whatever seed you choose.
It handles all the mounts for you and technically uses a chroot, but does so in a namespace. The pkgdir/etc are mounted into the container, and it uses your system /var/db/repos and resolv.conf by default.
--root is only used for "layers" and is not really used in the binhost mode.
_________________
µgRD dev
Wiki writer |
|
| Back to top |
|
|
pingtoo
Veteran
Joined: 10 Sep 2021
Posts: 1363
Location: Richmond Hill, Canada
|
| Posted: Sun Jan 05, 2025 11:34 pm Post subject: |
|
|
I am currently working on my own project that may do what you looking for. It is not complete yet and it is very specific to my environment so I can not share exact content with you.
However I will share the design principle.
I use two layer of docker container.
The first layer of docker container is based on alpine which task is to setup storage for the inner docker to run build.
The inner docker container use catalyst to build a rootfs, as side effect of the rootfs build it will also create all packages for the rootfs. so in the end you will have tarball (rootfs) and a "packages" directory that can act like bin-host share.
The inner docker is gentoo stage3 based image with catalyst installed.
This whole initial setup take some effort but once it is done it will do what you are seeking with relative easy to maintain. it also setup possibility with docker volumes-from option so you can have separated web-server container to share the "packages" directory.
Using catalyst benefit that you will have a consisted build environment so modification be come easy.
I hope this give you ideas on your design. Please don't hesitate asking question should you need help. |
|
| Back to top |
|
|
techtruth
n00b
Joined: 01 Jan 2025
Posts: 6
|
| Posted: Mon Jan 06, 2025 1:35 am Post subject: Correction to my thinking... |
|
|
It occurred to me that I don't want to _run_ the binhost toolchain to compile packages for the binhost, because the binhost execs (gcc, clang, etc) will be compiled by the system toolchain with the binhost optimizations. It would be much better for me to run all the compiles with the system toolchains, such that the compiled packages are optimized for my laptop and the laptop is the only place they actually run or are #included from.
It seems best in my case to treat use flags and make.conf the same as a different architecture. I should pretend that I'm not able to use programs or libs that are not for my build system, but that build system can still compile stuff for my target (binhost).
If I am correct in my thinking, optimizations and libraries of the system exec's wont make a difference to the compiled packages for my laptop, because I will be using alternate use flags and make.conf of the binhost (/binhost/etc/portage/) to compile packages for the laptop.
You ever have an idea and end up closer to where you started than after you thought about all the details?  I must have read too much from the LinuxFromScratch manual and got it stuck in my head that I needed to compile my packages with a "clean toolchain" for when the build env goes away.
Testing this out tonight. I should be able to have 0 compile on laptop, yet all packages are binary and optimized for my laptop, with my use personal flags. |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2573
|
| Posted: Mon Jan 06, 2025 4:28 am Post subject: Re: Correction to my thinking... |
|
|
| techtruth wrote: | | It occurred to me that I don't want to _run_ the binhost toolchain to compile packages for the binhost, because the binhost execs (gcc, clang, etc) will be compiled by the system toolchain with the binhost optimizations. It would be much better for me to run all the compiles with the system toolchains, such that the compiled packages are optimized for my laptop and the laptop is the only place they actually run or are #included from. |
You cannot run the host toolchain inside the container. It defeats the purpose of containers.
Here's a Dockerfile for something I did recently, I hope it helps:
| Code: | FROM gentoo/stage3:latest
RUN emerge-webrsync
RUN echo "FEATURES=\"${FEATURES} getbinpkg\"" >> /etc/portage/make.conf
RUN getuto
RUN echo "dev-java/openjdk-bin headless-awt" >> /etc/portage/package.use/java
RUN emerge -q dev-java/openjdk-bin
|
You should be able to bring up a whole Gentoo installation this way. Adding buildpkg to FEATURES will make portage store binary packages for everything you build inside the container. Then you should be able to setup a binary packages host.
But are you sure what you're looking for is not discc?
And why don't you want to use the official binhost?
Best Regards,
Georgi |
|
| Back to top |
|
|
zen_desu
Tux's lil' helper
Joined: 25 Oct 2024
Posts: 94
|
| Posted: Mon Jan 06, 2025 4:47 am Post subject: |
|
|
It may be easier to start by making binpkgs with a more generic target, then you can try stuff like setting -mtune. This way you can get the process down, and then work on more complex settings.
_________________
µgRD dev
Wiki writer |
|
| Back to top |
|
|
techtruth
n00b
Joined: 01 Jan 2025
Posts: 6
|
| Posted: Mon Jan 06, 2025 4:51 am Post subject: Re: Correction to my thinking... |
|
|
| logrusx wrote: | | You cannot run the host toolchain inside the container. It defeats the purpose of containers. |
Sorry, got my jargon flopped around. I mean the binhost 'host' not the host system running the docker host. I suppose it would be better said as the docker container runtime (root) and the binhost inside that docker container runtime (binhost).
| logrusx wrote: | | But are you sure what you're looking for is not discc? |
I started there actually!  and that works if I want to issue compile commands in realtime. It doesn't let me update in the background though, like running a binhost in on a different server would.
| logrusx wrote: | | And why don't you want to use the official binhost? |
I want to use my special use flags and common_flags like -mtune and -march. I don't think I can do that with gentoo's default binhost as its not within my control. I'm sure there are a bunch of precompiled packages though. |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2573
|
| Posted: Mon Jan 06, 2025 6:08 am Post subject: Re: Correction to my thinking... |
|
|
| techtruth wrote: | | I want to use my special use flags and common_flags like -mtune and -march. |
What's the CPU you want to optimize for?
Best Regards,
Georgi |
|
| Back to top |
|
|
techtruth
n00b
Joined: 01 Jan 2025
Posts: 6
|
| Posted: Mon Jan 06, 2025 6:57 am Post subject: Re: Correction to my thinking... |
|
|
| logrusx wrote: | | What's the CPU you want to optimize for? |
AMD Ryzen 7 7840HS w/ Radeon 780M Graphics
portage make.conf file for my laptop, for more context.
| Code: | GRUB_PLATFORMS="efi-64"
COMMON_FLAGS="-march=znver4 -mshstk --param=l1-cache-line-size=64 --param=l1-cache-size=32 --param=l2-cache-size=1024 -mtune=znver4 -O3 -pipe"
FEATURES="distcc"
DISTCC_HOSTS="192.168.0.5"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
# Local has 16
# DistCC pool has 20 (desktop)
MAKEOPTS="-j36"
USE="dist-kernel wayland sound-server gtk3 udev -pulseaudio pipewire bluetooth dbus -gtk -systemd -elogind -cups -X -xwayland -systemd -x11-backend -xinerama -xkb -qt6 -qt5 -branding -valgrind"
VIDEO_CARDS="amdgpu radeonsi"
ACCEPT_LICENSE="@BINARY-REDISTRIBUTABLE"
|
|
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2573
|
| Posted: Mon Jan 06, 2025 7:02 am Post subject: |
|
|
Do you really need that? Do you use vector graphics or matrix multiplication et.c.? Do you really need to compile on another computer?
Best Regards,
Georgi
p.s. -systemd should be set by your profile(openrc, right?), why is it in your make.conf USE variable? Also it's there twice. |
|
| Back to top |
|
|
techtruth
n00b
Joined: 01 Jan 2025
Posts: 6
|
| Posted: Mon Jan 06, 2025 7:59 am Post subject: |
|
|
| logrusx wrote: | | Do you really need that? Do you use vector graphics or matrix multiplication et.c.? Do you really need to compile on another computer? |
No, I don't think I need to compile on another computer, I just want to do it for my own experimenting and customization. I do have some future projects that could use this, but no pressing needs. I also don't think I mess with vector graphics and matrix multiplications on anything outside maybe SVG images.
| logrusx wrote: | | p.s. -systemd should be set by your profile(openrc, right?), why is it in your make.conf USE variable? Also it's there twice. |
Ahh, good catch on the dupe! My profile is the generic amd64/23.0
| Code: | | /etc/portage/make.profile: symbolic link to ../../var/db/repos/gentoo/profiles/default/linux/amd64/23.0 |
|
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2573
|
| Posted: Mon Jan 06, 2025 8:12 am Post subject: |
|
|
| techtruth wrote: | | I just want to do it for my own experimenting and customization. |
OK, disregard that line, I just needed to dig deeper as oftentimes users end up with a more complex solution than it needs to be.
On the note of running the host toolchain (the host being the binhost running in the container). Yes, this is how you should run it. You can also run your real host's toolchain but I don't know exactly how to do that. You could run portage giving it a different root and it'll build everything there, you can set it up to build binary packages along the way. The only caveat I can think of is you should modify your make.conf during that emerge session to match your target system settings, just like when using distcc.
Best Regards,
Georgi |
|
| Back to top |
|
|
techtruth
n00b
Joined: 01 Jan 2025
Posts: 6
|
| Posted: Mon Jan 06, 2025 11:12 am Post subject: |
|
|
| logrusx wrote: | | OK, disregard that line, I just needed to dig deeper as oftentimes users end up with a more complex solution than it needs to be |
No worries! Reduce when ridiculous is a great optimization path 
| logrusx wrote: | | The only caveat I can think of is you should modify your make.conf during that emerge session to match your target system settings, just like when using distcc. |
I was hoping to leave the host (inside docker) system's make.conf alone, and only make the /binhost/etc/portage/make.conf and have that be used. |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2573
|
| Posted: Mon Jan 06, 2025 11:14 am Post subject: |
|
|
I think you can do that too if you just use the stage 3 for the toolchain and emerge in a root.
Best Regards,
Georgi |
|
| Back to top |
|
|
|
Portage & Programming
