View previous topic :: View next topic |
Author |
Message |
Guenther Brunthaler Apprentice
Joined: 22 Jul 2006 Posts: 219 Location: Vienna
|
Posted: Sat Jan 18, 2025 10:26 pm Post subject: How to favor source-built over binary packages? |
|
|
I chose Gentoo because it is a source-based Distribution.
However, much to my dislike, a lot of packages were installed as *-bin rather than being built from source.
For instance, when I try to merge "jdk", "openjdk-bin" would be merged by default as a dependency rather than "openjdk" which is also available.
Of course I could merge "openjdk" manually, but this is a lot of additional work in general because I needed to review all dependencies of all packages being merged, always checking for unwanted *-bin packages.
How can I reverse the portage default, so that binary packages are only chosen if there is no way to initially build the source package, i.e. for the famous compiler bootstrap problem when a compiler needs itself to be compiled.
But in all other cases, I want everything being built from source!
If I liked binary packages so much, I could have stayed with Debian.
BTW, I did not explicitly set any USE flags or FEATURES for requesting binary packages:
Code: |
$ emerge --info | sed 's/[^ ]*="/\n&/g' | grep -E '^(USE|FEATURES)='
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live ccache config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
USE="X a52 aac acl acpi alsa amd64 asm audiofile avif blas bluetooth bpf branding bzip2 cairo cdda cdr cet clang compress-xz crypt cuda cups dbus deduplicate designer dist-kernel dot dracut dri dts dvd dvdr encode eselect-ldso exif ffmpeg fftw flac fontconfig fuse gd gdbm gif git glusterfs gmp gpm grub gtk gtk-doc gui handbook html iconv icu initramfs inotify jack java jemalloc jpeg jpeg2k jpegxl json kf6compat kyotocabinet ladspa lame lapack lcms libffi libnotify libsamplerate libtirpc lv2 lvm lzo mad magic man matroska mbox mng modules modules-compress mono mp3 mp4 mpeg mpi mtp multilib ncurses nls nvenc odbc offlinehelp ogg openal opencl opencv opengl openmp openvpn opus pam pango pcap pcre pdf png policykit portaudio posix postgres ppds qemu qml qt5 qt6 readline scanner sdl sdl-image seccomp sid smp sndfile socks5 sound sox speech speex spell sqlite ssl startup-notification strip svg syslog taglib test-rust threads tidy tiff timidity tk truetype ttf udev unicode upower usb usbredir uuid v4l vaapi vde vdpau vnc vorbis vpx vte vulkan wayland webkit webp wifi wmf wxwidgets x264 xattr xcb xft xinetd xml xmp xmpp xpm xscreensaver xv xvid zip zlib"
|
|
|
Back to top |
|
|
sam_ Developer
Joined: 14 Aug 2020 Posts: 2093
|
Posted: Sat Jan 18, 2025 10:39 pm Post subject: |
|
|
The default is only ever binary for cases where there's a bootstrapping problem (Java, Rust). Someone is currently working on wiring up mrustc so we can have a bootstrap chain for Rust and we've discussed figuring that out for Java too.
Can you list what the other packages were other than Java and Rust, given they're very much a special case (that you identified)?
(With regard to generally avoiding binaries, I plan on pursing bug 885827 at some point, but I don't think it would've really helped here without more data at least.) |
|
Back to top |
|
|
netfab Veteran
Joined: 03 Mar 2005 Posts: 1963 Location: 127.0.0.1
|
Posted: Sun Jan 19, 2025 9:07 am Post subject: |
|
|
Quote: |
How to favor source-built over binary packages?
|
Mask all -bin packages :
/etc/portage/package.mask wrote: |
*/*-bin
|
Then, I guess that emerge will try to find a way without any -bin.
And if it cannot, it will ask you an extra action to unmask the package it needs. |
|
Back to top |
|
|
bstaletic Guru
Joined: 05 Apr 2014 Posts: 456
|
Posted: Sun Jan 19, 2025 11:21 am Post subject: |
|
|
Portage will prefer rust-bin over mrustc -> rust:1.74.1 -> rust:1.75.0 -> ... -> rust:1.84.0
Similarly, portage will prefer ada-bootstrap over gnat-gpl -> gcc:11 -> gcc:12 -> gcc:13 -> gcc:14
One can mask */*-bin and dev-lang/*-bootstrap, but some language compilers/interpreters can not be bootstrapped. As far as I know, the list is:- dev-lang/rust - work in progress
- dev-java/openjdk - gentoo-bootstrap overlay has a way to achieve openjdk bootstrap without a binary.
- dev-lang/go - There's this request for go slots. Again, gentoo-bootstrap has a way to bootstrap.
- sys-devel/gcc - If you're looking at this, I think you're well beyond what a distro can provide.
- dev-lang/pypy - Has a complete bootstrap path, but depends on python:2.7.
@sam_, I am guessing you are aware of it already, but in case you aren't. gentoo-bootstrap overlay only adds go-1.4 and go-1.17.3. Then just uses $CC to build go-1.4.
To be honest, while I like the idea of being able to bootstrap the world, I am completely fine with gentoo not providing that in ::gentoo.
I bet most people do not care about that and those that do may as well do extra work.
Gentoo devs are putting a massive effort to provide bootstrap paths and I shudder at the thought of maintaining that however many years from now. Keeping python:2.7 around in ::gentoo in 2040 is not something I'd wish on anyone. Maybe by 2040 we will get rid of pypy and have USE=+jit in CPython. Maybe gcc-rs will be able to compile rust and make mrustc obsolete. Maybe...
Cincerely, thank you to all the gentoo developers! |
|
Back to top |
|
|
sam_ Developer
Joined: 14 Aug 2020 Posts: 2093
|
Posted: Sun Jan 19, 2025 1:19 pm Post subject: |
|
|
bstaletic wrote: | Portage will prefer rust-bin over mrustc -> rust:1.74.1 -> rust:1.75.0 -> ... -> rust:1.84.0
Similarly, portage will prefer ada-bootstrap over gnat-gpl -> gcc:11 -> gcc:12 -> gcc:13 -> gcc:14
One can mask */*-bin and dev-lang/*-bootstrap, but some language compilers/interpreters can not be bootstrapped. As far as I know, the list is:- dev-lang/rust - work in progress
- dev-java/openjdk - gentoo-bootstrap overlay has a way to achieve openjdk bootstrap without a binary.
- dev-lang/go - There's this request for go slots. Again, gentoo-bootstrap has a way to bootstrap.
- sys-devel/gcc - If you're looking at this, I think you're well beyond what a distro can provide.
- dev-lang/pypy - Has a complete bootstrap path, but depends on python:2.7.
@sam_, I am guessing you are aware of it already, but in case you aren't. gentoo-bootstrap overlay only adds go-1.4 and go-1.17.3. Then just uses $CC to build go-1.4.
To be honest, while I like the idea of being able to bootstrap the world, I am completely fine with gentoo not providing that in ::gentoo.
I bet most people do not care about that and those that do may as well do extra work.
Gentoo devs are putting a massive effort to provide bootstrap paths and I shudder at the thought of maintaining that however many years from now. Keeping python:2.7 around in ::gentoo in 2040 is not something I'd wish on anyone. Maybe by 2040 we will get rid of pypy and have USE=+jit in CPython. Maybe gcc-rs will be able to compile rust and make mrustc obsolete. Maybe...
Cincerely, thank you to all the gentoo developers! |
mrustc support is not yet merged. When it is and once it's matured, the plan is to make it the default. But we already said Rust is a special case (ditto Java, which I mentioned we plan to work on).
With regard to Go, yes, that's on our list also, although I don't think go-1.4 is needed -- gccgo should be fine to build go-1.17 or go-1.18 or so.
With regard to ada-bootstrap: there is no source bootstrap path for an Ada compiler. gnat-gpl is also a bunch of binaries from adacore.
All of these are notably the compiler case as mentioned in OP's post rather than something else...
(And thanks -- I think it is worth saying that while we are interested in doing this & working on it, it is a huge amount of work, far more than people often seem to realise - like in that other thread..) |
|
Back to top |
|
|
bstaletic Guru
Joined: 05 Apr 2014 Posts: 456
|
Posted: Sun Jan 19, 2025 2:35 pm Post subject: |
|
|
sam_ wrote: | mrustc support is not yet merged. |
It was merged ~6 hours ago, between my previous and your previous comment. |
|
Back to top |
|
|
sam_ Developer
Joined: 14 Aug 2020 Posts: 2093
|
Posted: Sun Jan 19, 2025 2:39 pm Post subject: |
|
|
That doesn't change that the default will be adjusted, nor that my original post was right at the time of posting.. |
|
Back to top |
|
|
Guenther Brunthaler Apprentice
Joined: 22 Jul 2006 Posts: 219 Location: Vienna
|
Posted: Sun Jan 19, 2025 2:39 pm Post subject: |
|
|
sam_ wrote: |
Can you list what the other packages were other than Java and Rust, given they're very much a special case (that you identified)?
|
Well, "seabios-bin", but that's about it.
Thanks a lot for your explanation!
I did not realize Java had a bootstrapping problem and assumed it could easily be bootstrapped using gcj.
Obviously I was wrong, but I mistook this as a sign that Gentoo prefers binary packages for no obvious reasons at least in some cases.
Now things are clear.
I assume, now that the binary packages have been installed, I can replace them by the source-based packages to be built by the binary compilers.
I will try this as soon as my currently running "emerge" has finished (which might take a couple of days, estimating from its progress so far). |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|