| View previous topic :: View next topic |
| Author |
Message |
theskyentist
n00b
Joined: 03 Jan 2025
Posts: 4
|
 Posted: Fri Jan 03, 2025 1:33 pm Post subject: Ebuild runtime dependencies for Binary Package installations |
 |
|
Hello!
I'm new to Gentoo and Portage and trying to do the following:
I'd like to install specific packages from the Binary Package Host for the reason you'd expect, to avoid long & often compile times for frequently updated applications.
However, I'd still like to install the runtime dependencies for the package from the ebuild host, i.e. compile them from source (and all of their dependencies) since they tend to be small. This is quite similar to the behaviour of installing www-client/firefox-bin from the ebuild repository which compiles its runtime dependencies from source, but still installs firefox as a binary package.
Including the --getbinpkg option forces all runtime dependencies to be installed from source. I've tried to just compile dependencies by combining the --only-deps and the --with-bdeps=n options, but it still tries to install the build dependencies.
For example, let's say I want to install dev-qt/qtwebengine, of which a build (but not runtime) dependency is net-libs/nodejs, How could I install all of the runtime dependencies of qtwebengine as ebuilds without installing the build dependencies (e.g. nodejs) and then install the binary package? Ideally without adding all of the runtime dependencies to the world set.
I've tried to search the forums for an answer to this question but have so far come up empty. I'm also open to advice on why this might be non-adviseable thing to do in general.
Thanks in advance! |
|
| Back to top |
|
 |
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9620
Location: beyond the rim
|
| Posted: Fri Jan 03, 2025 2:26 pm Post subject: |
|
|
I don't think that is possible without specifying all packages you want to be installed manually (you could use --oneshot to avoid adding them to world if that is your main concern) with the current implementation of binary package support. Wouldn't even know how to define a user interface for this without adding a new option dedicated to this very specific use case.
It might be doable by creating a custom package set class, but I doubt you want to get into portage programming for this. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54668
Location: 56N 3W
|
| Posted: Fri Jan 03, 2025 2:44 pm Post subject: |
|
|
theskyentist,
Welcome to Gentoo.
I not sure I understand completely,
You want to install the run time dependences of a package using portage, than the package itself from the binhost.
You do not want the build time dependences of the package you will install from the the binhost.
Here is where my confusion comes in ...
prevents ALL build time dependences from being built, even build time dependences of the packages you want to build.
Maybe a worked example would help find a way forward?
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22925
|
| Posted: Fri Jan 03, 2025 2:49 pm Post subject: |
|
|
| As part of that worked example, I would like to see the emerge --pretend --verbose output of the commands you ran that you hoped would work, but did not. It seems strange to me that --only-deps --getbinpkg --with-bdeps=n would not do what you want, though I have never tried to replicate your workflow. Perhaps someone more familiar with the binpkg mechanics knows a reason why that should not work. I'd also like to see what happens if you use --only-deps --getbinpkgonly --with-bdeps=n, since that should force Portage to use binpkgs or explain why it cannot. There are reasons it might refuse, such as USE flag mismatches or absence of compatible versions, which I think it will show when --getbinpkgonly forbids it falling back to building from source. |
|
| Back to top |
|
|
theskyentist
n00b
Joined: 03 Jan 2025
Posts: 4
|
| Posted: Sat Jan 04, 2025 11:36 am Post subject: |
|
|
Hey everyone,
Thanks for the quick replies! I'm happy to provide a worked example.
I'm specifically looking at QTWebEngine.
Taking at look at the ebuild for this package, it lists nodeJS as a build dependency.
| Code: | BDEPEND="
$(python_gen_any_dep 'dev-python/html5lib[${PYTHON_USEDEP}]')
dev-util/gperf
net-libs/nodejs[ssl]
sys-devel/bison
sys-devel/flex
"
|
So when I run: emerge -pavt dev-qt/qtwebengine --with-bdeps=n
I get the following:
| Code: | [ebuild N ] dev-qt/qtwebengine-6.8.1:6/6.8.1::gentoo USE="alsa jumbo-build opengl qml system-icu vulkan widgets -accessibility -bindist -custom-cflags -designer -geolocation -kerberos -pdfium -pulseaudio -screencast -test -vaapi -webdriver" 0 KiB
[ebuild N ] dev-python/html5lib-1.1-r2::gentoo USE="-test" PYTHON_TARGETS="python3_12 (-pypy3) -python3_10 -python3_11 -python3_13" 0 KiB
[ebuild N ] dev-python/webencodings-0.5.1-r2::gentoo USE="-test" PYTHON_TARGETS="python3_12 (-pypy3) -python3_10 -python3_11 -python3_13" 0 KiB
[ebuild N ] dev-python/six-1.17.0::gentoo USE="-doc -test" PYTHON_TARGETS="python3_12 (-pypy3) -python3_10 -python3_11 -python3_13 (-python3_13t)" 0 KiB
[ebuild N ] net-libs/nodejs-22.4.1-r1:0/22::gentoo USE="icu npm snapshot ssl system-icu system-ssl -corepack -debug -doc -inspector (-lto) -pax-kernel -test" CPU_FLAGS_X86="sse2" 0 KiB
[ebuild N ] net-libs/ngtcp2-1.7.0::gentoo USE="gnutls ssl (-openssl) -static-libs -test" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild N ] dev-libs/simdjson-3.11.0:0/24::gentoo USE="all-impls -test -tools" 0 KiB
[ebuild N ] dev-qt/qtwebchannel-6.8.1:6/6.8.1::gentoo USE="qml -test" 0 KiB
[ebuild N ] dev-libs/libevent-2.1.12-r1:0/2.1-7::gentoo USE="clock-gettime ssl -debug -malloc-replacement -static-libs -test -verbose-debug -verify-sig" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild N ] app-arch/snappy-1.1.10-r1:0/1.1::gentoo USE="-test" ABI_X86="(64) -32 (-x32)" CPU_FLAGS_X86="avx avx2" 0 KiB |
Note that nodeJS is still being merged. This is only because it is a dependency of qtwebengine, I've gone through and checked the dependencies of all other requested packages to emerge and none of them would merge nodeJS.
I'm unclear as to why with-bdeps=n seems to still include nodeJS.
I would have expected the following workflow to work for me:
emerge --ask --only-deps --with-bdeps=n dev-qt/qtwebengine (fetch & compile runtime dependencies)
emerge --ask --getbinpkg dev-qt/qtwebengine (install just the pre-compiled binary since all run-time dependencies are already merged)
But the first step appears to still bring in at least one build dependency.
Thanks again for taking the time to look at this! |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54668
Location: 56N 3W
|
| Posted: Sat Jan 04, 2025 1:05 pm Post subject: |
|
|
theskyentist,
| Code: | | emerge -pavt dev-qt/qtwebengine --with-bdeps=n |
explicitly builds qtwebengine.
| Code: | | [ebuild N ] dev-qt/qtwebengine-6.8.1:6/6.8.1::gentoo |
it's flagged as New too.
Without the colour, I can't see if its a binary or not but I suspect not.
| Code: | | emerge --ask --only-deps --with-bdeps=n dev-qt/qtwebengine |
There is no - in the middle of --onlydeps.
Is that just in your post?
I get ... | Code: | | emerge: error: unrecognized arguments: --only-deps |
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
theskyentist
n00b
Joined: 03 Jan 2025
Posts: 4
|
| Posted: Sat Jan 04, 2025 3:47 pm Post subject: |
|
|
NeddySeagoon,
You're correct,
would attempt to build qtwebengine. My understanding is that adding --with-bdeps=n would not pull in build dependencies (likely causing a build failure in many cases, hence why it defaults to yes for installation actions). Therefore combining this with the --onlydeps flag (apologizes for the typo there) would hopefully compile all run-time dependencies for qtwebengine and then allow me to only fetch the binary package itself.
The displayed behavior however does not appear to be the case, where build dependencies are still attempted to be merged (e.g. nodeJS in the example I provided). I've also tried:
emerge -avtp dev-qt/qtwebengine --with-bdeps-auto=n --with-bdeps=n --onlydeps
And this still wants to merge nodeJS, which is a build dependency and not a run-time dependency (assuming I am understanding the ebuild correctly.
Hopefully this clarifies the situation. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10687
Location: Somewhere over Atlanta, Georgia
|
| Posted: Sat Jan 04, 2025 4:35 pm Post subject: |
|
|
theskyentist,
I think you're misunderstanding what --with-bdeps=n does. It doesn't keep any given package from being a build-time dependency. If a package is a build-time dependency of what you're asking to be built—and it's not currently installed—then regardless of the state of the --with-bdeps option, it will be installed because it's needed in order to build the package that you've asked to build. That's what being a build-time dependency means.
What --with-bdeps actually does is it prevents the build-time dependencies from being updated when an update that's not strictly necessary for the specified package to be built is available.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22925
|
| Posted: Sat Jan 04, 2025 4:55 pm Post subject: |
|
|
| NeddySeagoon wrote: | | Code: | | [ebuild N ] dev-qt/qtwebengine-6.8.1:6/6.8.1::gentoo |
Without the colour, I can't see if its a binary or not but I suspect not. |
If OP had managed to request a binary package, it would read For me, using glibc as an example: | Code: | # emerge -p sys-libs/glibc
These are the packages that would be merged, in order:
[ebuild R ] sys-libs/glibc-2.40-r5:2.2
# emerge -pk sys-libs/glibc
These are the packages that would be merged, in order:
[binary R ] sys-libs/glibc-2.40-r5:2.2 |
| theskyentist wrote: |
The displayed behavior however does not appear to be the case, where build dependencies are still attempted to be merged (e.g. nodeJS in the example I provided). I've also tried:
emerge -avtp dev-qt/qtwebengine --with-bdeps-auto=n --with-bdeps=n --onlydeps
And this still wants to merge nodeJS, which is a build dependency and not a run-time dependency (assuming I am understanding the ebuild correctly. |
What was the full output of the command that tried to build nodejs? Since you have --tree enabled, we can check use that to check whether some package that you are building has its own dependency on nodejs. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9620
Location: beyond the rim
|
| Posted: Sun Jan 05, 2025 4:25 am Post subject: |
|
|
| theskyentist wrote: |
So when I run: emerge -pavt dev-qt/qtwebengine --with-bdeps=n
I get the following:
| Code: | [ebuild N ] dev-qt/qtwebengine-6.8.1:6/6.8.1::gentoo USE="alsa jumbo-build opengl qml system-icu vulkan widgets -accessibility -bindist -custom-cflags -designer -geolocation -kerberos -pdfium -pulseaudio -screencast -test -vaapi -webdriver" 0 KiB
[ebuild N ] dev-python/html5lib-1.1-r2::gentoo USE="-test" PYTHON_TARGETS="python3_12 (-pypy3) -python3_10 -python3_11 -python3_13" 0 KiB
[ebuild N ] dev-python/webencodings-0.5.1-r2::gentoo USE="-test" PYTHON_TARGETS="python3_12 (-pypy3) -python3_10 -python3_11 -python3_13" 0 KiB
[ebuild N ] dev-python/six-1.17.0::gentoo USE="-doc -test" PYTHON_TARGETS="python3_12 (-pypy3) -python3_10 -python3_11 -python3_13 (-python3_13t)" 0 KiB
[ebuild N ] net-libs/nodejs-22.4.1-r1:0/22::gentoo USE="icu npm snapshot ssl system-icu system-ssl -corepack -debug -doc -inspector (-lto) -pax-kernel -test" CPU_FLAGS_X86="sse2" 0 KiB
[ebuild N ] net-libs/ngtcp2-1.7.0::gentoo USE="gnutls ssl (-openssl) -static-libs -test" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild N ] dev-libs/simdjson-3.11.0:0/24::gentoo USE="all-impls -test -tools" 0 KiB
[ebuild N ] dev-qt/qtwebchannel-6.8.1:6/6.8.1::gentoo USE="qml -test" 0 KiB
[ebuild N ] dev-libs/libevent-2.1.12-r1:0/2.1-7::gentoo USE="clock-gettime ssl -debug -malloc-replacement -static-libs -test -verbose-debug -verify-sig" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild N ] app-arch/snappy-1.1.10-r1:0/1.1::gentoo USE="-test" ABI_X86="(64) -32 (-x32)" CPU_FLAGS_X86="avx avx2" 0 KiB |
Note that nodeJS is still being merged. This is only because it is a dependency of qtwebengine, I've gone through and checked the dependencies of all other requested packages to emerge and none of them would merge nodeJS.
I'm unclear as to why with-bdeps=n seems to still include nodeJS. |
Because you effectively requested to build qtwebengine from source, so nodeJS is required. --with-bdeps is intended to control inclusion of build dependencies in the dependency graph when they are not strictly required (when they are already satisfied, but could be updated), it has no effect in this case. As said, I don't think what you want to do is possible atm as the options for selecting binary packages work on a global basis, so when you're selecting the binary package of qtwebengine you will also get the binary package for the dependencies when available. And when selecting the ebuild version you will always get the build dependencies as well. |
|
| Back to top |
|
|
theskyentist
n00b
Joined: 03 Jan 2025
Posts: 4
|
| Posted: Sun Jan 05, 2025 10:38 am Post subject: |
|
|
@Genone and @John R. Graham
I was definitely wrong about the behaviour of --with-bdeps, so I appreciate the clarification on the usage of that option, it seems like that is therefore not the way to go for attempting to do what I am trying to achieve.
@Hu is correct that running emerge -avtG dev-qt/qtwebengine --binpkg-respect-use=n would yield a binary package installation, but the runtime dependencies would also be installed from the binary host:
| Code: | [binary N ] dev-qt/qtwebengine-6.8.1-r1-3:6/6.8.1::gentoo USE="alsa bindist jumbo-build opengl pulseaudio qml system-icu vulkan widgets -accessibility -custom-cflags -designer -geolocation -kerberos -pdfium -screencast -test -vaapi -webdriver" 72,820 KiB
[binary N ] dev-libs/libevent-2.1.12-r1-2:0/2.1-7::gentoo USE="clock-gettime ssl -debug -malloc-replacement -static-libs -test -verbose-debug -verify-sig" ABI_X86="(64) -32 (-x32)" 340 KiB
[binary N ] dev-qt/qtwebchannel-6.8.1-1:6/6.8.1::gentoo USE="qml -test" 150 KiB
[binary N ] app-arch/snappy-1.1.10-r1-1:0/1.1::gentoo USE="-test" ABI_X86="(64) -32 (-x32)" CPU_FLAGS_X86="-avx -avx2" 70 KiB
|
I think @Genone's original suggestion to --oneshot the required packages (thereby not polluting the world set with dependencies that may be removed at a later date) is probably the best way forward. After these discussions there appear to be two strategies in mind:
Install all dependencies (including build+run), install binary package, clean dependencies.
emerge --avt dev-qt/qtwebengine --onlydeps #install build+run dependencies
emerge --avtG dev-qt/qtwebengine --binpkg-respect-use=n #install binary package
emerge --depclean #clean build dependencies
Install only runtime dependencies with oneshot, install binary package
emerge --avt dev-libs/libevent dev-qt/qtwebchannel app-arch/snappy --oneshot #install runtime dependencies but dont add to world
emerge --avtG dev-qt/qtwebengine --binpkg-respect-use=n # install binary package
The second is clearly more work but avoid installing unnecessary packages in the short term, and could likely be scripted.
Are there other options I should consider or are there clear flaws with the process I've outlined?
Thanks again for everyone's time and attention on this! |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54668
Location: 56N 3W
|
| Posted: Sun Jan 05, 2025 11:25 am Post subject: |
|
|
theskyentist,
You could write a set or several sets listing build time dependencies.
That will aid scripting. Call them set1 and set2.
emerge @set1 will build and install the list in set1 and add @set1 to world.
emerge @set2 -1G will install set2 from the binhost and not list it in world.
It's not a new idea but it may help your scripting.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
|
Portage & Programming
