| View previous topic :: View next topic |
| Author |
Message |
jpsollie
Guru
Joined: 17 Aug 2013
Posts: 310
|
 Posted: Sun Sep 29, 2024 5:04 am Post subject: how do you guys handle qtwebengine |
 |
|
I'm fed up ...
every time I perform a emerge --update --deep --newuse --with-bdeps=y @system @world, it fails at the package qtwebengine.
mostly (yes, murphy's law) at the end of this heavy package,
I can then start digging through, experimenting with whatever comes to mind.
every time, i mean since QT 5.10, it was qtwebengine which broke the upgrade process.
I can't imagine being the only one who's afraid of compiling qtwebengine,
so how do you people deal with it when using kde plasma?
_________________
The power of Gentoo optimization (not overclocked): [img]https://www.passmark.com/baselines/V10/images/503714802842.png[/img] |
|
| Back to top |
|
 |
asturm
Developer
Joined: 05 Apr 2007
Posts: 9202
|
| Posted: Sun Sep 29, 2024 6:25 am Post subject: |
|
|
KDE Plasma itself can do without qtwebengine.
If your system lacks resources as determined by your build params, adjust build parameters in make.conf or per-package via package.env. Or increase resources hardware-wise.
Last edited by asturm on Sun Sep 29, 2024 7:01 am; edited 1 time in total |
|
| Back to top |
|
|
dmpogo
Advocate
Joined: 02 Sep 2004
Posts: 3395
Location: Canada
|
| Posted: Sun Sep 29, 2024 6:51 am Post subject: |
|
|
| got rid of everything that depends on it and run USE=-webengine |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2244
|
| Posted: Sun Sep 29, 2024 7:12 am Post subject: |
|
|
Binhost
Binhost x86_64-v3 if your CPU supports it.
| /etc/portage/binrepos.conf/gentoobinhost.conf: |
[binhost-x86_64]
priority = 9998
#sync-uri = https://ftp.snt.utwente.nl/pub/os/linux/gentoo/releases/amd64/binpackages/23.0/x86-64/
sync-uri = https://ftp.snt.utwente.nl/pub/os/linux/gentoo/releases/amd64/binpackages/23.0/x86-64/
[binhost-x86_64-v3]
priority = 9999
#sync-uri = https://ftp.snt.utwente.nl/pub/os/linux/gentoo/releases/amd64/binpackages/23.0/x86-64-v3/
sync-uri = https://ftp.snt.utwente.nl/pub/os/linux/gentoo/releases/amd64/binpackages/23.0/x86-64-v3/
|
Here the x86_64-v3 will take precedence and this way if a package is not available in v3 it'll be taken from the base x86_64 binrepo. If it's not available there too, then portage builds it from source.
Then adjust use flags if necessary. I think qtwebengine particularly does not need anything other than default, but nodejs for example needs -inspector. And I hate that crap but firefox:rapid requires it to build. However if you build chromium, it requires nodejs[inspector] which does not satisfy the binary package. However I used to use ungoogled-chromium-bin::pf4public which saved the day.
If you want to use it only for qtwebengine you update i separately with --getbinpkg. There still isn't a way to selectively chose binary packages other than doing it manually, sometimes terminating the source ebuild and then running the --getbinpkg query for that package manually and then starting the update over.
Another option is:
This will fetch the binary package from the binhost (-g/--getbinpkg) and then use it locally (-K/--usepkgonly). Still you need to update it separately.
EDIT: I just came up with an idea based on the above: emerge -gFpv <package(s)>. This will only fetch the binary package(s). Then continue the update with --usepkg. It'll use whatever binary packages that are compatible from the cache.
p.s. a related thread where it turned out to be a low memory condition: https://forums.gentoo.org/viewtopic-t-1170988.html Chromium and qtwebengine are notorious for that. Even more after gcc support for building chromium was dropped. Clang seems to have higher memory usage peeks. I've observed that since long long ago.
Best Regards,
Georgi
Last edited by logrusx on Sun Sep 29, 2024 9:44 am; edited 3 times in total |
|
| Back to top |
|
|
Banana
Moderator
Joined: 21 May 2004
Posts: 1660
Location: Germany
|
|
| Back to top |
|
|
lars_the_bear
Guru
Joined: 05 Jun 2024
Posts: 474
|
| Posted: Sun Sep 29, 2024 8:43 am Post subject: |
|
|
I've adjusted the package-specific USE flags so that I can use the binary version. I'm not going to try to compile that monster again on the hardware I have. I'm not sure what the implications are of doing this; but I'm installing qtwebengine at all, because FreeCAD has a dependency on it. I have no idea how (or even if) FreeCAD uses qtwebengine, and I'm not particular bothered whether it works properly.
BR, Lars. |
|
| Back to top |
|
|
Josef.95
Advocate
Joined: 03 Sep 2007
Posts: 4646
Location: Germany
|
| Posted: Sun Sep 29, 2024 9:27 am Post subject: |
|
|
But, what is the real problem?
Can you post the output from "emerge --info" and the error message (the build.log)?
/edit:
As first step you can check | Code: | | grep -i Killed /path/to/the/build.log |
and check the dmesg output. |
|
| Back to top |
|
|
jpsollie
Guru
Joined: 17 Aug 2013
Posts: 310
|
| Posted: Sun Sep 29, 2024 3:18 pm Post subject: |
|
|
that's exactly my point:
it's always something else: the build system, libraries, the compiler, binutils, ...
every time it crashes, but at every update it finds a new update to crash on.
For 6.7.2 and 6.7.3, the error log simply states:
| Code: |
[78/106] : && /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/linker_ulimit.sh /usr/lib/distcc/bin/x86_64-pc-linux-gnu-g++ -fPIC -march=znver1 -mtune=znver2 -O3 -pipe -Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-Bsymbolic-functions -Wl,--no-undefined -Wl,--version-script,/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/api/WebEngineCore.version @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_objects.rsp -Wl,--gc-sections -Wl,--enable-new-dtags -shared -Wl,-soname,libQt6WebEngineCore.so.6 -o lib64/libQt6WebEngineCore.so.6.7.3 src/core/api/CMakeFiles/WebEngineCore.dir/Unity/unity_1_cxx.cxx.o src/core/api/CMakeFiles/WebEngineCore.dir/Unity/unity_0_cxx.cxx.o -Wl,-rpath,::::::: -Wl,--start-group @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_archives.rsp -Wl,--end-group -Wl,--no-fatal-warnings @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_ldir.rsp @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_libs.rsp -Wl,--no-fatal-warnings /usr/lib64/libxkbcommon.so /usr/lib64/libQt6Quick.so.6.7.3 /usr/lib64/libQt6OpenGL.so.6.7.3 /usr/lib64/libQt6Gui.so.6.7.3 /usr/lib64/libGLX.so /usr/lib64/libOpenGL.so /usr/lib64/libQt6QmlModels.so.6.7.3 /usr/lib64/libQt6WebChannel.so.6.7.3 /usr/lib64/libQt6Qml.so.6.7.3 /usr/lib64/libQt6Network.so.6.7.3 /usr/lib64/libQt6QmlBuiltins.a /usr/lib64/libQt6Core.so.6.7.3 && : FAILED: lib64/libQt6WebEngineCore.so.6.7.3 : && /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/linker_ulimit.sh /usr/lib/distcc/bin/x86_64-pc-linux-gnu-g++ -fPIC -march=znver1 -mtune=znver2 -O3 -pipe -Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-Bsymbolic-functions -Wl,--no-undefined -Wl,--version-script,/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/api/WebEngineCore.version @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_objects.rsp -Wl,--gc-sections -Wl,--enable-new-dtags -shared -Wl,-soname,libQt6WebEngineCore.so.6 -o lib64/libQt6WebEngineCore.so.6.7.3 src/core/api/CMakeFiles/WebEngineCore.dir/Unity/unity_1_cxx.cxx.o src/core/api/CMakeFiles/WebEngineCore.dir/Unity/unity_0_cxx.cxx.o -Wl,-rpath,::::::: -Wl,--start-group @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_archives.rsp -Wl,--end-group -Wl,--no-fatal-warnings @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_ldir.rsp @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_libs.rsp -Wl,--no-fatal-warnings /usr/lib64/libxkbcommon.so /usr/lib64/libQt6Quick.so.6.7.3 /usr/lib64/libQt6OpenGL.so.6.7.3 /usr/lib64/libQt6Gui.so.6.7.3 /usr/lib64/libGLX.so /usr/lib64/libOpenGL.so /usr/lib64/libQt6QmlModels.so.6.7.3 /usr/lib64/libQt6WebChannel.so.6.7.3 /usr/lib64/libQt6Qml.so.6.7.3 /usr/lib64/libQt6Network.so.6.7.3 /usr/lib64/libQt6QmlBuiltins.a /usr/lib64/libQt6Core.so.6.7.3 && :
ninja: build stopped: subcommand failed.
* ERROR: dev-qt/qtwebengine-6.7.3::gentoo failed (compile phase):
* ninja -v -j70 -l12 failed
*
* Call stack:
* ebuild.sh, line 136: Called src_compile
* environment, line 3579: Called cmake_src_compile * environment, line 1650: Called cmake_build
* environment, line 1617: Called eninja
* environment, line 2099: Called die
* The specific snippet of code:
* "$@" || die -n "${*} failed" *
* If you need support, post the output of `emerge --info '=dev-qt/qtwebengine-6.7.3::gentoo'`,
* the complete build log and the output of `emerge -pqv '=dev-qt/qtwebengine-6.7.3::gentoo'`.
* The complete build log is located at '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/temp/build.log'.
* The ebuild environment file is located at '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/temp/environment'. * Working directory: '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build'
* S: '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3'
|
so yes, that doesn't say anything
_________________
The power of Gentoo optimization (not overclocked): [img]https://www.passmark.com/baselines/V10/images/503714802842.png[/img] |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2244
|
| Posted: Sun Sep 29, 2024 3:28 pm Post subject: |
|
|
This is not relevant output. There must be something above. I see it's only 78 of 108 so I guess you could share the whole build.log. Or at least 100 lines above what you've shared.
Best Regards,
Georgi |
|
| Back to top |
|
|
jpsollie
Guru
Joined: 17 Aug 2013
Posts: 310
|
| Posted: Sun Sep 29, 2024 4:15 pm Post subject: |
|
|
| logrusx wrote: | This is not relevant output. There must be something above. I see it's only 78 of 108 so I guess you could share the whole build.log. Or at least 100 lines above what you've shared.
Best Regards,
Georgi |
You're too optimistic:
| Code: |
ebuild /var/db/repos/gentoo/dev-qt/qtwebengine/qtwebengine-6.7.3.ebuild compile
>>> Existing ${T}/environment for 'qtwebengine-6.7.3' will be sourced. Run
>>> 'clean' to start with a fresh environment.
>>> Checking qtwebengine-6.7-patchset-13.tar.xz's mtime...
>>> Checking qtwebengine-everywhere-src-6.7.3.tar.xz's mtime...
>>> WORKDIR is up-to-date, keeping...
* checking ebuild checksums ;-) ... [ ok ]
* checking auxfile checksums ;-) ... [ ok ]
* checking miscfile checksums ;-) ... [ ok ]
* Checking for at least 8 GiB disk space at "/var/tmp/portage/dev-qt/qtwebengine-6.7.3/temp" ... [ ok ]
* Checking for at least 360 MiB disk space at "/usr" ... [ ok ]
>>> It appears that 'setup' has already executed for 'qtwebengine-6.7.3'; skipping.
>>> Remove '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/.setuped' to force setup.
>>> It appears that 'unpack' has already executed for 'qtwebengine-6.7.3'; skipping.
>>> Remove '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/.unpacked' to force unpack.
>>> It appears that 'prepare' has already executed for 'qtwebengine-6.7.3'; skipping.
>>> Remove '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/.prepared' to force prepare.
>>> It appears that 'configure' has already executed for 'qtwebengine-6.7.3'; skipping.
>>> Remove '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/.configured' to force configure.
>>> Compiling source in /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3 ...
* Source directory (CMAKE_USE_DIR): "/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3"
* Build directory (BUILD_DIR): "/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build"
ninja -v -j70 -l12 WebEngineCore_sync_all_public_headers
[1/1] cd /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/api && /usr/lib64/qt6/libexec/syncqt @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/api/WebEngineCore_syncqt_all_args
* Source directory (CMAKE_USE_DIR): "/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3"
* Build directory (BUILD_DIR): "/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build"
ninja -v -j70 -l12
[0/31] cd /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core && /usr/bin/ninja -j70 -l12 -v -C /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64 QtWebEngineCore ninja: Entering directory `/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64' ninja: no work to do.
[1/31] cd /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core && /usr/bin/ninja -j70 -l12 -v -C /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64 convert_dict
ninja: Entering directory `/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64'
ninja: no work to do.
[3/31] : && /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/linker_ulimit.sh /usr/lib/distcc/bin/x86_64-pc-linux-gnu-g++ -fPIC -march=znver1 -mtune=znver2 -O3 -pipe -Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-Bsymbolic-functions -Wl,--no-undefined -Wl,--version-script,/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/api/WebEngineCore.version @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_objects.rsp -Wl,--gc-sections -Wl,--enable-new-dtags -shared -Wl,-soname,libQt6WebEngineCore.so.6 -o lib64/libQt6WebEngineCore.so.6.7.3 src/core/api/CMakeFiles/WebEngineCore.dir/Unity/unity_1_cxx.cxx.o src/core/api/CMakeFiles/WebEngineCore.dir/Unity/unity_0_cxx.cxx.o -Wl,-rpath,::::::: -Wl,--start-group @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_archives.rsp -Wl,--end-group -Wl,--no-fatal-warnings @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_ldir.rsp @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_libs.rsp -Wl,--no-fatal-warnings /usr/lib64/libxkbcommon.so /usr/lib64/libQt6Quick.so.6.7.3 /usr/lib64/libQt6OpenGL.so.6.7.3 /usr/lib64/libQt6Gui.so.6.7.3 /usr/lib64/libGLX.so /usr/lib64/libOpenGL.so /usr/lib64/libQt6QmlModels.so.6.7.3 /usr/lib64/libQt6WebChannel.so.6.7.3 /usr/lib64/libQt6Qml.so.6.7.3 /usr/lib64/libQt6Network.so.6.7.3 /usr/lib64/libQt6QmlBuiltins.a /usr/lib64/libQt6Core.so.6.7.3 && :
FAILED: lib64/libQt6WebEngineCore.so.6.7.3
: && /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/linker_ulimit.sh /usr/lib/distcc/bin/x86_64-pc-linux-gnu-g++ -fPIC -march=znver1 -mtune=znver2 -O3 -pipe -Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-Bsymbolic-functions -Wl,--no-undefined -Wl,--version-script,/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/api/WebEngineCore.version @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_objects.rsp -Wl,--gc-sections -Wl,--enable-new-dtags -shared -Wl,-soname,libQt6WebEngineCore.so.6 -o lib64/libQt6WebEngineCore.so.6.7.3 src/core/api/CMakeFiles/WebEngineCore.dir/Unity/unity_1_cxx.cxx.o src/core/api/CMakeFiles/WebEngineCore.dir/Unity/unity_0_cxx.cxx.o -Wl,-rpath,::::::: -Wl,--start-group @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_archives.rsp -Wl,--end-group -Wl,--no-fatal-warnings @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_ldir.rsp @/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/QtWebEngineCore_libs.rsp -Wl,--no-fatal-warnings /usr/lib64/libxkbcommon.so /usr/lib64/libQt6Quick.so.6.7.3 /usr/lib64/libQt6OpenGL.so.6.7.3 /usr/lib64/libQt6Gui.so.6.7.3 /usr/lib64/libGLX.so /usr/lib64/libOpenGL.so /usr/lib64/libQt6QmlModels.so.6.7.3 /usr/lib64/libQt6WebChannel.so.6.7.3 /usr/lib64/libQt6Qml.so.6.7.3 /usr/lib64/libQt6Network.so.6.7.3 /usr/lib64/libQt6QmlBuiltins.a /usr/lib64/libQt6Core.so.6.7.3 && :
ninja: build stopped: subcommand failed.
* ERROR: dev-qt/qtwebengine-6.7.3::gentoo failed (compile phase):
* ninja -v -j70 -l12 failed
*
* Call stack:
* ebuild.sh, line 136: Called src_compile
* environment, line 3579: Called cmake_src_compile
* environment, line 1650: Called cmake_build
* environment, line 1617: Called eninja
* environment, line 2099: Called die
* The specific snippet of code:
* "$@" || die -n "${*} failed"
*
* If you need support, post the output of `emerge --info '=dev-qt/qtwebengine-6.7.3::gentoo'`,
* the complete build log and the output of `emerge -pqv '=dev-qt/qtwebengine-6.7.3::gentoo'`.
* The complete build log is located at '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/temp/build.log'.
* The ebuild environment file is located at '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/temp/environment'.
* Working directory: '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build'
* S: '/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3'
|
_________________
The power of Gentoo optimization (not overclocked): [img]https://www.passmark.com/baselines/V10/images/503714802842.png[/img] |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3638
Location: Rasi, Finland
|
| Posted: Sun Sep 29, 2024 4:24 pm Post subject: |
|
|
Hmm... | Quote: | | *snip*: | | ninja -v -j70 -l12 failed |
|
Ninja is running 70 concurrent compiler jobs at maximum, but limited if load average goes above 12.00.
There's quite a possibility for ninja to first spawn 70 jobs then eventually settling around 12 if nothing else is straining your system. I always assume each job takes one gig of RAM while compiling big packages. 70 concurrent jobs would surely kill any machines I own.
Maybe check your MAKEOPTS?
EDIT: Just saw you last post. MAKEOPTS may not be the problem here, but does not hurt to check.
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2244
|
| Posted: Sun Sep 29, 2024 4:31 pm Post subject: |
|
|
| Code: | | dmesg | grep killed |
Best Regards,
Georgi |
|
| Back to top |
|
|
jpsollie
Guru
Joined: 17 Aug 2013
Posts: 310
|
| Posted: Sun Sep 29, 2024 4:56 pm Post subject: |
|
|
| logrusx wrote: | | Code: | | dmesg | grep killed |
Best Regards,
Georgi |
| Code: |
janpieter@desktopjp ~ $ dmesg | grep killed
janpieter@desktopjp ~ $
|
The desktop machine has 64GB of ram for 12 CPUs (and 256GB of swap), but links to a 64 core distcc server, so that's where the -j70 -l12 comes from
*edit* some additional info:
| Code: |
COMMON_FLAGS="-march=znver1 -mtune=znver2 -O3 -pipe"
#MACHINE_FLAGS="-march=znver1 -mabm -madx -maes -mavx -mavx2 -mbmi -mbmi2 -mclflushopt -mclzero -mcx16 -mf16c -mfma -mfsgsbase -mfxsr -mlzcnt -mmmx -mmovbe -mmwaitx -mpclmul -mpopcnt -mprfchw -mrdrnd -mrdseed -msahf -msha -msse -msse2 -msse3 -msse4.1 -msse4.2 -msse4a -mssse3 -mxsave -mxsavec -mxsaveopt -mxsaves --param=l1-cache-line-size=64 --param=l1-cache-size=32 --param=l2-cache-size=512"
CFLAGS="${COMMON_FLAGS} ${MACHINE_FLAGS}"
CXXFLAGS="${COMMON_FLAGS} ${MACHINE_FLAGS}"
FCFLAGS="${COMMON_FLAGS} ${MACHINE_FLAGS}"
FFLAGS="${COMMON_FLAGS} ${MACHINE_FLAGS}"
# NOTE: This stage was built with the bindist Use flag enabled
PORTDIR="/var/db/repos/gentoo"
DISTDIR="/var/cache/distfiles"
PKGDIR="/var/cache/binpkgs"
BINPKG_COMPRESS="lz4"
BINPKG_FORMAT="gpkg"
FEATURES="nostrip -network-sandbox -ebuild-locks -sandbox -ipc-sandbox -pid-sandbox buildpkg distcc" # -userfetch -userpriv"
EMERGE_DEFAULT_OPTS="${EMERGE_DEFAULT_OPTS} --usepkg"
#DISTCC_IO_TIMEOUT=30
DISTCC_LOG="/tmp/distcc.log"
#DISTCC_VERBOSE=1
MAKEOPTS="-j70 -l12"
#MAKEOPTS="-j12"
|
... and the tail of distcc log:
| Code: |
distcc[3967] 11823232 bytes from /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/api/CMakeFiles/WebEngineCore.dir/Unity/unity_0_cxx.cxx compiled on linuxserver in 11.2919s, rate 1023kB/s distcc[3967] compile /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/api/CMakeFiles/WebEngineCore.dir/Unity/unity_0_cxx.cxx on linuxserver/70 completed ok
distcc[3967] elapsed compilation time 11.292266s
distcc[3967] (dcc_exit) exit: code 0; self: 0.004300 user 0.004300 sys; children: 0.607579 user 0.099740 sys
distcc[3973] (dcc_scan_args) called for link? i give up
distcc[3973] exec on localhost: x86_64-pc-linux-gnu-g++ -fPIC -march=znver1 -mtune=znver2 -O3 -pipe -Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-Bsymbolic-functions -Wl,--no-undefined -Wl,--version-script,/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/api/WebEngineCore.version /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/accessibility_tree_formatter_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/browser_accessibility_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/authentication_dialog_controller.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/autofill_client_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/autofill_popup_controller.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/browser_accessibility_manager_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/browser_main_parts_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/browser_message_filter_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/browsing_data_remover_delegate_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/certificate_error_controller.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/chromium_overrides.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/client_cert_select_controller.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/client_hints.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/clipboard_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/color_chooser_controller.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/color_chooser_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/qt_messages.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/compositor.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/content_gpu_client_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/display_overrides.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/display_software_output_surface.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/native_skia_output_device.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/content_browser_client_qt.o /var/tmp/p
distcc[3974] (dcc_execvp) ERROR: failed to exec x86_64-pc-linux-gnu-g++: Argument list too long
distcc[3974] (dcc_exit) exit: code 110; self: 0.000000 user 0.003660 sys; children: 0.000000 user 0.000000 sys
distcc[3973] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc[3973] ERROR: compile (null) on localhost failed with exit code 110
distcc[3973] elapsed compilation time 0.011420s
distcc[3973] (dcc_exit) exit: code 110; self: 0.014649 user 0.000000 sys; children: 0.000000 user 0.003913 sys
distcc[4910] (dcc_scan_args) called for link? i give up
distcc[4910] exec on localhost: x86_64-pc-linux-gnu-g++ -fPIC -march=znver1 -mtune=znver2 -O3 -pipe -Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-Bsymbolic-functions -Wl,--no-undefined -Wl,--version-script,/var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/api/WebEngineCore.version /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/accessibility_tree_formatter_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/browser_accessibility_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/authentication_dialog_controller.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/autofill_client_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/autofill_popup_controller.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/browser_accessibility_manager_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/browser_main_parts_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/browser_message_filter_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/browsing_data_remover_delegate_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/certificate_error_controller.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/chromium_overrides.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/client_cert_select_controller.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/client_hints.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/clipboard_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/color_chooser_controller.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/color_chooser_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/qt_messages.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/compositor.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/content_gpu_client_qt.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/display_overrides.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/display_software_output_surface.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/native_skia_output_device.o /var/tmp/portage/dev-qt/qtwebengine-6.7.3/work/qtwebengine-everywhere-src-6.7.3_build/src/core/RelWithDebInfo/x86_64/obj/QtWebEngineCore/content_browser_client_qt.o /var/tmp/p distcc[4911] (dcc_execvp) ERROR: failed to exec x86_64-pc-linux-gnu-g++: Argument list too long
distcc[4911] (dcc_exit) exit: code 110; self: 0.000000 user 0.003651 sys; children: 0.000000 user 0.000000 sys
distcc[4910] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc[4910] ERROR: compile (null) on localhost failed with exit code 110
distcc[4910] elapsed compilation time 0.011288s
distcc[4910] (dcc_exit) exit: code 110; self: 0.008630 user 0.005753 sys; children: 0.000000 user 0.003859 sys
|
_________________
The power of Gentoo optimization (not overclocked): [img]https://www.passmark.com/baselines/V10/images/503714802842.png[/img]
Last edited by jpsollie on Sun Sep 29, 2024 5:14 pm; edited 2 times in total |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2244
|
| Posted: Sun Sep 29, 2024 5:08 pm Post subject: |
|
|
| jpsollie wrote: | | distcc server |
I'm not very competent to analyze this setup. I hope others will come up with ideas of what might be going on. But what you describe is not normal. Even I didn't have such problems when I run a quad core CPU from 2007 with 6GB RAM.
But I strongly suggest using the binhost for that particular package. I have a 16T CPU with 32GB of RAM and I don't waste time compiling it although it would take 2 to 3 hours max. Compiling those(chromium derived pieces or crap) on whatever hardware is a waste of resource.
Best Regards,
Georgi |
|
| Back to top |
|
|
pingtoo
Veteran
Joined: 10 Sep 2021
Posts: 1138
Location: Richmond Hill, Canada
|
| Posted: Sun Sep 29, 2024 5:15 pm Post subject: |
|
|
| jpsollie wrote: | | logrusx wrote: | This is not relevant output. There must be something above. I see it's only 78 of 108 so I guess you could share the whole build.log. Or at least 100 lines above what you've shared.
Best Regards,
Georgi |
You're too optimistic:
| Code: |
ebuild /var/db/repos/gentoo/dev-qt/qtwebengine/qtwebengine-6.7.3.ebuild compile
...
... |
|
You are picking up from an previous failed job.
So why you are using "ebuild" to run? Why not using emerge so we can see from beginning where the error? |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2244
|
| Posted: Sun Sep 29, 2024 5:24 pm Post subject: |
|
|
| pingtoo wrote: | | jpsollie wrote: | | logrusx wrote: | This is not relevant output. There must be something above. I see it's only 78 of 108 so I guess you could share the whole build.log. Or at least 100 lines above what you've shared.
Best Regards,
Georgi |
You're too optimistic:
| Code: |
ebuild /var/db/repos/gentoo/dev-qt/qtwebengine/qtwebengine-6.7.3.ebuild compile
...
... |
|
You are picking up from an previous failed job. |
Exactly what I meant.
| pingtoo wrote: | | So why you are using "ebuild" to run? Why not using emerge so we can see from beginning where the error? |
I hadn't noticed that. Maybe to save time, but this is exactly how not to share error output. That's why the real cause is not visible. Whatever has failed has already failed and now jobs counting on that to do their job fail too.
There may still be low memory condition but this should be checked on the distcc server.
p.s. I'm not sure if the command I posted above greps for the right thing. I haven't had it on this computer and the old one is not functional.
Best Regards,
Georgi |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
