| View previous topic :: View next topic |
| Author |
Message |
jankom
Guru
Joined: 30 Aug 2021
Posts: 334
Location: USA
|
 Posted: Mon Sep 23, 2024 10:31 am Post subject: qtwebengine compile fails |
 |
|
Hello,
I try to solve Gentoo problems myself, but am not proficient enough to help others yet. However, this time I'm stuck and need help.
I update my system every 2 to 3 months, last time was in June, this year (2024). Compiling all packages took more than 3 days, but finally qtwebengine failed. It was among last 3 of 341 packages. tried again, now with only 3 packages left, and in about 20 hours qtwebengine failed again after completing almost all of the [30117} tasks. The failure was a ninja.
Presently compiling again with "bindist" use flag enabled, and after 3 hours I'm, at [8066/30117].
Any ideas, please?
jankom |
|
| Back to top |
|
 |
fedeliallalinea
Administrator
Joined: 08 Mar 2003
Posts: 31199
Location: here
|
| Posted: Mon Sep 23, 2024 11:13 am Post subject: |
|
|
You should share the build.log and your emerge --info.
Without any information is hard to help you.
Check your dmesg to see if the problem could be an out of memory error.
_________________
Questions are guaranteed in life; Answers aren't. |
|
| Back to top |
|
|
jankom
Guru
Joined: 30 Aug 2021
Posts: 334
Location: USA
|
| Posted: Mon Sep 23, 2024 11:49 am Post subject: |
|
|
Here is part of the failure message | Code: | * Messages for package dev-qt/qtwebengine-6.7.2:
* Log file: /var/log/portage/log/dev-qt:qtwebengine-6.7.2:20240920-030743.log
* Amount of main memory is insufficient, but amount
* of main memory combined with swap is sufficient.
* Build process may make computer very slow!
* ERROR: dev-qt/qtwebengine-6.7.2::gentoo failed (compile phase):
* ninja -v -j4 -l0 failed
*
* Call stack:
* ebuild.sh, line 136: Called src_compile
* environment, line 3589: Called cmake_src_compile
* environment, line 1656: Called cmake_build
* environment, line 1623: Called eninja
* environment, line 2105: Called die
* The specific snippet of code:
* "$@" || die -n "${*} failed"
|
I was checking memory state and sometimes swap was used but not all. dmsg has already lot more entries.
Compiling still proceeding, now at [12920/30117]
Thanks |
|
| Back to top |
|
|
Ionen
Developer
Joined: 06 Dec 2018
Posts: 2820
|
| Posted: Mon Sep 23, 2024 12:00 pm Post subject: |
|
|
The build process usually does not use "that much" ram but there are occasional sudden spikes in usage and that's where it may fail. ebuild may say it's sufficient w/ swap, but that's an approximation and does not account for other things using ram at same time.
May want to check "dmesg"'s output, if haven't rebooted since last failure then it may mention having run out of memory.
If it's not ram, will ideally need the full build.log to know what happened. The bottom part is not saying much beside "something went wrong earlier on" and "here are the things you should provide if need support".
Note that there's also the option to just a binpkg of it, which tend to be a good idea for low end hardware that needs more than 3-4 hours to painfully compile this.
https://wiki.gentoo.org/wiki/Gentoo_Binary_Host_Quickstart
https://distfiles.gentoo.org/releases/amd64/binpackages/23.0/x86-64/dev-qt/qtwebengine/
One downside of binpkgs is that you'll be forced to enable USE=bindist which disables proprietary codecs (e.g. can't play h264 videos), albeit youtube and such has vp9/av1/opus free alternatives nowadays. Not that it matters if not going to browse the web with qtwebengine either way. |
|
| Back to top |
|
|
eschwartz
Developer
Joined: 29 Oct 2023
Posts: 166
|
| Posted: Mon Sep 23, 2024 1:56 pm Post subject: |
|
|
| jankom wrote: | Here is part of the failure message
|
Unfortunately, that's the only part that doesn't contain the failure message.
Please post the *entire* /var/log/portage/log/dev-qt:qtwebengine-6.7.2:20240920-030743.log |
|
| Back to top |
|
|
jankom
Guru
Joined: 30 Aug 2021
Posts: 334
Location: USA
|
| Posted: Tue Sep 24, 2024 7:41 am Post subject: Progress report |
|
|
| Still compiling, after about 24 hours: [24553/30117]. Will report if success or requested additional info if failure. Thank you all for your interest in helping - jankom |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2224
|
| Posted: Tue Sep 24, 2024 11:30 am Post subject: Re: Progress report |
|
|
| Ionen wrote: | | The build process usually does not use "that much" ram but there are occasional sudden spikes in usage and that's where it may fail. |
Yes, I've observed such behavior both for qtwebengine and comparable versions of chromium. What I used to do back then, I think it was the times of double digit chromium major versions was, to create a zram device and put swap there. This would save me from such situations. It would trade CPU time for memory, but it has it's limits of course. What should be considered is that zram takes 10 percent of the configured maximum size just to maintain an empty device.
It could save me from those sudden spikes which happen when more GCC/clang processes overlap in their high memory consumption stages. Back then I had an HDD so swap was highly undesirable as it could render the computer unresponsive.
| jankom wrote: | | Still compiling, after about 24 hours: [24553/30117]. Will report if success or requested additional info if failure. Thank you all for your interest in helping - jankom |
What's stopping you from using the binhost, at least for that particular package? If you don't want to configure a binhost, you can download the binary packages and use them locally too.
Best Regards,
Georgi |
|
| Back to top |
|
|
jankom
Guru
Joined: 30 Aug 2021
Posts: 334
Location: USA
|
| Posted: Tue Sep 24, 2024 1:32 pm Post subject: |
|
|
| Quote: | | What's stopping you from using the binhost, at least for that particular package? |
That'll be my next step.
When I do occasional system updates I do | Code: | eix-sync
emerge -avuND --with-bdeps=y @world |
then would continue with six more commands. I'm still at the "emerge -avUND..." step that failed after updating 338 of 341 packages. This is when I posted my first cry for help - while making another attempt by modifying /etc/portage/package.use file by adding a line | Code: | | dev-qt/qtwebengine bindist |
The "emerge -avUND..." process is still running, now at [27941/30117]. Hopefully soon if fails or finishes with the remaining two other packages.
I thought adding the bindist use flag in package.use file would make a difference. There was one other thing different: previously I was using ovpn to a remote server connecting to the outside world (Internet) on top of the Internet access via local ISP. I did find that sometimes during the compile process my ovpn connection stopped. This time I'm connecting to Internet without ovpn. But this could just a red herring.
xosview shows me the 4 cpu-s, the memory, swap and page activity, and so far everything is within normal operating conditions.
Will see...
jankom |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2224
|
| Posted: Tue Sep 24, 2024 4:58 pm Post subject: |
|
|
I think you're mistaking binary packages with use flags. Bindist is a use flag regarding components that are distributed as binaries due to licensing issues. Ref: equery uses qtwebengine:
| Code: | * Found these USE flags for dev-qt/qtwebengine-6.7.2:
U I
...
+ + bindist : Flag to enable or disable options for prebuilt (GRP) packages (eg. due to licensing issues) |
Here's how you configure a binhost: https://wiki.gentoo.org/wiki/Gentoo_Binary_Host_Quickstart
If your CPU is capable of executing x86_64-v3 code, there's a note pointing to a page that explains how to configure it. It contains instructions on how to check if your CPU supports it.
In your original command --with-bdeps defaults to yes, so why are you putting it there? It regards build-time dependencies, not binary packages. It'll default to no when you use binary packages as they don't need build-time dependencies. They are already built.
Best Regards,
Georgi |
|
| Back to top |
|
|
jankom
Guru
Joined: 30 Aug 2021
Posts: 334
Location: USA
|
| Posted: Tue Sep 24, 2024 7:45 pm Post subject: |
|
|
First of all, thank you Georgi. - and yes, I understand that bindist use flag is different from binary packages. My problem, or question, is how to proceed not to cause any demage to my system.
Presently the process failed again, same way, at the ninja step, almost at the very end. bindist use flag or the ovpn red herring made no difference. Here is what the failure message suggested to post: | Quote: | Portage 3.0.65 (python 3.12.6-final-0, default/linux/amd64/23.0/split-usr/desktop, gcc-13, glibc-2.39-r6, 6.6.30-gentoo x86_64)
=================================================================
System Settings
=================================================================
System uname: Linux-6.6.30-gentoo-x86_64-Intel-R-_Celeron-R-_CPU_J3455_@_1.50GHz-with-glibc2.39
KiB Mem: 7968552 total, 6936228 free
KiB Swap: 4194300 total, 4068848 free
Timestamp of repository gentoo: Tue, 17 Sep 2024 16:30:01 +0000
Head commit of repository gentoo: 3f620d6e08033ff38eef2fd0e09e5c1a4dcfcf05
sh bash 5.2_p26-r6
ld GNU ld (Gentoo 2.41 p5) 2.41.0
app-misc/pax-utils: 1.3.7::gentoo
app-shells/bash: 5.2_p26-r6::gentoo
dev-build/autoconf: 2.71-r7::gentoo
dev-build/automake: 1.16.5-r2::gentoo
dev-build/cmake: 3.30.2::gentoo
dev-build/libtool: 2.4.7-r4::gentoo
dev-build/make: 4.4.1-r1::gentoo
dev-build/meson: 1.5.1::gentoo
dev-java/java-config: 2.3.4::gentoo
dev-lang/perl: 5.40.0::gentoo
dev-lang/python: 3.11.9::gentoo, 3.12.6::gentoo
dev-lang/rust-bin: 1.80.1::gentoo
sys-apps/baselayout: 2.15::gentoo
sys-apps/openrc: 0.54.2::gentoo
sys-apps/sandbox: 2.39::gentoo
sys-devel/binutils: 2.41-r5::gentoo, 2.42-r1::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang: 17.0.6::gentoo, 18.1.8::gentoo
sys-devel/gcc: 13.3.1_p20240614::gentoo
sys-devel/gcc-config: 2.11::gentoo
sys-devel/llvm: 17.0.6::gentoo, 18.1.8-r1::gentoo
sys-kernel/linux-headers: 6.6-r1::gentoo (virtual/os-headers)
sys-libs/glibc: 2.39-r6::gentoo
Repositories:
gentoo
location: /var/db/repos/gentoo
sync-type: rsync
sync-uri: rsync://rsync.gentoo.org/gentoo-portage
priority: -1000
volatile: False
sync-rsync-verify-jobs: 1
sync-rsync-extra-opts:
sync-rsync-verify-max-age: 24
sync-rsync-verify-metamanifest: yes
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@BINARY-REDISTRIBUTABLE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"
LEX="flex"
MAKEOPTS="-j4"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X a52 aac acl acpi alsa amd64 branding bzip2 cairo cdda cdr cet crypt cups dbus dri dts dvd dvdr elogind encode exif flac gdbm gif gpm gtk gui iconv icu ipv6 jpeg kf6compat lcms libnotify libtirpc mad mng mp3 mp4 mpeg multilib ncurses nls ogg opengl openmp pam pango pcre pdf png policykit ppds qml qt5 qt6 readline sdl seccomp sound spell split-usr ssl startup-notification svg test-rust tiff truetype udev udisks unicode upower usb vorbis vulkan wxwidgets x264 xattr xcb xft xml xv xvid zlib" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="efi-64" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-US" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_12" RUBY_TARGETS="ruby31 ruby32" SANE_BACKENDS="hp epson2 net" VIDEO_CARDS="intel fbdev vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
|
How should I complete the system update? Yes, I can configure the binhost, but how will it complete the system update?
Sorry for not completely understand the picture - jankom |
|
| Back to top |
|
|
eschwartz
Developer
Joined: 29 Oct 2023
Posts: 166
|
| Posted: Tue Sep 24, 2024 8:42 pm Post subject: |
|
|
| jankom wrote: | First of all, thank you Georgi. - and yes, I understand that bindist use flag is different from binary packages. My problem, or question, is how to proceed not to cause any demage to my system.
Presently the process failed again, same way, at the ninja step, almost at the very end. bindist use flag or the ovpn red herring made no difference. Here is what the failure message suggested to post: |
The message should suggest to post two things:
- the emerge --info which you just posted
- the full contents of build.log containing details on which compilation commands succeeded, which failed, and what the output of the compiles were.
| jankom wrote: | How should I complete the system update? Yes, I can configure the binhost, but how will it complete the system update?
Sorry for not completely understand the picture - jankom |
If you enable the binhost, then you will get:
| Code: | Calculating dependencies... done!
Dependency resolution took 3.59 s (backtrack: 0/20).
[binary R ] dev-qt/qtwebengine-6.7.2-4
>>> Running pre-merge checks for dev-qt/qtwebengine-6.7.2
--2024-09-24 16:43:59-- https://gentoo.osuosl.org/releases/amd64/binpackages/23.0/x86-64-v3/dev-qt/qtwebengine/qtwebengine-6.7.2-4.gpkg.tar
Resolving gentoo.osuosl.org... 2600:3402:200:227::2, 2600:3404:200:237::2, 2605:bc80:3010::134, ...
Connecting to gentoo.osuosl.org|2600:3402:200:227::2|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 70799360 (68M) [application/x-tar]
Saving to: ‘/var/cache/binpkgs-gentoobinhost/dev-qt/qtwebengine/qtwebengine-6.7.2-4.gpkg.tar.partial’
/var/cache/binpkgs-gentoobinhost/dev-qt/qtwebengine/ 100%[=====================================================================================================================>] 67.52M 10.8MB/s in 11s
2024-09-24 16:44:10 (6.27 MB/s) - ‘/var/cache/binpkgs-gentoobinhost/dev-qt/qtwebengine/qtwebengine-6.7.2-4.gpkg.tar.partial’ saved [70799360/70799360]
* qtwebengine-6.7.2-4.gpkg.tar MD5 SHA1 size ;-) ... [ ok ]
>>> Emerging binary (1 of 1) dev-qt/qtwebengine-6.7.2::gentoo
* qtwebengine-6.7.2-4.gpkg.tar MD5 SHA1 size ;-) ... [ ok ]
>>> Extracting info
>>> Extracting dev-qt/qtwebengine-6.7.2
>>> Installing (1 of 1) dev-qt/qtwebengine-6.7.2::gentoo
* Install additional packages for optional runtime features:
* www-plugins/chrome-binary-plugins for Widevine DRM support (protected media playback)
*
* This version of Qt WebEngine is based on Chromium version 118.0.5993.220, with
* additional security fixes up to 125.0.6422.142. Extensive as it is, the
* list of backports is impossible to evaluate, but always bound to be behind
* Chromium's release schedule.
*
* In addition, various online services may deny service based on an outdated
* user agent version (and/or other checks). Google is already known to do so.
*
* tl;dr your web browsing experience will be compromised.
>>> Completed (1 of 1) dev-qt/qtwebengine-6.7.2::gentoo
|
No compilation will be done... |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
