qtwebengine taking more than a day to emerge

Hu · Administrator Joined: 06 Mar 2007 Posts: 22626

If avoiding big updates is a sufficient priority for you, another option would be to force Portage to ignore subslots. Take the icu upgrade now. Refuse to rebuild the consuming packages. Rely on @preserved-rebuild to retain the old parts of icu until qtwebengine/libreoffice no longer need those old parts. That way, as soon as you rebuild something that uses icu, you switch it to the new version, even if you hold a few packages back for special circumstances.

Verdazil · Posted: Wed Mar 20, 2019 6:26 pm Post subject: Time required to emerge package

fedeliallalinea · Posted: Wed Mar 20, 2019 7:11 pm Post subject:

In my system is normal

Verdazil · Posted: Wed Mar 20, 2019 7:19 pm Post subject:

fedeliallalinea · Posted: Wed Mar 20, 2019 7:28 pm Post subject:

You enabled/disabled jumbo-build?
_________________
Questions are guaranteed in life; Answers aren't.

NeddySeagoon · Posted: Wed Mar 20, 2019 7:45 pm Post subject:

Verdazil,

That command gives elapsed time. Elapsed time depends on what else you are doing at the time.
By default, portage runs things with a nice value of 3. Your interactive tasks run at a nice value of 0. The means your tasks are always run ahead of emerges when there are several things competing for CPU time.

Other factors can extend elapsed time, e.g. swapping. You need to compare build conditions and build times together. Elapsed times alone are not very useful.

Verdazil · Posted: Wed Mar 20, 2019 8:34 pm Post subject:

saellaven · l33t Joined: 23 Jul 2006 Posts: 654

I don't have that package on my system, but C++ programs can take a notoriously long time to compile and each version of gcc has gotten slower at it. Is it possible you switched versions of gcc in the meantime?

It's also possible you were compiling multiple QT/KDE packages (all C++) at the same time as part of an upgrade, and they all slowed each other down.

Verdazil · Posted: Thu Mar 21, 2019 4:26 am Post subject:

gcc version did not change

NeddySeagoon · Posted: Thu Mar 21, 2019 8:29 pm Post subject:

Verdazil,

It can't be that bad to update several packages at the same time, provided you have the resource to support it.
The emerge command can do it if you give the correct option.

Look up the --jobs= option
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Naib · Posted: Thu Mar 21, 2019 8:38 pm Post subject:

IO can also slow this down
_________________

Verdazil · Posted: Fri Mar 22, 2019 4:35 am Post subject:

Ant P. · Watchman Joined: 18 Apr 2009 Posts: 6920

cyril42e · n00b Joined: 10 Oct 2014 Posts: 3

It seems @fedeliallalinea had the right hint, except that it is because the jumbo-build use flag was enabled by default before 5.11 and disabed by default starting with 5.12:

Chiitoo · Posted: Sat May 25, 2019 8:31 pm Post subject:

I initially set 'jumbo-build' to be enabled by default during testing, and I believe I intended to reverse it before it ever hit the official Gentoo ebuild repository, but kind of forgot, I maybe guess.

It can lead to an increased memory usage during compilation, causing build failures on machines sporting less than enough RAM, and as such, having it disabled by default makes more sense to me. For those having no such issues, it's probably still better to have it enabled (faster builds).

Apologies for the confusion!
_________________
Kindest of regardses.

Dragonix

Hi,

as this thread is one of the first results when googling for "gentoo qtwebengine" I thought I'd share my experience with the recent update to qtwebengine-5.12.3.
I've got 3 gentoo boxes (Sandy Bridge/32GB Ram/-j12, Skylake/16 GB Ram/-j5, Core2Duo/4GB Ram + 4 GB Swap/-j3), and on every one of these systems I noticed a significant increase in the compile time. Finally I realized, that qtwebengine was previously built with "jumbo-build" enabled, which became disabled for 5.12.3. After re-enabling "jumbo-build", the compile times went back to normal (Sandy Bridge: 2 hours back to 1 hour, Skylake: 3 hours back to 1.5 hours, Core2Duo: 14 hours back to 6.5 hours).

So, at least for me, enabling "jumbo-build" lead to a great improvement in regards to compile time - even on my smallest system with only 4GB Ram+4GB Swap (in my experience, swap is needed irrespective of "jumbo-build", 4GB Ram don't suffice).

Hope this helps someone,
Matthias

<3 · Veteran Joined: 21 Oct 2004 Posts: 1083

I don't know what is going on with webkit/blink but the compile times have gotten way out of hand lately. It's even worse than GCC or libreoffice.

pjp · Administrator Joined: 16 Apr 2002 Posts: 20484

Merged the previous post. The times for qtwebengine look a lot like the discussion here.
_________________
Quis separabit? Quo animo?

Dragonix · Posted: Wed May 29, 2019 7:07 am Post subject:

@<3:
Do you have "jumbo-build" enabled? As I've written above your post, starting with 5.12.3 "jumbo-build" became disabled as a default (at least for me). Reenabling "jumbo-build" reduced the compile times massively (again, at least for me).
Nevertheless, perhaps you're running out of RAM - did you try to lower the MAKEOPTS to -j4? I'm wondering because even my much older Core2Duo System (4GB RAM+4GB Swap) is faster than your Quadcore (6.5 hours with jumbo-build, 14 hours without jumbo-build). Alternatively, there's something wrong with my installation...

mikefot · l33t Joined: 19 Nov 2014 Posts: 709

Thanks for pointing out the

Chiitoo · Posted: Wed May 29, 2019 11:57 am Post subject:

Yeah, with 6 GiBs of RAM, and 8 threads, I'd imagine it's very likely that swapping will occur, which may indeed slow everything down. A lot.

It might be interesting to try if lowering MAKEOPTS and enabling USE="jumbo-build" would be better, than higher MAKEOPTS but USE="-jumbo-build". Not many will want to actually test this out of course, since it takes such a long time...

If I go with MAKEOPTS="-j16" (Ryzen 7 1700), I'll dip over into swap by over 8 GiBs, and I have 16 GiBs of RAM. I need to go with '-j11' or even '-j10' to avoid swapping. (This is with USE="jumbo-build".)

I run with Qt 5.13.9999 currently (always the latest versions), to help with packaging it all for Gentoo (I mainly work on the qt overlay at this time), and I often build all of it at least once per day. I, too, would like it if the Chromium within would become less insane. :]

(I make use of 'ccache' a lot, so I'm not always building things from scratch. With regards to the WebEngine, the longer builds only happen when Chromium is updated for it, which, for normal use, would definitely happen during a new release, so 'ccache' won't help there.)
_________________
Kindest of regardses.

skellr · Posted: Wed May 29, 2019 8:41 pm Post subject:

skellr · Posted: Wed May 29, 2019 9:35 pm Post subject:

Just installed falkon to see what would happen...

alamahant · Advocate Joined: 23 Mar 2019 Posts: 3916

C++ sucks in compile time..
qt5 is ridiculous..
So is net-libs/webkit-gtk...
rustc also tragical.
They take much longer to compile even than a fully blown kernel.
Its the abominable C++

I usually use an --exclude flag in emerge until things start to get nasty..

<3 · Veteran Joined: 21 Oct 2004 Posts: 1083

Thanks for the tips, for years I thought about upgrading my RAM but 6GB seemed plenty now my hand might be forced to upgrade. Unfortunately The max my laptop will hold is 16GB, I might need to consider rebuilding my old gaming rig and using that as my primary PC with newer parts and using that as a discc sever for my laptop. For now I have re-enabled cache and use = jumbo-build. For other programs -j9 works fine on my lap top even with limited memory, is it possible to set a low job number for qtwebengine only?