qtwebengine taking more than a day to emerge

urcindalo

Hi!

Whenever portage decides to emerge qtwebengine, it takes almost forever

:

fedeliallalinea · Posted: Mon Jan 14, 2019 2:55 pm Post subject:

qtwebengine include chromium code, and no, webkit-gtk isn't equivalent to qtwebengine in term of compilation time.
In my i7-4930K_CPU_@_3.40GHz

urcindalo · Posted: Mon Jan 14, 2019 3:11 pm Post subject:

Thanks for the info.

By "equivalent" I mean source code size: I also show libreoffice compilation time.

In your case, qtwebengine compilation time doubles that of webkit-gtk, but in mine it is up to six or seven times larger because, as a side note, I must say the compilation times showed by splat are not real: last one took a little bit more than two days, not 1.4 as reported.
So, I find it strange it takes that long.

For instance, in a more powerful box I have the compilation times for qtwebengine and webkit-gtk are:

Chiitoo · Posted: Mon Jan 14, 2019 9:50 pm Post subject: ><)))°€

For a bit of a comparison, I have a log of 6 hours, 36 minutes, and 6 seconds on qtwebengine-5.11.1, on an AMD Athlon 7750 dual core processor with around 3 GiBs of RAM. I do forget if that build made use of distcc or not, but I'd still probably expect yours to be able to do better.

The Qt WebEngine build really likes RAM. A lot. Which you seem to have around 4 GiB, and my first suspect therefore would be major swapping during the build.

If you have been building with USE="jumbo-build", I might try without it, and/or lowering the MAKEOPTS value.

Or/and add more RAM, if possible. :]
_________________
Kindest of regardses.

urcindalo · Posted: Tue Jan 15, 2019 7:59 am Post subject: Re: ><)))°€

mael · n00b Joined: 28 Feb 2018 Posts: 37 Location: Germany

asturm · Developer Joined: 05 Apr 2007 Posts: 9275

urcindalo · Posted: Tue Jan 15, 2019 11:57 am Post subject: Re: ><)))°€

mael · n00b Joined: 28 Feb 2018 Posts: 37 Location: Germany

you should also check if your cpu or memory is temporary clocked down because longtime compilings like qtwebengine can stress out bad cooling systems in some boxes or notebooks and therefore activate some slowing safety measures inside your online cpus
_________________
Gigabyte X99-SLI
Intel Core i7-5960X 16 Threads @ 3 Ghz
64 GB DDR4 RAM @ 2,13 Ghz
AMD Radeon 290X 8 GB RAM
Creative Soundblaster Z
Areca ARC-1883ix-16
16 Samsung 860 EVO 250 GB SSDs @ 3,5 TB RAID 6
Hauppauge HVR-5525
Firmwares are up-to-date

urcindalo · Posted: Wed Jan 16, 2019 8:02 am Post subject:

brokendreams · n00b Joined: 11 Nov 2018 Posts: 22 Location: clinton,ia US

Adding --quiet to the emerge command will speed this up some. As well as adding -w to cflags and cxxflags (disables printing of most warning messages). Shaved 30mins off my build time (Intel 7700hq cpu). Started at 2 hrs...dropped it down to just under 90mins and thats with all use flags enabled. And portage tmpdir on tmpfs.

curmudgeon · Veteran Joined: 08 Aug 2003 Posts: 1744

I hate having to emerge qtwebengine (and recently spent the day emerging it because of an dev-libs/icu update, and then had to emerge it a couple of days later when a newer version of qtwebengine itself became stable (GRRRRR!).

It takes between five and fifteen hours on my machines, and used to crash more often than not. The biggest problem (which has only appeared in the most recent versions) is that each thread takes up about 2.5 GB of RAM, so if you have lots of cores, but not a corresponding amount of RAM, you will thrash the hard drive by swapping everything at best, or exhaust memory and crash.

I can at least get it to compile by limiting threads to one on machines with 4 GB of RAM, and two on machines with 8 GB of RAM (instead of the usual -j5). That actually seems faster (haven't done any empirical tests), because the processes aren't always stalled waiting for RAM to become available).

The easiest way to do that is:

mike155 · Posted: Wed Jan 16, 2019 6:22 pm Post subject:

urcindalo · Posted: Thu Jan 17, 2019 9:54 am Post subject:

sao98021 · Posted: Thu Jan 17, 2019 11:22 am Post subject:

urcindalo · Posted: Thu Jan 17, 2019 3:06 pm Post subject:

dmpogo · Advocate Joined: 02 Sep 2004 Posts: 3425 Location: Canada

sao98021 · Posted: Thu Jan 17, 2019 8:09 pm Post subject:

mael · n00b Joined: 28 Feb 2018 Posts: 37 Location: Germany

he is like me probably a qt-5/kde-5 user, so migrating to palemoon and other gtk+ based stuff is not what he wants, right? are you using falkon by any chance?
_________________
Gigabyte X99-SLI
Intel Core i7-5960X 16 Threads @ 3 Ghz
64 GB DDR4 RAM @ 2,13 Ghz
AMD Radeon 290X 8 GB RAM
Creative Soundblaster Z
Areca ARC-1883ix-16
16 Samsung 860 EVO 250 GB SSDs @ 3,5 TB RAID 6
Hauppauge HVR-5525
Firmwares are up-to-date

dmpogo · Advocate Joined: 02 Sep 2004 Posts: 3425 Location: Canada

dmpogo · Advocate Joined: 02 Sep 2004 Posts: 3425 Location: Canada

Honestly speaking, it would go long way in aleviating problems with long compilations, if there was some coordination between releases of large packages and their dependencies which cause recompilation. icu is one of the main culprits. All the time there is update to icu, recompilation, and then couple of days later there is update to qtwebengine or libreoffice, and you start all over again. Or in reverse order.

xor_ · n00b Joined: 09 Aug 2009 Posts: 10

dmpogo · Advocate Joined: 02 Sep 2004 Posts: 3425 Location: Canada

Jaglover · Posted: Wed Jan 23, 2019 12:34 am Post subject:

ccache may be evil, but it helps in such cases.
_________________
My Gentoo installation notes.
Please learn how to denote units correctly!

mike155 · Posted: Wed Jan 23, 2019 12:45 am Post subject: