[solved] icu rebuild of firefox failure

pjp · Administrator Joined: 16 Apr 2002 Posts: 20542

EDIT: The second rebuild succeeded, so probably hardware related.

This is on a build host in a chroot for a laptop.

I first did a rebuild of whatever icu wanted with firefox deselected from world (to avoid rebuilding it and nodejs at the same time).

After that, I then rebuilt nodejs using --oneshot. That went fine. To rebuild firefox, I used my normal update @world command.

This seemed notable:

eccerr0r · Posted: Fri Dec 20, 2024 8:21 am Post subject:

Well, assuming the fact other people were able to build the same firefox and looks like clang crashed with a stack trace, something going on with your clang. Does it crash in the same place each time if you just rerun it (save your old build for reference) - checked for hardware issues?

Is there a specific reason why you compiled icu and nodejs separately, though it shouldn't matter, they may pick up older deps that weren't absolutely necessary. Agreed it should be safe. I don't think I've had issues building nodejs with anything else at the same time, it's the qtwebengine thing that needs to run by itself...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

pjp · Administrator Joined: 16 Apr 2002 Posts: 20542

I'm trying to rebuild it again now. I'm not sure how long it took before failure.

It indeed could be hardware. I have suspected possible heat issues in the past, but haven't ruled out memory. In the past, I've only seen "internal compiler error", but maybe that was only from gcc and not clang.

Reducing from -j3 to -j1 for bigger packages has helped. gcc, clang, firefox and nodejs off the top of my head. I've been using -j2 for firefox for a while without issues, so it seemed odd that it happened on a rebuild due to icu change and not a new firefox version.

The possible heat issue is why I try to limit how many large packages get built at any given time. From the output of which packages icu was causing a rebuild, they've all been rebuilt except firefox. I only mentioned just in case.

I'll update when it fails or completes. I'll see about scheduling a memtest. I guess I could do that overnight sometime.
_________________
Quis separabit? Quo animo?

eccerr0r · Posted: Fri Dec 20, 2024 6:40 pm Post subject:

Since you're chrooting, I assume the machine you're using to build is a desktop or larger system? If so it's worth to resolve overheating issues...I'd be kind of upset if I can't use all cores on a cpu full tilt on a desktop -- Your X4 has 16GB RAM and that would make me mad if I couldn't run all 4 cores at the same time. My i7-2700k is actually on the borderline, it reaches 90°C when running on all 4 cores 8 threads but that's fine, at least it doesn't crash or give ICEs.

Yeah Internal Compiler Error is a gcc thing. Clang gives a more useful stack trace it appears, but I can't make heads or tails out of it.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

logrusx · Advocate Joined: 22 Feb 2018 Posts: 2541

I just compiled Firefox-133.03 today, but I'm using the binhost for the most of my packages, so I believe clnag/llvm were compiled on the binhost. Can you try those?

Best Regards,
Georgi

p.s. out there some say 139 might be concurrent access to a file or null pointer. IDK if those statements have merit. Did you check your dmesg for segfaults?

pjp · Administrator Joined: 16 Apr 2002 Posts: 20542

pjp · Administrator Joined: 16 Apr 2002 Posts: 20542

logrusx · Advocate Joined: 22 Feb 2018 Posts: 2541

Hu · Administrator Joined: 06 Mar 2007 Posts: 22892

The emptytree rebuild is recommended for various good reasons (discussed back when this was new), but it's not required if you don't mind not getting the benefits it was meant to bring. As with so many things, it depends on what you value most. I migrated one special purpose system to 23.0 without an emptytree rebuild, because I decided that the savings in write cycles and install time mattered more to me than the technical improvements the new profile brought. The system was sufficiently isolated that the security improvements in 23.0 did not seem immediately necessary for it. That system has gradually picked up the 23.0 improvements as ordinary maintenance prompted package builds to get newer versions of things. I didn't actively disable anything that 23.0 brings; I only deferred picking up its improvements until they were "free" as part of other work. I may have encountered some problems that an emptytree would have avoided, but if so, I fixed them readily enough that they do not come to mind now.

pjp · Administrator Joined: 16 Apr 2002 Posts: 20542

pjp · Administrator Joined: 16 Apr 2002 Posts: 20542

sam_ · Developer Joined: 14 Aug 2020 Posts: 2047

FWIW, I did write that up in the end on the wiki at https://wiki.gentoo.org/wiki/On_occasional_emptytree. We also had a few real bug reports (IIRC initially manifesting as users seeking support) where they immediately showed up because of the -e and the changes in the profile, and it was then obvious what the problem/cause was, so I feel a bit vindicated.

(It was BIND_NOW-related breakage, for the interested.)

pjp · Administrator Joined: 16 Apr 2002 Posts: 20542

Yes, that's what I was remembering.

In retrospect, an occasional emptytree makes sense, it's just unfortunate for old hardware.
_________________
Quis separabit? Quo animo?

logrusx · Advocate Joined: 22 Feb 2018 Posts: 2541

eccerr0r · Posted: Sat Dec 21, 2024 1:04 am Post subject:

I would not emptytree just yet. Since it worked the second time around, the binary on disk is intact and you should be good - the corruption occurred in RAM/CPU.

However once you get verified good hardware or fix your hardware, then emptytree would be suggested just to make sure there are no bad binaries around that haven't been checked.

I've not had too many issues with corruption recently, knock on wood. Had to clear this out of my system when I was overclocking and bought unknown and bad RAM that I assumed was good, so each time now I have to go ahead and do validation cycles on any hardware I acquire before committing it to production, even if it's for home use. The recent fubar with intel probably would have counted towards more corruption cases...

I do not have end-to-end ECC/parity hardware so there's a nonzero chance I still have SDC...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

pjp · Administrator Joined: 16 Apr 2002 Posts: 20542

@logrusx,

I tried early on, but there were issues with some security based gcc flags. I removed those back then. I've been wanting to do it for a while, but I have to be prepared to deal ONLY with that if anything happens. I'll get ready this weekend and may start early next week. I'll probably run memtest first and see what replacement RAM would cost (if I can even get it locally). Assuming that's the underlying problem.

@eccerr0r,

emptytree is only for the migration to the 23 profile. I may not be able to do anything about the hardware, depending on what the issue is.

I haven't seen anything about Intel's latest issues.
_________________
Quis separabit? Quo animo?

eccerr0r · Posted: Sat Dec 21, 2024 1:13 am Post subject:

If you're overclocking or even not, you can also try slowing down timing of your components in BIOS if the option is available.

Granted yes most big box computers don't have these options so that's out of the question, sometimes I feel glad with my self-assembled machines that do have these options just so I could tweak them to see if I can edge out marginal hardware and get some use out of them.

(And if it's some known bad RAM locations, it's possible to tell Linux to not use them, so that's also an option. I've been using this frequently though recently I didn't need it anymore and memtest86+ tests fine now...)

Oh... and I didn't emptytree the 23 profile migration on any of my boxes (yet or ever)... so far so good, haven't seen any weird behavior.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

pjp · Administrator Joined: 16 Apr 2002 Posts: 20542

It's a custom build, but I don't overclock. The presumably marginal gains never seemed worth the potential downside for a mainstay system.

I thought about marking the bad RAM locations when I mentioned memtest earlier, so if that's the issue, hopefully that helps. I should have thought of that years ago.
_________________
Quis separabit? Quo animo?