| View previous topic :: View next topic |
| Author |
Message |
ExecutorElassus
Veteran
Joined: 11 Mar 2004
Posts: 1445
Location: Berlin, Germany
|
 Posted: Tue Jun 11, 2024 6:27 pm Post subject: Profile 23 migration: at what point is it safe to reboot? |
 |
|
I'm fine with following all the instructions given here. However, a quick question: the last item on the list is to 'emerge --ask --emptytree @world'. Is there any point before that when it is not safe to reboot the system? For example, usually when I update the portage tree the first thing I do is update the kernel and then reboot. After performing all the steps in those update instructions prior to the last step, can I still update the kernel, reboot, and then proceed with updating all the rest of @world? Or are there essential packages I need to update first that aren't already covered (for example, gcc, glibc, and libtool are already mentioned).
Cheers,
EE |
|
| Back to top |
|
 |
Banana
Moderator
Joined: 21 May 2004
Posts: 1484
Location: Germany
|
|
| Back to top |
|
|
ExecutorElassus
Veteran
Joined: 11 Mar 2004
Posts: 1445
Location: Berlin, Germany
|
| Posted: Tue Jun 11, 2024 8:19 pm Post subject: |
|
|
| OK, so I'll leave it up until I finish Step 16. Thanks. |
|
| Back to top |
|
|
ExecutorElassus
Veteran
Joined: 11 Mar 2004
Posts: 1445
Location: Berlin, Germany
|
| Posted: Thu Jun 13, 2024 6:59 am Post subject: |
|
|
welp, 24 hours later and I smacked into the same problem I've had for a while now -- emerging qtwebengine-6* ate up all the memory, froze the system, and forced a hard reboot -- and now I have the following problem: there were 60 packages left to go on 'emerge --emptytree @world' when the system froze, but now after reboot 'emerge -puD world' shows far fewer. So there are packages that won't get rebuilt, but I don't want to re-run the whole '--emptytree' rebuild again.
Is there a way to figure out which packages still need rebuilding? And while it'll probably be a separate post, is there a way to emerge qtwebengine that doesn't eat up all my memory and freeze the system? I have 32GB of RAM and 6GB of swap, and it's apparently not enough.
Cheers,
EE |
|
| Back to top |
|
|
NichtDerHans
Apprentice
Joined: 27 Jan 2023
Posts: 150
|
| Posted: Thu Jun 13, 2024 7:05 am Post subject: |
|
|
My ideas:
- Make an .env with less jobs for qtwebengine (takes longer of course  )
- emerge --resume
| Quote: | Resume emerge
If an emerge of several packages is interrupted (e.g. ctrl+c, crash...), the emerge may be resumed from the failed package with the --resume option. The --keep-going and --skipfirst options may also be of interest. See the emerge man page for details. |
https://wiki.gentoo.org/wiki/Emerge
Hope this helps. 
Edit: If you have already tried "emerge -puD world", --resume will probably no longer work for the --emptytree.  |
|
| Back to top |
|
|
ExecutorElassus
Veteran
Joined: 11 Mar 2004
Posts: 1445
Location: Berlin, Germany
|
| Posted: Thu Jun 13, 2024 7:12 am Post subject: |
|
|
| Quote: | | Make an .env with less jobs for qtwebengine (takes longer of course ) |
Uh … so how do I do that? I figured I'd probably want to make some custom setting to throttle qtwebengine, but I've never done that before.
Cheers,
ETA: I checked out this page, but the instructions are unclear. Do I create the file /etc/portage/env/makeopts-jobs-10.conf, and then add just the line
| Code: | | dev-qt/qtwebengine makeopts-jobs-10.conf |
to /etc/portage/package.env? |
|
| Back to top |
|
|
sdauth
l33t
Joined: 19 Sep 2018
Posts: 601
Location: Ásgarðr
|
| Posted: Thu Jun 13, 2024 8:56 am Post subject: |
|
|
It is correct I think, on one of my machine with only 4GB of RAM, I use this to emerge gcc.
/etc/portage/env/cores
(note I use a folder for package.env)
/etc/portage/package.env/gcc
| Code: | | sys-devel/gcc cores |
In your case, it could be useful so only qtwebengine is throttled.
Oherwise, without reducing the cores used, you can also pass --jobs 1 to your emerge command to bypass whatever number of concurrent emerge jobs you set in make.conf.
I suspect you have enough RAM to compile qtwebengine but it fails because your jobs (parallel emerge) value is too high. In 99% of cases, it doesn't cause any issue but during a world rebuild, you can have two or more ram eater packages trying to compile simultaneously with a too high --jobs value.
Last edited by sdauth on Thu Jun 13, 2024 9:57 am; edited 1 time in total |
|
| Back to top |
|
|
logrusx
Veteran
Joined: 22 Feb 2018
Posts: 1757
|
| Posted: Thu Jun 13, 2024 9:21 am Post subject: |
|
|
That might be a annoying detail but it's not throttling. If it was throttling, you could run your CPU at its lowest possible clock, but that would still eat up all memory.
I hate people and packages who incorporate chromium in them. That may make for up to 4 chromiums on your system: qtwebengine:5 and :6, nodejs and chromium istself.
Another thing you may do is disable hyper-threading altogether:
| Code: | | echo off > /sys/devices/system/cpu/smt/control |
The general rule is you need at least 2GB of RAM per thread. The fewer the threads, the more likely is they'll coincide at their peak memory usage, that's my observation from when I run 2-core/2-thread and 4-core-4/threads CPU's not so long ago. When I switched to a 16thread CPU with 32Gb of RAM I virtually have never hit an out of memory condition, despite the integrated video taking 4GB for itself. The ratio is even lower than 2GB per thread, but they are just more concurrent jobs that do not have peak memory consumption during their whole lifetime, so they are less likely to consume all the memory at the same time. Also be careful not to run parallel emerge jobs. If you have that in you command or emerge default opts, remove it and add --jobs= to your emerges you want to run in parallel.
One thing that may aid you in some degree is zram. When I run those 2 and 4 thread CPU's I used to setup a swap device into a ZRAM device so I can prevent OOM's and in the same time not slow down the compilation too much, so this is an option as well. Everything depends on your setup and if you share what your hardware is, you can get better suggestions how to deal with those.
Best Regards,
Georgi |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20117
|
| Posted: Thu Jun 13, 2024 5:22 pm Post subject: |
|
|
| ExecutorElassus wrote: | | Is there a way to figure out which packages still need rebuilding? |
This post has been referenced to reconstruct a list, but requires app-portage/eix.
Another option may be to reference the emerge --resume history file. Per 'man emerge' the file is located at /var/cache/edb/mtimedb.
That's a json text file, but with package information from the "resume_backup" section and what did get emerged (form emerge.log) you may be able to reconstruct a list of what didn't finish. It depends on what the "resume_backup" section contains.
Or you may be able to simply look at what packages are installed on the system and compare that to the packages that were successfully installed (emerge.log).
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
|
Portage & Programming
