| View previous topic :: View next topic |
| Author |
Message |
pietinger
Moderator
Joined: 17 Oct 2006
Posts: 5419
Location: Bavaria
|
 Posted: Sat Feb 22, 2025 2:12 pm Post subject: emerge uses more CPU cores (threads) than permitted |
 |
|
... and only at the start of the compilation phase.
I am posting here because I am not sure if this is a bug and needs a bug report. But I suspect a bug.
I usually do my word update at night (unattended) so I don't know when this behavior started. Today I could observe (with btop) that during today's word-update almost all packages after the configuration phase started the compilation phase immediately with 32 threads and after these were “processed”, portage “only” started the allowed 24 threads. My setting in make.conf is very simple:
| Code: | RUSTFLAGS="${RUSTFLAGS} -C target-cpu=native"
MAKEOPTS="-j24 |
Some packages (e.g. qtwebengine) even had this behavior several times: sometimes 32 threads were started, then 24 and suddenly 32 again in the middle. Could this be due to a missing/incorrect setting?
_________________
https://wiki.gentoo.org/wiki/User:Pietinger |
|
| Back to top |
|
 |
Hu
Administrator
Joined: 06 Mar 2007
Posts: 23126
|
| Posted: Sat Feb 22, 2025 2:24 pm Post subject: |
|
|
I have seen reports like this before. It is unfortunate, but it is not a Portage bug. When you set MAKEOPTS, you tell GNU Make (and certain other build systems that, for compatibility, try to understand at least some GNU Make options) how many jobs it can start. Some build systems just guess on their own how many parallel jobs they should start. For example, dev-build/ninja states in its documentation:That page does indicate that explicitly passing -j overrides the default of guessing the number of allowed threads.
I seem to recall other build systems that were even less well behaved, but I cannot recall names now.
A full solution may require that emerge direct the kernel to limit the resources allowed to the individual packages, such as through cgroup settings. |
|
| Back to top |
|
|
pietinger
Moderator
Joined: 17 Oct 2006
Posts: 5419
Location: Bavaria
|
| Posted: Sat Feb 22, 2025 3:13 pm Post subject: |
|
|
Hu, thank you very much for your (detailed) answer. If I understand correctly, I only have two options: Either there is the possibility to give NINJA this parameter (for each run), or it is only possible via the cgroups? (otherwise I have to live with the behavior)
_________________
https://wiki.gentoo.org/wiki/User:Pietinger |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
