make's --load-average not honored?

Zucca

So. I had set MAKEOPTS="--jobs=8 --load-average=7.80" and Portage was compiling nodejs and rust at the same time (checked that via qlop -r). But still my load average was 16.00.

I went to investigate this a bit and saw that rust didn't use make but nodejs did. I would then assume the nodejs compiling process to slow down (to halt basically) because the rust compiling process would create a load of around 8.

I'm certain that before --load-average has been honored. But for some moths, at least, this hasn't been the case, I think. The last @world update really caught my eye at load average.

I've now loosened my MAKEOPTS (to avoid such high loads) and it won't match what I stated at the beginning. Otherwise the following output is the same.

pingtoo · Posted: Tue Sep 19, 2023 6:14 pm Post subject:

May be your are thinking something else.

the MAKEOPTS is only used for per make job, so in your example each of nodejs job and rust job have limit set to concurrently run up to 8 tasks while system load average below 7.8.

So while system load is above 7.8 I think, each make job will not submit more tasks.

May be you are thinking using "EMERGE_DEFAULT_OPTS=--jobs=N --load-average-X.Y". Please see Gentoo wiki EMERGE_DEFAULT_OPTS

Zucca · Posted: Tue Sep 19, 2023 7:21 pm Post subject:

Hu · Administrator Joined: 06 Mar 2007 Posts: 22626

As a minor point, per info make, it looks like --load-average can never cause Make to drop to 0 jobs. It will always run at least 1, no matter how bad the load:

pingtoo · Posted: Tue Sep 19, 2023 8:32 pm Post subject:

I too observe similar pattern when I update with my RPI 4B with a distcc farm of 10 helper nodes.

However I did notice in my case the job count will not go above only when the load average persist for long time (said more than a few minutes). Also my load average refresh rate is a 10 seconds so I don't know if in between the 10 seconds may be somehow the load were drop to below the constraint.

Genone · Posted: Thu Sep 21, 2023 1:13 pm Post subject:

--load-avg is more of a suggestion than a hard limit. Also it only applies to spawning new make processes, so if make is launching other programs and those fork into multiple processes, that is not affected by the initial make options (if other tools launch multiple make processes, that is a different story). Also the documentation doesn't specify which load-avg is used, e.g. if it is using the 15 minutes average it will of course take quite a while before the limit is reached.

Btw, gnu make did apparently change the implementation for --load-avg in the most recent version 4.4: