intel_clc uses infinite memory compiling media-libs/mesa

[jharvell]

When I emerge media-libs/mesa-24.2.1 with VIDEO_CARDS=intel in /etc/portage/make.conf, it eats up all the system memory with /usr/bin/intel_clc processes. My system has 32 GiB RAM and normally has 64 GiB swap on a swap partition on an NVMe disk. In order to investigate this problem further, I also added a 1 TiB external USB disk as a second swap device. I'm running with MAKEOPTS="--jobs=7", so there are 7 intel_clc processes.

Here's an excerpt from /proc/<pid>/status for one of them:

VmPeak: 20455092 kB
VmSize: 20455092 kB
VmLck: 0 kB
VmPin: 0 kB
VmHWM: 4491320 kB
VmRSS: 4240568 kB
RssAnon: 4238044 kB
RssFile: 2524 kB
RssShmem: 0 kB
VmData: 20178824 kB
VmStk: 148 kB
VmExe: 5312 kB
VmLib: 141436 kB
VmPTE: 39916 kB
VmSwap: 15945984 kB
HugetlbPages: 0 kB


At this point VmSize is about 20 GiB, and it's the same on the other 6 of them. The VmSize just keeps rising steadily on all of them. The max I've seen them is all at about 40 GiB before giving up and terminating the emerge.

/usr/bin/intel_clc is from package dev-utils/intel_clc. But it's built from the same mesa source code, and emerge forces them to be the same version. I disabled LTO for both dev-util/intel_clc and media-libs/mesa and it doesn't make a difference. I am compiling with -Ofast, so I'll try -O3 and -O2 just in case.

I found a bug report for intel_clc leaking memory. That bug has been fixed, and I've verified it's in the versions of dev-util/intel_clc and media-libs/mesa that I'm building. I even looked in /var/tmp/portage while they were building and inspected the source code manually to confirm that fix really is there.

This is an emerge of a new system. So I'm stuck with an unusable system unless I unset VIDEO_CARDS. But eventually I'll have to fix this if I want to use X/KDE.

[pietinger]

[jharvell]

Thanks for the tip.

I did an emege -a1v =dev-utils/intel_clc-24.1.6 =media-libs/mesa-24.1.6. That successfully installed intel_clc, and while it was building mesa I saw from htop that the intel_clc processs I had previously seen eating up memory ran and finished. I didn't let mesa-24.1.6 finish since I don't really need to rebuild the version that's already there from stage3.

I masked mesa and intel_clc versions greater than 24.1.6 and emerge -avuDN @world resolved all dependencies and is building. If I later run into dependency issues due to that package mask I'll have to deal with it then.

Thanks again.

[jharvell]

I replaced -Ofast in /etc/porage/make.conf with -O3, then I rebuilt the entire system with emerge -eav @world. After that finished, I removed the package masks '>dev-utils/intel_clc-24.1.6 and >media-libs/mesa-24.1.6' then ran emerge -avuDN @world. The result was dev-utils/intel_clc-24.2.1 and media-libs/mesa-24.2.1 were successfully installed with no excessive memory usage during build.

[Hu]

-Ofast is known to break otherwise compliant programs, so it should only be used on programs that are specifically developed to work correctly under the rules it applies.