Stage 1 + 3 install: High sys time

[SphereII]

Good day.

At work, we've decided to update our internal build system, and are now bringing over our internal Gentoo build as well.

Our existing system is a dual processor, with 4G of RAM virtual machine, running on an ESX blade cluster. Currently configured, it takes roughly 400 minutes to perform a build of Stage 1 and Stage 3. We set emerge jobs to 5, with MAKEOPTS set to 1. We are also using ccache as a FEATURES.

On our new system, we have three build machines, each with 8 processors (2.33ghz each), with 4 to 8G of RAM. This has typically allowed us to reduce our build times by 1/3 to 2/3rds on all our other software packages. However, in the case of our Gentoo build, we are seeing an average of 700 minutes for the same build to be done, using the same scripts and same settings. We've taken steps to try to improve this, by setting up a RAM disk on /var/tmp, which has improved build times slightly, but not nearly as much as anticipated. We've increased and decreased the number of jobs we perform, as well as play around with the MAKEOPTS setting, with not much of an improvement. Oddly enough, decreasing the number of processors has decreased build times.

While monitoring through top, we see a very high sys time. For the 700 minute build, we could be in sys time for 2300 minutes. Is this normal for Gentoo Stage 1 and 2?

For a test, I did a strace on emerge glibc, and this is the results I get.