| View previous topic :: View next topic |
| Author |
Message |
RIA77
Guru
Joined: 24 Feb 2016
Posts: 396
|
 Posted: Thu May 04, 2023 1:38 pm Post subject: Ryzen 5500, help me to improve emerge times. |
 |
|
Hello and thank you in advance.
I have been using multilib profile
| Code: | # These settings were set by the catalyst build script that automatically
# built this stage.
# Please consult /usr/share/portage/config/make.conf.example for a more
# detailed example.
COMMON_FLAGS="-O3 -march=native -pipe"
ACCEPT_KEYWORDS="~amd64"
FEATURES="candy fixlafiles unmerge-orphans parallel-install"
CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
MAKEOPTS="--jobs 12 --load-average 13"
EMERGE_DEFAULT_OPTS="--jobs 18"
VIDEO_CARDS="radeon radeonsi amdgpu"
# NOTE: This stage was built with the bindist Use flag enabled
PORTDIR="/var/db/repos/gentoo"
DISTDIR="/var/cache/distfiles"
PKGDIR="/var/cache/binpkgs"
USE="-systemd -intel -pulseaudio -printsupport -bluetooth -kde -gnome -cd -dvd -bluetooth -pipewire -qt5 -gnome -qt4 -clamav -cups upower tkip lm-sensors elogind networkmanager X"
# This sets the language of build output to English.
# Please keep this setting intact when reporting bugs.
LC_MESSAGES=C
ACCEPT_LICENSE="* -@EULA"
GENTOO_MIRRORS="http://tux.rainside.sk/gentoo/ \
http://mirror.wheel.sk/gentoo \
http://ftp.romnet.org/gentoo/"
|
I have 16 gigs of ram.
| Code: |
tmpfs /var/tmp/portage tmpfs size=12G,uid=portage,gid=portage,mode=775,nosuid,noatime,nodev 0 0
mount /var/tmp/portage |
I have one problem, when I used cpu frequency as performance, conservative, ondemand, it seems that nothing happens. Cpu temperature is 40 celsius.
But when I have been using schedutil, it goes to 60.
Should I use schedutil to improve emerge time, and why performance governor is not rising cpu temperature ?
| Code: | Sun Jan 22 14:33:32 2023 >>> sys-devel/llvm-15.0.7
merge time: 31 minutes and 33 seconds.
Thu Mar 23 13:21:56 2023 >>> sys-devel/llvm-16.0.0
merge time: 42 minutes and 53 seconds.
Sat Apr 8 13:19:31 2023 >>> sys-devel/llvm-16.0.1
merge time: 45 minutes and 24 seconds.
Sat Apr 22 13:16:03 2023 >>> sys-devel/llvm-16.0.2
merge time: 42 minutes and 34 seconds.
Thu May 4 14:27:43 2023 >>> sys-devel/llvm-16.0.3
merge time: 55 minutes and 26 seconds.
|
Fastest is to use nvme drive to build, but when I mount tmpfs, this is the slowest build ever.
How to save nvme drive and build in RAM and speed up emerge time ? |
|
| Back to top |
|
 |
alamahant
Advocate
Joined: 23 Mar 2019
Posts: 3925
|
| Posted: Thu May 04, 2023 1:54 pm Post subject: |
|
|
You mount the tmpfs but you are not using it
You have
| Code: |
PORTDIR="/var/db/repos/gentoo"
DISTDIR="/var/cache/distfiles"
PKGDIR="/var/cache/binpkgs"
|
You need to add in make.conf
| Code: |
PORTAGE_TMPDIR=/var/tmp/portage
|
_________________
 |
|
| Back to top |
|
|
Banana
Moderator
Joined: 21 May 2004
Posts: 1792
Location: Germany
|
| Posted: Thu May 04, 2023 2:02 pm Post subject: |
|
|
Are you sure you want to use -O3 ? https://wiki.gentoo.org/wiki/GCC_optimization#-O
| Quote: | | MAKEOPTS="--jobs 12 --load-average 13" |
This could lead to problems as well. Your CPU has 12 "cores" but using them all, does not really help everytime. Using a lower number could reduce your build times.
| Code: | | EMERGE_DEFAULT_OPTS="--jobs 18" |
try to reduce this number as well
try to play with those numbers to get a good result, but keep in mind that using the highest may not result in the best performance.
_________________
Forum Guidelines
PFL - Portage file list - find which package a file or command belongs to.
My delta-labs.org snippets do expire
Last edited by Banana on Sat May 06, 2023 7:49 pm; edited 1 time in total |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54629
Location: 56N 3W
|
| Posted: Thu May 04, 2023 8:30 pm Post subject: |
|
|
RIA77,
| Quote: | | I have 16 gigs of ram. |
| Code: | MAKEOPTS="--jobs 12 --load-average 13"
EMERGE_DEFAULT_OPTS="--jobs 18" |
You don't have enough RAM for those options.
MAKEOPTS="--jobs 12 can take 24G RAM per parallel package being built and you allow EMERGE_DEFAULT_OPTS="--jobs 18" 18 of them.
That's gcc using 432G RAM before you allocate any for /var/tmp/portage in tmpfs.
The poor wee thing will be swapping.
With 16G RAM, the kernel will want some, so at most MAKEOPTS="--jobs 7 is safe together with EMERGE_DEFAULT_OPTS="--jobs 1"
That allows gcc to use 14G RAM.
Swapping does not mean using a swap partition. The content of any RAM that has a permanent home on HDD can be discarded and reloaded.
That's a form of swapping too. It takes time and adds nothing to the build process.
is generally a bad thing. It enables optimisations that make the output code bigger.
If the bigger code (working set) is too big for the CPU cache, the some code is ejected from the cache to be able to execute other but related code.
This results in 'cache thrashing' while code is continually refetched from much slower main memory.
You have a CPU with 12 threads. However, gcc and 12 threads requires 24G real RAM, just for gcc.
Its not all gloom and doom. That 2G per thread is for big C++ packages.
Start at | Code: | MAKEOPTS="--jobs 7"
EMERGE_DEFAULT_OPTS="--jobs 1" |
and play with bigger numbers of | Code: | | MAKEOPTS="--jobs X" |
up to X=12.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
RIA77
Guru
Joined: 24 Feb 2016
Posts: 396
|
| Posted: Thu May 04, 2023 9:29 pm Post subject: |
|
|
Thank you to all of you.
What about "load average" ? |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54629
Location: 56N 3W
|
| Posted: Thu May 04, 2023 9:37 pm Post subject: |
|
|
RIA77,
Not all build systems respect load average. Personally, I don't use it.
I still see the load average double what it can theoretically reach, based on --jobs= and MAKEOPTS=
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2185
|
| Posted: Fri May 05, 2023 10:02 am Post subject: |
|
|
I've recently added --load-average=31 to my MAKEOPTS on my 5750x setup (16 processors with hyperthreading = nominal 32 threads) and found that when emerge decided in its wisdom to compile firefox and libreoffice at the same time, firefox got just one thread (probably stalled, I'm not an expert at reading top) and libreoffice got the rest.
I also have an ENV file override of "--load-average=15" for qtwebengine, so my 32GB machine doesn't run out of space. Last time emerge included qtwebengine in its updates I noticed it maxed out around 24GB memory usage (this compilation with LLVM/clanag, which is rumoured to need less RAM), and the emerge ran satisfactorily.
_________________
Greybeard |
|
| Back to top |
|
|
|
Kernel & Hardware
