Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Ryzen 5500, help me to improve emerge times.
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
RIA77
Guru
Guru


Joined: 24 Feb 2016
Posts: 399

PostPosted: Thu May 04, 2023 1:38 pm    Post subject: Ryzen 5500, help me to improve emerge times. Reply with quote

Hello and thank you in advance.

I have been using multilib profile

Code:
# These settings were set by the catalyst build script that automatically
# built this stage.
# Please consult /usr/share/portage/config/make.conf.example for a more
# detailed example.
COMMON_FLAGS="-O3 -march=native -pipe"
ACCEPT_KEYWORDS="~amd64"
FEATURES="candy fixlafiles unmerge-orphans parallel-install"
CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
MAKEOPTS="--jobs 12 --load-average 13"
EMERGE_DEFAULT_OPTS="--jobs 18"
VIDEO_CARDS="radeon radeonsi amdgpu"
# NOTE: This stage was built with the bindist Use flag enabled
PORTDIR="/var/db/repos/gentoo"
DISTDIR="/var/cache/distfiles"
PKGDIR="/var/cache/binpkgs"
USE="-systemd -intel -pulseaudio -printsupport -bluetooth -kde -gnome -cd -dvd -bluetooth -pipewire -qt5 -gnome -qt4 -clamav -cups upower tkip lm-sensors elogind networkmanager X"
# This sets the language of build output to English.
# Please keep this setting intact when reporting bugs.
LC_MESSAGES=C
ACCEPT_LICENSE="* -@EULA"

GENTOO_MIRRORS="http://tux.rainside.sk/gentoo/ \
    http://mirror.wheel.sk/gentoo \
    http://ftp.romnet.org/gentoo/"

I have 16 gigs of ram.

Code:

tmpfs           /var/tmp/portage                tmpfs   size=12G,uid=portage,gid=portage,mode=775,nosuid,noatime,nodev  0 0

mount /var/tmp/portage


I have one problem, when I used cpu frequency as performance, conservative, ondemand, it seems that nothing happens. Cpu temperature is 40 celsius.
But when I have been using schedutil, it goes to 60.
Should I use schedutil to improve emerge time, and why performance governor is not rising cpu temperature ?


Code:
Sun Jan 22 14:33:32 2023 >>> sys-devel/llvm-15.0.7
       merge time: 31 minutes and 33 seconds.

     Thu Mar 23 13:21:56 2023 >>> sys-devel/llvm-16.0.0
       merge time: 42 minutes and 53 seconds.

     Sat Apr  8 13:19:31 2023 >>> sys-devel/llvm-16.0.1
       merge time: 45 minutes and 24 seconds.

     Sat Apr 22 13:16:03 2023 >>> sys-devel/llvm-16.0.2
       merge time: 42 minutes and 34 seconds.

     Thu May  4 14:27:43 2023 >>> sys-devel/llvm-16.0.3
       merge time: 55 minutes and 26 seconds.

Fastest is to use nvme drive to build, but when I mount tmpfs, this is the slowest build ever.
How to save nvme drive and build in RAM and speed up emerge time ?
Back to top
View user's profile Send private message
alamahant
Advocate
Advocate


Joined: 23 Mar 2019
Posts: 3928

PostPosted: Thu May 04, 2023 1:54 pm    Post subject: Reply with quote

You mount the tmpfs but you are not using it
You have
Code:

PORTDIR="/var/db/repos/gentoo"
DISTDIR="/var/cache/distfiles"
PKGDIR="/var/cache/binpkgs"

You need to add in make.conf
Code:

PORTAGE_TMPDIR=/var/tmp/portage

_________________
:)
Back to top
View user's profile Send private message
Banana
Moderator
Moderator


Joined: 21 May 2004
Posts: 1801
Location: Germany

PostPosted: Thu May 04, 2023 2:02 pm    Post subject: Reply with quote

Are you sure you want to use -O3 ? https://wiki.gentoo.org/wiki/GCC_optimization#-O

Quote:
MAKEOPTS="--jobs 12 --load-average 13"

This could lead to problems as well. Your CPU has 12 "cores" but using them all, does not really help everytime. Using a lower number could reduce your build times.

Code:
EMERGE_DEFAULT_OPTS="--jobs 18"

try to reduce this number as well

try to play with those numbers to get a good result, but keep in mind that using the highest may not result in the best performance.
_________________
Forum Guidelines

PFL - Portage file list - find which package a file or command belongs to.
My delta-labs.org snippets do expire


Last edited by Banana on Sat May 06, 2023 7:49 pm; edited 1 time in total
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54643
Location: 56N 3W

PostPosted: Thu May 04, 2023 8:30 pm    Post subject: Reply with quote

RIA77,


Quote:
I have 16 gigs of ram.

Code:
MAKEOPTS="--jobs 12 --load-average 13"
EMERGE_DEFAULT_OPTS="--jobs 18"


You don't have enough RAM for those options.
MAKEOPTS="--jobs 12 can take 24G RAM per parallel package being built and you allow EMERGE_DEFAULT_OPTS="--jobs 18" 18 of them.
That's gcc using 432G RAM before you allocate any for /var/tmp/portage in tmpfs.
The poor wee thing will be swapping.

With 16G RAM, the kernel will want some, so at most MAKEOPTS="--jobs 7 is safe together with EMERGE_DEFAULT_OPTS="--jobs 1"
That allows gcc to use 14G RAM.

Swapping does not mean using a swap partition. The content of any RAM that has a permanent home on HDD can be discarded and reloaded.
That's a form of swapping too. It takes time and adds nothing to the build process.

Code:
COMMON_FLAGS="-O3
is generally a bad thing. It enables optimisations that make the output code bigger.
If the bigger code (working set) is too big for the CPU cache, the some code is ejected from the cache to be able to execute other but related code.
This results in 'cache thrashing' while code is continually refetched from much slower main memory.

You have a CPU with 12 threads. However, gcc and 12 threads requires 24G real RAM, just for gcc.

Its not all gloom and doom. That 2G per thread is for big C++ packages.

Start at
Code:
MAKEOPTS="--jobs 7"
EMERGE_DEFAULT_OPTS="--jobs 1"
and play with bigger numbers of
Code:
MAKEOPTS="--jobs X"
up to X=12.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
RIA77
Guru
Guru


Joined: 24 Feb 2016
Posts: 399

PostPosted: Thu May 04, 2023 9:29 pm    Post subject: Reply with quote

Thank you to all of you.
What about "load average" ?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54643
Location: 56N 3W

PostPosted: Thu May 04, 2023 9:37 pm    Post subject: Reply with quote

RIA77,

Not all build systems respect load average. Personally, I don't use it.
I still see the load average double what it can theoretically reach, based on --jobs= and MAKEOPTS=
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Goverp
Advocate
Advocate


Joined: 07 Mar 2007
Posts: 2187

PostPosted: Fri May 05, 2023 10:02 am    Post subject: Reply with quote

I've recently added --load-average=31 to my MAKEOPTS on my 5750x setup (16 processors with hyperthreading = nominal 32 threads) and found that when emerge decided in its wisdom to compile firefox and libreoffice at the same time, firefox got just one thread (probably stalled, I'm not an expert at reading top) and libreoffice got the rest.

I also have an ENV file override of "--load-average=15" for qtwebengine, so my 32GB machine doesn't run out of space. Last time emerge included qtwebengine in its updates I noticed it maxed out around 24GB memory usage (this compilation with LLVM/clanag, which is rumoured to need less RAM), and the emerge ran satisfactorily.
_________________
Greybeard
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum