Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
PC configuration
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
apiaio
Guru
Guru


Joined: 04 Dec 2008
Posts: 417

PostPosted: Sat Aug 27, 2022 5:21 pm    Post subject: PC configuration Reply with quote

I am going to purchase new PC. Probably the last one. I am not youngest.

Presently I have PC with i7 processor - 8 threads, 16 GB RAM and MB Z-170M-PLUS.
I have picked out
INTEL Core i7-12700K Alder Lake (25M Cache, 3,8/4,9 GHz), Procesor (BX8071512700K) - 20 threads, 32 GB RAM and MB ASROCK B660M-HDV

What I expect is shorter compilation time of the packages like e.g. qtwebengine. In the present compilation qtwebengine takes
some 2 hours and 15 minutes. I use SSD and new PC will have SSD installed also.

I will change
Code:
MAKEOPTS="-j9"
to
Code:
MAKEOPTS="-j17"
in make.conf
and
Code:
tmpfs                   /var/tmp/portage tmpfs  size=32G,uid=portage,gid=portage,mode=775,noatime 0 0
in /etc/fstab

Concerning of the compilation time what I can expect?
Is the selection of processor, RAM and fast disc main part of good PC configuration for Gentoo?
Should I make any others changes in any configuration files?
Have I omitted something?
Does exist any PC configuration tailored for Gentoo?

Thanks
Back to top
View user's profile Send private message
pietinger
Moderator
Moderator


Joined: 17 Oct 2006
Posts: 4244
Location: Bavaria

PostPosted: Sat Aug 27, 2022 7:35 pm    Post subject: Reply with quote

apiaio,

please keep in mind that every compile job costs memory. How much memory depends on every package. Some packages need not more than 1 GB per job, others until 2 GB per job and I dont want to talk about "rust" ... ;-)

You surely know that you will be faster with compiling if you set a maximum of jobs that there will be no swaping (means: sometimes you are faster with -j8 instead of -j9 or higher).

Now, IF you put /var/tmp/portage also into memory, you need even more memory/job. Again you must prevent swaping.

You will buy a very fast CPU with many (logical) cores, being able to compile many jobs at the same time ... IF there is enough memory. This leads me to my personal recommendation (as user) to buy 64 GB RAM. My suggestion for this would be:

MAKEOPTS="-j20" and /var/tmp/portage tmpfs size=18G

If you will stay with 32 GB RAM you could use a configuration where you make exceptions for some packages, maybe:

MAKEOPTS="-j20" and /var/tmp/portage tmpfs size=12G

and =>

Code:
# mkdir -p /var/tmp/notmpfs
# chown portage:portage /var/tmp/notmpfs
# mkdir /etc/portage/env

# nano -w /etc/portage/env/notmpfs.conf
=>
PORTAGE_TMPDIR="/var/tmp/notmpfs"
<=


# nano -w /etc/portage/env/monster.conf
=>
MAKEOPTS="-j12"
<=


# nano -w /etc/portage/package.env
=>
app-office/libreoffice monster.conf
dev-lang/rust monster.conf  notmpfs.conf
dev-qt/qtwebengine monster.conf
<=

(these settings are rather conservative; maybe you can go higher)

I would guess you will compile qtwebengine in one hour or less ... :-)
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54308
Location: 56N 3W

PostPosted: Sat Aug 27, 2022 9:21 pm    Post subject: Reply with quote

apiaio,

I did that last year. Here's what I chose and why.

That CPU can use 40G RAM at 2G per thread today. You will need a minimum of 64G and two empty slots to fill later.

Do you mean SSD or NVMe?
NVMe can be 10x faster than SSD as its a faster interface.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
apiaio
Guru
Guru


Joined: 04 Dec 2008
Posts: 417

PostPosted: Sun Aug 28, 2022 8:03 am    Post subject: Reply with quote

Thanks for answers.

Why should I buy 64G RAM, when chosen CPU can use 40G RAM only? 24G will be useless.
Back to top
View user's profile Send private message
Spanik
l33t
l33t


Joined: 12 Dec 2003
Posts: 949
Location: Belgium

PostPosted: Sun Aug 28, 2022 8:27 am    Post subject: Reply with quote

Interesting thread. But there are a few things I do not really get. I got this setup and thought it would be faster then the previous 10 years old one. But it isn't. An update @world can still need a night to complete.
- Epyc 7402P 24 cores, 48 threads
- 64GB ram (4x16GB) each on it own memory channel
- nvme Samsung 970 EVO pcie 3 x4
- Supermicro H11SSL-i all pcie lanes are separate straight to the cpu
- makeopts = "-j25"

pietinger wrote:
You surely know that you will be faster with compiling if you set a maximum of jobs that there will be no swaping (means: sometimes you are faster with -j8 instead of -j9 or higher).

Now, IF you put /var/tmp/portage also into memory, you need even more memory/job. Again you must prevent swaping.

You will buy a very fast CPU with many (logical) cores, being able to compile many jobs at the same time ... IF there is enough memory. This leads me to my personal recommendation (as user) to buy 64 GB RAM. My suggestion for this would be:

Compiling still is a mostly linear thing. And indeed Libreoffice took ages. Rust was another issue and I went with the .bin. From what I see when looking at the system monitor there is a lot of swapping between cores going on and the cpu isn't the limiting factor. I never see the used memory go even half of what is available. Now I can add another 64GB and use the 4 remaining memory channels as well. But it is a heavy investment that wouldn't make much sense for the other use of the pc.
_________________
Expert in non-working solutions
Back to top
View user's profile Send private message
pietinger
Moderator
Moderator


Joined: 17 Oct 2006
Posts: 4244
Location: Bavaria

PostPosted: Sun Aug 28, 2022 9:29 am    Post subject: Reply with quote

apiaio wrote:
Why should I buy 64G RAM, when chosen CPU can use 40G RAM only? 24G will be useless.

This is a misunderstanding. Neddy took your 20 cores and multiplied it with 2 GB because there is a Gentoo recommendation to have 2 GB / job. As I told you most packages dont need so much. If you compile a "small" package needing only 1 GB / job and you use all 20 cores then you would need only 20 GB ram for this emerge. Now, what is with big packages ? Take qtwebengine as example. It needs up to 2 Gb / job. Why you can set -j12 (=> needing 24 GB) AND tmpfs size 12 GB (would be 36 GB and not 32 GB) ? Because when you emerge you have different processes:
1. After unpacking the package into /var/tmp/portage you will fill this directory a little bit;
2. Now you compile source files and get object files - you will fill this dir more and more; at the same time you need your ram for gcc doing its jobs;
3. after all compiling, all object files will be linked together; this will fill your tmp dir even more; at this time you dont need your ram for gcc (only for ldd)

Look at the last step: Here you will need ALL memory in /var/tmp/portage BUT because there is no compiling anymore you can do settings which would go over your ram at the same time.

Now: rust ... :-( Rust needs up to 4 GB per comile job ... So - even with 64 GB ram - it could be possible that -j20 would be to much. 20 jobs * 4 GB would be 80 GB needed ram. So, even if you take 64 GB ram, it is possible that you must do an excemption rule for rust.
Back to top
View user's profile Send private message
pietinger
Moderator
Moderator


Joined: 17 Oct 2006
Posts: 4244
Location: Bavaria

PostPosted: Sun Aug 28, 2022 10:09 am    Post subject: Reply with quote

P.S.: Please dont forget, you will use also other applications than emerge. If you use software which is capable of using all cores (like my "stockfish"; a chess engine) these programs ALSO needs memory for every thread ... ;-)

Yes, often 64 GB is not really needed, but it would be sad if you cant use your powerful CPU because of not enough memory in other moments ... 8)
Back to top
View user's profile Send private message
C5ace
Guru
Guru


Joined: 23 Dec 2013
Posts: 474
Location: Brisbane, Australia

PostPosted: Sun Aug 28, 2022 10:37 am    Post subject: Reply with quote

My main box:

No tempfs. Just a 7200RPM rotating rust drive. Desktop is XFce4. Compiling is done in a Xfce Terminal Emulator.

uname -a:
AMD Ryzen 9 3950X 16-Core Processor with 32 threads and 64Gb Ram.

make.conf:
MAKEOPTS="-j33"

qlop:
2021-11-20T21:32:25 >>> dev-qt/qtwebengine-5.15.2_p20211019: 32′57″
2022-08-17T16:08:14 >>> app-office/libreoffice-7.3.4.2-r1: 16′51″
2022-07-02T11:50:37 >>> dev-lang/rust-1.60.0: 15′24″
2022-03-10T10:46:08 >>> www-client/chromium-99.0.4844.51: 1:24:03
022-08-26T16:58:10 >>> www-client/firefox-91.13.0: 13′26″
2022-07-02T11:24:16 >>> sys-libs/glibc-2.34-r13: 1′59″

Maximum memory usage was about 35Gb. No swapping.

Tried using tmpfs and found practically no change. Probably because the HDD has a 64MB cache, Xfce Desktop only uses about 600Mb RAM. This leaves heaps of free RAM for Gentoo to use as buffer/cache.
_________________
Observation after 30 years working with computers:
All software has known and unknown bugs and vulnerabilities. Especially software written in complex, unstable and object oriented languages such as perl, python, C++, C#, Rust and the likes.
Back to top
View user's profile Send private message
apiaio
Guru
Guru


Joined: 04 Dec 2008
Posts: 417

PostPosted: Sun Aug 28, 2022 10:53 am    Post subject: Reply with quote

@pietinger
Thank you for comprehensive information. It will be 64GB.

NeddySeagoon wrote:
apiaio,
Do you mean SSD or NVMe?
NVMe can be 10x faster than SSD as its a faster interface.

I intend to buy
Quote:
CRUCIAL P1 Int. Disk SSD 500 GB/M.2 2280/M.2 PCI-Express NVMe CT500P1SSD8

and 1 or 2 TB HDD.
Back to top
View user's profile Send private message
Spanik
l33t
l33t


Joined: 12 Dec 2003
Posts: 949
Location: Belgium

PostPosted: Sun Aug 28, 2022 11:01 am    Post subject: Reply with quote

C5ace wrote:

qlop:
2021-11-20T21:32:25 >>> dev-qt/qtwebengine-5.15.2_p20211019: 32′57″
2022-08-17T16:08:14 >>> app-office/libreoffice-7.3.4.2-r1: 16′51″
2022-07-02T11:50:37 >>> dev-lang/rust-1.60.0: 15′24″
2022-03-10T10:46:08 >>> www-client/chromium-99.0.4844.51: 1:24:03
022-08-26T16:58:10 >>> www-client/firefox-91.13.0: 13′26″
2022-07-02T11:24:16 >>> sys-libs/glibc-2.34-r13: 1′59″


What do these number represent? I ran it on my desktop and I must say that the numbers shown with qlop feel/are very different from what you would time if you started a times when the compiling starts until installing starts when running an emerge of the same package.
_________________
Expert in non-working solutions
Back to top
View user's profile Send private message
pietinger
Moderator
Moderator


Joined: 17 Oct 2006
Posts: 4244
Location: Bavaria

PostPosted: Sun Aug 28, 2022 11:26 am    Post subject: Reply with quote

C5ace wrote:
AMD Ryzen 9 3950X 16-Core Processor with 32 threads and 64Gb Ram.

make.conf:
MAKEOPTS="-j33"

Your -j33 comes from an old recommendation (nr. of cores + 1). New measurement showed that -j32 would be a little bit faster (not much). Reason: Switching between threads costs extra cpu time. So, I always recommend to use nr. of of cores as a maximum.

The "problems" with fast CPU is sufficient memory and ... heat ;-)
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54308
Location: 56N 3W

PostPosted: Sun Aug 28, 2022 11:54 am    Post subject: Reply with quote

apiaio,

Today, with a 20 thread CPU, gcc alone can use 40G RAM.
You will want /var/tmp/portage in RAM too. Some individual packages check for 13G.

When you have the RAM the kernel buffers build products in filesystem cache anyway. /var/tmp/portage in RAM saves writes to you NVMe/SSD that will never be read.
Its not really a speed up thing.

That 53G so far. The kernel will want some space for itself, so will your desktop and any applications you want to run.
64G today is a reasonable starting point.

20 years ago I had a 1 core k6-2 with 512MB RAM. Memory and CPU requirements have grown since then :)
Expect that trend to continue, even accelerate.
With an eye to coping with that, expect to add another 64G RAM in the fairly near future.
That dictates a motherboard that can take 4x32G RAM sticks today but only two will be fitted.

CRUCIAL P1 Int. CT500P1SSD8 is marked end of life in the UK. Its a 4 lane PCIe Gen 3 device.
Your ASROCK B660M-HDV motherboard supports 1 Hyper M.2 (PCIe Gen4 x4), among other things.
You should choose a PCIe Gen 4 NVMe card for the best performance in that motherboard.
Also, your motherboard only has two DIMM slots which gives
Quote:
Max. capacity of system memory: 64GB

_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
apiaio
Guru
Guru


Joined: 04 Dec 2008
Posts: 417

PostPosted: Sun Aug 28, 2022 4:41 pm    Post subject: Reply with quote

Thanks Neddy.

I will keep in mind your recommendations. I will have to look around for proper components.
Back to top
View user's profile Send private message
KCmovestoSD
n00b
n00b


Joined: 06 Aug 2022
Posts: 20
Location: California

PostPosted: Sun Aug 28, 2022 5:36 pm    Post subject: Reply with quote

I have a recently upgraded system kinda similar to your build. It's a very nice system. I am curious where I could've changed my build around. I built it with only gaming in mind on Windows in mind. I game less and less now and picked up Rust and Python. Moved to Linux and haven't looked back. I "feel" like if I went team Red my build would be closer tailored to it's current use. But, it's not like I'm compiling a browser and DE daily...

It's a beast of a build you have there.

I have been thinking about increasing my amount of RAM, though I can't seem to justify spending even more money on this system.
_________________
Motherboard: PRIME-D WiFi
CPU: 12600k
RAM: 32GB 3200mhz
GPU: 6800XT & RTX 3070ti
Storage: 2x 1TB gen4 nvme, 500GB gen2 nvme
Back to top
View user's profile Send private message
C5ace
Guru
Guru


Joined: 23 Dec 2013
Posts: 474
Location: Brisbane, Australia

PostPosted: Mon Aug 29, 2022 6:07 am    Post subject: Reply with quote

Spanik:

Clean emerge:
Code:

emerge -av qtwebengine

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] dev-libs/re2-0.2022.06.01:0/9::gentoo  USE="icu" 0 KiB
[ebuild  N     ] dev-qt/qtwebchannel-5.15.5:5/5.15::gentoo  USE="qml -debug -test" 0 KiB
[ebuild  N     ] dev-qt/qtwebengine-5.15.5_p20220618:5/5.15::gentoo  USE="alsa jumbo-build system-ffmpeg system-icu -bindist -debug -designer -geolocation -kerberos -pulseaudio -screencast -test -widgets" 0 KiB

Total: 3 packages (3 new), Size of downloads: 0 KiB

Command:
time emerge qtwebengine
.....
.....
real   33m49.220s
user   941m47.056s ????
sys   60m41.061s   ????

qlop -v
2022-08-29T14:57:49 >>> dev-libs/re2-0.2022.06.01: 7s
2022-08-29T14:57:56 >>> dev-qt/qtwebchannel-5.15.5: 9s
2022-08-29T14:58:05 >>> dev-qt/qtwebengine-5.15.5_p20220618: 33′28″
      Date    time             atom           version      time min/sec to build

Stopwatch was 33m:54sec

pietinger:
make.conf:
MAKEOPTS="-j33" or MAKEOPTS="-j32" makes no difference.

May be able to get a little more time if I set max. CPU temperature in BIOS from 70C to 80C or 90C.
cpuspeed shows top about 4,100Ghz. This will go to 4,700Ghz if max CPU temperature is set to 90C and probably blow the CPU in our summer when we have up to +45C in the shade. Had last summer +51C in my office when the air conditioner was dead.
_________________
Observation after 30 years working with computers:
All software has known and unknown bugs and vulnerabilities. Especially software written in complex, unstable and object oriented languages such as perl, python, C++, C#, Rust and the likes.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54308
Location: 56N 3W

PostPosted: Mon Aug 29, 2022 8:31 am    Post subject: Reply with quote

C5ace,

Code:
time emerge qtwebengine

You just missed a gold star. :)

That command added qtwebengine to your world file. You probably didn't want to do that as qtwebengine is pulled in as a dependency of something else.
You missed --oneshot.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Goverp
Advocate
Advocate


Joined: 07 Mar 2007
Posts: 2014

PostPosted: Mon Aug 29, 2022 9:07 am    Post subject: Reply with quote

C5ace wrote:
Code:
...
user   941m47.056s ????
sys   60m41.061s   ????
...
Stopwatch was 33m:54sec

The times are the sum of the times spent executing on each individual core, so 32 cores=32 times the average clock time per core.
_________________
Greybeard
Back to top
View user's profile Send private message
C5ace
Guru
Guru


Joined: 23 Dec 2013
Posts: 474
Location: Brisbane, Australia

PostPosted: Mon Aug 29, 2022 1:56 pm    Post subject: Reply with quote

NeddySeagoon:

Don't worry. Not using anything that depends on qtwebkit. This exercise was followed by 'emerge --unmerge qtwebkit' and 'emerge -- depclean'.

Goverp:

Thanks for the info regarding time and number of cores.
_________________
Observation after 30 years working with computers:
All software has known and unknown bugs and vulnerabilities. Especially software written in complex, unstable and object oriented languages such as perl, python, C++, C#, Rust and the likes.
Back to top
View user's profile Send private message
apiaio
Guru
Guru


Joined: 04 Dec 2008
Posts: 417

PostPosted: Mon Jan 02, 2023 6:39 pm    Post subject: Reply with quote

After some testing of the new PC
Code:
# qlop qtwebengine
...
2022-11-28T18:27:21 >>> dev-qt/qtwebengine: 2:20:58
2022-12-04T17:42:10 >>> dev-qt/qtwebengine: 2:19:43
2023-01-01T18:22:34 >>> dev-qt/qtwebengine: 42′47″
This is what I awaited.
I am surprised , because the startup from /dev/nvme0n1p3 is a little bit longer than used to be
in old installation with SATA III ssd. I used the old disc with MBR partition table while NVMe
has GPT partiton table. Can it be reason?
But it is not so important, because startup is up to 10 seconds anyway,
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum