View previous topic :: View next topic |
Author |
Message |
apiaio Guru
Joined: 04 Dec 2008 Posts: 417
|
Posted: Sat Aug 27, 2022 5:21 pm Post subject: PC configuration |
|
|
I am going to purchase new PC. Probably the last one. I am not youngest.
Presently I have PC with i7 processor - 8 threads, 16 GB RAM and MB Z-170M-PLUS.
I have picked out
INTEL Core i7-12700K Alder Lake (25M Cache, 3,8/4,9 GHz), Procesor (BX8071512700K) - 20 threads, 32 GB RAM and MB ASROCK B660M-HDV
What I expect is shorter compilation time of the packages like e.g. qtwebengine. In the present compilation qtwebengine takes
some 2 hours and 15 minutes. I use SSD and new PC will have SSD installed also.
I will change to in make.conf
and Code: | tmpfs /var/tmp/portage tmpfs size=32G,uid=portage,gid=portage,mode=775,noatime 0 0 | in /etc/fstab
Concerning of the compilation time what I can expect?
Is the selection of processor, RAM and fast disc main part of good PC configuration for Gentoo?
Should I make any others changes in any configuration files?
Have I omitted something?
Does exist any PC configuration tailored for Gentoo?
Thanks |
|
Back to top |
|
|
pietinger Moderator
Joined: 17 Oct 2006 Posts: 4244 Location: Bavaria
|
Posted: Sat Aug 27, 2022 7:35 pm Post subject: |
|
|
apiaio,
please keep in mind that every compile job costs memory. How much memory depends on every package. Some packages need not more than 1 GB per job, others until 2 GB per job and I dont want to talk about "rust" ...
You surely know that you will be faster with compiling if you set a maximum of jobs that there will be no swaping (means: sometimes you are faster with -j8 instead of -j9 or higher).
Now, IF you put /var/tmp/portage also into memory, you need even more memory/job. Again you must prevent swaping.
You will buy a very fast CPU with many (logical) cores, being able to compile many jobs at the same time ... IF there is enough memory. This leads me to my personal recommendation (as user) to buy 64 GB RAM. My suggestion for this would be:
MAKEOPTS="-j20" and /var/tmp/portage tmpfs size=18G
If you will stay with 32 GB RAM you could use a configuration where you make exceptions for some packages, maybe:
MAKEOPTS="-j20" and /var/tmp/portage tmpfs size=12G
and =>
Code: | # mkdir -p /var/tmp/notmpfs
# chown portage:portage /var/tmp/notmpfs
# mkdir /etc/portage/env
# nano -w /etc/portage/env/notmpfs.conf
=>
PORTAGE_TMPDIR="/var/tmp/notmpfs"
<=
# nano -w /etc/portage/env/monster.conf
=>
MAKEOPTS="-j12"
<=
# nano -w /etc/portage/package.env
=>
app-office/libreoffice monster.conf
dev-lang/rust monster.conf notmpfs.conf
dev-qt/qtwebengine monster.conf
<=
|
(these settings are rather conservative; maybe you can go higher)
I would guess you will compile qtwebengine in one hour or less ... |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54308 Location: 56N 3W
|
Posted: Sat Aug 27, 2022 9:21 pm Post subject: |
|
|
apiaio,
I did that last year. Here's what I chose and why.
That CPU can use 40G RAM at 2G per thread today. You will need a minimum of 64G and two empty slots to fill later.
Do you mean SSD or NVMe?
NVMe can be 10x faster than SSD as its a faster interface. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
apiaio Guru
Joined: 04 Dec 2008 Posts: 417
|
Posted: Sun Aug 28, 2022 8:03 am Post subject: |
|
|
Thanks for answers.
Why should I buy 64G RAM, when chosen CPU can use 40G RAM only? 24G will be useless. |
|
Back to top |
|
|
Spanik l33t
Joined: 12 Dec 2003 Posts: 949 Location: Belgium
|
Posted: Sun Aug 28, 2022 8:27 am Post subject: |
|
|
Interesting thread. But there are a few things I do not really get. I got this setup and thought it would be faster then the previous 10 years old one. But it isn't. An update @world can still need a night to complete.
- Epyc 7402P 24 cores, 48 threads
- 64GB ram (4x16GB) each on it own memory channel
- nvme Samsung 970 EVO pcie 3 x4
- Supermicro H11SSL-i all pcie lanes are separate straight to the cpu
- makeopts = "-j25"
pietinger wrote: | You surely know that you will be faster with compiling if you set a maximum of jobs that there will be no swaping (means: sometimes you are faster with -j8 instead of -j9 or higher).
Now, IF you put /var/tmp/portage also into memory, you need even more memory/job. Again you must prevent swaping.
You will buy a very fast CPU with many (logical) cores, being able to compile many jobs at the same time ... IF there is enough memory. This leads me to my personal recommendation (as user) to buy 64 GB RAM. My suggestion for this would be: |
Compiling still is a mostly linear thing. And indeed Libreoffice took ages. Rust was another issue and I went with the .bin. From what I see when looking at the system monitor there is a lot of swapping between cores going on and the cpu isn't the limiting factor. I never see the used memory go even half of what is available. Now I can add another 64GB and use the 4 remaining memory channels as well. But it is a heavy investment that wouldn't make much sense for the other use of the pc. _________________ Expert in non-working solutions |
|
Back to top |
|
|
pietinger Moderator
Joined: 17 Oct 2006 Posts: 4244 Location: Bavaria
|
Posted: Sun Aug 28, 2022 9:29 am Post subject: |
|
|
apiaio wrote: | Why should I buy 64G RAM, when chosen CPU can use 40G RAM only? 24G will be useless. |
This is a misunderstanding. Neddy took your 20 cores and multiplied it with 2 GB because there is a Gentoo recommendation to have 2 GB / job. As I told you most packages dont need so much. If you compile a "small" package needing only 1 GB / job and you use all 20 cores then you would need only 20 GB ram for this emerge. Now, what is with big packages ? Take qtwebengine as example. It needs up to 2 Gb / job. Why you can set -j12 (=> needing 24 GB) AND tmpfs size 12 GB (would be 36 GB and not 32 GB) ? Because when you emerge you have different processes:
1. After unpacking the package into /var/tmp/portage you will fill this directory a little bit;
2. Now you compile source files and get object files - you will fill this dir more and more; at the same time you need your ram for gcc doing its jobs;
3. after all compiling, all object files will be linked together; this will fill your tmp dir even more; at this time you dont need your ram for gcc (only for ldd)
Look at the last step: Here you will need ALL memory in /var/tmp/portage BUT because there is no compiling anymore you can do settings which would go over your ram at the same time.
Now: rust ... Rust needs up to 4 GB per comile job ... So - even with 64 GB ram - it could be possible that -j20 would be to much. 20 jobs * 4 GB would be 80 GB needed ram. So, even if you take 64 GB ram, it is possible that you must do an excemption rule for rust. |
|
Back to top |
|
|
pietinger Moderator
Joined: 17 Oct 2006 Posts: 4244 Location: Bavaria
|
Posted: Sun Aug 28, 2022 10:09 am Post subject: |
|
|
P.S.: Please dont forget, you will use also other applications than emerge. If you use software which is capable of using all cores (like my "stockfish"; a chess engine) these programs ALSO needs memory for every thread ...
Yes, often 64 GB is not really needed, but it would be sad if you cant use your powerful CPU because of not enough memory in other moments ... |
|
Back to top |
|
|
C5ace Guru
Joined: 23 Dec 2013 Posts: 474 Location: Brisbane, Australia
|
Posted: Sun Aug 28, 2022 10:37 am Post subject: |
|
|
My main box:
No tempfs. Just a 7200RPM rotating rust drive. Desktop is XFce4. Compiling is done in a Xfce Terminal Emulator.
uname -a:
AMD Ryzen 9 3950X 16-Core Processor with 32 threads and 64Gb Ram.
make.conf:
MAKEOPTS="-j33"
qlop:
2021-11-20T21:32:25 >>> dev-qt/qtwebengine-5.15.2_p20211019: 32′57″
2022-08-17T16:08:14 >>> app-office/libreoffice-7.3.4.2-r1: 16′51″
2022-07-02T11:50:37 >>> dev-lang/rust-1.60.0: 15′24″
2022-03-10T10:46:08 >>> www-client/chromium-99.0.4844.51: 1:24:03
022-08-26T16:58:10 >>> www-client/firefox-91.13.0: 13′26″
2022-07-02T11:24:16 >>> sys-libs/glibc-2.34-r13: 1′59″
Maximum memory usage was about 35Gb. No swapping.
Tried using tmpfs and found practically no change. Probably because the HDD has a 64MB cache, Xfce Desktop only uses about 600Mb RAM. This leaves heaps of free RAM for Gentoo to use as buffer/cache. _________________ Observation after 30 years working with computers:
All software has known and unknown bugs and vulnerabilities. Especially software written in complex, unstable and object oriented languages such as perl, python, C++, C#, Rust and the likes. |
|
Back to top |
|
|
apiaio Guru
Joined: 04 Dec 2008 Posts: 417
|
Posted: Sun Aug 28, 2022 10:53 am Post subject: |
|
|
@pietinger
Thank you for comprehensive information. It will be 64GB.
NeddySeagoon wrote: | apiaio,
Do you mean SSD or NVMe?
NVMe can be 10x faster than SSD as its a faster interface. |
I intend to buy Quote: | CRUCIAL P1 Int. Disk SSD 500 GB/M.2 2280/M.2 PCI-Express NVMe CT500P1SSD8 |
and 1 or 2 TB HDD. |
|
Back to top |
|
|
Spanik l33t
Joined: 12 Dec 2003 Posts: 949 Location: Belgium
|
Posted: Sun Aug 28, 2022 11:01 am Post subject: |
|
|
C5ace wrote: |
qlop:
2021-11-20T21:32:25 >>> dev-qt/qtwebengine-5.15.2_p20211019: 32′57″
2022-08-17T16:08:14 >>> app-office/libreoffice-7.3.4.2-r1: 16′51″
2022-07-02T11:50:37 >>> dev-lang/rust-1.60.0: 15′24″
2022-03-10T10:46:08 >>> www-client/chromium-99.0.4844.51: 1:24:03
022-08-26T16:58:10 >>> www-client/firefox-91.13.0: 13′26″
2022-07-02T11:24:16 >>> sys-libs/glibc-2.34-r13: 1′59″ |
What do these number represent? I ran it on my desktop and I must say that the numbers shown with qlop feel/are very different from what you would time if you started a times when the compiling starts until installing starts when running an emerge of the same package. _________________ Expert in non-working solutions |
|
Back to top |
|
|
pietinger Moderator
Joined: 17 Oct 2006 Posts: 4244 Location: Bavaria
|
Posted: Sun Aug 28, 2022 11:26 am Post subject: |
|
|
C5ace wrote: | AMD Ryzen 9 3950X 16-Core Processor with 32 threads and 64Gb Ram.
make.conf:
MAKEOPTS="-j33" |
Your -j33 comes from an old recommendation (nr. of cores + 1). New measurement showed that -j32 would be a little bit faster (not much). Reason: Switching between threads costs extra cpu time. So, I always recommend to use nr. of of cores as a maximum.
The "problems" with fast CPU is sufficient memory and ... heat |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54308 Location: 56N 3W
|
Posted: Sun Aug 28, 2022 11:54 am Post subject: |
|
|
apiaio,
Today, with a 20 thread CPU, gcc alone can use 40G RAM.
You will want /var/tmp/portage in RAM too. Some individual packages check for 13G.
When you have the RAM the kernel buffers build products in filesystem cache anyway. /var/tmp/portage in RAM saves writes to you NVMe/SSD that will never be read.
Its not really a speed up thing.
That 53G so far. The kernel will want some space for itself, so will your desktop and any applications you want to run.
64G today is a reasonable starting point.
20 years ago I had a 1 core k6-2 with 512MB RAM. Memory and CPU requirements have grown since then :)
Expect that trend to continue, even accelerate.
With an eye to coping with that, expect to add another 64G RAM in the fairly near future.
That dictates a motherboard that can take 4x32G RAM sticks today but only two will be fitted.
CRUCIAL P1 Int. CT500P1SSD8 is marked end of life in the UK. Its a 4 lane PCIe Gen 3 device.
Your ASROCK B660M-HDV motherboard supports 1 Hyper M.2 (PCIe Gen4 x4), among other things.
You should choose a PCIe Gen 4 NVMe card for the best performance in that motherboard.
Also, your motherboard only has two DIMM slots which gives Quote: | Max. capacity of system memory: 64GB |
_________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
apiaio Guru
Joined: 04 Dec 2008 Posts: 417
|
Posted: Sun Aug 28, 2022 4:41 pm Post subject: |
|
|
Thanks Neddy.
I will keep in mind your recommendations. I will have to look around for proper components. |
|
Back to top |
|
|
KCmovestoSD n00b
Joined: 06 Aug 2022 Posts: 20 Location: California
|
Posted: Sun Aug 28, 2022 5:36 pm Post subject: |
|
|
I have a recently upgraded system kinda similar to your build. It's a very nice system. I am curious where I could've changed my build around. I built it with only gaming in mind on Windows in mind. I game less and less now and picked up Rust and Python. Moved to Linux and haven't looked back. I "feel" like if I went team Red my build would be closer tailored to it's current use. But, it's not like I'm compiling a browser and DE daily...
It's a beast of a build you have there.
I have been thinking about increasing my amount of RAM, though I can't seem to justify spending even more money on this system. _________________ Motherboard: PRIME-D WiFi
CPU: 12600k
RAM: 32GB 3200mhz
GPU: 6800XT & RTX 3070ti
Storage: 2x 1TB gen4 nvme, 500GB gen2 nvme |
|
Back to top |
|
|
C5ace Guru
Joined: 23 Dec 2013 Posts: 474 Location: Brisbane, Australia
|
Posted: Mon Aug 29, 2022 6:07 am Post subject: |
|
|
Spanik:
Clean emerge:
Code: |
emerge -av qtwebengine
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild N ] dev-libs/re2-0.2022.06.01:0/9::gentoo USE="icu" 0 KiB
[ebuild N ] dev-qt/qtwebchannel-5.15.5:5/5.15::gentoo USE="qml -debug -test" 0 KiB
[ebuild N ] dev-qt/qtwebengine-5.15.5_p20220618:5/5.15::gentoo USE="alsa jumbo-build system-ffmpeg system-icu -bindist -debug -designer -geolocation -kerberos -pulseaudio -screencast -test -widgets" 0 KiB
Total: 3 packages (3 new), Size of downloads: 0 KiB
Command:
time emerge qtwebengine
.....
.....
real 33m49.220s
user 941m47.056s ????
sys 60m41.061s ????
qlop -v
2022-08-29T14:57:49 >>> dev-libs/re2-0.2022.06.01: 7s
2022-08-29T14:57:56 >>> dev-qt/qtwebchannel-5.15.5: 9s
2022-08-29T14:58:05 >>> dev-qt/qtwebengine-5.15.5_p20220618: 33′28″
Date time atom version time min/sec to build
Stopwatch was 33m:54sec
|
pietinger:
make.conf:
MAKEOPTS="-j33" or MAKEOPTS="-j32" makes no difference.
May be able to get a little more time if I set max. CPU temperature in BIOS from 70C to 80C or 90C.
cpuspeed shows top about 4,100Ghz. This will go to 4,700Ghz if max CPU temperature is set to 90C and probably blow the CPU in our summer when we have up to +45C in the shade. Had last summer +51C in my office when the air conditioner was dead. _________________ Observation after 30 years working with computers:
All software has known and unknown bugs and vulnerabilities. Especially software written in complex, unstable and object oriented languages such as perl, python, C++, C#, Rust and the likes. |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54308 Location: 56N 3W
|
Posted: Mon Aug 29, 2022 8:31 am Post subject: |
|
|
C5ace,
Code: | time emerge qtwebengine |
You just missed a gold star. :)
That command added qtwebengine to your world file. You probably didn't want to do that as qtwebengine is pulled in as a dependency of something else.
You missed --oneshot. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
Goverp Advocate
Joined: 07 Mar 2007 Posts: 2014
|
Posted: Mon Aug 29, 2022 9:07 am Post subject: |
|
|
C5ace wrote: | Code: | ...
user 941m47.056s ????
sys 60m41.061s ????
...
Stopwatch was 33m:54sec
|
|
The times are the sum of the times spent executing on each individual core, so 32 cores=32 times the average clock time per core. _________________ Greybeard |
|
Back to top |
|
|
C5ace Guru
Joined: 23 Dec 2013 Posts: 474 Location: Brisbane, Australia
|
Posted: Mon Aug 29, 2022 1:56 pm Post subject: |
|
|
NeddySeagoon:
Don't worry. Not using anything that depends on qtwebkit. This exercise was followed by 'emerge --unmerge qtwebkit' and 'emerge -- depclean'.
Goverp:
Thanks for the info regarding time and number of cores. _________________ Observation after 30 years working with computers:
All software has known and unknown bugs and vulnerabilities. Especially software written in complex, unstable and object oriented languages such as perl, python, C++, C#, Rust and the likes. |
|
Back to top |
|
|
apiaio Guru
Joined: 04 Dec 2008 Posts: 417
|
Posted: Mon Jan 02, 2023 6:39 pm Post subject: |
|
|
After some testing of the new PC
Code: | # qlop qtwebengine
...
2022-11-28T18:27:21 >>> dev-qt/qtwebengine: 2:20:58
2022-12-04T17:42:10 >>> dev-qt/qtwebengine: 2:19:43
2023-01-01T18:22:34 >>> dev-qt/qtwebengine: 42′47″ | This is what I awaited.
I am surprised , because the startup from /dev/nvme0n1p3 is a little bit longer than used to be
in old installation with SATA III ssd. I used the old disc with MBR partition table while NVMe
has GPT partiton table. Can it be reason?
But it is not so important, because startup is up to 10 seconds anyway, |
|
Back to top |
|
|
|