View previous topic :: View next topic |
Author |
Message |
eccerr0r Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
Joined: 01 Jul 2004 Posts: 9891 Location: almost Mile High in the USA
|
Posted: Fri Oct 05, 2018 10:09 pm Post subject: |
|
|
Yeah, I took a chunk out of my SSD's life trying to compile I believe it was webkit or qt. I only had 3 processes going on this dual core (-j3) with 2GB RAM, and it was swapping quite a bit though it was kind of hard to notice since it swaps fairly quickly.
The SSD is now probably getting close to a decade old, and I just hit... 1% lifetime usage (it's now about 30 erase cycles on the MLC). Portage_tmpdir is on the SSD as well. That last huge batch update I did on this SSD took around ~2 erase cycles to complete, and the swapping was a good chunk of it... and having to restart broken merges kills... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
josephg l33t
![l33t l33t](/images/ranks/rank_rect_4.gif)
![](images/avatars/5277610355ba555d6aa280.png)
Joined: 10 Jan 2016 Posts: 783 Location: usually offline
|
Posted: Fri Oct 05, 2018 10:56 pm Post subject: |
|
|
that's why i have -gk in make.conf on my netbook |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
Jaglover Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
![](images/avatars/179708169458f2999e44e26.gif)
Joined: 29 May 2005 Posts: 8291 Location: Saint Amant, Acadiana
|
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
josephg l33t
![l33t l33t](/images/ranks/rank_rect_4.gif)
![](images/avatars/5277610355ba555d6aa280.png)
Joined: 10 Jan 2016 Posts: 783 Location: usually offline
|
Posted: Sat Oct 06, 2018 12:11 am Post subject: |
|
|
that's interesting.. i wonder if you can cross compile arm in nfs chroot from intel |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
eccerr0r Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
Joined: 01 Jul 2004 Posts: 9891 Location: almost Mile High in the USA
|
Posted: Sat Oct 06, 2018 12:14 am Post subject: |
|
|
Actually build memory use isn't the problem, the fact that build memory use is high, all other operations on that computer like running Firefox, etc., is also high, and performance will suffer likewise.
However if you still do not completely use up all your RAM, then maybe things are okay. I just felt that amd64 2GB machine even when not compiling seemed to use up all RAM well before my x86 machine did with 2GB RAM. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
josephg l33t
![l33t l33t](/images/ranks/rank_rect_4.gif)
![](images/avatars/5277610355ba555d6aa280.png)
Joined: 10 Jan 2016 Posts: 783 Location: usually offline
|
Posted: Sat Oct 06, 2018 12:26 pm Post subject: |
|
|
one of the reasons i still continue with 32bit on all my systems |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
1clue Advocate
![Advocate Advocate](/images/ranks/rank-G-1-advocate.gif)
Joined: 05 Feb 2006 Posts: 2569
|
Posted: Tue Oct 09, 2018 1:34 am Post subject: |
|
|
IMO Gentoo is not happy in 2 GiB no matter what. I think that anything under 4 and you want to have a compiler farm with bigger hardware somewhere on your network.
I have successfully installed Gentoo on a Raspberry Pi B+ without the compiler farm, but when I was done I pretty much trashed it as it would be a full-time job just to do updates on it. |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
eccerr0r Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
Joined: 01 Jul 2004 Posts: 9891 Location: almost Mile High in the USA
|
Posted: Sat Oct 13, 2018 4:42 pm Post subject: |
|
|
I tried chrooting into my nfs mounted Atom from my i7...
When I tried running 'emerge' I got greeted with:
Code: | /tmp # chroot /mnt0
/ # emerge
Illegal instruction (core dumped)
/ # exit |
No idea where it dumped core...
I still sort of update a 1GB P-m, it's still tolerable but swaps quite a bit. Less than 1GB I don't even try. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
tribute58 n00b
![n00b n00b](/images/ranks/rank_rect_0.gif)
Joined: 14 Oct 2018 Posts: 7
|
Posted: Sun Oct 14, 2018 2:55 pm Post subject: |
|
|
I'm running Gentoo with 64bit kernel + userspace on an Acer Aspire One D255 (Atom N550 1.5GHz dualcore + HT with max 2GB Ram). And it works!
The only problematic package, in case of ram, is chromium. It didn't respect -jn and ran with at least 8 threads, if I remember correctly. Maybe one can override it... I didn't dig into this as I use firefox, which compiles just fine.
My make.conf uses march=native -O2 pipe and -j2! Running updates over night is not a problem as long as you don't try to experiment with LTO.
The only bin package I'm using is LibreOffice, but thats a question of CPU performance...
I don't think -Os helps, neither compiling nor running programs, but I never checked that. Of course, it reduces the instruction memory footprint, but has no influence on data, so the overall gain will be negligible. |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
eccerr0r Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
Joined: 01 Jul 2004 Posts: 9891 Location: almost Mile High in the USA
|
Posted: Sun Oct 14, 2018 3:18 pm Post subject: |
|
|
My Atom 32 bit with 2GB RAM is still chugging away at updates. It was compiling rust when I looked at it last night, and is STILL compiling rust this morning...
Code: | atom /tmp # uname -a
Linux atom 4.9.95-gentoo #1 SMP Mon Jun 18 21:51:25 MDT 2018 i686 Intel(R) Atom(TM) CPU N270 @ 1.60GHz GenuineIntel GNU/Linux
atom /tmp # qlop -c
* dev-lang/rust-1.29.1
started: Sat Oct 13 20:34:01 2018
elapsed: 12 hours, 53 minutes, 11 seconds |
CPU limited not RAM limited I think, though I don't quite get how rustc allocates memory. It appears to be multithreaded but once I caught it pretty much using all RAM but didn't touch swap.
I did notice rustc use more than 2 threads on my quad core machines, so it must have some internal thread usage limitation. The current rustc execution has a resident size of 1.2GB, though I saw 1.5GB last night.
I switched over to swapping onto a USB HDD just in case, but current swap usage is 0. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
cord Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/8391414414aa54e6971a84.png)
Joined: 28 Apr 2007 Posts: 346
|
Posted: Sun Oct 14, 2018 6:14 pm Post subject: |
|
|
How long dev-qt/qtwebkit compilation took? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
eccerr0r Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
Joined: 01 Jul 2004 Posts: 9891 Location: almost Mile High in the USA
|
Posted: Sun Oct 14, 2018 7:45 pm Post subject: |
|
|
I don't have qtwebkit or webkit-gtk installed on this machine, plus sometimes I have distcc working and sometimes not, so hard to tell which merges were done under what circumstances.
Also not sure why I have a few entries with really short merge times, but this one is probably accurate:
rust: Sat Oct 13 20:27:49 2018: 57930 seconds
Gcc is bad too, this is 7.3:
gcc: Sat Oct 13 09:19:24 2018: 27702 seconds
32-bit -- so far, neither package swaps while building using MAKEOPTS=-j2 (since this Atom is a dual thread machine.) on 2GB.
The 64-bit setup seems more swappy with the same amount of RAM. Alas overall speed is faster for the 64-bit because that's a Core2 Duo which is many times faster than the dual thread single core Atom. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
n05ph3r42 Tux's lil' helper
![Tux's lil' helper Tux's lil' helper](/images/ranks/rank_rect_1.gif)
Joined: 11 Jul 2016 Posts: 135
|
Posted: Sun Oct 14, 2018 8:07 pm Post subject: |
|
|
cord:
As for hardware RAM limitations - i suggest to try other RAM, better of same manufacturer, same as soldered on your mb. Mostly this "limitations" is only in specifications.
Also you can try https://wiki.gentoo.org/wiki/Distcc to speedup compilation, if you have other machines on your network. |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
tribute58 n00b
![n00b n00b](/images/ranks/rank_rect_0.gif)
Joined: 14 Oct 2018 Posts: 7
|
Posted: Sun Oct 14, 2018 8:51 pm Post subject: |
|
|
eccerr0r wrote: | The 64-bit setup seems more swappy with the same amount of RAM. |
You're right, measuring absolute numbers, but the difference is negligible. I started with a 32 bit System, but soon made the step to 64 bit, because the extra Ram usage is much less than often communicated. Using a lightweight DE and activating only necessary services, plugins, ... is important.
cord wrote: | How long dev-qt/qtwebkit compilation took? |
You can't compare that to your system. eccerr0rs system has one physical core, mine has two (as yours has), but both have much less IPC than your C2D. So if swapping doesn't occur, you're compile times will be seriously faster than ours. |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
eccerr0r Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
Joined: 01 Jul 2004 Posts: 9891 Location: almost Mile High in the USA
|
Posted: Thu Nov 08, 2018 6:13 am Post subject: |
|
|
BTW, portage/"emerge" uses ~500MB of RAM on an amd64 box I'm updating right now (2GB RAM) ... a quarter of its RAM is used by portage for emerge!
I suspect I'm not comparing apples with apples, but a 32-bit install I'm maintaining, emerge used up about 252MB of RAM. The difference: the 32-bit install (1GB RAM...) has an xfce4 setup, other 64-bit has full GNOME setup too; both have firefox-60 and its dependencies.
Anyone see similar memory use? No wonder the 2GB amd64 machine is running fairly crappy when doing updates. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
cord Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/8391414414aa54e6971a84.png)
Joined: 28 Apr 2007 Posts: 346
|
Posted: Thu Nov 15, 2018 5:22 pm Post subject: |
|
|
eccerr0r wrote: | BTW, portage/"emerge" uses ~500MB of RAM on an amd64 box I'm updating right now (2GB RAM) ... a quarter of its RAM is used by portage for emerge! |
Is there difference between "emerge --sync" / "emerge-webrsync"? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
eccerr0r Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
Joined: 01 Jul 2004 Posts: 9891 Location: almost Mile High in the USA
|
Posted: Thu Nov 15, 2018 8:05 pm Post subject: |
|
|
Those don't make portage use much memory. Anyway --sync could be using rsync or git I believe, webrsync is rsync. Both use an external program to do the sync and isn't really using portage.
Doing the dependency calculations unfortunately is serialized with doing the actual make compilation, unlike doing a sync which does not need to be (and should not be) running while computing dependencies... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
Anon-E-moose Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
![](images/avatars/3064969284ad468b81342a.jpg)
Joined: 23 May 2008 Posts: 6222 Location: Dallas area
|
Posted: Thu Nov 15, 2018 8:08 pm Post subject: |
|
|
cord wrote: | eccerr0r wrote: | BTW, portage/"emerge" uses ~500MB of RAM on an amd64 box I'm updating right now (2GB RAM) ... a quarter of its RAM is used by portage for emerge! |
Is there difference between "emerge --sync" / "emerge-webrsync"? |
Just time, webrsync downloads portage as a tarball instead of individual packages, then unpacks them. May be some memory difference but I doubt much. _________________ UM780, 6.12 zen kernel, gcc 13, openrc, wayland |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
cord Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/8391414414aa54e6971a84.png)
Joined: 28 Apr 2007 Posts: 346
|
Posted: Fri Nov 16, 2018 4:40 pm Post subject: |
|
|
It needs 500 MiB RAM to unpack 50 MiB archive? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
eccerr0r Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
Joined: 01 Jul 2004 Posts: 9891 Location: almost Mile High in the USA
|
Posted: Fri Nov 16, 2018 5:25 pm Post subject: |
|
|
No... again, 500MB to compute dependencies. When doing a sync, dependencies are not computed - heck, portage doesn't even load the ebuilds to compute anything... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
eccerr0r Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
Joined: 01 Jul 2004 Posts: 9891 Location: almost Mile High in the USA
|
Posted: Thu Nov 29, 2018 6:28 pm Post subject: |
|
|
For curiosity sake, dev-lang/rust of the few runs I tried took:
Atom 1.6GHz dual thread, 32-bit: 16 hours (std dev less than 5 mins)
Pentium-M 1.6GHz single thread, 32-bit: 7 hours (std dev is on the order of an hour, so I have to get more accurate data...)
I have to use rust as a gauge because rust does not distribute, so I know it's not cheating :)
Anyway, I didn't expect this kind of discrepancy, thought the two processors were closer together in terms of speed. Alas the Atom is not affected by meltdown or other CPU security issues...
I will need to do some testing of 32- vs 64- bit. The only data I have right now is flawed:
32 bit QEMU KVM using 2 cores on Core2Quad 9550S: just under 3 hours
64bit baremetal using 4 cores on Core2Quad 9550S: just over 1 hour.
If I extrapolate the 32-bit number, it would be under 1.5 hours in 32 bit. That would mean that QEMU is quite slow adding 50% time penalty or 33% speed penalty... Of course this has no bearing on 32- vs 64- on baremetal. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
Marcih Apprentice
![Apprentice Apprentice](/images/ranks/rank_rect_2.gif)
![](images/avatars/10212797815bcc419b421c8.png)
Joined: 19 Feb 2018 Posts: 213
|
Posted: Thu Nov 29, 2018 9:24 pm Post subject: |
|
|
eccerr0r wrote: | Atom 1.6GHz dual thread, 32-bit: 16 hours (std dev less than 5 mins)
Pentium-M 1.6GHz single thread, 32-bit: 7 hours (std dev is on the order of an hour, so I have to get more accurate data...). |
That's interesting, what exact processors are those? I have a notebook and a 2-in-1 with a Pentium M 770 (2.33GHz, single core) and an Atom Z3735F (1.33GHz, quad-core), both have Gentoo with all the installed packages and the kernel compiled using GCC's native optimizations. I have yet to do any scientific testing but the Atom seems to be (less than) half the speed of the Pentium M (~20 mins vs 1 hr compiling glibc) and I'm wondering why that is. If the Atoms deal with instructions differently, shouldn't the compiler optimize for that, making them not suck as much? The Pentium is simple, it's just a more power-efficient PIII, no change in how it executes instructions as far as I know, but the Atom family may have a more esoteric set-up (I know the rest of the device does, I haven't managed to get audio working on it after almost a year ). _________________
Bones McCracker wrote: | It wouldn't be so bad, if it didn't suck. |
NeddySeagoon wrote: | The problem with leaving is that you can only do it once and it reduces your influence. |
|
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
1clue Advocate
![Advocate Advocate](/images/ranks/rank-G-1-advocate.gif)
Joined: 05 Feb 2006 Posts: 2569
|
Posted: Thu Nov 29, 2018 9:33 pm Post subject: |
|
|
Marcih wrote: | eccerr0r wrote: | Atom 1.6GHz dual thread, 32-bit: 16 hours (std dev less than 5 mins)
Pentium-M 1.6GHz single thread, 32-bit: 7 hours (std dev is on the order of an hour, so I have to get more accurate data...). |
That's interesting, what exact processors are those? I have a notebook and a 2-in-1 with a Pentium M 770 (2.33GHz, single core) and an Atom Z3735F (1.33GHz, quad-core), both have Gentoo with all the installed packages and the kernel compiled using GCC's native optimizations. I have yet to do any scientific testing but the Atom seems to be (less than) half the speed of the Pentium M (~20 mins vs 1 hr compiling glibc) and I'm wondering why that is. If the Atoms deal with instructions differently, shouldn't the compiler optimize for that, making them not suck as much? The Pentium is simple, it's just a more power-efficient PIII, no change in how it executes instructions as far as I know, but the Atom family may have a more esoteric set-up (I know the rest of the device does, I haven't managed to get audio working on it after almost a year ). |
I have a c2758 based board running Gentoo. It also sucks pretty bad for compile times compared to my i7 920, but for encryption/compression/networking tasks it goes neck and neck with the i7. The atom has extra hardware for acceleration of encryption and compression, and the board was designed as a network device with 7x Intel nics. So I suspect that this atom from eccerr0r also has some sort of extra on-chip hardware that makes some things faster, but not compilation. |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
eccerr0r Watchman
![Watchman Watchman](/images/ranks/rank-G-2-watchman.gif)
Joined: 01 Jul 2004 Posts: 9891 Location: almost Mile High in the USA
|
Posted: Thu Nov 29, 2018 9:57 pm Post subject: |
|
|
The atom is an old n270 single core dual thread, the Pentium-M is just a Dothan 1.6GHz.
I also suspect that the "rust" workload is really not what an atom is designed for. I haven't had a definitive distcc-less gcc run to compare but I suspect gcc also isn't quite a "normal" workload.
On the other hand firefox itself seems about the same or maybe a tad worse on the Pentium-M and the Atom. But I haven't run a true benchmark on them to compare, but I would think that a browser should be in the intended workloads of the Atom... _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
|