| View previous topic :: View next topic |
| Author |
Message |
Princess Nell
l33t
Joined: 15 Apr 2005
Posts: 927
|
 Posted: Wed Dec 11, 2024 8:31 pm Post subject: nodejs compile time |
 |
|
Excerpt:
| Code: |
2017-05-05T21:37:48 >>> net-libs/nodejs-6.9.4: 8′43″
2018-04-15T19:19:29 >>> net-libs/nodejs-8.11.1: 17′30″
2019-11-23T23:42:46 >>> net-libs/nodejs-12.13.0: 10:02:58
2020-05-20T21:21:05 >>> net-libs/nodejs-14.2.0: 1:02:04
2022-05-21T19:52:36 >>> net-libs/nodejs-16.14.2: 59′15″
2022-06-18T21:22:48 >>> net-libs/nodejs-18.3.0: 1:13:32
2023-05-27T23:54:43 >>> net-libs/nodejs-20.2.0: 1:25:51
2024-07-13T21:33:24 >>> net-libs/nodejs-22.3.0: 2:50:16
2024-08-17T20:11:17 >>> net-libs/nodejs-22.4.1-r1: 3:19:54
2024-10-26T23:33:21 >>> net-libs/nodejs-22.4.1-r1: 2:45:26
|
To put this into context, the emerge history goes back at least one, if not two build machines, and each new box was faster than the previous. I'm willing to accept the 2nd last entry as an outlier, but do we have an explanation for the compile time for this ebuild? On my system, it is required by all things mozilla, direct and derived, so I can't avoid it.
| Code: |
Installed versions: 22.4.1-r1(0/22)^t(01:18:34 27/10/24)(icu npm snapshot ssl system-icu system-ssl -corepack -debug -doc -inspector -lto -pax-kernel -test CPU_FLAGS_X86="sse2")
|
I could probably disable npm, but not sure for how much gain. |
|
| Back to top |
|
 |
Banana
Moderator
Joined: 21 May 2004
Posts: 1811
Location: Germany
|
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20540
|
| Posted: Sat Dec 14, 2024 5:20 am Post subject: |
|
|
08:17:35 - 13 03:12:24
That was fun.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
gorg86
Guru
Joined: 20 May 2011
Posts: 320
|
| Posted: Mon Dec 16, 2024 8:12 pm Post subject: |
|
|
| Post your emerge --info, those compile times are very long even for a potato PC. |
|
| Back to top |
|
|
sam_
Developer
Joined: 14 Aug 2020
Posts: 2043
|
| Posted: Mon Dec 16, 2024 8:45 pm Post subject: |
|
|
Someone reported this to me on IRC a few months ago and we did a little bit of digging. We found the substantial increase between iirc 20 and 22 started with an import of newer V8.
I filed a handful of GCC bugs (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=116286, https://gcc.gnu.org/bugzilla/show_bug.cgi?id=116285) for some kind of slow compilations but they weren't smoking guns and I don't really feel like we got to the bottom of it. The difference for those 2 bugs I filed wasn't really ideal in terms of something we could easily reduce to make it more actionable, either. |
|
| Back to top |
|
|
bstaletic
Guru
Joined: 05 Apr 2014
Posts: 412
|
| Posted: Mon Dec 16, 2024 8:48 pm Post subject: |
|
|
Those times match mine:
| qlop -vtm nodejs: | 2023-05-01T01:35:20 >>> net-libs/nodejs-20.0.0: 1:32:03
2023-05-09T20:13:52 >>> net-libs/nodejs-20.1.0: 1:33:35
2023-05-18T21:38:58 >>> net-libs/nodejs-20.2.0: 1:26:32
2023-07-01T06:27:01 >>> net-libs/nodejs-20.3.1: 1:29:50
2023-08-13T12:01:04 >>> net-libs/nodejs-20.5.1: 1:31:38
2023-09-10T09:07:16 >>> net-libs/nodejs-20.6.0: 1:47:03
2023-09-13T17:40:33 >>> net-libs/nodejs-20.6.1: 1:35:12
2023-09-30T12:46:06 >>> net-libs/nodejs-20.7.0: 1:28:57
2023-10-23T08:42:28 >>> net-libs/nodejs-20.8.1: 1:27:23
2023-12-14T07:43:39 >>> net-libs/nodejs-20.9.0: 1:22:38
2024-01-29T02:05:34 >>> net-libs/nodejs-20.11.0: 1:26:11
2024-03-05T00:02:17 >>> net-libs/nodejs-20.11.1: 1:20:58
2024-08-05T16:15:26 >>> net-libs/nodejs-22.5.1: 3:14:25
2024-08-21T08:51:24 >>> net-libs/nodejs-22.6.0: 2:53:33
2024-08-31T23:00:21 >>> net-libs/nodejs-22.7.0: 3:12:56
2024-09-17T09:37:26 >>> net-libs/nodejs-22.8.0: 2:52:32
2024-11-11T20:38:17 >>> net-libs/nodejs-22.10.0: 3:17:04 |
nodejs is a massive pain in the ass to compile. I did at some point turn on LTO globally. |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20540
|
| Posted: Wed Dec 18, 2024 4:46 am Post subject: |
|
|
| gorg86 wrote: | | Post your emerge --info, those compile times are very long even for a potato PC. |
If that's regarding mine, the results are a bit misleading. I had forgotten that I changed from -j3 to -j1 to prevent intermittent failures. It has been building with -j1 for some time... I'll try -j2 to see it works.
It seems likely that I switched from -j3 to -j1 with nodejs-20.5.1: | Code: | net-libs/nodejs-20.2.0: 1:29:57
net-libs/nodejs-20.2.0: 1:35:28
net-libs/nodejs-20.2.0: 1:33:24
net-libs/nodejs-20.5.1: 4:10:36
net-libs/nodejs-20.6.1: 4:18:17
net-libs/nodejs-20.11.0: 4:12:00
net-libs/nodejs-20.11.0: 4:28:08
net-libs/nodejs-20.12.1: 4:02:24
net-libs/nodejs-22.3.0: 6:53:02
net-libs/nodejs-22.4.1-r1: 6:56:54
net-libs/nodejs-22.4.1-r1: 6:55:44
net-libs/nodejs-22.4.1-r1: 6:54:49 |
I've been delaying an icu induced rebuild of nodejs / firefox, so I'll see what happens later this week.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2536
|
| Posted: Wed Dec 18, 2024 6:19 am Post subject: |
|
|
With the consideration I'm assuming most of the commenters here do not seem to need nodejs for something other than a build time dependency, I would urge moving to binary packages for the ones that require nodejs as a build dependency. Nodejs is too big of a crap to waste any amount of time on it.
p.s. I hate that piece of .... for it made so much more .... possible. If it wasn't for firefox and calibre, it wouldn't exist on my computer.
Best Regards,
Georgi |
|
| Back to top |
|
|
gorg86
Guru
Joined: 20 May 2011
Posts: 320
|
| Posted: Wed Dec 18, 2024 11:35 am Post subject: |
|
|
@pjp I was talking to the OP  |
|
| Back to top |
|
|
Perfect Gentleman
Veteran
Joined: 18 May 2014
Posts: 1256
|
| Posted: Wed Dec 18, 2024 1:34 pm Post subject: |
|
|
| Code: | 2023-10-18T08:17:13 >>> net-libs/nodejs-20.8.1: 24m47s
2023-10-18T17:48:05 >>> net-libs/nodejs-20.8.1: 26m44s
2023-10-26T17:54:18 >>> net-libs/nodejs-21.1.0: 29m43s
2023-11-20T06:19:49 >>> net-libs/nodejs-21.1.0: 29m10s
2023-11-20T17:40:27 >>> net-libs/nodejs-21.2.0: 29m17s
2023-11-28T05:46:47 >>> net-libs/nodejs-21.2.0: 30m24s
2023-12-01T19:54:27 >>> net-libs/nodejs-21.3.0: 29m34s
2023-12-02T11:59:05 >>> net-libs/nodejs-21.3.0: 31m22s
2023-12-06T22:32:52 >>> net-libs/nodejs-21.4.0: 28m09s
2023-12-27T21:36:43 >>> net-libs/nodejs-21.5.0: 29m31s
2024-01-31T11:55:21 >>> net-libs/nodejs-21.6.1: 29m07s
2024-02-16T11:35:22 >>> net-libs/nodejs-21.6.2: 31m56s
2024-04-16T11:13:08 <<< net-libs/nodejs-21.7.3: 23s
2024-04-16T11:13:55 >>> net-libs/nodejs-20.12.2: 23m34s
2024-04-16T12:13:51 >>> net-libs/nodejs-21.7.3: 28m32s
2024-04-26T19:07:00 >>> net-libs/nodejs-22.0.0: 36m10s
2024-05-04T05:22:37 >>> net-libs/nodejs-22.1.0: 35m04s
2024-05-10T07:53:23 >>> net-libs/nodejs-22.1.0: 42m10s
2024-05-19T17:41:01 >>> net-libs/nodejs-22.2.0: 46m44s
2024-06-18T08:27:41 >>> net-libs/nodejs-22.3.0: 38m04s
2024-07-03T18:53:01 >>> net-libs/nodejs-22.4.0: 46m15s
2024-07-10T19:03:30 >>> net-libs/nodejs-22.4.1: 41m37s
2024-07-14T23:42:10 >>> net-libs/nodejs-22.4.1: 40m34s
2024-07-22T07:43:54 >>> net-libs/nodejs-22.5.1: 40m38s
2024-08-08T18:49:18 >>> net-libs/nodejs-22.6.0: 46m17s
2024-08-23T06:59:24 >>> net-libs/nodejs-22.7.0: 39m42s
2024-09-02T07:13:29 >>> net-libs/nodejs-22.7.0: 39m59s
2024-09-07T04:43:14 >>> net-libs/nodejs-22.8.0: 38m27s
2024-10-06T22:49:46 >>> net-libs/nodejs-22.9.0: 37m44s
2024-10-10T18:37:57 >>> net-libs/nodejs-22.9.0: 40m12s
2024-10-18T16:54:36 >>> net-libs/nodejs-23.0.0: 46m17s
2024-10-26T11:57:24 >>> net-libs/nodejs-23.1.0: 42m41s
2024-11-11T12:12:39 >>> net-libs/nodejs-23.1.0: 46m39s
2024-11-12T20:05:15 >>> net-libs/nodejs-23.2.0: 46m02s
2024-11-22T15:19:17 >>> net-libs/nodejs-23.3.0: 47m31s
2024-12-07T06:27:28 >>> net-libs/nodejs-23.3.0: 45m04s
2024-12-15T16:49:08 >>> net-libs/nodejs-23.4.0: 47m11s
|
i7-4770k
Last edited by Perfect Gentleman on Thu Dec 19, 2024 12:06 am; edited 1 time in total |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20540
|
| Posted: Wed Dec 18, 2024 9:23 pm Post subject: |
|
|
| gorg86 wrote: | | @pjp I was talking to the OP :lol: |
I wondered but couldn't tell as their results were a riced potato compared to my own :)
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3802
Location: Rasi, Finland
|
| Posted: Wed Dec 18, 2024 10:14 pm Post subject: |
|
|
| Code: | NBLK-WAX9X ~ # qlop -vtm nodejs | awk '(NR == 1); END {print}'
2023-04-14T03:01:44 >>> net-libs/nodejs-18.14.2: 1:25:31
2024-12-03T17:22:41 >>> net-libs/nodejs-22.4.1-r1: 4:31:02 |
... sure - build time has increased.
I wish bun or deno were alternatives already.
_________________
..: Zucca :..
| My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
Yamakuzure
Advocate
Joined: 21 Jun 2006
Posts: 2305
Location: Adendorf, Germany
|
| Posted: Thu Dec 19, 2024 7:22 am Post subject: |
|
|
This is on a Dell Precision Laptop with Intel Core i7-10875H CPU @ 2.30GHz (8 cores HT) with 64GB RAM: | Code: | sed@sed-m7550 ~ $ qlop -vtm nodejs
2024-10-29T09:37:53 >>> net-libs/nodejs-22.10.0: 2:06:34
2024-11-11T12:34:27 >>> net-libs/nodejs-22.10.0: 1:26:22
2024-11-12T07:54:13 >>> net-libs/nodejs-22.10.0: 1:13:27
2024-12-09T10:11:15 >>> net-libs/nodejs-22.10.0: 1:28:01
2024-12-11T20:55:18 >>> net-libs/nodejs-22.12.0: 1:10:49 |
I am using ccache if that matters.
Also my make.conf contains: | Code: | EMERGE_DEFAULT_OPTS="${EMERGE_DEFAULT_OPTS} --jobs=7 --load-average=10.0"
MAKEOPTS="-j17 -l12" |
I still think this is rather long, but it is a laptop after all.
_________________
Edited 220,176 times by Yamakuzure |
|
| Back to top |
|
|
Kaorukun
n00b
Joined: 27 Aug 2014
Posts: 6
|
| Posted: Sun Dec 29, 2024 1:25 am Post subject: |
|
|
We came a long way, baby...
Increment from 3 minutes to 40+ in the last 6 years indeed is a bit concerning. Also, why would it need to recompile the same version several times?
| Code: | 2018-10-26T22:11:06 >>> net-libs/nodejs-8.12.0: 2′57″
2019-08-11T11:05:38 >>> net-libs/nodejs-8.12.0: 3′12″
2019-08-15T00:34:56 >>> net-libs/nodejs-8.12.0: 3′08″
2019-10-09T21:48:57 >>> net-libs/nodejs-8.12.0: 3′20″
2019-12-07T10:38:30 >>> net-libs/nodejs-12.13.0: 6′33″
2019-12-28T20:03:02 >>> net-libs/nodejs-12.13.0: 6′25″
2020-03-14T13:07:28 >>> net-libs/nodejs-12.14.0: 6′32″
2020-03-24T12:45:55 >>> net-libs/nodejs-12.16.1: 7′02″
2020-06-05T07:20:13 >>> net-libs/nodejs-14.2.0: 8′24″
2020-06-17T21:00:14 >>> net-libs/nodejs-14.4.0: 8′15″
2020-07-12T13:39:59 >>> net-libs/nodejs-14.4.0: 8′08″
2020-10-10T04:31:58 >>> net-libs/nodejs-14.11.0: 8′49″
2020-11-12T20:03:39 >>> net-libs/nodejs-14.15.0: 9′40″
2020-11-25T19:56:31 >>> net-libs/nodejs-14.15.1: 9′40″
2021-01-09T22:14:38 >>> net-libs/nodejs-14.15.4: 9′27″
2021-03-02T18:06:30 >>> net-libs/nodejs-14.16.0: 10′26″
2021-04-24T08:53:50 >>> net-libs/nodejs-14.16.1: 10′19″
2021-07-15T19:09:31 >>> net-libs/nodejs-14.16.1: 10′20″
2021-08-01T00:35:06 >>> net-libs/nodejs-14.17.3: 10′44″
2021-09-10T19:15:23 >>> net-libs/nodejs-14.17.6: 9′03″
2022-03-17T23:51:16 >>> net-libs/nodejs-14.17.6: 10′51″
2022-05-26T03:13:26 >>> net-libs/nodejs-16.14.2: 15′52″
2022-06-27T19:58:15 >>> net-libs/nodejs-18.3.0: 19′33″
2022-08-07T02:03:05 >>> net-libs/nodejs-18.6.0: 19′45″
2022-10-02T21:27:08 >>> net-libs/nodejs-18.9.1: 20′04″
2022-11-12T14:15:07 >>> net-libs/nodejs-18.12.1: 20′03″
2022-11-23T23:21:36 >>> net-libs/nodejs-18.12.1: 19′09″
2023-03-08T00:05:47 >>> net-libs/nodejs-18.14.2: 21′40″
2023-04-28T12:04:49 >>> net-libs/nodejs-18.16.0: 20′35″
2023-05-08T18:27:52 >>> net-libs/nodejs-18.16.0: 21′33″
2023-05-25T20:03:08 >>> net-libs/nodejs-20.2.0: 21′24″
2023-06-04T14:28:38 >>> net-libs/nodejs-20.2.0: 22′03″
2023-06-23T20:16:12 >>> net-libs/nodejs-20.2.0: 22′23″
2023-08-16T06:42:14 >>> net-libs/nodejs-20.5.1: 22′56″
2023-09-29T22:09:40 >>> net-libs/nodejs-20.6.1: 21′45″
2024-02-02T21:47:28 >>> net-libs/nodejs-20.11.0: 33′06″
2024-02-13T21:22:53 >>> net-libs/nodejs-20.11.0: 21′17″
2024-04-19T18:08:48 >>> net-libs/nodejs-20.12.1: 19′53″
2024-07-06T08:46:42 >>> net-libs/nodejs-22.3.0: 39′56″
2024-08-18T09:24:31 >>> net-libs/nodejs-22.4.1-r1: 40′31″
2024-10-29T20:25:10 >>> net-libs/nodejs-22.4.1-r1: 41′11″
2024-12-08T10:34:29 >>> net-libs/nodejs-22.4.1-r1: 41′50″
2024-12-23T14:23:58 >>> net-libs/nodejs-22.4.1-r1: 43′19″
2024-12-23T16:08:20 >>> net-libs/nodejs-22.4.1-r1: 42′32″
2024-12-27T14:59:39 >>> net-libs/nodejs-22.4.1-r1: 49′36″
2024-12-27T23:05:25 >>> net-libs/nodejs-22.4.1-r1: 41′04″
|
Ryzen 7 2700X, 8-core, 32G, with /var/tmp mounted at tmpfs |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3802
Location: Rasi, Finland
|
| Posted: Sun Dec 29, 2024 9:36 am Post subject: |
|
|
My poor ex-Chormebox. :(
Intel(R) Celeron(R) 2955U (2) @ 1.40 GHz only two slow cores with no hyperthreading.
Last time I went nuts and added all kinds of (stupid) optimization flags to nodejs. Including -lto which probably contributes most of the compilation time.
| Code: | | 2024-12-26T01:43:48 >>> net-libs/nodejs-22.4.1-r1: 15 hours, 32 minutes, 58 seconds |
Fun.
_________________
..: Zucca :..
| My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
| Quote: | | I am NaN! I am a man! |
Last edited by Zucca on Sun Dec 29, 2024 10:57 am; edited 1 time in total |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3802
Location: Rasi, Finland
|
| Posted: Sun Dec 29, 2024 9:53 am Post subject: |
|
|
| Perfect Gentleman wrote: | | i7-4770k |
Do you use ccache or some other magic?
My ThinkCentre with Intel(R) Core(TM) i5-7600T (4) @ 3.70 GHz
| Code: | | 2024-10-14T03:58:31 >>> net-libs/nodejs-22.4.1-r1: 2 hours, 7 minutes, 4 seconds |
EDIT: It seems that hyperthreading is the key in this case.
_________________
..: Zucca :..
| My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2536
|
| Posted: Sun Dec 29, 2024 10:28 am Post subject: |
|
|
| Zucca wrote: | My poor ex-Chormebox. :(
Intel(R) Celeron(R) 2955U (2) @ 1.40 GHz only two slow cores with no hyperthreading.
Last time I went nuts and added all kinds of (stupid) optimization flags to nodejs. Inluding -lto which probably contributes most of the compilation time.
| Code: | | 2024-12-26T01:43:48 >>> net-libs/nodejs-22.4.1-r1: 15 hours, 32 minutes, 58 seconds |
Fun. |
Do you really need nodejs? Do you really need to build it from source? The only reason I would build nodejs from source is because the binhost does not provide nodejs[inspector] which is a requirement to build chromium. However I've switched to Firefox and opted for ungoogled-chromium-bin as a backup, so I don't need nodejs[inspector] and can use the package from the binhost.
Best Regards,
Georgi |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3802
Location: Rasi, Finland
|
| Posted: Sun Dec 29, 2024 10:57 am Post subject: |
|
|
| logrusx wrote: | | Do you really need nodejs? Do you really need to build it from source? |
Probably no to both.
That box has "evolved" into just a basic test-box. It does still act as a some kind of firewall for my TV. I had plans to transform it into a home automation box. But since it just "sits there idling", I occasionally ssh into it and update it. Then it might run updates for over 24 hours, but the compilation time really doesn't matter there.
My last post generally showed that with low end machines that are "in use" where one does not have time too much to wait for packages to compile, really should ask the questions you asked me. ;)
_________________
..: Zucca :..
| My gentoo installs: | init=/sbin/openrc-init
-systemd -logind -elogind seatd |
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
Yamakuzure
Advocate
Joined: 21 Jun 2006
Posts: 2305
Location: Adendorf, Germany
|
| Posted: Sun Dec 29, 2024 11:04 am Post subject: |
|
|
For me it is qtwebengine: | Code: | ~ $ equery d net-libs/nodejs
* These packages depend on net-libs/nodejs:
dev-qt/qtwebengine-5.15.16_p20241115 (net-libs/nodejs[ssl])
dev-qt/qtwebengine-6.8.1 (net-libs/nodejs[ssl]) |
The price to pay for running Plasma...
_________________
Edited 220,176 times by Yamakuzure |
|
| Back to top |
|
|
keekkenen
n00b
Joined: 05 Oct 2024
Posts: 20
|
| Posted: Sun Dec 29, 2024 11:16 am Post subject: |
|
|
| Yamakuzure wrote: | | The price to pay for running Plasma... |
I use Plasma without qtwebengine (the way without pay)
_________________
7950x3d / x670e MSI Tomagawk / Sapphire RX 7800XT 16Gb / G.Skill 64Gb 5600 / A-Data Legend 960 2Tb (x2), A-Data SX8200PNP 256Gb |
|
| Back to top |
|
|
Josef.95
Advocate
Joined: 03 Sep 2007
Posts: 4682
Location: Germany
|
| Posted: Sun Dec 29, 2024 1:06 pm Post subject: |
|
|
| qlop -vH nodejs: | 2024-09-12T21:46:47 >>> net-libs/nodejs-22.4.1-r1: 22 minutes, 44 seconds
2024-11-10T23:51:29 >>> net-libs/nodejs-22.4.1-r1: 22 minutes, 16 seconds
2024-12-08T01:36:28 >>> net-libs/nodejs-22.4.1-r1: 22 minutes, 58 seconds |
(On a Thinkpad P16v Gen1 Laptop, with i7-13800H CPU, and 32GB RAM)
| Kaorukun wrote: | | Also, why would it need to recompile the same version several times? |
First rebuild here was with new dev-libs/icu-76.1-r1 (subslot change with USE=system-icu),
second rebuild was triggered with new dev-libs/simdjson-3.11.2 update with subslot change. |
|
| Back to top |
|
|
|
Portage & Programming
