| View previous topic :: View next topic |
| Author |
Message |
nurali
Apprentice
Joined: 17 Nov 2022
Posts: 179
Location: Somewhere,Earth
|
 Posted: Thu Jun 20, 2024 1:49 am Post subject: Go programs build take too long |
 |
|
Hello all:
Are there special make.conf variables for go programs?
Seems like -j12 in make.conf is not working for go-written programs in my computer
I just installed gentoo on my new home server(not finished yet, stucking on go), I did not see any output on console for a long time, and I checked htop, found only one logical CPU is 100%
Any solution? |
|
| Back to top |
|
 |
kgdrenefort
Apprentice
Joined: 19 Sep 2023
Posts: 295
Location: Somewhere in the 77
|
| Posted: Thu Jun 20, 2024 6:24 am Post subject: |
|
|
Hello,
If you really think you -j12 value isn't effective for that, seems weird but I'm not experienced enough to state if it's goofy, so let's try something else.
Pick up a program with this behaviour, runs emerge while using:
| Code: | | MAKEOPTS="-j1" emerge yourcommand |
If it's taking way more time, indeed the j value for make is taken as it should.
Some programs, while compiled, don't use as much as other. Compiling GCC is way heavier than htop for example, it won't use as much ressources as GCC will ask for.
Hope it helps.
Regards,
GASPARD DE RENEFORT Kévin
_________________
Traduction wiki, pour praticiper.
Custom logos/biz card/website. |
|
| Back to top |
|
|
Juippisi
Developer
Joined: 30 Sep 2005
Posts: 742
Location: /home
|
| Posted: Thu Jun 20, 2024 7:01 am Post subject: |
|
|
Go programs are generally slow because you'll have to download all the go modules which does take time. I'm not sure if it helps, but try FEATURES="parallel-fetch". Then, all the modules must be compiled one-by-one (because they depend on each other) which is slow. Although compiling one module should utilize compiler parallelization, the whole process is still slow.
tail -f the build log to see what's happening. I'd imagine it's the download phase that takes time for you.
EDIT: actually I just remembered we don't allow listing the modules anymore, but they're tarballed. Those "deps" may still be a rather big download and unpacking. |
|
| Back to top |
|
|
nurali
Apprentice
Joined: 17 Nov 2022
Posts: 179
Location: Somewhere,Earth
|
| Posted: Thu Jun 20, 2024 11:25 am Post subject: |
|
|
| kgdrenefort wrote: | Hello,
If you really think you -j12 value isn't effective for that, seems weird but I'm not experienced enough to state if it's goofy, so let's try something else.
Pick up a program with this behaviour, runs emerge while using:
| Code: | | MAKEOPTS="-j1" emerge yourcommand |
If it's taking way more time, indeed the j value for make is taken as it should.
Some programs, while compiled, don't use as much as other. Compiling GCC is way heavier than htop for example, it won't use as much ressources as GCC will ask for.
Hope it helps.
Regards,
GASPARD DE RENEFORT Kévin |
Thanks for replying
That is a good idea, I can not test it right now, but maybe later I can try that as you said
It took me one and half hour to compile dev-lang/go with -j12 set, only thing I can do was staring at that one single thread running 100%, makes me feel like: other 15 was saying:hey look at this one, working so hard, let's dance around him.... |
|
| Back to top |
|
|
nurali
Apprentice
Joined: 17 Nov 2022
Posts: 179
Location: Somewhere,Earth
|
| Posted: Thu Jun 20, 2024 11:35 am Post subject: |
|
|
| Juippisi wrote: | Go programs are generally slow because you'll have to download all the go modules which does take time. I'm not sure if it helps, but try FEATURES="parallel-fetch". Then, all the modules must be compiled one-by-one (because they depend on each other) which is slow. Although compiling one module should utilize compiler parallelization, the whole process is still slow.
tail -f the build log to see what's happening. I'd imagine it's the download phase that takes time for you.
EDIT: actually I just remembered we don't allow listing the modules anymore, but they're tarballed. Those "deps" may still be a rather big download and unpacking. |
Thank you for replying
I am a bit confused actually, the package is in main tree, shouldn't it's dep modules are in distfiles? If they are, there should not be a fetching problem, just like some package's crates, or pip modules comes from main tree
Or Go has features like fetching modules in compile time?
This is the output when it was stuck and only one thread is working
| Code: | | Building Go bootstrap cmd/go (go_ bootstrap) using GO toolchain1 |
|
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22447
|
| Posted: Thu Jun 20, 2024 12:14 pm Post subject: |
|
|
| Most build systems understand at least enough of GNU Make's options that -j should work. When this was slow, was the build process actually invoked with -j12? If not, it would almost certainly be single-threaded. If it was invoked that way, then your issue is that you hit a step that does not run in parallel even when encouraged to do so. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
