| View previous topic :: View next topic |
| Author |
Message |
mattst88
Developer
Joined: 28 Oct 2004
Posts: 422
|
 Posted: Wed Jul 24, 2019 4:18 am Post subject: no-distcc-env: disable distcc on a per-package basis |
 |
|
I have some slow systems that benefit a lot from FEATURES="distcc" (and perhaps FEATURES="distcc-pump" but not sure), but setting this often leads to problems.
Packages fail to compile under distcc sometimes. Worse, sometimes a package doesn't benefit from distcc and your high MAKEOPTS value now overloads the poor system that thought it was going to get some help.
I've created app-portage/no-distcc-env: a collection of package.env files to disable distcc on a per-package basis. With this installed, the hope is to make it safe to use FEATURES="distcc distcc-pump" globally. Set LOCAL_MAKEOPTS=... to a reasonable value for your system alone, and MAKEOPTS=... for your distcc network.
For example, if my slow system has a single-core CPU but I have 20 CPUs among all the machines in my distcc network, in make.conf I would set
| Code: | FEATURES="... distcc ..."
MAKEOPTS="-j20"
LOCAL_MAKEOPTS="-j1" |
By default packages will build with distcc, but if one doesn't (e.g., dev-libs/libunistring, bug 620738) it'll automatically have FEATURES="-distcc" and will replace MAKEOPTS with LOCAL_MAKEOPTS.
The package.env files installed by the package are named in the typical cate-gory/package style. E.g., /etc/portage/package.env/no-distcc/dev-libs/libunistring, which contains
| Code: | | dev-libs/libunistring no-distcc.conf # Bug 620738 |
This way, using INSTALL_MASK (see "man 5 make.conf") you can selectively disable the installation of these files for specific packages or categories if you're feeling adventurous.
My hope is that people will contribute to the project as they find packages that are incompatible with distcc (but the best method of contributing remains to be seen). |
|
| Back to top |
|
 |
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9830
Location: almost Mile High in the USA
|
| Posted: Wed Jul 24, 2019 2:05 pm Post subject: |
|
|
I know it's probably bad style to reply to tips and tricks, but what about the packages that are simply temporarily do not build properly with distcc?
Some packages continually fail, some packages fail for some people and not others (like www-client/firefox). Ideally it's dynamic, perhaps portage should simply retry packages that fail - if it notices distcc is enabled, disable it and retry? Maybe only in conjunction with --keep-going or perhaps another option to allow alerting of the user that distcc is failing and should be investigated?
As for specific packages, I still am not too sure of the status of waf (sys-libs/tdb sys-libs/ldb sys-libs/talloc sys-libs/tevent and specifically net-fs/samba) because theoretically it should work -- but I have not had much success for many months building through portage with distcc enabled.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
bunder
Bodhisattva
Joined: 10 Apr 2004
Posts: 5937
|
| Posted: Wed Jul 24, 2019 3:16 pm Post subject: |
|
|
i used to use distcc quite a bit, it's nice to see people still using it, this sounds like it would work, but i wonder if this would be easier to set up in an eclass similar to flag-o-matic's strip-flags...
_________________
| Neddyseagoon wrote: | | The problem with leaving is that you can only do it once and it reduces your influence. |
banned from #gentoo since sept 2017 |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
