View previous topic :: View next topic |
Author |
Message |
juliedeville n00b
Joined: 14 Oct 2024 Posts: 39
|
Posted: Fri Oct 25, 2024 3:45 pm Post subject: Trying to get distcc to work for cross-compiling |
|
|
Hello,
I followed these instructions https://forums.gentoo.org/viewtopic-p-8844162.html#8844162 to setup distcc on two machines. I am not getting any useful errors or logs, and it does not appear to be compiling on both. Networking is setup, and I am able to ssh into the helper machine. I believe I had it working earlier, because I saw messages when merging a package, but it might have been when I was merging distcc itself.
I am not sure where I went wrong, any assistance is appreciated. |
|
Back to top |
|
|
pingtoo Veteran
Joined: 10 Sep 2021 Posts: 1325 Location: Richmond Hill, Canada
|
Posted: Fri Oct 25, 2024 4:14 pm Post subject: |
|
|
If you wish help from other, don't let other guessing You have yet sharing anything for us to start understand what is the problem. All we know is from your statement distcc is not working.
Please use the terminology in the linked post, show us what does it mean Quote: | it does not appear to be compiling on both |
What kind of machines both are?
What is cross compile target?
How cross compile tool chain setup?
Using your posted link as example, tell us the setup so we can have place to start. Don't expect everybody read through the link content. |
|
Back to top |
|
|
juliedeville n00b
Joined: 14 Oct 2024 Posts: 39
|
Posted: Fri Oct 25, 2024 4:24 pm Post subject: |
|
|
Sorry, haha
One is a thinkpad t450s with an Intel i7-5600u, and one is an asus prebuilt desktop with an intel i7-7700
The target is the thinkpad
The profile on both machines is default/linux/amd64/23.0/hardened
I am still learning about tool chains |
|
Back to top |
|
|
juliedeville n00b
Joined: 14 Oct 2024 Posts: 39
|
Posted: Fri Oct 25, 2024 4:34 pm Post subject: |
|
|
Okay, I added the line CC="distcc" to make.conf on the target, and before that, I tested to make sure distcc itself was working by writing a c program and compiling using distcc, and it worked, so it seems the issue is was just getting it to work with portage. Now, I get this error when running emerge
Code: | compiling source in /var/tmp/portage/app-text/tree-2.1.1-r1/work/unix-tree-2.1.1 ...
make -j9 -l5 CC=distcc 'CFLAGS=-O2 -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE' 'LDFLAGS=-Wl,-O1
-Wl,--as-needed -Wl,-z,pack-relative-relocs'
distcc -O2 -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -c -o tree.o tree.c
distcc -O2 -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -c -o list.o list.c
distcc -O2 -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -c -o hash.o hash.c
distcc[27] (dcc_talk_to_include_server) Warning: INCLUDE_SERVER_PORT not set - did you forget to run under 'pump'?
distcc[27] (dcc_build_somewhere) Warning: failed to get includes from include server, preprocessing locally
distcc -O2 -pipe -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -c -o color.o color.c
distcc[28] (dcc_talk_to_include_server) Warning: INCLUDE_SERVER_PORT not set - did you forget to run under 'pump'?
distcc[28] (dcc_build_somewhere) Warning: failed to get includes from include server, preprocessing locally |
I am not sure where to set "INCLUDE_SERVER_PORT" |
|
Back to top |
|
|
juliedeville n00b
Joined: 14 Oct 2024 Posts: 39
|
Posted: Fri Oct 25, 2024 4:52 pm Post subject: |
|
|
I think I have got it now. I had the hosts file on the target set to "192.166.7.79,cpp, lzo." I removed "cpp," and I am no longer getting the error. I was under the impression that portage would return messages specifically for distcc at the beginning of an emerge process, and when I run "sudo distccmon-text 1" on the helper, it wasn't displaying anything in the output.
I think I have got it working now, because I can hear it compiling! |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54623 Location: 56N 3W
|
Posted: Fri Oct 25, 2024 6:17 pm Post subject: |
|
|
distccmon is run on the system needing help, not the helper.
distcc-pump has been broken for years. It needs the same (identical) include files on the helpers as the system needing help.
That allows the helpers local files to be used which saves network bandwidth.
Portage no longer supports pump mode.
Your topic title is misleading as you are not cross compiling.
Cross compiling would be using your i7 to build arm code for say, a raspberry pi.
That works too. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
juliedeville n00b
Joined: 14 Oct 2024 Posts: 39
|
Posted: Sat Oct 26, 2024 3:46 pm Post subject: |
|
|
Oh I see. I misunderstood the term. Also sorry, I meant to type 'target' and not 'helper' regarding distccmon. Currently I am having an issue where the helper machine will randomly stop compiling.. |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54623 Location: 56N 3W
|
Posted: Sat Oct 26, 2024 4:06 pm Post subject: |
|
|
You need distcc installed on both systems as it's in two part.
distccd, which runs on the helpers and listens for cries for help
By default it will start one distccd process for each logical core on the helper.
distccd need not be run on the system needing help.
local compilation will be used anyway when there are problems.
Hence, localhost need not be included in the list of helpers. If it's there at all, it should be last.
When you do not give a maximum nuber of concurrent jobs to distribute to helpers, the default is 4, as if you had used /4
If a helper can do more than that, by all means increase it.
But! MAKEOPS on the weaker system may need to pe increased too.
This is a double edged sword as not everything can be distributed and you don't want to swamp the weaker sysem when it has to do all the work.
You say that the helper randomly stops helping.
Does that mean what it says or does distribution stop?
There should be error messages.
Only C and C++ compile jobs can be distributed.
Preprocessing and linking will still be carried out locally.
All other code will still be built locallaly to ... e.g. rust. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
juliedeville n00b
Joined: 14 Oct 2024 Posts: 39
|
Posted: Sun Oct 27, 2024 4:34 pm Post subject: |
|
|
Quote: | Only C and C++ compile jobs can be distributed.
Preprocessing and linking will still be carried out locally.
All other code will still be built locallaly to ... e.g. rust. |
Oh I was unaware of that, that could be my issue. I am not sure though, because I was compiling gimp earlier, which is mostly written in C, and the helper was not helping at all. It is still going on the target.
Quote: |
distccd need not be run on the system needing help.
local compilation will be used anyway when there are problems.
Hence, localhost need not be included in the list of helpers. If it's there at all, it should be last.
|
So I need it installed, but not running as a service?
Quote: |
When you do not give a maximum nuber of concurrent jobs to distribute to helpers, the default is 4, as if you had used /4
If a helper can do more than that, by all means increase it.
But! MAKEOPS on the weaker system may need to pe increased too.
This is a double edged sword as not everything can be distributed and you don't want to swamp the weaker sysem when it has to do all the work. |
Gotcha, thank you. Do you, by any chance, know how to determine the optimal number of jobs, and load on each machine in MAKEOPS, and in the distcc config on the helper?
Quote: | You say that the helper randomly stops helping.
Does that mean what it says or does distribution stop?
There should be error messages.
|
I think this might be because it is compiling non-C/C++ code, but I am not sure. I don't get any error messages, and distcc is still running on the helper, but it isn't utilizing the cores at all. I had it running yesterday using much more resources, but I am not sure what I did (if anything) to change it.
Sorry btw, I just recently switched to getnoo, and I have never used a source-based distro before. I appreciate you taking time out of your day to help. |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9838 Location: almost Mile High in the USA
|
Posted: Sun Oct 27, 2024 4:59 pm Post subject: |
|
|
It is difficult to tell what the optimal number of jobs is -- it depends on the package and also within the package itself (which is difficult to change, so you have to carefully choose one that's not optimal for half of the run.)
Distcc is used only under certain circumstances and is the reason why it's not used often. The things to check:
- distcc is installed and FEATURES=distcc (which you got, but it needs to be checked!)
- networking is working (which you got, but needs to be checked!). ipv6 setup is annoying.
- Only works for gcc and clang, and cross compile works. However, things like xgcc, rust, haskell, perl, python, llvm-tblgen, ld-bfd, etc. don't get distributed.
- target machine must have same architecture cross or native gcc (arm vs i686 vs amd64 vs... mingw!). Clang is a bit different as it seems it has cross compile support built-in.
- target machine must have the same version of gcc enabled (make sure with gcc-config!), or same version of clang installed
- target machine with similar version (like gcc-13 vs gcc-11) may or may not be used depending on how different the preprocessed code is. I found that gcc about 50% of the time will still use mismatched gcc version. This percentage drops to near 0% for mismatched clang.
I hacked up a patch for distcc (distccmon-text, distccmon-gui) so that it works a bit better despite having pid namespaces enabled for portage (default). It's a dirty hack but "works for me" ... might have to post it somewhere if I haven't already, else you can disable pid namespaces (FEATURES=-pid-sandbox I think, and imho my dirty hack is cleaner than using -pid-sandbox but ymmv). _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54623 Location: 56N 3W
|
Posted: Fri Nov 01, 2024 10:02 am Post subject: |
|
|
There is no single optimal MAKEOPTS.
e.g. gcc is written in C++ but it mostly cannot use distcc due to the way it builds
Pass 1 builds C and C++ with any random compiler on the system.
Pass 2 does the same thing but uses the compiler built in pass 1. That only exists on the local host.
Pass 3 builds C and C++ using the compiler from pass 2, then compares the outputs of pass 2 and pass 3.
If all is well, the other languages are built and its all installed.
It is possible to set MAKEOPTS on a per package basis, or you could run your own binhost by building in a chroot on one of your helpers.
That would not use distcc and the more powerful box would lend it's power to everything.
There is a wiki page. I would normally post a link but I'm on my phone right now. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9838 Location: almost Mile High in the USA
|
Posted: Sun Dec 01, 2024 5:41 pm Post subject: |
|
|
Rust is also one of those annoying packages like gcc
Phase 0: Build local copy of LLVM using any C compiler. This can be distributed.
Phase 1: build Rust with (originally portage-downloaded) downloaded copy of rust binary, or (currently) rust-bin. This technically can be distributed alas there is no distrust today, so it's run local.
Phase 2: Build rust with the newly compiled rust. Just like gcc, this necessarily is local and cannot be distributed.
There is no MAKEOPTS that will optimize this.
One of these days perhaps cluster computing would be part of portage and if the machine needing compilation help's filesystems are NFS mounted, then it would be possible for the helper to run whatever command is needed...alas it would open another can of worms. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|