Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Trying to get distcc to work for cross-compiling
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
juliedeville
n00b
n00b


Joined: 14 Oct 2024
Posts: 27

PostPosted: Fri Oct 25, 2024 3:45 pm    Post subject: Trying to get distcc to work for cross-compiling Reply with quote

Hello,

I followed these instructions https://forums.gentoo.org/viewtopic-p-8844162.html#8844162 to setup distcc on two machines. I am not getting any useful errors or logs, and it does not appear to be compiling on both. Networking is setup, and I am able to ssh into the helper machine. I believe I had it working earlier, because I saw messages when merging a package, but it might have been when I was merging distcc itself.

I am not sure where I went wrong, any assistance is appreciated.
Back to top
View user's profile Send private message
pingtoo
Veteran
Veteran


Joined: 10 Sep 2021
Posts: 1213
Location: Richmond Hill, Canada

PostPosted: Fri Oct 25, 2024 4:14 pm    Post subject: Reply with quote

If you wish help from other, don't let other guessing :) You have yet sharing anything for us to start understand what is the problem. All we know is from your statement distcc is not working.

Please use the terminology in the linked post, show us what does it mean
Quote:
it does not appear to be compiling on both


What kind of machines both are?

What is cross compile target?

How cross compile tool chain setup?

Using your posted link as example, tell us the setup so we can have place to start. Don't expect everybody read through the link content.
Back to top
View user's profile Send private message
juliedeville
n00b
n00b


Joined: 14 Oct 2024
Posts: 27

PostPosted: Fri Oct 25, 2024 4:24 pm    Post subject: Reply with quote

Sorry, haha

One is a thinkpad t450s with an Intel i7-5600u, and one is an asus prebuilt desktop with an intel i7-7700

The target is the thinkpad

The profile on both machines is default/linux/amd64/23.0/hardened

I am still learning about tool chains
Back to top
View user's profile Send private message
juliedeville
n00b
n00b


Joined: 14 Oct 2024
Posts: 27

PostPosted: Fri Oct 25, 2024 4:34 pm    Post subject: Reply with quote

Okay, I added the line CC="distcc" to make.conf on the target, and before that, I tested to make sure distcc itself was working by writing a c program and compiling using distcc, and it worked, so it seems the issue is was just getting it to work with portage. Now, I get this error when running emerge

Code:
compiling source in /var/tmp/portage/app-text/tree-2.1.1-r1/work/unix-tree-2.1.1 ...
make -j9 -l5 CC=distcc 'CFLAGS=-O2 -pipe  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE' 'LDFLAGS=-Wl,-O1
-Wl,--as-needed -Wl,-z,pack-relative-relocs'
distcc -O2 -pipe  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -c -o tree.o tree.c
distcc -O2 -pipe  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -c -o list.o list.c
distcc -O2 -pipe  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -c -o hash.o hash.c
distcc[27] (dcc_talk_to_include_server) Warning: INCLUDE_SERVER_PORT not set - did you forget to run under 'pump'?
distcc[27] (dcc_build_somewhere) Warning: failed to get includes from include server, preprocessing locally
distcc -O2 -pipe  -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -c -o color.o color.c
distcc[28] (dcc_talk_to_include_server) Warning: INCLUDE_SERVER_PORT not set - did you forget to run under 'pump'?
distcc[28] (dcc_build_somewhere) Warning: failed to get includes from include server, preprocessing locally


I am not sure where to set "INCLUDE_SERVER_PORT"
Back to top
View user's profile Send private message
juliedeville
n00b
n00b


Joined: 14 Oct 2024
Posts: 27

PostPosted: Fri Oct 25, 2024 4:52 pm    Post subject: Reply with quote

I think I have got it now. I had the hosts file on the target set to "192.166.7.79,cpp, lzo." I removed "cpp," and I am no longer getting the error. I was under the impression that portage would return messages specifically for distcc at the beginning of an emerge process, and when I run "sudo distccmon-text 1" on the helper, it wasn't displaying anything in the output.

I think I have got it working now, because I can hear it compiling!
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54572
Location: 56N 3W

PostPosted: Fri Oct 25, 2024 6:17 pm    Post subject: Reply with quote

distccmon is run on the system needing help, not the helper.

distcc-pump has been broken for years. It needs the same (identical) include files on the helpers as the system needing help.
That allows the helpers local files to be used which saves network bandwidth.

Portage no longer supports pump mode.

Your topic title is misleading as you are not cross compiling.
Cross compiling would be using your i7 to build arm code for say, a raspberry pi.
That works too.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
juliedeville
n00b
n00b


Joined: 14 Oct 2024
Posts: 27

PostPosted: Sat Oct 26, 2024 3:46 pm    Post subject: Reply with quote

Oh I see. I misunderstood the term. Also sorry, I meant to type 'target' and not 'helper' regarding distccmon. Currently I am having an issue where the helper machine will randomly stop compiling..
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54572
Location: 56N 3W

PostPosted: Sat Oct 26, 2024 4:06 pm    Post subject: Reply with quote

You need distcc installed on both systems as it's in two part.
distccd, which runs on the helpers and listens for cries for help
By default it will start one distccd process for each logical core on the helper.

distccd need not be run on the system needing help.
local compilation will be used anyway when there are problems.
Hence, localhost need not be included in the list of helpers. If it's there at all, it should be last.

When you do not give a maximum nuber of concurrent jobs to distribute to helpers, the default is 4, as if you had used /4
If a helper can do more than that, by all means increase it.
But! MAKEOPS on the weaker system may need to pe increased too.
This is a double edged sword as not everything can be distributed and you don't want to swamp the weaker sysem when it has to do all the work.

You say that the helper randomly stops helping.
Does that mean what it says or does distribution stop?
There should be error messages.

Only C and C++ compile jobs can be distributed.
Preprocessing and linking will still be carried out locally.
All other code will still be built locallaly to ... e.g. rust.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
juliedeville
n00b
n00b


Joined: 14 Oct 2024
Posts: 27

PostPosted: Sun Oct 27, 2024 4:34 pm    Post subject: Reply with quote

Quote:
Only C and C++ compile jobs can be distributed.
Preprocessing and linking will still be carried out locally.
All other code will still be built locallaly to ... e.g. rust.


Oh I was unaware of that, that could be my issue. I am not sure though, because I was compiling gimp earlier, which is mostly written in C, and the helper was not helping at all. It is still going on the target.

Quote:

distccd need not be run on the system needing help.
local compilation will be used anyway when there are problems.
Hence, localhost need not be included in the list of helpers. If it's there at all, it should be last.


So I need it installed, but not running as a service?

Quote:

When you do not give a maximum nuber of concurrent jobs to distribute to helpers, the default is 4, as if you had used /4
If a helper can do more than that, by all means increase it.
But! MAKEOPS on the weaker system may need to pe increased too.
This is a double edged sword as not everything can be distributed and you don't want to swamp the weaker sysem when it has to do all the work.


Gotcha, thank you. Do you, by any chance, know how to determine the optimal number of jobs, and load on each machine in MAKEOPS, and in the distcc config on the helper?

Quote:
You say that the helper randomly stops helping.
Does that mean what it says or does distribution stop?
There should be error messages.

I think this might be because it is compiling non-C/C++ code, but I am not sure. I don't get any error messages, and distcc is still running on the helper, but it isn't utilizing the cores at all. I had it running yesterday using much more resources, but I am not sure what I did (if anything) to change it.

Sorry btw, I just recently switched to getnoo, and I have never used a source-based distro before. I appreciate you taking time out of your day to help.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9812
Location: almost Mile High in the USA

PostPosted: Sun Oct 27, 2024 4:59 pm    Post subject: Reply with quote

It is difficult to tell what the optimal number of jobs is -- it depends on the package and also within the package itself (which is difficult to change, so you have to carefully choose one that's not optimal for half of the run.)

Distcc is used only under certain circumstances and is the reason why it's not used often. The things to check:
- distcc is installed and FEATURES=distcc (which you got, but it needs to be checked!)
- networking is working (which you got, but needs to be checked!). ipv6 setup is annoying.
- Only works for gcc and clang, and cross compile works. However, things like xgcc, rust, haskell, perl, python, llvm-tblgen, ld-bfd, etc. don't get distributed.
- target machine must have same architecture cross or native gcc (arm vs i686 vs amd64 vs... mingw!). Clang is a bit different as it seems it has cross compile support built-in.
- target machine must have the same version of gcc enabled (make sure with gcc-config!), or same version of clang installed
- target machine with similar version (like gcc-13 vs gcc-11) may or may not be used depending on how different the preprocessed code is. I found that gcc about 50% of the time will still use mismatched gcc version. This percentage drops to near 0% for mismatched clang.

I hacked up a patch for distcc (distccmon-text, distccmon-gui) so that it works a bit better despite having pid namespaces enabled for portage (default). It's a dirty hack but "works for me" ... might have to post it somewhere if I haven't already, else you can disable pid namespaces (FEATURES=-pid-sandbox I think, and imho my dirty hack is cleaner than using -pid-sandbox but ymmv).
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum