| View previous topic :: View next topic |
| Author |
Message |
Tin
Guru
Joined: 22 Dec 2005
Posts: 305
Location: Namur, Belgium
|
 Posted: Tue Apr 08, 2008 6:50 pm Post subject: does distcc need specific kernel stuff ? |
 |
|
Hi everybody,
I am using gentoo and distcc on two machines from more than two years now, without any problem.
I just added a new host in my home network and configure it like the other (copy of make.conf) and adapt some arch parameters because the first two machines are pentium 4 and pentium M, and the new one is an amd athlon.
Here are my three machines :
diva (192.168.1.2)
rebel (192.168.1.3)
junior (192.168.1.7) (the new one)
| diva: |
diva ~ # cat /etc/make.conf
# These settings were set by the catalyst build script that automatically built this stage
# Please consult /etc/make.conf.example for a more detailed example
CFLAGS="-Os -march=pentium4 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j7"
FEATURES="parallel-fetch distcc"
diva ~ # distcc-config --get-hosts
localhost 192.168.1.3 192.168.1.7
|
| rebel: |
rebel ~ # cat /etc/make.conf
CFLAGS="-Os -march=pentium-m -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j7"
FEATURES="parallel-fetch distcc"
rebel ~ # distcc-config --get-hosts
localhost 192.168.1.2 192.168.1.7
|
| junior: |
junior ~ # cat /etc/make.conf
# These settings were set by the catalyst build script that automatically
# built this stage.
# Please consult /etc/make.conf.example for a more detailed example.
CFLAGS="-Os -march=athlon-xp -pipe -fomit-frame-pointer"
CXXFLAGS="${CFLAGS}"
# This should not be changed unless you know exactly what you are doing. You
# should probably be using a different stage, instead.
CHOST="i686-pc-linux-gnu"
MAKEOPTS="-j7"
junior ~ # distcc-config --get-hosts
localhost 192.168.1.2 192.168.1.3
|
I set the log level to debug on the three hosts to be sure that there was no connection error, and I do a "tail -f /tmp/distccd.log &", so I a sure not to miss some info (I set the log file to /tmp/distccd.log on all hosts)
When I compile on diva
The three hosts are compiling together, the debug are filling my konsole buffers
When I compile on rebel
same thing than on diva, the three hosts are compiling (great isn't it)
When I compile on junior
junior is compiling alone
NO distcc error message in the emerge (no message like "distcc failed to distribute : connection refused" or such kind of stuff)
NO messages in the distccd logs on diva or rebel
Of course, the three distccd are running (checking with top -u distcc) because they are running fine when I compile on rebel or diva.
So the only difference between the installation are :
For junior kernel compilation, I start with an empty kernel and add only what I needed (great to have a 1.1Mb kernel, even for sport only)
I had to add by hand pci, ide support, tcp-ip, packet networking, bla bla bla, so, ....
I guess that ONE parameter in the kernel is missing for distcc to be able to "broadcast" ?
I googled about distcc and kernel but found nothing about specific needs for distcc.
If you need my .config file, tell me
(I don't like to post it because it is so big....)
Thank you very much for your precious help 
_________________
Tin, the gentoobie |
|
| Back to top |
|
 |
cyrillic
Watchman
Joined: 19 Feb 2003
Posts: 7313
Location: Groton, Massachusetts USA
|
| Posted: Tue Apr 08, 2008 7:36 pm Post subject: |
|
|
| Distcc doesn't need anything special in the kernel, you just need to add FEATURES="distcc" to junior's /etc/make.conf |
|
| Back to top |
|
|
Tin
Guru
Joined: 22 Dec 2005
Posts: 305
Location: Namur, Belgium
|
| Posted: Tue Apr 08, 2008 7:40 pm Post subject: |
|
|
| cyrillic wrote: | | Distcc doesn't need anything special in the kernel, you just need to add FEATURES="distcc" to junior's /etc/make.conf |
lol of course I did some cleaning in the /etc/make.conf files before put them with all the use flags on the forum, and I deleted this line on junior :
FEATURES="parallel-fetch distcc"
Sorry for my stupidity, I should triple-check my post before click on "post this message"
So, the distcc feature was present since the beginning (because I simply copied the /etc/make.conf and adapt only for the processor difference point of view)
I did not post the entire content of the file to make it easier to read.
_________________
Tin, the gentoobie |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54808
Location: 56N 3W
|
| Posted: Tue Apr 08, 2008 7:48 pm Post subject: |
|
|
Tin,
With the given
| Code: | # distcc-config --get-hosts
localhost 192.168.1.2 192.168.1.3 |
junior working alone may be right.
The helpers only compile if they are given work to do. and the server (giving out jobs) has to do all the preprocessing and linking.
It follows that as soon as a preprocessed package is available, it shuold be sent to a helper, not compiled locally.
From this, the fastest helper should be first in the list and locahost should be last, so that localhost only compiles when the helpers are all busy (otherwise it only preprocesses and links)
You configs have the opposite of that. Work that could be shared is first assigned to localhost.
Slower hosts may never be able to distribute configured this way.
Its not clear from your post if you apply job limits to each helper. That can help to distribute work more evenly.
The syntax is hostname/jobs. zero is a valid value for jobs. localhost/0 says to never build on localhost, which means nothing happens if helpers cannot be reached.
Lastly, parallel make is broken on some packages, so the ebuilds force -j1.
With your config, helpers would never be used for such a package.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Tin
Guru
Joined: 22 Dec 2005
Posts: 305
Location: Namur, Belgium
|
| Posted: Tue Apr 08, 2008 8:06 pm Post subject: |
|
|
I understand that localhost should be the last one, but be sure that with a -j7 makeopt and building the entire kde or things like mplayer or koffice, there are enough things to compile to fill up the three hosts 
I put localhost as last host, restart distccd but no change for the moment.
Even with localhost NOT set into the distccd hosts, no messages, no error, no distributed compilation.
I guess that this messages are not important :
| Code: |
junior linux # /etc/init.d/distccd restart
* Stopping distccd ... [ ok ]
* Starting distccd ...
distccd[18808] (dcc_should_be_inetd) stdin is a tty; assuming --daemon mode
distccd[18808] (dcc_discard_root) discarded root privileges, changed to uid=240 gid=2 [ ok ]
|
This is really puzzling because junior is really compiling when both the other are emerging, and when junior is compiling, it is alone, without error, warning, anything to give me any idea...
_________________
Tin, the gentoobie |
|
| Back to top |
|
|
Tin
Guru
Joined: 22 Dec 2005
Posts: 305
Location: Namur, Belgium
|
| Posted: Tue Apr 08, 2008 8:45 pm Post subject: |
|
|
I guess you were right, as I saw some jobs dist-compiled from junior to diva ! VICTORY !
I will double-check the parameters on rebel, because there are nothing in its logs, but I feel that your explanation was right.
Anyway, for a new installation, there are a LOT to compile, so it is a bit puzzling to see that with 3 machines, only one is used, and the second just a bit.
Thank you for your help, I will continue to check the compilations and try to find out if there are not other reason why rebel and diva are not compiling (or just a bit).
Perhaps if I put -j 13 the work will be distributed
I will post here if I find anything interesting.
_________________
Tin, the gentoobie |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54808
Location: 56N 3W
|
| Posted: Tue Apr 08, 2008 8:59 pm Post subject: |
|
|
Tin,
It goes like this on all your boxes. System does the configure stage then runs gcc locally to build what it just configured.
While thats happening, it does the next configure, if the local job count is not exceeded, it runs that locally too.
The next configure starts ... you now have gcc * 2, a configure and maybe some linking going on and nothing has been sent to helpers.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Networking & Security
