| View previous topic :: View next topic |
| Author |
Message |
rsnfunky
Tux's lil' helper
Joined: 30 Dec 2007
Posts: 93
|
 Posted: Thu Jan 09, 2025 2:03 pm Post subject: Installation - Distcc, fixed IP and CFLAGS (make.conf) |
 |
|
Hi All,
I have 2 AMD PCs:
1) Old AMD Athlon II- 965 with 4 Cores (OpenRC)
2) Newer AMD 8700G with 8 cores (Systemd)
Facts (Aim to build faster on old PC):
1) I have installed distcc and binutils on both machines. They are the same versions:
a) GCC: sys-devel/gcc-14.2.1_p20241221
b) binutils: sys-devel/gcc-14.2.1_p20241221
2) Used the gentoo distcc guide and set up the servers on both PCs. Also on the Old PC setup the /etc/distcc/hosts file as: 192.168.1.101/16 (as the new PC has 8 cores, hence 16)
3) In the new PC have edited the allowed machines on /etc/systemd/system/distccd.service.d/00gentoo.conf as Environment="ALLOWED_SERVERS=127.0.0.1 --allow 192.168.1.105/16 --log-level error"
4) In the old PC, have edited the make.conf as: MAKEOPTS="-j24 -l8" and FEATURES="distcc" COMMON_FLAGS="-O2 -pipe -march=amdfam10"
5) I can ping the new PC and test mentioned on the gentoo guide also functions from the old PC
Issues:
1) The New PC's CPU utilization is not more than 15% ever (and that too intermittent) meaning that full 16 processes are not running on the new CPU and also distccmon-text shows nothing
2) How do I set a fixed IP on both the machines unable to do that on systemd one. Hence on every reboot need to set up the hosts file
3) Output of resolve-march-native gives a long output. Is there any benefits of the other parameters to be added to CFLAGS?
| Code: | resolve-march-native
-march=amdfam10 --param=l1-cache-line-size=64 --param=l1-cache-size=64 --param=l2-cache-size=512
|
Output of the test mentioned on gentoo guide:
| Code: | distcc gcc -c test.c -o test.o
distcc[25066] (dcc_trace_version) distcc 3.4 x86_64-pc-linux-gnu; built Jan 5 2025 09:46:49
distcc[25066] (dcc_recursion_safeguard) safeguard level=0
distcc[25066] (main) compiler name is "distcc"
distcc[25066] (dcc_scan_args) scanning arguments: gcc -c test.c -o test.o
distcc[25066] (dcc_scan_args) found input file "test.c"
distcc[25066] (dcc_scan_args) found object/output file "test.o"
distcc[25066] compile from test.c to test.o
distcc[25066] (dcc_gcc_rewrite_fqn) Re-writing call to 'gcc' to 'x86_64-pc-linux-gnu-gcc' to support cross-compilation.
distcc[25066] (dcc_get_hostlist) not reading /root/.distcc/hosts: No such file or directory
distcc[25066] (dcc_parse_hosts_file) load hosts from /etc/distcc/hosts
distcc[25066] (dcc_parse_hosts) found tcp token "192.168.1.101/16"
distcc[25066] (dcc_lock_host) got cpu lock on 192.168.1.101/16 slot 0 as fd3
distcc[25066] (dcc_lock_host) got cpu lock on localhost slot 0 as fd4
distcc[25066] (dcc_note_state) note state 3, file "(NULL)", host "localhost"
distcc[25066] (dcc_strip_dasho) result: x86_64-pc-linux-gnu-gcc -c test.c
distcc[25066] (dcc_spawn_child) forking to execute: x86_64-pc-linux-gnu-gcc -E test.c
distcc[25066] (dcc_spawn_child) child started as pid25067
distcc[25066] (dcc_strip_local_args) result: x86_64-pc-linux-gnu-gcc -c test.c -o test.o
distcc[25066] exec on 192.168.1.101/16: x86_64-pc-linux-gnu-gcc -c test.c -o test.o
distcc[25066] (dcc_note_state) note state 2, file "test.c", host "192.168.1.101"
distcc[25067] (dcc_new_pgrp) entered process group
distcc[25067] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc[25066] (dcc_connect_by_name_real) connecting to 192.168.1.101 port 3632
distcc[25066] (dcc_connect_by_addr) started connecting to 192.168.1.101:3632
distcc[25066] (dcc_select_for_write) select for write on fd5
distcc[25066] (dcc_note_state) note state 4, file "(NULL)", host "(NULL)"
distcc[25066] (dcc_x_token_int) send DIST00000001
distcc[25066] (dcc_x_token_int) send ARGC00000005
distcc[25066] (dcc_x_token_int) send ARGV00000017
distcc[25066] (dcc_x_token_string) send string 'x86_64-pc-linux-gnu-gcc'
distcc[25066] (dcc_x_token_int) send ARGV00000002
distcc[25066] (dcc_x_token_string) send string '-c'
distcc[25066] (dcc_x_token_int) send ARGV00000006
distcc[25066] (dcc_x_token_string) send string 'test.c'
distcc[25066] (dcc_x_token_int) send ARGV00000002
distcc[25066] (dcc_x_token_string) send string '-o'
distcc[25066] (dcc_x_token_int) send ARGV00000006
distcc[25066] (dcc_x_token_string) send string 'test.o'
distcc[25066] (dcc_note_state) note state 3, file "(NULL)", host "(NULL)"
distcc[25066] (dcc_collect_child) cpp child 25067 terminated with status 0
distcc[25066] (dcc_collect_child) cpp times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc[25066] cpp test.c on localhost completed ok
distcc[25066] (dcc_unlock) release lock fd4
distcc[25066] (dcc_x_file) send 20431 byte file /tmp/distcc_04f7d6a2.i with token DOTI and compression 69
distcc[25066] (dcc_x_token_int) send DOTI00004fcf
distcc[25066] (dcc_compile_remote) client finished sending request to server
distcc[25066] (dcc_note_state) note state 5, file "(NULL)", host "192.168.1.101"
distcc[25066] (dcc_select_for_read) select for read on fd5 for 300s
distcc[25066] (dcc_r_token_int) got DONE00000001
distcc[25066] (dcc_r_result_header) got response header
distcc[25066] (dcc_note_state) note state 6, file "(NULL)", host "(NULL)"
distcc[25066] (dcc_r_token_int) got STAT00000000
distcc[25066] (dcc_r_token_int) got SERR00000000
distcc[25066] (dcc_r_file) received 0 bytes to file /tmp/distcc_server_stderr_06bed6a2.txt
distcc[25066] (dcc_r_token_int) got SOUT00000000
distcc[25066] (dcc_r_token_int) got DOTO00000610
distcc[25066] (dcc_r_file) received 1552 bytes to file test.o
distcc[25066] (dcc_r_file_timed) 1552 bytes received in 0.000346s, rate 4380kB/s
distcc[25066] 20431 bytes from test.c compiled on 192.168.1.101 in 0.1296s, rate 154kB/s
distcc[25066] (dcc_unlock) release lock fd3
distcc[25066] compile test.c on 192.168.1.101/16 completed ok
distcc[25066] elapsed compilation time 0.131154s
distcc[25066] (dcc_exit) exit: code 0; self: 0.000000 user 0.008094 sys; children: 0.044767 user 0.007309 sys
distcc[25066] (dcc_cleanup_tempfiles_inner) deleted 2 temporary files
|
|
|
| Back to top |
|
 |
pingtoo
Veteran
Joined: 10 Sep 2021
Posts: 1368
Location: Richmond Hill, Canada
|
| Posted: Thu Jan 09, 2025 4:54 pm Post subject: Re: Installation - Distcc, fixed IP and CFLAGS (make.conf) |
|
|
| rsnfunky wrote: | Issues:
1) The New PC's CPU utilization is not more than 15% ever (and that too intermittent) meaning that full 16 processes are not running on the new CPU and also distccmon-text shows nothing
2) How do I set a fixed IP on both the machines unable to do that on systemd one. Hence on every reboot need to set up the hosts file
3) Output of resolve-march-native gives a long output. Is there any benefits of the other parameters to be added to CFLAGS?
| Code: | resolve-march-native
-march=amdfam10 --param=l1-cache-line-size=64 --param=l1-cache-size=64 --param=l2-cache-size=512
|
|
1) Your expectation for distccc is wrong. distcc can only work on C/C++ related compile task. Not all gentoo packages are C/C++ based. And certainly the entire emerge process is not always compiling something so some of emerge task will not distribute to distcc helper. And distccmon-text is based on your sampling rate, it is very possible that it cannot capture the transaction because the transaction period is too short and landed in between sampling step.
2) depend on which Network Management you are using there are different way to configure static IP (or you can setup dhcp server to always serve same IP for same MAC)
3) When in use with distcc, using march/mcpu=native is dangours because distcc helper may have different CPU subarch than you target, therefor get the right CFLAGS for your target is necessary. However if you choose disable distcc than use march/mcpu=native is better (in case you target machine have performance/efficient cores)
I think you really need to compare the entire build time between with distcc helper and without to understand if using distcc have any benefit.
I use distcc because my systems are SBC which have limited memory and use SD card for file system. so using distcc help use remote memory for C/C++ compiling task and reduce some of I/O to my SD cards. if the build ran faster just a side effect for my goal. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Installing Gentoo
