| View previous topic :: View next topic |
| Author |
Message |
leyvi
Apprentice
Joined: 08 Sep 2023
Posts: 184
|
 Posted: Sun Mar 09, 2025 2:45 pm Post subject: app-shells/bash pgo way too long |
 |
|
| I have a pretty fast PC, and app-shells/bash has been in the PGO stage of building for the last 3.5 hours. Is that normal, or should I try starting the build process again? |
|
| Back to top |
|
 |
Banana
Moderator
Joined: 21 May 2004
Posts: 1921
Location: Germany
|
|
| Back to top |
|
|
leyvi
Apprentice
Joined: 08 Sep 2023
Posts: 184
|
| Posted: Sun Mar 09, 2025 3:15 pm Post subject: |
|
|
Yeah, I confirmed with htop that everything is running. These are the last couple lines of the build log: | Code: | make[1]: Entering directory '/var/tmp/portage/app-shells/bash-5.2_p37/work/bash-5.2/builtins'
make[1]: warning: -j12 forced in submake: resetting jobserver mode.
***********************************************************
make[1]: 'pipesize.h' is up to date.
make[1]: Leaving directory '/var/tmp/portage/app-shells/bash-5.2_p37/work/bash-5.2/builtins'
* *
* GNU bash, version 5.2.37(1)-release (x86_64-pc-linux-gnu)
* *
***********************************************************
declare -r SHELLOPTS="braceexpand:hashall:interactive-comments"
Testing /var/tmp/portage/app-shells/bash-5.2_p37/work/bash-5.2/bash
version: 5.2.37(1)-release
versinfo: 5 2 37 1 release x86_64-pc-linux-gnu
HOSTTYPE = x86_64
OSTYPE = linux-gnu
MACHTYPE = x86_64-pc-linux-gnu
Any output from any test, unless otherwise noted, indicates a possible anomaly
run-alias
run-appendop
run-arith
run-arith-for
run-array
warning: all of these tests will fail if arrays have not
warning: been compiled into the shell
warning: the BASH_ARGC and BASH_ARGV tests will fail if debugging support
warning: has not been compiled into the shell
run-array2
warning: all of these tests will fail if arrays have not
warning: been compiled into the shell
run-assoc
warning: all of these tests will fail if arrays have not
warning: been compiled into the shell
|
Also, here's the process tree: | Code: | UID PID PPID PGID SID C SZ RSS PSR STIME TTY TIME CMD
portage 6251 6248 6248 6248 0 656 1780 8 13:25 pts/3 00:00:00 [app-shells/bash-5.2_p37] sandbox /usr/lib/portage/python3.13/ebuild.sh compile
portage 6252 6251 6248 6248 0 2995 7380 0 13:25 pts/3 00:00:00 bash /usr/lib/portage/python3.13/ebuild.sh compile
portage 6273 6252 6248 6248 0 2996 6340 1 13:25 pts/3 00:00:00 bash /usr/lib/portage/python3.13/ebuild.sh compile
portage 7895 6273 6248 6248 0 2289 4696 9 13:25 pts/3 00:00:00 bash /usr/lib/portage/python3.13/ebuild-helpers/emake CFLAGS=-w -march=znver4 -mtune=znver4 -O3 -pipe -flto=full -std=gnu17 -fprofile-update=atomic -fprofile-dir=/var/tmp/portage/app
portage 7897 7895 6248 6248 0 1869 3692 1 13:25 pts/3 00:00:00 make -j12 -l12.0 CFLAGS=-w -march=znver4 -mtune=znver4 -O3 -pipe -flto=full -std=gnu17 -fprofile-update=atomic -fprofile-dir=/var/tmp/portage/app-shells/bash-5.2_p37/temp/pgo -fpro
portage 7971 7897 6248 6248 0 2157 4048 8 13:25 pts/3 00:00:00 /bin/sh -c ( cd ./tests && \ ?BUILD_DIR=/var/tmp/portage/app-shells/bash-5.2_p37/work/bash-5.2 PATH=/var/tmp/portage/app-shells/bash-5.2_p37/work/bash-5.2/tests:$PATH THIS_SH=/va
portage 7972 7971 6248 6248 0 2157 4244 7 13:25 pts/3 00:00:00 /bin/sh run-all
portage 8532 7972 6248 6248 0 2157 4080 5 13:25 pts/3 00:00:00 sh run-assoc
portage 8533 8532 6248 6248 0 2467 5116 2 13:25 pts/3 00:00:00 /var/tmp/portage/app-shells/bash-5.2_p37/work/bash-5.2/bash ./assoc.tests
portage 8561 8533 6248 6248 99 2435 5388 1 13:25 pts/3 03:57:46 /var/tmp/portage/app-shells/bash-5.2_p37/work/bash-5.2/bash ./assoc2.sub
|
|
|
| Back to top |
|
|
Banana
Moderator
Joined: 21 May 2004
Posts: 1921
Location: Germany
|
|
| Back to top |
|
|
sam_
Developer
Joined: 14 Aug 2020
Posts: 2200
|
| Posted: Mon Mar 10, 2025 6:39 am Post subject: |
|
|
| We had problems in the past where some of the bash tests (if you can call them that...) would deadlock with PGO, but I don't remember what I did about it in the end (I did make some remark in a commit message, I think). |
|
| Back to top |
|
|
leyvi
Apprentice
Joined: 08 Sep 2023
Posts: 184
|
| Posted: Mon Mar 10, 2025 8:06 am Post subject: |
|
|
OK, quick update:
Ran emerge overnight, bash profiling script ran for 8.5 hours and didn't finish. I don't think that's supposed to happen...
Here's the output of emerge --info: | Code: | Portage 3.0.67 (python 3.13.2-final-0, default/linux/amd64/23.0/split-usr/desktop/plasma, gcc-14, glibc-2.41, 6.13.5-gentoo x86_64)
=================================================================
System Settings
=================================================================
System uname: Linux-6.13.5-gentoo-x86_64-AMD_Ryzen_9_7900X_12-Core_Processor-with-glibc2.41
KiB Mem: 64485616 total, 715608 free
KiB Swap: 82313208 total, 80376724 free
Timestamp of repository gentoo: Mon, 10 Mar 2025 04:00:00 +0000
Head commit of repository gentoo: e637fd91e5a37440affa6818149feeecedae1f9d
Timestamp of repository guru: Sat, 08 Mar 2025 17:38:04 +0000
Head commit of repository guru: 0a33a0a7da0d2ece9e224c38019a038f7b578086
Timestamp of repository kde: Sat, 08 Mar 2025 17:04:22 +0000
Head commit of repository kde: 69dcea95c94a6e39b71a733f76df43009440c553
Timestamp of repository librewolf: Sat, 08 Mar 2025 14:19:13 +0000
Head commit of repository librewolf: afc2ab32929e9367574026f7912289d345cc79bb
Timestamp of repository pentoo: Mon, 10 Mar 2025 03:33:18 +0000
Head commit of repository pentoo: 1742de4bcb1a026213826ce8c842c3c9a0c5fb32
Timestamp of repository steam-overlay: Sat, 08 Mar 2025 17:04:20 +0000
Head commit of repository steam-overlay: 5f5031d1937ace119bb3839e9283c29238b0a634
Timestamp of repository waffle-builds: Sat, 08 Mar 2025 17:04:24 +0000
Head commit of repository waffle-builds: cf126602213d1314cdf46f2487d1e5ac55d70dc2
Timestamp of repository wayland-desktop: Sat, 08 Mar 2025 17:04:24 +0000
Head commit of repository wayland-desktop: 0e70afb8c2f48c26c23bb645c986a3c54d0aee09
Timestamp of repository x11: Sun, 16 Feb 2025 18:52:01 +0000
Head commit of repository x11: f632d3fce377d94054c4f597487bac82e61eb33e
sh bash 5.2_p37
ld GNU ld (Gentoo 2.44 p1) 2.44.0
ccache version 4.10.2 [enabled]
app-misc/pax-utils: 1.3.8::gentoo
app-shells/bash: 5.2_p37::gentoo
dev-build/autoconf: 2.13-r8::gentoo, 2.71-r7::gentoo, 2.72-r1::gentoo
dev-build/automake: 1.16.5-r2::gentoo, 1.17-r2::gentoo
dev-build/cmake: 3.31.6::gentoo
dev-build/libtool: 2.5.4::gentoo
dev-build/make: 4.4.1-r100::gentoo
dev-build/meson: 1.7.0::gentoo
dev-java/java-config: 2.3.4::gentoo
dev-lang/perl: 5.40.1::gentoo
dev-lang/python: 3.10.16_p2::gentoo, 3.11.11_p2::gentoo, 3.12.9::gentoo, 3.13.2::gentoo, 3.13.2-r100::gentoo
dev-lang/rust: 1.84.1-r1::gentoo
dev-lang/rust-bin: 1.85.0-r1::gentoo
dev-util/ccache: 4.10.2-r1::gentoo
llvm-core/clang: 19.1.7::gentoo, 20.1.0-r1::gentoo
llvm-core/lld: 17.0.6::gentoo, 19.1.7::gentoo, 20.1.0::gentoo
llvm-core/llvm: 17.0.6-r3::gentoo, 18.1.8-r6::gentoo, 19.1.7::gentoo, 20.1.0::gentoo
sys-apps/baselayout: 2.17::gentoo
sys-apps/openrc: 0.60.1::gentoo
sys-apps/sandbox: 2.45::gentoo
sys-devel/binutils: 2.44::gentoo
sys-devel/binutils-config: 5.5.2::gentoo
sys-devel/gcc: 14.2.1_p20250301::gentoo
sys-devel/gcc-config: 2.12.1::gentoo
sys-kernel/linux-headers: 6.13::gentoo (virtual/os-headers)
sys-libs/glibc: 2.41::gentoo
Repositories:
gentoo
location: /var/db/repos/gentoo
sync-type: rsync
sync-uri: rsync://rsync.gentoo.org/gentoo-portage
priority: -1000
volatile: False
sync-rsync-extra-opts:
sync-rsync-verify-max-age: 24
sync-rsync-verify-metamanifest: yes
sync-rsync-verify-jobs: 16
guru
location: /var/db/repos/guru
sync-type: git
sync-uri: https://github.com/gentoo-mirror/guru.git
masters: gentoo
volatile: False
kde
location: /var/db/repos/kde
sync-type: git
sync-uri: https://github.com/gentoo-mirror/kde.git
masters: gentoo
volatile: False
librewolf
location: /var/db/repos/librewolf
sync-type: git
sync-uri: https://github.com/gentoo-mirror/librewolf.git
masters: gentoo
volatile: False
my-ebuilds
location: /var/db/repos/my-ebuilds
masters: gentoo
volatile: False
pentoo
location: /var/db/repos/pentoo
sync-type: git
sync-uri: https://github.com/gentoo-mirror/pentoo.git
masters: gentoo
volatile: False
steam-overlay
location: /var/db/repos/steam-overlay
sync-type: git
sync-uri: https://github.com/gentoo-mirror/steam-overlay.git
masters: gentoo
volatile: False
waffle-builds
location: /var/db/repos/waffle-builds
sync-type: git
sync-uri: https://github.com/gentoo-mirror/waffle-builds.git
masters: gentoo
volatile: False
wayland-desktop
location: /var/db/repos/wayland-desktop
sync-type: git
sync-uri: https://github.com/gentoo-mirror/wayland-desktop.git
masters: gentoo
volatile: False
x11
location: /var/db/repos/x11
sync-type: git
sync-uri: https://github.com/gentoo-mirror/x11.git
masters: gentoo
volatile: False
Binary Repositories:
gentoobinhost
priority: 1
sync-uri: https://gentoo.osuosl.org/releases/amd64/binpackages/23.0/x86-64-v3
Installed sets: @esteam
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
AR="llvm-ar"
AS="llvm-as"
CBUILD="x86_64-pc-linux-gnu"
CC="clang"
CFLAGS="-w -march=znver4 -mtune=znver4 -O3 -pipe -flto=full"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt /var/bind /var/lib/i2pd/certificates"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php8.3/ext-active/ /etc/php/cgi-php8.3/ext-active/ /etc/php/cli-php8.3/ext-active/ /etc/php/fpm-php8.3/ext-active/ /etc/php/phpdbg-php8.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CPPFLAGS="-w -march=znver4 -mtune=znver4 -O3 -pipe -flto=full"
CXX="clang++"
CXXFLAGS="-w -march=znver4 -mtune=znver4 -O3 -pipe -flto=full"
DISTDIR="/mnt/Bulk-Storage/cache/distfiles"
EMERGE_DEFAULT_OPTS="--ask -j1 -l12.0 --backtrack=500 --keep-going=y --with-bdeps=y --complete-graph=y --columns"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-w -march=znver4 -mtune=znver4 -O3 -pipe -flto=full"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live ccache clean-logs config-protect-if-modified distlocks ebuild-locks fail-clean fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox strict test test-fail-continue unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-w -march=znver4 -mtune=znver4 -O3 -pipe -flto=full"
GENTOO_MIRRORS="https://mirror.isoc.org.il/pub/gentoo/ https://gentoo.mirror.garr.it/ http://gentoo.mirror.garr.it/ https://mirrors.evoluso.com/gentoo/ http://mirrors.evoluso.com/gentoo/ https://mirror.leaseweb.com/gentoo/ http://mirror.leaseweb.com/gentoo/ rsync://mirror.leaseweb.com/gentoo/ https://ftp.snt.utwente.nl/pub/os/linux/gentoo http://ftp.snt.utwente.nl/pub/os/linux/gentoo ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo rsync://ftp.snt.utwente.nl/gentoo"
LANG="en_US.utf8"
LD="ld.lld"
LDFLAGS="-Wl,--lto-O3 -fuse-ld=lld"
LEX="flex"
MAKEOPTS="-j12 -l12.0"
NM="llvm-nm"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
RANLIB="llvm-ranlib"
RUSTFLAGS="-C opt-level=3 -C target-cpu=znver4 -C lto"
SHELL="/bin/bash"
STRIP="llvm-strip"
USE="32 64 X a52 aac acl acpi activities alsa amd64 bash-completion bluetooth bluray boost bpf branding brotli bzip2 cairo cdda cdr cet clang crypt cups custom-cflags cxx dbus declarative dhcpcd doc dri dts dvd dvdr ebpf elogind encode exif fapi ffmpeg flac gdbm geoclue gif go graphite gtk gui gzip haptic hip http2 http3 hwaccel iconv icu introspection ipv4 ipv6 jit jpeg kde kf6compat lcms libass libnotify libreadline libtirpc llvm lto lz4 mad markdown matroska mng mp3 mp4 mpeg multilib ncurses network nfs nft nls nptl offload ogg opencl opencv opengl openmp openssl orc pango pcre pdf pgo pkcs11 plasma png policykit postproc ppds profile pulseaudio python qml qt6 rar readline rocm rust rustfmt samba screencast script sdl semantic-desktop sound spatialaudio spell split-usr sqlite ssl startup-notification strip svg syslog system-* test test-rust threads tiff tls tools tpm trayicon truetype udev udisks unicode upnp upower usb usbredir vaapi verify-sig vorbis vpx vulkan wayland webp websockets widgets wpa_supplicant wxwidgets x264 x265 x32 xattr xcb xft xml xv xvid xz zeroconf zip zlib" ABI_X86="64 32" ADA_TARGET="gcc_14" AMDGPU_TARGETS="gfx1102" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx avx2 avx512f avx512dq avx512cd avx512bw avx512vl avx512vbmi f16c fma3 mmx mmxext pclmul popcnt rdrand sha sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="efi-64" GUILE_SINGLE_TARGET="3-0" GUILE_TARGETS="3-0" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en-US en he" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LLVM_SLOT="20" LLVM_TARGETS="AArch64 AMDGPU ARC ARM AVR BPF CSKY DirectX Hexagon Lanai LoongArch M68k MSP430 Mips NVPTX PowerPC RISCV SPIRV Sparc SystemZ VE WebAssembly X86 XCore Xtensa" LUA_SINGLE_TARGET="luajit" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-2" POSTGRES_TARGETS="postgres16" PYTHON_SINGLE_TARGET="python3_12" PYTHON_TARGETS="python3_13 python3_12 python3_11 python3_10 pypy3 pypy3_11" RUBY_TARGETS="ruby34 ruby33 ruby32" VIDEO_CARDS="amdgpu radeonsi radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset: ADDR2LINE, ARFLAGS, ASFLAGS, CCLD, CONFIG_SHELL, CPP, CTARGET, CXXFILT, ELFEDIT, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, READELF, SIZE, STRINGS, YACC, YFLAGS
=================================================================
Package Settings
=================================================================
app-shells/bash-5.2_p37::gentoo was built with the following:
USE="net nls pgo (readline) verify-sig -afs -bashlogger -examples -mem-scramble -plugins" ABI_X86="(64)"
CFLAGS="-w -march=znver4 -mtune=znver4 -O2 -pipe -fuse-linker-plugin -flto -ftree-vectorize"
CXXFLAGS="-w -march=znver4 -mtune=znver4 -O2 -pipe -fuse-linker-plugin -flto -ftree-vectorize"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance buildpkg-live ccache config-protect-if-modified distlocks fail-clean fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"
|
|
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2821
|
| Posted: Mon Mar 10, 2025 9:23 am Post subject: |
|
|
Sorry for the of topic, what do you need optimized bash for?
Best Regards,
Georgi |
|
| Back to top |
|
|
leyvi
Apprentice
Joined: 08 Sep 2023
Posts: 184
|
| Posted: Mon Mar 10, 2025 9:46 am Post subject: |
|
|
| bash is running pretty much all the time on my system. Building bash with lots of optimization is a good way to save CPU time. |
|
| Back to top |
|
|
Banana
Moderator
Joined: 21 May 2004
Posts: 1921
Location: Germany
|
|
| Back to top |
|
|
leyvi
Apprentice
Joined: 08 Sep 2023
Posts: 184
|
| Posted: Mon Mar 10, 2025 12:02 pm Post subject: |
|
|
| I didn't quite do that; I tried building with GCC, LTO, and -O3, and everything worked fine, even with PGO (I was using LLVM before). Maybe enabling USE=pgo should require GCC to be used? |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2821
|
| Posted: Mon Mar 10, 2025 12:12 pm Post subject: |
|
|
| leyvi wrote: | | bash is running pretty much all the time on my system. Building bash with lots of optimization is a good way to save CPU time. |
Most of the time bash does nothing, what's to say something CPU intensive so you're saving close to nothing. It's not even arguable if the time wasted in optimization is made up for.
Do yourself a favor and put your time in something more productive. You're getting close to nothing if at all from optimizing bash.
Best Regards,
Georgi |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
