View previous topic :: View next topic |
Author |
Message |
tld Veteran
Joined: 09 Dec 2003 Posts: 1848
|
Posted: Thu Nov 28, 2024 1:16 pm Post subject: Full rebuild required on gcc-14, gcc-15? |
|
|
[Administrator note: this post, and the first 12 responses, were originally attached to the topic gcc-14 - cannot build sys-apps/lm-sensors. -Hu]
Sorry if this is a bit OT but just to clarify: I keep seeing posts about a recompile of everything. That's not a requirement for gcc 14 is it? Also. will that be a requirement for gcc 15?
Thanks!
Tom |
|
Back to top |
|
|
logrusx Advocate
Joined: 22 Feb 2018 Posts: 2505
|
Posted: Thu Nov 28, 2024 1:28 pm Post subject: |
|
|
tld wrote: | That's not a requirement for gcc 14 is it? Also. will that be a requirement for gcc 15?
Thanks!
Tom |
No and no. The fact that somebody does something odd doesn't mean anything.
Best Regards,
Georgi |
|
Back to top |
|
|
CaptainBlood Advocate
Joined: 24 Jan 2010 Posts: 3977
|
Posted: Thu Nov 28, 2024 1:49 pm Post subject: |
|
|
I have no general answer.
Nevertheless when LTO is involved there, it should mentionned there are LTO step signature which must be matching when 2 packages are involved.
Let's supposed a dependency package D for package A has been LTO build in the past with a gcc that signs LTO level=X
Current gcc involved in building package A with LTO may be distant enough with the one used above to now sign LTO level=X+1 and would require the that very same LTO level=X+1 in the dependency package too. Build will break otherwise.
This happened here with D from gcc:13 and A build attempt with gcc:14.
It also happened with D from early gcc:14 and much later gcc:14.
Please note gcc ebuild doesn't inform when LTO step has changed.
This is something you get to know the hard way.
Back in the past I used to rebuild world after installing a new gcc.
I recall I had a feeling things to be snappier. Maybe better GUI environment.
Code: | wc -l /var/lib/portage/world
758 /var/lib/portage/world | is way too much nowadays.
Thks 4 ur attention, interest & support. _________________ USE="-* ..." in /etc/portage/make.conf here, i.e. a countermeasure to portage implicit braces, belt & diaper paradigm
LT: "I've been doing a passable imitation of the Fontana di Trevi, except my medium is mucus. Sooo much mucus. " |
|
Back to top |
|
|
CaptainBlood Advocate
Joined: 24 Jan 2010 Posts: 3977
|
Posted: Thu Nov 28, 2024 1:56 pm Post subject: |
|
|
Moreover, how to identify packages requiring clang/gcc tool stack?
Vast majority of such ebuilds don't tell easily.
Thks 4 ur attention, interest & support. _________________ USE="-* ..." in /etc/portage/make.conf here, i.e. a countermeasure to portage implicit braces, belt & diaper paradigm
LT: "I've been doing a passable imitation of the Fontana di Trevi, except my medium is mucus. Sooo much mucus. "
Last edited by CaptainBlood on Thu Nov 28, 2024 2:14 pm; edited 1 time in total |
|
Back to top |
|
|
logrusx Advocate
Joined: 22 Feb 2018 Posts: 2505
|
Posted: Thu Nov 28, 2024 2:03 pm Post subject: |
|
|
CaptainBlood wrote: | I have no general answer.
Nevertheless when LTO is involved there, it should mentionned there are LTO step signature which must be matching when 2 packages are involved. |
I might be wrong but I think there's no way for that to be true. LTO should be transparent to the outside world. Packages depend on the ABI not the way the libraries were optimized. LTO information is lost once the optimization is finished. The optimization itself should not impact neither program correctness nor ABI - the only things visible/important to the outside world. Nothing from the optimization of one package should spill into another. That would break ABI/correctness.
CaptainBlood wrote: | Back in the past I used to rebuild world after installing a new gcc.
I recall I had a feeling things to be snappier. Maybe better GUI environment. |
That's not necessary and has never been. /well maybe there were special cases but those must have been one offs/
Best Regards,
Georgi |
|
Back to top |
|
|
freke Veteran
Joined: 23 Jan 2003 Posts: 1035 Location: Somewhere in Denmark
|
Posted: Thu Nov 28, 2024 2:17 pm Post subject: |
|
|
tld wrote: | Sorry if this is a bit OT but just to clarify: I keep seeing posts about a recompile of everything. That's not a requirement for gcc 14 is it? Also. will that be a requirement for gcc 15?
Thanks!
Tom |
Simply done to catch/find bugs/ebuilds that needs to fixed for the new versions, they're getting stricter and stricter on errors in the code. |
|
Back to top |
|
|
CaptainBlood Advocate
Joined: 24 Jan 2010 Posts: 3977
|
Posted: Thu Nov 28, 2024 2:25 pm Post subject: |
|
|
Code: | /var/log/portage/media-gfx:splashutils-1.5.4.4-r9:20241027-213142.log.gz:lto1: fatal error: bytecode stream in file '/usr/lib64/libjpeg.a' generated with LTO version 14.0 instead of the expected 13.1 |
Code: | /var/log/portage/app-emulation:qemu-8.2.3:20240908-140310.log.gz:lto1: fatal error: bytecode stream in file '/usr/lib64/libz.a' generated with LTO version 13.1 instead of the expected 14.0 |
Code: | /var/log/portage/dev-qt:qt3d-5.15.14:20241107-160924.log.gz:lto1: fatal error: bytecode stream in file '/usr/lib64/libQt5OpenGLExtensions.a' generated with LTO version 13.1 instead of the expected 14.0 |
Code: | /var/log/portage/x11-libs:motif-2.3.8-r5:20241105-133058.log.gz:lto1: fatal error: bytecode stream in file '/usr/lib64/libfl.a' generated with LTO version 13.1 instead of the expected 14.0 |
Thks 4 ur attention, interest & support. _________________ USE="-* ..." in /etc/portage/make.conf here, i.e. a countermeasure to portage implicit braces, belt & diaper paradigm
LT: "I've been doing a passable imitation of the Fontana di Trevi, except my medium is mucus. Sooo much mucus. "
Last edited by CaptainBlood on Thu Nov 28, 2024 2:31 pm; edited 1 time in total |
|
Back to top |
|
|
CaptainBlood Advocate
Joined: 24 Jan 2010 Posts: 3977
|
Posted: Thu Nov 28, 2024 2:30 pm Post subject: |
|
|
World should be avoided.
Wise cherry picking specific packages to rebuild may have interesting benefit locally.
Thks 4 ur attention, interest & support. _________________ USE="-* ..." in /etc/portage/make.conf here, i.e. a countermeasure to portage implicit braces, belt & diaper paradigm
LT: "I've been doing a passable imitation of the Fontana di Trevi, except my medium is mucus. Sooo much mucus. " |
|
Back to top |
|
|
sam_ Developer
Joined: 14 Aug 2020 Posts: 2020
|
Posted: Thu Nov 28, 2024 3:13 pm Post subject: |
|
|
logrusx wrote: | CaptainBlood wrote: | I have no general answer.
Nevertheless when LTO is involved there, it should mentionned there are LTO step signature which must be matching when 2 packages are involved. |
I might be wrong but I think there's no way for that to be true. LTO should be transparent to the outside world. Packages depend on the ABI not the way the libraries were optimized. LTO information is lost once the optimization is finished. The optimization itself should not impact neither program correctness nor ABI - the only things visible/important to the outside world. Nothing from the optimization of one package should spill into another. That would break ABI/correctness.
|
The issue is with static libraries. See bug 926120. |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22809
|
Posted: Thu Nov 28, 2024 3:17 pm Post subject: |
|
|
LTO bytecode in static archives can cause problems, yes. The solution to that is (1) don't build static archives if you can avoid it, and (2) if you cannot avoid it, prioritize rebuilding the packages that provide those static archives. Most packages will not have built static archives, and so will not need to be rebuilt due to a newer LTO bytecode version. On my system, I have more than 10 times as many shared objects in /usr/lib64 than I have static archives. Of those static archives I reviewed for this post, none came from packages that respect USE=static-libs. Only 26 packages would need to be rebuilt to cover all my static archives, out of more than 1000+ packages if I ran emerge -e @world. |
|
Back to top |
|
|
CaptainBlood Advocate
Joined: 24 Jan 2010 Posts: 3977
|
Posted: Thu Nov 28, 2024 3:29 pm Post subject: |
|
|
I understand I might be problematic if binary packages D are involved... Maybe not.
The solution here is to rebuild dependancy package with later gcc.
Then rebuild any installed packages that relies on it with later gcc.
Thks 4 ur attention, interest & support. _________________ USE="-* ..." in /etc/portage/make.conf here, i.e. a countermeasure to portage implicit braces, belt & diaper paradigm
LT: "I've been doing a passable imitation of the Fontana di Trevi, except my medium is mucus. Sooo much mucus. " |
|
Back to top |
|
|
CaptainBlood Advocate
Joined: 24 Jan 2010 Posts: 3977
|
Posted: Thu Nov 28, 2024 3:34 pm Post subject: |
|
|
@Hu
I was just answering Quote: | I think there's no way for that to be true |
A corner case likely, that nevertheless exist.
Thks 4 ur attention, interest & support. _________________ USE="-* ..." in /etc/portage/make.conf here, i.e. a countermeasure to portage implicit braces, belt & diaper paradigm
LT: "I've been doing a passable imitation of the Fontana di Trevi, except my medium is mucus. Sooo much mucus. " |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22809
|
Posted: Thu Nov 28, 2024 3:39 pm Post subject: |
|
|
While emerge --emptytree will achieve that result, it is a very expensive way to achieve a result that could be achieved far more cheaply with some targeted intervention. I sought to explain the cheap way (rebuild exactly those packages you need), and document how much cheaper it can be. For my system, it is a difference of 26 versus 1000+ packages, and most of those 26 look relatively cheap to build, so the time and power savings is even more favorable than just 26/1000, since I don't have to rebuild huge packages like llvm and firefox, because those didn't install any static archives.
The Gentoo bug that sam_ linked shows that there is interest in making this work without user intervention, but for now, users who globally apply LTO, even to packages that install static archives, will need to give the package manager a bit of extra guidance when doing a compiler upgrade.
logrusx is mostly correct though: LTO should not change ABI, and the problem is one case where LTO information is not discarded by the end of the build, and is instead persisted in a static archive for later builds to find and trip over. |
|
Back to top |
|
|
CaptainBlood Advocate
Joined: 24 Jan 2010 Posts: 3977
|
Posted: Thu Nov 28, 2024 5:40 pm Post subject: |
|
|
Hopefully useful one-liner for any cc LTO interested user if any.: Code: | find /usr/lib* -name *.a|xargs equery -q b|sort|uniq
app-accessibility/julius-4.6
app-accessibility/mimic1-1.3.0.1-r1
app-admin/syslog-ng-4.6.0
app-arch/bzip2-1.0.8-r5
app-portage/hackport-0.7.2.2
app-text/qpdf-11.9.1
dev-db/mariadb-10.6.17
dev-db/postgresql-14.15
dev-db/postgresql-15.10
dev-db/postgresql-16.6
dev-db/postgresql-17.2
dev-debug/valgrind-3.23.0_p3
dev-haskell/adjunctions-4.4
dev-haskell/aeson-2.0.3.0
dev-haskell/alsa-core-0.5.0.1-r2
dev-haskell/alsa-mixer-0.3.0
dev-haskell/ansi-terminal-0.10.3
dev-haskell/appar-0.1.8
dev-haskell/asn1-encoding-0.9.6
dev-haskell/asn1-parse-0.9.5
dev-haskell/asn1-types-0.3.3
dev-haskell/assoc-1.0.2
dev-haskell/async-2.2.4-r1
dev-haskell/attoparsec-0.14.4
dev-haskell/base16-bytestring-0.1.1.7
dev-haskell/base64-bytestring-1.0.0.3
dev-haskell/base-compat-0.12.1
dev-haskell/base-compat-batteries-0.12.1
dev-haskell/basement-0.0.12
dev-haskell/base-orphans-0.8.6
dev-haskell/base-unicode-symbols-0.2.4.2
dev-haskell/bifunctors-5.5.11
dev-haskell/blaze-builder-0.4.2.1
dev-haskell/blaze-html-0.9.1.2-r2
dev-haskell/blaze-markup-0.8.2.8-r1
dev-haskell/byteorder-1.0.4
dev-haskell/cabal-doctest-1.0.9
dev-haskell/call-stack-0.3.0
dev-haskell/case-insensitive-1.2.1.0
dev-haskell/cereal-0.5.8.1
dev-haskell/colour-2.3.5
dev-haskell/comonad-5.0.8-r1
dev-haskell/conduit-1.3.4.2
dev-haskell/conduit-extra-1.3.4
dev-haskell/connection-0.3.1-r1
dev-haskell/contravariant-1.5.3
dev-haskell/cookie-0.4.4
dev-haskell/cryptohash-sha256-0.11.101.0
dev-haskell/cryptonite-0.28
dev-haskell/data-default-class-0.1.2.0
dev-haskell/data-fix-0.3.2-r1
dev-haskell/dbus-1.2.27
dev-haskell/diff-0.4.0
dev-haskell/distributive-0.6.2.1
dev-haskell/dlist-1.0
dev-haskell/echo-0.1.3
dev-haskell/ed25519-0.0.5.0-r1
dev-haskell/edit-distance-0.2.2.1-r1
dev-haskell/extensible-exceptions-0.1.1.4
dev-haskell/fgl-5.7.0.3
dev-haskell/free-5.1.9
dev-haskell/hashable-1.4.0.2-r1
dev-haskell/hinotify-0.3.10
dev-haskell/hourglass-0.2.12
dev-haskell/http-4000.4.1-r1
dev-haskell/http-client-0.7.11
dev-haskell/http-client-tls-0.3.6.1
dev-haskell/http-conduit-2.3.8
dev-haskell/http-types-0.12.3
dev-haskell/indexed-traversable-0.1.2
dev-haskell/indexed-traversable-instances-0.1
dev-haskell/integer-logarithms-1.0.3.1-r2
dev-haskell/invariant-0.5.5
dev-haskell/iproute-1.7.8
dev-haskell/kan-extensions-5.2.5
dev-haskell/language-c-0.8.3-r1
dev-haskell/lens-5.1.1
dev-haskell/libmpd-0.9.2.0
dev-haskell/memory-0.15.0-r1
dev-haskell/mime-types-0.1.0.9
dev-haskell/mono-traversable-1.0.15.1
dev-haskell/network-3.1.2.5
dev-haskell/network-uri-2.6.4.1
dev-haskell/old-locale-1.0.0.7
dev-haskell/onetuple-0.3.1
dev-haskell/parallel-3.2.2.0
dev-haskell/parsec-numbers-0.1.0
dev-haskell/pem-0.2.4
dev-haskell/primitive-0.7.3.0
dev-haskell/profunctors-5.6.2-r1
dev-haskell/quickcheck-2.14.2
dev-haskell/random-1.2.1
dev-haskell/reflection-2.1.6
dev-haskell/regex-base-0.94.0.1
dev-haskell/regex-compat-0.95.2.1-r1
dev-haskell/regex-posix-0.96.0.0
dev-haskell/regex-tdfa-1.3.2
dev-haskell/resolv-0.1.2.0-r1
dev-haskell/resourcet-1.2.4.2
dev-haskell/safe-exceptions-0.1.7.0
dev-haskell/scientific-0.3.7.0-r2
dev-haskell/semialign-1.2.0.1
dev-haskell/semigroupoids-5.3.7-r1
dev-haskell/semigroups-0.20
dev-haskell/socks-0.6.1
dev-haskell/split-0.2.3.4
dev-haskell/splitmix-0.1.0.3
dev-haskell/statevar-1.2.1
dev-haskell/streaming-commons-0.2.2.4
dev-haskell/strict-0.4.0.1
dev-haskell/syb-0.7.2.1
dev-haskell/tagged-0.8.6.1
dev-haskell/tar-0.5.1.1-r3
dev-haskell/text-short-0.1.5-r1
dev-haskell/th-abstraction-0.4.3.0
dev-haskell/th-compat-0.1.3
dev-haskell/these-1.1.1.1
dev-haskell/th-lift-0.8.2
dev-haskell/time-compat-1.9.6.1-r1
dev-haskell/timezone-olson-0.2.0
dev-haskell/timezone-series-0.1.9
dev-haskell/tls-1.5.8
dev-haskell/transformers-base-0.4.5.2
dev-haskell/transformers-compat-0.6.6
dev-haskell/typed-process-0.2.6.0
dev-haskell/unliftio-core-0.1.2.0
dev-haskell/unordered-containers-0.2.17.0
dev-haskell/utf8-string-1.0.1.1
dev-haskell/uuid-types-1.0.5-r1
dev-haskell/vector-0.12.3.1
dev-haskell/vector-algorithms-0.8.0.4
dev-haskell/void-0.7.3
dev-haskell/witherable-0.4.2-r1
dev-haskell/x11-1.10.2
dev-haskell/x11-xft-0.3.4
dev-haskell/x509-1.7.5
dev-haskell/x509-store-1.6.7
dev-haskell/x509-system-1.6.6
dev-haskell/x509-validation-1.6.11
dev-haskell/xml-1.3.14
dev-haskell/xml-conduit-1.9.1.1
dev-haskell/xml-types-0.3.8
dev-haskell/zlib-0.6.2.3
dev-lang/ghc-9.0.2-r4
dev-lang/gprolog-1.5.0-r1
dev-lang/ispc-1.25.3
dev-lang/ocaml-4.14.2
dev-lang/tcl-8.6.14
dev-lang/tk-8.6.14
dev-lang/yasm-1.3.0-r2
dev-libs/boost-1.85.0-r1
dev-libs/ffcall-2.4-r1
dev-libs/glib-2.78.6
dev-libs/hts_engine-1.10
dev-libs/libbsd-0.11.8
dev-libs/libpcre2-10.44-r1
dev-libs/libpcre-8.45-r3
dev-libs/nss-3.107
dev-lisp/clisp-2.49.92-r1
dev-lisp/ecl-23.9.9-r1
dev-ml/findlib-1.9.6-r1
dev-ml/ocamlbuild-0.15.0
dev-python/numpy-2.1.3
dev-qt/designer-5.15.14
dev-qt/qtbase-6.7.3-r2
dev-qt/qtcore-5.15.14
dev-qt/qtdeclarative-5.15.14
dev-qt/qtdeclarative-6.7.3-r4
dev-qt/qtgui-5.15.14
dev-qt/qtmultimedia-6.7.3
dev-qt/qtquick3d-6.7.3
dev-util/dialog-1.3.20240307
dev-util/shellcheck-0.9.0-r2
media-gfx/splashutils-1.5.4.4-r10
media-libs/compface-1.5.2-r1
media-libs/libjpeg-turbo-3.0.3-r1
media-libs/libpng-1.6.44
media-libs/libsdl-1.2.68
media-libs/libsdl2-2.30.7-r1
media-libs/lsp-plugins-1.2.14
media-sound/ardour-8.10-r3
net-misc/dhcp-4.4.3_p1-r6
net-misc/xrdp-9999
sci-physics/clhep-2.4.7.1
sys-apps/attr-2.5.2-r1
sys-apps/dtc-1.7.1
sys-apps/iproute2-6.11.0-r2
sys-apps/pkgcore-0.12.28
sys-apps/portage-3.0.66.1-r1
sys-boot/gnu-efi-3.0.18-r5
sys-devel/binutils-2.42-r2
sys-devel/binutils-2.43-r2
sys-devel/bpf-toolchain-14.2.0_p1
sys-devel/flex-2.6.4-r6
sys-devel/gcc-13.3.1_p20241115
sys-devel/gcc-14.2.1_p20241116
sys-devel/llvm-15.0.7-r7
sys-devel/llvm-18.1.8-r6
sys-devel/llvm-19.1.4
sys-fs/eudev-3.2.14
sys-libs/binutils-libs-2.42-r2
sys-libs/compiler-rt-15.0.7-r1
sys-libs/compiler-rt-18.1.8-r1
sys-libs/compiler-rt-19.1.4
sys-libs/compiler-rt-sanitizers-15.0.7-r1
sys-libs/compiler-rt-sanitizers-18.1.8-r1
sys-libs/compiler-rt-sanitizers-19.1.4
sys-libs/glibc-2.40-r5
sys-libs/libxcrypt-4.4.36-r3
sys-libs/zlib-1.3.1-r1
x11-misc/xmobar-0.44.1 | It outputs a raw package list to work from to decide what to rebuild.
Plz note: This doesn't help to define which dependent packages to rebuild afterwards.
I don't know how to retrieve which package EFFECTIVELY depends on another.
Although 'equery d' can't retrieve for effectiveness,it may help to discard some above packages from rebuild e.g.;
Code: | equery d app-admin/syslog-ng
* These packages depend on app-admin/syslog-ng:
virtual/logger-0-r1 (app-admin/syslog-ng) | Sorry no interesting one-liner there.
Thks 4 ur attention, interest & support. _________________ USE="-* ..." in /etc/portage/make.conf here, i.e. a countermeasure to portage implicit braces, belt & diaper paradigm
LT: "I've been doing a passable imitation of the Fontana di Trevi, except my medium is mucus. Sooo much mucus. " |
|
Back to top |
|
|
eschwartz Developer
Joined: 29 Oct 2023 Posts: 240
|
Posted: Fri Nov 29, 2024 6:54 am Post subject: |
|
|
Code: | emerge ---oneshot --ask /usr/lib64/lib*.a |
|
|
Back to top |
|
|
CaptainBlood Advocate
Joined: 24 Jan 2010 Posts: 3977
|
Posted: Fri Nov 29, 2024 10:47 am Post subject: |
|
|
eschwartz wrote: | Code: | emerge ---oneshot --ask /usr/lib64/lib*.a |
| Nice one-liner,
Typo fix: Code: | emerge --oneshot --ask /usr/lib64/lib*.a |
Thks 4 ur attention, interest & support. _________________ USE="-* ..." in /etc/portage/make.conf here, i.e. a countermeasure to portage implicit braces, belt & diaper paradigm
LT: "I've been doing a passable imitation of the Fontana di Trevi, except my medium is mucus. Sooo much mucus. " |
|
Back to top |
|
|
|