| View previous topic :: View next topic |
| Author |
Message |
Demo
Tux's lil' helper
Joined: 25 Nov 2022
Posts: 126
|
 Posted: Sat Jun 01, 2024 2:12 pm Post subject: Why rebuild the whole system!! |
 |
|
Hello folks ...
As you may have already guessed, I did receive the newest update mail. I have to change the system profile to 23.0 and I have to rebuild the whole system, again! The question is, if I have already CET use flag enabled in my make.conf, and I have the merged usr profile also, why do I have to rebuild??! Or rather, why portage insists on rebuilding major packages, like gcc where it was already merged with the CET flag?!
| Code: | $ equery u sys-devel/gcc
[ Legend : U - final flag setting for installation]
[ : I - package is installed with flag ]
[ Colors : set, unset ]
* Found these USE flags for sys-devel/gcc-14.1.1_p20240518:
U I
- - ada : Build the ADA language (GNAT) frontend
+ + cet : (Restricted to >=sys-devel/gcc-10)
Enable support for control flow hijacking protection. On amd64, this provides Intel
Control Flow Enforcement Technology (CET). On arm64, this provides Branch Target
Identification (BTI) and Pointer Authentication Code (PAC) support. This is only
effective on amd64 or arm64. Only provides benefits on newer CPUs. For Intel, the
CPU must be at least as new as Tiger Lake. For AMD, it must be at least as new as
Zen 3. This is harmless on older CPUs, but provides no benefit either. For ARM64,
PAC was introduced in armv8.3-a, and BTI was introduced in armv8.5-a. When combined
with USE=hardened on amd64, GCC will set -fcf-protection by default when building
software. The effect is minimal on systems which do not support it, other than a
possible small increase in codesize for the NOPs. The generated code is therefore
compatible with i686 at the earliest. On arm64, GCC will set
-mbranch-protection=standard by default when building software.
- - d : Enable support for the D programming language
+ + debug : Enables GCC's 'checking' facility via --enable-checking=yes,extra,rtl. This adds
checks to various compiler passes for integrity and input validation. This can help
catch possible miscompilations early as well as latent bugs which could become real
problems in future, but at the cost of slower compile times when using GCC.
Unrelated to backtraces.
- - doc : Add extra documentation (API, Javadoc, etc). It is recommended to enable per
package instead of globally
+ + fortran : Add support for fortran
+ + go : Build the GCC Go language frontend.
+ + graphite : Add support for the framework for loop optimizations based on a polyhedral
intermediate representation
- - hardened : Activate default security enhancements for toolchain (gcc, glibc, binutils)
+ + jit : Enable libgccjit so other applications can embed gcc for Just-In-Time compilation.
+ + lto : Build using Link Time Optimizations (LTO). Note that GCC is always built with
support for building other programs with LTO. This USE flag is for whether GCC
itself is built and optimized with LTO.
- - modula2 : Build the GCC Modula-2 language frontend.
+ + nls : Add Native Language Support (using gettext - GNU locale utilities)
- - objc : Build support for the Objective C code language
- - objc++ : Build support for the Objective C++ language
- - objc-gc : Build support for the Objective C code language Garbage Collector
+ + openmp : Build support for the OpenMP (support parallel computing), requires
>=sys-devel/gcc-4.2 built with USE="openmp"
+ + pgo : Build GCC using Profile Guided Optimization (PGO). GCC will build itself and then
analyze the just-built binary and then rebuild itself using the data obtained from
analysis of codepaths taken. It does not affect whether GCC itself supports PGO
when building other software. This substantially increases the build time needed
for building GCC itself.
+ + rust : Build support for the Rust language, installs gccrs.
+ + sanitize : Build support for various sanitizer functions (ASAN/TSAN/etc...) to find runtime
problems in applications.
+ + ssp : Build packages with stack smashing protection on by default
+ + systemtap : enable systemtap static probe points
+ + test : Enable dependencies and/or preparations necessary to run tests (usually controlled
by FEATURES=test but can be toggled independently)
+ + valgrind : Enable annotations for accuracy. May slow down runtime slightly. Safe to use even
if not currently using dev-debug/valgrind
- - vanilla : Do not add extra patches which change default behaviour; DO NOT USE THIS ON A
GLOBAL SCALE as the severity of the meaning changes drastically
- - vtv : Build support for virtual table verification (a C++ hardening feature). This does
not control whether GCC defaults to using VTV> Note that actually using VTV breaks
ABI and hence the whole system must be built with -fvtable-verify.
+ + zstd : Enable support for ZSTD compression |
Now I did select the suitable 23.0 profile, and synced the repos, and moved the binpkgs folder to another name - binbpkgs -, but still all that doesn't explain why major packages being rebuilt ... |
|
| Back to top |
|
 |
Banana
Moderator
Joined: 21 May 2004
Posts: 1730
Location: Germany
|
|
| Back to top |
|
|
Demo
Tux's lil' helper
Joined: 25 Nov 2022
Posts: 126
|
| Posted: Sat Jun 01, 2024 4:58 pm Post subject: |
|
|
What I understood from the thread is that I could just go on normally updating packages as I did before the profile switch, - and of course don't ask for help when the system breaks because I didn't follow the beaten path - but then that doesn't explain why a major package like gcc has to be rebuilt if it's already built with the needed use flags?!
I optimize my builds because I have a relatively slow machine - intel core i7 6700 - and it helps compiling gcc for the skylake architecture. Because the gcc binaries use architecture specific extensions, most of the builds go faster, but gcc itself takes ages, especially the 14th version, it takes twice as long as the 13th and more.
I just wanted to understand the rational behind portage behavior and the reason the whole world set needs rebuilding for. It's understandable with split and merged usr but it's not clear with changing the profile ... |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2421
|
| Posted: Sat Jun 01, 2024 6:04 pm Post subject: |
|
|
| sam_ wrote: |
There are other changes mentioned, like DT_RELR, -fstack-clash-protection, and RELRO. |
| sam_ wrote: |
DT_RELR uses a different representation for relocations which is more efficient, which reduces binary sizes and startup time. Stack clash protection introduces checks when making stack allocations as a security measure. RELRO is again another security measure where mappings are marked as read-only immediately (it also has a nice property in that it tells you immediately if a binary is broken, not later on at runtime when something enters a codepath). None of them break ABI though so programs will still run. |
So it is possible not to rebuild the whole system, but it's impossible to guide all users what to do, neither help everybody to restore what they broke in the process, so it's your responsibility to find out how to do what you want.
Best Regards,
Georgi |
|
| Back to top |
|
|
Demo
Tux's lil' helper
Joined: 25 Nov 2022
Posts: 126
|
| Posted: Sat Jun 01, 2024 6:35 pm Post subject: |
|
|
Thank you for your replies, I do appreciate your help ...
It would be useful if the information was delivered in stages. The first stage would be the message that was sent that the profile update is going to happen. Within that message a link/links for further details about what will be actually changed and the possibly affected profiles and packages. Within the resources reachable by the link/links additional information about the possible ABI changes and the conflicts it may impose along with some workarounds and troubleshooting advise.
The already provided links is a trial, but it needs to be followed by more organized information to better explain what's going on. I mean the developers can do it at least to show case their effort and to be recognized for what they're doing, consider it selling your work even if it's already sold, it's still added to their record among the users.
I'd really appreciate it if the developers considered providing such information/details in an organized manner, and no one will complain about much or missing information, they'll give each group what they'd need separately. |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2421
|
| Posted: Sat Jun 01, 2024 6:46 pm Post subject: |
|
|
| Demo wrote: |
I'd really appreciate it if the developers considered providing such information/details in an organized manner, and no one will complain about much or missing information, they'll give each group what they'd need separately. |
Now that you say that I believe that all that work should have been tracked by bugs. Maybe you could find them and get more information.
Also have in mind the developers lives are already too complicated and even the tiniest amount of additional work could be overburden.
However I believe things will only get better, judging by the past.
BTW, have you noticed that cet is useless on your CPU:
| Quote: | + + cet : (Restricted to >=sys-devel/gcc-10)
Enable support for control flow hijacking protection. On amd64, this provides Intel Control Flow Enforcement Technology (CET). On arm64, this provides Branch Target Identification (BTI) and
Pointer Authentication Code (PAC) support. This is only effective on amd64 or arm64. Only provides benefits on newer CPUs. For Intel, the CPU must be at least as new as Tiger Lake. For AMD, it
must be at least as new as Zen 3. This is harmless on older CPUs, but provides no benefit either. For ARM64, PAC was introduced in armv8.3-a, and BTI was introduced in armv8.5-a. When combined
with USE=hardened on amd64, GCC will set -fcf-protection by default when building software. The effect is minimal on systems which do not support it, other than a possible small increase in
codesize for the NOPs. The generated code is therefore compatible with i686 at the earliest. On arm64, GCC will set -mbranch-protection=standard by default when building software.
|
So the effect on speed you're noticing looks like placebo effect and you can switch to binary packages as well. At least you won't torture yourself with compile times.
Best Regards,
Georgi |
|
| Back to top |
|
|
Demo
Tux's lil' helper
Joined: 25 Nov 2022
Posts: 126
|
| Posted: Sat Jun 01, 2024 6:54 pm Post subject: |
|
|
I'm not a software developer, however I do know that part of good software development is good documentation. I've watched university teachers talking about that point on and on in many youtube programming courses from MIT or Stanford I recall.
I did check on the bug list that was provided, and believe me it's far from clear let alone organized.
I hope that the developers find the time/chance to provide better documentation.
If they are short on working hands, I'm available for hiring  , it'd be my pleasure to participate in the gentoo community! |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20485
|
| Posted: Sun Jun 02, 2024 3:14 am Post subject: |
|
|
I feel for the developers on news announcements. I'm on the side of more is better, but preferably Just The Right Amount. Not enough information and people will ask for more. Too much information and people will ask for less.
I don't know if this helps, but link 1 in the news announcement may include some of the information you mentioned.
https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_profile_transition
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Demo
Tux's lil' helper
Joined: 25 Nov 2022
Posts: 126
|
| Posted: Sun Jun 02, 2024 4:18 pm Post subject: |
|
|
| Quote: | | I feel for the developers on news announcements. I'm on the side of more is better, but preferably Just The Right Amount. Not enough information and people will ask for more. Too much information and people will ask for less. |
I suggested that information would be provided in stages, each for it's respective audience who will actually seek after it ...
| Quote: | | https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_profile_transition |
I did check on it, and it's just a bugs list.
Now there's another mess to deal with, which is python dependencies, but that's for another thread ... |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
