| View previous topic :: View next topic |
| Author |
Message |
ipic
Guru
Joined: 29 Dec 2003
Posts: 446
Location: UK
|
 Posted: Mon Feb 17, 2025 2:45 pm Post subject: How to make temporary change to package.env? |
 |
|
This question arises for me because I want to experiment with building the kernel with LLVM, with a view to experimenting with a rust build in the future.
I have scripted the kernel build process, and that part is straight forward.
I checked that the LLVM/Rust tool chain is OK from the kernel point of view:
| Code: |
~ # cd /usr/src/linux
/usr/src/linux # export LLVM=1
/usr/src/linux # make rustavailable
Rust is available!
|
My script then does the equivalent of this:
| Code: |
~ # cd /usr/src/linux
/usr/src/linux # export LLVM=1
/usr/src/linux # make distclean
/usr/src/linux # zcat /proc/config.gz >.config
/usr/src/linux # make olddefconfig
/usr/src/linux # make menuconfig
/usr/src/linux # make -j64
/usr/src/linux # make modules_install
|
That works - and the script can either declare the LLVM=1 environment variable or not - depending on which tool chain I wish to use.
Since modules are involved - I know I can't mix two instances of the same kernel version this way - a single version has to be one or the other.
My question then relates to the next part of the kernel build, where this command is run:
| Code: | | emerge --oneshot --jobs @module-rebuild |
I have set up a Clang profile in /etc/portage/env:
| Code: |
~ # cat /etc/portage/env/compiler-clang
# Normal settings here
COMMON_FLAGS="-O2 -march=native"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
CC="clang"
CPP="clang-cpp" # necessary for xorg-server and possibly other packages
CXX="clang++"
AR="llvm-ar"
NM="llvm-nm"
RANLIB="llvm-ranlib"
|
From the documentation I can find, if I want modules in @module-rebuild to match the kernel tool chain, I have to place an entry in /etc/portage/env/package.env for each package, specifying use of compiler-clang.
There are two problems with this from my point of view:
- I have to know what packages are in @module-rebuild and manually change the settings for each build
- I have to manually change the entries depending on whether I am using LLVM=1 or not.
So, the question is - can I temporarily override /etc/portage/env/package.env settings for an emerge invocation - so that all packages in that invocation use the same settings, without having to know which packages are being emerged? |
|
| Back to top |
|
 |
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2743
|
| Posted: Mon Feb 17, 2025 6:20 pm Post subject: |
|
|
Perhaps do something in /etc/portage/bashrc. Or change thing manually. To my knowledge there's no space for scripting in package.env and limited scripting in env/category/package. Entries in package.env are processed by python and in env/category/package you can have variable expansion. Don't take my words to face value, I'm not an expert in that area. This is what I remember I once read in a bug that I can't find anymore.
Best Regards,
Georgi |
|
| Back to top |
|
|
pingtoo
Veteran
Joined: 10 Sep 2021
Posts: 1506
Location: Richmond Hill, Canada
|
| Posted: Mon Feb 17, 2025 8:08 pm Post subject: |
|
|
You can try | Code: | COMMON_FLAGS="-O2 -march=native" \
CFLAGS="-O2 -march=native" \
CXXFLAGS="-O2 -march=native" \
CC="clang" \
CPP="clang-cpp" \
CXX="clang++" \
AR="llvm-ar" \
NM="llvm-nm" \
RANLIB="llvm-ranlib" \
emerge --oneshot --jobs @module-rebuild
|
|
|
| Back to top |
|
|
ipic
Guru
Joined: 29 Dec 2003
Posts: 446
Location: UK
|
| Posted: Mon Feb 17, 2025 10:03 pm Post subject: |
|
|
| logrusx wrote: | Perhaps do something in /etc/portage/bashrc. Or change thing manually. To my knowledge there's no space for scripting in package.env and limited scripting in env/category/package. Entries in package.env are processed by python and in env/category/package you can have variable expansion. Don't take my words to face value, I'm not an expert in that area. This is what I remember I once read in a bug that I can't find anymore.
Best Regards,
Georgi |
I'd forgotten about /etc/portage/bashrc - thanks for pointing it out.
Interestingly, I have an entry in there that says this:
| Code: |
# This hook is neccesary for automatic updating of the cfg-update index, please do not modify it!
pre_pkg_setup() {
[[ $ROOT = / ]] && cfg-update --index
}
|
I guess cfg-update placed that there, although the file is not owned by cfg-update.
I see this statement in the Gentoo WiKi page for /etc/portage/bashrc (at https://wiki.gentoo.org/wiki//etc/portage/bashrc)
| Quote: | In order to execute manually-specified code during an ebuild, there are two distinct ways to hook into the ebuild process: via hook functions, and via environment variables.
In the context of /etc/portage/bashrc, hook functions should be used, as this file will be sourced multiple times during the build - at least once per phase, and several other times as well... |
That seems to be saying don't set environment variables in this file - which seems to rule out doing | Code: | export CC="clang"
...etc |
in pre_pkg_setup |
|
| Back to top |
|
|
ipic
Guru
Joined: 29 Dec 2003
Posts: 446
Location: UK
|
| Posted: Mon Feb 17, 2025 10:11 pm Post subject: |
|
|
| pingtoo wrote: | You can try | Code: | COMMON_FLAGS="-O2 -march=native" \
CFLAGS="-O2 -march=native" \
CXXFLAGS="-O2 -march=native" \
CC="clang" \
CPP="clang-cpp" \
CXX="clang++" \
AR="llvm-ar" \
NM="llvm-nm" \
RANLIB="llvm-ranlib" \
emerge --oneshot --jobs @module-rebuild
|
|
Thanks for the suggestion.
Not being a portage/bash expert, does the pre-existence of environment variables stop emerge from going off and applying it's own values from make.conf?
If the pre-set values are used, would this work?
| Code: |
source /etc/portage/env/compiler-clang
emerge --oneshot --jobs @module-rebuild |
That would be neat (if it works), using the same setup as for other Clang builds. |
|
| Back to top |
|
|
pingtoo
Veteran
Joined: 10 Sep 2021
Posts: 1506
Location: Richmond Hill, Canada
|
| Posted: Mon Feb 17, 2025 10:53 pm Post subject: |
|
|
| ipic wrote: | | pingtoo wrote: | You can try | Code: | COMMON_FLAGS="-O2 -march=native" \
CFLAGS="-O2 -march=native" \
CXXFLAGS="-O2 -march=native" \
CC="clang" \
CPP="clang-cpp" \
CXX="clang++" \
AR="llvm-ar" \
NM="llvm-nm" \
RANLIB="llvm-ranlib" \
emerge --oneshot --jobs @module-rebuild
|
|
Thanks for the suggestion.
Not being a portage/bash expert, does the pre-existence of environment variables stop emerge from going off and applying it's own values from make.conf?
If the pre-set values are used, would this work?
| Code: |
source /etc/portage/env/compiler-clang
emerge --oneshot --jobs @module-rebuild |
That would be neat (if it works), using the same setup as for other Clang builds. |
I know using single line will work (the example I post) I have not try source a file. But it should work the same as the simple line. You just need to be careful when using source command, from that point onward those variables are set, so if you have follow up emerge for something else you may need to unset those variable prior to run emerge.
apparently emerge favour run time environment over configuration file, i.e. env/package.env. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9627
Location: beyond the rim
|
| Posted: Tue Feb 18, 2025 1:52 pm Post subject: |
|
|
| ipic wrote: |
That seems to be saying don't set environment variables in this file - which seems to rule out doing | Code: | export CC="clang"
...etc |
in pre_pkg_setup |
You can set env variables there. But as the file is sourced multiple times during the build (and maybe even for each phase) you should keep the code in global scope to a minimum. Also you should not set variables there that have any special meaning to portage itself as that will lead to undefined/misleading behavior. For example setting CFLAGS is technically ok, but the value you set in bashrc will not be picked up by emerge --info (one reason why package.env exists) and may bypass sanity checks/filters performed by the ebuild or eclasses. That is why this is generally discouraged in favor of package.env.
| Quote: |
Thanks for the suggestion.
Not being a portage/bash expert, does the pre-existence of environment variables stop emerge from going off and applying it's own values from make.conf? |
No, settings from the calling environment will usually override settings from config files.
| Quote: |
If the pre-set values are used, would this work?
Code:
source /etc/portage/env/compiler-clang
emerge --oneshot --jobs @module-rebuild
That would be neat (if it works), using the same setup as for other Clang builds. |
Yes, but keep in mind that the values set by the source command will persist in that bash session after the emerge job has finished. So I would not recommend doing it that way, better just do a full script for it. |
|
| Back to top |
|
|
ipic
Guru
Joined: 29 Dec 2003
Posts: 446
Location: UK
|
| Posted: Tue Feb 18, 2025 2:12 pm Post subject: |
|
|
I ran a few test builds with the following settings:
- just run emerge @module-rebuild
- run emerge @module-rebuild with entry for the module in /etc/portage/env/package.env tagging compiler-clang
- run `source /etc/portage/env/compiler-clang ; energe @module-rebuild
- run the command as @pingtoo suggested
In ALL of these cases, the build output is unchanged, and shows this:
| Code: |
>>> Emerging (1 of 1) app-admin/ryzen_smu-0.1.5::gentoo
* ryzen_smu-v0.1.5.tar.bz2 BLAKE2B SHA512 size ;-) ...
* Determining the location of the kernel source code
* Found kernel source directory:
* /usr/src/linux
* Found sources for kernel version:
* 6.13.3-gentoo-r1
* Checking for suitable kernel configuration options ...
* Preparing x86_64-pc-linux-gnu toolchain for kernel modules (override with KERNEL_CHOST) ...
* Toolchain picked for kernel modules (override with KERNEL_CC, _LD, ...): '/usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-clang-19' '/usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-clang++-19' '/usr/lib/llvm/19/bin/ld.lld' '/usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-ar' '/usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-nm' '/usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-objcopy' '/usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-objdump' '/usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-readelf' '/usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-strip'
>>> Unpacking source...
|
Also, in ALL these cases the module builds, installs, and then loads correctly against the LLVM/Clang built kernel.
Looking at the selected tools on my system I see this:
| Code: |
~ # ls -lh /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-clang-19
lrwxrwxrwx 1 root root 8 Feb 8 09:13 /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-clang-19 -> clang-19
~ # ls -lh /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-clang++-19
lrwxrwxrwx 1 root root 10 Feb 8 09:13 /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-clang++-19 -> clang++-19
~ # ls -lh /usr/lib/llvm/19/bin/ld.lld
-rwxr-xr-x 1 root root 23K Feb 11 10:45 /usr/lib/llvm/19/bin/ld.lld
~ # ls -lh /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-ar
lrwxrwxrwx 1 root root 7 Dec 7 12:55 /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-ar -> llvm-ar
~ # ls -lh /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-nm
lrwxrwxrwx 1 root root 7 Dec 7 12:55 /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-nm -> llvm-nm
~ # ls -lh /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-objcopy
lrwxrwxrwx 1 root root 12 Dec 7 12:55 /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-objcopy -> llvm-objcopy
~ # ls -lh /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-objdump
lrwxrwxrwx 1 root root 12 Dec 7 12:55 /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-objdump -> llvm-objdump
~ # ls -lh /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-readelf
lrwxrwxrwx 1 root root 12 Dec 7 12:55 /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-readelf -> llvm-readelf
~ # ls -lh /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-strip
lrwxrwxrwx 1 root root 10 Dec 7 12:55 /usr/lib/llvm/19/bin/x86_64-pc-linux-gnu-strip -> llvm-strip
~ #
|
It thus appears that, without any additional external 'help', a portage kernel module package will select the tool chain that matches the kernel source it is compiling/linking against.
A quick search for 'Gentoo KERNEL_CHOST' doesn't turn up anything useful, so not sure what format it would need to be to replicate the env settings I have.
I'm not sure how robust the automatic selection done by portage will be, given I'm not sure what is causing those symlinks to be placed in usr/lib/llvm/19/bin
However, given it appears to 'Just Work' (trademark) - I'm going to go with it. Makes my kernel script much simpler  |
|
| Back to top |
|
|
ipic
Guru
Joined: 29 Dec 2003
Posts: 446
Location: UK
|
| Posted: Tue Feb 18, 2025 2:14 pm Post subject: |
|
|
| Genone wrote: |
Yes, but keep in mind that the values set by the source command will persist in that bash session after the emerge job has finished. So I would not recommend doing it that way, better just do a full script for it. |
Good point.
In my case, the commands are being issued from a python scrip, so each command gets its own environment setup as inherited from the calling task.
However, as above, turns out portage is way cleverer than I gave it credit for. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
