| View previous topic :: View next topic |
| Author |
Message |
Shadow_Fury
Apprentice
Joined: 20 Apr 2021
Posts: 180
Location: 11.435765792823453, 143.05926743686274
|
 Posted: Sat Sep 07, 2024 1:00 pm Post subject: Help needed with understanding package.env stacking |
 |
|
(question originally form here)
I'm trying to understand how to use /etc/portage/package.env and /etc/portage/package.env work.
here's what i have:
make.conf (relevant section):
| Code: |
SANITIZERS="-fsanitize=cfi"
HARDEN_FLAGS="-flto -ftrivial-auto-var-init=zero -fomit-frame-pointer -ftrapv -mretpoline -fpie -fPIE -fPIC -fstack-clash-protection -Wl,-z,separate-code ${SANITIZERS}"
COMMON_FLAGS="-mtune=native -march=native -O2 -pipe ${HARDEN_FLAGS} -Wno-error=implicit-function-declaration"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
|
/etc/portage/env/safestack:
| Code: |
COMMON_FLAGS="${COMMON_FLAGS} -fsanitize=safe-stack"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
|
/etc/portage/env/default-visibility:
| Code: |
COMMON_FLAGS="${COMMON_FLAGS} -fvisibility=default"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
|
/etc/portage/package.env/00-default:
the wiki would suggest that you can mix portage/env profiles:
| wiki wrote: |
Notice that it is possible to reference several files in /etc/portage/env for each package.
|
but when i try it, they behave weirdly;
If i do:
/etc/portage/package.env/libffi:
| Code: |
dev-libs/libffi -safestack
|
safestack disables fine for libffi (the clang flag disappears)
but, when i do this:
/etc/portage/package.env/libffi:
| Code: |
dev-libs/libffi -safestack default-visibility
|
it re-appears...
can someone explain to me why this is wrong, and how to do it right?
(this all started because clang safestack doesn't seem to respect -fno-sanitize i.e. -fsanitise=safe-stack is not overwritten by -fno-sanitize=safe-stack)
Last edited by Shadow_Fury on Mon Sep 09, 2024 7:13 pm; edited 1 time in total |
|
| Back to top |
|
 |
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9605
Location: beyond the rim
|
| Posted: Sat Sep 07, 2024 1:58 pm Post subject: |
|
|
| Don't think negation is supported in package.env as these are just filenames, not tokens (and -safestack is a valid filename). |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2390
|
| Posted: Sat Sep 07, 2024 6:07 pm Post subject: |
|
|
One more remark, I'm not sure you can have them both changing the same variables.
Also your /etc/portage/package.env/00-default: does not make sense. If you want something as a default, put it in make.conf, package.env is intended for *per-package* configuration.
Best Regards,
Georgi |
|
| Back to top |
|
|
Shadow_Fury
Apprentice
Joined: 20 Apr 2021
Posts: 180
Location: 11.435765792823453, 143.05926743686274
|
| Posted: Sat Sep 07, 2024 8:10 pm Post subject: |
|
|
| logrusx wrote: |
Also your /etc/portage/package.env/00-default: does not make sense. If you want something as a default, put it in make.conf, package.env is intended for *per-package* configuration.
Best Regards,
Georgi |
i'd like to put it in make.conf, but there appears to be no way to then override just that flag using the env system. unlike most clang flags, it isn't overwritten by -fno-sanitize=safestack (for some reason, i'm not sure why).
i can think of a few other ways to do this but they all seem a bit hacky. if there is a good way to do this, i'd appreciate to know it. (to clarify, i want it enabled by default, with the option to disable it on specific packages. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9605
Location: beyond the rim
|
| Posted: Sun Sep 08, 2024 4:24 am Post subject: |
|
|
Just to be certain, what is the exact content of $CFLAGS in all four cases:
a) no package.env entries
b) only the */* entry
c) with the libffi -safestack entry
d) with the libffi -safestack defaultvisibility entry
I wouldn't expect the -safestack entries doing anything but trigger errors about missing files. Also I have another suspicion, but would have to see the actual variable content to confirm that. |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2390
|
| Posted: Sun Sep 08, 2024 4:35 am Post subject: |
|
|
-safestack is wrong analogy to use flags. To disable safestack it should not be present on the line at all.
Remove the */* entry and compose your configs the way you want them for every package. This is not a use flag and the negation should not work as already mentioned.
Best Regards,
Georgi |
|
| Back to top |
|
|
Ralphred
l33t
Joined: 31 Dec 2013
Posts: 646
|
| Posted: Sun Sep 08, 2024 5:10 am Post subject: |
|
|
| logrusx wrote: | | -safestack is wrong analogy to use flags. To disable safestack it should not be present on the line at all. |
So a "nostafestack" /etc/portage/env file negating the effects of /etc/portage/env/safestack being applied to */* would be the way forward in this case? |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2390
|
| Posted: Sun Sep 08, 2024 6:12 am Post subject: |
|
|
| Ralphred wrote: | | logrusx wrote: | | -safestack is wrong analogy to use flags. To disable safestack it should not be present on the line at all. |
So a "nostafestack" /etc/portage/env file negating the effects of /etc/portage/env/safestack being applied to */* would be the way forward in this case? |
No. |
|
| Back to top |
|
|
Ralphred
l33t
Joined: 31 Dec 2013
Posts: 646
|
| Posted: Sun Sep 08, 2024 8:14 am Post subject: |
|
|
Lol, so as OP wants -fsanitize=safe-stack as default, and to disable it at will, you are suggesting he should maintain /etc/portage/package.env/safestack by running | Code: | | EIX_LIMIT=0 eix --pure-packages --format '<category>\\<name> safestack\n' > /etc/portage/package.env/safestack |
then filtering out the things it shouldn't be applied to is somehow a preferable option to using COMMON_FLAGS="${COMMON_FLAGS/ -fsanitize=safe-stack/"} or setting a "-fsanitize=safe-stack free" variable from make.conf to allow nested negation within the portage/{env,package.env} framework?
[/code] |
|
| Back to top |
|
|
sam_
Developer
Joined: 14 Aug 2020
Posts: 1946
|
| Posted: Sun Sep 08, 2024 8:34 am Post subject: |
|
|
Why not just -fno-sanitize=...? (Keeping in mind my previous remarks about this endeavour being IMO ill-advised.)
EDIT: Ah, nevermind, OP answered that before in the other thread (although I think it'd be worth reporting that as a Clang bug).
Last edited by sam_ on Sun Sep 08, 2024 10:10 am; edited 1 time in total |
|
| Back to top |
|
|
logrusx
Advocate
Joined: 22 Feb 2018
Posts: 2390
|
| Posted: Sun Sep 08, 2024 9:13 am Post subject: |
|
|
| Ralphred wrote: | Lol, so as OP wants -fsanitize=safe-stack as default, and to disable it at will, you are suggesting he should maintain /etc/portage/package.env/safestack by running | Code: | | EIX_LIMIT=0 eix --pure-packages --format '<category>\\<name> safestack\n' > /etc/portage/package.env/safestack |
then filtering out the things it shouldn't be applied to is somehow a preferable option to using COMMON_FLAGS="${COMMON_FLAGS/ -fsanitize=safe-stack/"} or setting a "-fsanitize=safe-stack free" variable from make.conf to allow nested negation within the portage/{env,package.env} framework?
|
I'm not suggesting anything. Just pointing out this idea is bad and it won't work, as it's already evident from OP's attempts.
Both putting a global entry in package.env and putting all packages in package.env are bad ideas. It's *per-package* configuration, not system-wide configuration. Negation doesn't seem to be possible.
Your idea seems to be possible but impractical. Either there must be another way of doing what OP wants or doing it in itself is a bad idea.
Best Regards,
Georgi |
|
| Back to top |
|
|
Ralphred
l33t
Joined: 31 Dec 2013
Posts: 646
|
| Posted: Sun Sep 08, 2024 7:30 pm Post subject: |
|
|
| logrusx wrote: | | I'm not suggesting anything. |
Indeed.
| Shadow_Fury wrote: | | i want it enabled by default, with the option to disable it on specific packages. |
Including -fsanitize=safe-stack in make.conf and performing parameter substitution by way of | /etc/portage/env/-safestack: | COMMON_FLAGS="${COMMON_FLAGS/ -fsanitize=safe-stack/"}
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}" |
does work with | /etc/portage/package.env/libffi: | | dev-libs/libffi -safestack default-visibility |
Portage complains - but it works...
Preserving COMMON_FLAGS at the point of -fsanitize=safe-stack addition will obviously work, but it looks so ugly: | /etc/portage/make.conf snippet: | COMMON_FLAGS="-mtune=native -march=native -O2 -pipe ${HARDEN_FLAGS} -Wno-error=implicit-function-declaration"
NOSS_FLAGS="${COMMON_FLAGS}
COMMON_FLAGS="${COMMON_FLAGS} -fsanitize=safe-stack" |
| /etc/portage/env/default-visibility: | COMMON_FLAGS="${COMMON_FLAGS} -fvisibility=default"
NOSS_FLAGS="${NOSS_FLAGS} -fvisibility=default"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}" |
| /etc/portage/env/someotheroption: | COMMON_FLAGS="${COMMON_FLAGS} -some-valid=option"
NOSS_FLAGS="${NOSS_FLAGS} -some-valid=option"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}" |
| /etc/portage/env/nosafestack: | COMMON_FLAGS="${NOSS_FLAGS}"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}" |
| /etc/portage/package.env/libffi: | | dev-libs/libffi nosafestack default-visibility |
|
|
| Back to top |
|
|
Shadow_Fury
Apprentice
Joined: 20 Apr 2021
Posts: 180
Location: 11.435765792823453, 143.05926743686274
|
| Posted: Mon Sep 09, 2024 7:13 pm Post subject: |
|
|
| Ralphred wrote: | | logrusx wrote: | | I'm not suggesting anything. |
Indeed.
| Shadow_Fury wrote: | | i want it enabled by default, with the option to disable it on specific packages. |
Including -fsanitize=safe-stack in make.conf and performing parameter substitution by way of | /etc/portage/env/-safestack: | COMMON_FLAGS="${COMMON_FLAGS/ -fsanitize=safe-stack/"}
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}" |
does work with | /etc/portage/package.env/libffi: | | dev-libs/libffi -safestack default-visibility |
Portage complains - but it works... |
tried this.
it doesn't work brilliantly. specifically, it entirely clears COMMON_FLAGS. so, things like "-O2 -pipe" are gone.
i also tried the apporach of defining a custom envvar in make.conf:
| Code: |
SAN_SAFESTACK="-fsanitize=safe-stack"
|
having no-safestack clear it:
and moving the setting of CFLAGS, CXXFLAGS etc. from make.conf to:
portage/env/merge:
| Code: |
COMMON_FLAGS="${COMMON_FLAGS} ${SAN_SAFESTACK}"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"
FCFLAGS="${COMMON_FLAGS}"
FFLAGS="${COMMON_FLAGS}"
|
then, i changed package.env/libffi:
| Code: |
dev-libs/libffi no-safestack
|
and 00-default:
this didn't work.
i moved 00-default to zz-default (trying to change the load order),
didn't work either
this works:
package.env/libffi:
| Code: |
dev-libs/libffi no-safestack merge
|
but only if 00-default or zz-default with merge don't exist.
i'm thoroughly confused now.
i'm also contemplating injecting a sed command via /etc/portage/bashrc that checks a custom ENVVAR and zeros any flags i pass to it.
input with other ideas is welcome. |
|
| Back to top |
|
|
Ralphred
l33t
Joined: 31 Dec 2013
Posts: 646
|
| Posted: Mon Sep 09, 2024 8:28 pm Post subject: |
|
|
| Shadow_Fury wrote: |
this didn't work.
i moved 00-default to zz-default (trying to change the load order),
didn't work either
this works:
package.env/libffi:
| Code: |
dev-libs/libffi no-safestack merge
|
|
Yeah, I played with 00-default and zz-default, and the concept "merge" too, it seems random (or at least unpredictable).
That's why I put -fsanitize=safe-stack in make.conf, then it doesn't matter "what order things get run it" because for no-safestack to work it has to be there, and we know it is.
| Code: | | specifically, it entirely clears COMMON_FLAGS. so, things like "-O2 -pipe" are gone. |
Yeah, that didn't happen here, I set multiple "broken options" in make.conf and pulled them out one by one, if I only pulled one out it failed on the other "unknown option"s.
https://wiki.gentoo.org/wiki//etc/portage/bashrc is an interesting read, taking what is says onboard about not stamping on CFLAGS and when to run things, I'm off to break my make.conf for some playing... |
|
| Back to top |
|
|
Ralphred
l33t
Joined: 31 Dec 2013
Posts: 646
|
| Posted: Mon Sep 09, 2024 9:24 pm Post subject: |
|
|
| Shadow_Fury wrote: | | Contemplating injecting a sed command via /etc/portage/bashrc that checks a custom ENVVAR and zeros any flags i pass to it. |
This works like a charm, code snippets: | /etc/portage/make.conf snippet: | ##test
NEGATE_FLAGS=""
COMMON_FLAGS="${COMMON_FLAGS} -fsanitize=safe-stack -mbrokenoption -mbrokenoption2"
##endtest |
| /etc/portage/env/nosafestack: | | NEGATE_FLAGS="${NEGATE_FLAGS} -fsanitize=safe-stack" |
| /etc/portage/env/nobrokenoption: | | NEGATE_FLAGS="${NEGATE_FLAGS} -mbrokenoption" |
| /etc/portage/env/nobrokenoption2: | | NEGATE_FLAGS="${NEGATE_FLAGS} -mbrokenoption2" |
| /etc/portage/bashrc: | function pre_src_compile() {
for local_flag in ${NEGATE_FLAGS};do
CFLAGS="${CFLAGS/ ${local_flag}/}"
CXXFLAGS="${CXXFLAGS/ ${local_flag}/}"
done
} |
I use dwm as my test emerge for things like this, so with no /etc/portage/package.env/dwm I get | Code: | x86_64-pc-linux-gnu-gcc: error: unrecognized argument to ‘-fsanitize=’ option: ‘safe-stack’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-mbrokenoption’
x86_64-pc-linux-gnu-gcc: error: unrecognized command-line option ‘-mbrokenoption2’ |
With | /etc/portage/package.env/dwm: | | x11-wm/dwm nosafestack nobrokenoption nobrokenoption2 |
It works. I also added them one at a time, in various orders, and removed the "##test" lines from make.conf, just to check. |
|
| Back to top |
|
|
Shadow_Fury
Apprentice
Joined: 20 Apr 2021
Posts: 180
Location: 11.435765792823453, 143.05926743686274
|
| Posted: Wed Sep 11, 2024 3:49 pm Post subject: |
|
|
| Ralphred wrote: | | Shadow_Fury wrote: | | Contemplating injecting a sed command via /etc/portage/bashrc that checks a custom ENVVAR and zeros any flags i pass to it. |
This works like a charm, code snippets: | /etc/portage/make.conf snippet: | ##test
NEGATE_FLAGS=""
COMMON_FLAGS="${COMMON_FLAGS} -fsanitize=safe-stack -mbrokenoption -mbrokenoption2"
##endtest |
| /etc/portage/env/nosafestack: | | NEGATE_FLAGS="${NEGATE_FLAGS} -fsanitize=safe-stack" |
| /etc/portage/env/nobrokenoption: | | NEGATE_FLAGS="${NEGATE_FLAGS} -mbrokenoption" |
| /etc/portage/env/nobrokenoption2: | | NEGATE_FLAGS="${NEGATE_FLAGS} -mbrokenoption2" |
| /etc/portage/bashrc: | function pre_src_compile() {
for local_flag in ${NEGATE_FLAGS};do
CFLAGS="${CFLAGS/ ${local_flag}/}"
CXXFLAGS="${CXXFLAGS/ ${local_flag}/}"
done
} |
|
had to do something slightly different, since my system doesn't like the ${//} replacement syntax for some reason.
here's what i came up with:
/etc/portage/bashrc:
| Code: |
function pre_pkg_setup() {
for local_flag in ${NEGATE_FLAGS}; do
COMMON_FLAGS=`echo ${COMMON_FLAGS} | sed "s/${local_flag}//g"`
CFLAGS=`echo ${CFLAGS} | sed "s/${local_flag}//g"`
CXXFLAGS=`echo ${CXXFLAGS} | sed "s/${local_flag}//g"`
FCFLAGS=`echo ${CFLAGS} | sed "s/${local_flag}//g"`
FFLAGS=`echo ${FFLAGS} | sed "s/${local_flag}//g"`
done
}
|
i also hooked pre-setup so it substitutes as early as possible, the since libffi EBUILD runs a compiler test in the configure phase.
it seems to work fine.
i have one final question, though am unsure whether i should split it off into a separate thread...
is it possible to do this kind of thing per ABI? safe-stack has an issue right now when compiled for 32-bits, so i'm wondering whether i can disable the flag for only when it's compiling the 32-bit version of a package, and not the 64-bit version. (it this does deserve it's own thread, please let me know and i'll fix it.)
Last edited by Shadow_Fury on Wed Sep 11, 2024 4:28 pm; edited 2 times in total |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22601
|
| Posted: Wed Sep 11, 2024 4:03 pm Post subject: |
|
|
| Shadow_Fury wrote: | had to do something slightly different; i use dash as the sh shell, and it doesn't support the ${//} replacement syntax.
here's what i came up with:
/etc/portage/bashrc: |
So even though the file is named /etc/portage/bashrc, and is evaluated by the shell that Portage runs, your use of /bin/sh -> dash causes this file to be evaluated using dash, not bash? That seems wrong. I think many ebuilds use bash-specific features, and the specification even allows for that. Therefore, it seems strange to me that Portage would use /bin/sh for this step. |
|
| Back to top |
|
|
Shadow_Fury
Apprentice
Joined: 20 Apr 2021
Posts: 180
Location: 11.435765792823453, 143.05926743686274
|
| Posted: Wed Sep 11, 2024 4:27 pm Post subject: |
|
|
| Hu wrote: | | Shadow_Fury wrote: | had to do something slightly different; i use dash as the sh shell, and it doesn't support the ${//} replacement syntax.
here's what i came up with:
/etc/portage/bashrc: |
So even though the file is named /etc/portage/bashrc, and is evaluated by the shell that Portage runs, your use of /bin/sh -> dash causes this file to be evaluated using dash, not bash? That seems wrong. I think many ebuilds use bash-specific features, and the specification even allows for that. Therefore, it seems strange to me that Portage would use /bin/sh for this step. |
..Ignore me... was trying to figure out why the substitution syntax wasn't working.
... also, upon further examination, i haven't changed the symlink over yet, anyway...
sorry, i was being an idiot |
|
| Back to top |
|
|
Ralphred
l33t
Joined: 31 Dec 2013
Posts: 646
|
| Posted: Wed Sep 11, 2024 7:27 pm Post subject: |
|
|
| Shadow_Fury wrote: | | had to do something slightly different, since my system doesn't like the ${//} replacement syntax for some reason. |
You know, I wrote the sed version too, but as the ${//} wasn't even making portage complain (like it does if I put in portage/env) I thought, "yeah, it'll work, I'll take those #'d out sed lines out"
| Shadow_Fury wrote: | i also hooked pre-setup so it substitutes as early as possible, the since libffi EBUILD runs a compiler test in the configure phase.
it seems to work fine. |
There was a comment in the bashrc docs about flags being generated by the build, so I thought "Ahh, should put it as late as humanly possible in case that's the flag someone is trying to negate!" - but that's the difference, you have a use case and I'm just tinkering with stuff...
| Shadow_Fury wrote: | | i have one final question, though am unsure whether i should split it off into a separate thread |
I think the context that this thread offers doesn't make it inappropriate to ask here, but if you start a new one you'll get more fresh eyes on it IMHO. |
|
| Back to top |
|
|
|
Portage & Programming
