View previous topic :: View next topic |
Author |
Message |
pjp Administrator
Joined: 16 Apr 2002 Posts: 20469
|
Posted: Sat Oct 26, 2024 3:14 am Post subject: RFC: linux-firmware ebuild patch, include firmware filter |
|
|
This would replace my old method that altered the upstream "truth" file. That worked for ~2 years, but as of linux-firmware-20241017, it will no longer work.
Before submitting a bug as a proposed change, I'm looking for feedback on what I should fix.
This version requires USE=savedconfig but with NO savedconfig containing a list of firmware. I suspect this isn't what would be implemented, but it's hat I could think to do with existing functionality.
Instead of the current savedconffig file, I'm using a "filter" file that contains one awk pattern per line, later used to select only the firmware to be installed to the image / live file system. This does cause the ebuild to issue the warning about "no savedconfig" file.
Nevertheless, when that condition is true (no savedconfig file) the patch next checks for the existence of the filter file. If present, it "mimics" similar functionality as the old savedconfig file method, but using the awk pattern matching on the firmware to be included.
If adopted, I expect those who maintain the ebuild would do it differently, but I'm working with the constraints of it being a new capability and my lack of knowledge about "the right way" to do things with ebuilds.
My two (tested and working) examples: /etc/portage/savedconfig/sys-kernel/linux-firmware.filter: | rtl_nic/rtl8(168|411)
amd-ucode/microcode_amd(_fam16h)?.bin | /etc/portage/savedconfig/sys-kernel/linux-firmware.filter: | i915/kbl_
iwlwifi-8265 |
The ebuild patch...
I couldn't find how to refer to the savedconfig file using variables, which is similar to how I'm referencing the "filter" file, so that is hard coded. I thought that's what "${S}/${PN}.conf" did, but no.
The if block could be streamlined, but for now, I've chosen to leave the original in place and duplicate some of the settings.
Regarding the method using awk, the find output didn't like being piped to awk, so find is wrapped with redirection. My original method used an actual file in the same position, so maybe it's a requirement rather than relying on a pipe?
sed is used to strip the leading './' path characters from find's output.
It's mostly straightforward if a bit ugly. Code: | # diff -u gentoo/sys-kernel/linux-firmware/linux-firmware-20241017-r2.ebuild local/sys-kernel/linux-firmware/linux-firmware-20241017-r2.ebuild
--- gentoo/sys-kernel/linux-firmware/linux-firmware-20241017-r2.ebuild 2024-10-22 12:11:21.000000000 -0600
+++ local/sys-kernel/linux-firmware/linux-firmware-20241017-r2.ebuild 2024-10-25 17:10:22.186148992 -0600
@@ -284,6 +284,7 @@
local FW_OPTIONS=( "-v" )
local files_to_keep=
+ local filter_file="/etc/portage/savedconfig/sys-kernel/linux-firmware.filter"
if use savedconfig; then
if [[ -s "${S}/${PN}.conf" ]]; then
@@ -291,6 +292,14 @@
grep -v '^#' "${S}/${PN}.conf" 2>/dev/null > "${files_to_keep}" || die
[[ -s "${files_to_keep}" ]] || die "grep failed, empty config file?"
FW_OPTIONS+=( "--firmware-list" "${files_to_keep}" )
+ elif [[ -s "${filter_file}" ]]; then
+ files_to_keep="${T}/files_to_keep.lst"
+ pushd "${WORKDIR}/${P}" &>/dev/null || die
+ awk 'NR==FNR{patterns[$1]; next} {for (p in patterns) if ($0 ~ p) print $0}' \
+ "${filter_file}" <(find |sed -e 's:\./::') > "${files_to_keep}" || die
+ popd &>/dev/null || die
+ [[ -s "${files_to_keep}" ]] || die "filter failed, pattern match error?"
+ FW_OPTIONS+=( "--firmware-list" "${files_to_keep}" )
fi
fi |
One thing that may be useful is an example of how to test the patterns. Perhaps a separate script? _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
pa4wdh l33t
Joined: 16 Dec 2005 Posts: 881
|
Posted: Sat Oct 26, 2024 7:09 am Post subject: |
|
|
I like your idea.
Last time i took the time to decrease the number of firmware files on my system (which i admit is quite some time ago ...) i had to list the files i do not want. The problem is that i don't know what has been added or where versions have changed.
Your method seems to allow to select files i do want, which is much more useful as i know what hardware i have and which files they need. The use of patters allows to write a pattern that matches the current and future versions so you don't have to track updates manually. _________________ The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world
My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20469
|
Posted: Sat Oct 26, 2024 3:30 pm Post subject: |
|
|
Potentially missing updates with the official method was the original motivator. Something about that just seemed very wrong.
The find statement needs to be fixed to handle files with spaces. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22570
|
Posted: Sat Oct 26, 2024 4:37 pm Post subject: |
|
|
Piping output from find ought to work. How did it fail for you, and what was the invocation?
You can avoid the sed by directing find to print without the leading .: find -printf '%P\n'. |
|
Back to top |
|
|
Goverp Advocate
Joined: 07 Mar 2007 Posts: 2164
|
Posted: Sat Oct 26, 2024 7:04 pm Post subject: |
|
|
IIUC this is so far just a proposal, in which case may I add a couple of suggestions:
a) allow very simple matching by filename, though permitting more extensive if desired. (This could almost be "grep -f <patternsfile>" against tree listing of the firmware directory, but using GLOB rather than regex...
[a SMOP, perhaps just a sed replace "*" with ".*", "?" with "." and "." with "[.]"]
so valid patterns might be: Code: | *amd_fam17h.bin
amdgpu/*
amdgpu/*polaris10* | and the Code: | /etc/portage/savedconfig/sys-kernel/linux-firmware.filter | could be a directory and the contents merged, so I might have
linux-firmware.filter/cpu
and
linux-firmware.filter/gpu
I appreciate making suggestions is so much easier than implementing them! _________________ Greybeard |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20469
|
Posted: Sat Oct 26, 2024 8:47 pm Post subject: |
|
|
Hu wrote: | Piping output from find ought to work. How did it fail for you, and what was the invocation? | Good question. I didn't make notes about it. I'll try to reproduce it. In general I'm guessing a pipe would be preferred due to pipefail checking.
Hu wrote: | You can avoid the sed by directing find to print without the leading .: find -printf '%P\n'. | Thanks. I almost never think of find's formatted output. I don't think I've ever tried to combine filename safety with formatted output, so that could be interesting. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20469
|
Posted: Sat Oct 26, 2024 9:05 pm Post subject: |
|
|
Goverp wrote: | IIUC this is so far just a proposal, in which case may I add a couple of suggestions: | Correct. Thank you.
Goverp wrote: | a) allow very simple matching by filename, though permitting more extensive if desired. (This could almost be "grep -f <patternsfile>" against tree listing of the firmware directory, but using GLOB rather than regex...
[a SMOP, perhaps just a sed replace "*" with ".*", "?" with "." and "." with "[.]"] | For now it's tied to awk pattern matching. I don't know what it uses. I suspect some stuff would just work. I should be able to add some additional tests.
Whether or not awk remains is likely to be influenced by whatever prospective maintainers think is a better solution. At the time, awk was the only way I was able identify a solution.
Goverp wrote: | so valid patterns might be: Code: | *amd_fam17h.bin
amdgpu/*
amdgpu/*polaris10* |
| I'll add those for testing. And I'll try to think of a way to use grep. But I think it would be an even uglier solution. Unless I'm missing something obvious.
Goverp wrote: | and the Code: | /etc/portage/savedconfig/sys-kernel/linux-firmware.filter | could be a directory and the contents merged, so I might have
linux-firmware.filter/cpu
and
linux-firmware.filter/gpu
I appreciate making suggestions is so much easier than implementing them! | I don't think that would be too difficult, Using one file as I have it now or all files in a directory _shouldn't_ be that much different. But using a directory seems like a bigger change as that affects stuff outside the ebuild, requiring (I presume) different approval and consideration.
Since I'm "abusing" savedconfig, I expect that to need some rework. At a minimum, the error / warning message would need to change.
I'll try to do some of those tests / changes tonight. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|