| View previous topic :: View next topic |
| Author |
Message |
vimja
n00b
Joined: 26 Mar 2022
Posts: 23
Location: Bern, Switzerland
|
 Posted: Mon Jan 30, 2023 4:20 pm Post subject: Building an uncompressed kernel for ARM64 |
 |
|
Hi
I have written an ebuild for a custom kernel for the ARM64 based MNT Reform2 laptop.
The ebuild is based on sys-kernel/gentoo-kernel which is extended with some patches and configuration from the MNT Reform2 project. You can find the ebuild here: https://git.chaostreffbern.ch/vimja/mnt-reform2-overlay/-/tree/master/sys-kernel/mnt-reform2-kernel
As it stands right now, the ebuild produces a working kernel. However, the image placed at /boot/vmlinuz-6.1.4... is gzip compressed whereas I need an uncompressed kernel image for u-boot to be able to load it.
Sure I could update the u-boot configuration and boot script to handle the gzip compressed kernel but I would really like to be able to use the upstream u-boot to make installing Gentoo easier and more seamless.
I have tried getting an uncompressed kernel by setting CONFIG_KERNEL_GZIP=n and CONFIG_KERNEL_UNCOMPRESSED=y, but that did not work. I still ended up with a GZIP compressed image. Trying to find out why that is, I ran ebuild [...] prepare and then manually ran make olddefconfig which appears to be pretty much the first thing the ebuild does after preparing the sources. Doing so removed my carefully placed options, both CONFIG_KERNEL_GZIP and CONFIG_KERNEL_UNCOMPRESSED.
I tried to come up with a working configuration through manually editing the config file and also using menuconfig, but to no avail. Each time I ran make olddefconfig, the options were lost.
So now I'm thinking that I'm probably doing something wrong. Any ideas? |
|
| Back to top |
|
 |
Hu
Administrator
Joined: 06 Mar 2007
Posts: 23053
|
| Posted: Mon Jan 30, 2023 5:31 pm Post subject: |
|
|
| As I read the kernel configuration language, KERNEL_UNCOMPRESSED depends on HAVE_KERNEL_UNCOMPRESSED. If you do not have the latter, I would not be surprised if make olddefconfig is removing KERNEL_UNCOMPRESSED. HAVE_KERNEL_UNCOMPRESSED is automatically set on S/390, and not on any other architecture. However, as I understand it, this compression is an internal feature. The bootloader should neither know nor care that the kernel is mostly gzip-compressed. What output led you to believe that this is a problem? |
|
| Back to top |
|
|
vimja
n00b
Joined: 26 Mar 2022
Posts: 23
Location: Bern, Switzerland
|
| Posted: Mon Jan 30, 2023 5:55 pm Post subject: |
|
|
| Hu wrote: | | What output led you to believe that this is a problem? |
Here is how I came to think that this configuration option might be what I am looking for:
| Code: |
file /boot/vmlinuz-6.1.4-mnt-reform2
|
| Code: |
/boot/vmlinuz-6.1.4-mnt-reform2: Linux kernel ARM64 boot executable Image, little-endian, 4K pages
|
This is what the output looks for the kernel from the upstream Debian image. Using this, the system boots. Here is the same command run against the Kernel from my ebuild:
| Code: |
file /boot/vmlinuz-6.1.4-mnt-reform2
|
| Code: |
/boot/vmlinuz-6.1.4-mnt-reform2: gzip compressed data, max compression, from Unix, original size modulo 2^32 37235200
|
If I try to boot using this kernel, u-boot complains about Bad Linux ARM64 Image magic!.
So I went looking for the mechanism that causes the kernel image in /boot to be GZIP compressed. This option is what I have found.
To be honest, I am not sure if this option is really the correct one. I could be very much off on this. |
|
| Back to top |
|
|
vimja
n00b
Joined: 26 Mar 2022
Posts: 23
Location: Bern, Switzerland
|
| Posted: Mon Jan 30, 2023 6:06 pm Post subject: |
|
|
| Hu wrote: | | HAVE_KERNEL_UNCOMPRESSED is automatically set on S/390, and not on any other architecture. |
With this hint and some help from the matrix channel, I have now come up with a thing that might be working.
I have patched arch/arm64/Kconfig like so:
| Code: |
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -209,6 +209,14 @@
select HAVE_KPROBES
select HAVE_KRETPROBES
select HAVE_GENERIC_VDSO
+ select HAVE_KERNEL_BZIP2
+ select HAVE_KERNEL_GZIP
+ select HAVE_KERNEL_LZ4
+ select HAVE_KERNEL_LZMA
+ select HAVE_KERNEL_LZO
+ select HAVE_KERNEL_UNCOMPRESSED
+ select HAVE_KERNEL_XZ
+ select HAVE_KERNEL_ZSTD
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
select KASAN_VMALLOC if KASAN
|
This prevents make olddefconfig from removing the configuration. Here is the .config after the prepare stage:
| Code: |
grep CONFIG_KERNEL .config
|
| Code: |
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZ4 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_LZO is not set
CONFIG_KERNEL_MODE_NEON=y
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_ZSTD is not set
CONFIG_KERNEL_GZIP=n
CONFIG_KERNEL_UNCOMPRESSED=y
|
and
| Code: |
grep CONFIG_HAVE_KERNEL .config
|
(empty result).
And here after the make olddefconfig
| Code: |
grep CONFIG_KERNEL .config
|
| Code: |
# CONFIG_KERNEL_GZIP is not set
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_LZO is not set
# CONFIG_KERNEL_LZ4 is not set
# CONFIG_KERNEL_ZSTD is not set
CONFIG_KERNEL_UNCOMPRESSED=y
CONFIG_KERNEL_MODE_NEON=y
|
and
| Code: |
grep CONFIG_HAVE_KERNEL .config
|
| Code: |
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_HAVE_KERNEL_LZ4=y
CONFIG_HAVE_KERNEL_ZSTD=y
CONFIG_HAVE_KERNEL_UNCOMPRESSED=y
|
Where as before, without the patch, both results would be empty after the make olddefconfig step.
As mentioned in my previous post, I am not entirely certain that this option is actually what controls whether the kernel image will be compressed or not. I'll give it a try and let you know. It will be a while though, as compiling the kernel takes 8 hours on this hardware. |
|
| Back to top |
|
|
vimja
n00b
Joined: 26 Mar 2022
Posts: 23
Location: Bern, Switzerland
|
| Posted: Mon Jan 30, 2023 6:49 pm Post subject: |
|
|
| vimja wrote: |
So I went looking for the mechanism that causes the kernel image in /boot to be GZIP compressed. This option is what I have found.
To be honest, I am not sure if this option is really the correct one. I could be very much off on this. |
Having looked a bit more, I think I have found some more hints supporting the theory that this option controls how the kernel image gets compressed.
Here is the relevant section of the base configuration for sys-kernel/gentoo-kernel: https://github.com/projg2/gentoo-kernel-config/blob/master/base.config#L61-L63
| Quote: |
| Code: |
## Use gzip compression for better compatibility
## https://bugs.gentoo.org/784599
CONFIG_KERNEL_GZIP=y
|
|
In combination with the discussion in the linked bug, I do actually think that I'm on the right track. |
|
| Back to top |
|
|
pingtoo
Veteran
Joined: 10 Sep 2021
Posts: 1459
Location: Richmond Hill, Canada
|
| Posted: Mon Jan 30, 2023 7:21 pm Post subject: |
|
|
| vimja wrote: | | Hu wrote: | | What output led you to believe that this is a problem? |
Here is how I came to think that this configuration option might be what I am looking for:
| Code: |
file /boot/vmlinuz-6.1.4-mnt-reform2
|
| Code: |
/boot/vmlinuz-6.1.4-mnt-reform2: Linux kernel ARM64 boot executable Image, little-endian, 4K pages
|
This is what the output looks for the kernel from the upstream Debian image. Using this, the system boots. Here is the same command run against the Kernel from my ebuild:
| Code: |
file /boot/vmlinuz-6.1.4-mnt-reform2
|
| Code: |
/boot/vmlinuz-6.1.4-mnt-reform2: gzip compressed data, max compression, from Unix, original size modulo 2^32 37235200
|
If I try to boot using this kernel, u-boot complains about Bad Linux ARM64 Image magic!.
So I went looking for the mechanism that causes the kernel image in /boot to be GZIP compressed. This option is what I have found.
To be honest, I am not sure if this option is really the correct one. I could be very much off on this. |
In general u-boot expect kernel binary be wrapped in u-boot image format. which is anything with some special header tell u-boot the content is compressed or not and where to place in ram.
It looks to me your ebuild produced the kernel first, then ran gzip on the produced kernel. that seems missing a step that wrap the gziped kernel in to u-boot image format. have you try to use
mkimage to create a uImage? |
|
| Back to top |
|
|
vimja
n00b
Joined: 26 Mar 2022
Posts: 23
Location: Bern, Switzerland
|
| Posted: Mon Jan 30, 2023 7:36 pm Post subject: |
|
|
| pingtoo wrote: |
In general u-boot expect kernel binary be wrapped in u-boot image format. which is anything with some special header tell u-boot the content is compressed or not and where to place in ram.
|
I do have a u-boot image or boot script. It is made using mkimage as suggested by you. I have documented both the script it's creation in the wiki: https://wiki.gentoo.org/wiki/MNT_Reform#boot_script
Indeed, as you mention, the booi command used in this script to load the kernel is able to load a compressed kernel, given the right parameters.
However, the MNT Reform2 u-boot does not have GZIP support compiled in: https://source.mnt.re/reform/reform-boundary-uboot/-/blob/master/mntreform-config#L1322
I can, and in fact I have, compiled a version of u-boot with GZIP support. However, I would really prefer to be able to use the boot loader provided by MNT, in order to reduce the friction for new users.
Their u-boot does support LZMA compression though. I considered using that instead. But I think the mechanism for swtiching from GZIP to LZMA might be the same as disabling compression altogether. However I would really like to avoid having to figure out a bunch of additional memory parameters for u-boot. Plus the reference Debian image uses an uncompressed kernel, so all in all I figured it would be the safest option to find a way to build an uncompressed kernel.. |
|
| Back to top |
|
|
pingtoo
Veteran
Joined: 10 Sep 2021
Posts: 1459
Location: Richmond Hill, Canada
|
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54782
Location: 56N 3W
|
| Posted: Mon Jan 30, 2023 9:50 pm Post subject: |
|
|
vimja,
The uncompressed kernel should be at arch/arm64/boot/Image once make has run.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
vimja
n00b
Joined: 26 Mar 2022
Posts: 23
Location: Bern, Switzerland
|
| Posted: Tue Jan 31, 2023 6:23 pm Post subject: |
|
|
| vimja wrote: |
Having looked a bit more, I think I have found some more hints supporting the theory that this option controls how the kernel image gets compressed.
|
My new kernel has built and I am now pretty sure that I was on the wrong track all along! While my options were retained in the configuration, the image is still compressed:
| Code: |
grep --perl-regexp '(CONFIG_KERNEL|CONFIG_HAVE_KERNEL)' /boot/config-6.1.8-mnt-reform2
|
| Code: |
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_HAVE_KERNEL_LZ4=y
CONFIG_HAVE_KERNEL_ZSTD=y
CONFIG_HAVE_KERNEL_UNCOMPRESSED=y
# CONFIG_KERNEL_GZIP is not set
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_LZO is not set
# CONFIG_KERNEL_LZ4 is not set
# CONFIG_KERNEL_ZSTD is not set
CONFIG_KERNEL_UNCOMPRESSED=y
CONFIG_KERNEL_MODE_NEON=y
|
| Code: |
file /boot/vmlinuz-6.1.8-mnt-reform2
|
| Code: |
/boot/vmlinuz-6.1.8-mnt-reform2: gzip compressed data, max compression, from Unix, original size modulo 2^32 37349888
|
So the compression of this file is quite definitely not controlled by the config.
| NeddySeagoon wrote: |
The uncompressed kernel should be at arch/arm64/boot/Image once make has run. |
Not quite. There is however, a file at arch/arm64/boot/Image.gz. But that isn't the main problem I'm facing. I know how I can get an uncompressed image, the thing is how I can get the ebuild to install it for me with as little of a hassle as possible.
Yes, that is right. I rely on the eclasses provided by Gentoo for most of the work. The changes in my ebuild compared to the reference sys-kernel/gentoo-kernel are very minimal.
But that is also what appears to be my problem. parona over in the matrix channel has looked into this a bit further and found the specific parts of the eclasses that are of concern:
This would be the code that selects the file that will be installed: https://gitweb.gentoo.org/repo/gentoo.git/tree/eclass/dist-kernel-utils.eclass#n64
The way it looks it is hardcoded to use Image.gz, so there is not much I can do without ugly hackery :/ |
|
| Back to top |
|
|
pingtoo
Veteran
Joined: 10 Sep 2021
Posts: 1459
Location: Richmond Hill, Canada
|
| Posted: Tue Jan 31, 2023 8:33 pm Post subject: |
|
|
I don't think you need to make a ugly hackery. You can just follow standard ebuild flow. | example: | src_install() {
local myversion="-mnt-reform2"
local img_path=$(find ${S}/arch/arm64 -name Image -print)
installkernel ${PV}${myversion} $img_path ${S}/System.map
} |
As I stated earlier I am no ebuild expert so take above code as guide, it is most likely have error in it. But the idea is to just find the file "Image" which is the uncompressed kernel then call sys-kernel/installkernel-gentoo to install into /boot/vmlinuz-xxx-mnt-reform2. |
|
| Back to top |
|
|
Nowa
Developer
Joined: 25 Jun 2014
Posts: 473
Location: Nijmegen
|
| Posted: Mon Jan 13, 2025 8:57 am Post subject: |
|
|
| pingtoo wrote: | | example: | src_install() {
local myversion="-mnt-reform2"
local img_path=$(find ${S}/arch/arm64 -name Image -print)
installkernel ${PV}${myversion} $img_path ${S}/System.map
} |
|
It is not possible to call installkernel in the install phase, access to ROOT is required so it must be done in the postinst phase.
| Quote: | | Not quite. There is however, a file at arch/arm64/boot/Image.gz. But that isn't the main problem I'm facing. I know how I can get an uncompressed image, the thing is how I can get the ebuild to install it for me with as little of a hassle as possible. |
Two solutions:
- Custom kernel installation hook to uncompress the to-be-installed kernel image before 90-loaderentry.install does its thing, or the easier solution:
- Enable CONFIG_EFI_ZBOOT for your kernel, we have support for this in the eclass, but this option is not enabled by default. It is however enabled if the secureboot USE flag is enabled, and it is also enabled by default in the prebuilt sys-kernel/gentoo-kernel-bin. It is simply for backwards compatibility reasons that it is not enabled by default always.
_________________
OS: Gentoo 6.10.12-gentoo-dist, ~amd64, 23.0/desktop/plasma/systemd
MB: MSI Z370-A PRO
CPU: Intel Core i9-9900KS
GPU: Intel Arc A770 16GB & Intel UHD Graphics 630
SSD: Samsung 970 EVO Plus 2 TB
RAM: Crucial Ballistix 32GB DDR4-2400 |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Kernel & Hardware
