| View previous topic :: View next topic |
| Author |
Message |
carcajou
Apprentice
Joined: 10 Jun 2008
Posts: 248
|
 Posted: Tue Apr 09, 2024 9:18 pm Post subject: [solved] 6.6 kernel cannot boot after profile upgrade (LUKS) |
 |
|
Hi guys.
I moved from 17.1 to 23.0 profile last weekend. As part of upgrade process, the latest kernel was recompiled. However, I can't longer boot the system. I am using LUKS encrypted root partition and during boot process I am asked for passphrase. I can type in the passphrase, but boot process stands still, only the cursor is blinking. There are no error messages. Same kernel worked fine before profile upgrade.
Previous kernel 6.6.16 still works normally (but was not recompiled during profile switch).
I am using sys-kernel/gentoo-kernel (initramfs strip -debug -generic-uki -hardened -modules-compress -modules-sign -savedconfig -secureboot -test) with default config. Initramfs is built with dracut. The only thing I noticed is that size of initramfs image differs in 1MB:
| Code: |
# du -sh /boot/*
260K config-6.6.16-gentoo-dist
260K config-6.6.21-gentoo-dist
144K EFI
14M grub
12M initramfs-6.6.16-gentoo-dist.img
11M initramfs-6.6.21-gentoo-dist.img
14M intel-uc.img
9.1M System.map-6.6.16-gentoo-dist
9.1M System.map-6.6.21-gentoo-dist
15M vmlinuz-6.6.16-gentoo-dist
15M vmlinuz-6.6.21-gentoo-dist
|
| Code: |
# diff config-6.6.21-gentoo-dist config-6.6.16-gentoo-dist
3c3
< # Linux/x86 6.6.21 Kernel Configuration
---
> # Linux/x86 6.6.16 Kernel Configuration
5c5
< CONFIG_CC_VERSION_TEXT="x86_64-pc-linux-gnu-gcc (Gentoo 13.2.1_p20240210 p14) 13.2.1 20240210"
---
> CONFIG_CC_VERSION_TEXT="x86_64-pc-linux-gnu-gcc (Gentoo 13.2.1_p20240113-r1 p12) 13.2.1 20240113"
18d17
< CONFIG_GCC_ASM_GOTO_OUTPUT_WORKAROUND=y
|
| Code: |
# cat /etc/dracut.conf.d/*.conf
omit_dracutmodules+=" systemd systemd-initrd dracut-systemd i18n systemd-udevd "
early_microcode="no"
hostonly="yes"
compress="gzip"
|
| Code: |
Portage 3.0.61 (python 3.11.8-final-0, default/linux/amd64/23.0/split-usr/desktop/plasma, gcc-13, glibc-2.38-r11, 6.6.16-gentoo-dist x86_64)
=================================================================
System uname: Linux-6.6.16-gentoo-dist-x86_64-Intel-R-_Core-TM-_i7-10700_CPU_@_2.90GHz-with-glibc2.38
KiB Mem: 32618732 total, 27881400 free
KiB Swap: 2097148 total, 2097148 free
Timestamp of repository gentoo: Mon, 08 Apr 2024 20:04:07 +0000
Head commit of repository gentoo: ba24104b30818a8b52871ac02f3c997a34177f3a
Timestamp of repository guru: Mon, 08 Apr 2024 11:48:11 +0000
Head commit of repository guru: d5d04abc6181cc961117b6e09c8d171ed45569e4
sh bash 5.1_p16-r6
ld GNU ld (Gentoo 2.41 p5) 2.41.0
app-misc/pax-utils: 1.3.7::gentoo
app-shells/bash: 5.1_p16-r6::gentoo
dev-build/autoconf: 2.71-r6::gentoo
dev-build/automake: 1.16.5-r2::gentoo
dev-build/cmake: 3.28.3::gentoo
dev-build/libtool: 2.4.7-r4::gentoo
dev-build/make: 4.4.1-r1::gentoo
dev-build/meson: 1.3.2::gentoo
dev-java/java-config: 2.3.3-r1::gentoo
dev-lang/perl: 5.38.2-r2::gentoo
dev-lang/python: 3.11.8_p1::gentoo, 3.12.2_p1::gentoo
dev-lang/rust-bin: 1.75.0::gentoo
sys-apps/baselayout: 2.14-r2::gentoo
sys-apps/openrc: 0.54::gentoo
sys-apps/sandbox: 2.38::gentoo
sys-devel/binutils: 2.41-r5::gentoo
sys-devel/binutils-config: 5.5::gentoo
sys-devel/clang: 17.0.6::gentoo
sys-devel/gcc: 13.2.1_p20240210::gentoo
sys-devel/gcc-config: 2.11::gentoo
sys-devel/llvm: 17.0.6::gentoo
sys-kernel/linux-headers: 6.6-r1::gentoo (virtual/os-headers)
sys-libs/glibc: 2.38-r11::gentoo
Repositories:
gentoo
location: /var/db/repos/gentoo
sync-type: git
sync-uri: https://github.com/gentoo-mirror/gentoo.git
priority: -1000
volatile: False
guru
location: /var/db/repos/guru
sync-type: git
sync-uri: https://github.com/gentoo-mirror/guru.git
masters: gentoo
volatile: False
Binary Repositories:
binhost
priority: 9999
sync-uri: https://mirror.kumi.systems/gentoo/releases/amd64/binpackages/23.0/x86-64-v3
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="@FREE @BINARY-REDISTRIBUTABLE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe"
DISTDIR="/var/cache/distfiles"
ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GDK_PIXBUF_MODULE_FILE GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR XDG_STATE_HOME"
FCFLAGS="-march=native -O2 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs binpkg-multi-instance binpkg-request-signature buildpkg-live config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-march=native -O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-z,pack-relative-relocs"
LEX="flex"
MAKEOPTS="-j16 -l16"
PKGDIR="/var/cache/binpkgs"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
SHELL="/bin/bash"
USE="X a52 aac acl acpi activities alsa amd64 bash-completion bluetooth branding bzip2 cairo caps cdda cdr cet crypt cups dbus declarative dri dts dvd dvdr elogind encode exif flac flatpak gdbm gif gpm gtk gui heif iconv icu ipv6 jpeg kde kwallet lcms libnotify libtirpc lm-sensors lzma mad mng mp3 mp4 mpeg multilib ncurses networkmanager nls ogg opengl openmp pam pango pcre pdf pipewire plasma png policykit ppds pulseaudio qml qt5 readline screencast sdl seccomp semantic-desktop sound spell split-usr ssl startup-notification svg test-rust tiff truetype udev udisks unicode upower usb vaapi verify-sig vim-syntax vorbis vulkan wayland webp widgets wxwidgets x264 x265 xattr xcb xft xml xv xvid zip zlib zstd" ABI_X86="64" ADA_TARGET="gcc_12" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_anon authn_dbm authn_file authz_dbm authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir env expires ext_filter file_cache filter headers include info log_config logio mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 aes avx avx2 f16c fma3 pclmul popcnt rdrand sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 ntrip navcom oceanserver oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 tsip tripmate tnt ublox" GRUB_PLATFORMS="efi-64" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-US" LCD_DEVICES="bayrad cfontz glk hd44780 lb216 lcdm001 mtxorb text" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php8-1" POSTGRES_TARGETS="postgres15" PYTHON_SINGLE_TARGET="python3_11" PYTHON_TARGETS="python3_11" RUBY_TARGETS="ruby31" VIDEO_CARDS="intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipp2p iface geoip fuzzy condition tarpit sysrq proto logmark ipmark dhcpmac delude chaos account"
Unset: ADDR2LINE, AR, ARFLAGS, AS, ASFLAGS, CC, CCLD, CONFIG_SHELL, CPP, CPPFLAGS, CTARGET, CXX, CXXFILT, ELFEDIT, EMERGE_DEFAULT_OPTS, EXTRA_ECONF, F77FLAGS, FC, GCOV, GPROF, INSTALL_MASK, LC_ALL, LD, LFLAGS, LIBTOOL, LINGUAS, MAKE, MAKEFLAGS, NM, OBJCOPY, OBJDUMP, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PYTHONPATH, RANLIB, READELF, RUSTFLAGS, SIZE, STRINGS, STRIP, YACC, YFLAGS
|
Thank you.
Last edited by carcajou on Thu Apr 11, 2024 10:27 pm; edited 1 time in total |
|
| Back to top |
|
 |
erg_samowzbudnik
Apprentice
Joined: 09 Sep 2011
Posts: 223
Location: European sticks
|
| Posted: Wed Apr 10, 2024 11:01 am Post subject: |
|
|
Perhaps this is not the answer you seek, just my two cents.
For me, dracut stopped generating working initramfs with upgrade to 23.0 profile.
I switched to genkernel:
| Code: | | genkernel --lvm --luks initramfs |
If you do find out what changed, please let know. genkernel takes forever, I would gladly go back to dracut |
|
| Back to top |
|
|
carcajou
Apprentice
Joined: 10 Jun 2008
Posts: 248
|
| Posted: Wed Apr 10, 2024 9:35 pm Post subject: |
|
|
@erg_samowzbudnik
Thank you, but I would like to avoid using genkernel, primary because I want to use Portage to perform the upgrades as part of regular upgrade procedure.
I have used lsinitrd tool to inspect both initramfs images. It is difficult to compare the diff line by line, but this is the main difference:
| Code: |
Image: initramfs-6.6.21-gentoo-dist.img: 12M
========================================================================
Version: dracut-060
Arguments: --force --verbose --kernel-image '/usr/src/linux-6.6.21-gentoo-dist/arch/x86/boot/bzImage' --no-uefi
dracut modules:
crypt
dm
kernel-modules
kernel-modules-extra
rootfs-block
terminfo
udev-rules
usrmount
base
fs-lib
shutdown
|
| Code: |
Image: initramfs-6.6.16-gentoo-dist.img: 12M
========================================================================
Version: dracut-060
Arguments: --force --verbose --kernel-image '/usr/src/linux-6.6.16-gentoo-dist/arch/x86/boot/bzImage'
dracut modules:
crypt
dm
kernel-modules
kernel-modules-extra
rootfs-block
terminfo
udev-rules
usrmount
base
fs-lib
shutdown
|
So dracut for 6.6.21 passes --no-uefi option. I also did try to build manually image, but even with this manually generated image the result it the same - nothing happens after passphrase input. Following command was used:
| Code: |
dracut --force --verbose --kver 6.6.21-gentoo-dist --hostonly
|
I am not sure where to proceed from here? Is it possible that because some binary packages were used during upgrade process, that there is "something" that is causing this? Any tip is appreciated.
I also noticed another issue on shutdown/reboot - I have to switch to another shell and poweroff manually. Shutting down or rebooting from Plasma leaves the black screen with blinking cursor. There is this message in /var/log/messages:
| Code: |
/usr/bin/power[16228]: Failed to execute /usr/bin/poweroff: No such file or directory |
^ Solved the issue with poweroff/reboot. Recompiled sys-auth/elogind as replacement for binary package that was pulled during profile switch. Not sure if that is real cause of the issue, but it solved it for me. |
|
| Back to top |
|
|
figueroa
Advocate
Joined: 14 Aug 2005
Posts: 3005
Location: Edge of marsh USA
|
| Posted: Thu Apr 11, 2024 3:06 am Post subject: |
|
|
Experimentally, try the binary version of the gentoo-kernel. If that doesn't work as intended, ditch all the binary packages and compile those locally.
_________________
Andy Figueroa
hp pavilion hpe h8-1260t/2AB5; spinning rust x3
i7-2600 @ 3.40GHz; 16 gb; Radeon HD 7570
amd64/23.0/split-usr/desktop (stable), OpenRC, -systemd -pulseaudio -uefi |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22649
|
| Posted: Thu Apr 11, 2024 3:08 pm Post subject: |
|
|
| kukibl wrote: | | Code: | | /usr/bin/power[16228]: Failed to execute /usr/bin/poweroff: No such file or directory |
|
This looks like fallout from confusion over split-usr versus merged-usr. The traditional location of poweroff is /sbin/poweroff. Invoking it from /usr/bin can work if the system is merged-usr, since everything from /sbin will have been dumped into /usr/bin. Otherwise, it needs to be invoked from its actual path, or found via a search of $PATH. |
|
| Back to top |
|
|
carcajou
Apprentice
Joined: 10 Jun 2008
Posts: 248
|
| Posted: Thu Apr 11, 2024 10:32 pm Post subject: |
|
|
Gentoo-kernel-bin did not solve it, the issue remained. In the end I picked only packages from "sys-" categories and reinstalled them from source in order to replace binaries. That solved the issue for me (it was around 60 packages in total).
Thank you all for the feedback.
| Hu wrote: | | This looks like fallout from confusion over split-usr versus merged-usr. |
Does this mean that it is not good idea to use binaries for split-usr profiles yet (or ever)? |
|
| Back to top |
|
|
Atha
Apprentice
Joined: 22 Sep 2004
Posts: 230
|
| Posted: Fri Apr 12, 2024 10:44 am Post subject: |
|
|
| kukibl wrote: | Gentoo-kernel-bin did not solve it, the issue remained. In the end I picked only packages from "sys-" categories and reinstalled them from source in order to replace binaries. That solved the issue for me (it was around 60 packages in total).
Thank you all for the feedback.
| Hu wrote: | | This looks like fallout from confusion over split-usr versus merged-usr. |
Does this mean that it is not good idea to use binaries for split-usr profiles yet (or ever)? |
I don't think so. In my experience it's dracut that is the problem, most of the times.
Example: I just recently switched from genkernel to UKI, including generating my own secure boot keys, and now I'm using dracut as well. I also enabled secure boot on my system firmware. Genkernel was not very easy to setup in this scenario, and it was actually the missing continued development of genkernel that made me switch in the first place.
So, now I'm a new user to dracut. As it was, with the switch to UKI (unified kernel images) and secure boot, I also created a new ESP (EFI System Partition) + XBOOTLDR partition (both VFAT) configuration (before it was ESP as VFAT + /boot as ext2/ext4 minus journal).
I already had a working UKI, and systemd-boot picked it up, everything worked. In the UKI I'm using dracut. So, I had removed /boot and added XBOOTLDR. As it later turned out to be the case this only worked because the XBOOTLDR partition was already in place when I created the UKI.
Now, booted with the working kernel and having removed the old ones I couldn't use anymore anyway, I also recreated the ESP, so it had a new volume ID, you know, this 12AB-34CD kind of 1980's version of a UUID. So, what happened now was that the UKI refused to boot, specifically the dracut part: it kept saying it cannot find the ESP (the old one, with the old volume ID). PLEASE ANYONE TELL MY WHY THIS IS NECESSARY? The UKI was already loaded, and dracut should have just ignored this, because in my fstab the new ESP was defined anyways.
The workaround was the funniest: I just created a 1 MB VFAT ESP partition on an USB stick (or an external USB HDD, cannot remember) and gave it the missing volume ID that dracut was looking for. There was nothing on that partition. Now I could boot into my system again, but I had to recompile the kernel image, re-run install-kernel and everything to get an updated UKI.
How stupid is that?!?
And now I have the same issue: dracut refuses to boot when the image is created while a bcachefs filesystem is mounted. I tried bcachefs as my new /var partitions filesystem and I am very happy with it. All previous UKIs work fine. Every new one fails, it simply reboots. I have to unmount my bcachefs, then create the new kernel UKI, and then dracut can boot WITH the bcachefs now in place from my fstab.
So you see: it really is dracut, that is to blame. Often. Most of the time.
I think, dracut is too big. It checks too much. Hence all those errors.
_________________
Think for yourself and let others enjoy the privilege of doing so too. – Voltaire |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22649
|
| Posted: Fri Apr 12, 2024 2:52 pm Post subject: |
|
|
| kukibl wrote: | | Hu wrote: | | This looks like fallout from confusion over split-usr versus merged-usr. |
Does this mean that it is not good idea to use binaries for split-usr profiles yet (or ever)? |
I think it would be best to only use binaries built for the same type of usr you have locally. When on a merged-usr system, use binaries built for merged-usr. When on a split-usr system, use binaries built for split-usr. In many cases, you can probably get away with using the "wrong" type, and using binaries that expect split-usr will probably work on merged-usr, assuming they do not suffer file collisions during installation. You were trying to use a binary that expected merged-usr[1], but running it on a split-usr system. The safest course is not to use foreign usr binaries.
[1]: More specifically, it used the wrong path to something, and because merged-usr dumps everything into one directory, people on merged-usr systems can get away with using the wrong path. |
|
| Back to top |
|
|
carcajou
Apprentice
Joined: 10 Jun 2008
Posts: 248
|
| Posted: Fri Apr 12, 2024 8:47 pm Post subject: |
|
|
| Hu wrote: | You were trying to use a binary that expected merged-usr[1], but running it on a split-usr system. The safest course is not to use foreign usr binaries.
|
I am not sure I get this... 
My old profile was "default/linux/amd64/17.1/desktop/plasma". Respecting the profile upgrade table https://wiki.gentoo.org/wiki/Project:Toolchain/23.0_update_table, correct new profile is "default/linux/amd64/23.0/split-usr/desktop/plasma" and binhost directory is "amd64/binpackages/23.0/x86-64-v3".
What did I do wrong? How can I know if the binary expects split-usr or merge-usr? |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22649
|
| Posted: Fri Apr 12, 2024 10:20 pm Post subject: |
|
|
| That seems correct to me. Did you get /usr/bin/power from the binhost in that directory? What package provides /usr/bin/power? Perhaps whoever built it is now always configuring for merged-usr paths, even when building on a split-usr profile. |
|
| Back to top |
|
|
carcajou
Apprentice
Joined: 10 Jun 2008
Posts: 248
|
| Posted: Sat Apr 13, 2024 10:55 am Post subject: |
|
|
| Hu wrote: | | What package provides /usr/bin/power? Perhaps whoever built it is now always configuring for merged-usr paths, even when building on a split-usr profile. |
I am not sure. There is not/was not one on my system. Some Google result showed me that it could be related to elogind, so I recompiled it considering the elogind package at that moment was a binary one. It solved the issue. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Kernel & Hardware
