NVMe drive not booting kernel from (u)EFI

[AECFXI]

I'm migrating my Gentoo installation from an older SATA-connected SSD to a newer NVME M.2 connected SSD. The NVMe root partition is an F2FS filesystem type. I am also migrating from GRUB2 BIOS to GRUB UEFI or rEFInd, or an EFI stub. The original intent was rEFInd specifically.

I have NVME enabled in the kernel, as well as F2FS, the EFI system partition and the steps for EFI stub support. My kernel is 5.10.52.

I have the new drive partitioned appropriately, with a FAT32 (appropriate per rEFInd) partition /dev/nvme0n1p1 that has the boot and efi partition labels enabled. I copied my gentoo install over from the old SSD onto /dev/nvme0n1p3 with a cp -a, and updated the fstab to reflect the new partitions.

Booting into the "SystemRescue" live environment, my new system root directory /dev/nvme0n1p3 is detected and I can enter into it from the live environment. Although KDE isn't working, it seems to be an otherwise functional gentoo system and I installed GRUB UEFI from within it.

I've been able to identify both rEFInd and the GRUB UEFI in my bios configuration and I've been able to enter both the rEFInd and grub's UEFI environments. They both correctly locate my kernel and I can navigate options, etc.

However neither rEFInd and GRUB UEFI can boot the kernel and enter the /dev/nvme0n1p3 root directory. In GRUB UEFI it hangs at "Loading Linux 5.10.52-gentoo...". In rEFInd, it hangs after a nearly identical prompt including the text "using load options", and shows my /dev/nvme0n1p3 address as the intended root directory as well as the kernel image being used.

I am at a loss for options here. This is the behavior I would expect from my kernel if I didn't nave NVME or F2FS support enabled, but support is enabled. Clearly there is no problem with the motherboard's bios detecting the UEFI and EFI boot information on the NVME drive and starting up those environments. Clearly the problem isn't a unqiue rEFInd or GRUB UEFI issue as both environments demonstrate identical behavior. And clearly my gentoo install on /dev/nvme0n1p3 is reasonably operational as I can enter a working gentoo system on that partition from the SystemRescue live system. F2FS shows every indication of being a reasonably supported filesystem, and this is exactly the use case I would expect for it: having the base system on a high speed NAND drive running F2FS.

Let me know what logs or other information I can upload to help get support.

[Jaglover]

[AECFXI]

[Jaglover]

I wonder what you mean by "entering root". Bootloader locates the kernel and attempts to load it. From your problem description it seems it hangs while trying to load the kernel, therefore it seems to me like there is some trouble reading the kernel image. You sure this kernel is good? And the filesystem is not corrupted in ESP? Once the kernel is loaded it runs, initializes the hardware and finally attempts to access root, but yours never gets that far.
_________________
My Gentoo installation notes.
Please learn how to denote units correctly!

[AECFXI]

[pietinger]

[Tony0945]

Please post your refind.conf and refind-linux.conf. IMO, placing kernels on FAT is a bad idea, but should be possible (they may have become corrupted).
You also need UEFI support in the kernels. I tripped up on this going from non-EFI to EFI.

On my system these files are:
/boot/efi/EFI/refind/refind.conf
and /boot/efi/EFI/refind/refind.conf
Use separate pastebins, please.

I'm thinking bad boot parameters.

EDIT:
I mount the FI partition as /boot/efi
It's a good sign that refind attempts to load the jernel. That means it found it.

[AECFXI]

Re: corrupted vfat partition, fsck.vfat reveals:

- Differences between boot sector and its backup
- 'This is mostly harmless. Differences: (offset:original/backup)'
- copied original to backup (should have done backup to original oops)
- removed dirty bit
- used 'make install' to replace the kernel image in /boot
- Unchanged behavior

This appears consistent with how I'm unable to cleanly unmount the fat32 partition when the kernel fails to load, and forced to hard-reset the system.

I also used mkfs.vfat to create a fresh fat32 partition to redo the rEFInd boot method with no change in behavior.

I can understand the desire not to have the kernel on a vulnerable fat32 filesystem, but it was how I was instructed to create the EFI/UEFI partition and where I was instructed to put the kernel. It seems like rEFInd may support an ext4 (for example) /boot partition with a EFI partition mounted /boot/EFI, and then rEFInd reads the kernel file written on /boot, but it's not clear from the wiki how exactly how to achieve this and I feel like I've reasonably ruled out a fat corruption for my present issues.

---

I re-attempted a direct EFI stub. I emptied the /boot directory except for the singular file EFI/Gentoo/bzImage-5.10.52.efi with EFI/Gentoo as the only folders.

I then installed an EFI entry with the command "efibootmgr -c -d /dev/nvme0n1 -p 1 -L "Gentoo" -l '\EFI\Gentoo\bzImage-5.10.52.efi'

The kernel configuration requirements were also fulfilled, where my build-in kernel command string is "root=/dev/nvme0n1p3".

When booting this way, my computer hangs on a bios flash screen I don't normally see with no kernel output.

---

Re: pietinger:

I agree that it doesn't quite seem like a kernel problem in that I never get any output from the kernel, so it doesn't seem like the kernel actually gets executed because of this, but it should also be observed that GRUB UEFI and rEFInd both locate the kernel file and their final output reports the appropriate kernel version from the file I'd placed into the partition, linux 5.10.52, and GRUB/rEFInd's final report is that they are loading it.

---

Re: Tony0945

I would love for this to be a wrong boot parameters problem because that would be immediately fixable.

Atm I used 'make install' to drop the linux kernel into /boot which reEFInd is successfully locating and providing a boot option for. Moving the kernels and refind_linux.conf into EFI/Gentoo/ gives me a gentoo icon in the rEFInd menu but no other change in behavior.

/boot/refind_linux.conf (alongside my kernels)
https://pastebin.com/fpaTX9s3

/boot/EFI/refind/refind.conf
https://pastebin.com/0Vcrcmds

---

The last item of consideration didn't seem applicable at first, but given there is so little to work off here maybe it could be something of consideration. I have a GA-X79-UD3 motherboard from 2014 that was not released with NVMe support, nor was an official update for NVMe support ever released. Instead, I'm running a community modded BIOS that has been patched to include NVMe support. It's been attested by other users as working, and I've so far been able partition, format, read, and write execute files on my NVMe drive. fdisk speed tests show 3gb/s read/writes with the drive, which are the expected values. The EFI labeled partition on the NVMe drive populates within the BIOS menu as a boot option and these rEFInd and GRUB environments load up and work fine off my NVMe drive, it's just when I launch the kernel I get no activity and no output. This is my modded bios: https://www.win-raid.com/t2800f16-Bios-Driver-Modules-Versions-Question-For-GA-X-UD-Rev-F-modded.html#msg89146

---

Going to blow up the /boot partition again and try for a GRUB BIOS, will report my findings.

[AECFXI]

Also, I am not using an initramfs setup. My perception was that it was for a specific use case involving split system partitions which I don't have, but I can try setting up an initramfs if it would potentially give me a working system.

[AECFXI]

I GOT IT WORKING!

There was ONE SINGULAR CHANGE to my kernel that was required that made rEFInd (my most recent re-attempt at a bootloader) work perfectly.

Device Drivers -> Graphics Support -> Frame Buffer Devices ->
<*> Support for frame buffer devices --->
[*] EFI-based Framebuffer Support

CONFIG_FB_EFI was unset, and setting it made everything work in an instant. I believe this kernel configuration option should be added to the EFI System Partition gentoo wiki page as a kernel configuration requirement, I will make an edit later this evening. This configuration option referenced in this thread: https://forums.gentoo.org/viewtopic-t-1096052-start-0.html but the thread's discussion and resolution was a little bit difficult for me to parse on my first read through.

[Tony0945]

First, why are you mounting root as read only?

[Tony0945]

Ah yes, the frame buffer! No, I don't think it's in any instructions. I just enabled everything EFI. Didn't think about that.
I did think "maybe the vga="

Also consider PARTUUID as in my example. But if there is only one bootable device, it is moot.

Another tip: If you want to make another kernel the default just use "touch". The actual build time isnt used, rather the file creation time which touch alters.

Welcome to Gentoo and refind!

[AECFXI]

[Jaglover]

So your kernel was booting, but you were unable to see it and thought it hung.
_________________
My Gentoo installation notes.
Please learn how to denote units correctly!

[AECFXI]

[Tony0945]

2006. You've been around about as long as me. I was fooled by the "noob".

As i think about it, there s no reason for the kernel to be rw. The filesystem must be rw.
I've always made them rw.
Regarding PARTUIID. I had hoped that LABEL would work, but label works in /etc/fstab but not on kernel command line.

Yeah, our paths crossed while posting.

I spent three hours at the eye doctor today wearing a mask. my lungs and brain have not quite recovered.

[DONAHUE]

kernel cmdline can use PARTLABEL or PARTUUID
_________________
Defund the FCC.

[wjb]

Maybe confirm what EFI thinks it has

[Tony0945]

[pietinger]

[Tony0945]

Yes, it is a gpt disk. my understanding is that you can't haven EFI booting system without gpt.

[pietinger]

[Tony0945]

[Goverp]

[pietinger]