initramfs... how often do you rebuild?

eccerr0r

Ugh. It looks like I ran into a longstanding "bug" with Gentoo lvm USE, it doesn't quite handle caches. As I have a custom initramfs it means it's time to recut it with a new lvm so I can try caching my rust spinners.
Here's the sizes of my initramfs... Note the timestamp of the old one should actually be much older, I extracted the binaries much earlier and changed the scripts slightly (which I did script...)

Goverp · Advocate Joined: 07 Mar 2007 Posts: 2178

Since the vote missed my option, I didn't.

My kernel has an initramfs cpio list, so it's built from my system's current binaries every time I make a new kernel.

Good points of doing it this way is it avoids precisely the problem you have.
Bad points are (1) you might get something you didn't want if the current binaries are wrong. Shouldn't really be an issue if you keep old kernels for backup; (2) occaisionally you need to update the cpio list as the binaries (if you use dynamic ones) might change soname versions. I got hit by this a few months back; as it was the first time for several years, it took me a while to work out what was wrong. So you can argue that if your initramfs works, why risk breaking it? (3) If you build your kernel as non-root, it's possible that privileged programs don't get included in the initramfs due to read access restrictions.

I use a script to build the cpio list, so all my initramfs is dynamically linked (including busybox).
_________________
Greybeard

pa4wdh · l33t Joined: 16 Dec 2005 Posts: 881

I usually try to avoid using an initramfs in the first place, i never liked them

I have one system where i really can't (it's a VPS with it's root on LVM) and there i use genkernel to build one when i update the kernel.
_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world

My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com

NeddySeagoon · Posted: Mon Mar 29, 2021 10:33 am Post subject:

eccerr0r,

My initrd is usersace tools to start raid, start LVM, and mount root, /usr and /var from their logical volumes.
Its 12 years old next month, its like firmware, so why would i ever rebuild it?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Whissi · Retired Dev Joined: 12 Jan 2011 Posts: 222

Yes, that's an important difference: An initramfs can be anything. So the poll doesn't make much sense:

If you use your initramfs for userspace tools only, you probably do not have to rebuild until you want a new feature/bugfix.
But once you put kernel modules into your initramfs you will have to rebuild with every kernel build.
_________________
Regards,
Whissi

Gatsby · Posted: Mon Mar 29, 2021 1:36 pm Post subject:

No initramfs, never needed one. My three Gentoo systems with separate /usr partition work fine without an initramfs.
_________________
"Its your Gentoo, your way. When it breaks, you can keep all the pieces."
-- NeddySeagoon@forums.gentoo.org

flysideways · Guru Joined: 29 Jan 2005 Posts: 490

I don't use any now.

Maybe back when MythTV did not yet have storage groups and my video was stored on a software raid array, not sure. I do have a distant, non-distinct memory of going through the process of creating them and listing them in grub.

eccerr0r · Posted: Mon Mar 29, 2021 3:12 pm Post subject:

That is true depending on your use model.

I'm assuming that people don't put kernel modules in the initramfs and theoretically you do not need to rebuild initramfs every kernel release.

... except when the userspace tools have a bug or lack a feature you need and you need to rebuild it.

It's been many years, probably near a decade since I built my initramfs and now that I needed a new feature, it needs to be rebuilt. Just judging by the size of the binaries it must have been a very long time...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

Hu · Administrator Joined: 06 Mar 2007 Posts: 22626

For systems that need an initramfs, I have the kernel build system construct and embed it, so that I do not need to keep multiple files together. This is inefficient, since it means I have multiple nearly redundant copies, one per kernel. It also means that any time a user tool changes its requirements (such as by gaining a dependency on a new library), I need to modify the list of files I put into the initramfs. Despite these problems, I do it this way because the initramfs construction is completely automatic when no changes are required, because I don't need to keep track of which kernel uses which initramfs, and because rebuilding it every time means that I have some confidence that I can rebuild it. As others have described above, some people have an initramfs that has not been changed in so long that there is a real question of whether they still can rebuild it and get something that works. With mine, every time something breaks, I learn about it the first time I try to boot a new kernel, when it is usually fairly fresh what has changed and what could be responsible.

eccerr0r · Posted: Mon Mar 29, 2021 4:35 pm Post subject:

Exactly.

I had to go through two revs already to fix the userspace tools that stopped working in my initramfs. First one I caught by simply chrooting into the initramfs tree and found that the dynamic linker no longer works with the new binaries, luckily no reboot cycle needed. Second one seems to be an lvm change where it no longer populates /dev automatically where it had in the past. This required a reboot and manual recovery so that I could boot again (of course I had the old initramfs around so I could boot. Luckily no modules are in the initramfs, so despite a new kernel was built, it could still use the old binaries just fine.)

Actually I may or may not have had to rebuild initramfs, but I didn't want to take a chance for my lvm issue.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

alamahant · Advocate Joined: 23 Mar 2019 Posts: 3916

I always use dracut because i have my "/" on lvm.
gentoo-kernel-bin automatically updates initramfs,but if i use my manually compiled kernel,then i rebuild it also manually.
On a related note is it possible to boot root on lvm without initramfs?
_________________

eccerr0r · Posted: Mon Mar 29, 2021 6:09 pm Post subject:

Yeah, traditionally it seems that initramfs is necessary to have root on lvm. However I recall seeing an option to autodetect lvm PV/VG/LVs in kernel, and if this is the case, no initramfs is needed anymore.

Neddy and I have a special case where we have root on LVM over MD, this would be kind of tricky for the kernel to figure out (it'd need to first scan for MD, then scan for LV, and if not done in that order, there's no chance to properly assemble.

This had been working fine for me for many years - my initramfs dates back when I made it for my old 4x500G array. It ported straight over to the upgrade - a 3x2T array - until I wanted to muck with my LV configuration more and the old lvm binary needed to be upgraded. This potentially and later found out to definitely cause problems for the initramfs - simply replacing the lvm binary was insufficient. I had to also muck with libraries and the mounting script too!
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

erm67 · l33t Joined: 01 Nov 2005 Posts: 653 Location: EU

I rebuild the initramfs when the system fails to boot :-)

and it happens always at the wrong time .....
RAID/LVM kernel autodetect was deprecated a decade ago ...
https://raid.wiki.kernel.org/index.php/RAID_Boot
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net

NeddySeagoon · Posted: Tue Mar 30, 2021 10:29 am Post subject:

eccerr0r,

I've done root in raid5 only with no initrd.
I've done root in LVM only with no initrd, it's fragile, don't do it.

I've tried root in LVM on raid5 with no initrd. The kernel does it in the wrong order :(

For amusement only.
I don't see the kernel devs interested in a patch to fix that, as kernel raid autodetect has been depreciated for many years.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Zucca · Posted: Tue Mar 30, 2021 10:37 am Post subject:

pa4wdh · l33t Joined: 16 Dec 2005 Posts: 881

NeddySeagoon · Posted: Tue Mar 30, 2021 12:43 pm Post subject:

Zucca,

I do manage my raid and lvm separately. It's the same kernel code in both places.
Mostly because of old habits. My first use of raid was raid only, so I was vaguely familiar with mdadm.
When I wanted to add LVM, in another intall, keeping it separate seemed the natural way to go.

You are correct that the rebuild is user space only. When you boot LVM directly from the kernel (no initrd), the kernel cannot check that LVM is clean, so raid mode LVMs are not supported.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Zucca · Posted: Tue Mar 30, 2021 12:57 pm Post subject:

I've been creating a initramfs script with a slightly different approach - it only gathers files and their dependencies and crams them into (optionally) compressed cpio. It can be placed under /etc/kernel/postinst.d for automatic initramfs creation during kernel install.
User is responsible for everything else.
This way I intend to create more or less custom initramfs initrscript (linuxrc) for each of my setups.
My test initramfs is now around 2MiB instead of around 40MiB what dracut created. 95% less in size is quite a lot, but I'm not too thrilled about that since the size wasn't my concern, while it's still a nice bonus. It can speed up my setups which have /boot (and /boot/efi) on a slower medium.

I started this little project because I switched to gentoo-kernel, and dracut with default configs failed to create an initramfs which could boot my OpenRC based systems. And now I can even create somewhat universal (more "bloated") initramfs for emergency/rescue usage.

Now that I've learned (not mastered) how to easily craft my own, I don't see myself going "initramfsless".
_________________
..: Zucca :..

eccerr0r · Posted: Tue Mar 30, 2021 3:02 pm Post subject:

That is what I should do... find a way to point at binaries I need to copy to initramfs and have it automatically pick up the libraries needed. Been doing that manually for now (though finding lvm's deps got everything mdadm needed, and busybox was static.)

Still would not have prevented the failure I saw when the script no longer works with the new binaries... so I needed to do the rebuild anyway.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?