More atomic upgrades with portage

[mid-kid]

Hello, I rarely write on this forum, but I felt this was the right way to discuss this, so here I am.

As of the recent years, I've been upgrading my machine on about a monthly basis. This allows me to be sure that no unexpected changes or breakage happens during the month, and allows me to coordinate my upgrades with free time to spend fixing any possible breakage/new bugs in new software. Unfortunately, this also means each upgrade is *long*, usually ranging from 12 to 24 hours, depending on what packages decide to upgrade. There's a lot of time where I'll want to use my machine during the upgrade.

Portage only ensures system consistency before and after an upgrade, but during an upgrade, it's anyone's game. There's many packages that don't deal with this well, one very annoying example being Qt. If you update dev-qt/qtcore, all Qt applications will stop working until all of the Qt components they use have been upgraded as well. This happens with things like PYTHON_TARGETS as well, where the gentoo news item will generally recommend rebuilding twice to prevent breakage, by first enabling the new python target, and afterwards disabling the old one. There's other packages with this issue, but less frequently so.

Looking at today's upgrade, I'm seeing dev-qt/qtcore will be upgraded as the 59th package, yet dev-qt/qtgui is package 323, somewhere near the end of the upgrade. For a huge portion of the coming 12 to 24 hours, I won't be able to use any Qt application, making me unable to use my password manager, among other things.

Granted, Gentoo isn't the only distribution with this problem. Every single binary distribution deals with this as well, and some have even decided to deal with this by forcing the user to reboot their machine to apply any non-flatpak update (kde neon, gnome software center). The big difference between those and Gentoo is that the time between the start of an upgrade and the point where the system is fully consistent again, is much, much shorter.

This is why I was thinking, if there'd possibly a way to improve this situation in Gentoo. If not make upgrades fully atomic, then at least increase atomicity by shortening the timespan during which the system is broken.

Gentoo has binpkgs, and I feel that leveraging this feature, or otherwise delaying the package install stage to when everything is finished building, would be key to improving this. emerge --buildpkgonly complains about not all dependencies being satisfied before build, and portage has no way of retaining the "image" directories until the transaction is done, so those are out of the question. Two solutions I can think of, are either somehow leveraging --quickpkg-direct to install every currently installed package into a chroot, then building binpkgs in there, and afterwards installing them on the host, OR, setting up an overlayfs chroot, and building inside of that such that installed files end up in the overlay instead of the real system, and afterwards installing the binpkgs on the host.

There's drawbacks to both of them, --quickpkg-direct won't copy over all configuration files, and an overlayfs chroot is hard to get right in terms of mounts, especially if the host system is split up over multiple drives or partitions.
I'm wondering, has anyone tried anything like this before? What are your thoughts? Maybe something like this could be implemented in portage itself?

[Banana]

I do updates more often and thus the amount of changes is smaller.
Maybe using https://wiki.gentoo.org/wiki/Binary_package_guide provided by another machine could solve your case? This way you make sure the updates a working on a not-so-important machine.
_________________
Forum Guidelines

PFL - Portage file list - find which package a file or command belongs to.
My delta-labs.org snippets do expire

[sublogic]

I wonder if a solution can be cobbled together using LVM snapshots ? (LVM= Logical Volume Manager).

1. create a snapshot of root (and /var, if a separate filesystem)
   
2. convince portage to update the snapshot(s)
   
3. merge the snapshot(s)

I use the LVM but I haven't played with snapshots. No experience in the matter. Especially step 3.

Maybe safer,

1. create snapshot(s)
   
2. start emerge
   
3. meanwhile, set up a chroot to the snapshot(s) and work from there
   
4. delete the snapshot(s)

[Goverp]

I build all the binaries in a chroot, and then install them into the main system. That's not to make the main system updates more atomic, it's because I also load up a laptop from the same binaries. a few packages (currently one on each system) in fact get compiled natively on the main and laptop boxes because of differing CPU_FLAGS_X86 optiizations.

I set up the chroot as you describe, using quickpkg; the config files aren't a problem - the ones put in the package are the ones you want anyway. More to the point, you should only be installing binary packages that have been updated, and so they should contain the latest configs. In fact, I don't think you need to populate the chroot with old binaries.

My setup uses bind mounts to share bits of the portage configuration between the chroot and the main system. Basically /etc/portage, /var/db/repos and /var/lib/portage/world. You may as well include /var/cache/distfiles and /var/cache/binpackages too. That way any alterations you need to get the compilation to work in the chroot propagate to the main system.
_________________
Greybeard

[Dr.Willy]

Have you considered distributions that do atomic upgrades out-of-the-box, like Guix or Nix?
_________________
gentoo repos: kakoune | oil | hyper-v

[mid-kid]

Thanks for the replies, this is all very interesting. Unfortunately, I currently don't have the time to try different approaches, so it'll remain mostly a thought experiment for now.

I think the LVM solution is very interesting, but I'm not sure it won't have any drawbacks, aside from the obvious part where I'll have to reformat my drive with LVM.
I'll have to play with it sometime.

[Goverp]