systemd considered harmful -- return to Gentoo

sitquietly

I had decided to ignore my own best judgment and run systemd linux on a couple of systems where linux and freebsd both import the same 8TB data pool. The pool mounts datasets for users directly into their home directories. This has proved very handy and has much better behavior than the alternative of symlinks into the pool (especially since I make heavy use of stow for putting dotfiles into place).

Didn't work and I should have seen it coming. Feel stupid. Well, actually systemd is stupid -- mounting filesystems onto mount points is not deterministic! There is always a race condition in systemd which is only avoided by "workarounds". This became hotly evident to me when my zfs mounts were sometimes there and sometimes NOT.

The systemd folk will kindly tell you that you are responsible for avoiding race failures by using workarounds such as zfs mount-generator or (maybe) zfs legacy mounts instead of standard, easy zfs mounts.

Or just don't use zfs :roll:

-- it doesn't play well with systemd-linux.

My foray into systemd-linux was quickly terminated and I returned to OpenRC Gentoo. I love deterministic startup.

Deterministic System: a deterministic system is a system in which no randomness is involved in the development of future states of the system. A deterministic model will thus always produce the same output from a given starting condition or initial state. e.g. systemd is non-deterministic and is considered harmful.

Zucca · Posted: Mon Aug 09, 2021 9:16 pm Post subject:

For some special mounts to work properly with systemd I needed to create (/modify) the .mount unit files. It's a pita.

I still use systemd on one laptop. I keep it as a testing ground to watch the progress (? degress/congress :P) of systemd.
Still many kinks so be solved.

(Moved topic to Gentoo Chat.)
_________________
..: Zucca :..

Hu · Administrator Joined: 06 Mar 2007 Posts: 22624

I understand the OP was primarily intended as a rant, but I am curious as to what went wrong. What exactly was done that systemd behaved non-deterministically? In what way were the mounts non-deterministic? If you are not stacking one mount below another, the precise order in which filesystems mount should not matter. If you are stacking one below another, then you need to enforce an ordering. Although openrc does not, as far as I know, use mount --fork, the functionality for non-deterministic mounts is provided in mount itself, so non-systemd users can still potentially get in trouble with dependent mounts.

Where does systemd acknowledge the problem that led to OP and blame the user for holding it wrong?

sitquietly · Posted: Tue Aug 10, 2021 4:56 am Post subject:

ulenrich · Veteran Joined: 10 Oct 2010 Posts: 1483

sitquietly says: My alternative system hides all race conditions behind a big wall of artificial tardiness.

Naib · Posted: Tue Aug 10, 2021 9:32 am Post subject:

Event driven startup is always better then time-triggered startup with regards to efficient use of time and validating a precursor exists.

This however is dependent on the event-generator generating its event at at point in time which corresponds to when it itself is in a valid state and the event supervisor acknowledging what is an event as the event to indicate completion/readyness. If the event is generated early in its logic OR what is being monitored as an event is synonymous with other startup activities, well you will have a false indication of availability and all the corresponding race conditions that comes with it.

If accessing a socket is required for a process to complete but accessing a socket is used by some supervisor to indicate completion then the flaw is with the supervisor and the method used to determine availability. If the process accesses a socket prematurely but does not require aspects of that connection to complete then the flaw is with the process for signaling too early

Someone made a mistake, either architecturally or in implementation

[Moderator edit: rephrased last line based on a user complaint. -Hu]
_________________

NeddySeagoon · Posted: Tue Aug 10, 2021 10:30 am Post subject:

Naib.

... or both.

sitquietly,

As soon as you trigger things with events, like interrupts, your system is no longer deterministic. However it can still be robust without being deterministic.
I'm sure Naib will have something to say about that assertion :)
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Naib · Posted: Tue Aug 10, 2021 10:54 am Post subject:

Just a bit

But I think this is an ontology concern here. How we use the term "deterministic" might not be as concise as the OP or others. 100% an event driven architecture cannot be considered deterministic if that which triggers the event is itself non-deterministic

BuT I KnOw HpTtpD wIlL sTaRt AfTeR dHcP lEaSiNg

That's not deterministic

that's just a clearly defined sequence based upon events and as long as the producer and consumer act on the correct stimulus it can be robust
_________________

sitquietly · Posted: Tue Aug 10, 2021 1:39 pm Post subject:

mrbassie

guitou · Guru Joined: 02 Oct 2003 Posts: 534 Location: France

Hi.

Don't know especially about zfs, but if using fstab, you may enforce systemd mount ordering using specific mount options: x-systemd-{before,after}.
Not as straightforward as just relying on mount point definitions order, but it does the job.

++
Gi)

Fitzcarraldo · Posted: Tue Aug 10, 2021 11:16 pm Post subject:

Have not tried it myself, but Mount filesystem in certain order using /etc/fstab states that the mount order for systemd can be specified in /etc/fstab as per the following example:

Hu · Administrator Joined: 06 Mar 2007 Posts: 22624

The fstab shown by Fitzcarraldo seems not to need ordering. Neither mount seemingly depends on the other. OP: I was hoping for an example fstab that systemd mishandles, so I could understand whether there is enough information present that it should have been able to do the right thing.

sitquietly · Posted: Wed Aug 11, 2021 12:15 pm Post subject:

Hu · Administrator Joined: 06 Mar 2007 Posts: 22624

If there are no fstab entries for zfs, how does zfs get mounted at all? How can systemd screw this up if zfs is all automagically mounted outside the normal fstab flow? Is there some special openrc init script / systemd unit file required for zfs, and only the openrc init script is actually correct?

mrbassie · Posted: Thu Aug 12, 2021 5:52 pm Post subject:

Morality124 · Tux's lil' helper Joined: 20 Feb 2018 Posts: 102

Hu · Administrator Joined: 06 Mar 2007 Posts: 22624

I was hoping for an answer that would make sense to those of us who are not proficient with ZFS pools, so that we could explore whether the problem was an inherent limitation in systemd, a defect in how ZFS integrates with systemd, or a bug in systemd. What kind of services are these? /etc/init.d/zfs-mount-my-local-filesystems?

Parallelism is normally a good thing, and if openrc only works because it is not parallel, that is bad and should be fixed. Proper support for parallelism should make ZFS pools work properly under both systemd and openrc. On the other hand, if openrc would work properly even when the non-default parallel startup is enabled, it would be interesting to explore why parallelism works under openrc and not under systemd.

alamahant · Advocate Joined: 23 Mar 2019 Posts: 3916