slow fast.com when running from memory

[o5gmmob8]

I am running from my system from memory and it is snappy, apps start instantly. But, one thing I am trying to work through is why when measuring bandwidth on fast.com, I get numbers all over the board, it isn't stable.

Generally, when running from memory, it is measured much slower than what it I normally get when running from a disk - I've not run a test where it was as fast or faster. I have 16 GB of ram, 10 of which is allocated for the squashfs image copied in memory, about 3 GB then is used by /run as tmpfs. I don't believe I am seeing any errors and otherwise, CPU usage looks low.

I ran my system from dracut's dmsquash-live and it runs normally, so I must be doing something off, but I'm not sure what it is. The only thing that comes to mind is I have 2 overlays because I have my /home as a separate squashfs image from root so I can update it without having to wait for rebuilding root.

Oddly enough, if I download a large file such as the Windows 10 ISO, I pull about what I expect until I hit the file size limit on the /run partition.

[zen_desu]

My guess would be that it's the overlays + squashfs being slow. Maybe there is a chance that you're hitting memory bandwidth limits?

I have a similar setup and run my laptop in RAM most of the time using ugrd's overlayfs module which makes a single overlay over the rootfs. I did a fair bit of testing and it seems to be faster and have better battery life that way.

The design here is that i can boot it without the overlay to do updates, the overlay option is the default boot option so changes don't persist unless I tell it to boot differently. I think this is a bit simpler and more effective than managing multiple system images, and can be used with btrfs subvols too.
_________________
µgRD dev
Wiki writer

[o5gmmob8]

Ok, with 1 overlayfs, it seems normal, with 2, I only experience that behavior with fast.com. Everything else seems fine.

Hmm, I will experiment more. I'm not sure how I will use and maintain this in the long run. I would estimate I would reboot to my spinning disks, apply updates, generate a new image, then reboot back to the in memory. If I'm not making many changes, then 1 overlayfs would be simplest.

[zen_desu]

[o5gmmob8]

Thanks - I am going back to a single squashfs image.

I *think* I can append my home partition by running the command again after the root image is built. I think if I exclude directories then the directories don't appear in the image built unless I specify files later and to me, it becomes more cumbersome that way.

While we're on the topic of squashfs images, what compression are you using? I historically used lz4 because it was fast and efficient, but was wondering if you have experience with others.

[szatox]

I use xz because it was way better than the only alternative at the time (gzip?) and I got used to it. If you want to optimize it, whatever offers the highest compression ratio will probably be the right choice.

You _can_ append new files to a squashfs, but you can't reclaim space from deleted files.
Why do you even need your /home included in overlayfs though? Just mount it as a branch on top of your rootfs, like everyone else.
_________________
Make Computing Fun Again

[o5gmmob8]

I did a quick POC today just with lz4 and zstd. For my root filesystem and home configuration files, my image went from 7 GB with lz4 to 5 GB with zstd.

I believe I was using -12 with lz4 for the compression level, but for zstd, I did not pass any options.

I historically used lz4 as 5 or so odd years back lz4 worked better for me in practice for the same application. I think zstd is newer and thus may be better.