Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Is Gentoo bad for a microSD
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
thegrind
n00b
n00b


Joined: 22 Sep 2024
Posts: 26

PostPosted: Sun Oct 20, 2024 4:56 am    Post subject: Is Gentoo bad for a microSD Reply with quote

I have a microSD on a Rpi5. I am looking to compile a kernel and headless chromium. Will all this emerging and compiling really chew up my card? I bought a better performing but I'd like to move it over after I'm done with some heavy compiling/emerging. Thoughts?
Back to top
View user's profile Send private message
Zucca
Moderator
Moderator


Joined: 14 Jun 2007
Posts: 3777
Location: Rasi, Finland

PostPosted: Sun Oct 20, 2024 6:08 am    Post subject: Reply with quote

I'd maybe use an USB stick to store Portage temporary files (PORTAGE_TMPDIR).
_________________
..: Zucca :..

My gentoo installs:
init=/sbin/openrc-init
-systemd -logind -elogind seatd

Quote:
I am NaN! I am a man!
Back to top
View user's profile Send private message
Banana
Moderator
Moderator


Joined: 21 May 2004
Posts: 1801
Location: Germany

PostPosted: Sun Oct 20, 2024 7:42 am    Post subject: Reply with quote

They have improved over the years, but I would suggest either: Build on different PC, use binhost or use an extranl drive on your Pi.
_________________
Forum Guidelines

PFL - Portage file list - find which package a file or command belongs to.
My delta-labs.org snippets do expire
Back to top
View user's profile Send private message
pa4wdh
l33t
l33t


Joined: 16 Dec 2005
Posts: 892

PostPosted: Sun Oct 20, 2024 9:20 am    Post subject: Reply with quote

I have been running a Raspberry Pi 2 based NTP server since 2016 running Gentoo from a microSD card. It's biggest compile job is gcc (running the job from disk and using lots of swap because it only has 1GB of memory). So far i've replaced the SD card twice, but never because it failed. I usually get low on space for the rootfs and wait for nice discounts on cards.
Now it has a 128GB Samsung card i bought for about 20 euros using f2fs as a filesystem. As far as i know sdcards don't have smart-like functions so i can't tell how much has been read/written on the card.
_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world

My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com
Back to top
View user's profile Send private message
Naib
Watchman
Watchman


Joined: 21 May 2004
Posts: 6069
Location: Removed by Neddy

PostPosted: Sun Oct 20, 2024 10:56 am    Post subject: Reply with quote

Things have improved but the old trope still exists (but is still something to be mindful of)... >100,000writes means you are likely to face physical failure before bit failure.
Gentoo and linux does increase the number of writes but there are some things you can do to prolong it

1. disable atime
2. disable logging - or selectively
3. use a non-journaled FS (or use FSFS)
4. use gentoo binhost
_________________
#define HelloWorld int
#define Int main()
#define Return printf
#define Print return
#include <stdio>
HelloWorld Int {
Return("Hello, world!\n");
Print 0;
Back to top
View user's profile Send private message
flysideways
Guru
Guru


Joined: 29 Jan 2005
Posts: 499

PostPosted: Sun Oct 20, 2024 11:56 am    Post subject: Reply with quote

With the Pi 5 comes the option of using an nvme adapter.

This is from an sd card in a Pi 5

Code:
mirror ~ # hdparm -tT /dev/mmcblk0

/dev/mmcblk0:
 Timing cached reads:   3392 MB in  2.00 seconds = 1696.44 MB/sec
 Timing buffered disk reads: 272 MB in  3.01 seconds =  90.43 MB/sec


and this is from my Gentoo Desktop Pi 5 using an nvme ssd in a Pimoroni NVMe Base.
Code:
funf ~ # hdparm -tT /dev/nvme0n1

/dev/nvme0n1:
 Timing cached reads:   4472 MB in  2.00 seconds = 2236.43 MB/sec
 Timing buffered disk reads: 1708 MB in  3.00 seconds = 569.12 MB/sec


I had been using external usb ssd drives before the Pi 5 for the faster read/write, they are faster than usb flash drives but more cumbersome.

The NVMe Base makes for an aesthetically appealing package along with the stock cooling fan.

It continues to amaze me how the Pi 5 outperforms probably the first ten years of Gentoo boxes that I have had.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54643
Location: 56N 3W

PostPosted: Sun Oct 20, 2024 1:02 pm    Post subject: Reply with quote

Personally, if you must use an SD card, choose a good quality card, not a noname one and over provision it.
Many SD cards do wear levelling now, so F2FS will fight the inbuilt wear levelling, if you have it.

Does fstrim -av trim your card, or is trim not supported for you?
That's a good thing to do once a month to keep free space erased to keep write speeds up.
if trim is supported, you probabably have wear levelling too but there is no way to be sure.

I use a 512Gb Sandisk card with a desktop install. It want's to build Chromium every week.
Its a ~arm64 install except Chromium, which is stable.

The only reason I don't use NVMe is that Pi5 has to be silent, so passively cooled.
Maybe I could do some metalwork to use a bottom NVMe but that's for the future.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
pingtoo
Veteran
Veteran


Joined: 10 Sep 2021
Posts: 1339
Location: Richmond Hill, Canada

PostPosted: Sun Oct 20, 2024 2:33 pm    Post subject: Reply with quote

I use RPI5 for my daily drive. It is gentoo based on SD card. However I seldom compile on it. I use other nodes on my network that provide NBD then I pool those NBD with LVM and mount the lv as /var/tmp. Just give you some idea for possible workaround the SD card write concern.
Back to top
View user's profile Send private message
Goverp
Advocate
Advocate


Joined: 07 Mar 2007
Posts: 2187

PostPosted: Sun Oct 20, 2024 7:13 pm    Post subject: Reply with quote

NeddySeagoon wrote:
...
Many SD cards do wear levelling now, so F2FS will fight the inbuilt wear levelling, if you have it.
...

I must nail this particular canard, as it keeps appearing. F2FS does nothing about wear levelling - it leaves it to the underlying hardware.
AFAIK it's main tricks are to do with trimming and improving file deletion processing; IIUC it puts files into separate buckets by type (I think just the ".jpg"/".tgz" type, not MIME), trying to manage file component sizes to match the hardware's allocation units.

For more info, though way out of date, the Linux Weekly News tear-down is informative.

While I'm at it, another canard I see on Reddit "it doesn't journal like Btrfs/ext4". A bit of a strange claim, given it's a journal filesystem. Note, not journaling, journal - the filesystem IS a journal, nothing else.
Also people continue to recycle claims of unreliability from 10 years ago, when it was very new. I've used it on my desktop NVMe, laptop SSD and several Raspberry Pi SD cards, and not had a problem for probably 5 years - the last time I had an issue was when an SD card went irreparably read-only, which I think is actually a hardware failure state probably brought on by writing the systemd journal to the card at the wrong level, and generating far too much IO.

Rant over.
_________________
Greybeard
Back to top
View user's profile Send private message
pingtoo
Veteran
Veteran


Joined: 10 Sep 2021
Posts: 1339
Location: Richmond Hill, Canada

PostPosted: Sun Oct 20, 2024 7:23 pm    Post subject: Reply with quote

Goverp,

Thank you for sharing your f2fs experience. I think I will try it for my next stage4 build for rpi5. However I have one concern. I remember from somewhere it suggest f2fs are tie to kernel. i.e change kernel version may cause the file system on disk not usable. Is this true? or it was but changed so now it no longer is the case?
Back to top
View user's profile Send private message
Ralphred
l33t
l33t


Joined: 31 Dec 2013
Posts: 668

PostPosted: Sun Oct 20, 2024 8:34 pm    Post subject: Reply with quote

NeddySeagoon wrote:
Maybe I could do some metalwork to use a bottom NVMe but that's for the future.
I've messed with these hats extensively, my "passive cooling" preference is the double bottom hat, the long pcie cable kit (folded to get access to m.2 slots) and the hat fitted upside down (at an angle) on top of ~30/40mm of modified* nylon standoffs - it looks like a tuk tuk, but stays pretty cool on both boards.

*modified with a two pairs of pliers, a tiny blowtorch, and a 99p protractor from the corner shop
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9847
Location: almost Mile High in the USA

PostPosted: Mon Oct 21, 2024 5:43 am    Post subject: Reply with quote

most SD cards are probably using TLC or QLC flash memory and probably will only get 800-1000 erase cycles before becoming unreliable. Wear leveling will increase apparent erase cycle life. It appears there are "industrial" microsd cards that have SLC in them... these probably can do the 100K cycles.

I'm not sure if SD cards have become "fully associative" like most nvme/sata versus "set associative" wear leveling - at least this was a thing in the past. I also highly doubt it has inactive pool recycling like properly designed nvme/sata SSDs as it would really kill interactive performance that's slow enough as it is...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
lars_the_bear
Guru
Guru


Joined: 05 Jun 2024
Posts: 537

PostPosted: Mon Oct 21, 2024 7:15 am    Post subject: Reply with quote

I don't care how many writes my Raspberry Pi SD cards can tolerate. SD cards large enough for a Pi root filesystem are cheap, and I regard them as disposable. When I'm actively working on a Pi project, every so often I pull the SD card and copy it. This takes a few minutes, but I don't have to sit and watch it. Then, if an SD fails, I just copy the back-up image to an new SD and I'm good to go. In practice, I've never had an SD card fail, if it looked like it was in good order when I bought it. The ones that have failed are the ones I bought in bulk at too-good-to-be-true prices, and seemed dodgy right from the start.

BR, Lars.
Back to top
View user's profile Send private message
Goverp
Advocate
Advocate


Joined: 07 Mar 2007
Posts: 2187

PostPosted: Mon Oct 21, 2024 9:21 am    Post subject: Reply with quote

pingtoo wrote:
... I remember from somewhere it suggest f2fs are tie to kernel. i.e change kernel version may cause the file system on disk not usable. Is this true? or it was but changed so now it no longer is the case?

There was a particular one-off occurrence that burnt relatively early adopters, but I've not seen a problem since that one.

Two gotchas that I believe still active: 1) if you go to the Arch wiki article on F2FS, it recommends using options at mkfs time to add checksums to metadata. Those options increase the size of the metadata. That in itself is not a problem, but at least up to a year ago, the GRUB F2FS driver can't handle this version of the metadata. So either run without metadata, or use a separate /ext4 or whatever /boot, or don't use GRUB (say use the kernel EFI stub).
I opened a Gentoo bug report, but, not unreasonably, the response was to ask me to raise it with upstream. As far as I can tell, the only way to get GRUB developers to respond to a bug report is to include a patch :-) and I've not gotten around to it.

2) There was a problem with the boot time fstab-driven fsck, in that if the partition needed the fsck, it hung. I got round it by putting an "fsck -p" routine into my initramfs,. It resulted from fsck.f2fs handling whichever options were passed incorrectly/defaulted at boot time, leaving me with having to run "fsck -f" to fix the problem rather than just report it.
Conversely, I've just read a report where someone's fsck barfed because it couldn't find a checkpoint (in 2023) no idea how that ended.
_________________
Greybeard
Back to top
View user's profile Send private message
pingtoo
Veteran
Veteran


Joined: 10 Sep 2021
Posts: 1339
Location: Richmond Hill, Canada

PostPosted: Mon Oct 21, 2024 1:41 pm    Post subject: Reply with quote

Goverp wrote:
pingtoo wrote:
... I remember from somewhere it suggest f2fs are tie to kernel. i.e change kernel version may cause the file system on disk not usable. Is this true? or it was but changed so now it no longer is the case?

There was a particular one-off occurrence that burnt relatively early adopters, but I've not seen a problem since that one.

Two gotchas that I believe still active: 1) if you go to the Arch wiki article on F2FS, it recommends using options at mkfs time to add checksums to metadata. Those options increase the size of the metadata. That in itself is not a problem, but at least up to a year ago, the GRUB F2FS driver can't handle this version of the metadata. So either run without metadata, or use a separate /ext4 or whatever /boot, or don't use GRUB (say use the kernel EFI stub).
I opened a Gentoo bug report, but, not unreasonably, the response was to ask me to raise it with upstream. As far as I can tell, the only way to get GRUB developers to respond to a bug report is to include a patch :-) and I've not gotten around to it.

2) There was a problem with the boot time fstab-driven fsck, in that if the partition needed the fsck, it hung. I got round it by putting an "fsck -p" routine into my initramfs,. It resulted from fsck.f2fs handling whichever options were passed incorrectly/defaulted at boot time, leaving me with having to run "fsck -f" to fix the problem rather than just report it.
Conversely, I've just read a report where someone's fsck barfed because it couldn't find a checkpoint (in 2023) no idea how that ended.

Thank you very much for the information.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9847
Location: almost Mile High in the USA

PostPosted: Mon Oct 21, 2024 2:55 pm    Post subject: Reply with quote

lars_the_bear wrote:
I don't care how many writes my Raspberry Pi SD cards can tolerate. SD cards large enough for a Pi root filesystem are cheap, and I regard them as disposable.

For me the main problem is downtime and the frustration of such in an always-on situation. Technically hard drives are disposable too, but there's a reason why I run RAID on my server...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
gentoo_ram
Guru
Guru


Joined: 25 Oct 2007
Posts: 502
Location: San Diego, California USA

PostPosted: Wed Oct 23, 2024 8:09 pm    Post subject: Reply with quote

You have options.

1. I run an RPi5 with the NVME HAT+ option. It's running a real NVME drive direct over a PCI-E link. Increases performance and deals with the MMC wear issue.
2. I boot an RPI4 with a USB3-to-NVME adapter with an NVME drive inside. From what I can tell that's higher throughput than the SDCard and deals with the wear issue.
3. I put my /var/tmp/portage on a ZRAM device so it's all compiled in compressed RAM. Installs would happen over SDCard and/or NVME as appropriate. Would also reduce SDCard write wear.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9847
Location: almost Mile High in the USA

PostPosted: Wed Oct 23, 2024 10:53 pm    Post subject: Reply with quote

I have a RPi2 ...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum