Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Low disk space: recommend changes to wiki and DIY threads
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
clytle374
Apprentice
Apprentice


Joined: 01 Aug 2006
Posts: 221

PostPosted: Mon Oct 31, 2022 3:03 pm    Post subject: Low disk space: recommend changes to wiki and DIY threads Reply with quote

I run an old computer with a small SSD drive, needed more space and found these pages.

Sticky on freeing up disk space
https://forums.gentoo.org/viewtopic-t-30547.html

Wiki on SSDs
https://wiki.gentoo.org/wiki/SSD

And a wiki on freeing up disk space
https://wiki.gentoo.org/wiki/Knowledge_Base:Freeing_disk_space

These should have using savedconfig for firmware included as firmware has gotten to nearly 1Gb in size now. I realized this when recently the firmware file was updated and I remembered the savedconfig.

I don't have edit privileges on the wiki or sticky FAQ subsection and thought it should probably be added.

Thanks
Cory
_________________
NooB since RedHat 6.1
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54605
Location: 56N 3W

PostPosted: Mon Oct 31, 2022 7:14 pm    Post subject: Reply with quote

clytle374.

The Wiki Talk pages are open to all. Even if you can't fix a page, you can create a talk page entry.
The wiki will then email all contributors to the page ... who can fix it.

You post here is likely to be missed.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9835
Location: almost Mile High in the USA

PostPosted: Mon Oct 31, 2022 7:28 pm    Post subject: Reply with quote

Surprised existing pages don't include more stuff that can be trimmed...

make.conf: FEATURES="nodoc noinfo" or you can delete a lot of the extra documentation stored in /usr/share/doc/* and /usr/share/info/* respectively. /usr/share/gtk-doc could also be trimmed.

If you're really manly you can try FEATURES="noman" to not install man pages. (hmm... this probably could be worded better...)

Then there's localization information - you can blow away the locales that you don't use in /usr/share/locale/ /usr/share/help ... come to think of it, a lot of /usr/share/ can be blown away if needed.

Making sure you go down to just one python/gcc/llvm/... version is also helpful.

Also do consider, do you really need to run portage/emerge on some of these embedded systems. That's yet another optimization, though you should think long and hard before omitting them, though it may make sense in some situations.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54605
Location: 56N 3W

PostPosted: Mon Oct 31, 2022 8:06 pm    Post subject: Reply with quote

eccerr0r,

It easy to put the repos and distfiles on external media.
As you hint, the requirements for maintaining Gentoo and running Gentoo are quite different and the two cases can be separated.

Until a few years ago I had a Gentoo install running from an early 8G SSD. An Acer One Netbook with the 32 bit Intel Atom
Needless to say, it didn't build there.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Goverp
Advocate
Advocate


Joined: 07 Mar 2007
Posts: 2183

PostPosted: Tue Nov 01, 2022 10:40 am    Post subject: Reply with quote

clytle374 wrote:
...
These should have using savedconfig for firmware included as firmware has gotten to nearly 1Gb in size now. I realized this when recently the firmware file was updated and I remembered the savedconfig.
...

<ediit>I forgot about building firmware into the kernel</edit>
A bit of digging on the web, and a bit of experimentation, found the following useful command: to run at least 48 hours after your last update to /lib/firmware, and with a reboot in the last 24 hours, and/or if you build your firmware into the kernel, a kernel build in the last 24 hours:
Code:
find /lib/firmware -atime 2

It lists all firmware files accessed since the reboot. If you're building a list for savedconfig, don't forget to scan /etc for drivers (such as USB scanners) that might download firmware on demand. (The use of -atime 2 instead of -atime 1 is down to find rounding rules - see man find for details.)

This assumes you have not totally disabled atime updates by specifying noatime for your rootfs in /etc/fstab; there's not a lot of advantage in that nowadays; the kernel assumes lazyatime,relatime, which have negligible performance impact, and ensure atime gets bumped once a day for accessed files. (I wonder if the lazyatime feature is responsible for the delay when the OpenRc shutdown remounts / as R/O? IIUC that's when the kernel has to update the atime values when running with lazyatime.)
_________________
Greybeard
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9835
Location: almost Mile High in the USA

PostPosted: Tue Nov 01, 2022 5:08 pm    Post subject: Reply with quote

atime updates don't cost too much in terms of slowing things down, but the main reason for not enabling them nowadays is due to flash memory wear... which might be a large percentage of total writes to a small embedded system. It's still useful to enable them temporarily to do the accounting, find out what you need, and then disable it later.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 20501

PostPosted: Thu Nov 03, 2022 3:00 am    Post subject: Reply with quote

clytle374 wrote:
I run an old computer with a small SSD drive, needed more space and found these pages.

Sticky on freeing up disk space
https://forums.gentoo.org/viewtopic-t-30547.html
clytle374 wrote:
These should have using savedconfig for firmware included as firmware has gotten to nearly 1Gb in size now. I realized this when recently the firmware file was updated and I remembered the savedconfig.

I don't have edit privileges on the wiki or sticky FAQ subsection and thought it should probably be added.
I've added a reference to the wiki page documentation for 'savedconfig'. Thanks.
_________________
Quis separabit? Quo animo?
Back to top
View user's profile Send private message
clytle374
Apprentice
Apprentice


Joined: 01 Aug 2006
Posts: 221

PostPosted: Sun Dec 11, 2022 3:08 am    Post subject: Reply with quote

Thanks everyone, I forgot about this post until I started running out of space again.

eccerr0r wrote:
Surprised existing pages don't include more stuff that can be trimmed...

make.conf: FEATURES="nodoc noinfo" or you can delete a lot of the extra documentation stored in /usr/share/doc/* and /usr/share/info/* respectively. /usr/share/gtk-doc could also be trimmed.

If you're really manly you can try FEATURES="noman" to not install man pages. (hmm... this probably could be worded better...)

Then there's localization information - you can blow away the locales that you don't use in /usr/share/locale/ /usr/share/help ... come to think of it, a lot of /usr/share/ can be blown away if needed.

Making sure you go down to just one python/gcc/llvm/... version is also helpful.

Also do consider, do you really need to run portage/emerge on some of these embedded systems. That's yet another optimization, though you should think long and hard before omitting them, though it may make sense in some situations.


The FEATURES= nodoc noinfo noman don't trigger rebuilds, will it take effect as things are rebuilt due to other changes?

This system has a small SSD drive that the system runs on. distfiles are on another drive and /tmp is a ram drive.

Thanks
Cory
_________________
NooB since RedHat 6.1
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9835
Location: almost Mile High in the USA

PostPosted: Sun Dec 11, 2022 7:15 am    Post subject: Reply with quote

correct, they are applied on package installation, so it doesn't affect the current system until the packages get re-merged.
You could also just manually delete /usr/share/doc/ if you feel lucky.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54605
Location: 56N 3W

PostPosted: Sun Dec 11, 2022 12:06 pm    Post subject: Reply with quote

clytle374,

Put /usr/src on the external drive too. Its only needed to build kernels and external modules.
Likewise, the repos are only needed a build and install time, so can go with distfiles.
If you want to do binary package installs, the piece(s) of the repo to support that are required.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9835
Location: almost Mile High in the USA

PostPosted: Sun Dec 11, 2022 6:58 pm    Post subject: Reply with quote

curious how big peoples' DE/GUI Gentoo installs are (with portage tree, but minus user data, distfiles, etc.)
Seems like around

i686 with XFCE/Firefox: 10GB
amd64 multilib with XFCE/Firefox: 16GB

These also aren't fully cleaned of superfluous linux-firmware files, so perhaps these can go down a little more.

I don't remember if these numbers are with kernel sources installed, which is about 1GB these days. Note these numbers do not leave any room to build additional packages so a 20GB HDD is not big enough for an amd64 multilib self-hosting (i.e. can emerge for itself for updates), though for i686 it may work with some file space juggling...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
szatox
Advocate
Advocate


Joined: 27 Aug 2013
Posts: 3468

PostPosted: Sun Dec 11, 2022 7:32 pm    Post subject: Reply with quote

My amd64 nomultilib with lxde (and a lot of other things installed) is just under 20GB.
This does not include RAM/SWAP and temporary files required for a big build (like a web browser) to succeed.

Although I do have some headless servers running of 20GB drives, I wouldn't expect them to be able to compile GUI applications. System takes around 9Gb there, but being headless it has no need for GUI stuff.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9835
Location: almost Mile High in the USA

PostPosted: Sun Dec 11, 2022 8:56 pm    Post subject: Reply with quote

Just checked one of my amd64-multilib XFCE+Firefox installs with minimal extras, looks like 21GB used that includes 2 kernels in /usr/src and a 4GB swapfile. So yeah 16GB is possible, but need at least 32GB to be self hosting, if not more. I did not prune doc/info/manpages or /lib/firmware.

Been thinking about how to set up a very small setup with all the distcc compilers set up along with PXE boot... Would be nice to share the same NFS but things like /var and /run are probably going to be issues, but would be nice to share binaries, etc... hmm.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
szatox
Advocate
Advocate


Joined: 27 Aug 2013
Posts: 3468

PostPosted: Sun Dec 11, 2022 11:41 pm    Post subject: Reply with quote

Quote:
Been thinking about how to set up a very small setup with all the distcc compilers set up along with PXE boot

By far the best thing I came up with was a shared squashfs image copied to RAM at boot.

I had some scripts that would derive hostname from IP address and announce machine's presence using avahi. Distcc can use avahi.
Back in the day I had to patch distcc to let it use hosts discovered on ipv6 link-local, it's been a few years ago so might have been fixed upstream. Prioritizing ipv4 over ipv6 (or disabling it completely) might do the trick as well.
A single image makes sure you will have the same version of toolchain running everywhere, which takes care of gcc clashes, and copying it to RAM at boot saves a lot of bandwidth later, this makes a noticeable difference even on fast networks.
Also, this way satellites won't crash just because you're rebooting NFS, though this also means you might want to add a hook to mass-reboot all the satellites after creating a new image.
Back to top
View user's profile Send private message
Goverp
Advocate
Advocate


Joined: 07 Mar 2007
Posts: 2183

PostPosted: Mon Dec 12, 2022 9:02 am    Post subject: Reply with quote

NeddySeagoon wrote:
clytle374,

Put /usr/src on the external drive too. Its only needed to build kernels and external modules.
Likewise, the repos are only needed a build and install time, so can go with distfiles.
If you want to do binary package installs, the piece(s) of the repo to support that are required.

Alternatively, if you want them, you can use squashfs images of both /usr/src and /var/db/repos/gentoo. To use the former, you need to build kernels outside /usr/src/linux, but the kernel guys have always recommended building in user space, not from root. I'll dig out my scripts for handling /usr/src (I'm on the wrong machine just now).
_________________
Greybeard
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum