Excessive trim [SOLVED]

Tony0945 · Posted: Mon Jan 23, 2017 11:16 pm Post subject: Excessive trim [SOLVED]

I have a Crucial Mx200 250GB SSD. Following recommendations I have a cron job that runs trim twice a day:

NeddySeagoon · Posted: Mon Jan 23, 2017 11:33 pm Post subject:

Tony0945,

Have a look at smartctl -a ...

frostschutz · Advocate Joined: 22 Feb 2005 Posts: 2977 Location: Germany

Tony0945 · Posted: Tue Jan 24, 2017 12:49 am Post subject:

smartctl -a /dev/sda

Tony0945 · Posted: Tue Jan 24, 2017 1:10 am Post subject: Re: Excessive trim

frostschutz · Advocate Joined: 22 Feb 2005 Posts: 2977 Location: Germany

NeddySeagoon · Posted: Tue Jan 24, 2017 10:00 am Post subject:

Tony0945,

Think of trim being an instruction to the drive that it may perform an erase.
Erase is not write. Further, erase is an expensive operation time wise. The drive has to keep track of erased blocks, for low level block allocation, so it won't erase already erased blocks.

Regardless of fstrim, the drive will do what it wants. Think of fstrim as giving the drive permission to erase the trimmed blocks.
Its not forced to, it will make a note and get back to it.

discard works much the same way. You can't tell exactly when the erase will be performed. Its up to the drive.

Erase blocks are bigger that write blocks. There may be advantages (to the drive) in delaying erase cycles, so that write amplification is minimised.
e.g. If an entire erase block is to be trimmed, it contains no data to be kept.

trim is a complex process in SSD firmware and they don't all get it right.
There is at least one drive in the wild that trims still used data but that's what backups are for.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Tony0945 · Posted: Tue Jan 24, 2017 7:48 pm Post subject:

Gentlemen: Thank you for your input. I respect you both based on your work in these forums.

Before I mark this [SOLVED] (and I'm not so sure where I should have swap - SSD or HDD) please confirm that the wiki, https://wiki.gentoo.org/wiki/SSD, is all wet

Roman_Gruber · Posted: Tue Jan 24, 2017 8:21 pm Post subject:

A bit offtopic:

I never used trim on my SSDs. I have 3 of those. And i swap in them regularly as system or backpu drives. the file system is wiped and recreated during the backup stage.

and before that i used the SSD for ages without backup... without trim. Is it really that necessary for /?

frostschutz · Advocate Joined: 22 Feb 2005 Posts: 2977 Location: Germany

@Tony0945: fstrim is what pretty much all distros use these days, just not twice a day. :lol:

As for swap, I don't have any. If you're not tight on RAM you'd only need it for hibernate. Hibernate on SSD is faster than HDD but not THAT much (hibernation image is one blob of consecutive data, that's the one thing HDD aren't horrible at). Also depends on how often you do it... something you have to try out for yourself. If you don't need hibernation that's for the best, so many things can go wrong with it.

Tony0945 · Posted: Tue Jan 24, 2017 11:27 pm Post subject:

No hibernation, just a server that runs 24/7. I think those big fstrim's were from reboots. Apparently on shutdown, the kernel forgets what was trimmed before and trims all free space. I'm surprised. it shouldn't be that hard to save a binary file with the data on shutdown and restore it on reboot. Of course, the drive could be messed with by e.g. sysrescuecd between shutdown and reboot. I only reboot when I build a new kernel which is fairly often these days. I doubt if the swapfile was ever used. The system has 8G but the APU reserves 1G for graphics. Maybe a few ebuilds.

What about the advice to NOT use discard? Some sources say ext4 uses it automatically now. I'm running 4.9.x kernels.

frostschutz · Advocate Joined: 22 Feb 2005 Posts: 2977 Location: Germany

A.S. Pushkin

As I've been battling my 850 PRO SSD and though they are fast I'll be going back to
mechanical HDDs.

I've heard various remarks about TRIM, use it don't use it, but I fail to see that it does anything to restore
last capacity to my drives. I began with Gentoo installed on a 128GB 850 PRO and filled it up? I thought
that perhaps I just needed a larger disk for Gentoo so I upgraded to a 256GB 850 PRO. Well, the same thing
is happening.

/home is located on a Seagate Baracuda

I built this box for 3D modeling, video and audio editing and I can no longer create DVDs with DeVeDeng.

Now I'm not sure if this is just Samsung SSDs ot all, but I'm done with SSDs. I own three 850 PROs and a fast
boot is of little value to me if I can not do my work.
_________________
ASPushkin

"In a time of universal deceit - telling the truth is a revolutionary act." -- George Orwell

NeddySeagoon · Posted: Mon Apr 17, 2017 3:39 pm Post subject:

A.S. Pushkin,

Do you use eclean?
What about --depclean?
/var/tmp/portage fills with any failed ebuilds.
/var/log/portage can contain logs for everything you have ever built.
Do you rotate your logs in /var/logs ?

There are a few other locations that grow until you prune them too.
/boot - old kernels and initrds
/lib/modules - loadable modules for all the kernels you have ever made.
/usr/src - bits of kernels that are not --depclean ed.

Cleaning out your junk is as important as keeping your Gentoo up to date.
You can easily fill 250G in a year or so if you don't git rid of the rubbish.

discard/fstrim has nothing to do with cleaning out the filesystem. It informs the SSD that hosts the filesystem that it may erase once used space that is no longer in use.
This maintains SSD speed as there is no need to perform the slow erase operation before a block can be rewritten. The drive should do that in response to the discard/fstrim command. well before the blocks are reused.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

A.S. Pushkin · Posted: Fri Apr 21, 2017 2:54 am Post subject: eclean

I'm been trying to use emerge --depclean, but I'm a bit leery of removing too much.

I've got some logs, but total at this time less that 1GB.

Rotation is another area I'm uncertain, that is how to set up rotation, so I must assume I do not. Suggestions
as I do have syslogng installed.

/boot -
/lib/modules
/usr/src

All seem alright

When I run fstrim -v /

It returns no more than about 11GB.

Thanks
_________________
ASPushkin

"In a time of universal deceit - telling the truth is a revolutionary act." -- George Orwell

NeddySeagoon · Posted: Fri Apr 21, 2017 8:14 am Post subject:

A.S. Pushkin,

Run

Anon-E-moose · Posted: Fri Apr 21, 2017 9:55 am Post subject:

I run a mix of ssd and old hd tanks (lol) on my mostly server system.

On the ssd I have /, /usr/portage and a mostly static partition, I fstrim daily on root and /usr and weekly on /x.

On one of the hd's, it's my "storage" filesystem, media (sound and movies) along with a few odds and ends.
It gets written hard every once in a while, and read a lot, and I'll swap it over to ssd when the 1 gig come down in price.

On the other hd, I have /var, and /home, both of which get pretty frequent writes along with reads.

/var/tmp/portage and /tmp are ram disks

How you set up your ssd or a mix of ssd/hd should be dependent on how you use your system.

Edit to add:
I keep all distfiles and ebuilds backed up daily onto a backup drive that only gets mounted for backups,
and then do a eclean-dist every 2-3 months (that will keep the size of /distfiles down.

When in doubt do "eclean-dist -p" to see what would be removed.
_________________
UM780, 6.12 zen kernel, gcc 13, openrc, wayland

A.S. Pushkin · Posted: Mon May 01, 2017 1:02 pm Post subject:

Thank you NeddySeagoon and Anon-E-moose!

I clearly have got to become more adept at du!

Anon-E-moose how do you run fstrim on those particular partitions? Do you use cron or
do you do it manually? I've been concerned that running fstrim too much was bad idea
for the SSD longevity. I've heard it was particularly bad for Linux.

Thanks
_________________
ASPushkin

"In a time of universal deceit - telling the truth is a revolutionary act." -- George Orwell

NeddySeagoon · Posted: Mon May 01, 2017 1:13 pm Post subject:

A.S. Pushkin,

fstrim is harmless to the drive. It informs the drive of which blocks the filesystem is not using.
The drive will not erase already erased blocks.
Its also up to the drive if and when it takes any action at all based on this information.

While that's correct, its not complete.
The erase block size is bigger than the write block size. This leads to 'write amplification' as the drive needs to move used written blocks around to empty an erase block before it erases it.
How and when the drive decides to do this is up to the drive firmware.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

A.S. Pushkin · Posted: Wed May 03, 2017 7:44 pm Post subject:

NeddySeagoon thanks for the confirmation on what takes place.

I'm puzzled still. I did run fstrim on locations you suggest.

In this search I realized I should use sys-apps/smartmontools and installed it along with sys-apps/gsmartcontrol, which is very helpful.

I'm wondering if my NEW drive is worn out?

NeddySeagoon · Posted: Wed May 03, 2017 8:10 pm Post subject:

A.S. Pushkin,

fstrim does not free any space on your filesystem.

When you delete a file, the data remains in place on your hard drive. Only the pointers to it are removed.
The space occupied by the file is returned to the filesystem free space.

fstrim tells the drive about the free space un the filesystem. The drive can then erase it well before it will be reallocated.
Compared to reads and writes, erase is a very slow operation,
A magnetic hard drive does not require a separate erase operation an it can be overwritten by a write.
SSDs must be erased before they can be written.

The total tumber of blocks written to your drive is

Anon-E-moose · Posted: Thu May 04, 2017 9:52 am Post subject:

A.S. Pushkin · Posted: Sat May 06, 2017 3:13 pm Post subject:

NeddySeagoon you put me the wondering what was happening. I assumed it was just a SSD problem.

On my part there is some confusion about the automounter, but I finally discovered what happened.
Perhaps there is something not configured properly? But this is what happened with all my disk space.

I assumed my 1TB drive in the hotswap rack was being mounted at /media. It was not. It was being
mounted at /run/media/
When I created two directories they actually created in / rather than the 1TB drive. After actually moving
the files I thought were on the 1TB drive from /media space on my root drive to the 1TB drive USE% went from
100% to 10%.

I clearly have more to do, but will take into account the plans used by you and Anon-E-moose.

Thanks.
_________________
ASPushkin

"In a time of universal deceit - telling the truth is a revolutionary act." -- George Orwell

NeddySeagoon · Posted: Sat May 06, 2017 6:41 pm Post subject:

A.S. Pushkin,

You SSD will be none the worse for being filled up once.
You should run fstrim now that its 90% free so the drive will erase those blocks just moved to your 1TB drive.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.