How to tell for sure that a USB stick is broken?

[nxe9]

Hi, I have a 32 GB USB stick and after copying some files to the stick sometimes some files are corrupted. I compared the crc32 checksums of the files on the host with the files on the stick and the data were not consistent. The usb stick contains only one ext4 partition. In some cases the files on the stick are good but after copying another file some of the previous files become corrupt. I can reproduce the problem very often. At the latest after I move a few larger files.

My attempts so far

- Tests on two difference gentoo hosts. On both machines the usb stick corrupts files.
- Check the host with memtest. OK.
- Another stick. OK, all checksums all fine.
- Test with badblock multiple times on two hosts, „badblocks -w -s -o error.log /dev/sdX“. OK, no error.
- The problem is not because I unmounted the device incorrectly or removed the pendrive too quickly.

It seems that my usb stick is corrupt but how to make a sufficiently informative test? The badblock test doesn’t produce any error….

[dmpogo]

Check the logs /var/log/syslog or whatever logging system you are using. If there are write errors and logging is set to capture error level message, something maybe there

And

Reformat it, try again copying,
reformat with vfat, copy again

But it is probably dying

[jpsollie]

it may also be an issue with USB link power management ... though I do not see why that would make badblocks succeed
_________________
The power of Gentoo optimization (not overclocked): [img]https://www.passmark.com/baselines/V10/images/503714802842.png[/img]

[Banana]

There is https://www.cyberciti.biz/faq/linux-check-the-physical-health-of-a-usb-stick-flash-drive/ which lists some possibilities. No guarantee how good they are, but some sound reasonable
_________________
Forum Guidelines

PFL - Portage file list - find which package a file or command belongs to.
My delta-labs.org snippets do expire

[nxe9]

Thank you for your suggestions!

@dmpogo
I reformatted it many times, however, it didn't help.

/var/log/syslog, sdb is my usb stick:

[dmpogo]

I am unsure about badblocks program, but overall filesystem tries to use the disk if it is responsive. In the age of rotating disks the failure sometimes was taking time, bad blocks will develop, filesystem will remap them so that they are not used, and disk could still be used for quite a while.

[NeddySeagoon]

Turn off UAS if its in use.

The kernel knows about a lot of devices with broken UAS suppor and will refuse to use it.
Equally, thre are lots of devices the kernel does not know about with broken UAS support.

Test on USB 2 if you can. That does not support UAS by design.
Should that not be possible, you need to de it the hard way with a kernel parameter.

Testing needs to start with partitioning, since nothing can be trusted on the drive.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[nxe9]

It is a bit strange for me. I ran

[logrusx]

I have an Adata USB 3 stick which behaved very bad on my old desktop computer as well as when plugged into my smart TV. However with my newer laptop I haven't noticed any problems for 3 years now. I don't use it that often, but it hasn't failed a single time.

On my old computer and I'm not sure but I think my old ThinkPad laptop too, I was forced to use an older USB 2 flash drive to get reliable data transfer.

What are the hosts you tried it on?

Also there are those fake drives which report bigger size than the flash chip really is. I can easily imagine how the controller ignores a more significant bit and overrides addresses that have already been written.

Best Regards,
Georgi

[NeddySeagoon]

ehci-pci is the PCI end of USB2.
One reset is harmless, when a device is added.
A stream is a problem.

badblocks has been useless for many years.
Even USB flash drives do wear levelling and sector sparing.
Badblocks could trigger either or both mechanisms.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[nxe9]

[logrusx]

[pingtoo]

[NeddySeagoon]

when you say the data on the drive is corrup,
what size files have you been testing with?
how do you know that data is being read from the drive?

To be sure that the data is written, you need to issue a sync command.
it will not return while you have 'dirty' write caches.

Now you can drop caches to force the kernel to read the device, and not compare with the in cache copy of the file.

I think you have a USB 2 pendrive so UAS cannot be an issue.

Big files can take several minuets to sync over USB 2, even if the flash write speed is not the limiting factor
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[nxe9]

@logrusx:
This pendrive is not important to me. It is a $5 device. This is just a technical issue that interests me.

@pingtoo:
I don't know exactly. I can only provide the name of the stick and information you request. After inserting the stick into USB 3.0 port dmesg shows me among other things

[logrusx]

[nxe9]

The brand is German.
https://de.wikipedia.org/wiki/Intenso

This is a company that has been around for many years, but produces quite cheap products. Well, that's not the point here. In conclusion, it's a pity that I don't have the ability to detect errors with any specific tool. And regarding this pendrive, it's true, I have no choice but to throw it into the trash. Because everything indicates that there is something wrong with this stick.

Regarding the f3 and badblocks program, I think the problem is that they work on the principle of standardized data write. If the packages created by these programs do not generate an error, it will not detect them. It would be useful to have a tool that generates some random data on the host and on the pendrive and compares them with each other. I think that writing such a program yourself would not be difficult.

One more thing, I wonder how this pendrive will behave on Windows. I think I can check it later and let you know.

[NeddySeagoon]

nex9,

umount gives you a free sync, which is why it takes so long.
The prompt does not return until the data is committed to the USB stick and its safeto remove it.

in short, there is nothing wrong with your process.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[nxe9]

Further results:
- My second, new pendrive, same model, creates the same problems on my gentoo systems.
- Files do not get corrupted on Windows. However, I used ntfs there instead of ext4.

It seems as if the system had problems managing this pendrive correctly.

I will perform more tests soon to try to narrow down the problem:
- ntfs instead of ext4 on gentoo
- Another linux distro like ubuntu live cd.

I'll let you know later.

[NeddySeagoon]

nxe9,

ntfs and ext4 are both journalled filesystems.
The journal requires extra writes.

With ext4, the journal can be turned off to save the these writes.
That may be worth trying.

add -O ^has_journal to the command when you make the filesystem or play with tune2fs
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[pingtoo]

[nxe9]

Thank you for your help so far.

@NeddySeagoon
When invoking

[pingtoo]

[nxe9]

On windows I tested the data transfer only on the blue (usb 3.0) port.

When the pendrive is inserted into the blue or black slot the result in the device manager is the same.

https://i.postimg.cc/28mWg1Cr/devman.png

However, a change is visible in the Windows settings.

Blue: https://i.postimg.cc/tRFPr3QB/winset.png

Black: https://i.postimg.cc/dVVytg7f/winset2.png

So I conclude that Windows operates on USB 3.0.

[dmpogo]