large file to usb -> kernel panic [solved]

houtworm · Last edited by houtworm on Tue Jan 10, 2017 5:10 pm; edited 1 time in total

I tried to create a bootable usb stick from a 3,5 Gb iso image.
So at first I tried

eccerr0r · Posted: Sun Jan 08, 2017 9:02 pm Post subject:

That's not good...

How about a stable version of the kernel (4.4.39)?
Are you using vanilla or gentoo-sources all around?

Do you have a dump of the panic? Is it the same error each time?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

houtworm · Posted: Sun Jan 08, 2017 9:33 pm Post subject:

I use only gentoo-sources

I will install gentoo-sources and try it again.
With dd on the commandline I can see the kernel dump but I have to write it down because the computer will not respond anymore.

I don't know if the error is the same every time. Perhaps I will try it twice to see it :-)

But first I must reinstall grub because that other OS has ruined my bootsector.
_________________
niemand is onbekwamer, dan een timmerman zonder hamer

Kees

houtworm · Posted: Mon Jan 09, 2017 1:01 am Post subject:

Ok, I installed kernel gentoo-sources-3.4.113-r1 and tried again.
And.. got another kernel panic:

Tony0945 · Posted: Mon Jan 09, 2017 1:36 am Post subject:

Pardon me for what may be a silly question, but was /dev/sdc mounted during the dd operation? Are you sure the device is /dev/sdc? Did you try bs=8192k as the wiki said? I can see writing on a mounted filesystem causing a panic.

Hu · Administrator Joined: 06 Mar 2007 Posts: 23064

That panic looks to be more about the lack of progress writing to the device than due to any confusion associated with changing the filesystem while it was mounted.

OP: what manufacturer/model USB stick is this?

houtworm · Posted: Mon Jan 09, 2017 2:07 am Post subject:

It is a kingston usb stick.
I tried bs=8192 -> kernel panic
then bs=8192k -> kernel panic, did not take long
no bs -> kernel panic, after 20 minutes writing

of course /dev/sdc was not mounted.
Yes I am sure that /dev/sdc was my usb stick.
_________________
niemand is onbekwamer, dan een timmerman zonder hamer

Kees

eccerr0r · Posted: Mon Jan 09, 2017 3:07 am Post subject:

Do you have any other sticks to test with, is it just this one stick?

My guess is that this stick gave up on a write for whatever reason but reported that it's still trying - and the kernel got confused when it couldn't make forward process writing. Especially if there was a barrier... This is bad behavior in any case though I'm not exactly sure what the best way to handle this is when there's data to be written in flight.

It could also be a USB hardware issue...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

yilmi · n00b Joined: 03 Jan 2017 Posts: 3

Panic on a hung task seems to be enabled on your system (see https://www.kernel.org/doc/Documentation/sysctl/kernel.txt)
Try : sysctl -w kernel.hung_task_panic=0

However, you device shouldn't be unresponsive for a 120s (default).... it can also be caused by a hardware error or some cheap USB sticks/SD cards that have "smart" controllers that might produce some unexpected behavior (https://www.bunniestudios.com/blog/?p=3554)

houtworm · Posted: Mon Jan 09, 2017 9:03 pm Post subject:

Ok what I have tried..
downgrade core-utils to 8.23 to see if dd has perhaps changed but I still got a kernel panic.
I noticed in gkrellm at the disk activity that the writing suddenly stopt but the reading continues.
At that moment dd can't be killed and I can't reboot either so... reset switch.

I will try another usb stick and see if that works.

I will also try the hint from yilmi.

I will know the result within 10 minutes.
_________________
niemand is onbekwamer, dan een timmerman zonder hamer

Kees

houtworm · Posted: Mon Jan 09, 2017 9:32 pm Post subject:

Ok another stick, from lexar this time.
and.. another panic and now I could see it in the logfile, thanks to yilmi:

eccerr0r · Posted: Mon Jan 09, 2017 10:00 pm Post subject:

Different coreutils shouldn't affect this issue, this is a kernel or more likely hardware problem.

It should not panic or oops or WARN due to a unresponsive disk.

What USB hardware are you using?

This is weird, blkdev_get is used during opening a device - but the bulk write has already happened... or at least part of it... Did it write anything to the flash disk?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

houtworm · Posted: Mon Jan 09, 2017 10:17 pm Post subject:

It happens on both computers.
The other pc has a gygabyte x790ud3 mobo
Ths usb2 port works ok. I have copied my gentoo system to it without problems, from an externel harddrive connected to the usb2 port.
There is written on the stick. Perhaps if I can mount an iso.. I can see more.

it looks like everything is on the stick.
I will try if I can boot the pc from it.
..no does not work.
but it looks like everything is written to the stick
du -c gives the same result for the iso and for the stick

weird things happen..
_________________
niemand is onbekwamer, dan een timmerman zonder hamer

Kees

Hu · Administrator Joined: 06 Mar 2007 Posts: 23064

houtworm · Posted: Tue Jan 10, 2017 2:41 am Post subject:

I think it is a kernel panic because my machine blocked every time with flashing keyboard leds. Until I did what yilmi wrote:

houtworm · Posted: Tue Jan 10, 2017 9:08 am Post subject:

Ok, I wrote another iso to the stick and that worked. No bs=xx used with dd
It was 368 Mb instead of 3,5 gb

So it goes wrong if the filesize is too large.
_________________
niemand is onbekwamer, dan een timmerman zonder hamer

Kees

houtworm · Posted: Tue Jan 10, 2017 5:09 pm Post subject:

ok problem solved!

When using direct i/o it works:

Tony0945 · Posted: Tue Jan 10, 2017 5:28 pm Post subject:

Never heard of the direct i/o flag before. It would seem that writing to a device IS direct.
But I found this: http://www.alexonlinux.com/what-is-direct-io-anyway

Maybe you don't have enough swap to cache 3.5G?

Thanks for the thread. One learns more every day.

NeddySeagoon · Posted: Tue Jan 10, 2017 5:47 pm Post subject:

Tony0945,

swap is only used for dynamically allocated RAM. RAM content that does not have a permanent home on disk.
Thus dirty buffers - waiting to be written, will never be written to swap. They will just wait their turn to be written.

Meanwhile, as RAM fills up with dirty buffers, everything grinds to a halt.
Dynamically allocated RAM is pushed to swap, read only data/code is flushed, as it can be reread later and RAM is filled up by the read task of the copy operation.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Tony0945 · Posted: Tue Jan 10, 2017 6:20 pm Post subject:

NeddySeagoon · Posted: Tue Jan 10, 2017 6:29 pm Post subject:

Tony0945,

He would still have got the panic due to the hung task but up to that point, everything would look normal.
It appears that 120 sec was not enough to write enough to keep the kernel from detecting a hung task.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

Tony0945 · Posted: Tue Jan 10, 2017 7:07 pm Post subject:

Why did --directio solve that problem? What mechanism was bypassed?

NeddySeagoon · Posted: Tue Jan 10, 2017 7:30 pm Post subject:

Tony0945,

The kernels cache was bypassed.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

eccerr0r · Posted: Tue Jan 10, 2017 8:30 pm Post subject:

EIther way, there is still a problem with the kernel or hardware. This is merely a workaround.

Being able to panic or hang the kernel in userland should not be possible (other than by deliberate corruption, but that's a different issue.)
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

houtworm · Posted: Tue Jan 10, 2017 11:42 pm Post subject:

Anybody can try it.
I just tried a copy: