[SOLVED] Gentoo dies for no reason

[robak]

Hi folks.

i'm habing trouble with my gentoo system for some weeks now. the system dies suddenly and randomly. the x-server freezes, rebooting doesnt work (ALT-F1 -> CTRL-ALT-DEL) only Magic SysRq keys work. so it seems the kernel is still running but syslog isn't logging any troubles. This problem appears every 3-4 days.

Here some infos about my system (today state, i'm doing daily updates):
hardware:

[NeddySeagoon]

robak,

I suspect memory or motherboard problems.

Try a few cycles of memtest. If it finds something, it does not always mean RAM is faulty.

It could also be power related. Does the system dying correlate with power surges or do you have a proper UPS?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[robak]

memtest ran for about 12hrs without any error. i have no UPS but a 350W BeQuiet PSU. i took some power measurements and the system takes about 90W at full load, about 50W at idle (that was some days ago, sry i missed to mention it)
by idle i mean all hdds (except system-hdd) are in stand-by and by full-load all cpu-cores are under heavy load plus all hdds are running.

i also tried another mainboard, same problems.

my last try today is to unplug any USB HDDs. i will see in a few days if that helps.
and oh, all HDDs are encrypted with truecrypt except the system hdd.

ok, just to be as clear as possible, here's a list of all hdds:

System: external USB drive, power via USB (changed this today, internal now)
Data: four internal hdds
one external with own PSU (changed this today, internal now)
_________________
run this in your gentoo-bash:
"grep -R -i -A2 -B2 'on fire' /usr/src/linux/*"

[NeddySeagoon]

robak,

At first sight, a 350w PSU seems plenty but its not the headline total power that matters, its the limit per voltage, per output and per combinations if any.
You normally hit the 3.3v/5v limit before you get anywhere near the headline power.

Also many PSUs have several independent 12v supplies. At 350w, you will not have more than two. Each one must not be overloaded.
All the gory detail will be on the label on the PSU. The only way to check for the PSU running at or close to its limit is to look at each DC supply separately and in combination as the label on the PSU says.

By external power surges, I had in mind 'brownouts'. This is when the supply to the PC drops out of voltage specification but does cut out altogether.
Such events can cause the DC power supplies inside the PC to go out of spec and the system to behave in odd ways without the PWR OK signal going false and triggering a reset.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[robak]

thanks for your explanation, NeddySeagoon.
here are the datas for my PSU (its a 300W PSU, my mistake):
Output:
3.3V -> 23A
5V -> 23A
max combined power: 150W

12V1 -> 18A
12V2 -> 18A
max combined current: 21A
max combined power: 252W

-12V -> 0.5A (max power: 6W)
5VSB -> 2A (max power: 10W)

each HDD is label with this power consumption:
5V -> 0.72A
12V -> 0.52A

if you are right, i'm wondering why the system crashes after several days.
_________________
run this in your gentoo-bash:
"grep -R -i -A2 -B2 'on fire' /usr/src/linux/*"

[NeddySeagoon]

robak,

The actual PSU current limits will change with load and temperature.

Now we need to know how your system is wired to calculate the loads. We also need to know a little about your motherbaord, RAM and how the CPU is powered.
If you are using the extra ATX +12v connector, that will provide power to the CPU core and nothing else. The drive spin motors will be operated from the other 12v supply. The motors will have a spin up surge at least twice the 0.52A advertised.

The -12v and 5vSTBY can be ignored. The -12v is used only by serial ports and the 5vSTBY for switching the system on/off and possibly powering a USB root hub, so the mouse or keyboard can bring the system out of standby.

When you add up all the max combined power, you get well past 300w ... I make it 408w, so there is a further implicit restriction, that the 3.3v, +5v, +12v power must not exceed 300w. (Ignoring the -12v and 5v STBY.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[robak]

yes, i'm using the extra +12V ATX connector for the cpu. two harddrives are connected to one +12V supply the other three to the other +12V supply.
the cpu is a Core2Quad 9550 E0-stepping, as i mentioned in my first post. it's running at fixed 1,1375V core-voltage.

(just to be clear, i tested the cpu at 1,100V with mersenne prime95 test for 48hrs with two setups; first, one instance with four threads for 24hrs (8k-6M FFT size) and second, four instances with one thread each with the same settings for another 24hrs. so i think the cpu is not the problem for my crashes.)

i already posted the rest of my hardware setup in my first post. everything else is at stock voltage.
_________________
run this in your gentoo-bash:
"grep -R -i -A2 -B2 'on fire' /usr/src/linux/*"

[mattmatteh]

2 things to try 1) setting the max cpu freq to the lowest and see long you can go. 2) go into the bios and underclock the ram. i have had random freezes and took me a month to find out it was bad ram. memtest kinda found problems but nothing consistant.

[NeddySeagoon]

robak,

Your CPU draws 100A at 1.1375v, thats 114W.
All that comes out of the extra 12v supply provided by the additional 4 pin ATX connector.
There is a DC-DC converter right next to the CPU to convert the 12v to the 1.1375v the CPU needs. That converter is not 100% effecient, so lets say the power out of the PSU is 120w, as thats 10A on the +12v supply for the CPU.

We will count your hard drives at 1A each on the 12v, to allow for spin up, thats another 5A ...or 15A total on the +12v, or 180W
That leaves 6A more for other things. It leaves 120W for everything else in your system.

Lets count the 5v for the hard drives at 1A each too ... thats 5A or 25W. Now we are up to 205W

The next power hungry device is the GPU, and after that any plug in cards.
The Intel 4500 chipset needs 24W ... thats 229W and we have yet to take account of your RAM and any plug in cards, CDROM or floppy.

The 229W is a worst case figure (so far) but it does not provide the whole picture. Thats static load.
The load in a PC is anything but static. For example, the CPU can go from next to nothing to full load in a few clock cycles. The PSU has to cope with this dynamic load too or you get all sorts of odd behavior.

The only way to test for dynamic load is with specialist test equipment or by substitution. Do you have a spare PSU you can swap with to test?
PSU dynamic load capability tends to reduce with age too.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[robak]

@mattmatteh: 1st) i always do this before i test cpus at highest multiplicator to check if i get crashes when the cpu is in idle mode. 2nd) already done this; checkes with memtest for 12hrs, no errors but system still crashed randomly, so i set ram back to stock frequency.

@NeddySeagoon: 120W for cpu, well, ok for the VERY worst case since intel specs are talking about 100A max for my cpu, but since the TDP is 95W which is never reached i would calc 80W as worst case for the cpu. but ok, lets take your 120W, just to be sure.
but the 180W (cpu+harddrives) are splitted on three 12V lines or am i taking something wrong?

this 300W PSU is about one year old but i have another 500W PSU in the age of 3 years. if the system crashes again, i'll test with that one.

i have neither an optical drive nor a floppy.

the system is up for 1d5h now since i unplugged the usb-drives and build them into the case. *fingers crossed*

thanks for your great help so far.
_________________
run this in your gentoo-bash:
"grep -R -i -A2 -B2 'on fire' /usr/src/linux/*"

[dmpogo]

[NeddySeagoon]

robak,

You PSU label says

[robak]

since I unplugged the USB HDDs and connected them internally I had no crash. so I think it was an power problem at the USB hub.
nevertheless, thanks for your help !
_________________
run this in your gentoo-bash:
"grep -R -i -A2 -B2 'on fire' /usr/src/linux/*"