olli.bo
Apprentice
Joined: 16 Jul 2003
Posts: 208
Location: Germany
|
 Posted: Thu Jul 26, 2007 7:42 am Post subject: OCFS2 - Kernel panic |
 |
|
Hi,
I have a three node cluster with ocfs2. On every node I have the same fs mounted. If i change a file on the mounted ocfs2-fs it is changed on every other node too. So the cluster works perfect I think.
In the background I have a working drbd (on two nodes) exported with gnbd for the third.
The problem is the following:
If I copy on one node files into the ocfs2-fs and while it is copying I power off one other node (for simulating a power cut) the remaining two nodes are getting a kernel panic after ca. 12 seconds.
If I do no copy or something like this power cut works perfect. After some seconds I can write files in the remaining two nodes and when the third node is restarted and mounted the ocfs2-fs the files are there too...
If I only copy files without power cut after ca. 1-3 minutes I'm getting kernel panic.
Here is the panic I'm getting:
| Code: | | Kernel panic - not syncing: ocfs2 is very sorry to be fencing this system by panicing |
See http://oss.oracle.com/projects/ocfs2/dist/documentation/ocfs2_faq.html - point 74 for details. This seems to be exactly my problem but using the deadline-scheduler dosn't seem to work...
Setting the heartbeat in /config/cluster/xencluster/heartbeat/dead_threshold to 121 changes the 12 seconds until panicing to 60, so this doesn't seem to make sense.
Is here anyone who can help me...?
Here are some configuration files and command outputs (same on every node):
/etc/ocfs2/cluster.conf
| Code: | node:
ip_port = 7777
ip_address = 184.1.72.201
number = 0
name = xentest01
cluster = xencluster
node:
ip_port = 7777
ip_address = 184.1.72.202
number = 1
name = xentest02
cluster = xencluster
node:
ip_port = 7777
ip_address = 184.1.72.203
number = 2
name = xentest03
cluster = xencluster
cluster:
node_count = 3
name = xencluster |
/sbin/mounted.ocfs2 /dev/gnbd/xen
| Code: | Device FS Nodes
/dev/gnbd/xen ocfs2 xentest01, xentest02, xentest03 |
/etc/fstab
| Code: | none /config configfs defaults
none /dlm ocfs2_dlmfs defaults
/dev/gnbd/xen /xen ocfs2 noauto 0 0 |
/etc/default/o2cb
| Code: | #
# This is a configuration file for automatic startup of the O2CB
# driver. It is generated by running /etc/init.d/o2cb configure.
# Please use that method to modify this file
#
# O2CB_ENABELED: 'true' means to load the driver on boot.
O2CB_ENABLED=true
# O2CB_BOOTCLUSTER: If not empty, the name of a cluster to start.
O2CB_BOOTCLUSTER=xencluster
# O2CB_HEARTBEAT_THRESHOLD: Iterations before a node is considered dead.
O2CB_HEARTBEAT_THRESHOLD=121 |
/etc/conf.d/ocfs2
| Code: | # Copyright 1999-2006 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo-x86/sys-fs/ocfs2-tools/files/ocfs2.conf,v 1.1 2006/07/20 05:13:14 dberkholz Exp $
# Put your cluster names here, separated by space, ie.
OCFS2_CLUSTER="xencluster"
O2CB_HEARTBEAT_THRESHOLD=121 |
/etc/cluster/cluster.conf
| Code: | <?xml version="1.0"?>
<cluster name="xencluster" config_version="5">
<cman>
</cman>
<clusternodes>
<clusternode name="xentest01">
<fence>
<method name="single">
<device name="human" nodename="xentest01"/>
</method>
</fence>
</clusternode>
<clusternode name="xentest02">
<fence>
<method name="single">
<device name="human" nodename="xentest02"/>
</method>
</fence>
</clusternode>
<clusternode name="xentest03">
<fence>
<method name="single">
<device name="human" nodename="xentest03"/>
</method>
</fence>
</clusternode>
</clusternodes>
<fencedevices>
<fencedevice name="human" agent="fence_manual"/>
</fencedevices>
</cluster>
|
/etc/drbd.conf
| Code: | global { usage-count yes; }
common { syncer { rate 10M; } }
resource xencluster {
protocol C;
net {
# cram-hmac-alg sha1;
shared-secret "XXXXXXXX";
allow-two-primaries;
after-sb-0pri discard-least-changes;
}
on xentest01 {
device /dev/drbd0;
disk /dev/sdb1;
address 184.1.72.201:7789;
meta-disk internal;
}
on xentest02 {
device /dev/drbd0;
disk /dev/sdb1;
address 184.1.72.202:7789;
meta-disk internal;
}
} |
uname -r
dmesg | grep scheduler
| Code: | io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered (default)
io scheduler cfq registered |
I formatted the ocfs2 with
| Code: | | mkfs.ocfs2 -N 3 /dev/gnbd/xen |
thx
olli |
|
Other Things Gentoo
