JFS or ext3, suggestions?

leviathan4444 · n00b Joined: 08 Oct 2006 Posts: 13 Location: Champaign, IL

Hi,

I've searched through the forums, but haven't really found the ultimate answer to my question. I'm reconfiguring a server for a small business that I work for. We currently have about 1.4TB of data, and will be expanding the server to 3TB. The integrity of data is critical, I can't afford there to be any data corruption. I previously used reiserfs v3.6, and the only problem I've ever had is now that the server is up to 98% or so utilization of space, its gotten a bit slow. Due to stagnating development on reiserfs, I think it would be best to switch to another filesystem. I've read up on all of them. I've read that XFS can have some corruption issues -- I read a forum posting by a guy that said he migrated 9 systems to XFS, and within around 6 months, they all had data corruption. This rules out XFS. ext4 is out, as it simply isn't proven enough yet. This pretty much boils it down to ext3, and JFS. I do have a UPS hooked up to this server that will auto power it down in case of power failure. It should also be noted that I'm planning on using pretty much all the available space in one filesystem, and the 3TB filesystem I will create will probably need to be expanded in the future. Running out of inodes isn't a huge issue, as most of the files we create are 50-100MB or larger. Data integrity is most important followed by performance. I would appreciate any thoughts and/or recommendations on this issue that you guys might have. Thanks so much!

gentoo_ram · Posted: Tue Mar 24, 2009 10:51 pm Post subject:

Huh, I've been using XFS for years. Haven't had any data corruption issues. The risk with XFS is that if the machine goes down, then some data might be lost since the data and metadata aren't flushed at the same time. But this is a possible issue no matter what if the power is lost. It sounds you have the power issues in hand though.

leviathan4444 · n00b Joined: 08 Oct 2006 Posts: 13 Location: Champaign, IL

Thanks for replying. Its good to hear the positive review of XFS. If you've been using it successfully for a few years, then I might add XFS to my list of filesystems to consider.

FizzyWidget · Posted: Tue Mar 24, 2009 11:55 pm Post subject:

only issue i seem to have with ext3 is the fact it never uses the whole of the lvm i have made and then ontop of that reserves 5% of the space, which i turn off anyway, not really required on a storage drive
_________________
I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch.

agent_jdh · Veteran Joined: 08 Aug 2002 Posts: 1783 Location: Scotland

I used to run JFS exclusively on two boxes here for a few years - switched each drive to ext3 because it gave a big performance increase (on the same hardware), especially syncing the portage tree. Not had any data corruption/loss issues with either FS, and that includes plenty of hard resets.
_________________
Jingle Jangle Jewellery

tuber · Apprentice Joined: 12 Nov 2004 Posts: 267

My gripe with ext3 is that it can't be resized dynamically (while mounted), while JFS can.

leviathan4444 · n00b Joined: 08 Oct 2006 Posts: 13 Location: Champaign, IL

I really appreciate all the replies to my question so far. Any other thoughts or suggestions would be most welcome.

I probably should have mentioned before that this setup will be used for a moderate number of smaller files (not many tiny ones though,) and a large number of large files 100MB -> ≈1.2GB.

FizzyWidget · Posted: Wed Mar 25, 2009 10:44 am Post subject:

it all depends on what you are going to do with the drive and fs, as different file systems have difference strengths and weaknesses

http://linuxgazette.net/102/piszcz.html - bit old now, and i suppose there are newer ones about, i just found that in my book marks, but it should give you and idea of what i mean
_________________
I know 43 ways to kill with a SKITTLE, so taste my rainbow bitch.

jormartr · Apprentice Joined: 02 Jan 2008 Posts: 174

I have been reading though the internet about jfs and xfs, looking for a better filesystem (compared to ext3).

From what I have readen, on several websites, blogs, benchmarks, forums and so on..., XFS is the best one to choose when you have to deal with big files on big disk systems. It scales very well when you add more disks and cpu's to the system. Sounds really nice, but to be so good it aggresively caches data, so you must rely on stable hardware and battery back up, if not you may end with filesystem corruption.

JFS seems a nice "overall features" good fs. Has a very low cpu usage and in general terms seems faster than ext3.

This is the impression I have after reading some sites, but of course, some of these sometimes does not have good words on these file systems. On some of them read histories about xfs corruption, and histories of JFS being slower than ext3.

As my home hardware is not enterprise quality, I am afraid to go for XFS. My home machine is attached to a UPS, but I sometimes play with it, so I can expect some hungs from time to time., and I have decided to try JFS this weekend.

rambam · Posted: Wed Mar 25, 2009 11:18 am Post subject:

I use JFS for the volume my /home is mounted on and ext3 for the rest.
I have been happy with that combination.
_________________
The soul, when accustomed to superfluous things, acquires a strong habit of desiring things. This desire is without limit, while things which are necessary are few in number.

djinnZ · Posted: Wed Mar 25, 2009 12:52 pm Post subject:

The only real problem with jfs (i never more use it, so my report is sure outdated) is than in case of corruption you can not easy fix the filesystem but in fact, xfs is really slow as delete a large amount of files (a real pain for the tmp or var directory management) but can be easy (rare necessity) defragmented, reiser 3.6 hardly lock itself but I have experienced some data loss (files vanished) and suffer from fragmentation more than ext3, ext3 in front of xfs or reiser is slow etc... every filesystem has good and bad points.

The solution of mine is to have the root, the home and the portage distifiles (the main portage tree is on squash) in xfs and the var, the tmp and some shared for users in reiser (3.6 not 4).
Hard caching, necessity of backups for power and data etc. for xfs are urban legends (referred to solved issues) now.
The only filesystem has really caused troubles (an entire partition loss) in my experience is ext3.

But for a large filesystem as you think (more than 3TB) the only suggestion is to verify if the problem on remove the dirty flag is solved in jfs now, the security is the only question because the performance of xfs and jfs are similar.
_________________
scita et risus abundant in ore stultorum sed etiam semper severi insani sunt :wink:

mala tempora currunt...mater stultorum semper pregna est :evil:

Murpy'sLaw:If anything can go wrong, it will - O'Toole's Corollary:Murphy was an optimist :wink:

energyman76b · Advocate Joined: 26 Mar 2003 Posts: 2048 Location: Germany

reiserfs is STABLE. Not stagnant, STABLE. Means, no stupid featuritis. Bugfixes, nothing else.

ext3 was never as stable as reiserfs - so if it works well for you, why switch away?
Especially why switch to fs that don't care about your data? ext3 doesn't care by default and jfs can't even forced to care.

Stay with reiserfs.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males

I identify as a dirty penismensch.

alex.blackbit · Advocate Joined: 26 Jul 2005 Posts: 2397

i never had data corruption problems with xfs in many years.
on my machines jfs did not behave nicely in the past.
i got corrupted filesystems on power outage that were very difficult to repair.
i'd recommend xfs.

energyman76b · Advocate Joined: 26 Mar 2003 Posts: 2048 Location: Germany

don't use ext3, except if you love occasional 480sec pauses:
http://marc.info/?l=linux-kernel&m=123789077610123&w=2
(and the 10 before that)
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males

I identify as a dirty penismensch.

mathfeel · l33t Joined: 03 Aug 2004 Posts: 700

leviathan4444 · n00b Joined: 08 Oct 2006 Posts: 13 Location: Champaign, IL

First off thanks again for all the posts. I really appreciate all the input.

energyman76b - I like your thoughts on reiserfs3.6. I do indeed aggre that its stable. However, I am concerned for the future. From my understanding, there are apparently only two guys left who do bugfixes on reiserfs3.6, and these two guys have other priorities. I read this in a posting somewhere, while researching the development status on reiser. Hans is, of course in jail, and namesys is apparently dead. I have encounted the slowdown with reiser that I have read about, but I think that's to be expected when you have 15GB out of 1.4TB available. We sometimes get pauses that can last up to a minute, but they always work themselves out. These pauses have only cropped up now that the system is approaching 100% utilization of space. This has been my only problem with reiserfs so far. I'm concerned that a future change in the kernel, or some other issue might cause a problem with reiser in the future that might not get fixed. I'm also concerned that it may eventually be deprecated from the kernel anyway. I have used it in the past on other systems, as well as on our current system, and its predecessor, without any trouble. If the whole jail thing hadn't happened, I wouldn't even be considering a switch. I might decide to stick with it, but I'm leaning away from it. Also, thanks for the post on ext3 delays.

Carps - Thanks for the filesystem benchmark link.

djinnZ - thanks for all the info. I'll look up jfs and the dirty flag issue. Interesting that you had an entire partition fail with ext3.

alex.blackbit - thanks for the vote for XFS. I am reconsidering using this filesystem.

All in all, I still haven't made up my mind yet. If Hans wasn't in jail, or Namesys wasn't defunct, I would most likely just continue with resier3.6. However, since this is not the case, and even though reiser3.6 is quite stable, I'm worried about how its future, which forces me to consider alternatives. Currently I'm warming up to XFS, cooling on ext3, and still considering jfs with reiserfs as a fallback. Currently I'm running some tests on our old system, and I might just benchmark all the filesystems with bonnie++ and see where that leaves me. Any other thoughts you guys have would be most welcome. Thanks!

rrbrussell · n00b Joined: 21 Jun 2004 Posts: 64

I would suggest:

ext4: stable for me, resizeable, data journalling, in my experience about 25% to 50% faster than ext3 with data=journal on both
ext3: stable, resizeable, data journalling
xfs: works

Up until a few months ago I was using XFS regularly on my data server/router but it just got to slow and I upgraded to ext3. My main system uses ext3 for / and ext4 for just about everything else. I have had a few software crashes and power failures with no troubles from any of these file systems.

Mod edit: Fixed the list BBcode for you. --timeBandit

energyman76b · Advocate Joined: 26 Mar 2003 Posts: 2048 Location: Germany

have you even read the thread I posted? even Linus has nothing good to say about ext4. Ext4 is braindead. An fs that does not care about data is an fs that should not exist.

ext3 does not care. ext4 does not care. xfs does only care partially.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males

I identify as a dirty penismensch.

energyman76b · Advocate Joined: 26 Mar 2003 Posts: 2048 Location: Germany

emm.. jfs is in so deep maintanence mode that there is only a patch or two in years - and you are worried about reiserfs' future?
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males

I identify as a dirty penismensch.

timeBandit · Posted: Thu Mar 26, 2009 4:20 am Post subject:

energyman76b · Advocate Joined: 26 Mar 2003 Posts: 2048 Location: Germany

djinnZ · Posted: Thu Mar 26, 2009 7:12 pm Post subject:

ccp · n00b Joined: 14 Sep 2007 Posts: 62

Hi leviathan4444,

This is my understanding of your question, you have 1.4 TB data on a reiserfs file system. it is at 98% full. you will migrate this data to a new 3TB storage(file system TBD) the this new storage expect to expend very soon. filesize >=50MB~100MB.

And you said "Data integrity is most important" then it follow by "performance".

My question,

what do you mean by "Data integrity"? do you mean it is same as when they were saved on to the disk or you are actually thinking "data availability"?. if you mean former then how do you know your existing data have not corrupt yet? do you have some kind of check sum database for each file? or you application can verify "data integrity" on the fly?(thinking DB)

if you are sure you have the "data integrity" state now then just make sure you have a good backup of the data and have formal procedure to practice restore periodocally to ensure you have good backup then you are done.

once you have a good backup/restore practice then file system type choose become less relevant as all you care about would be which file system can give you most flexibility and best performance. you want to have a file system that can give you expandability on the fly and on the fly snapshot and quick back online when it went dirty (crash). be concern of you new storage does it have enough performance to meet you ultimate wost case to restore entire data set in time you business defined requirement.

think about it, how long you have the 1.4TB under your care and how often you have data corruption problem cause by file system and assume you do have corruption problem, are they really file system bug or just system crash cause lost of data?

I think if you take my suggested view you may see a whole new ball game now:)

Ping.

leviathan4444 · n00b Joined: 08 Oct 2006 Posts: 13 Location: Champaign, IL

djinnZ: Thanks for the info on ext3, I'll definitely avoid using it then. I guess that just leaves me with jfs, xfs, and reiserfs3.6.

ccp:

ccp · n00b Joined: 14 Sep 2007 Posts: 62

leviathan4444,

So you do realize the important of backup, that is good.

Now we analyze the problem, there are multiple possibility cause data corruptions; it can be cause by OS (file system code) or it can be cause by bad application code(bug), or it can be bad application logic(misunderstand OS API). the file system was design to store files under most common access pattern i.e. application open a file, write data, close file. file system manage create inode, link inode to the data segments, delete file by delete inode and from data segments. most file system can handle directory content with 10s of thousands of files. may be 10 to 100 of concurrent creation/modification of file system tree in different area.

So do you have a very unusual access pattern? i.e very very large amount of files store in one directory or very very high rate of modify directory tree? because unless you have these unusual access pattern the chance of file system corruption by itself is very very rare, it happen more often is the underlying storage system have problem which cause file system corruption, and this is really nothing file system can do to prevent it happen.

and trying to find a perfect solution to hope that so bad thing will never happen is just not practical why not spent the effort to find a solution to address when bad thing happen how to best recover it quickly.

I don't think there are kernel file system hacker in here to read this forum/thread. so do you really want to take our word on your business future?

on the other hand we can discuss this in a pure technical fashion.

if you think your access pattern worth to discuss then tell us about it we can see if the chance of corruption is high and may be we can together do some testing to find and possible file system problem or solution.