| View previous topic :: View next topic |
| Author |
Message |
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
 Posted: Wed Mar 09, 2005 4:36 am Post subject: Some ext3 Filesystem Tips |
 |
|
Copyright (c) 2005 Peter Gordon
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.2
or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
Texts. A copy of the license can be found here.
Overview
I'm a big fan of the Third Extended ("ext3") filesystem. It's in-kernel and userspace code has been tried, tested, fixed, and improved upon more than almost every other Linux-compatible filesystem. It's simple, robust, and extensible. In this article I intend to explain some tips that can improve both the performance and the reliability of the filesystem.
In the document, /dev/hdXY will be used as a generic partition. You should replace this with the actual device node for your partition, such as /dev/hdb1 for the first partition of the primary slave disk or /dev/sda2 for the second partition of your first SCSI or Serial ATA disk.
I: Using The tune2fs and e2fsck Utilities
Before we begin, we need to make sure you are comfortable with using the tune2fs utility to alter the filesystem options of an ext2 or ext3 partition. Please make sure to read the tune2fs man page:It's generally a good idea to run a filesystem check using the e2fsck utility after you've completed the alterations you wish to make on your filesystem. This will verify that your filesystem is clean and fix it if needed. You should also read the manual page for the e2fsck utility if you have not yet done so:
 WARNING: Make sure any filesystems are cleanly unmounted before altering them with the tune2fs or e2fsck utilities! (Boot from a LiveCD such as Knoppix if you need to.) Altering or tuning a filesystem while it is mounted can cause severe corruption! You have been warned!
II: Using Directory Indexing
This feature improves file access in large directories or directories containing many files by using hashed binary trees to store the directory information. It's perfectly safe to use, and it provides a fairly substantial improvement in most cases; so it's a good idea to enable it:
| Code: | | # tune2fs -O dir_index /dev/hdXY |
This will only take effect with directories created on that filesystem after tune2fs is run. In order to apply this to currently existing directories, we must run the e2fsck utility to optimize and reindex the directories on the filesystem: | Code: | # e2fsck -D /dev/hdXY
|
 Note: This should work with both ext2 and ext3 filesystems. Depending on the size of your filesystem, this could take a long time. Perhaps you should go get some coffee 
III: Enable Full Journaling
By default, ext3 partitions mount with the 'ordered' data mode. In this mode, all data is written to the main filesystem and its metadata is committed to the journal, whose blocks are logically grouped into transactions to decrease disk I/O. This tends to be a good default for most people. However, I've found a method that increases both reliability and performance (in some situations): journaling everything, including the file data itself (known as 'journal' data mode). Normally, one would think that journaling all data would decrease performance, because the data is written to disk twice: once to the journal then later committed to the main filesystem, but this does not seem to be the case. I've enabled it on all nine of my partitions and have only seen a minor performance loss in deleting large files. In fact, doing this can actually improve performance on a filesystem where much reading and writing is to be done simultaneously. See this article written by Daniel Robbins on IBM's website for more information:
http://www-106.ibm.com/developerworks/linux/library/l-fs8.html#4
In fact, putting /usr/portage on its own ext3 partition with journal data mode seems to have decreased the time it takes to run emerge --sync significantly. I've also seen slight improvements in compile time.
There are two different ways to activate journal data mode. The first is by adding data=journal as a mount option in /etc/fstab. If you do it this way and want your root filesystem to also use it, you should also pass rootflags=data=journal as a kernel parameter in your bootloader's configuration. In the second method, you will use tune2fs to modify the default mount options in the filesystem's superblock: | Code: | | # tune2fs -O has_journal -o journal_data /dev/hdXY |
Please note that the second method may not work for older kernels. Especially Linux 2.4.20 and below will likely disregard the default mount options on the superblock. If you're feeling adventurous you may also want to tweak the journal size. (I've left the journal size at the default.) A larger journal may give you better performance (at the cost of more disk space and longer recovery times). Please be sure to read the relevant section of the tune2fs manual before doing so: | Code: | | # tune2fs -J size=$SIZE /dev/hdXY |
IV: Disable Lengthy Boot-Time Checks
WARNING: Only do this on a journalling filesystem such as ext3. This may or may not work on other journalling filesystems such as ReiserFS or XFS, but has not been tested. Doing so may damage or otherwise corrupt other filesystems. You do this AT YOUR OWN RISK.
Hmm..It seems that our ext3 filesystems are still being checked every 30 mounts or so. This is a good default for many because it helps prevent filesystem corruption when you have hardware issues, such as bad IDE/SATA/SCSI cabling, power supply failures, etc. One of the driving forces for creating journalling filesystems was that the filesystem could easily be returned to a consistent state by recovering and replaying the needed journalled transactions. Therefore, we can safely disable these mount-count- and time-dependent checks if we are certain the filesystem will be quickly checked to recover the journal if needed to restore filesystem and data consistency. Before you do this please make sure your filesystem entry in /etc/fstab has a positive integer in its 6th field (pass) so that it is checked at boot time automatically. You may do so using the following command: | Code: | | # tune2fs -c 0 -i 0 /dev/hdXY |
V: Checking The Filesystem Options Using tune2fs
Well, now that we've tweaked our filesystem, we want to make sure those tweaks are applied, right?  Surprisingly, we can do this options iusing the tune2fs utility quite easily. To list all the contents of the filesystem's superblock, we can pass the "-l" (lowercase "L") option to tune2fs: | Code: | | # tune2fs -l /dev/hdXY |
Unlike the other tune2fs calls, this can be run on a mounted filesystem without harm, since it doesn't access or attempt to change the filesystem at such a low level.
This will give you a lot of information about the filesystem, including the block/inode information, as well as the filesystem features and default mount options, which we are looking for. If all goes well, the relevant part of the output should include "dir_index" and "has_journal" flags in the Filesystem features listing, and should show a default mount option of "journal_data".
This concludes this filesystem tweaking guide for now. Happy hacking! 
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF
Last edited by codergeek42 on Sun Jan 08, 2006 5:38 pm; edited 5 times in total |
|
| Back to top |
|
 |
i92guboj
Bodhisattva
Joined: 30 Nov 2004
Posts: 10315
Location: Córdoba (Spain)
|
| Posted: Wed Mar 09, 2005 6:54 am Post subject: |
|
|
Nice thing!
Just one thing: woundn't it be:
| Code: |
tune2fs -O dir_index,has_journal /dev/hdXY
tune2fs -o journal_data /dev/hdXY
|
in place of:
| Code: |
tune2fs -O dir_index /dev/hdXY
tune2fs -o has_journal,journal_data /dev/hdXY
|
Editer for clarification: notice that 'has_journal' parameter is valid for '-O' (upper case 'o') modifier, not for '-o' (lower case 'o'). |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Wed Mar 09, 2005 3:50 pm Post subject: |
|
|
Gah! I can't believe I didn't see that.  Thanks 6thpink...
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
jetsaredim
n00b
Joined: 28 Sep 2003
Posts: 40
Location: North East US
|
| Posted: Wed Mar 09, 2005 7:07 pm Post subject: |
|
|
is there a way to list the options for a particular ext3 fs options that have been set?
I'm not sure if things are working correctly... I formatted the FS with mke2fs -j /dev/vg/ext3, but when I went to mount it it seems to be using the ext2 kernel module... |
|
| Back to top |
|
|
saffy
n00b
Joined: 31 Jan 2004
Posts: 14
Location: London, UK
|
| Posted: Wed Mar 09, 2005 7:37 pm Post subject: |
|
|
Thanks!
You are absolutely right about ext3 and full journal mode. I have several systems running a combination of XFS and ReiserFS and my main workstation has used both for various periods. Recently I backed up my workstation and switched to ext3 with full journal and have to report that the performance is nothing short of amazing! I also don't 'notice' disk access as much as I used to with XFS or ReiserFS, ext3 appears to be much smoother with disk access not interrupting normal workstation activity during heavy usage.
I would also recommend adding orlov and commit=9999 to you mount options.
As you can tell I am now a big fan of ext3 full journal mode 
_________________
===============================>
saffy |
|
| Back to top |
|
|
i92guboj
Bodhisattva
Joined: 30 Nov 2004
Posts: 10315
Location: Córdoba (Spain)
|
| Posted: Wed Mar 09, 2005 7:44 pm Post subject: |
|
|
| codergeek42 wrote: | | Gah! I can't believe I didn't see that. Thanks 6thpink... |
Thanks to u, I just got an error and looked into man page  Now my super-ext3-home reads smooth. The rest of my fs's are reiserfs, I think I'm gonna try, since I really prefer a hard-stable system, and ext3 if for me the best fs all over the world when it comes to reliability. |
|
| Back to top |
|
|
jetsaredim
n00b
Joined: 28 Sep 2003
Posts: 40
Location: North East US
|
| Posted: Wed Mar 09, 2005 7:47 pm Post subject: |
|
|
So, if I were looking for the best performance, I would run the commands in the original post?
I'm trying to do some performance testing for some research I'm working on... Its not easy trying to find good recommendations of performance tuning options for any of the filesystems (ext3, reiserfs, etc...) |
|
| Back to top |
|
|
i92guboj
Bodhisattva
Joined: 30 Nov 2004
Posts: 10315
Location: Córdoba (Spain)
|
| Posted: Wed Mar 09, 2005 8:16 pm Post subject: |
|
|
| jetsaredim wrote: | So, if I were looking for the best performance, I would run the commands in the original post?
I'm trying to do some performance testing for some research I'm working on... Its not easy trying to find good recommendations of performance tuning options for any of the filesystems (ext3, reiserfs, etc...) |
Yes, codergeek corrected it. |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Wed Mar 09, 2005 9:03 pm Post subject: |
|
|
| jetsaredim wrote: | | is there a way to list the options for a particular ext3 fs options that have been set? |
`tune2fs -l /dev/hdXY` will list the current contents of the filesystem's superblock.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
acasto
Apprentice
Joined: 06 Feb 2004
Posts: 236
Location: Durka-Durka-Stan
|
| Posted: Mon Mar 21, 2005 10:33 am Post subject: Re: Some ext3 Filesystem Tips |
|
|
| codergeek42 wrote: |
There are two different ways to activate journal data mode. The first is by adding journal=data as a mount option in /etc/fstab. If you do it this way and want your root filesystem to also use it, you should also pass rootfsflags=data=journal as a kernel parameter in your bootloader's configuration. |
Shouldn't that be data=journal for the option to /etc/fstab? Or does the order not matter?
- Adam
_________________
Leerrroooooyyyyyyyy JENKINS!!!!1111...................
"You know the Nazi's had pieces of flare.. that they made the Jews wear." |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Mon Mar 21, 2005 4:38 pm Post subject: Re: Some ext3 Filesystem Tips |
|
|
| acasto wrote: |
Shouldn't that be data=journal for the option to /etc/fstab? Or does the order not matter?
|
Your kernel has to mount your root filesystem before it can read the /etc/fstab file that it contains, so only putting it in there would effect everything _except_ your root partition . It's kind of like the chicken-and-egg problem...
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
acasto
Apprentice
Joined: 06 Feb 2004
Posts: 236
Location: Durka-Durka-Stan
|
| Posted: Mon Mar 21, 2005 7:34 pm Post subject: Re: Some ext3 Filesystem Tips |
|
|
| codergeek42 wrote: | | Your kernel has to mount your root filesystem before it can read the /etc/fstab file that it contains, so only putting it in there would effect everything _except_ your root partition . It's kind of like the chicken-and-egg problem... |
I thought that when it boots up and initially mounts the root filesystem as read-only, that it then looks at /etc/fstab and applies it upon remount to read-write. I just ask becaue I've been using data=journal in my /etc/fstab on the root filesystem for about the last year and it shows up as being mounted properly with that option.
- Adam
_________________
Leerrroooooyyyyyyyy JENKINS!!!!1111...................
"You know the Nazi's had pieces of flare.. that they made the Jews wear." |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Mon Mar 21, 2005 7:40 pm Post subject: |
|
|
Hm.. perhaps since Gentoo's initscripts do remount the root filesystem that does apply. Good point, Adam. Thanks.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
OneOfMany
Tux's lil' helper
Joined: 19 Nov 2003
Posts: 108
Location: Portland, OR USA
|
| Posted: Fri Apr 08, 2005 5:13 am Post subject: |
|
|
| I think the above post was pointing out that the above instructions shouldn't be "journal=data", but "data=journal" (for "/etc/fstab"). |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Fri Apr 08, 2005 3:05 pm Post subject: |
|
|
| OneOfMany wrote: | | I think the above post was pointing out that the above instructions shouldn't be "journal=data", but "data=journal" (for "/etc/fstab"). |
I'm stupid. I didn't even notice that. Thanks OneOfMany
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
barrct
n00b
Joined: 13 Apr 2004
Posts: 63
Location: Naples, FL
|
| Posted: Fri Apr 08, 2005 8:24 pm Post subject: |
|
|
I _should_ be running all of my drives and partitions in ext3, but I seem to end up with corruptions regularly. Is there a way to view the journal, or see if it is functioning?
_________________
Linux William 2.4.24 #3 SMP Wed Apr 28 19:28:29 EDT 2004 sparc64 sun4u TI UltraSparc II (BlackBird) GNU/Linux
cpu : TI UltraSparc II (BlackBird)
ncpus probed : 8
http://www.barrtechnology.com
http://chris.barrtechnology.com |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Fri Apr 08, 2005 8:41 pm Post subject: |
|
|
| barrct wrote: | | I _should_ be running all of my drives and partitions in ext3, but I seem to end up with corruptions regularly. Is there a way to view the journal, or see if it is functioning? |
Malfunctioning? That's definitely something I don't expect ext3 to do. Perhaps your hardware is going dead? Do other filesystems (such as ReiserFS, XFS, or *shudder* FAT32) work ok on the same disk(s)?
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
irondog
l33t
Joined: 07 Jul 2003
Posts: 715
Location: Voor mijn TV. Achter mijn pc.
|
| Posted: Fri Apr 08, 2005 9:18 pm Post subject: |
|
|
Is there any way to do defragmentation on ext3? My experience is that havily used ext3 partitions become slower and slower while the amount of files in the filesystem and used diskspace don't significally change.
_________________
Alle dingen moeten onzin zijn. |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Fri Apr 08, 2005 9:21 pm Post subject: |
|
|
ext3 automagically handles defragmentation as you use it. The best thing I can think of is to try re-optimizing the filesystem structure by running the following: | Code: | | # e2fsck -D /dev/hdXY |
(/dev/hdXY should be unmounted as explained in the first post in order to avoid filesystem corruption.)
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
barrct
n00b
Joined: 13 Apr 2004
Posts: 63
Location: Naples, FL
|
| Posted: Mon Apr 11, 2005 8:17 pm Post subject: |
|
|
| codergeek42 wrote: | | barrct wrote: | | I _should_ be running all of my drives and partitions in ext3, but I seem to end up with corruptions regularly. Is there a way to view the journal, or see if it is functioning? |
Malfunctioning? That's definitely something I don't expect ext3 to do. Perhaps your hardware is going dead? Do other filesystems (such as ReiserFS, XFS, or *shudder* FAT32) work ok on the same disk(s)? |
The hardware has seemed to be fine, I haven't used it much, well at all really in anything other than our main server since it required an HVD SCSI connection that isn't exactly a common thing.
As for the hardware going bad, well it should be, it seems to happen on every drive, and every drive is mirrored to another array. The setup in like this.
Server
||--- Array 1 A B C D D D D
|--- Array 2 A B C D D D D
Each array is on a different controller and different cables, and the drives are mirrored across the two arrays, PLUS the 4D drives are in a RAID 5 that is mirrored.
So if there is something going bad, I think that if ext3 can't catch it, then the RAID5 should and if it's the card or cable, the mirror should.
right?
So is there any way to test the journal? To view it? To view it's size?
_________________
Linux William 2.4.24 #3 SMP Wed Apr 28 19:28:29 EDT 2004 sparc64 sun4u TI UltraSparc II (BlackBird) GNU/Linux
cpu : TI UltraSparc II (BlackBird)
ncpus probed : 8
http://www.barrtechnology.com
http://chris.barrtechnology.com |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Tue Apr 12, 2005 2:58 am Post subject: |
|
|
That I'm not sure of sorry 
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
barrct
n00b
Joined: 13 Apr 2004
Posts: 63
Location: Naples, FL
|
| Posted: Tue Apr 12, 2005 12:36 pm Post subject: |
|
|
Yea, it's got me fairly perplexed as well. Of course, I was makeing shure that my backups are running and I saw this.
| Code: | Apr 12 03:02:25 William kjournald starting. Commit interval 5 seconds
Apr 12 03:02:25 William EXT3 FS 2.4-0.9.19, 19 August 2002 on md(9,3), internal journal
Apr 12 03:02:25 William EXT3-fs: mounted filesystem with ordered data mode.
Apr 12 03:09:53 William kjournald starting. Commit interval 5 seconds
Apr 12 03:09:53 William EXT3-fs: mounted filesystem with ordered data mode. |
So that leads me to think that the journal is running at least.
_________________
Linux William 2.4.24 #3 SMP Wed Apr 28 19:28:29 EDT 2004 sparc64 sun4u TI UltraSparc II (BlackBird) GNU/Linux
cpu : TI UltraSparc II (BlackBird)
ncpus probed : 8
http://www.barrtechnology.com
http://chris.barrtechnology.com |
|
| Back to top |
|
|
c0bblers
Guru
Joined: 28 Mar 2003
Posts: 403
|
| Posted: Tue Apr 12, 2005 7:44 pm Post subject: |
|
|
Hi,
After having used XFS for a LONG time I decided to switch totally to ext3 after playing with ubuntu on ext3. Here's what I noticed....disk head movement (or what sounds like it anyway). With XFS the disk is thrashed to within an inch of it's life at times compared to ext3, this probably isn't as noticable with some disks but my Barracudas are whisper quiet when the head isn't moving. It's particularly noticable when scrolling through a list of emails in a folder in Evolution, which for some reason causes a LOT of HD activity on XFS....ext3 is whisper quiet by comparison. I've noticed no slow down of any note, even with the full data journalling codergeek suggests. In fact, some apps seem to start up quicker...though that may be a placebo effect. All in all I'm very happy with ext3 so far, plus I have the warm fuzzy feeling of full data journalling keeping my precious data together. I'm officially a convert.
Cheers,
James |
|
| Back to top |
|
|
codergeek42
Bodhisattva
Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)
|
| Posted: Tue Apr 12, 2005 7:49 pm Post subject: |
|
|
| c0bblers wrote: | | I'm officially a convert. |
Excellent.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF |
|
| Back to top |
|
|
Crazor
Tux's lil' helper
Joined: 23 Apr 2003
Posts: 131
|
| Posted: Wed Apr 13, 2005 5:35 pm Post subject: |
|
|
| saffy wrote: | | I would also recommend adding orlov and commit=9999 to you mount options. |
when does one benefit from using orlov?
and what exactly does commit=9999 mean? |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
