View previous topic :: View next topic |
Author |
Message |
mikkoc Apprentice
Joined: 24 May 2007 Posts: 231
|
Posted: Sun Oct 26, 2008 9:10 pm Post subject: |
|
|
Schwinni wrote: |
You want to use extents:
Code: | tune2fs -O extents /dev/sda8 |
Do that before you mount it as ext4. |
Thanks, I already have 256 inode size.
One more question:
can I do "tune2fs -O extents /dev/sda8" while the partition is mounted?
If not, is it ok to do it from an ubuntu liveCD?
EDIT: ok, it seems everything went fine:
EXT3-fs: sda8: couldn't mount because of unsupported optional features (40).
EXT4-fs: barriers enabled
EXT4-fs: delayed allocation enabled
EXT4-fs: file extents enabled
EXT4-fs: mballoc enabled
EXT4-fs: mounted filesystem with ordered data mode.
EXT4 FS on sda8, internal journal on sda8:8
What about the first line? Does it try to mount as ext3 or what? |
|
Back to top |
|
|
Schwinni Apprentice
Joined: 02 Sep 2004 Posts: 214 Location: quadrant1.earth. germany.wuerzburg
|
Posted: Sun Oct 26, 2008 11:01 pm Post subject: |
|
|
Have a look at the thread, I have exactly the same line in my log
The funny thing is that it only appears for my root partition.
For the home partition everything is normal.
If I wouldn't have a separate boot partition with ext2 I would say it is because grub doesn't handle ext4, but this way...
I really don't know what it is.
I also fscked my partitions now several times and everything seems to be fine.
I think I try to find out something about that message on the ext4 mailing list or on the IRC channel... |
|
Back to top |
|
|
boris64 Veteran
Joined: 04 Oct 2003 Posts: 1770 Location: Vechelde/Peine
|
Posted: Mon Oct 27, 2008 1:56 pm Post subject: |
|
|
Yeah, that's what i found, too. The patches from those sites (grub4ext4) don't apply cleanly to
an unpatched grub-0.97 and grub-1.97, which claims to have preliminary ext4 support, doesn't even
build on amd64 because of missing 32bit lzo.
Did i miss something important? So, anyone out there who got it compiled
and who is already able to boot from an ext4 partition? _________________ boris64.net 200x / visit my desktop / try these tiny kernel patches |
|
Back to top |
|
|
octoploid n00b
Joined: 21 Oct 2006 Posts: 65
|
Posted: Mon Oct 27, 2008 5:17 pm Post subject: |
|
|
boris64 wrote: |
Yeah, that's what i found, too. The patches from those sites (grub4ext4) don't apply cleanly to
an unpatched grub-0.97 and grub-1.97, which claims to have preliminary ext4 support, doesn't even
build on amd64 because of missing 32bit lzo.
Did i miss something important? So, anyone out there who got it compiled
and who is already able to boot from an ext4 partition? |
Yes, I switched my root partition to ext4 yesterday and my system (amd64) boots without
problems with grub2.
You'll have to install grub2 by hand (the version in portage does not work), but is is rather straightforward.
Code: | svn co svn://svn.savannah.gnu.org/grub/trunk/grub2/
cd grub2
./configure
make
# make install
# grub-install /dev/sda
# vim /boot/grub/grub.cfg
|
The grub2 syntax is different. This is my boot/grub/grub.cfg:
Code: | # Timeout for menu
set timeout=6
# Set default boot entry as Entry 0
set default=0
# Fallback to the second entry.
set fallback=2
# Entry 0 - Load Linux kernel
menuentry "Gentoo" {
set root=(hd0,1)
linux /boot/kernel254 root=/dev/sda1 rootflags=commit=60 hpet=force fbcon=rotate:3
}
menuentry "git kernel" {
set root=(hd0,1)
linux /usr/src/linux/arch/x86_64/boot/bzImage root=/dev/sda1 rootflags=commit=60 hpet=force fbcon=rotate:3
}
menuentry "old_kernel" {
set root=(hd0,1)
linux /boot/kernel_old root=/dev/sda1 rootflags=commit=60 hpet=force
}
menuentry "memtest" {
set root=(hd0,1)
linux /boot/memtest86plus/memtest.bin
}
|
More info here: http://www.gnu.org/software/grub/grub-2.en.html _________________ Myself and mine gymnastic ever |
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 3003 Location: Bay Area, CA
|
Posted: Tue Oct 28, 2008 3:44 am Post subject: |
|
|
is there a real world comparison of ext3 and ext4 somewhere? what I am interested in is:
1. large number of small files. How well does ext4 handle this case? space consumption and speed wise. This is one case where both speed can be increased and space consumption reduced like reiser4 does
.
2. small number of large files. e.g. how many fragments does the ext4 create for a 4GB file on a 70% used file system of 30GB? how fast is the delete of large files? Its slow on ext3/ext2.
3. there is no online de-fragmentation tool. Please don't flame me with "linux FSs don't fragment". Anybody who has created multiple virtual machines and have a full enough FS, knows that all FSs fragment.
These are the things that bother me on ext3. Does ext4 help with any of these? |
|
Back to top |
|
|
wrc1944 Advocate
Joined: 15 Aug 2002 Posts: 3456 Location: Gainesville, Florida
|
|
Back to top |
|
|
octoploid n00b
Joined: 21 Oct 2006 Posts: 65
|
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 3003 Location: Bay Area, CA
|
Posted: Thu Oct 30, 2008 7:05 am Post subject: |
|
|
So, I tested ext4dev in 2.6.27.4. Stay away from it. Its worse than ext3 in every sense. Large files have many many (41 against 770) more fragments compared to ext3 (so, what's up with extents?). Seq write b/w drops 20%. And pdflush locks up cpu during large (5GB) file copy and kjournald2 goes into 'D' state, requiring a reboot. About the only thing going for it was that the delete of large files was much quicker than ext3. The tests were done on the same physical hardware and same partition, on an amd64 system. |
|
Back to top |
|
|
lucke n00b
Joined: 11 May 2005 Posts: 35
|
Posted: Thu Oct 30, 2008 10:11 am Post subject: |
|
|
They apparently fixed that bug with excessive number of extents. It's comparable to xfs now in this regard. All in all, it's dandy on .28. |
|
Back to top |
|
|
octoploid n00b
Joined: 21 Oct 2006 Posts: 65
|
Posted: Thu Oct 30, 2008 10:33 am Post subject: |
|
|
devsk wrote: | So, I tested ext4dev in 2.6.27.4. Stay away from it. Its worse than ext3 in every sense. Large files have many many (41 against 770) more fragments compared to ext3 (so, what's up with extents?). Seq write b/w drops 20%. And pdflush locks up cpu during large (5GB) file copy and kjournald2 goes into 'D' state, requiring a reboot. About the only thing going for it was that the delete of large files was much quicker than ext3. The tests were done on the same physical hardware and same partition, on an amd64 system. |
Hmm, strange. I don't observe any of these symptoms on my machine (amd64).
I'm running 2.6.28-rc2 and started with fresh filesystems ("mkfs.ext4") to which
I copied all my data from ext3 backups, including my ~250GB music collection.
fsck.ext4 is dramatically faster than the ext3 version. Firing up my music player (cmus)
now only takes a second (reading the entire directory structure of my collection); on ext3
I had to wait 10-15 sec before I could use it. update-eix is blazingly fast.
Granted, these may all be effects of a freshly populated file system, but I definitely don't
see any downsides compared to ext3. _________________ Myself and mine gymnastic ever |
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 3003 Location: Bay Area, CA
|
Posted: Thu Oct 30, 2008 3:33 pm Post subject: |
|
|
may be the difference comes between ext4dev of 2.6.27.4 and ext4 of 2.6.28. But my experiments on the same hardware and partition under the same OS, same tools, same everything, paints ext4dev in bad shape. I will wait for 2.6.28 and see if it changes. |
|
Back to top |
|
|
asturm Developer
Joined: 05 Apr 2007 Posts: 9280
|
Posted: Thu Oct 30, 2008 8:24 pm Post subject: |
|
|
That's because ext4dev != ext4 |
|
Back to top |
|
|
kernelOfTruth Watchman
Joined: 20 Dec 2005 Posts: 6111 Location: Vienna, Austria; Germany; hello world :)
|
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 3003 Location: Bay Area, CA
|
Posted: Thu Oct 30, 2008 11:32 pm Post subject: |
|
|
genstorm wrote: | That's because ext4dev != ext4 | do you have data to support that? I doubt they put in so much of speed between 2.6.27.4 and 2.6.28-rc2. |
|
Back to top |
|
|
agent_jdh Veteran
Joined: 08 Aug 2002 Posts: 1783 Location: Scotland
|
Posted: Thu Oct 30, 2008 11:58 pm Post subject: |
|
|
devsk wrote: | genstorm wrote: | That's because ext4dev != ext4 | do you have data to support that? I doubt they put in so much of speed between 2.6.27.4 and 2.6.28-rc2. |
When you make menuconfig your kernel and you see ext4dev, it's not at the latest level. There's a link to the kernel.org repository for the latest level of patches for ext4 with 2.6.27 on the first page of this topic. People using 2.6.28-rc kernels already have them. Once the patches are applied, when you make menuconfig, ext4dev is gone, and has been replaced by simply ext4 (albeit with an option for compatability with ext4dev partitions).
This would probably fix the performance issues you're seeing. Although YMMV. _________________ Jingle Jangle Jewellery |
|
Back to top |
|
|
Schwinni Apprentice
Joined: 02 Sep 2004 Posts: 214 Location: quadrant1.earth. germany.wuerzburg
|
Posted: Fri Oct 31, 2008 9:20 am Post subject: |
|
|
But be careful:
2.6.27-ext4-2 cannot patch "ext4/dir.c" because this file itself was patched from 2.6.27.3 to 2-6-27-4.
I didn't have the time to look at the rejected hunk yesterday.
The safest way is to wait for the next patch set (2.6.27-ext4-3).
devsk wrote: | genstorm wrote: | That's because ext4dev != ext4 | do you have data to support that? I doubt they put in so much of speed between 2.6.27.4 and 2.6.28-rc2. |
These are the broken out patches from 2.6.27-ext4-2. They do more than just renaming extdev to ext4:
http://www2.kernel.org/pub/linux/kernel/people/tytso/ext4-patches/2.6.27-ext4-2/broken-out/
Best regards
Chris |
|
Back to top |
|
|
Schwinni Apprentice
Joined: 02 Sep 2004 Posts: 214 Location: quadrant1.earth. germany.wuerzburg
|
Posted: Fri Oct 31, 2008 1:30 pm Post subject: |
|
|
Seems like the failing hunk is included both in the kernel patch set for 2.6.27.4 and in 2.6.27-ext4-2:
From patch-2.6.27.4:
Code: | diff --git a/fs/ext4/dir.c b/fs/ext4/dir.c
index ec8e33b..d1d6487 100644
--- a/fs/ext4/dir.c
+++ b/fs/ext4/dir.c
@@ -102,6 +102,7 @@ static int ext4_readdir(struct file * filp,
int err;
struct inode *inode = filp->f_path.dentry->d_inode;
int ret = 0;
+ int dir_has_error = 0;
sb = inode->i_sb;
@@ -148,9 +149,13 @@ static int ext4_readdir(struct file * filp,
* of recovering data when there's a bad sector
*/
if (!bh) {
- ext4_error (sb, "ext4_readdir",
- "directory #%lu contains a hole at offset %lu",
- inode->i_ino, (unsigned long)filp->f_pos);
+ if (!dir_has_error) {
+ ext4_error(sb, __func__, "directory #%lu "
+ "contains a hole at offset %Lu",
+ inode->i_ino,
+ (unsigned long long) filp->f_pos);
+ dir_has_error = 1;
+ }
/* corrupt size? Maybe no more blocks to read */
if (filp->f_pos > inode->i_blocks << 9)
break; |
From 2.6.27-ext4-2:
Code: | diff --git a/fs/ext4/dir.c b/fs/ext4/dir.c
index ec8e33b..3ca6a2b 100644
--- a/fs/ext4/dir.c
+++ b/fs/ext4/dir.c
.
.
.
@@ -102,6 +102,7 @@ static int ext4_readdir(struct file * filp,
int err;
struct inode *inode = filp->f_path.dentry->d_inode;
int ret = 0;
+ int dir_has_error = 0;
sb = inode->i_sb;
@@ -148,9 +149,13 @@ static int ext4_readdir(struct file * filp,
* of recovering data when there's a bad sector
*/
if (!bh) {
- ext4_error (sb, "ext4_readdir",
- "directory #%lu contains a hole at offset %lu",
- inode->i_ino, (unsigned long)filp->f_pos);
+ if (!dir_has_error) {
+ ext4_error(sb, __func__, "directory #%lu "
+ "contains a hole at offset %Lu",
+ inode->i_ino,
+ (unsigned long long) filp->f_pos);
+ dir_has_error = 1;
+ }
/* corrupt size? Maybe no more blocks to read */
if (filp->f_pos > inode->i_blocks << 9)
break;
.
.
. |
|
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 3003 Location: Bay Area, CA
|
Posted: Sat Nov 01, 2008 12:24 am Post subject: |
|
|
Schwinni wrote: | But be careful:
2.6.27-ext4-2 cannot patch "ext4/dir.c" because this file itself was patched from 2.6.27.3 to 2-6-27-4.
I didn't have the time to look at the rejected hunk yesterday.
The safest way is to wait for the next patch set (2.6.27-ext4-3).
devsk wrote: | genstorm wrote: | That's because ext4dev != ext4 | do you have data to support that? I doubt they put in so much of speed between 2.6.27.4 and 2.6.28-rc2. |
These are the broken out patches from 2.6.27-ext4-2. They do more than just renaming extdev to ext4:
http://www2.kernel.org/pub/linux/kernel/people/tytso/ext4-patches/2.6.27-ext4-2/broken-out/
Best regards
Chris | Have all these patches been accepted in 2.6.28-rc2? |
|
Back to top |
|
|
Schwinni Apprentice
Joined: 02 Sep 2004 Posts: 214 Location: quadrant1.earth. germany.wuerzburg
|
Posted: Sat Nov 01, 2008 12:55 am Post subject: |
|
|
I don't know. At least on http://ext4.wiki.kernel.org/index.php/Ext4_patchsets it says Quote: | Periodically, Ted will take the ext4 patch queue and create a stable patchset for people to apply. |
Because these patch sets doesn't contain all tests from the git tree but only the stable things, I tend to say yes, probably they are accepted in 2.6.28.
But to make sure you'd have to ask that on the ext4 mailing list. |
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 3003 Location: Bay Area, CA
|
Posted: Sat Nov 01, 2008 1:35 am Post subject: |
|
|
I got this reject while applying the patch:
Code: | fs/ext4/dir.c.rej
***************
*** 102,107 ****
int err;
struct inode *inode = filp->f_path.dentry->d_inode;
int ret = 0;
sb = inode->i_sb;
--- 102,108 ----
int err;
struct inode *inode = filp->f_path.dentry->d_inode;
int ret = 0;
+ int dir_has_error = 0;
sb = inode->i_sb;
***************
*** 148,156 ****
* of recovering data when there's a bad sector
*/
if (!bh) {
- ext4_error (sb, "ext4_readdir",
- "directory #%lu contains a hole at offset %lu",
- inode->i_ino, (unsigned long)filp->f_pos);
/* corrupt size? Maybe no more blocks to read */
if (filp->f_pos > inode->i_blocks << 9)
break;
--- 149,161 ----
* of recovering data when there's a bad sector
*/
if (!bh) {
+ if (!dir_has_error) {
+ ext4_error(sb, __func__, "directory #%lu "
+ "contains a hole at offset %Lu",
+ inode->i_ino,
+ (unsigned long long) filp->f_pos);
+ dir_has_error = 1;
+ }
/* corrupt size? Maybe no more blocks to read */
if (filp->f_pos > inode->i_blocks << 9)
break;
|
As per Schwinni's post above, I think this is fine. so, I should be able to continue, right? |
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 3003 Location: Bay Area, CA
|
Posted: Sat Nov 01, 2008 3:15 am Post subject: |
|
|
OK, just booted with the patch and the tests on the same physical partition are much more favorable now. Number of fragments went down substantially. Sequential write/read b/w on large files went up beyond ext3 by 20%, I think mostly because of reduced number of fragments.
Had no effect on portage tree compared to ext2: same results, give or take few millisecs.
I can live with that.
Does anybody know anything about this error from bonnie++?
Code: | Delete files in sequential order...Can't delete file 0004996uWK9H6o
Cleaning up test directory after error.
| It doesn't happen on ext[23]. |
|
Back to top |
|
|
octoploid n00b
Joined: 21 Oct 2006 Posts: 65
|
Posted: Sat Nov 01, 2008 5:54 am Post subject: |
|
|
devsk wrote: | OK, just booted with the patch and the tests on the same physical partition are much more favorable now. Number of fragments went down substantially. Sequential write/read b/w on large files went up beyond ext3 by 20%, I think mostly because of reduced number of fragments.
Had no effect on portage tree compared to ext2: same results, give or take few millisecs.
I can live with that.
Does anybody know anything about this error from bonnie++?
Code: | Delete files in sequential order...Can't delete file 0004996uWK9H6o
Cleaning up test directory after error.
| It doesn't happen on ext[23]. |
Yes, this bug is already fixed in 2.6.28-rc2:
http://git.kernel.org/?p=linux/kernel/git/tytso/ext4.git;a=commitdiff;h=3c37fc86d20fe35be656f070997d62f75c2e4874 _________________ Myself and mine gymnastic ever |
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 3003 Location: Bay Area, CA
|
Posted: Sat Nov 01, 2008 6:10 am Post subject: |
|
|
doesn't apply on top of the other patch ext4-2. Did someone generate that patch against ext4-2?
edit: it applied with "patch -p1 -l". Cut&paste mangled the tabs.
Last edited by devsk on Sat Nov 01, 2008 4:33 pm; edited 1 time in total |
|
Back to top |
|
|
fldc2 Tux's lil' helper
Joined: 14 Jun 2006 Posts: 112 Location: Motala, Sweden
|
Posted: Sat Nov 01, 2008 12:19 pm Post subject: |
|
|
The only problem i've had with ext4 so far is that the torrents i download get lots of failed chunks for some reason, no matter what client is used, my workaround at this time is to download on a partition with a different filesystem |
|
Back to top |
|
|
devsk Advocate
Joined: 24 Oct 2003 Posts: 3003 Location: Bay Area, CA
|
Posted: Sat Nov 01, 2008 4:39 pm Post subject: |
|
|
ext4 is really shining with large files. I used to get upto 3000 fragments for a 650MB VM vmem file after repeated suspend/restore cycles. It has got stuck with 2 so far, which means suspend/resume of VM takes just 5-10 seconds, instead of worst cases 30-60 seconds. I just hope it doesn't oops on me.... |
|
Back to top |
|
|
|