Linux Filesystems Comparison: ext2, ext3, xfs, Reiserfs

Copyright(©); 2001 Jim Bray under the terms of the GNU Free Documentation License:
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation.

Update 12/25/01

I've been running mostly xfs for a while now, currently in a 2.4.17-xfs kernel. I've had no problems with it. I've turned my second machine into an ext3 machine, and it runs fine. Ext3 mounts about as quickly as xfs, checks a little slower, and maybe runs a little slower, but not annoyingly. I haven't yet done a proper comparison; I'll probably wait until ext3 is considered released, since xfs is now quite mature and ext3 just hit the stock kernels. There is no question that xfs eats a lot more core than ext3 (see below). An xfs developer told me once that xfs, while perfectly OK for desktop use, was really intended for server use, so hopefully all this extra code means it scales better than ext3.

Update note: I recently upped my ram in the xfs box from 192MB to 320, then 450 MB. There was a noticeable speedup in some xfs operations. It is clearly grabbing a good chunk of ram and doing efficient caching with it.

Kernel Size Compare, ext3 vs. xfs

Other than the wired-in filesystem, the kernels below are identical configurations.

I use the GRUB bootloader. Until recently it didn't understand xfs, so I had to have an ext2 or ext3 /boot partition, but now it reads xfs so this is no longer necessary. Having a separate /boot has an advantage in that it can be left unmounted most of the time, making it a little harder for problems to make the system completely dead in the water, so I'm sticking with this setup.

I don't expect to ever try Reiserfs again. It was a slow nuisance when I did, and I was really annoyed by the way its documentation and even kernel config help hyped itself and asked for money (still does, as of 12/25/01):

If you like it, you can pay us to add new features to it that you need, buy a support contract, or pay us to port it to another OS.
Somehow this is of a piece with it being named after a person. Fortunately the most excellent Steven Tweedie didn't saddle us with tweediefs, and the SGI folks chose xfs instead of sgifs.

Older notes, dealing with Reiserfs

This is quite crude and preliminary at this point. Some scribbled notes, some timings of commands. Hopefully it will improve. Suggestions welcome. If you have a test you'd like run, drop me a line and I'll consider it. If there appears to be any interest in this, I'll work on it more.

I've got one kernel with Reiserfs, and another with xfs. That is what I'm mainly running and using: linux-2.4.1-XFS, built from the SGI Linux tree checked out via CVS. One has to wonder a lot of things about Linux decisions, such as why, when CVS is the obvious Right Way and used by everything else, Official Linux is still done with patches. I'd frankly like to see other versions of the kernel, such as the SGI version, become popular to stir things up a bit.

I've done some crude tests, using things like time tar -cf - /xfs1|cat >/dev/null. So far my crude tests don't show any overwhelming speed advantage for ext2, Reiserfs [Hereinafter Rfs], or xfs. The xfs commands are better than the Rfs commands, and make.Rfs apparently can't build a small filesystem (not under about 100MB, anyway), which limits its usefulness for me. I haven't tried putting xfs on a floppy, but I made one in my tiny swap partition without any trouble.

xfs is big. I built modules to see just how big. It needs these three:

-rw-r--r-- 1 root root 49498 Feb 4 01:16 pagebuf.o -rw-r--r-- 1 root root 489727 Feb 4 01:16 xfs.o -rw-r--r-- 1 root root 18095 Feb 4 01:16 xfs_support.o so it comes in at over 500k.

Rfs is smaller:

-rw-r--r-- 1 root root 183474 Feb 3 01:51 reiserfs.o scribbles: reiserfs: slow mkfs ? slow mount ? slow check, very annoying Pedantry, demanding 'Yes', etc. slow find slow ls -R won't make 80MB filesystem. xfs: builds no problem (but kgcc) cmds build no problem very fast makefs instant check

In the notes below, 'mozilla' refers to the mozilla source-tree, 'sgi' refers to the SGI Linux (and associated commands) tree, and 'mp3' refers to my MP3 collection)

**************************************************** ext2: gilgamesh:/big2[2]#time du -s mozilla 399700 mozilla real 0m26.617s user 0m0.450s sys 0m2.110s ----------------------------- xfs: gilgamesh:/xfs1[2]#time du -s mozilla 380860 mozilla real 0m25.888s user 0m2.110s sys 0m6.290s ----------------------------- Rfs: 1: gilgamesh:/reiser1[2]time du -s mozilla 379447 mozilla real 0m14.371s user 0m0.410s sys 0m8.550s gilgamesh:/reiser1[2] 2: gilgamesh:/reiser1[2]time du -s mozilla 379447 mozilla real 0m0.936s user 0m0.260s sys 0m0.530s 1: gilgamesh:/reiser1[2]time du -s sgi 177729 sgi real 0m7.213s user 0m0.220s sys 0m3.580s gilgamesh:/reiser1[2] 2: gilgamesh:/reiser1[2]time du -s sgi 177729 sgi real 0m0.267s user 0m0.080s sys 0m0.150s **************************************************** in cache: ext2: gilgamesh:/src[1]time du -s sgi 183728 sgi real 0m0.246s ext2 quicker here user 0m0.080s sys 0m0.130s xfs: gilgamesh:/xfs1[1]time du -s sgi 178200 sgi real 0m0.973s user 0m0.440s sys 0m0.350s **************************************************** sgi tar: ext2: 1: gilgamesh:/src[2]time tar -cf - sgi|cat >/dev/null real 0m38.523s user 0m1.450s sys 0m7.440s 2: gilgamesh:/src[2]time tar -cf - sgi|cat >/dev/null real 0m35.557s user 0m1.460s sys 0m6.520s ----------------------------- xfs: 1: gilgamesh:/xfs1[1]time tar -cf - sgi|cat >/dev/null real 0m58.774s user 0m1.960s sys 0m9.290s 2: gilgamesh:/xfs1[1]time tar -cf - sgi|cat >/dev/null real 0m57.802s user 0m2.000s sys 0m7.880s ----------------------------- Rfs: 1: gilgamesh:/reiser1[2]time tar -cf - sgi|cat >/dev/null real 0m56.035s user 0m1.380s sys 0m9.030s 2: [second time much longer!] gilgamesh:/reiser1[2]time tar -cf - sgi|cat >/dev/null real 1m0.395s user 0m1.240s sys 0m9.420s 3: gilgamesh:/reiser1[2]time tar -cf - sgi|cat >/dev/null real 1m0.026s user 0m1.420s sys 0m8.730s **************************************************** mp3 tar: ----------------------------- ext2: 1: gilgamesh:/big1[2]time tar -cf - mp3|cat >/dev/null real 0m44.996s user 0m0.940s sys 0m16.410s 2: gilgamesh:/big1[2]time tar -cf - mp3|cat >/dev/null real 0m41.829s user 0m0.740s sys 0m12.930s ----------------------------- Rfs: 1: gilgamesh:/reiser1[2]time tar -cf - mp3|cat >/dev/null real 0m37.501s user 0m0.630s sys 0m15.180s gilgamesh:/reiser1[2] 2: gilgamesh:/reiser1[2]time tar -cf - mp3|cat >/dev/null real 0m36.016s user 0m1.100s sys 0m15.580s **************************************************** mozilla tar: ----------------------------- ext2: 1: gilgamesh:/big2[2]time tar -cf - mozilla|cat >/dev/null real 2m55.327s user 0m3.990s sys 0m19.080s 2: gilgamesh:/big2[2]time tar -cf - mozilla|cat >/dev/null real 3m6.853s user 0m3.740s sys 0m18.400s ----------------------------- Rfs: 1: gilgamesh:/reiser1[2]time tar -cf - mozilla|cat >/dev/null real 2m9.134s user 0m3.930s sys 0m25.660s gilgamesh:/reiser1[2] 2: gilgamesh:/reiser1[2]time tar -cf - mozilla|cat >/dev/null real 2m5.753s user 0m3.660s sys 0m26.370s **************************************************** Rfs copy: gilgamesh:/big1[3]time cp -a mp3 /reiser1 real 1m45.615s user 0m0.710s sys 0m29.660s gilgamesh:/src[2]time cp -a sgi /reiser1/ real 1m31.105s user 0m1.160s sys 0m27.790s gilgamesh:/src[2] **************************************************** ls -lR mozilla: ----------------------------- ext2: 1: [not in cache] gilgamesh:/big2[1]time ls -lR mozilla >/dev/null real 0m39.801s user 0m5.180s sys 0m2.430s 2: gilgamesh:/big2[1]time ls -lR mozilla >/dev/null real 0m9.435s user 0m4.970s sys 0m2.960s 3: gilgamesh:/big2[1]time ls -lR mozilla >/dev/null real 0m6.927s user 0m5.110s sys 0m0.820s ------------------------- Rfs: 1: gilgamesh:/reiser1[1]time ls -lR mozilla >/dev/null real 0m18.101s user 0m5.470s sys 0m8.570s 2: gilgamesh:/reiser1[1]time ls -lR mozilla >/dev/null real 0m7.109s user 0m5.020s sys 0m1.050s 3: gilgamesh:/reiser1[1]time ls -lR mozilla >/dev/null real 0m7.518s user 0m5.280s sys 0m1.160s ****************************************************