Visit Our UNIX and Linux User Community


Gfs2 vs xfs vs ext4


 
Thread Tools Search this Thread
Special Forums Hardware Filesystems, Disks and Memory Gfs2 vs xfs vs ext4
# 8  
Old 11-19-2009
Quote:
Originally Posted by king_hippo
1. What type of storage are you using.
Penguin x8dtn, 24 1tb disks
Can find anything about this setup. Do you have specs/a link ?

Quote:
2. What type of disks
sata
Definitely not optimal for the amount of files or the access patterns you describe.

Quote:
3. If using a controller based array, what you optimized for sequential or random operation.
Sounds like it should be random optimized, but depends on the array or RAID controller and its capabilities whether or not this a selectable.

Quote:
4. Will you mirror on-host or using hardware RAID
controller based raid 10. six disks per raid set

Seems like a reasonable choice.


Quote:
4. What is your storage block size.
Quote:
64k
Quote:
5. What is the average size of your writes
Quote:
dunno, guess is a few meg. No more than 16m. These are email attachments
That is the size of the objects not the size of the writes. iostats for the file systems would be good here.

Quote:
6. What is the breakdown of reads/writes
not sure
This might be essential information, depends on the pattern, but caching correctly might improve performance drastically.

Quote:
9. How many files per directory do you have ( files includes sub-directories) in the largest directory.
Quote:
millions
This is in all probability your main bottleneck, more file systems or directories with fewer files could improve this.
# 9  
Old 11-19-2009
I was told they did have it turned on. Basically at the time we started this layout we were the largest email system in the world (still are I think). The only we we had to handle the i/o load we were getting was small files and basically deep random directories. reiser was the only file system that could handle it. Red Hat assigned about 5 engineers to work with us and couldn't get ext close. They built us a custom gf2 after numerous attempts. It is now in the public release, I forget which ver.
# 10  
Old 11-20-2009
Quote:
Originally Posted by king_hippo
I was told they did have it turned on.
We've been screaming at you to tune it for days... If it was tuned in the first place, you might have said so! We're only trying to help you here.

If ext3 isn't fast enough for you, I don't think ext4 is going to be either. But performance in whatever filesystem you choose could be improved a lot by not having millions of files in one directory.
# 11  
Old 11-21-2009
You may want to think about some sort of higher end LVM/filesystem configuration.

Simply changing filesystems without putting extra intelligence and disk I/O bandwidth between the OS and directory access is not going to help much.
# 12  
Old 11-23-2009
Quote:
Originally Posted by jim mcnamara
You may want to think about some sort of higher end LVM/filesystem configuration.

Simply changing filesystems without putting extra intelligence and disk I/O bandwidth between the OS and directory access is not going to help much.
I don't think the OP has posted any hard numbers on the I/O performance he is getting, nor exact specifics of his RAID setup (connection type, RAID blocksize, etc.) Without all that it's hard to solve his problem.
# 13  
Old 11-30-2009
Quote:
Originally Posted by jim mcnamara
You may want to think about some sort of higher end LVM/filesystem configuration.

Simply changing filesystems without putting extra intelligence and disk I/O bandwidth between the OS and directory access is not going to help much.
Can you point me to a resource that I can read more about high level filesystem config or putting extra intelligence and disk I/O bandwidth between the OS and directory access?

Previous Thread | Next Thread
Test Your Knowledge in Computers #458
Difficulty: Medium
NTP uses a flat, peer-to-peer system of time sources.
True or False?

8 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

VMWare,XFS and iSCSI issues!

I have a RHEL6 VM that requires the use of remote storage using iSCSI and XFS for the mount point. Here's the issue: With XFS you can't use the _netdev option for your mount point (pause for network) so my mount point doesn't mount properly because the network isn't up yet. I've moved the... (3 Replies)
Discussion started by: ksfolsom
3 Replies

2. Red Hat

Convert ext4 to ext3

Is there any way to conver ext4 to ext3 filesystem without formatting the partition/disk .. Had ext3 filesystem and had converted it to ext4 by issuing following command # tune2fs -O extents,uninit_bg,dir_index /dev/sda1 # fsck -pf /dev/sda1 # blkid /dev/sda1 /dev/sda1:... (1 Reply)
Discussion started by: Shirishlnx
1 Replies

3. Red Hat

XFS - Custom Kernel or Module?

Hey everyone. I am going to be using XFS for a project coming up. We're running RHEL 5.5. Simply typing modprobe xfs works just fine. The kernel module loads without any issue. Is there any issue with doing this and inserting "modprobe xfs" into /etc/rc.modules? Is there a major reason to... (0 Replies)
Discussion started by: msarro
0 Replies

4. SuSE

iscsi network share + xfs

Hello, we got a MD3000i used as a network share between two servers (say A and B). The problem we are facing is that file/directories created by server A are not visible on server B (and viceversa). It's not a problem with permission (chmod 777 doesn't help). The MD3000i was at first used ony... (0 Replies)
Discussion started by: neutrino
0 Replies

5. Linux

GFS2 needed or not?

Hey everyone. I am in the process of setting up an iSCSI SAN to function as a log storage device for a number of servers. All of the initiators see the volume, and originally I formatted it with ext3 and went on my merry way. However after some research I'm having concerns that I should nuke the... (1 Reply)
Discussion started by: msarro
1 Replies

6. UNIX for Advanced & Expert Users

[XFS] How to use real-time subvolume

Hi! I created filesystem XFS on partition hda8 with subvolume real-time on partition hda5: mkfs.xfs -r rtdev=/dev/hda5 /dev/hda8 and i mounted it: mount -t xfs -o rtdev=/dev/hda5 /dev/hda8 /xfs But I don't know how can I use this partition hda5 with subvolume real-time. I don't know how to... (3 Replies)
Discussion started by: Cadi2108
3 Replies

7. Filesystems, Disks and Memory

ext4 - ready for production system?

Gidday, Are you using ext4 for production system? Or is it better to opt for a more conservative strategy, like ext3 for instance? What are your experiences? Thanks in advance, Loc. (3 Replies)
Discussion started by: Loic Domaigne
3 Replies

8. Solaris

Does Solaris support XFS filesytem?

Hi, Does solaris support the XFS file sytems? If so, how can I mount a shared directory from another machine (which is using XFS)? thanks (3 Replies)
Discussion started by: orahi001
3 Replies

Featured Tech Videos