Solaris

Can you elaborate on that point?
I would not expect the presence of snapshots to have a significant effect on the number of I/Os.

more snapshots = more writing to the delta log which contributes to greater I/O. if you have complex zfs systems it's exponental. snapshot -r /rpool will snapshot every subordinate fs and will then cause any changes to have to be written to each snapshot. 50 subs, 10 snapshots you get the idea...

I'm afraid I don't get it. Snapshots are read-only by design so they cannot be the target of write operations. On the other hand creating them can have a small overhead and destroying them might have a bigger overhead. The latter is to be balanced with the fact having snapshots reduce the number of I/Os in case of file removal, as the data blocks, being still referenced by the snapshot(s) need not to be marked as free.

while admittedly I don't know the specifics of how it works I do know that a zfs snapshot is a delta value of the FS. so it must be recording those deltas somewhere. The older the snap the larger the file and the more snaps the more writes.

I can only tell you from practical experience that removing snapshots DOES improve performance.

The delta is not written. The data already exists in the original filesystem.

For instance, you have 4 files sized 20 GB on a zfs filesystem inside a zpool sized 100 GB.
Zpool current space utilization is 80%
For the sake of argument, we have only one filesystem in that zpool.

A snapshot has been made on that zfs filesystem.
You delete 1 of 4 files sized 20 GB.

Zpool will remain on 80%, since the snapshot is referencing on the deleted data, the data is not actually deleted from the zpool.

You issue zfs destroy on the snapshot. This operation actually deletes data from the zpool.

This is how i understand it, feel free to correct me

As for ARC :

Problem is it doesn't work well if a program requests a very large memory chunk (such as Oracle database), since it will request memory, if it is not given in certain time it will start swapping.

This is why i avoid zfs filesystems in general for Oracle database and use ASM with limited ZFS arc.
Take a look at the documentation regarding ZFS and databases. It requires a lot of love and attention.

I'd rather give that love to something else and run ASM
Chip in a good SSD or a local flash cache card as a CACHE device for Oracle, pin a couple of monster indexes in it and go get some beer

On the other hand, on several TB Solaris Cluster with ZFS which is a NFS server, i haven't touched that tunable.
Machines work fine with 95% memory consumed, mostly by filesystems, using the memory as ARC (which is desired).

This is incorrect, a snapshot is a frozen dataset content. What you call the delta value is written once on the live file system. The snapshots delta is already there, no need to record it. Yes, if the file system is evolving.
No, there is no write inflation.
Perhaps had you rolling snapshots in place?

Hi Jllagre,Peasant,Don

Any idea on how to solve my issue regarding DB hang when backup(RMAN)? I already give sar -d output earlier. How to fine tune the zfs parameter especially the arc_max?
Is that change the parameter will cause data loss? Please help.