Solaris

Well, the 4GB ARC is now full but while uncapped, it used to contain three times more data (12.9GB).

It is not possible to know how much this cache size reduction does affect performance without getting at least hits and miss statistics but if there is a large increase in misses, the performance impact might be huge.

Most of the ZFS ARC on Oracle DB servers is log file data that will never be read back. It'll just sit in the ARC until something else needs the RAM - there's no point in caching that data. I bet the cache hit rate is still > 99%.

Brendan's blog » Activity of the ZFS ARC

Anything over 95% doesn't really improve performance.

I'm thinking however the OP is doing backups is causing ZFS to block IO for a while, thus the DB aio_wait() calls time out. Maybe the backup process used is to take a snapshot of the ZFS file system will the DB is active, and then use ZFS send on the snapshot? (Although that really won't produce a viable backup of an Oracle DB....)

As I posted earlier, if you want to use the memory to speed up an Oracle DB, use it as the DB's SGA.

Yes, the closer the cache is to the DB, the better.

Hi achenlejlliagre,
We are using RMAN backup. The DB is LDOM which we hosted in CDOM host. Futher check find out that, even LDOM level no io queue but saw io queue at CDOM hostserver while backup running. Is that normal

Please show the iostat output on control domain for specified disks when the backup is running ?

Code:

/usr/bin/iostat -xcnzCTd 3 10

Adjust the frequency (10) to your needs (run it when the problem appears)

Regards
Peasant.