Login or Register to Ask a Question and Join Our Community


Solaris

Help with ZFS arc cache

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Solaris Help with ZFS arc cache
# 1  
Old 12-13-2010
Help with ZFS arc cache
Greetings Forumers!

I have a Solaris 10u9 M5000 with 32GB RAM and have noticed the ZFS arc cache is consuming large amount of memory. Here's what I see on my system:
Code:
# echo ::memstat|mdb -k
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                     534548              4176   13%
ZFS File Data             2898766             22646   70%
Anon                       253850              1983    6%
Exec and libs               23498               183    1%
Page cache                  22613               176    1%
Free (cachelist)            33639               262    1%
Free (freelist)            350437              2737    9%

Total                     4117351             32166
Physical                  4095740             31997

How can I limit this excessive use of memory without impacting the zones that run on this system?


Thanks is Advance!
# 2  
Old 12-13-2010
Correct me where I missed the point - the system allocates 'unused' memory which it gives up when needed. You still have lots of free pages. This an m4000:
Code:
# echo ::memstat |mdb -k
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                     398633              3114    5%
Anon                      1074821              8397   13%
Exec and libs               58856               459    1%
Page cache                6013479             46980   73%
Free (cachelist)           229681              1794    3%
Free (freelist)            459750              3591    6%

Total                     8235220             64337
Physical                  8213603             64168

Is page cache is out of control? No. The system is doing to this m4000 exactly what you see on your box. Using idle memory to good advantage.

Are you experiecing problems? Really sluggish performance, lots of swapping?

---------- Post updated at 08:48 ---------- Previous update was at 08:42 ----------

Here found it - the Evil Tuning Guide

ZFS Evil Tuning Guide - Siwiki

Read the arc cache section.
This User Gave Thanks to jim mcnamara For This Post:
bluescreen
# 3  
Old 12-13-2010
As far as I know limiting the zfs cache could make sense because of the overhead associated with the dynamic resizing.

Oracle/Sun consultants recommended the following settings in /etc/system on our database servers with 64G of physical memory:

Code:
* ZFS tuning
set zfs:zfs_immediate_write_sz=8000
set zfs:metaslab_df_free_pct=4
set zfs:zfs_nocacheflush=1
set zfs:zfs_vdev_max_pending=32
* limit zfs 16G
set zfs:zfs_arc_max=17179869184

This User Gave Thanks to radoulov For This Post:
bluescreen
# 4  
Old 12-13-2010
Quote:
Originally Posted by jim mcnamara
Are you experiecing problems? Really sluggish performance, lots of swapping?
Actually no. However, I have created other zones on ZFS pools. These zones run Oracle software that looks for available RAM before starting. If there is no RAM available, the software pukes out an error msg and dies.

I also found this link about limiting the amount of ZFS ARC in memory.

I'm just looking for ways to get these zones up and running while managing memory effectively. I'm also new at using ZFS and zones so I'm going through a bit of a learning curve too. Smilie

Any help is greatly appreciated.
# 5  
Old 12-13-2010
There are additional notes on MOS about this.
# 6  
Old 12-13-2010
Quote:
Originally Posted by radoulov
As far as I know limiting the zfs cache could make sense because of the overhead associated with the dynamic resizing.

Oracle/Sun consultants recommended the following settings in /etc/system on our database servers with 64G of physical memory:

Code:
* ZFS tuning
set zfs:zfs_immediate_write_sz=8000
set zfs:metaslab_df_free_pct=4
set zfs:zfs_nocacheflush=1
set zfs:zfs_vdev_max_pending=32
* limit zfs 16G
set zfs:zfs_arc_max=17179869184

I was think about setting this limit (16G) on my M5000.
# 7  
Old 12-13-2010
You could also check ZFS Best Practices Guide.
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Check zfs cache usage for Solaris 11.3

Hi All, How do we calculate the zfs cache value from Solaris 11.3 ?? I normally use memstat but learnt that is not a good practice. Any other way to get the value ? (2 Replies)
Discussion started by: jayadanabalan
2 Replies

2. Solaris

ZFS : Can arc size value exceed Physical RAM ?

Hi, kstat -p -m zfs -n arcstats -s size returns zfs:0:arcstats:size 8177310584 this values is approx (7.61 GB) but my Physical Memory size is only 6144 Megabytes. Can this happen ? if yes, then how can I find free memory on the system. BTW, I ran the kstat commands from a Non... (2 Replies)
Discussion started by: sapre_amit
2 Replies

3. Linux

File cache /Page cache Linux

Hi All, could any one point out any open source test-suites for "File cache" testing and as well as performance test suites for the same. Currently my system is up with Linux/ext4. Regards Manish (0 Replies)
Discussion started by: hmanish
0 Replies

4. Solaris

Zfs::zpool.cache file

Hi All, I am trying to read zpool.cache file to find out pool information like pool name, devices it uses and all properties. File seems to be in packed format.I am not sure how to unpack it. But from opensolaris code base we can see that they have used libz for uncompromising this file, but... (0 Replies)
Discussion started by: shailesh_111
0 Replies

5. Solaris

ZFS ARC cache issue

We are having a server running zfs root with 64G RAM and the system has 3 zones running oracle fusion app and zfs cache is using 40G memory as per kstat zfs:0:arcstats:size. and system shows only 5G of memory is free rest is taken by kernel and 2 remaining zones. Now my problem is that... (2 Replies)
Discussion started by: fugitive
2 Replies

6. Linux

getting info on Cache Size, Data Cache etc..

Hi all I saw in Microsoft web site www.SysInternals.com a tool called CoreInfo from able to print out on screen the size of the Data and Instruction caches of your processor, the Locigal to Physical Processor mapping, the number of the CPU sockets. etc.. Do you know if in Linux is available a... (2 Replies)
Discussion started by: manustone
2 Replies

7. Solaris

ARP Cache

Dear all, We are testing two of our servers for mq series connectivity. The scenario is, when one machine is shutting down it's services there are some scripts that do a dns update, which removes the ip address and relates it to the ip address of the other node on our dns server, and the update... (7 Replies)
Discussion started by: earlysame55
7 Replies

8. UNIX for Advanced & Expert Users

UBC cache vs. Metadata cache

hi, What is the difference between UBC cache and Metadata cache ? where can i find UBC cache Hits and Metadata cache Hits in hp-ux? Advanced thanx for the help. (2 Replies)
Discussion started by: sushaga
2 Replies
Login or Register to Ask a Question