Login or Register to Ask a Question and Join Our Community


Solaris

Problem with Swap consumption

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Solaris Problem with Swap consumption
# 1  
Old 03-15-2011
Problem with Swap consumption
Hi Experts,

I have M4000 server with 132 GB Physical memory. 4 sparse zones are running under this server, which are running multiple applications. I am not getting any pointer, where swap space is getting consumed. Almost 97% of swap space is being used. I checked all /tmp (of zones as well), which are almost blank.
Code:
# zfs list | grep -i swap
rpool/swap 4.05G 60.5G 4.05G -
# swap -l
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 256,1 16 8388592 8157216
# swap -s
total: 53587272k bytes allocated + 19471528k reserved = 73058800k used, 1925368k available
# vmstat 2 2
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr lf s0 s1 s2 in sy cs us sy id
2 0 0 4122144 31798056 75 328 0 0 0 0 0 0 87 86 -0 16897 60434 24387 10 6 84
0 0 0 1924984 30637104 28 54 0 0 0 0 0 0 165 165 0 15363 11150 17851 2 3 95
# df -k | grep -i swap
swap 1934264 480 1933784 1% /etc/svc/volatile
swap 524288 15472 508816 3% /tmp
swap 1933920 136 1933784 1% /var/run
# prstat -s size
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
12628 rmdadmin 1999M 941M sleep 29 10 0:01:31 0.0% java/86
17905 rmdadmin 1946M 669M sleep 29 10 2:18:05 0.0% java/84
6141 tomsadmd 1499M 1356M sleep 59 0 9:34:34 0.0% BIService/1622
4148 lsradmin 1427M 1401M sleep 29 10 1:16:38 0.0% BIService/43
7116 35810 1389M 825M sleep 29 10 14:06:38 0.1% java/151
7097 35810 1351M 705M sleep 29 10 12:39:33 0.2% java/149
19028 35810 1335M 1023M sleep 29 10 6:30:59 0.0% java/71
16462 sys11adm 1298M 1087M sleep 59 0 2:24:41 0.0% java/122
11745 mfradmd 1212M 103M sleep 59 0 0:14:28 0.0% java/24
17 sys11adm 1185M 963M sleep 47 4 1:24:11 0.0% java/57
16529 sys11adm 1180M 318M sleep 59 0 0:36:09 0.0% java/39
17863 rmdadmin 1171M 451M sleep 59 0 1:15:28 0.0% java/61
16436 sys11adm 1154M 373M sleep 59 0 11:05:52 0.0% java/66
41 sys11adm 1141M 311M sleep 47 4 0:33:32 0.0% java/39
22658 35810 1132M 823M sleep 29 10 8:17:06 0.0% java/93
6106 tomsadmd 1062M 745M sleep 59 0 2:44:37 0.0% java/1796
21044 apsadmsy 1033M 305M sleep 59 0 76:27:58 0.0% java/199
23037 24943 976M 814M sleep 59 0 3:05:00 0.0% java/75
19257 obieed 965M 641M sleep 47 4 9:32:09 0.0% java/44
14442 24943 930M 838M sleep 59 0 0:19:57 0.0% BIService/21
831 24943 858M 665M sleep 59 0 31:20:25 0.0% java/39
21096 apsadmsy 846M 420M sleep 29 10 4:40:07 0.0% java/79
19029 35810 827M 651M sleep 29 10 3:39:34 0.0% java/45
22659 35810 815M 641M sleep 29 10 3:29:17 0.0% java/44
7947 35810 797M 468M sleep 29 10 1:23:46 0.0% java/39
852 tomsadmd 794M 610M sleep 59 0 6:07:22 0.0% java/77
8884 vdwadmd 793M 587M sleep 59 0 6:31:07 0.0% java/108
7946 35810 793M 459M sleep 29 10 1:22:50 0.0% java/39
12636 lssadmd 792M 516M sleep 59 0 8:33:14 0.0% java/80
6055 weradmd 757M 607M sleep 59 0 2:47:18 0.0% java/90
17324 24943 734M 353M sleep 59 0 2:25:37 0.0% java/42
20217 lssadmd 730M 530M sleep 59 0 42:02:53 0.1% java/104
25202 sys9admd 723M 580M sleep 59 0 1:09:51 0.0% java/144
28867 hpsuadmd 712M 374M sleep 59 0 11:44:01 0.0% java/67
21152 apsadmsy 708M 368M sleep 29 10 3:52:47 0.0% java/65
19764 hpsuadmd 701M 502M sleep 59 0 60:57:07 0.0% java/58
15493 24943 700M 472M sleep 59 0 3:19:35 0.0% java/91
23111 lssadmd 690M 381M sleep 59 0 6:07:23 0.0% java/43
27861 fpradmin 689M 564M sleep 59 0 3:26:04 0.0% java/96
26187 sppsadmd 685M 520M sleep 59 0 3:44:05 0.0% java/69
27870 fpradmin 670M 581M sleep 59 0 3:35:13 0.0% BIService/20
27162 56022 664M 596M sleep 59 0 1:11:02 0.0% java/75
Total: 688 processes, 20735 lwps, load averag

Please help me, as it is impacting my productioni server.

Regards
Abhishek Singh


Moderator's Comments:
Mod Comment Please use [code] and [/code] tags when posting code, data or logs etc. to preserve formatting and enhance readability, thanks.
Last edited by zaxxon; 03-16-2011 at 09:46 AM.. Reason: code tags
# 2  
Old 03-16-2011
I was under the impression that all memory had to be backed by swap on most UNIX systems...
# 3  
Old 03-16-2011
It does not look like any of the virtual filesystems like TMPFS which can use up swap space are a problem.

However long running Java applications has a habit of slowly eating up memory and swap. How long since this system was rebooted?

The entire swap partition is treated as a single global resource to processes running in both global and non-global zones. If you are running Solaris 10 8/07 or later, you can limit the amount of swap on a per-zone basis using the 'swap' property of the capped-memory resource to cap the amount of virtual memory (VM) that a zone uses. This can also be set dynamically with the resource control zone.max-swap (check if this is correct name.) Have you tried setting a cap on swap space usage per zone?
# 4  
Old 03-16-2011
fpmurphy,
Thanks for your prompt reply.
Server is not rebooted since long time (over a year). Here is the config for four zones -
Code:
 
Zone1 - 
capped-memory:
physical: 6G
[swap: 12G]
[locked: 6G]
rctl:
name: zone.max-swap
value: (priv=privileged,limit=12884901888,action=deny)
rctl:
name: zone.max-locked-memory
value: (priv=privileged,limit=6442450944,action=deny)
 
Zone2 -
No capped memory or swap dedicated
 
Zone3 -
capped-memory:
physical: 4G
 
Zone4 - 
capped-memory:
physical: 4G

We can set up capped swap space for other zones also, but I feel 97% utilization is too high, so I was concerned, if something is eating up too much.

Regards
Last edited by solaris_1977; 03-16-2011 at 01:41 PM.. Reason: Putting in Tag
# 5  
Old 03-16-2011
Try to use dtrace to see what is eating up your swap space. You can find some pretty good dtrace scripts here that will help you figure it out.

Top Ten DTrace (D) Scripts
This User Gave Thanks to sboots For This Post:
solaris_1977
# 6  
Old 03-16-2011
Is there a problem with processes not getting the memory they request? If not, it's probably just that the page cache has grown over the extended uptime.
# 7  
Old 03-16-2011
Thanks all for the help.
sboots, I was able to see that there is no problem with any process. It is just extended uptime (428 days), which might have consumed mem and cpu as achenle said. I will talk to client to reboot the box.
Thanks again for your expertise.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. HP-UX

Swap device file and swap sapce

Hi I have an integrity machine rx7620 and rx8640 running hp-ux 11.31. I'm planning to fine tune the system: - I would like to know when does the memory swap space spill over to the device swap space? - And how much % of memory swap utilization should be specified (swap space device... (6 Replies)
Discussion started by: lamoul
6 Replies

2. Solaris

Coredumps and swap - was part of Solaris Mem Consumption

We have Sun OS running on spark : SunOS ciniwnpr67 5.10 Generic_118833-24 sun4u sparc SUNW,Sun-Fire-V440 Having Physical RAM : Sol10box # prtconf | grep Mem Memory size: 8192 Megabytes My Top Output is : 130 processes: 129 sleeping, 1 on cpu CPU states: 98.8% idle, 0.2% user, 1.0%... (39 Replies)
Discussion started by: rajwinder
39 Replies

3. UNIX for Advanced & Expert Users

Swap space problem

Hy! Recently i had a problem with one of mine Tru64 machines. It started to kill processes because of low amount of swap space. It said that it went below 10 %. But when i ran swapon -s it said: In-use space: 12 % So, the system couldn't accept any new ssh connections plus it killed most of... (1 Reply)
Discussion started by: veccinho
1 Replies

4. UNIX and Linux Applications

/dev/swap problem

Hi I am sory but my english is very BAD:mad: I have a problem with UnixWare 7.1.1 When I start the pc i have a message "/sbin/swap can¨t determine size /dev/swap" ??? Please help me (0 Replies)
Discussion started by: alminpulic
0 Replies

5. HP-UX

problem mirroring swap

Im following the directions from Mirroring the Root File System and Primary Swap # Mirror the root logical volume to the above disk: lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c0t3d0 # Mirror the primary swap logical volume: lvextend -m 1 /dev/vg00/prswaplv /dev/dsk/c0t3d0 # ... (1 Reply)
Discussion started by: csaunders
1 Replies

6. AIX

problem with swap

Good morning I have a problem with a partition (P570). A user's process take all the memory, and the swap. So the partition shutdown. How can I limit a process to not take all the swap ? (4 Go of memroy, 1 Go of swap) thank you (2 Replies)
Discussion started by: pascalbout
2 Replies

7. UNIX for Advanced & Expert Users

Strange swap space problem, pls help.

During Sun Solaris 8 installation, I did allocate 1 G for the swap partition. By doing a "df -k" shows the swap space usage is only 1% even during the application server is heavily processing. However, when I do a "vmstat", it shows that the swap memory free space is only 8816 out of 1419100. ... (1 Reply)
Discussion started by: champion
1 Replies

8. UNIX for Advanced & Expert Users

Swap Space Problem?

Over the last couple of days my laptop has been stalling terribly on bootup and when starting up applications (even a terminal takes ages to come up) Whilst trying to figure out whats happened I noticed that nothing is being swapped out. Output from top command shows that I have approx 500mb... (2 Replies)
Discussion started by: silvaman
2 Replies

9. UNIX for Advanced & Expert Users

swap problem

Hi all, when I execute a below script, I'm getting the following error, how can I solve it. $ sim_rep1.sh ====================== swap space below 10 percent free Unable to obtain requested swap space ======================== Thanks Krishna (1 Reply)
Discussion started by: krishna
1 Replies

10. Filesystems, Disks and Memory

problem with swap space

I have sun solaris 8 for intell with 128m physiccal ram and swap of 148. Oracle requires to have 512M swap space. Is there a way I can change the swap space on intell machine without repartioning the box:? what if i create a link to /swap from another place???? pls advise Jigar (1 Reply)
Discussion started by: jigarlakhani
1 Replies
Login or Register to Ask a Question