Swap memory usage keeps increasing.

 
Thread Tools Search this Thread
Operating Systems Linux Red Hat Swap memory usage keeps increasing.
# 1  
Old 05-24-2017
Swap memory usage keeps increasing.

Hi

the version is RedHat 6.2 (Oracle DB server)

I don't know why swap memory usage keeps increasing...

I used to check swap memory usage Free -m and another way.

Code:
             total       used       free     shared    buffers     cached
Mem:         32183      31861        322          0        409      28413
-/+ buffers/cache:       3038      29145
Swap:        10236       5325       4911

and used below command to check each PID's swap memory usage
Code:
[root@ ~]# for proc in /proc/*; do cat $proc/smaps 2>/dev/null | awk '/Swap/{swap+=$2}END{print swap "\t'`readlink $proc/exe`'"}'; done | sort -n | awk '{total+=$1}/[0-9]/;END{print total "\tTotal"}'
0       /bin/bash
0       /bin/bash
0       /bin/bash
0       /bin/bash
0       /bin/bash
0       /bin/dbus-daemon
0       /bin/gawk
0       /bin/sort
0       /bin/su
0       /opt/hp/hp-health/bin/hpasmlited
0       /opt/hp/hp-health/bin/hp-asrd
0       /opt/hp/hp-snmp-agents/nic/bin/cmanicd
0       /opt/hp/hp-snmp-agents/server/bin/cmahealthd
0       /opt/hp/hp-snmp-agents/server/bin/cmahostd
0       /opt/hp/hp-snmp-agents/server/bin/cmapeerd
0       /opt/hp/hp-snmp-agents/server/bin/cmaperfd
0       /opt/hp/hp-snmp-agents/server/bin/cmasm2d
0       /opt/hp/hp-snmp-agents/server/bin/cmasm2d
0       /opt/hp/hp-snmp-agents/server/bin/cmastdeqd
0       /opt/hp/hp-snmp-agents/server/bin/cmathreshd
0       /opt/hp/hp-snmp-agents/storage/bin/cmaeventd
0       /opt/hp/hp-snmp-agents/storage/bin/cmafcad
0       /opt/hp/hp-snmp-agents/storage/bin/cmaidad
0       /opt/hp/hp-snmp-agents/storage/bin/cmaided
0       /opt/hp/hp-snmp-agents/storage/bin/cmasasd
0       /opt/hp/hp-snmp-agents/storage/bin/cmascsid
0       /opt/VRTSralus/bin/beremote
0       /oracle/app/11.2/grid/bin/crsd.bin
0       /oracle/app/11.2/grid/bin/cssdagent
0       /oracle/app/11.2/grid/bin/cssdmonitor
0       /oracle/app/11.2/grid/bin/evmd.bin
0       /oracle/app/11.2/grid/bin/evmlogger.bin
0       /oracle/app/11.2/grid/bin/gipcd.bin
0       /oracle/app/11.2/grid/bin/gpnpd.bin
0       /oracle/app/11.2/grid/bin/mdnsd.bin
0       /oracle/app/11.2/grid/bin/ocssd.bin
0       /oracle/app/11.2/grid/bin/octssd.bin
0       /oracle/app/11.2/grid/bin/ohasd.bin
0       /oracle/app/11.2/grid/bin/ologgerd
0       /oracle/app/11.2/grid/bin/oraagent.bin
0       /oracle/app/11.2/grid/bin/oraagent.bin
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/oracle
0       /oracle/app/11.2/grid/bin/orarootagent.bin
0       /oracle/app/11.2/grid/bin/orarootagent.bin
0       /oracle/app/11.2/grid/bin/osysmond.bin
0       /oracle/app/11.2/grid/bin/tnslsnr
0       /oracle/app/11.2/grid/opmn/bin/ons
0       /oracle/app/11.2/grid/opmn/bin/ons
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /oracle/app/oracle/product/11.2/bin/oracle
0       /root/nmon/bin/nmon_x86_64_rhel5
0       /sbin/audispd
0       /sbin/auditd
0       /sbin/init
0       /sbin/klogd
0       /sbin/mingetty
0       /sbin/mingetty
0       /sbin/mingetty
0       /sbin/mingetty
0       /sbin/mingetty
0       /sbin/mingetty
0       /sbin/multipathd
0       /sbin/portmap
0       /sbin/rpc.statd
0       /sbin/syslogd
0       /sbin/udevd
0       /skcc/ACEMAgent/3.0/bin/ACEMAgent.exe
0       /skcc/ACEMAgent/3.0/bin/INWatchDog.exe
0       /usr/bin/python
0       /usr/bin/readlink
0       /usr/bin/xfs
0       /usr/bin/Xorg
0       /usr/lib/Acronis/Agent/acronisagent
0       /usr/lib/Acronis/BackupAndRecovery/arsm
0       /usr/lib/Acronis/BackupAndRecovery/mms
0       /usr/libexec/gdmgreeter
0       /usr/libexec/gdm-rh-security-token-helper
0       /usr/libexec/hald-addon-acpi
0       /usr/libexec/hald-addon-keyboard
0       /usr/libexec/hald-addon-keyboard
0       /usr/libexec/hald-addon-keyboard
0       /usr/libexec/hald-addon-storage
0       /usr/libexec/hald-runner
0       /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/bin/java
0       /usr/sbin/atd
0       /usr/sbin/cimserver
0       /usr/sbin/crond
0       /usr/sbin/gdm-binary
0       /usr/sbin/gdm-binary
0       /usr/sbin/gpm
0       /usr/sbin/hald
0       /usr/sbin/hpiod
0       /usr/sbin/irqbalance
0       /usr/sbin/ntpd
0       /usr/sbin/pcscd
0       /usr/sbin/rpc.idmapd
0       /usr/sbin/sshd
0       /usr/sbin/sshd
0       /usr/sbin/sshd
0       /usr/sbin/sshd
0       /usr/sbin/sshd
0       /usr/sbin/uuidd
0       /usr/sbin/xinetd
0       Total

but....the result value is 0.....

I don't know why there is no swap memory usage but there is existing swap memory usage in free -m command's result.

Can you help me why the swap memory usage keeps increasing?

I attach another command result

Code:
[root@~]# vmstat 3 10
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0 5453296 294920 419980 29099820    0    0   240    85    0    1  1  1 99  0  0
 0  0 5453296 294548 419992 29099828    0    0    32   171 1207 5943  0  0 99  0  0
 0  0 5453296 292984 420004 29099824    0    0    27   162 1200 5613  0  0 99  0  0
 0  0 5453296 293480 420016 29099828    0    0    48   554 1145 5199  0  0 99  0  0
 0  0 5453296 294224 420024 29099840    0    0    28    50 1129 6146  0  0 99  0  0
 1  0 5453296 295864 420040 29099840    0    0    33  1172 1154 5213  0  0 99  0  0
 0  0 5453296 295144 420068 29099832    0    0    53   107 1269 5804  0  0 99  0  0
 0  0 5453296 299752 420092 29099940    0    0   112   118 1137 5295  0  0 99  0  0
 0  0 5453296 299196 420092 29099944    0    0    53    74 1121 5331  1  0 99  0  0
 0  0 5453296 299988 420104 29099952    0    0    43   166 1111 5427  0  0 99  0  0

Code:
top - 21:02:39 up 71 days,  4:41,  2 users,  load average: 1.28, 1.19, 1.11
Tasks: 394 total,   1 running, 393 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.8%us,  0.3%sy,  0.0%ni, 98.8%id,  0.1%wa,  0.0%hi,  0.0%si,  0.0%st

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  SWAP COMMAND
13547 acem      18   0  522m 112m 1684 S  2.0  0.3   1005:09 410m ACEMAgent.exe
11918 root      15   0  352m  35m  14m S  1.0  0.1 491:31.07 317m orarootagent.bi
11977 oracle    RT   0  382m 110m  53m S  1.0  0.3 540:01.24 271m ocssd.bin
17219 root      18   0 12216 1264  600 S  1.0  0.0   0:32.60  10m nmon_x86_64_rhe
13262 root      15   0 1707m  25m  13m S  0.7  0.1 472:50.90 1.6g orarootagent.bi
18120 oracle    15   0 12.2g  76m  50m S  0.7  0.2 406:27.19  12g oracle
 5861 root      RT   0  150m 4280 2816 S  0.3  0.0  85:00.28 145m multipathd
11394 root      15   0  381m  51m  21m S  0.3  0.2 110:03.00 329m ohasd.bin
11832 oracle    15   0  371m  36m  15m S  0.3  0.1 327:39.28 334m oraagent.bin
11934 root      RT   0  246m  87m  55m S  0.3  0.3 378:02.72 159m osysmond.bin
13597 oracle    15   0 12.2g  47m  42m S  0.3  0.1 163:53.72  12g oracle
18726 oracle    15   0 12.2g  47m  42m S  0.3  0.1 202:53.48  12g oracle
18728 oracle    15   0 12.2g  58m  53m S  0.3  0.2 187:58.09  12g oracle
25375 oracle    15   0 12.2g  46m  42m S  0.3  0.1 197:19.75  12g oracle
25377 oracle    15   0 12.2g  48m  43m S  0.3  0.2 197:12.23  12g oracle
27058 oracle    15   0 12.2g  72m  67m S  0.3  0.2 182:48.25  12g oracle
28400 root      15   0 13000 1376  848 R  0.3  0.0   0:00.04  11m top

Code:
[root@ ~]# sar -r | grep Average
Average:       299196  32657104     99.09    379724  29115012   5151813   5330559     50.85    219135

# 2  
Old 05-24-2017
From what you have posted, during the time you executed those commands there was no swapping.
But, your system did swap some period of time, hence the metrics.

Considering the cached column in your output, your system has cached 29 GB of data in memory.

For your output i can see you are running databases, is this Oracle ASM or filesystems in question ?

If you are running Oracle ASM, i would recommend tuning the kernel file system caching parameters and some others if required.

Here is a nice link :
https://access.redhat.com/sites/defa...hel6_1.2_1.pdf

The swappiness parameters is most important to notice in the document and apply to your environment.

If filesystems are in question, it would be required to pinpoint the specific time of memory starvation using metrics from cron and reports from users.
The kernel tuning if you are using filesystems should be done differently, since your database / system benefits greatly from memory for read io.
Read doesn't hit disks so often, so they have more time to service write request and repopulating cache, making system faster.
# 3  
Old 05-24-2017
I suspect that culprit and owner are both Oracle. What else but Oracle would fight to keep 5 gigabytes of idle memory just sitting around? Oracle keeps its own caches, not relying on the hosts, and they could end up in swap if they're sitting idle when Oracle outgrows system memory. This is why it's so important to tune Oracle to your system.

Swapped cache is an amusing concept.

Last edited by Corona688; 05-24-2017 at 02:46 PM..
# 4  
Old 05-24-2017
Quote:
Originally Posted by Peasant
Considering the cached column in your output, your system has cached 29 GB of data in memory.
Only sort of. The kernel will readily give up cache when needed. Large amounts of cache show a system that's not currently under memory pressure, but could be doing a fair amount of I/O. Large amounts of swap show a system that was under memory pressure at some point in the past but may not be now. Stuff that ends up in swap tends to stay, since idle memory gets swapped, not busy memory; it'll leave swap when asked for and no sooner.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Red Hat

Swap memory usage is high in Linux

Hi , There is one following alert . Message : cdm:Average (2 samples) swap memory usage is now 91%, which is above the warning threshold (90%) Here is my findings. Output of TOP command in Linux server. top - 14:21:44 up 6 days, 4:48, 1 user, load average: 2.55, 2.06,... (3 Replies)
Discussion started by: Maddy123
3 Replies

2. UNIX for Dummies Questions & Answers

Memory usage per user,percent usage,sytem time in ksh

Let's say i have 20 users logged on Server. How can I know how much memory percent used each of them is using with system time in each user? (2 Replies)
Discussion started by: roy1912
2 Replies

3. AIX

How to monitor the IBM AIX server for I/O usage,memory usage,CPU usage,network..?

How to monitor the IBM AIX server for I/O usage, memory usage, CPU usage, network usage, storage usage? (3 Replies)
Discussion started by: laknar
3 Replies

4. UNIX for Dummies Questions & Answers

Increasing swap space

I searched the archives first, but found that there are alot of mixed answers on whether swap space can or can not be increased. Some postings said swap space can be increased using the swap or growfs commands while other postings said you can not increase the permanent size of the swap space. ... (1 Reply)
Discussion started by: Kevin1166
1 Replies

5. Solaris

current CPU usage, memory usage, disk I/O oid(snmp)

Hi, I want to monitor the current cpu usage, monitor usage , disk I/o and network utlization for solaris using SNMP. I want the oids for above tasks. can you please tell me that Thank you (2 Replies)
Discussion started by: S_venkatesh
2 Replies

6. HP-UX

how can I find cpu usage memory usage swap usage and logical volume usage

how can I find cpu usage memory usage swap usage and I want to know CPU usage above X% and contiue Y times and memory usage above X % and contiue Y times my final destination is monitor process logical volume usage above X % and number of Logical voluage above can I not to... (3 Replies)
Discussion started by: alert0919
3 Replies

7. UNIX for Dummies Questions & Answers

var memory usage increasing

Hi all Just a couple days ago my /var/ partition was at 63% now its shot up to 83%. I checked my log file sizes and they same ok # ls -l |more total 38820 -rw-r----- 1 root root 3754 Aug 1 19:26 acpid drwxrwxrwx 2 root root 4096 Aug 24 04:11 admserv -rw------- ... (28 Replies)
Discussion started by: mcraul
28 Replies

8. HP-UX

Increasing swap size

We increased our server's RAM 8 -> 32 GB RAM. swap memory is currently 10 GB. With which command I can increase this memory? (1 Reply)
Discussion started by: akyuceisik
1 Replies

9. Programming

memory increasing

On Alpha True64 UNIX platform, we use "ps aux" command to display the process information, it also displays the %MEM,VSZ, RRS field. The question is that which field really indicates the program's memeory is increasing, thanks (0 Replies)
Discussion started by: Frank2004
0 Replies

10. Programming

memory increasing on UNIX

hi, i'm writing a program that is running always... but, i have a problem that memory is increasing.. i want to know whether does stack size inscease or not, when program is running.. i don't know what is the cause stack or heap... i need some help.. :( (1 Reply)
Discussion started by: netfer
1 Replies
Login or Register to Ask a Question