Emergency UNIX and Linux Support

Please post your urgent questions here for highest visibility. Posting a new thread to this forum requires Bits. We monitor this forum to help people with emergencies, but we do not guarantee response time or answers. This forum is "best effort" only. Members who reply to posts here receive a bonus of 1000 Bits per reply.

Swap space (almost) full

👤 Login to reply

    #1  
Old 02-28-2015
solaris_1977 solaris_1977 is offline
Registered User
 
Join Date: Mar 2011
Last Activity: 22 June 2018, 11:18 AM EDT
Posts: 464
Thanks: 57
Thanked 4 Times in 4 Posts
Swap space (almost) full

Hello,
This is RHEL 5.7. swap is almost full, but I am not sure, what to release and how to release space. This is production server so I would like to try all possible options before reboot.
Code:
[root@tudcol10 ~]# top
top - 00:18:26 up 327 days,  7:01,  3 users,  load average: 0.16, 0.21, 0.18
Tasks: 782 total,   1 running, 781 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us,  0.1%sy,  0.0%ni, 99.8%id,  0.1%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  49322952k total, 35302516k used, 14020436k free,  1986316k buffers
Swap: 52428116k total, 52424792k used,     3324k free,  3224040k cached

[root@tudcol10 ~]# free -m
             total       used       free     shared    buffers     cached
Mem:         48166      34480      13686          0       1941       3148
-/+ buffers/cache:      29390      18776
Swap:        51199      51196          3
[root@tudcol10 ~]# pidof memcached

[root@tudcol10 ~]# cat /proc/meminfo
MemTotal:     49322952 kB
MemFree:      14006976 kB
Buffers:       1988464 kB
Cached:        3224596 kB
SwapCached:     524352 kB
Active:       29330340 kB
Inactive:      4358960 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     49322952 kB
LowFree:      14006976 kB
SwapTotal:    52428116 kB
SwapFree:         3324 kB
Dirty:            6848 kB
Writeback:           0 kB
AnonPages:    27986200 kB
Mapped:         280616 kB
Slab:          1237248 kB
PageTables:     251032 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  77089592 kB
Committed_AS: 113543976 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    301576 kB
VmallocChunk: 34359435251 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB
[root@tudcol10 ~]#

Sponsored Links
    #2  
Old 02-28-2015
Peasant's Unix or Linux Image
Peasant Peasant is offline Forum Advisor  
Registered User
 
Join Date: Mar 2011
Last Activity: 21 July 2018, 6:52 AM EDT
Posts: 1,197
Thanks: 34
Thanked 370 Times in 318 Posts
Check vmstat looking at PI column if actual swapping is occurring at a certain moment.

If not, you are fine at this moment (probably not due to free memory available on the machine from your post)

What should worry you that sometime in past swapping occurred and was used by a process(es) which is no longer running.

You might want to consider putting vmstat in cron to monitor your machine to pinpoint the time when swapping occurs, then identify what is being ran to cause such high memory usage.
Sponsored Links
    #3  
Old 02-28-2015
solaris_1977 solaris_1977 is offline
Registered User
 
Join Date: Mar 2011
Last Activity: 22 June 2018, 11:18 AM EDT
Posts: 464
Thanks: 57
Thanked 4 Times in 4 Posts
As of now, current issue is, we are installing a software which requires 1 GB of free swap space. Due to less space, that is failing. I was checking if I can make it work without adding more swap space. Here is vmstat output. Does it point to something ?
Code:
[root@tudcol10 ~]# vmstat 2 10
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0 52424664 13507380 2032568 3311216    0    0    19    14    0    0  0  0 100  0  0
 0  0 52424664 13507704 2032584 3311336    0    0     0    38 1290 3973  0  0 100  0  0
 0  1 52424664 13508008 2032604 3311320    0    0     0   228 1185 3972  0  0 100  0  0
 0  0 52424664 13508116 2032688 3311248    0    0     0   352 1150 4560  0  0 100  0  0
 2  0 52424664 13507632 2032696 3311348    0    0     0    74 1588 4365  0  0 99  0  0
 0  0 52424664 13507352 2032708 3311336    0    0     0   288 1134 3240  0  0 99  0  0
 0  0 52424664 13507384 2032732 3311320    0    0     0    70 1268 3880  0  0 100  0  0
 0  0 52424664 13507844 2032744 3311344    0    0     0   268 1205 3903  0  0 100  0  0
 0  0 52424664 13508116 2032752 3311364    0    0     0    60 1053 3531  0  0 100  0  0
 1  0 52424664 13508240 2032764 3311364    0    0     0    82 1320 3226  0  0 100  0  0
[root@tudcol10 ~]#

    #4  
Old 02-28-2015
fpmurphy's Unix or Linux Image
fpmurphy fpmurphy is offline
Registered User
 
Join Date: Dec 2003
Last Activity: 12 June 2016, 11:03 PM EDT
Location: /dev/ph
Posts: 4,996
Thanks: 73
Thanked 476 Times in 438 Posts
You need to identify what process(es) are currently using swap space. Here is one popular method:
Code:
for file in /proc/*/status 
do 
     awk '/VmSwap|Name/ {printf $2 " " $3} END { print ""}' $file
done | sort -k 2 -n -r

Sponsored Links
    #5  
Old 02-28-2015
jim mcnamara jim mcnamara is offline Forum Staff  
...@...
 
Join Date: Feb 2004
Last Activity: 21 July 2018, 12:51 AM EDT
Location: NM
Posts: 11,441
Thanks: 638
Thanked 1,176 Times in 1,082 Posts
Are you running oracle? The SGA is created then swap of the same size is allocated. There is never any swapping activity once the SGA's are created. To allow other processes and oracle to get what they need, we created 500GB of swap - on a machine with 256 GB memory. Many applications do this. The only workaround is to add swap because of badly behaved processes. In this particular scenario.
Sponsored Links
    #6  
Old 02-28-2015
solaris_1977 solaris_1977 is offline
Registered User
 
Join Date: Mar 2011
Last Activity: 22 June 2018, 11:18 AM EDT
Posts: 464
Thanks: 57
Thanked 4 Times in 4 Posts
This is OHS and httpd server. Here is output of script given by fpmurphy, I have grepped top processes
Code:
[root@tudcol10  ~]# /tmp/swap_check.sh | wc -l
794
[root@tudcol10  ~]#
[root@tudcol10  ~]# /tmp/swap_check.sh | grep httpd.worker | wc -l
142
[root@tudcol10  ~]# /tmp/swap_check.sh | grep odl_rotatelogs | wc -l
105
[root@tudcol10  ~]# /tmp/swap_check.sh | grep opmn | wc -l
76
[root@tudcol10  ~]#

Sponsored Links
    #7  
Old 03-01-2015
MadeInGermany MadeInGermany is offline Forum Staff  
Moderator
 
Join Date: May 2012
Last Activity: 21 July 2018, 3:53 AM EDT
Location: Simplicity
Posts: 4,156
Thanks: 365
Thanked 1,419 Times in 1,275 Posts
Display the top 10 memory consumers:
Code:
ps -e -o vsz,rss,comm | { read x; echo "$x"; sort -k2n | tail; }

Sponsored Links
👤 Login to reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
File system full, swap nikhil kasar Solaris 2 08-22-2012 04:10 AM
Solaris full /tmp - du and df different swap NOT filled click Solaris 2 11-17-2011 12:09 PM
How to reclaim the space which i used to increse the swap space on Xen, apm Linux 1 06-20-2011 02:09 PM
/tmp: File system full, swap space limit exceeded frustrated1 Solaris 5 03-18-2007 04:38 PM
swap space / paging space aaronh AIX 2 05-19-2004 10:06 AM



All times are GMT -4. The time now is 07:30 AM.

Unix & Linux Forums Content Copyrightę1993-2018. All Rights Reserved.
×
UNIX.COM Login
Username:
Password:  
Show Password





Not a Forum Member?
Forgot Password?