Mount point at 100%, but cannot see what is filling up


 
Thread Tools Search this Thread
Operating Systems Solaris Mount point at 100%, but cannot see what is filling up
# 8  
Old 07-14-2014
Dear hickd8,

Good point!

I never like NFS unless it's read-only, so publishing code for instance. It just seems to introduce too much complication to an operation otherwise. I always have files written to a common shared location being done with FTP (or SFTP, of course) It might be less efficient, but it's far easier to control and trace.


Robin

---------- Post updated at 03:15 PM ---------- Previous update was at 03:12 PM ----------

Quote:
Originally Posted by fretagi
Hi
Thanks again!

I am afraid I dont have
Code:
lsof

installed and the webpage that use to provide free utilities no longer does Introduction.
But using
Code:
fuser

it results in:

Code:
 fuser -c /nikira
/nikira:     4079com    4077c    3532co    3522com    1009c    1008c     995c   73128c   59575c   22574tom   64065tom   61756tom   59148tom   83983tom   56666c   43760c    1157om    1148o    1311ctom   75924ctom   56224ctm   56223ctm   56222ctm   56220ctm   56219ctm   56218ctm   56216ctm   56215ctm   56214ctm   56212ctm   56211ctm   56210ctom   56208cm   56056cm   56055com   55899ctm   26522com   26518co   26085com   26080co   25747com   25578co   24857com   24852co   78834ctom   74990ctom   72309ctom   98652tom   98605tom   31367ctm   43009com

So , I am not sure how to identify all these PIDs
As a rough and ready listing, you could:-
Code:
for pid in `fuser -c /nikira 2>/dev/null`
do
   ps -fp $pid | grep -v PPID
done


Robin
# 9  
Old 07-14-2014
the output has provided a long list of commands belonging to the application...
Code:
bash-3.00# for pid in `fuser -c /nikira 2>/dev/null`
> do
> ps -fp $pid | grep -v PPID
> done
  nikira  61140  48413   0 16:17:54 ?           0:00 sleep 10
  nikira  48413      1   0 16:15:04 ?           0:00 /bin/bash ./record_hashsplitter.sh 7
    root   1009   1008   0 16:04:31 pts/6       0:00 more -s /tmp/mpzPagFa
    root   1008    995   0 16:04:31 pts/6       0:00 sh -c more -s /tmp/mpzPagFa
    root    995  43760   0 16:04:31 pts/6       0:00 man fuser
  nikira  73128  56055   0 15:35:32 ?           0:00 sleep 10000
    root  59575  58160   0 15:32:44 pts/8       0:00 bash
  nikira  22574  98605   0 15:25:40 ?           0:00 /nikira/NIKIRATOOLS/APACHE64/bin/httpd -f /nikira/NIKIRATOOLS/APACHE64/conf/htt
  nikira  64065  98605   0 14:51:04 ?           0:00 /nikira/NIKIRATOOLS/APACHE64/bin/httpd -f /nikira/NIKIRATOOLS/APACHE64/conf/htt
  nikira  61756  98605   0 14:50:33 ?           0:00 /nikira/NIKIRATOOLS/APACHE64/bin/httpd -f /nikira/NIKIRATOOLS/APACHE64/conf/htt
  nikira  59148  98605   0 14:49:58 ?           0:00 /nikira/NIKIRATOOLS/APACHE64/bin/httpd -f /nikira/NIKIRATOOLS/APACHE64/conf/htt
  nikira  83983  98605   0 14:33:13 ?           0:00 /nikira/NIKIRATOOLS/APACHE64/bin/httpd -f /nikira/NIKIRATOOLS/APACHE64/conf/htt
  nikira  56666  56175   0 10:28:00 pts/9       0:00 -bash
    root  43760  43601   0 09:52:21 pts/6       0:02 bash
  nikira   1157   1148   0 06:53:01 ?           0:03 sqlplus -s /nolog
  nikira   1148   1144   0 06:53:01 ?           0:00 /bin/bash /nikira/NIKIRAROOT/bin/spark_tables_cleanup.sh 7
  nikira   1311  98652   0   Jul 11 ?           1:14 /nikira/NIKIRATOOLS/RUBYROOT/bin/ruby /nikira/NIKIRACLIENT/src/public/dispatch.
  nikira  75924      1   1   Jul 08 ?        8311:38 dbwriter -r /nikira/NIKIRAROOT/RangerData/DBWriterData/ -s /nikira/NIKIRAROOT/R
  nikira  56224  55899   0   Jul 08 ?           2:20 recorddispatcher -r RFF:///nikira/NIKIRAROOT/RangerData/AICumulativeGPRSData/ -
  nikira  56223  55899   0   Jul 08 ?           2:20 recorddispatcher -r RFF:///nikira/NIKIRAROOT/RangerData/AICumulativeVoiceData/
  nikira  56222  55899   0   Jul 08 ?           2:21 recorddispatcher -r RFF:///nikira/NIKIRAROOT/RangerData/SubscriberDataRecord/ -
  nikira  56220  55899   0   Jul 08 ?         740:24 recorddispatcher -r RFF:///nikira/NIKIRAROOT/RangerData/DataRecord_7/ -p TCP://
  nikira  56219  55899   0   Jul 08 ?         735:49 recorddispatcher -r RFF:///nikira/NIKIRAROOT/RangerData/DataRecord_6/ -p TCP://
  nikira  56218  55899   0   Jul 08 ?         754:20 recorddispatcher -r RFF:///nikira/NIKIRAROOT/RangerData/DataRecord_5/ -p TCP://
  nikira  56216  55899   0   Jul 08 ?         748:54 recorddispatcher -r RFF:///nikira/NIKIRAROOT/RangerData/DataRecord_4/ -p TCP://
  nikira  56215  55899   0   Jul 08 ?         744:29 recorddispatcher -r RFF:///nikira/NIKIRAROOT/RangerData/DataRecord_3/ -p TCP://
  nikira  56214  55899   0   Jul 08 ?         748:40 recorddispatcher -r RFF:///nikira/NIKIRAROOT/RangerData/DataRecord_2/ -p TCP://
  nikira  56212  55899   0   Jul 08 ?         755:14 recorddispatcher -r RFF:///nikira/NIKIRAROOT/RangerData/DataRecord_1/ -p TCP://
  nikira  56211  55899   0   Jul 08 ?           0:00 recordprocessor -r TCP://10.100.48.73:30002 -w TCP://10.100.48.75:20000 -c 1:TC
  nikira  56210  55899   0   Jul 08 ?           0:03 recordprocessor -r TCP://10.100.48.73:30001 -w TCP://10.100.48.75:20000 -c 1:TC
  nikira  56208  55899   0   Jul 08 ?           0:00 ssh -l nikira 10.100.48.75 /nikira_data01/NIKIRAROOT/bin/runprogram.sh programm
  nikira  56056  55899   0   Jul 08 ?           0:00 ssh -l nikira 10.100.48.74 /nikira_data01/NIKIRAROOT/bin/runprogram.sh programm
  nikira  56055  55899   0   Jul 08 ?           0:00 /bin/bash /nikira/NIKIRAROOT/sbin/licenseengine
  nikira  55899      1   0   Jul 08 ?           0:00 programmanager -f programmanager.conf
  nikira  26522  26518   0   Jul 08 ?          10:27 java -server -classpath ../config/:../lib/cereports.jar:../lib/itext-1.2.jar:..
  nikira  26518      1   0   Jul 08 ?           0:00 sh ./server.sh
  nikira  26085  26080   0   Jul 08 ?          10:31 java -server -classpath ../config/:../lib/cereports.jar:../lib/itext-1.2.jar:..
  nikira  26080      1   0   Jul 08 ?           0:00 sh ./server.sh
  nikira  25747  25578   1   Jul 08 ?        17863:38 java -server -classpath ../config/:../lib/cereports.jar:../lib/itext-1.2.jar:..
  nikira  25578      1   0   Jul 08 ?           0:01 sh ./tc.sh
  nikira  24857  24852   0   Jul 08 ?         440:39 java -server -classpath ../config/:../lib/cereports.jar:../lib/itext-1.2.jar:..
  nikira  24852      1   0   Jul 08 ?           0:00 sh ./sc.sh
  nikira  78834      1   0   Jun 30 ?          11:02 ./mcelrater -d nrtrde
  nikira  74990      1   0   Jun 30 ?        3904:46 ./mcelrater -d gprs
  nikira  72309      1   0   Jun 30 ?        1440:33 ./mcelrater -d gsm
  nikira  98652  98605   0   May 23 ?           2:58 /nikira/NIKIRATOOLS/APACHE64/bin/httpd -f /nikira/NIKIRATOOLS/APACHE64/conf/htt
  nikira  98605      1   0   May 23 ?           4:16 /nikira/NIKIRATOOLS/APACHE64/bin/httpd -f /nikira/NIKIRATOOLS/APACHE64/conf/htt
  nikira  31367      1   0   May 23 ?           7:08 memcached -p 12321
  nikira  43009      1   0   Oct 13 ?        106677:56 /bin/bash ./Nrtrde_VpmnID_populate_cdr.sh
bash-3.00#

# 10  
Old 07-14-2014
That's some serious CPU clocking. Can you stop/start each of these? I presume that they are services. Process 43009 would be the first to consider. It's been doing whatever for nine months. Could it be the log file for that script?



Robin
# 11  
Old 07-15-2014
I will consult the application admin, if he can stop all processes related to the application

---------- Post updated 15-07-14 at 09:37 AM ---------- Previous update was 14-07-14 at 04:36 PM ----------

after stopping all processes, them after a reboot, its now back to normal, thank you to all

---------- Post updated at 09:39 AM ---------- Previous update was at 09:37 AM ----------

How to get
Code:
lsof

This User Gave Thanks to fretagi For This Post:
# 12  
Old 07-16-2014
Search for unix lsof from your favourite internet search engine. I think it's written & held by SourceForge and is free to download & use (observing any copyright requirements)

You will need to pick the appropriate version for your OS. Probably easier to do that than to download the source code and compile it.



Robin
# 13  
Old 07-16-2014
Quote:
Code:
du -sh *

As a side note, the shell pattern * won't match filenames that start with a dot
(unless some non-standard shell option similar to bash's dotglob is active) ,
so the above command won't show you the disk usage of hidden files and directories.
This User Gave Thanks to radoulov For This Post:
# 14  
Old 07-16-2014
I'm late in that thread but anyway, you don't need lsof as it is relatively easy to identify using standard Solaris commands what processes have deleted files and on which file descriptors. Simply run as root:

Code:
find /proc/*/fd -links 0 ! -size 0 -ls

These 5 Users Gave Thanks to jlliagre For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to create a new mount point with 600GB and add 350 GBexisting mount point? IN AIX

How to create a new mount point with 600GB and add 350 GBexisting mount point Best if there step that i can follow or execute before i mount or add diskspace IN AIX Thanks (2 Replies)
Discussion started by: Thilagarajan
2 Replies

2. AIX

How to change the mount point of LV?

I have situation where my disk upon reboot, has its mount point as # LOGICAL VOLUME: disk4vol VOLUME GROUP: disk4vg LV IDENTIFIER: 00f609aa00004c0000000152414b786c.1 PERMISSION: read/write VG STATE: active/complete LV STATE: closed/syncd TYPE: jfs2 WRITE VERIFY: off MAX LPs: 512 PP SIZE: 512... (1 Reply)
Discussion started by: mrmurdock
1 Replies

3. UNIX for Beginners Questions & Answers

Can we have 2 mount point under the same name but at different directory?

guys i would like to know can we have 2 mount point which is same name but on different directory? (3 Replies)
Discussion started by: leecopper
3 Replies

4. Shell Programming and Scripting

Filling in the missing data point by awk

I am learning AWK by trying out examples whenever I need a specific conversion. I would like to edit the 'before.txt' so that all the missing data points between 140-150 are added and shown as 0. before.txt 145 2 148 13 149 17 to below, 140 0 141 0 142 0 143 0 144 0 145 2 146 0... (5 Replies)
Discussion started by: numareica
5 Replies

5. Red Hat

NFS mount point

Hi, Can you tell me something about NFS mount point ? Regards, Maddy (3 Replies)
Discussion started by: Maddy123
3 Replies

6. Shell Programming and Scripting

Mount point usage

Hi Guys, I have Solaris 9 and RHEL 5 boxes I implemented script to send me an email when my mount point is > 90. Now the ouput id like these: /dev/dsk/emcpower20a 1589461168 1509087840 64478720 96% /data1 /dev/dsk/emcpower21a 474982909 451894234 18338846 97% /data2... (2 Replies)
Discussion started by: Phuti
2 Replies

7. Solaris

Mount point in a server

Hi , How to find out mount point in a server ? OS -- SunOS 5.6 Generic sun4u sparc SUNW Thanks (4 Replies)
Discussion started by: Maddy123
4 Replies

8. Solaris

Mount Point Sorting?

Dear Gurus, Could it be possible to have the output of df -k sorted? The df -k output messed up after recent power trip. Also, is there any folders that I should look into to reduce the root size (other than /var/adm and /var/crash) after server crash? Many thanks in advance. ... (2 Replies)
Discussion started by: honmin
2 Replies

9. UNIX for Dummies Questions & Answers

auto mount point

hi can i know what is the command to create auto mount point in my unix server? is there any directory which i have to go? (1 Reply)
Discussion started by: legato
1 Replies

10. UNIX for Dummies Questions & Answers

mount point

hi people, I'm trying to create a mount point, but am having no sucess at all, with the following: mount -F ufs /dev/dsk/diskname /newdirectory but i keep getting - mount-point /newdirectory doesn't exist. What am i doing wrong/missing? Thanks Rc (1 Reply)
Discussion started by: colesy
1 Replies
Login or Register to Ask a Question