Unidentified File on /var Directory


 
Thread Tools Search this Thread
Operating Systems HP-UX Unidentified File on /var Directory
# 1  
Old 09-26-2013
Unidentified File on /var Directory

Hi All,

I'm having problem with /var directory which is keep increasing. Here's the output of bdf and du command
Code:
# uname -a
HP-UX rppmis1 B.11.11 U 9000/800 1153414645 unlimited-user license
 
# bdf /var
Filesystem          kbytes    used   avail %used Mounted on
/dev/vg00/lvol8    2613248 2139568  473392   82% /var
 
# du -skx /var
1615552 /var

bdf and du command reported different size of used space, 2,139,568 - 1,615,552 = 524,016 KB. Can someone tell me how to find this file ?

Here's detail of /var directory
Code:
# du -skx /var/* | sort -k2
16      /var/X11
518344  /var/adm
408     /var/asx
4520    /var/dmi
152     /var/dt
0       /var/home
6160    /var/hp
0       /var/ifor
0       /var/lost+found
18320   /var/mail
0       /var/news
0       /var/obam
86048   /var/opt
0       /var/parmgr
118832  /var/patches
11376   /var/preserve
14336   /var/rbootd
24      /var/run
16160   /var/sam
1032    /var/spool
16      /var/statmon
768752  /var/stm
49872   /var/tmp
712     /var/tombstones
0       /var/uucp
304     /var/vx
96      /var/yp

FYI, this is what I've done to reduce size of /var directory
- nulling log file such as /var/adm/wtmp
- trimming /var/mail/oracle using vi editor
- delete file such as /var/preserve/Ex*
- delete /var/adm/crash/crash.5 directory and its content
- delete an application log file on /var/tmp, initially I tried to null it with "cat /dev/null > trasym.ulma" command but it didn't work so I delete it. After I delete it, this file appeared again with same size before the deletion. After several times of deletion, finally this file is disappeared. Unfortunately I forgot to check with "fuser" command when delete this file to ensure that the respective process is already stop

Appreciate any feedback from all. Thanks
# 2  
Old 09-26-2013
I would always prefer to:-
Code:
du -k | sort -n

.... to get the biggest at the bottom. You can also:-
Code:
ls -l | sort -nk 5

.... to get the biggest files in the current directory.

Does that direct you anywhere?


I see that /var/preserve is quite large. This is normally where editor recovery files are left. Perhaps they could be pruned. There is also mail waiting to be read. Have a look in /var/mail and for each large file, get the user to read their mail. These are normally the output from cron jobs, so perhaps you have something that works, but issues messages that you need to take care of.


I hope that this helps, but feel free to ask more if you still need help.




Robin
Liverpool/Blackburn
UK
# 3  
Old 09-26-2013
There's no file missing. du reports the disk usage by files/directories. But there's more to that: file system infra structure/meta data like super blocks are consuming disk space. Plus, the OS reserves some space for emergency root access (not sure if this holds true for data file systems). This is what df reports. It goes without saying that the two infos differ.
This User Gave Thanks to RudiC For This Post:
# 4  
Old 09-27-2013
How much load do you have on your box, and what consumes midaemon?

Now, the file you deleted will only release its space if it were not used... since it was growing I doubt...
# 5  
Old 09-27-2013
I forgot to say something else, quite important. If you have removed a large file (or large total size of files) and there was no change in the usage, then it could be that the file was in-use. The only thing you will have done is is to remove the entry from the directory it is in. The filesystem as a whole will still have the blocks marked as used until the process that has it open ends / closes it.

Use can use the fuser command to look for deleted files that are still in use on a filesystem. It depends on your OS as to what flags to use.

You are in the HP-UX thread, but I'm only a 11.11, so my fuser does not have this as an option. On other platforms, you could use something like:-
Code:
fuser -duV /var


Not sure if this helps, but I thought I should bring it up.




Robin

Last edited by rbatte1; 10-07-2013 at 08:54 AM..
# 6  
Old 09-30-2013
Quote:
Originally Posted by vbe
How much load do you have on your box, and what consumes midaemon?

Now, the file you deleted will only release its space if it were not used... since it was growing I doubt...
Hi vbe,
It ranges between 1.00 and 2.00 on daily basis. Here's today load
Code:
System: rppmis1                                       Mon Sep 30 08:47:20 2013
Load averages: 0.58, 0.93, 1.16
706 processes: 697 sleeping, 9 running
Cpu states:
CPU   LOAD   USER   NICE    SYS   IDLE  BLOCK  SWAIT   INTR   SSYS
 0    0.61  17.6%   1.4%  40.9%  40.1%   0.0%   0.0%   0.0%   0.0%
 1    0.54   5.5%   1.4%  11.5%  81.6%   0.0%   0.0%   0.0%   0.0%
---   ----  -----  -----  -----  -----  -----  -----  -----  -----
avg   0.58  11.5%   1.4%  26.2%  60.9%   0.0%   0.0%   0.0%   0.0%
Memory: 1282912K (172324K) real, 3158708K (471108K) virtual, 23280K free  Page# 1/65
CPU TTY     PID USERNAME PRI NI   SIZE    RES STATE    TIME %WCPU  %CPU COMMAND
 1   ?    28672 autoprod 154 20   107M  1276K sleep    1:19 26.65 26.61 oracleMES
 1   ?     8633 oracle   231 20   107M  1184K run      0:02 71.36  9.94 oracleMES
 1   ?    23987 oracle   148 22  1764K   160K sleep   48:35  5.77  5.76 tar
 0   ?     7446 autoprod 154 20 24380K  3500K sleep   15:28  3.89  3.89 eff
 0   ?     7529 autoprod 154 20   107M  1128K sleep   11:49  2.84  2.83 oracleMES
 1   ?    13271 autoprod 154 20   108M  1096K sleep   23:54  2.70  2.70 oracleMES
 1   ?     4602 autoprod 154 20   107M  1452K sleep    0:10  1.83  1.82 oracleMES
 1   ?    13190 autoprod 154 20 38172K  4708K sleep   11:19  1.00  1.00 eff
 0 pts/ts 19475 budik    168 20  9816K  1292K sleep    0:23  0.89  0.88 top
 1   ?       37 root     152 20  1888K  1888K run     39:41  0.68  0.68 vxfsd
# 
# ps -ef | grep midaemon
    root  8663  8620  1 08:47:27 pts/tj    0:00 grep midaemon

As you can see, midaemon isn't running.

Quote:
Originally Posted by rbatte1
I forgot to say something else, quite important. If you have removed a large file (or large total size of files) and there was no change in the usage, then it could be that the file was in-use. The only thing you will have done is is to remove the entry from the directory it is in. The filesystem as a whole will still have the blocks marked as used until the process that has it open ends / closes it.

Use can use the fuser command to look for deleted files that are still in use on a filesystem. It depends on your OS as to what flags to use.

You are in the HP-UX thread, but I'm only a 11.11, so my fuser does have this as an option. On other platforms, you could use something like:-
Code:
fuser -duV /var


Not sure if this helps, but I thought I should bring it up.




Robin
Hi rbatte1,
Yes, I've removed a single large file with size about 1.5 GB from /var/tmp directory. It was an application log file and I thought it was safe to delete it. It took several times of deletion before this file is totally disappeared. After that I ran bdf command and see no change on usage. I thought this is the root cause but it only last for a while. This log file is generated again by the application but with a smaller size.

Beside this file, I've also deleted some other files. I was panic at that time so I didn't record properly what files on which directory that I've deleted. Maybe one or several of them caused this problem. Current conclusion is I have to find process that is still accessing these files and kill it in order to release those files.

I'm using HP-UX 11.11 too but fuser command only support following options
Code:
-c   Display the use of a mount point and any file beneath that
     mount point.  Each file must be a file system mount point.
-f   Display the use of the named file only, not the files
     beneath it if it is a mounted file system.
-u   Display the login user name in parentheses following each
     process ID.
-k   Send the SIGKILL signal to each process using each file.

Correct me if I'm wrong, since English isn't my native language Smilie Based on above options, fuser command can not be used to find process that is using deleted file.

Is there any other command that I can use for this purpose ?

Last edited by stu33; 09-30-2013 at 04:02 AM..
# 7  
Old 09-30-2013
There is an article on the HP ITRC site. You will have to be registered to have a look at it though.

https://h20565.www2.hp.com/portal/si...tte.cachetoken


I hope that it helps. I don't have the tool crashinfo available on my HPUX server. It seem that you have to request it from HP support, according to this article:-

Viewing Open Sockets and Ports with Crashinfo | HP® Support



I hope that this helps.

Robin
Liverpool/Blackburn
UK
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Transfer the logs being thrown into /var/log/messages into another file example /var/log/volumelog

I have been searching and reading about syslog. I would like to know how to Transfer the logs being thrown into /var/log/messages into another file example /var/log/volumelog. tail -f /var/log/messages dblogger: msg_to_dbrow: no logtype using missing dblogger: msg_to_dbrow_str: val ==... (2 Replies)
Discussion started by: kenshinhimura
2 Replies

2. Shell Programming and Scripting

Need a script for automatically cleaning up /var/spool/cups directory

Hi Friends, Actually in an linux server , there was printer jobs files occupying more space in /var/spool/cups so i want a script for deleting the files once in two week since i need the latest two weeks files. Thanks in advance..Waiting for the script. (2 Replies)
Discussion started by: Mohamed Thamim
2 Replies

3. UNIX for Dummies Questions & Answers

How to protect directory var/www in debian?

hi i need somme help to protect directory in debian with user and pass var/www thank's (6 Replies)
Discussion started by: azzeddine2005
6 Replies

4. SuSE

Suse 8 /var Directory

Hi there, I'm working on a Suse 8 based system and picked up that the crontab was missing. after some investigation i realized that the hole /var directory is missing. :mad: I would like to now if its common for this directory to go missing? Can I copy another Suse 8 /var directory to this... (10 Replies)
Discussion started by: malanve
10 Replies

5. Ubuntu

problem with /var/mail directory

Hi, There is no "administrator" file in "/var/mail" directory........ can any one tell me what could be the reason for that?...and also how to resolve that??? Thanks, ~Kavi (4 Replies)
Discussion started by: kavi.mogu
4 Replies

6. HP-UX

[Solved] /var directory in HP-UX is showing 95%?

Hi Everyone, My var directory is showing near to 100% ? What are the files should i delete to make it less?? Kindly suggest # bdf -i Filesystem kbytes used avail %used iused ifree %iuse Mounted on /dev/vg00/lvol3 1048576 107616 933616 10% 3342 29394 10% /... (10 Replies)
Discussion started by: yadvinder
10 Replies

7. Solaris

Changing of syslog file path instead of /var/log directory

Hi Please let me know how can we change the syslog file path from /var/log to /a directory in solaris Regards (4 Replies)
Discussion started by: amity
4 Replies

8. Solaris

Lost /var/sadm/install/contents file and /var/sadm/pkg

Hello, I recently found that my /var/sadm/install/contents, ~/admin/default, /var/spool/patch and /var/spool/pkg files were empty. This broke the pkginfo, pkgchk and other package related tools. The pkgmap no longer points to where the applications have been installed. I have replaced the... (0 Replies)
Discussion started by: ronin42
0 Replies

9. Solaris

Usage of /var/tmp/ directory on Solaris 10

Hi All, I have some threaded applications. Design of the application is such that one thread will decode some data and put it in data structure, And main thread will wait for another child threads pick up the decoded data. The data will be large decoded files. Once decoded data is picked by... (1 Reply)
Discussion started by: patilmuragesh
1 Replies

10. UNIX for Dummies Questions & Answers

Unix Swap File in /var directory

Thank you 98_1LE.... Please help required urgently... We are getting error message in SUN... Please give solution ERROR MESSAGE: NOTICE: alloc: /var: file system full Output of command "df -k" in our system is... Filesystem kbytes used avail capacity Mounted on... (1 Reply)
Discussion started by: wipro fluid power
1 Replies
Login or Register to Ask a Question