Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Can you use find with ps or doing find excluding file in use

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Can you use find with ps or doing find excluding file in use
# 1  
Old 09-08-2016
Can you use find with ps or doing find excluding file in use
Hi,

I am currently using the find below to remove old files. I am redirecting the listing to a file and then use a while-loop and do a rm

Code:
cd ${directory}
find . \( ! -name . -prune \) \( -type f -name "*.trc" -mtime +10 \) | sed 's#^./##' | sed "s#^#${directory}/#" 2>/dev/null | tee -a /tmp/rm_list_trc.txt
find . \( ! -name . -prune \) \( -type f -name "*.trm" -mtime +10 \) | sed 's#^./##' | sed "s#^#${directory}/#" 2>/dev/null | tee -a /tmp/rm_list_trm.txt

I've now just found out that some of the files that I am removing are actually still in use. Is it possible to tell find to do a fuser / lsof check and not do the remove if the file is in user?

Examples of these files that I want to exclude for getting removed are as below:


Code:
$ date
Fri Sep  9 07:50:44 NZST 2016
$ pwd
/u01/app/oracle/admin/qual1/bdump
$ ps -ef | grep qual1 | grep lgw
oracle   29112     1  0 Sep08 ?        00:00:01 ora_lgwr_qual13
$ ls -altr *lgw*trc
-rw-rw---- 1 oracle oinstall 955 Sep  8 11:35 qual13_lgwr_15798.trc
-rw-rw---- 1 oracle oinstall 805 Sep  8 11:40 qual13_lgwr_29112.trc
$ ls -altr *lgw*trm
ls: *lgw*trm: No such file or directory
$ date
Fri Sep  9 07:51:14 NZST 2016
$ ps -ef | grep qual1 | grep ora_p
oracle   21816     1  0 07:47 ?        00:00:00 ora_pz99_qual13
oracle   21818     1  0 07:47 ?        00:00:00 ora_pz98_qual13
oracle   29039     1  0 Sep08 ?        00:00:01 ora_pmon_qual13
oracle   29047     1  0 Sep08 ?        00:00:00 ora_psp0_qual13
$ ls -altr qual1*p*trc
-rw-rw---- 1 oracle oinstall 1770402 Sep  8 11:24 qual13_pmon_26250.trc
-rw-rw---- 1 oracle oinstall     890 Sep  8 22:00 qual13_pz98_31759.trc
-rw-rw---- 1 oracle oinstall     892 Sep  8 22:00 qual13_pz97_31903.trc

$ ls -altr *lgwr*
-rw-rw---- 1 oracle oinstall 955 Sep  8 11:35 qual13_lgwr_15798.trc
-rw-rw---- 1 oracle oinstall 805 Sep  8 11:40 qual13_lgwr_29112.trc
$ ps -ef | grep lgwr | grep prrq
oracle   29112     1  0 Sep08 ?        00:00:01 ora_lgwr_qual13
$ fuser qual13_lgwr_29112.trc
qual13_lgwr_29112.trc: 29112
$ pwd
/u01/app/oracle/admin/qual1/bdump
$ lsof /u01/app/oracle/admin/qual1/bdump/qual13_lgwr_29112.trc
COMMAND   PID   USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
oracle  29112 oracle    2w   REG  253,3      805 8357085 /u01/app/oracle/admin/qual1/bdump/qual13_lgwr_29112.trc
$

While these files may not have been modified for awhile and meet the find criteria for removal based on -mtime, we need to keep them as sometimes Oracle requests for these files if we log a call with them.

Currently, in the while-loop that I am using to do the rm, I am now doing a lsof or fuser now as well before doing the rm? Perhaps someone know of any better way of doing this.

Thanks in advance.
# 2  
Old 09-08-2016
If these files are all under a particular folder, lsof +D /path/to/folder will produce a list of files and folders in use inside that folder. Subtracting this list from the list of files you wish to delete may be more efficient than calling lsof dozens or hundreds of times.

On the other hand, deleting a file that's in use isn't necessarily a problem to the thing that's using it. It will stay on disk until it's finally closed, then disappear forever.
# 3  
Old 09-09-2016
Quote:
Originally Posted by Corona688
If these files are all under a particular folder, lsof +D /path/to/folder will produce a list of files and folders in use inside that folder. Subtracting this list from the list of files you wish to delete may be more efficient than calling lsof dozens or hundreds of times.

On the other hand, deleting a file that's in use isn't necessarily a problem to the thing that's using it. It will stay on disk until it's finally closed, then disappear forever.
Hi,

Thanks for your reply.

I like your idea of the lsof +D. Do you mean using diff on the lsof +D list and the list of files from the find?

I am not able to confirm what you mentioned on the second paragraph. We are not able to check whether the file was deleted before or after we shutdown the database. Correct me if I understand it differently, are you saying even if we had done a remove, the file will stay there because the process has it in use but it will eventually disappear if the process that has in use is killed?

We were expecting these files would have been getting 'modified' by the Oracle processes that has them in use or at least doing a 'touch' of them so it's modified bit is getting ignored by the find Smilie

I'll try and have a go with doing lsof.
# 4  
Old 09-09-2016
The modification time won't change if it's not modifying them. The access time might be more useful but many systems don't bother updating that for performance reasons.

diff won't give a good result -- it's meant for comparing source files, not lists of things. comm would work, if you sorted the two files.
Code:
$ man comm

COMM(1)                          User Commands                         COMM(1)



NAME
       comm - compare two sorted files line by line

SYNOPSIS
       comm [OPTION]... FILE1 FILE2

DESCRIPTION
       Compare sorted files FILE1 and FILE2 line by line.

       With  no  options,  produce  three-column  output.  Column one contains
       lines unique to FILE1, column two contains lines unique to  FILE2,  and
       column three contains lines common to both files.

       -1     suppress column 1 (lines unique to FILE1)

       -2     suppress column 2 (lines unique to FILE2)

       -3     suppress column 3 (lines that appear in both files)

...

It may take some work to produce lists that will compare right.

If you delete a file while it's open, it will no longer show in the directory listing but will still take up space on disk until it's closed, when it will finally disappear. (Which can be a big problem when you delete a giant logfile to free space and nothing happens.)
# 5  
Old 09-09-2016
A bit safer and simpler is
Code:
if cd ${directory}; then
  find . ! -name . -prune -type f -name "*.trc" -mtime +10 -atime +5 | sed "s#^\.#${directory}#" | tee -a /tmp/rm_list_trc.txt
  find . ! -name . -prune -type f -name "*.trm" -mtime +10 -atime +5 | sed "s#^\.#${directory}#" | tee -a /tmp/rm_list_trm.txt
else
  : "cd failed, we better don't delete anything"
fi

---------- Post updated at 10:59 ---------- Previous update was at 10:29 ----------

A solution with fuser, slow but reliable:
Code:
if cd "$directory"
then
  find . ! -name . -prune -type f \( -name "*.trc" -o -name "*.trm" \) -mtime +10 -atime +5 |
  while IFS=  read -r fn
  do
    if fuser "$fn" >/dev/null 2>&1
    then
      echo "'$fn' is in use, not deleted."
    else
      case $fn in
      *.trc) printf "%s\n" "$fn" >&3;;
      *.trm) printf "%s\n" "$fn" >&4;;
      esac
    fi
  done 3>>/tmp/rm_list_trc.txt 4>>/tmp/rm_list_trm.txt
else
  echo "cd '$directory' failed, nothing deleted."
fi

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Find file with extension and excluding directory

Hi, I have an inquiry on how do I use the find command in Solaris Unix to find some file ends with extension : txt, err in the root directory with modified date of 30days and this find command will also need to exclude b directory and its subdirectory. All the files from the above find criteria... (5 Replies)
Discussion started by: snowfrost88
5 Replies

2. Shell Programming and Scripting

Bash - Find files excluding file patterns and subfolder patterns

Hello. For a given folder, I want to select any files find $PATH1 -f \( -name "*" but omit any files like pattern name ! -iname "*.jpg" ! -iname "*.xsession*" ..... \) and also omit any subfolder like pattern name -type d \( -name "/etc/gconf/gconf.*" -o -name "*cache*" -o -name "*Cache*" -o... (2 Replies)
Discussion started by: jcdole
2 Replies

3. Shell Programming and Scripting

Find command excluding directories and some files

hello. I try to print a list of files but excluding some directories and some files. I would like to write a command for : find "from_dir" "ignore dir1, dir2, ..." "ignore file1, file2,...." "where file are older than 2017-02-03T06:00:00" Note that "DO_IT" is a local function in the script... (5 Replies)
Discussion started by: jcdole
5 Replies

4. Shell Programming and Scripting

Excluding directories from a find

I've looked at a few similar threads, but I can't bridge from those examples to what I'm working on, so I'm hoping someone can help. I want to extend the following statement find $PathToCheck -type f \( -not -iwholename "$ScriptDir/*" \) -exec md5sum "{}" \;>$NewSigs to exclude several... (9 Replies)
Discussion started by: nixie
9 Replies

5. Shell Programming and Scripting

Find all files for a user, excluding a directory

I have been searching, and cannot find an answer for this. I am trying to find all files for a user, lets call him (test001), and I want to exclude a specific directory. Here is the command I run, it finds all files: find / -user test001 I get this result: > find / -user test001 ... (4 Replies)
Discussion started by: steve2x4
4 Replies

6. UNIX for Dummies Questions & Answers

Need help in excluding a particular directory using Find commad

Hi, I have a directory structure as below /home/gad/Merl/a/a1.txt /home/gad/Merl/b/a1.txt /home/gad/Merl/c/a1.txt How can I find the file a1.txt but not from directory 'a' and it(the filw) should loaded 6 days ago.. Can any one pls help,quick reply much appriciated.. Thanks. (1 Reply)
Discussion started by: jagadish_gaddam
1 Replies

7. Solaris

how to find out the file's name excluding string?

Hello, Under one directory, I can use below command to find out the file names with string "Export terminated successfully without warnings" grep -i -l "Export terminated successfully without warnings" *.* My question is : how I find out the file names without including string "Export... (5 Replies)
Discussion started by: GreatJerry
5 Replies

8. UNIX for Dummies Questions & Answers

Excluding directories with find

How do I exclude directories with the find command on Solaris? I want to skip the directories /proc and /shared. find / -nouser -print This shows me all files and directories that don't have an owner but I need to skip /shared and /proc. I've been able to get it to work on Linux... (3 Replies)
Discussion started by: x96riley3
3 Replies

9. UNIX for Advanced & Expert Users

find excluding a directory and a file

Hi I have some 5 folders and two files in the current directory. I want to delete all, expect one folder(files in the folder too should not be deleted) and a file in the current directory. Lets say the folder and file that should not be deleted as 'a'(folder name) and 'b'(file name). Can you... (1 Reply)
Discussion started by: ammu
1 Replies

10. UNIX for Advanced & Expert Users

find excluding the hidden files

Hi , I am trying to use the find command with delete in a directory . Even though i use a wil character search the find command is checking the hidden files which inturn results in error . Can i avoid look that into the hidden files ?? I am using HP unix . find /cv1/ -name "ite*"... (3 Replies)
Discussion started by: arunkumar_mca
3 Replies
Login or Register to Ask a Question