pid.cleanup script.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting pid.cleanup script.
# 1  
Old 02-28-2011
pid.cleanup script.

Hi guys!

I have a directory in the production environment from which i have to delete files older then 40 minutes with .pid extention. I wrote a script below for the purpose.

Code:
#!/bin/bash
#
# Script to delete specific file older than N minutes.
# OLDERTHAN="40" #40 minutes

FOLDER="home/optima/pids/"
PID="*.pid"
OLDERTHAN="40"
if [ -e ${FOLDER}/${PID} ]
then
     ls_time=`ls -l ${FOLDER}/$PID`
     pid_h=`echo $ls_time | cut -d\ -f 8 | cut -d\: -f 1`
     pid_m=`echo $ls_time | cut -d\ -f 8 | cut -d\: -f 2`
     echo pid time=$pid_h:$pid_m
     let pid_time=(10#$pid_h*60)+10#$pid_m
     curr_h=`date | cut -d\ -f 4 | cut -d\: -f 1`
     curr_m=`date | cut -d\ -f 4 | cut -d\: -f 2`
     let curr_time=(10#$curr_h*60)+10#$curr_m
     echo curr_time=`date | cut -d\ -f 4`
     let diff=10#$curr_time-10#$pid_time
     #echo pid_time=$pid_time
     #echo curr_time=$curr_time
     echo diff=$diff minutes
     if [ $diff -ge $OLDERTHAN ]
     then
          echo "${PID} is older than $OLDERTHAN minutes"
          echo "Deleting ${PID}..."
          rm -f ${FOLDER}/${PID}
     else
          echo -e "${PID} is not older than $OLDERTHAN minutes"
     fi
else
     echo -e "${PID} not found."
fi

I created a test pid in /home/somefolder/pids to check if the script will delete old pids or not. I ran it & it showed the following result “*.pid not found.” Any advice?

Code:
# ls -ltr
total 96
-rw-r----- 1 root sys 0 Feb 28 09:00 test.pid
-rwxr-xr-x 1 root sys 15 Feb 28 11:52 optimamd_opx_LOD_GEN_110_00111010L.pid
-rwxr-xr-x 1 root sys 15 Feb 28 11:52 optimamd_opx_LOD_GEN_110_001110107.pid
-rwxr-xr-x 1 root sys 15 Feb 28 11:52 optimamd_opx_LOD_GEN_110_001110109.pid
-rwxr-xr-x 1 root sys 16 Feb 28 11:52 optimamd_opx_LOD_GEN_110_00111010H.pid
-rwxr-xr-x 1 root sys 15 Feb 28 11:52 optimamd_opx_LOD_GEN_110_00111009A.pid
-rwxr-xr-x 1 root sys 15 Feb 28 11:52 optimamd_opx_LOD_GEN_110_001110106.pid
#
 
/home/optima/run
 
-rwxrwxrwx 1 optima dba 1716 Jun 2 2010 NSN_MGWtest_DAP.sh
-rwxrwxrwx 1 optima dba 2828 Jul 7 2010 RunValidators_HUAWEI_BSC6000.sh.orig
-rwxrwxrwx 1 optima dba 5482 Jul 7 2010 NSN_MSS_DAP.sh
-rwxrwxrwx 1 optima dba 1729 Jul 7 2010 NSN_MGW_DAP.sh
-rwxrwxrwx 1 optima dba 244 Jul 7 2010 Run_FTP.sh
-rwxrwxrwx 1 optima dba 1914 Jul 7 2010 NSN_MGW_VAL.sh
-rwxrwxrwx 1 optima dba 6013 Jul 7 2010 NSN_MSS_VAL.sh
-rwxrwxrwx 1 optima dba 5138 Jul 9 2010 NSN_MSS_loader.sh
-rwxrwxrwx 1 optima dba 1715 Jul 9 2010 NSN_MGW_loader.sh
-rwxrwxrwx 1 optima dba 1460 Jul 10 2010 MOTOROLA_BSS_VALIDATOR.sh
-rwxrwxrwx 1 optima dba 6100 Oct 11 08:37 RunLoaders_HUAWEI_BSC6000.sh
-rwxrwxrwx 1 optima dba 2828 Oct 18 12:40 RunValidators_HUAWEI_BSC6000.sh
-rwxrwxrwx 1 optima dba 945 Nov 29 16:27 MOTOROLA_BSS_CMB.sh
-rwxrwxrwx 1 optima dba 2120 Dec 6 11:39 MOTOROLA_BSS_LOADERS.sh
-rwxrwxrwx 1 optima dba 228 Jan 10 11:31 OLD_RunProcessMonitor_004.sh
-rwx------ 1 root sys 252 Jan 19 12:34 Neighbour_single.sh
-rwx------ 1 root sys 228 Jan 26 12:07 RunProcessMonitor_004.sh
-rwxrwxrwx 1 root sys 957 Feb 28 11:39 pid_cleanup.sh
# ./pid_cleanup.sh
*.pid not found.


will changing
Code:
PID="*.pid"

to
Code:
PID="*pid* *.pid*".

help?

---------- Post updated at 06:44 AM ---------- Previous update was at 05:19 AM ----------

I replaced PID=”*.pid” with PID="*pid* *.pid*" and ran the script and got the below result. Kindly advice.
Code:
# ./pid_cleanup.sh
./pid_cleanup.sh: script:  not found.
./pid_cleanup.sh[10]: Syntax error at line 16 : `(' is not expected.
#

Moderator's Comments:
Mod Comment Please use code tags also in updates. Also please refrain from overly use of font formatting since it is awful to remove when editing your post, thanks. It would be also helpful if you'd use indention when posting your scripts as well.

Last edited by zaxxon; 02-28-2011 at 08:08 AM.. Reason: code tags, indention etc.
# 2  
Old 02-28-2011
The test
Code:
if [ -e ${FOLDER}/${PID} ]

does not accept multiple values this way. Putting a set -x in the head of your script shows following:
Code:
+ '[' -e home/optima/pids/optimamd_opx_LOD_GEN_110_00111009A.pid home/optima/pids/optimamd_opx_LOD_GEN_110_001110106.pid
 home/optima/pids/optimamd_opx_LOD_GEN_110_001110107.pid home/optima/pids/optimamd_opx_LOD_GEN_110_001110109.pid
 home/optima/pids/optimamd_opx_LOD_GEN_110_00111010H.pid home/optima/pids/optimamd_opx_LOD_GEN_110_00111010L.pid ']'
./mach.ksh: line 10: [: too many arguments
+ echo -e '*.pid not found.'
*.pid not found.

I suggest to put it this way and do not test with -e if the files exist, as the loop will only run on the files that fit to the expansion anyway and sowith exist:
Code:
for f in ${FOLDER}/*pid; do
   ls_time=`...
   ...
done

You will also get problems with these kind of cuts:
Code:
cut -d\ -f 8 | cut -d\: -f 1

If you want a blank as delimeter, write it like this:
Code:
cut -d" " -f 8 ...
#or
cut -d\  -f 8 ...

In the second line there are 2 blanks so cut will know that only the following blank right after the backslash is the delimeter, not the rest of the command too. I'd suggest you take the 1st solution as it is more clear to read.

And another more general thing is, that the programs or scripts that generate those pid files, should be able to clean them up themselves. It seems that there is something not working properly and that this script is just a workaround for the real problem.
This User Gave Thanks to zaxxon For This Post:
# 3  
Old 02-28-2011
Referring to post #1 (we'll ignore post #2).
The script contains a fundamental logic error. If there is more than one file matching the pattern *.pid the variable $PID contains multiple values. From that point on the script fails.

I'm not quite clear what you are trying to do here. Reading betweeen the lines, beware that deleting a ".pid" file will not stop the process with that Process ID.

The script needs restructuring to process each file in turn.

e.g.

Code:
#!/bin/bash
#
# Script to delete specific file older than N minutes.
# OLDERTHAN="40" #40 minutes

FOLDER="home/optima/pids/"
ls -1d "${FOLDER}"/*.pid 2>/dev/null | while read PID
do
OLDERTHAN="40"
if [ -e ${FOLDER}/${PID} ]
then
     ls_time=`ls -l ${FOLDER}/$PID`
     pid_h=`echo $ls_time | cut -d' ' -f8 | cut -d\: -f1`
     pid_m=`echo $ls_time | cut -d' ' -f8 | cut -d\: -f2`
     echo pid time=$pid_h:$pid_m
     let pid_time=(10#$pid_h*60)+10#$pid_m
     curr_h=`date | cut -d' ' -f4 | cut -d\: -f1`
     curr_m=`date | cut -d' ' -f4 | cut -d\: -f2`
     let curr_time=(10#$curr_h*60)+10#$curr_m
     echo curr_time=`date | cut -d' ' -f4`
     let diff=10#$curr_time-10#$pid_time
     #echo pid_time=$pid_time
     #echo curr_time=$curr_time
     echo diff=$diff minutes
     if [ $diff -ge $OLDERTHAN ]
     then
          echo "${PID} is older than $OLDERTHAN minutes"
          echo "Deleting ${PID}..."
          # Remove echo when thoroughly tested
          echo rm -f ${FOLDER}/${PID}
     else
          echo -e "${PID} is not older than $OLDERTHAN minutes"
     fi
else
     echo -e "${PID} not found."
fi
done

Keep the "rm" line as an "echo" until you are happy that your script does what you want.
Noted zaxxon comments and corrected "cut" statements.
Noted that the "date" line could be better but didn't change it.

Last edited by methyl; 02-28-2011 at 08:40 AM.. Reason: layout correct cut syntax
This User Gave Thanks to methyl For This Post:
# 4  
Old 02-28-2011
can be done with 1 find command

Code:
 
find /home/somefolder/pids -maxdepth 1 -type f -name "*.pid" -amin +40 -exec rm {} \;

This User Gave Thanks to xoops For This Post:
# 5  
Old 02-28-2011
Change it to -mmin (as -amin changes also when someone issues a read like cat on it) and xoops solution will be best, if your find supports that switch. Smilie Though you might check out the commented things by Methyl and me to reduce errors in further shell scripts.
This User Gave Thanks to zaxxon For This Post:
# 6  
Old 02-28-2011
Quote:
Originally Posted by zaxxon
And another more general thing is, that the programs or scripts that generate those pid files, should be able to clean them up themselves. It seems that there is something not working properly and that this script is just a workaround for the real problem.

:-) You got me there mate. This is a work around. I am trying to support my customer who is on the other side of the planet. And I cant have a remote session on his production server due to their IT Policey. I will look into your suggestion and will give you an Update. Thank you very much for your support.

---------- Post updated at 09:19 AM ---------- Previous update was at 08:43 AM ----------

Quote:
Originally Posted by methyl
Referring to post #1 (we'll ignore post #2).
The script contains a fundamental logic error. If there is more than one file matching the pattern *.pid the variable $PID contains multiple values. From that point on the script fails.

I'm not quite clear what you are trying to do here. Reading betweeen the lines, beware that deleting a ".pid" file will not stop the process with that Process ID.

The script needs restructuring to process each file in turn.

e.g.

Code:
#!/bin/bash
#
# Script to delete specific file older than N minutes.
# OLDERTHAN="40" #40 minutes
 
FOLDER="home/optima/pids/"
ls -1d "${FOLDER}"/*.pid 2>/dev/null | while read PID
do
OLDERTHAN="40"
if [ -e ${FOLDER}/${PID} ]
then
     ls_time=`ls -l ${FOLDER}/$PID`
     pid_h=`echo $ls_time | cut -d' ' -f8 | cut -d\: -f1`
     pid_m=`echo $ls_time | cut -d' ' -f8 | cut -d\: -f2`
     echo pid time=$pid_h:$pid_m
     let pid_time=(10#$pid_h*60)+10#$pid_m
     curr_h=`date | cut -d' ' -f4 | cut -d\: -f1`
     curr_m=`date | cut -d' ' -f4 | cut -d\: -f2`
     let curr_time=(10#$curr_h*60)+10#$curr_m
     echo curr_time=`date | cut -d' ' -f4`
     let diff=10#$curr_time-10#$pid_time
     #echo pid_time=$pid_time
     #echo curr_time=$curr_time
     echo diff=$diff minutes
     if [ $diff -ge $OLDERTHAN ]
     then
          echo "${PID} is older than $OLDERTHAN minutes"
          echo "Deleting ${PID}..."
          # Remove echo when thoroughly tested
          echo rm -f ${FOLDER}/${PID}
     else
          echo -e "${PID} is not older than $OLDERTHAN minutes"
     fi
else
     echo -e "${PID} not found."
fi
done

Keep the "rm" line as an "echo" until you are happy that your script does what you want.
Noted zaxxon comments and corrected "cut" statements.
Noted that the "date" line could be better but didn't change it.

Thanks methyl. That was very educating. You are right about the process thingy. I might end up getting a zombie process.

Let me explain what is going on a bit. There is a process, or a program, called the Process Monitor. Its job is to remove the failed programs.

Whenever a process is initiated, it generates an associated monitor file (.i.e. <exename>_<programid>.pid) file. The purpose of this file is to make sure that the new instances of a program are not started if the previous instance is still running.

Before a program starts an instance, it checks if an associated monitor file exists. If one does exist, then this indicates that an instance is already running and so the program immediately exits. If a monitor file does not exist, the program starts and creates a monitor file. This file is uniquely associated to the program instance using the PRID in a common directory (the default is $OPTIMA/PIDS). When the program has run, it removes the monitor file. The Process Monitor ensures that monitor files are removed if programs crash or hang.

By the way, I tried the following set of lines and it worked but with a catch.


Code:
#!/bin/ksh
# filetimes
filetime()
{
perl -e '
$mtime = (stat $ARGV[0])[9];
print $mtime
' $1
}
now=$(date +%s)
limit=$(( $now - 1800 )) # one half hour ago.
find /home/optima/pids -type f |\
while read filename 
do
dt=$(filetime $filename)
if [[ $dt -lt $limit ]] ; then
rm -f $filename # delete ALL files older then 30 minutes
fi
done


I have realized that the PIDS in /pids folder have the same updated time like the below figs

Quote:
total 96
Quote:
-rw-r----- 1 root sys 0 Feb 28 09:00 test.pid
-rwxr-xr-x 1 root sys 15 Feb 28 23:52 optimamd_opx_LOD_GEN_110_00111010L.pid
-rwxr-xr-x 1 root sys 15 Feb 28 23:52 optimamd_opx_LOD_GEN_110_001110107.pid
-rwxr-xr-x 1 root sys 15 Feb 28 23:52 optimamd_opx_LOD_GEN_110_001110109.pid
-rwxr-xr-x 1 root sys 16 Feb 28 23:52 optimamd_opx_LOD_GEN_110_00111010H.pid
-rwxr-xr-x 1 root sys 15 Feb 28 23:52 optimamd_opx_LOD_GEN_110_00111009A.pid
-rwxr-xr-x 1 root sys 15 Feb 28 23:52 optimamd_opx_LOD_GEN_110_001110106.pid
#


the script will not be able to delete anyone of the pids since the time is same and the loader will see that another process is running. I have hit the wall here Smilie
any advice dear?




---------- Post updated at 09:23 AM ---------- Previous update was at 09:19 AM ----------

Quote:
Originally Posted by xoops
can be done with 1 find command

Code:
 
find /home/somefolder/pids -maxdepth 1 -type f -name "*.pid" -amin +40 -exec rm {} \;

Thanks mate. I tried this before even posting here. Particular flavor of the solaris dosnt support -amin or -mmin of the find command

---------- Post updated at 09:38 AM ---------- Previous update was at 09:23 AM ----------

Standard Unix file systems don't store creation time. The only times
Unix stores for files are access time, modification time, and inode
change time.

Quote:
total 96
-rw-r----- 1 root sys 0 Feb 28 09:00 test.pid
-rwxr-xr-x 1 root sys 15 Feb 28 23:52 optimamd_opx_LOD_GEN_110_00111010L.pid
-rwxr-xr-x 1 root sys 15 Feb 28 23:52 optimamd_opx_LOD_GEN_110_001110107.pid
-rwxr-xr-x 1 root sys 15 Feb 28 23:52 optimamd_opx_LOD_GEN_110_001110109.pid
-rwxr-xr-x 1 root sys 16 Feb 28 23:52 optimamd_opx_LOD_GEN_110_00111010H.pid
-rwxr-xr-x 1 root sys 15 Feb 28 23:52 optimamd_opx_LOD_GEN_110_00111009A.pid
-rwxr-xr-x 1 root sys 15 Feb 28 23:52 optimamd_opx_LOD_GEN_110_001110106.pid
Oh GOD help ... I not cut for this Smilie

Last edited by sajid.shah; 02-28-2011 at 11:08 AM..
# 7  
Old 02-28-2011
so ,can you use "touch" to change the access time?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

UNIX script for cleanup

Hello, I need some help from unix guru's here..I am looking for some advanced level script to cleanup the directories and files from specific directories under a file system.. The folders are created under /opt/modules And under modules, there are multiple subfolders with the application... (6 Replies)
Discussion started by: mb525
6 Replies

2. Shell Programming and Scripting

Suggestion with script to cleanup

I need help with sed and awk scripts to search for Symmetrix ID=000090009902 and then grep its child disk devices associated to the dead paths and display them only, so that those dead devices can be removed. test01:/#powermt display dev=all Pseudo name=hdiskpower0 Symmetrix ID=000090009902... (0 Replies)
Discussion started by: aix_admin_007
0 Replies

3. UNIX for Advanced & Expert Users

Table Cleanup Script

I needed some help with a script to fetch and delete all records prior to 3 days from now connecting to sybase from sunos. I wrote the following script but not working..can someone please guide me with my code. Thanks #!/bin/ksh ##GET PREVIOUS DAY DATE dt=`date | awk... (3 Replies)
Discussion started by: moe458
3 Replies

4. Shell Programming and Scripting

Mail cleanup from ksh script, keeping 50 most recent msgs

I found some posts describing how to completely clean out a mailbox in Unix/Linux. But I want to keep the 50 most recent messages. Any ideas out there? Thanks! (3 Replies)
Discussion started by: OPTIMUS_prime
3 Replies

5. Shell Programming and Scripting

Suggestions/cleanup Bash script

Hello, beginner bash scripter here.. I was able to write a script and it works just fine. I'm just wondering if someone could chime in or any suggestions to make it cleaner or tighter so to speak. I have a disk to disk backup solution which uses 250GB disks. When one gets full I just po in a new... (7 Replies)
Discussion started by: woodson2
7 Replies

6. Shell Programming and Scripting

Cleanup script

Hi! I would like to write a script which remove some files, all beginning with the same prefix : prefix.1 doc/prefix.2 ../prefix.3 etc. So, I would create a file and chmod it executable. But I dont know how to pass a variable to a script. I would like to write something like ... (2 Replies)
Discussion started by: tipi
2 Replies

7. Shell Programming and Scripting

User Cleanup Script

Hi Guys, I've got an system setup to act as an sftp server. I have a script that allows me to create chroot users running a custom shell within their home directory, it also creates a subdirectory that they can write into. I'm trying to write a script (that I can cron at a later date) that checks... (3 Replies)
Discussion started by: King_Brucie
3 Replies

8. Shell Programming and Scripting

awk/sed/ksh script to cleanup /etc/group file

Many of my servers' /etc/group file have many userid's that does not exist in /etc/passwd file and they need to be deleted. This happened due to manual manipulation of /etc/passwd files. I need to do this for 40 servers. Can anyone help me in achieving this? Even reducing a step or two will be... (6 Replies)
Discussion started by: pdtak
6 Replies

9. UNIX for Dummies Questions & Answers

Session PID & socket connection pid

1. If I use an software application(which connects to the database in the server) in my local pc, how many PID should be registered? Would there be PID for the session and another PID for socket connection? 2. I noticed (through netstat) that when I logged in using the my software application,... (1 Reply)
Discussion started by: pcx26
1 Replies

10. Programming

printing ppid,child pid,pid

question: for the below program i just printed the value for pid, child pid and parent pid why does it give me 6 values? i assume ppid is 28086 but can't figure out why there are 5 values printed instead of just two! can someone comment on that! #include<stdio.h> #define DIM 8 int... (3 Replies)
Discussion started by: a25khan
3 Replies
Login or Register to Ask a Question