Comparing filename-substrings and remove unnecessary files - UNIX for Dummies Questions & Answers

cypher82 · #1 05-19-2008

hi folks...
i have to write a sript that removes unnecessary backup-files.
iam new to shell scripting so please be patient with me. and no its not homework

these files look like "javacore303330.1209029863.txt" where the first number is the PID and the second is the timestamp. so there can be a lot javacores with the same PID but i need only the 2 latest ones.
before:

Code:

javacore479430.1208522264.txt
javacore479430.1208522097.txt
javacore479430.1208521935.txt
javacore479430.1208521772.txt
javacore540754.1209554863.txt
javacore540754.1209551357.txt
javacore540754.1209551130.txt

after:

Code:

javacore479430.1208522264.txt
javacore479430.1208522097.txt
javacore540754.1209554863.txt
javacore540754.1209551357.txt

My basic approach looks like that:

Code:

#comparing every file with every file
#and inc. the counter
for ((  i = 1;  i <= ${number_of_files};  i++  ))
do 	for ((  j = 1;   j<= ${number_of_files-1};  j++  ))
         do
             if (pid_file.i = pid_file.j+1)
            and if counter<=2
            do counter++
            else remove_file file.j

my attempt:

Code:

#pid >> list
ls -t|grep javacore |cut -c 9-14  >> /home/user/list.txt
chmod a+x /home/user/list.txt

number_of_files=$(cat "/home/de151738/liste.txt" | wc -w)

#now the tough part
for ((  i = 1;  i <= ${number_of_files};  i++  ))
do 	for ((  j = 1;   j<= ${number_of_files-1};  j++  ))
         do
 #?        if {list.line.i = list.line.j}
 #           and if counter<=2 
 #          do   counter++
 #        else rm file.j

If anybody has some advice I would be glad. Maybe my basic approach could be already wrong

jim mcnamara · #2 05-19-2008

This is one way - probably not the most efficient:

Code:

#!/bin/ksh
# t.awk
awk -F. '{ print $1 }' filename | sort -u | \
while read corefile
do
     grep "$corefile" filename | tail -2
done

Code:

/home/jmcnama> cat filename
javacore479430.1208522264.txt
javacore479430.1208522097.txt
javacore479430.1208521935.txt
javacore479430.1208521772.txt
javacore540754.1209554863.txt
javacore540754.1209551357.txt
javacore540754.1209551130.txt

/home/jmcnama> t.awk
javacore479430.1208521935.txt
javacore479430.1208521772.txt
javacore540754.1209551357.txt
javacore540754.1209551130.txt

shamrock · #3 05-19-2008

What happens when there is a single javacore file corresponding to a certain PID. How do you handle that special case.

cypher82 · #4 05-26-2008

Hi!

Thanks for your answer, Jim. But something i did not understand. Your code just touches the strings, right? ('print $1') And not the files itself, or am I wrong? Maybe i have not understood the awk-command

@shamrock
that´s a good question too

cypher82 · #5 06-02-2008

----------up!----------------

cypher82 · #6 06-06-2008

Hello...
So thats my solution so far.
My only problem is that if iam logged in as root, it will delete all the files.
If iam logged in as normal user it works as it should.
This script is part of a cronjob so it must be root who will execute that.
Has anybody a solution for that problem?

Code:

#!/bin/ksh

clear
echo Deleted Files
echo '#############'
echo ' '

###create lists

cd /tmp/javacores
ls -t|grep "^[h-jH-J]" >> /tmp/javacores/listbefore.txt

awk -F. '{ print $1 }' /tmp/javacores/listbefore.txt | sort -u | 
while read corefile
do
     grep "$corefile" /tmp/javacores/listbefore.txt | tail -2  >> /tmp/javacores/listafter.txt
done

###compare lists

comm -13 listafter.txt listbefore.txt >> listcmp.txt

###remove files

for i in `cat listcmp.txt`
do

rm -r /tmp/javacores/$i
echo $i was deleted !
done
echo ' '

rm /tmp/javacores/listbefore.txt
rm /tmp/javacores/listafter.txt
rm /tmp/javacores/listcmp.txt

###write log-file
touch date`date +%F-%H-%M-%S`.log

Btw its not only javacores which should be deleted. "Heapdumps" also.
That´s why the

Code:

ls -t|grep "^[h-jH-J]"

.

Thanks in advance!

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
ksh: Comparing strings that contain spaces and working with substrings	nancylt723	Shell Programming and Scripting	2	02-19-2008 12:01 PM
Remove path from filename	borgeh	UNIX for Dummies Questions & Answers	3	08-23-2007 07:58 AM
remove filename prefix	peter.herlihy	UNIX for Dummies Questions & Answers	2	07-10-2002 10:08 PM
remove pound sign from filename	kristy	UNIX for Dummies Questions & Answers	4	02-13-2002 12:31 PM
remove unnecessary lines	nazri	UNIX for Dummies Questions & Answers	3	11-23-2001 05:06 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode