Comparing filename-substrings and remove unnecessary files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Comparing filename-substrings and remove unnecessary files
# 1  
Old 05-19-2008
Question Comparing filename-substrings and remove unnecessary files

hi folks...
i have to write a sript that removes unnecessary backup-files.
iam new to shell scripting so please be patient with me. and no its not homework Smilie
these files look like "javacore303330.1209029863.txt" where the first number is the PID and the second is the timestamp. so there can be a lot javacores with the same PID but i need only the 2 latest ones.
before:
Code:
javacore479430.1208522264.txt
javacore479430.1208522097.txt
javacore479430.1208521935.txt
javacore479430.1208521772.txt
javacore540754.1209554863.txt
javacore540754.1209551357.txt
javacore540754.1209551130.txt

after:
Code:
javacore479430.1208522264.txt
javacore479430.1208522097.txt
javacore540754.1209554863.txt
javacore540754.1209551357.txt

My basic approach looks like that:
Code:
#comparing every file with every file
#and inc. the counter
for ((  i = 1;  i <= ${number_of_files};  i++  ))
do 	for ((  j = 1;   j<= ${number_of_files-1};  j++  ))
         do
             if (pid_file.i = pid_file.j+1)
            and if counter<=2
            do counter++
            else remove_file file.j

my attempt:
Code:
#pid >> list
ls -t|grep javacore |cut -c 9-14  >> /home/user/list.txt
chmod a+x /home/user/list.txt

number_of_files=$(cat "/home/de151738/liste.txt" | wc -w)

#now the tough part
for ((  i = 1;  i <= ${number_of_files};  i++  ))
do 	for ((  j = 1;   j<= ${number_of_files-1};  j++  ))
         do
 #?        if {list.line.i = list.line.j}
 #           and if counter<=2 
 #          do   counter++
 #        else rm file.j

If anybody has some advice I would be glad. Maybe my basic approach could be already wrong Smilie
# 2  
Old 05-19-2008
This is one way - probably not the most efficient:
Code:
#!/bin/ksh
# t.awk
awk -F. '{ print $1 }' filename | sort -u | \
while read corefile
do
     grep "$corefile" filename | tail -2
done

Code:
/home/jmcnama> cat filename
javacore479430.1208522264.txt
javacore479430.1208522097.txt
javacore479430.1208521935.txt
javacore479430.1208521772.txt
javacore540754.1209554863.txt
javacore540754.1209551357.txt
javacore540754.1209551130.txt

/home/jmcnama> t.awk
javacore479430.1208521935.txt
javacore479430.1208521772.txt
javacore540754.1209551357.txt
javacore540754.1209551130.txt

# 3  
Old 05-19-2008
Question Single javacore file

What happens when there is a single javacore file corresponding to a certain PID. How do you handle that special case.
# 4  
Old 05-26-2008
Hi!

Thanks for your answer, Jim. But something i did not understand. Your code just touches the strings, right? ('print $1') And not the files itself, or am I wrong? Maybe i have not understood the awk-command Smilie

@shamrock
thatīs a good question too Smilie
# 5  
Old 06-02-2008
----------up!----------------
# 6  
Old 06-06-2008
Tools

Hello...
So thats my solution so far.
My only problem is that if iam logged in as root, it will delete all the files.
If iam logged in as normal user it works as it should.
This script is part of a cronjob so it must be root who will execute that.
Has anybody a solution for that problem?

Code:
#!/bin/ksh

clear
echo Deleted Files
echo '#############'
echo ' '

###create lists

cd /tmp/javacores
ls -t|grep "^[h-jH-J]" >> /tmp/javacores/listbefore.txt

awk -F. '{ print $1 }' /tmp/javacores/listbefore.txt | sort -u | 
while read corefile
do
     grep "$corefile" /tmp/javacores/listbefore.txt | tail -2  >> /tmp/javacores/listafter.txt
done

###compare lists

comm -13 listafter.txt listbefore.txt >> listcmp.txt

###remove files

for i in `cat listcmp.txt`
do

rm -r /tmp/javacores/$i
echo $i was deleted !
done
echo ' '

rm /tmp/javacores/listbefore.txt
rm /tmp/javacores/listafter.txt
rm /tmp/javacores/listcmp.txt

###write log-file
touch date`date +%F-%H-%M-%S`.log

Btw its not only javacores which should be deleted. "Heapdumps" also.
Thatīs why the
Code:
ls -t|grep "^[h-jH-J]"

. Smilie

Thanks in advance!
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script that should remove unnecessary commas between double quotes in CSV file

i have data as below 123,"paul phiri",paul@yahoo.com,"po.box 23, BT","Eco Bank,Blantyre,Malawi" i need an output to be 123,"paul phiri",paul@yahoo.com,"po.box 23 BT","Eco Bank Blantyre Malawi" (5 Replies)
Discussion started by: mathias23
5 Replies

2. Shell Programming and Scripting

How to remove sections of a filename?

Hello, I need some help with renaming some files by removing a certain portion of the filename. The current file name is: ABC_2013186197_20130708_203556.95336 I need to remove the 5 digits after the first "_". The new file name should be: ABC_197_20130708_203556.95336 I'm not quite... (5 Replies)
Discussion started by: bbbngowc
5 Replies

3. Shell Programming and Scripting

Comparing multiple substrings for a match

I have a tab-delimited file containing a large genetic dataset with binary base calls, in this format: Chr7 26021407 1/1:0,0,0:5 1/1:0,0,0:5 1/1:0,0,0:5 Chr7 26022023 1/1:0,0,0:3 1/1:0,0,0:3 1/1:28,3,0:5 Chr7 26022087 1/1:0,0,0:6 1/1:25,3,0:9 1/1:25,3,0:9 Chr7 26022656 1/1:0,0,0:3... (1 Reply)
Discussion started by: ljk
1 Replies

4. Shell Programming and Scripting

Remove parts from a filename

I want to remove the beginning and end of a filename and keep the middle. E.g. tempblast7114_1#21110932.out_ the current filename I want it to be called 7114_1#21 only How would I do this?? (3 Replies)
Discussion started by: avonm
3 Replies

5. Solaris

to clean unnecessary files and folders

I want to clean used solaris OS and then to give another developer. How can I understand that difference between system file/folder and others. I want to delete apart from the files/folders. I need tools or scripts like disk-cleanup or something like that. #usr>du -s -h * 6.6M 4lib ... (1 Reply)
Discussion started by: getrue
1 Replies

6. Shell Programming and Scripting

Filename from splitting files to have the same filename of the original file with counter value

Hi all, I have a list of xml file. I need to split the files to a different files when see the <ko> tag. The list of filename are B20090908.1100-20090908.1200_CDMA=1,NO=2,SITE=3.xml B20090908.1200-20090908.1300_CDMA=1,NO=2,SITE=3.xml B20090908.1300-20090908.1400_CDMA=1,NO=2,SITE=3.xml ... (3 Replies)
Discussion started by: natalie23
3 Replies

7. Shell Programming and Scripting

substrings from all files incl subdirs into csv with dir names

Greetings! I have multiple files, one per subdirectory, all with the same file name. All subdirectories are one level deep from the main directory. The data in the files is tab delimited between fields and record delimited with a newline. The subdirectory names have the date in the... (5 Replies)
Discussion started by: vtischuk@yahoo.
5 Replies

8. UNIX for Advanced & Expert Users

remove unnecessary comma from file

HI all, I have a file with following data - test1 "ABC,D",1234,"XYZ,QWER",1234 "SZXA",9870,"ASD,QWERT",234 "XZ,SD",9478,"ADCS,AXZ",876 "WESR",8764,"AQZXAS",9888 "WESR",9898,"WESDRTSAW",3323 I need to get rid of unnecessary commas in fields having double quotes. Ouput - ... (1 Reply)
Discussion started by: sumeet
1 Replies

9. Shell Programming and Scripting

ksh: Comparing strings that contain spaces and working with substrings

Forgive me. I am very new to kornshell scripts. The simplest things stop me dead in my tracks. Here are two such examples. I want to save the first 19 characters of the following string to a variable. "Operation Completed and blah blah blah" I know this works (from another thread): ... (2 Replies)
Discussion started by: nancylt723
2 Replies

10. UNIX for Dummies Questions & Answers

remove unnecessary lines

If I have a file with 5000 lines contains numbers. Some of the number are repeated and some are not. Among those repeated number, I only would like to keep only one. How do I remove those balance repeated number. Your help is much appreciated. Thank you. (3 Replies)
Discussion started by: nazri
3 Replies
Login or Register to Ask a Question