Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Delete files based on specific MMDDYYYY pattern in filename

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete files based on specific MMDDYYYY pattern in filename
# 1  
Old 08-08-2011
Delete files based on specific MMDDYYYY pattern in filename
Hi Unix gurus,

I am trying to remove the filenames based on MMDDYYYY in the physical name as such so that the directory always has the recent 3 files based on MMDDYYYY. "HHMM" is just dummy in this case. You wont have two files with different HHMM on the same day.


For example in a directory, I have files like
Code:
OPEN_INV_01012011_1345.xls
OPEN_INV_01022011_1230.xls
OPEN_INV_01032011_1145.xls
OPEN_INV_01042011_2456.xls
OPEN_INV_01012011_3456.txt
OPEN_INV_01022011_1134.txt
OPEN_INV_01032011_0812.txt
OPEN_INV_01042011_3467.txt

When I run the script it should delete the OPEN_INV_01012011_1345.xls and OPEN_INV_01012011_3456.txt

Note that the before the file extension, we always have "MMDDYYYY_HHMM"

I am using the following script:
This is what I am trying to do;

Code:
#!/usr/bin/ksh

archivedir=/opt/data/files/archive

typeset -i MAX_ARCHIVE_COUNT
typeset -i archive_file_count
typeset -i remove_archive_count

cd $archivedir
for archpref in $(ls | sed 's/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]\.[^.]*$//' | sort | uniq)
do
  archive_file_count=$(ls -1t ${archpref}* | wc -l)
  MAX_ARCHIVE_COUNT=3
  remove_archive_count=${archive_file_count}-${MAX_ARCHIVE_COUNT}
  if [ ${remove_archive_count} -gt 0 ]
  then
# List the files in date order (most recent first), suppress the first 3, and delete the rest
    rm $(ls -1rt | tail -${remove_archive_count})
  fi
done



Any help is greatly appreciated.

Thanks
Last edited by Scott; 08-09-2011 at 01:04 PM.. Reason: Added code tags
# 2  
Old 08-08-2011
Shankar, I think your problem is on this line:

Code:
for archpref in $(ls | sed 's/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]\.[^.]*$//' | sort | uniq)

Try running the commands in the parenthesis - starting with ls up to uniq - on the command line and see if it picks up any files. I believe the sed command should look like:
Code:
sed 's/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]_[0-9]*\.[^.]*$//'

Or, better still:
Code:
sed 's/[0-9]*\_[0-9]*\.[a-z]*//'

# 3  
Old 08-09-2011
G.pi,

I tried all your suggestions and it is still not working. It considers .txt and .xls as same set and removes 5 files leaving the count as 3.

The idea is to leave 3 files for each set (.xls and .txt).

I think the issue is with the "sort and uniq"

Current it shows the archnt=10 and remove_count=7. Which is wrong as it combines both extensions.
It should actually be archnt=4 and remove_count=1 for each set.


Please if anyone can throw some ideas or modify the sed command will be great.

Thanks in advance
# 4  
Old 08-11-2011
Shankar,

I had overlooked earlier response.

My earlier suggestion was based on your existing script. A quick (easy) solution would be to create 2 for loops, instead of the single one that you have now. The first one should only deal with .txt and the second, with .xls. So in the first for loop, your sed command would look something like this:
Code:
sed 's/[0-9]*\_[0-9]*\.txt//'

Make sure your rm command only removes the .txt files. So your rm command should look like this:
Code:
rm $(ls -1rt *.txt | tail -${remove_archive_count})

The second for loop should be identical, except, it will only deal with .xls.

Hope that helps.

- GP
This User Gave Thanks to g.pi For This Post:
vivek.goel.piet
# 5  
Old 08-12-2011
we can achieve this with sort and grep command
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete lines based on pattern match

BASH in Solaris 10 I have a log file like below. Whenever the pattern ORA-39083 is encountered, I want to delete the line which has this pattern and 3 lines below it. $ cat someLogfile.txt ORA-39083: Object type OBJECT_GRANT failed to create with error: ORA-01917: user or role 'CMPA' does... (4 Replies)
Discussion started by: kraljic
4 Replies

2. Shell Programming and Scripting

Delete lines that contain a pattern from specific line to the end.

Gents, I am trying to delete all lines which start with "H" character, but keeping the fist header. Example In the input file I will delete all lines starting from line 8 which contents character "H" to the end of the file. I try sed '8,10000{/^H/d;}' file But as don't know the end... (1 Reply)
Discussion started by: jiam912
1 Replies

3. UNIX for Dummies Questions & Answers

How to find a file based on pattern & return the filename if found?

Hi all, I am a newbie here. I have this requirement to find a file based on a pattern then return the filename if found. I created a script based on online tutorials. Though, I am stuck & really appreciate if anyone can have a quick look & point me to the right direction? #Script starts... (10 Replies)
Discussion started by: buster_t
10 Replies

4. Shell Programming and Scripting

Delete specific lines from files based on another file

I have some text files in a folder named ff as follows. I need to delete the lines (in-place editing)in these files based on another file aa.txt. 32bm.txt: 249 253 A P - 0 0 8 0, 0.0 6,-1.4 0, 0.0 2,-0.4 -0.287 25.6-102.0 -74.4 161.1 37.1 13.3 10.9 250... (2 Replies)
Discussion started by: aden
2 Replies

5. Shell Programming and Scripting

Search for duplicates and delete but remain the first one based on a specific pattern

Hi all, I have been trying to delete duplicates based on a certain pattern but failed to make it works. There are more than 1 pattern which are duplicated but i just want to remove 1 pattern only and remain the rest. I cannot use awk '!x++' inputfile.txt or sed '/pattern/d' or use uniq and sort... (7 Replies)
Discussion started by: redse171
7 Replies

6. Shell Programming and Scripting

Delete line based on count of specific character

I'm looking for what I hope might be a one liner along these lines: sed '/a line with more than 3 pipes in it/d' I know how to get the pipe count in a string and store it in a variable, but I'm greedy enough to hope that it's possible via regex in the /.../d context. Am I asking too much? ... (5 Replies)
Discussion started by: tiggyboo
5 Replies

7. Shell Programming and Scripting

Delete multiple lines starting with a specific pattern

Hi, just tried some script, awk, sed for the last 2 hours and now need help. Let's say I have a huge file of 800,000 lines like this : It's a tedious job to look through it, I'd like to remove those useless lines in it as there's a few thousands : Or to be even more precise : if line1 =... (6 Replies)
Discussion started by: Zurd
6 Replies

8. Shell Programming and Scripting

how to delete lines from a file which starts with a specific pattern

I need to delete those lines from a file, which starts with 45. How to do it? (3 Replies)
Discussion started by: mady135
3 Replies

9. Shell Programming and Scripting

Trimming sequences based on specific pattern

My files look like this And I need to cut the sequences at the last "A" found in the following 'pattern' -highlighted for easier identification, the pattern is the actual file is not highlighted. The expected result should look like this Thus, all the sequences would end with AGCCCTA... (2 Replies)
Discussion started by: Xterra
2 Replies

10. HP-UX

How to delete specific pattern in a file with SED?

I have one file which is having content as following... 0513468211,,,,20091208,084005,5,,2,3699310, 0206554475,,,,20090327,123634,85,,2,15615533 0206554475,,,,20090327,134431,554,,2,7246177 0103000300,,,,20090523,115501,89,,2,3869929 0736454328,,,,20091208,084005,75,,2,3699546... (7 Replies)
Discussion started by: ganesh.mandlik
7 Replies
Login or Register to Ask a Question