Listing files older than 30 days


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Listing files older than 30 days
# 8  
Old 10-24-2017
Quote:
Originally Posted by Home
. . .
Code:
./list.sh: line 11: /abc/datasource/checkstatus/CHECK_STATUS_PARTY_CODE_2017-08-31_114506.txt: Permission denied
./list.sh: line 15: 1508845131-1508796000: command not found
./list.sh: line 15: command not found
./list.sh: line 11: /abc/datasource/checkstatus/CHECK_STATUS_PARTY_CHANGE_2017-10-17_114526.txt: Permission denied
./list.sh: line 15: 1508845243-1508796000: command not found

If people only knew what line 11 and 15 were...

Quote:
Originally Posted by Home
. . .
but I ran the code with ./list.sh -x and results were the same.
-x is not an option to the script but to the shell running the script. Either use set -x or sh -x scriptname.

Don't be that economical with facts about your setup, and, please, read entire posts and answer ALL the questions asked...

Last edited by RudiC; 10-24-2017 at 11:49 AM.. Reason: typo...
This User Gave Thanks to RudiC For This Post:
# 9  
Old 10-25-2017
Thank you all for your replies. I copied the code in notepad++ and here is the lines for code:

Code:
line1 #!/bin/sh 
line2
line3 FOLDER=/abc/datasource/checkstatus
line4 
line5 TIMESTAMP=$(date +%s) 
line6
line7 for filename in $(find $FOLDER 
line8
line9 -maxdepth 1 -type f -name "CHECK_STATUS*"); do
line10
line11 f1=$($filename -Eo "[0-9]{4}[0-9-]+") 
line12
line13 f2=$(date -d "$f1" +%s)
line14
line15 if $"$TIMESTAMP-$f2" -gt 2592000; then 
line16
line17 echo $(basename "$filename")
line18
line19 fi
line20
line21 done
line22
line23 exit 0;

I ran the code with -x option,
Code:
bash -x ./list.sh

and here is the results:

Code:
-maxdepth 1 -type f -name "CHECK_STATUS*")'
++ /abc/datasource/checkstatus/CHECK_STATUS_BALANCE_2017-10-14_114534.txt -Eo '[0-9]{4}[0-9-]+'
./list.sh: line 11: /abc/datasource/checkstatus/CHECK_STATUS_BALANCE_2017-10-14_114534.txt: Permission denied
+ f1=
++ date -d '' +%s
+ f2=1508882400
+ 1508921365-1508882400 -gt 2592000
./list.sh: line 15: 1508921365-1508882400: command not found
+ for filename in '$(find $FOLDER

-maxdepth 1 -type f -name "CHECK_STATUS*")'
++ /abc/datasource/checkstatus/CHECK_STATUS_CODE_CHECK_2017-09-28_114530.txt-kvitto -Eo '[0-9]{4}[0-9-]+'
./list.sh: line 11: /abc/datasource/checkstatus/CHECK_STATUS_PARTY_2017-09-28_114530.txt: Permission denied
+ f1=
++ date -d '' +%s
+ f2=1508882400
+ 1508921365-1508882400 -gt 2592000
./list.sh: line 15: 1508921365-1508882400: command not found
+ for filename in '$(find $FOLDER

-maxdepth 1 -type f -name "CHECK_STATUS*")'
++ /abc/datasource/checkstatus/CHECK_STATUS_CODE_CHECK_2017-10-07_114714.txt-kvitto -Eo '[0-9]{4}[0-9-]+'
./list.sh: line 11: /abc/datasource/checkstatus/CHECK_STATUS_CHANGE_CHECK_2017-10-07_114714.txt: Permission denied
+ f1=
++ date -d '' +%s
+ f2=1508882400
+ 1508921365-1508882400 -gt 2592000
./list.sh: line 15: 1508921365-1508882400: command not found
+ for filename in '$(find $FOLDER

-maxdepth 1 -type f -name "CHECK_STATUS*")'
++ /abc/datasource/checkstatus/CHECK_STATUS_CODE_CHECK_2017-10-12_114530.txt-kvitto -Eo '[0-9]{4}[0-9-]+'
./list.sh: line 11: /abc/datasource/checkstatus/CHECK_STATUS_CODE_STATUS_CHECK_2017-10-12_114530.txt: Permission denied
+ f1=
++ date -d '' +%s
+ f2=1508882400
+ 1508921365-1508882400 -gt 2592000
./list.sh: line 15: 1508921365-1508882400: command not found
+ for filename in '$(find $FOLDER

---------- Post updated at 04:09 AM ---------- Previous update was at 04:03 AM ----------

Quote:
Originally Posted by Aia
It would be helpful if you post a few CHECK_STATUS* entries to examine.
CHECK_STATUS are simple files. Name of files are important and you could create a simple text file with the mentioned name.

Quote:
It would be helpful if you post what the errors are. You can post the output of the code running with the flag -x.
The results has been posted.

Quote:
It would be helpful if you post the os and shell type you are using.
I know the OS is linux, shell type is sh(bash). Please be more specific that what/where I should search.

---------- Post updated at 04:57 AM ---------- Previous update was at 04:09 AM ----------

Quote:
Originally Posted by rbatte1
Would you prefer to re-write this as Perl, or can you alter the f1= statement to meet your needs?
This part of the code is important to extract the date and rest of the code is somehow depended to it. I don't know how can I remove it and get the result that I want.
# 10  
Old 10-25-2017
Quote:
Originally Posted by Home
Thank you all for your replies. I copied the code in notepad++ and here is the lines for code:

Code:
line1 #!/bin/sh 
line2
line3 FOLDER=/abc/datasource/checkstatus
line4 
line5 TIMESTAMP=$(date +%s) 
line6
line7 for filename in $(find $FOLDER 
line8
line9 -maxdepth 1 -type f -name "CHECK_STATUS*"); do
line10
line11 f1=$($filename -Eo "[0-9]{4}[0-9-]+") 
line12
line13 f2=$(date -d "$f1" +%s)
line14
line15 if $"$TIMESTAMP-$f2" -gt 2592000; then 
line16
line17 echo $(basename "$filename")
line18
line19 fi
line20
line21 done
line22
line23 exit 0;

I ran the code with -x option,
Code:
bash -x ./list.sh

and here is the results:

Code:
-maxdepth 1 -type f -name "CHECK_STATUS*")'
++ /abc/datasource/checkstatus/CHECK_STATUS_BALANCE_2017-10-14_114534.txt -Eo '[0-9]{4}[0-9-]+'
./list.sh: line 11: /abc/datasource/checkstatus/CHECK_STATUS_BALANCE_2017-10-14_114534.txt: Permission denied
+ f1=
++ date -d '' +%s
+ f2=1508882400
+ 1508921365-1508882400 -gt 2592000
./list.sh: line 15: 1508921365-1508882400: command not found
+ for filename in '$(find $FOLDER

-maxdepth 1 -type f -name "CHECK_STATUS*")'
++ /abc/datasource/checkstatus/CHECK_STATUS_CODE_CHECK_2017-09-28_114530.txt-kvitto -Eo '[0-9]{4}[0-9-]+'
./list.sh: line 11: /abc/datasource/checkstatus/CHECK_STATUS_PARTY_2017-09-28_114530.txt: Permission denied
+ f1=
++ date -d '' +%s
+ f2=1508882400
+ 1508921365-1508882400 -gt 2592000
./list.sh: line 15: 1508921365-1508882400: command not found
+ for filename in '$(find $FOLDER

-maxdepth 1 -type f -name "CHECK_STATUS*")'
++ /abc/datasource/checkstatus/CHECK_STATUS_CODE_CHECK_2017-10-07_114714.txt-kvitto -Eo '[0-9]{4}[0-9-]+'
./list.sh: line 11: /abc/datasource/checkstatus/CHECK_STATUS_CHANGE_CHECK_2017-10-07_114714.txt: Permission denied
+ f1=
++ date -d '' +%s
+ f2=1508882400
+ 1508921365-1508882400 -gt 2592000
./list.sh: line 15: 1508921365-1508882400: command not found
+ for filename in '$(find $FOLDER

-maxdepth 1 -type f -name "CHECK_STATUS*")'
++ /abc/datasource/checkstatus/CHECK_STATUS_CODE_CHECK_2017-10-12_114530.txt-kvitto -Eo '[0-9]{4}[0-9-]+'
./list.sh: line 11: /abc/datasource/checkstatus/CHECK_STATUS_CODE_STATUS_CHECK_2017-10-12_114530.txt: Permission denied
+ f1=
++ date -d '' +%s
+ f2=1508882400
+ 1508921365-1508882400 -gt 2592000
./list.sh: line 15: 1508921365-1508882400: command not found
+ for filename in '$(find $FOLDER

---------- Post updated at 04:09 AM ---------- Previous update was at 04:03 AM ----------


CHECK_STATUS are simple files. Name of files are important and you could create a simple text file with the mentioned name.


The results has been posted.


I know the OS is linux, shell type is sh(bash). Please be more specific that what/where I should search.

---------- Post updated at 04:57 AM ---------- Previous update was at 04:09 AM ----------



This part of the code is important to extract the date and rest of the code is somehow depended to it. I don't know how can I remove it and get the result that I want.
We had hoped that you would know that you can't randomly split lines in shell script command lines into multiple lines without breaking the script. (Compare the 3 lines in your script above numbered line7, line8, and line9 with the isingle line in your script in post #1 in this thread:
Code:
for filename in $(find $FOLDER -maxdepth 1 -type f -name "CHECK_STATUS*"); do

If you don't understand why there is any difference, type the command:
Code:
find .

into your shell. Then type the command:
Code:
-maxdepth 1 -type f -name ".*"

into your shell. (These two commands are roughly equivalent to what you have in the command substitution shown in your script above.) And, then type the command:
Code:
find . -maxdepth 1 -type f -name ".*"

into your shell. (Which is roughly equivalent to what you had in post #1 in this thread.) Do you see the difference in the output produced?

In post #5, when Aia said:
Quote:
It would be helpful if you post a few CHECK_STATUS* entries to examine.
he wasn't asking you to show us the contents of those files. He was asking for the format of the naming convention used to name the files. If we have a clear description of the names that will be matched by the pattern CHECK_STATUS*, we can help you make your script MUCH more efficient. From the two names you supplied in later posts:
Code:
CHECK_STATUS_PARTY_CHANGE_2017-10-17_114526.txt
CHECK_STATUS_PARTY_CODE_2017-08-31_114506.txt

we might guess that the naming format is something like:
  1. The literal string CHECK_STATUS_,
  2. followed by an arbitrary string ending with an underscore character,
  3. followed by a ten character representation of a date in the format YYYY-MM-DD,
  4. followed by an underscore character,
  5. followed by six decimal digit characters,
  6. followed by the literal string .txt.
If the above is an accurate representation of the rules defining the names of the files you will be processing, there is no need to run find nor is there any need for what I assume was an attempt to run grep -Eo to extract the date from the filename.

You could also speed up your script and make it more efficient by moving into the directory that contains your files instead of processing absolute pathnames and then throwing away the directory hierarchy before printing the filename after determining that your date criteria are met.

If, in addition to the above comments, you also know that all files in the directory /abc/datasource/checkstatus with filenames ending in .txt are regular files, you might want to try the following simpler script to see if it comes close to doing what you want:
Code:
#!/bin/sh
cd /abc/datasource/checkstatus || exit
 
S30DAY=$((60*60*24*30))		# Seconds in 30 days.
TIMESTAMP=$(date '+%s')		# Seconds since the Epoch.

for filename in CHECK_STATUS*.txt
do	f1=${filename%_*}	# drop last "_" and everything after it
	f1=${f1##*_}		# drop everything before last "_" and it

	f2=$(date -d "$f1" '+%s')

	[ $((TIMESTAMP - f2)) -gt S30DAY ] && printf '%s\n' "$filename"
done

If all that we know is that the filenames start with the literal string CHECK_STATUS followed by an arbitrary string followed by a date followed by another arbitrary string, we could clean up your script a little bit to something like the following (using grep the way I think you intended to use it):
Code:
#!/bin/sh
cd /abc/datasource/checkstatus || exit
 
S30DAY=$((60*60*24*30))		# Seconds in 30 days.
TIMESTAMP=$(date '+%s')		# Seconds since the Epoch.

for filename in CHECK_STATUS*
do
	[ ! -f "$filename" ] && continue

	f1=$(printf '%s\n' "$filename" | grep -Eo '[0-9]{4}-[01][0-9]-[0-3][0-9]')

	f2=$(date -d "$f1" +%s)

	[ $((TIMESTAMP - f2)) -gt S30DAY ] && printf '%s\n' "$filename"
done

# 11  
Old 10-26-2017
Quote:
Originally Posted by Don Cragun
Code:
[ $((TIMESTAMP - f2)) -gt $S30DAY ] && printf '%s\n' "$filename"

-gt will need that to be expanded to an integer.
This User Gave Thanks to Aia For This Post:
# 12  
Old 10-26-2017
Quote:
Originally Posted by Aia
-gt will need that to be expanded to an integer.
Hi Aia,
Thanks for spotting that.

On the system I'm using, I don't have date -d and the bash on my system doesn't recognize printf '%(%s)T\n' YYYY-MM-DD. Therefore I tested the code I suggested using ksh. And, ksh version 93u+ expands the S30DAY variable in that test statement with or without the dollar sign.

- Don
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. HP-UX

listing processes older than n days

Hello; trying to find processes older than n days, mostly user shells Tried the following code on 11.31 box: in this case older than 5 days UNIX95= ps -ef -o user,pid,ppid,cpu,etime,stime | grep "-" | awk '{print $2}' | xargs ps -ef|grep -v '?' |\ awk '$5 !~ ""' | awk '($5 ~ "$(date "+%b")")... (6 Replies)
Discussion started by: delphys
6 Replies

2. UNIX for Dummies Questions & Answers

Files older than 50 days

Hi All, OS :- HP-UX wm5qa B.11.23 U ia64 1119805695 unlimited-user license I need to search files older than 50 days. I've used following command in order to search desired files, I also discoverd, it's showing today's files as well. Do you have any clue with this ? wmqa1> find .... (4 Replies)
Discussion started by: alok.behria
4 Replies

3. Shell Programming and Scripting

deleting files older than 7 days

Hi Guys, I am new to unix I am looking for a script to delete files older than 7 days but i also want to exclude certain directories (like arch,log .....) and also some files with extensions ( like .ksh, .ch, ..............) Thanks (1 Reply)
Discussion started by: MAYAMAYA0451
1 Replies

4. Shell Programming and Scripting

How to find files older than 30 days

Dear Friends, I have two queries. 1) I want to see the list of folders which were created 29 days ago. 2) I want to see the folders in which last created file is older than 29 days. Can it be done? Thank you in advance Anushree (4 Replies)
Discussion started by: anushree.a
4 Replies

5. Shell Programming and Scripting

delete files more than 15 days older

i have to delete files which are older than 15 days or more except the ones in the directory Current and also *.sh files i have found the command for files 15 days or more older find . -type f -mtime +15 -exec ls -ltr {} \; but how to implement the logic to avoid directory Current and also... (3 Replies)
Discussion started by: ali560045
3 Replies

6. Shell Programming and Scripting

Deleting files older than 7 days

Hi Guys, I want to delete folder/files older than 7 days. Im using the command below. find /test/test1 -mtime +7 -print0 | xargs -0 rm -Rf /test/test1/* which works ok, but it deletes the test1 folder as well which i dont want. The test1 folder will have a list of sub-folders which in... (4 Replies)
Discussion started by: shezam
4 Replies

7. UNIX for Dummies Questions & Answers

Find files older than 5 days and remove tem after listing

need help with this ... Find files older than 5 days and remove tem after listing list "test" file older than 5 days and then remove them (1 Reply)
Discussion started by: ypatel6871
1 Replies

8. UNIX for Dummies Questions & Answers

How can I delete files older than 7 days?

I will like to write a script that delete all files that are older than 7 days in a directory and it's subdirectories. Can any one help me out witht the magic command or script? Thanks in advance, Odogboly98:confused: (3 Replies)
Discussion started by: odogbolu98
3 Replies

9. Shell Programming and Scripting

Listing files older than 2 months

A script or command to list files older than 2 months in a specified directory and remove it. (3 Replies)
Discussion started by: pbekal
3 Replies
Login or Register to Ask a Question