extracting and using date from filenames in a loop


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers extracting and using date from filenames in a loop
# 1  
Old 08-29-2008
Data extracting and using date from filenames in a loop

HIya,

Having a dumb day whilst writing an archive process in Shell want to extract from the filename the date and archive into tar files based on this, I don't want to use mtime as it may not be the actual file date.

The files are

-rw-rw---- 1 user admin 100 Aug 29 11:10 Jav_Transmission_20080826.dat
-rw-rw---- 1 user admin 100 Aug 29 11:10 Jav_Transmission_20080827.dat
-rw-rw---- 1 user admin 100 Aug 29 11:11 Jav_Transmission_20080828.dat
-rw-rw---- 1 user admin 100 Aug 29 11:11 Jav_Transmission_20080829.dat

so for each of the days in the below dir I want to find a way to sort of

for dates in `ls -l |extract distinct file date in thias directory`

tar -cvf ${dates}.tar *${dates}*.dat
mv ${dates}.tar /arc

done

So of thing, any help gratefully received.
# 2  
Old 08-29-2008
Lets start by the dates:
Hint:
ls -ltr|awk '{print $6,$7}' |uniq
# 3  
Old 08-29-2008
In the couple of hours between submitting question for moderation and posting I have written the following

Code:
#Get list of unique dates for which we have files for
for dates in `ls *0* |grep -v $TODAY|grep -v $TODAYmin1|grep -v $TODAYmin2|grep -v $TODAYmin3| cat | tr "_" " " | awk '{print substr($3,1,8)}' | sort -u`
do
#Tar into daily archives
tar -cvf ${dates}.tar *${dates}*.dat
#Compress it
gzip ${dates}.tar

#Check the arc directory exists or make it
if [ -d $ARC_DIR ]
then
echo "This dirname [$ARC_DIR] exists"
else
echo "[$ARC_DIR] does not exist creating"
mkdir arc
fi 

#Move it into Arc directory
mv ${dates}.tar.gz arc/
#exit inner loop
done

#Now we must delete files outside of the retention period $3
#find arc/*.gz -mtime +{$RETENTION_PERIOD} -exec rm {} \;
find arc/*.gz -mtime +{$DAILY_RETENTION_PERIOD} -exec ls -rtl {} \;

cd ${BASE_DIR}
#loop

#exit outer loop
done

Which works fine, but thank you anyways.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Appending date to UNIX Filenames

Hello, I have a file name in the below format and have to append the date as _$currdate. kchik_UK_lo.txt_$currdate. The above should be the format and I dont want to put entire filename as above in the code, but it should give me the output as the above filename.Can anyone please help... (7 Replies)
Discussion started by: harika03
7 Replies

2. Shell Programming and Scripting

For/While Loop to Increment Filenames in a Script

Daily stupid question. I want to increment the file name everytime the script is run. So for example if the filename is manager.log and I run the script, I want the next sequence to be manager.log1. So to be clear I only want it to increment when the script is executed. So ./script... (10 Replies)
Discussion started by: metallica1973
10 Replies

3. Shell Programming and Scripting

Generate filenames in a loop

Hi all, I want to generate output files in a loop, run the same command on the same input file 1000 times and output in files with a new name each time, maybe a number appended to it. The output will be different each time as I`m sampling randomly from the input file. I want to do the... (3 Replies)
Discussion started by: newbie83
3 Replies

4. UNIX for Dummies Questions & Answers

Searching by date range from filenames

Hello all, i have tons of files in folder named like this (yyyymmdd): bookcollection20100729 bookcollection20100730 bookcollection20100731 bookcollection20100801 bookcollection20100802 etc. I need to find files with date range in there names lets say from 2010.07.30 - 2010.08.02 ... (10 Replies)
Discussion started by: Whit3H0rse
10 Replies

5. Shell Programming and Scripting

Extracting filenames

Hi I need to pull out the name of the file from the path. See, here is my loop that gets the files: dsxdir="/var/local/dsx/import" for dsxfile in $dsxdir/*.dsx; do dsxlog $reverb --info --module="$module" "$dsxfile" $dsximp $norule $oprange --dsn=$dsn --dbname=$dbname... (6 Replies)
Discussion started by: ladyAnne
6 Replies

6. Shell Programming and Scripting

How to extract date out of this filenames

I have filenames filenameA_fg_MMDDYY.tar.gz filenameASPQ_fg_MMDDYY.tar.gz filenameAFTOPHYYINGH_fg_MMDDYY.tar.gz filenameAGHYSW_fg_MMDDYY.tar.gz Is there a way I can extract the date out of these filenames? Thanks in advance (2 Replies)
Discussion started by: RubinPat
2 Replies

7. Shell Programming and Scripting

Whitespace in filenames in for loop in bash script

I'm trying to search all .odt files in a directory for a string in the text of the file. I've found a bash script that works, except that it can't handle whitespace in the filenames. #!/bin/bash if ; then echo "Usage: searchodt searchterm" exit 1 fi for file in $(ls *.odt); do ... (4 Replies)
Discussion started by: triplemaya
4 Replies

8. Shell Programming and Scripting

loop through numbered filenames

Hi I'm very new to this script thing, so please be gentle. I am trying to get a command - the mach2qtl command in the code below - to loop through a set of files. Each command should take the same two .dat and .ped files, but the .mlinfo and .mlprob files with filenames including 'chrom1' ... (7 Replies)
Discussion started by: polly_falconer
7 Replies

9. Shell Programming and Scripting

Perl: Extracting date from file name and comparing with current date

I need to extract the date part from the file name (20080221 in this ex) and compare it with the current date and delete it, if it is a past date. $file = exp_ABCD4_T-2584780_upto_20080221.dmp.Z really appreciate any help. thanks mkneni (4 Replies)
Discussion started by: MKNENI
4 Replies

10. Shell Programming and Scripting

changing filenames in a directory to a number within a loop

hey guys. i'm new to shell scripting but not new to programming. i want to write a script that will take all the files in the current directory that end with a particular filetype and change all their names to a number in order. so, it would take all the jpg files and sort them in alphabetical... (30 Replies)
Discussion started by: visitorQ
30 Replies
Login or Register to Ask a Question