Help with Archiving multiple files based on name and date
Dear Gurus,
I am a novice in shell scripts. I have a requirement where I need to move files every day from Current Folder to Archive folder.
Daily I will be receiving 5 files in the folder - /opt/data/feeds/.
The feeds folder has two sub-folders - Current and Archive.
For example the first day, the I receive file and names will be like:
The last 8 characters are the mmddyyyy.
Very first time Current folder will be empty so I just move the files from
/opt/data/feeds/ to /opt/data/feeds/Current.
Again the second day, I will be receiving the following files:
When I receive the files, I should the move the existing files from Current to Archive folder and then place the new files in the "Current" folder.
Also there is another condition where say for example on the third day, I receive only 3 out of 5.
In this case, the current folder should keep the file2 and file4 (which is from 08022011) and file1,3,5 (which is from 08032011). The file1,3,5(of 08022011) should be moved to Archive. The current folder should always have the most recent date for each file.
Also the archive folder should have only the last 6 days for each file.
So every day the current folder will have 5 files and Archive folder will have 30 files.
Please help me on this.
Thanks in advance.
Shankar
Last edited by radoulov; 08-01-2011 at 05:54 PM..
Reason: Code tags.
Usually, the trick is to put the files all in the archive, and as each comes in, link it to current without a date in the name, after removing any link from before.
How do you know the file is fully written and can be used? Discovery seems silly, when the producer could just install it.
These files are actually report outputs with .CSV extension. They are moved to this folder via application.
I am using something like below:
The above script just checks whether the file in Current directory is different from Archive directory and then does the move.
Daily you get files and the last 8characters of the file name will be current system date.
I am struggling to check the file names with the file saved date and then archiving for 6 days.
Any help with the script is highly appreciated.
Thanks
Last edited by radoulov; 08-01-2011 at 05:54 PM..
Reason: Code tags!
Well, if you list file names stripped of date and detect duplicates, you know who needs moving down. Don't overwork making file lists, env var and pipes are fine:
I am still not getting the script to work. The issue I notice is the file name.
The names I gave was an example.
If the file name format changes then the script won't work.
The only non changing criteria in the file name is that it always have the date(mmddyyyy) at the end. It can have any name/length as prefix.
For example: The files can be
File1_extract_08012011
Open_inv_08012011
BAXT_CONV_08012011
You can do it going through a sorted list with history in variables. If you hit a later file, you move the last file to archive. Here, * is a sorted list, and the ksh does it all but the mv internally, until 2040:
Really, polling sucks! The creator app should do this, too, after a good file create.
I am trying to work on a script where it is a *(star) delimited file has a multiple lines starts with RTG and 3rd column=TD8 I want to substring the date part and
I want to replace with currentdate minus 15 days. Here is an example. iam using AIX server
$ cat temp.txt
RTG*888*TD8*20180201~... (1 Reply)
My unix version is IBM AIX Version 6.1
I tried google my requirement and found the below answer,
find . -newermt “2012-06-15 08:13" ! -newermt “2012-06-15 18:20"
But newer command is not working in AIX version 6.1 unix
I have given my requirement below:
Input:
atr files:
... (1 Reply)
Hi All,
I do have an EBCDIC file sent from the z/os , this file has records with different record types in it, the type of record is identified by bytes 45-46 like
value 12 has employee record
value 14 has salaray record and etc....
we do now want to split the big ebcdic file into multiple... (3 Replies)
We have year folder say in a path /opt/informat/Archive a folder 2012.
And in the same folder /opt/informat/Archive we have different folders month based, like 201210, 201211, 201212, this have data for each month, ie files. Now time to time i need to move the monthly folders to the main folder... (1 Reply)
Hello team,
I have a number of files in a folder which are dated yesterday and today.Can i remove all the files which i created today based on date??
is there any syntax for this ?? (1 Reply)
Hi all
i am having so many files in my directory.Is there any option
to copy files based on date.
example
i am having file this
-rw-rw-r-- 1 ram user 1 Feb 2 17:12 abc
-rw-rw-r-- 1 ram user 1 Feb 2 17:12 bnw
-rwxrwxr-x 1 ram user 21122 Feb 4... (3 Replies)
hi all,
here is the description to my problem.
input parameters: $date1 & $date2
based on the range i need to select the archived files from the archived directory and moved them in to working directory.
can u please help me in writing the code to select the multiple files based on the... (3 Replies)
Hello friends,
I'm learning to script, and I need help.
How can I get the latest/newest files based on date? the format is as following:
Feb 07 19:25 TESTPWD_file_1vk6pn40_19519_1
Feb 07 19:46 TESTPWD_file_1uk6pn40_19518_2
Feb 07 19:47 TESTPWD_file_20k6pn40_19520_2
Feb 07 19:56... (5 Replies)
Hi Friends,
Can anyone help me with this:
To get the count of files that are existing in a directory created on a perticular date like in the example (01/08) .(having same pattern for the filename)
ex:
FileName Creted Date
FILE001 01/08/2007
FILE005 ... (6 Replies)
I am trying to write a shell script that will remove files in a directory based on the date. For instance, remove all files older than yesterday. Any ideas? (4 Replies)