Should pick latest file within past 3 days using UNIX script and perform steps in message below.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Should pick latest file within past 3 days using UNIX script and perform steps in message below.
# 1  
Old 12-04-2017
Should pick latest file within past 3 days using UNIX script and perform steps in message below.

Hi ,
Can anyone help me how do perform below requirement in unix.

Step1:we will receive multiple files weekly with same name(as below) in a folder(In folder we will have other files also def.dat,ghf.dat)

Filenames:

1) abc_20171204_052389.dat
2)abc_20171204_052428.dat

Moderator's Comments:
Mod Comment DON'T modify posts if others already referenced it, pulling the rug from under their feet!



Step2:we should pick the file which has received date within the past 3 days.

step3:We should read the files which has text ‘S A M P L E’ in the files starting at position 31 on line 19 .

step4:Once we identify the file which has text ‘S A M P L E’ than pick that file and should look for 'GENERIC' and copy all the data which has 'GENERIC' to
other file.

Last edited by RudiC; 12-05-2017 at 12:34 PM.. Reason: Add ICODE tags.
# 2  
Old 12-04-2017
Step 1 is a problem since you can't have more than one file with the same name.
# 3  
Old 12-04-2017
Welcome to the forum.

Please become accustomed to provide decent context info of your problem.

It is always helpful to carefully and detailedly phrase a request, and to support it with system info like OS and shell, related environment (variables, options), preferred tools, adequate (representative) sample input and desired output data and the logics connecting the two including your own attempts at a solution, and, if existent, system (error) messages verbatim, to avoid ambiguities and keep people from guessing.
# 4  
Old 12-04-2017
Sorry filenames recieved will have timestamp as below.

Step1:we will receive multiple files with different time stamps weekly with same name(as below) in a folder.

Filenames:

1) abc_20171204_052389.dat
2)abc_20171204_052428.dat

Last edited by Don Cragun; 12-04-2017 at 10:54 PM.. Reason: Add ICODE tags.
# 5  
Old 12-04-2017
How about this:

Code:
MIDNIGHT=$(date -d 0:0 +'%s')
((THREEDAYSAGO=MIDNIGHT - 3*3600*24))

for file in *_*_*.dat
do
   FILESTAMP=${file#*_}
   FILESTAMP=${FILESTAMP%_*}
   if [[ $(date -d $FILESTAMP +'%s') -ge $THREEDAYSAGO ]]
   then
       awk '
           NR==19{
              if (substr($0,31,11) == "S A M P L E")
                 EC=1
              exit
           }
           END { exit EC }' $file

       if [ $? -eq 1 ] 
       then
           # copy all data that has GENERIC to "other file"
           grep "GENERIC" $file >> ./"other file"
       fi
   fi
done

# 6  
Old 12-04-2017
Thanks for your reply .Can you please please explain(step by step) in detail?I'm new to scripting.

*When i try to execute MIDNIGHT=$(date -d 0:0 +'%s') the result is '1512363600' . what is this number?is it the date?

*((THREEDAYSAGO=MIDNIGHT - 3*3600*24)) ..why we are doing 3*36*24?

*if we have 3 files selected in 3 days ,each file might have different string in line no#19 position 31,we should only select the file which has text ‘S A M P L E’ in the file.how we are doing this?

*And also can you explain this logic:

Code:
for file in *_*_*.dat
do
   FILESTAMP=${file#*_}
   FILESTAMP=${FILESTAMP%_*}
   if [[ $(date -d $FILESTAMP +'%s') -ge $THREEDAYSAGO ]]
   then
       awk '
           NR==19{
              if (substr($0,31,11) == "S A M P L E")
                 EC=1
              exit
           }
           END { exit EC }' $file

Moderator's Comments:
Mod Comment Please use CODE tags (as required by forum rules) when displaying sample input, sample output, and code segments.

Last edited by Don Cragun; 12-05-2017 at 12:46 AM.. Reason: Add CODE and ICODE tags.
# 7  
Old 12-05-2017
While the community in here are happy to help people, be it simple or complex questions, the main objective is to help them help themselves. Amongst others, man pages are invaluable sources of info, e.g. man date:
Quote:
FORMAT controls the output. Interpreted sequences are:
.
.
.
%s seconds since 1970-01-01 00:00:00 UTC
This would answer your first question: 1512363600 is the number of seconds since "the epoque", of that day's midnight.
2. question: how many seconds does an hour have? how many hours a day?
3. man awk:
Quote:
7. Builtin-variables
.
.
.
NR current record number in the total input stream.
By default, lines are the records for awk, so NR==19 detects the 19th line (as requested)
Quote:
substr(s,i,n) / substr(s,i)
Returns the substring of string s, starting at index i, of length n. If n is omitted, the suffix of s, starting at i is returned.
so substr($0,31,11) will extract exactly that part of the line that needs to be compared to your sample text.

I think the logics should be clear by now.

Last edited by RudiC; 12-05-2017 at 06:08 AM..
This User Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

I have this list of files . Now I will have to pick the latest file based on some condition

3679 Jul 21 23:59 belk_rpo_error_**po9324892**_07212014.log 0 Jul 22 23:59 belk_rpo_error_**po9324892**_07222014.log 3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log 22 Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log 0 Jul 20 05:50... (5 Replies)
Discussion started by: LoneRanger
5 Replies

2. Shell Programming and Scripting

to pick the latest file modified in a directory

I wan to pick the latest modified file name and redirect it to a file .. ls -tr | tail -1 >file but this is printing file ins side the filename , can anyone help me out (5 Replies)
Discussion started by: vishwakar
5 Replies

3. UNIX for Dummies Questions & Answers

Need command to pick the latest file

Hi In my script i am trying to access mainframe server using FTP, in the server i have filee with the timestamp.I need to get the file with the latest timestamp among them . The server has the below files / ftp> cd /outbox 250 CWD command successful ftp> ls 200 PORT command successful... (4 Replies)
Discussion started by: laxmi131
4 Replies

4. UNIX for Dummies Questions & Answers

pick the very latest directory

Hi, I have some list of directories in the form datemonthyear e.g. 02082009, 03082009 and 04082009 etc. I need to pick the latest directory from the current working directory. Outcome: 05082009 This is the output am expecting. Thanks (6 Replies)
Discussion started by: venkatesht
6 Replies

5. Shell Programming and Scripting

how can i pick the latest log file as per below

in the below .. i want to pick the latest logfile which is having JPS.PR inside.. that means i want particularly "spgport040408041223.log:@@@@@@@@ 04:13:09 Adding: JPS.PR." which is latest among these.. is it possible to compare the current time with logfile time ? reptm@xblr0758rop>... (4 Replies)
Discussion started by: mail2sant
4 Replies

6. UNIX for Dummies Questions & Answers

unix script that will pick up first 10 file

Suppose I have a unix file which contain a lost of 60 files like filename1 filename2 ... .. ... filename60 I want to write a unix script that will pick up first 10 files in first run 10-20 files in 2 run 20-30 files in 3 run 30-40 files in 4 run 40-50 files in 5 run 50-60 files in 6... (1 Reply)
Discussion started by: er_zeeshan05
1 Replies

7. Shell Programming and Scripting

find files from the past 7 days

Hi All, I have a file which contains the listing of another directory: >cat list.dat -rwxr-xr-x 1 test staff 10240 Oct 02 06:53 test.txtdd -rwxrwxrwx 1 test staff 0 Oct 04 07:22 test.txx -rwxrwxrwx 1 test staff 132 Sep 16 2007 test_tt.sh... (6 Replies)
Discussion started by: deepakgang
6 Replies

8. UNIX for Dummies Questions & Answers

creating a CSV file for past 7 days

I have a requirement which will select the files with a specific naming convention which got created in past 7 days in a specific directory.Lets say the directory is /data/XYZ and the file names follow the below nomenclature like Daily_File*.txt I just need to create one CSV file which will... (12 Replies)
Discussion started by: dr46014
12 Replies

9. Shell Programming and Scripting

Script to pick out files from a file which are older than 15 days

Hi , I have a file abc.txt which actually contains a directory and file list -rwxrwxr-x 1 dmadmin mmasapp 334 Dec 03 2001 aafs_006.ksh -rwxrwxr-x 1 dmadmin mmasapp 1270 Nov 13 2001 mcl_deposit_ftp.ksh -rwxrwxr-x 1 dmadmin mmasapp 925 Oct 31 2001 ... (1 Reply)
Discussion started by: viv1
1 Replies

10. Shell Programming and Scripting

ls latest 4 days or specify days of files in the directory

Hi, I would like to list latest 2 days, 3 days or 4 days,etc of files in the directory... how? is it using ls? (3 Replies)
Discussion started by: happyv
3 Replies
Login or Register to Ask a Question