Pick the latest set of files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pick the latest set of files
# 1  
Old 02-03-2009
Pick the latest set of files

I have task in which I need to pickup a set of files from a directory
depending on the following criteria:
Every month 6 files are expected to arrive at /test.
The files come with date timestamp and the latest file set for the month needs to be used
Suppose this is the set of files that present in the test directory
1. a_20080905_214058_200808.txt
2. b_20080905_214058_200808.txt
3. c_20080905_214058_200808.txt
4. d_20080905_214058_200808.txt
5. e_20080905_214058_200808.txt
6. f_20080905_214058_200808.txt
7. a_20080906_214058_200809.txt
8. b_20080906_214058_200809.txt
9. c_20080906_214058_200809.txt
10. d_20080906_214058_200809.txt
11. e_20080906_214058_200809.txt
12. f_20080906_214058_200809.txt
13. d_20080906_114058_200809.txt
14. e_20080906_114058_200809.txt
15. f_20080906_114058_200809.txt
Then
7. a_20080906_214058_200809.txt
8. b_20080906_214058_200809.txt
9. c_20080906_214058_200809.txt
10. d_20080906_214058_200809.txt
11. e_20080906_214058_200809.txt
12. f_20080906_214058_200809.txt files should be picked up.
Criteria 200809 is the latest month 20080906 is the latest date and 214058 is the latest date.
Thanks in Advance

Last edited by w020637; 02-03-2009 at 03:51 PM..
# 2  
Old 02-03-2009
Code:
/bin/ls -1 *_* |
  cut -d_ -f4 |
  sort -rnu |
  head -1 |
  read latest
 
/bin/ls -l *${latest}*

# 3  
Old 02-03-2009
What's wrong with just doing this?

ls -r1 ?_*.txt | head -6

You can then just add some while logic around that.
# 4  
Old 02-03-2009
assume you have a file with the filenames - call it inputfile - this is a longer example - plus -rnu will not work across _ characters.
Code:
tr -s '_' ' ' <  inputfile | \
  sort -n -k 1.1,1.1 -k 2.1,2.8 -k 3.1,3.6 | \
  awk '{ arr[$1]=$0 } END {for (i in arr) { print arr[i] } }' | \
  tr '_' ' ' | sort > resultsfile

# 5  
Old 02-03-2009
Quote:
Originally Posted by jim mcnamara
assume you have a file with the filenames - call it inputfile - this is a longer example - plus -rnu will not work across _ characters.
Code:
tr -s '_' ' ' <  inputfile | \
  sort -n -k 1.1,1.1 -k 2.1,2.8 -k 3.1,3.6 | \
  awk '{ arr[$1]=$0 } END {for (i in arr) { print arr[i] } }' | \
  tr '_' ' ' | sort > resultsfile

i'm only pulling out the 4th field and piping to sort -rn.
( I tested it first; it works. )

My assumption was that the files were ~named~ with the
correct time-stamps, but the modification times on the files
themselves weren't necessarily correct....

If the file-modification-times are correct,
giannicello's solution works fine.
# 6  
Old 02-03-2009
Thanks All,

Jim, The results file had file names with '_' replaced by spaces.How can that be corrected?
The filemodification time is not consistent and hence would not solve the purpose.
Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Should pick latest file within past 3 days using UNIX script and perform steps in message below.

Hi , Can anyone help me how do perform below requirement in unix. Step1:we will receive multiple files weekly with same name(as below) in a folder(In folder we will have other files also def.dat,ghf.dat) Filenames: 1) abc_20171204_052389.dat 2)abc_20171204_052428.dat DON'T modify... (23 Replies)
Discussion started by: sunnykamal59
23 Replies

2. Shell Programming and Scripting

awk - Pick last value from set of rows

Input data COL_1,COL_2,COL_3,COL_4,COL_5,COL_6,COL_7,COL_8,COL_9,COL_10,COL_11,COL_12,COL_13 C,ABC,ABCD,3,ZZ,WLOA,2015-12-01,2016-12-01,975.73,ZZZ,P,111111.00,Y1 **GROUP1** C,ABC,ABCD,3,ZZ,WLOA,2015-12-01,2016-12-01,975.73,ZZZ,P,222222.00,Y1 **GROUP1**... (2 Replies)
Discussion started by: Ads89
2 Replies

3. Shell Programming and Scripting

I have this list of files . Now I will have to pick the latest file based on some condition

3679 Jul 21 23:59 belk_rpo_error_**po9324892**_07212014.log 0 Jul 22 23:59 belk_rpo_error_**po9324892**_07222014.log 3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log 22 Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log 0 Jul 20 05:50... (5 Replies)
Discussion started by: LoneRanger
5 Replies

4. Shell Programming and Scripting

to pick the latest file modified in a directory

I wan to pick the latest modified file name and redirect it to a file .. ls -tr | tail -1 >file but this is printing file ins side the filename , can anyone help me out (5 Replies)
Discussion started by: vishwakar
5 Replies

5. UNIX for Dummies Questions & Answers

How to pick only the latest files based on the timestamp?

I have a few log files which get generated on a daily basis..So, I need to pick only the ones which get generated for that particular day. -rw-r--r-- 1 staff 510732676 Apr 7 22:01 test.log040711 -rwxrwxrwx 1 staff 2147482545 Apr 7 21:30 test.log.2 -rwxrwxrwx 1 staff 2147482581 Apr 7 19:26... (43 Replies)
Discussion started by: win4luv
43 Replies

6. Shell Programming and Scripting

To find latest set of logs among new and old

Hi All I am writing a script which will select the latest logs (which are generated every night via a script) among old one and new. Script generates set of 3 logs each time it runs. Example : log-WedJun082011_bkt1.log log-WedJun082011_bkt2.log log-WedJun082011_bkt3.log I have... (1 Reply)
Discussion started by: ratneshnagori
1 Replies

7. UNIX for Dummies Questions & Answers

Need command to pick the latest file

Hi In my script i am trying to access mainframe server using FTP, in the server i have filee with the timestamp.I need to get the file with the latest timestamp among them . The server has the below files / ftp> cd /outbox 250 CWD command successful ftp> ls 200 PORT command successful... (4 Replies)
Discussion started by: laxmi131
4 Replies

8. Shell Programming and Scripting

Find the latest directory and loop through the files and pick the error messages

Hi, I am new to unix and shell scripting,can anybody help me in sctipting a requirement. my requirement is to get the latest directory the name of the directory will be like CSB.monthdate_time stamp like CSB.Sep29_11:16 and CSB.Oct01_16:21. i need to pick the latest directory. in the... (15 Replies)
Discussion started by: sudhir_83k
15 Replies

9. UNIX for Dummies Questions & Answers

pick the very latest directory

Hi, I have some list of directories in the form datemonthyear e.g. 02082009, 03082009 and 04082009 etc. I need to pick the latest directory from the current working directory. Outcome: 05082009 This is the output am expecting. Thanks (6 Replies)
Discussion started by: venkatesht
6 Replies

10. Shell Programming and Scripting

how can i pick the latest log file as per below

in the below .. i want to pick the latest logfile which is having JPS.PR inside.. that means i want particularly "spgport040408041223.log:@@@@@@@@ 04:13:09 Adding: JPS.PR." which is latest among these.. is it possible to compare the current time with logfile time ? reptm@xblr0758rop>... (4 Replies)
Discussion started by: mail2sant
4 Replies
Login or Register to Ask a Question