How to pick only the latest files based on the timestamp?


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers How to pick only the latest files based on the timestamp?
# 1  
Old 04-15-2011
How to pick only the latest files based on the timestamp?

I have a few log files which get generated on a daily basis..So, I need to pick only the ones which get generated for that particular day.
Code:
-rw-r--r-- 1 staff 510732676 Apr 7 22:01 test.log040711
-rwxrwxrwx 1 staff 2147482545 Apr 7 21:30 test.log.2
-rwxrwxrwx 1 staff 2147482581 Apr 7 19:26 test.log.1

There are 3 files generated on april 7....I need to get only those files(ALL 3 of them)

Please help

Last edited by Scott; 04-18-2011 at 05:29 PM..
win4luv
# 2  
Old 04-15-2011
Code:
ls -lrt test.log* | awk '/Apr  7/{x=$9}END{print x}'

---------- Post updated at 07:49 PM ---------- Previous update was at 07:46 PM ----------

Code:
d="Apr  7"
ls -lrt test.log* | awk  -v V="$d" '$0~V{x=$9}END{print x}'

Use nawk instead of awk if on SunOS / Solaris plateform
# 3  
Old 04-15-2011
Best add a : to the pattern so you do not get Apr 7 1971,

or script to touch marker files at midnight before and after, and find ... -newer marker1 ! -newer marker2 -type f

Of course, if they keep writing the files, they show up on two days. Better to create them with a start date in the name.
# 4  
Old 04-17-2011
you may try this.

Code:
ls -lrt | awk '{print $6$7" "$9 }' | grep Apr7 | awk '{print $2}'


Last edited by Scott; 04-18-2011 at 05:30 PM..
# 5  
Old 04-19-2011
Code:
ls -lrt | awk '{print $6$7" "$9 }' | grep Apr7 | awk '{print $2}'

works perfect...Thanks Smilie

But how do i make the date get generated dynamically...Since i want the latest files on a daily basis and avoid any manual intervention.

---------- Post updated 04-19-11 at 06:55 AM ---------- Previous update was 04-18-11 at 03:12 PM ----------

if you look at the code..lets say i incorporate the following logic:
Code:
ls -lrt | awk '{print $6$7" "$9 }' | grep Apr7 | awk '{print $2}'

It would fetch me the log files for a particular day, but i need to avoid this manual intervention where i need to enter the date ...So i need something which would pick the files daily automatically and mv it to some folder and from there i can take it forward...

Last edited by radoulov; 05-19-2011 at 12:19 PM.. Reason: Code tags.
win4luv
# 6  
Old 04-19-2011
Methods which process the output from "ls -la" are prone to spurious matches and problems with filenames containing space characters. None of the examples posted so far really work.

Another approach. Find all the files in the directory which were created after midnight. The filenames are available in the centre of the do-done loop if you need to do further processing.

Code:
YYYYMMDD="`date +%Y%m%d`"       # Reversed date yyyymmdd
REFERENCE=/var/tmp/reference.${YYYYMMDD}.$$
touch -t ${YYYYMMDD}0000 ${REFERENCE}
#
ls -1d * 2>/dev/null | while read FILENAME
do
     if [ -f "${FILENAME}" -a "${FILENAME}" -nt "${REFERENCE}" ]
     then
           echo "${FILENAME}"
     fi
done 
#
rm ${REFERENCE}

There is a better method using just "find" but we don't know whether your directory has subdirectories.
# 7  
Old 04-19-2011
Remember, find does not recurse for files, so glob in a find with only file names and use -newer. Usually, a glob of your files can be devised that does not pic up dirs, like '*.*'.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Need help with UNIX command to get the latest file from list of files with timestamp

Hi All, I have list of files like below with name abcxyz.timestamp. I need a unix command to pick the latest file from the list of below files. Here in this case the lates file is abcxyz.20190304103200. I have used this unix command "ls abcxyz*|tail -1" but i heard that it is not the appropriate... (2 Replies)
Discussion started by: rakeshp
2 Replies

2. Shell Programming and Scripting

Picking the latest file based on a timestamp for a Dynamic file name

Hi , I did the initial search but could not find what I was expecting for. 15606Always_9999999997_20160418.xml 15606Always_9999999998_20160418.xml 15606Always_9999999999_20160418.xml 9819Always_99999999900_20160418.xml 9819Always_99999999911_20160418.xmlAbove is the list of files I... (4 Replies)
Discussion started by: chillblue
4 Replies

3. Shell Programming and Scripting

I have this list of files . Now I will have to pick the latest file based on some condition

3679 Jul 21 23:59 belk_rpo_error_**po9324892**_07212014.log 0 Jul 22 23:59 belk_rpo_error_**po9324892**_07222014.log 3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log 22 Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log 0 Jul 20 05:50... (5 Replies)
Discussion started by: LoneRanger
5 Replies

4. UNIX for Dummies Questions & Answers

Sorting files based on timestamp and picking the latest file

Hi Friends, Newbie to shell scripting Currently i have used the below to sort data based on filenames and datestamp $ printf '%s\n' *.dat* | sort -t. -k3,4 filename_1.dat.20120430.Z filename_2.dat.20120430.Z filename_3.dat.20120430.Z filename_1.dat.20120501.Z filename_2.dat.20120501.Z... (12 Replies)
Discussion started by: robertbrown624
12 Replies

5. Shell Programming and Scripting

Urgent ...pls Sorting files based on timestamp and picking the latest file

Hi Friends, Newbie to shell scripting. Currently i have used the below to sort data based on filenames and datestamp $ printf '%s\n' *.dat* | sort -t. -k3,4 filename_1.dat.20120430.Z filename_2.dat.20120430.Z filename_3.dat.20120430.Z filename_1.dat.20120501.Z filename_2.dat.20120501.Z... (1 Reply)
Discussion started by: robertbrown624
1 Replies

6. Shell Programming and Scripting

sort the files based on timestamp and execute sorted files in order

Hi I have a requirement like below I need to sort the files based on the timestamp in the file name and run them in sorted order and then archive all the files which are one day old to temp directory My files looks like this PGABOLTXML1D_201108121235.xml... (1 Reply)
Discussion started by: saidutta123
1 Replies

7. UNIX for Dummies Questions & Answers

Need command to pick the latest file

Hi In my script i am trying to access mainframe server using FTP, in the server i have filee with the timestamp.I need to get the file with the latest timestamp among them . The server has the below files / ftp> cd /outbox 250 CWD command successful ftp> ls 200 PORT command successful... (4 Replies)
Discussion started by: laxmi131
4 Replies

8. Shell Programming and Scripting

Find the latest directory and loop through the files and pick the error messages

Hi, I am new to unix and shell scripting,can anybody help me in sctipting a requirement. my requirement is to get the latest directory the name of the directory will be like CSB.monthdate_time stamp like CSB.Sep29_11:16 and CSB.Oct01_16:21. i need to pick the latest directory. in the... (15 Replies)
Discussion started by: sudhir_83k
15 Replies

9. UNIX for Dummies Questions & Answers

pick the very latest directory

Hi, I have some list of directories in the form datemonthyear e.g. 02082009, 03082009 and 04082009 etc. I need to pick the latest directory from the current working directory. Outcome: 05082009 This is the output am expecting. Thanks (6 Replies)
Discussion started by: venkatesht
6 Replies

10. Shell Programming and Scripting

Pick the latest set of files

I have task in which I need to pickup a set of files from a directory depending on the following criteria: Every month 6 files are expected to arrive at /test. The files come with date timestamp and the latest file set for the month needs to be used Suppose this is the set of files that present... (5 Replies)
Discussion started by: w020637
5 Replies
Login or Register to Ask a Question