I have this list of files . Now I will have to pick the latest file based on some condition


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting I have this list of files . Now I will have to pick the latest file based on some condition
# 1  
Old 07-24-2014
I have this list of files . Now I will have to pick the latest file based on some condition

Code:
 
 3679   Jul 21 23:59 belk_rpo_error_**po9324892**_07212014.log
   0    Jul 22 23:59 belk_rpo_error_**po9324892**_07222014.log
   3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log
     22 Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log
      0 Jul 20 05:50 belk_rpo_error_**po9999992**_07202014.log
    411 Jul 21 06:30 belk_rpo_error_**po9999992**_07212014.log
    742 Jul 21 07:30 belk_rpo_error_**po9999991**_07212014.log
      0 Jul 23 2014 belk_rpo_error_**po9999991**_07232014.log

First column represents Size in Bytes.

For a PATRICULAR Order_No(Marked with ** **)
  1. If the latest file is 0 kB then we will discard it (rest of the files with same Order_no as well)
  2. if the latest file is non Zero then I will take it.(Only the latest one)
Then append the contents in a txt file . My expected output would be ::

Code:
 
 411    Jul 21 06:30 belk_rpo_error_**po9999992**_07212014.log
   3679 Jul 23 23:59 belk_rpo_error_**po9324892**_07232014.log
    22   Jul 22 06:30 belk_rpo_error_**po9324267**_07012014.log

I am at my wits end here. I cant seem to figure out how to compare dates in Unix. Any help is very appreciated.
# 2  
Old 07-24-2014
Dear LoneRanger,
I have a few to questions pose in response first:-
  • Is this homework/assignment? There are specific forums for these.
  • What have you tried so far?
  • What output/errors do you get?
  • Are the ** shown actually part of the file name or just to highlight the important part?
  • How are you generating the listing? We might be able to process it in-line rather than writing and processing a file.
  • What OS and version are you using?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)
Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.


We're all here to learn and getting the relevant information will help us all.
# 3  
Old 07-24-2014
Is this homework/assignment? There are specific forums for these.
What have you tried so far? No It's not.
What output/errors do you get? I am not getting expected output.
Are the ** shown actually part of the file name or just to highlight the important part? Just to highlight the important part
How are you generating the listing? We might be able to process it in-line rather than writing and processing a file.

I am generating it using
Code:
find . -name "*.log" -mtime -8

What OS and version are you using? I am not sure. Some flavor some Linux.
What are your preferred tools? (C, shell, perl, awk, etc.) Shell
What logical process have you considered? (to help steer us to follow what you are trying to achieve) I am trying to pick the filename , date part ,size and the order_number part and insert all into a DB table from shell script. do the comparison there an pick the right file name and pass it back into shell script for further processing.


My code so far :: it's not giving me expected output though-


Code:
 
 export LOGDIR=/home/rmsbatch/testdata
 
# TEMPDIR : Path where all non zero files need to be moved after search.
export TEMPDIR=/home/rmsbatch/testdata/TEMP
 # RESULTFILE : Path of the Final File 
export RESULTFILE=${TEMPDIR}/belk_rpo_error.txt
 ###############################################################################
#                                     MAIN                                    #
###############################################################################
set -x
if [ $# -ne 1 ]
  then
    echo "Please Provide Date range"
    echo "Example: ./belk_rpo_shell_script.ksh "
    exit 1
fi
 DATE_RANGE=$1
echo $DATE_RANGE
echo " SET ECHO OFF
SET NEWPAGE 0
SET SPACE 0
SET PAGESIZE 0
SET FEEDBACK OFF
SET HEADING OFF
SET TRIMSPOOL ON
SET TAB OFF
           select to_char(sysdate,'YYYYMMDD')
                                  from dual ;" > date.sql
 END_DATE=`sqlplus -s $UP <<EOF @date.sql
exit;
EOF`
 echo " SET ECHO OFF
SET NEWPAGE 0
SET SPACE 0
SET PAGESIZE 0
SET FEEDBACK OFF
SET HEADING OFF
SET TRIMSPOOL ON
SET TAB OFF
           select to_char(sysdate-$DATE_RANGE,'YYYYMMDD')
                  from dual ;" > date1.sql
 START_DATE=`sqlplus -s $UP <<EOF @date1.sql
exit;
EOF`
 echo $START_DATE
echo $END_DATE
 cd $LOGDIR
 if [ ! -d "$LOGDIR/tmp" ]; then
  mkdir $LOGDIR/tmp
fi
 touch -t ${START_DATE}0000 ./tmp/NEWSTARTDATE
touch -t ${END_DATE}2359 ./tmp/NEWENDDATE
 find . -type f ! -size 0 \( -newer ./tmp/NEWSTARTDATE -a \! -newer ./tmp/NEWENDDATE \) -print |grep '.log'|
 while read fname
do
cp -p $fname  $TEMPDIR/${fname}
  
  PONUM=$(echo "${fname:17:9}")
  find . -type f  -size 0 \( -newer ./tmp/NEWSTARTDATE -a \! -newer ./tmp/NEWENDDATE \) -print |grep $PONUM |  
   
  while read fn
  do
    rm -f $TEMPDIR/$fn
    rm -f $TEMPDIR/${fname}
  done 
done
  
 cd $TEMPDIR
 
if test -f $RESULTFILE; then
  rm -rf $RESULTFILE;
fi
# Create an empty $RESULTFILE
touch -t ${START_DATE} $RESULTFILE
  
 for f in `ls *`; do
  cat $f >> $RESULTFILE;
done
 
nawk '!x[$0]++' $RESULTFILE > temp
mv temp $RESULTFILE
 
sed '/@0PACKAGE_ERROR@1ORA/d' $RESULTFILE > temp
mv temp $RESULTFILE
 
if [ -d "$LOGDIR/tmp" ]; then
  rm -rf $LOGDIR/tmp
fi
 
#for f in `ls belk_rpo_error_po* 2>/dev/null`
#do
#  rm -rf $f
#done

# 4  
Old 07-24-2014
Something like
Code:
#! /bin/bash
ls -ctl | while read a b c d SIZE f g h NAME 
do
if [[ ${SIZE} -ne 0 ]] 
then
  echo  ${NAME},${SIZE}
  exit
fi
done

# 5  
Old 07-24-2014
For
Code:
find . -name "*.log" -mtime -8

..... do you mean
Code:
find . -name "*.log" -mtime -8 -ls

That could give the listing you have.

For the listing, am I okay to assume that it's better to show it as this:-
Code:
3679   Jul 21 23:59 belk_rpo_error_po9324892_07212014.log
   0    Jul 22 23:59 belk_rpo_error_po9324892_07222014.log
   3679 Jul 23 23:59 belk_rpo_error_po9324892_07232014.log
     22 Jul 22 06:30 belk_rpo_error_po9324267_07012014.log
      0 Jul 20 05:50 belk_rpo_error_po9999992_07202014.log
    411 Jul 21 06:30 belk_rpo_error_po9999992_07212014.log
    742 Jul 21 07:30 belk_rpo_error_po9999991_07212014.log
      0 Jul 23 2014 belk_rpo_error_po9999991_07232014.log

.... where the red highlights the selection bit and the bold are those to be picked up? Note that find does not give you a listing that is sorted in any way in particular. It just reads the directories and gives them to you in the order written, which may or may not be what you need.


Okay, now what about the rest of the questions?
  • Is this homework/assignment? There are specific forums for these.
  • What output/errors do you get?
  • What OS and version are you using? Run uname -a and paste the result in here in CODE tags.
  • What are your preferred tools? (C, shell, perl, awk, etc.)
As for "What logical process have you considered?" , well Smilie!

Okay let's make this simpler. From what you wrote, I think you want to take (maybe read in, maybe loop for all) the order number which in your examples is po........ and get the most recent file that is not empty.

Am I correct?



If so, perhaps something more like this might help:-
Code:
ls -1t *${order_no}* | while read file       # For each file matching the search in a simple list, newest first
do
   if [ -s $file ]            # Test if it's a file with bytes in it
   then
      use_file=$file
      break
   else
      echo "Ignoring empty file $file"
   fi
done

echo "I selected $use_file"

Let me know what you think and the answers to the other requests, if you could be so kind. There may be limitations based on what you have so that may set our choices.


Robin
# 6  
Old 07-24-2014
Robin , Thanks .

I will try your suggestion tomorrow and get back to you..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Should pick latest file within past 3 days using UNIX script and perform steps in message below.

Hi , Can anyone help me how do perform below requirement in unix. Step1:we will receive multiple files weekly with same name(as below) in a folder(In folder we will have other files also def.dat,ghf.dat) Filenames: 1) abc_20171204_052389.dat 2)abc_20171204_052428.dat DON'T modify... (23 Replies)
Discussion started by: sunnykamal59
23 Replies

2. UNIX for Dummies Questions & Answers

Sorting files based on timestamp and picking the latest file

Hi Friends, Newbie to shell scripting Currently i have used the below to sort data based on filenames and datestamp $ printf '%s\n' *.dat* | sort -t. -k3,4 filename_1.dat.20120430.Z filename_2.dat.20120430.Z filename_3.dat.20120430.Z filename_1.dat.20120501.Z filename_2.dat.20120501.Z... (12 Replies)
Discussion started by: robertbrown624
12 Replies

3. Shell Programming and Scripting

Urgent ...pls Sorting files based on timestamp and picking the latest file

Hi Friends, Newbie to shell scripting. Currently i have used the below to sort data based on filenames and datestamp $ printf '%s\n' *.dat* | sort -t. -k3,4 filename_1.dat.20120430.Z filename_2.dat.20120430.Z filename_3.dat.20120430.Z filename_1.dat.20120501.Z filename_2.dat.20120501.Z... (1 Reply)
Discussion started by: robertbrown624
1 Replies

4. Shell Programming and Scripting

to pick the latest file modified in a directory

I wan to pick the latest modified file name and redirect it to a file .. ls -tr | tail -1 >file but this is printing file ins side the filename , can anyone help me out (5 Replies)
Discussion started by: vishwakar
5 Replies

5. UNIX for Dummies Questions & Answers

How to pick only the latest files based on the timestamp?

I have a few log files which get generated on a daily basis..So, I need to pick only the ones which get generated for that particular day. -rw-r--r-- 1 staff 510732676 Apr 7 22:01 test.log040711 -rwxrwxrwx 1 staff 2147482545 Apr 7 21:30 test.log.2 -rwxrwxrwx 1 staff 2147482581 Apr 7 19:26... (43 Replies)
Discussion started by: win4luv
43 Replies

6. UNIX for Dummies Questions & Answers

Need command to pick the latest file

Hi In my script i am trying to access mainframe server using FTP, in the server i have filee with the timestamp.I need to get the file with the latest timestamp among them . The server has the below files / ftp> cd /outbox 250 CWD command successful ftp> ls 200 PORT command successful... (4 Replies)
Discussion started by: laxmi131
4 Replies

7. Shell Programming and Scripting

Find the latest directory and loop through the files and pick the error messages

Hi, I am new to unix and shell scripting,can anybody help me in sctipting a requirement. my requirement is to get the latest directory the name of the directory will be like CSB.monthdate_time stamp like CSB.Sep29_11:16 and CSB.Oct01_16:21. i need to pick the latest directory. in the... (15 Replies)
Discussion started by: sudhir_83k
15 Replies

8. Shell Programming and Scripting

how can i pick the latest log file as per below

in the below .. i want to pick the latest logfile which is having JPS.PR inside.. that means i want particularly "spgport040408041223.log:@@@@@@@@ 04:13:09 Adding: JPS.PR." which is latest among these.. is it possible to compare the current time with logfile time ? reptm@xblr0758rop>... (4 Replies)
Discussion started by: mail2sant
4 Replies

9. Shell Programming and Scripting

Pick the latest set of files

I have task in which I need to pickup a set of files from a directory depending on the following criteria: Every month 6 files are expected to arrive at /test. The files come with date timestamp and the latest file set for the month needs to be used Suppose this is the set of files that present... (5 Replies)
Discussion started by: w020637
5 Replies

10. Shell Programming and Scripting

ksh: How to get latest file from a list of files in a directory

Hi, I need to get the latest file from a list of files in a particular directory. Please could anyone help me out to get the file. Thank you, - Jay. (1 Reply)
Discussion started by: Jayathirtha
1 Replies
Login or Register to Ask a Question