Shell Programming and Scripting

Hi,

I am new to Unix shell scripting. Can you please help me with this immediate requirement to code.. The requirement is as given below.

In a directory say Y, I have files like this.

PP_100000_28062006_122731_746.dat
PP_100000_28062006_122731_745.dat
PP_100000_28062006_122734_745.dat
PP_100000_28062006_122732_745.dat
PP_100000_28062006_122801_745.dat

QQ_100001_28062006_122733_745.dat
QQ_100001_28062006_122731_745.dat

RR_100002_28062006_122731_745.dat
RR_100002_28062006_122732_745.dat
RR_100002_28062006_122729_745.dat

format: <type>_<somesequence>_DDMMYYYY_HHMMSS_<organisationID>

Now, I will have an input parameter coming in as 'Directory path' (path to Y) and type (can be PP, QQ or RR. If type is NULL, means I need to process all the 3 types)

Assuming we get type as PP

Now I need to pick up files of this type. We can see there are 5 files matching this string.
Now, I need to check the date and time stamp as available in the file names (and not the unix system date timestamp), compare it, and pick up the file with the lowest date timestamp in its name, first - and process it.
As you see I have 2 files with same timestamp here - PP_100000_28062006_122731_746.dat and PP_100000_28062006_122731_745.dat. In this case I have to process it in ascending order of <organisationID>. ie, PP_100000_28062006_122731_745.dat first, followed by PP_100000_28062006_122731_746.dat

Once processed, I have to pick the next lowest date time stamp file for processing.

I will be archiving successfully processed files in another directory, and renaming failed files to say FAILED (suffix) so that it wont be picked up again.

Any sample code/ pointers/ suggestions would be of great help.

Thanks a lot in advance,
Chindhu

Sorting the files is simple if you can keep your head with the sort key definitions (man sort):

Code:

...
patt=$1         ;# get pattern to match
# do some validation on pattern (your code)
#
# loop through the files in order
ls $patt*| sort -t'_' -k3.5,3.8 -k3.3,3.4 -k3.1,3.2 -k4 -k5| while read a
do
  print  "processing $a"
# process each file here (your code)

# check to see if it worked
  if [ (some condition to test whether it worked or not ) ]; then
    print "Processed successfully"
    mv $a (processed files directory)
  else
    print "Processing failed"
    mv $a FAILED$a
  fi
done
...

hope this helps

cheers

Hi Thestevew,

Thanks for your reply and sample code.
I will try this out.
I was actually trying out some logic with grep, cut and sort.

Thanks,
Chindhu

Hi Steve,

I have a smiliar requirement
But I was unable to figure out the sort command used .
If u can help me on that that will be great.


The are many files in a folder Which are of the below types

Ack_to_MDS_20070809141159.xml
Ack_to_MDS_20070809141157.xml
Ack_to_MDS_20070809141155.xml
Ack_to_MDS_20070809141151.xml
Ack_to_MDS_20070809141149.xml
Ack_to_MDS_20070809141148.xml
.
.
.
.

The files are of format Ack_to_MDS_YYYYMMDDHHMMSS.xml

The Content of the files are like this

Later I need to cat the files based on the timestamps present in the file format i.e Ack_to_MDS_YYYYMMDDHHMMSS.xml.
The Sort order is the oldest file contents needs to be at the top and the Latest file at the end.

i.e. If theese are the files

Ack_to_MDS_20070809141159.xml
Ack_to_MDS_20070809141157.xml
Ack_to_MDS_20070809141155.xml
Ack_to_MDS_20070809141151.xml
Ack_to_MDS_20070809141149.xml
Ack_to_MDS_20070809141148.xml


Then The Sorted Order is
Ack_to_MDS_20070809141148
Ack_to_MDS_20070809141149
Ack_to_MDS_20070809141151
Ack_to_MDS_20070809141155
Ack_to_MDS_20070809141157
Ack_to_MDS_20070809141159

Later The concatenation order is also as the above .

Then Append the Tags <ACKNOWLEDGEMENTS> and </ACKNOWLEDGEMENTS> to the first line and last line of the concatenated file.

Please Gurus
Need Help and Advice.

Thanks & Regards
Srikanth GR

Here you have , two ways :

If you want to keep the result of the concat process in 1 file:

Code:

ls Ack*|xargs -i ksh -c '(echo "<ACKNOWLEDGEMENTS>";cat {};echo "</ACKNOWLEDGEMENTS>") ' >file

If you want to change the source files:

Code:

ls Ack*|xargs -i ksh -c '(echo "<ACKNOWLEDGEMENTS>";cat {};echo "</ACKNOWLEDGEMENTS>") >{}.tmp;mv {}.tmp {}'