Shell Programming and Scripting

Hi Gurus, Experts,

I am facing a problem to concatenate all the files based on timeStamps.
The Problem is like this .

The are many files in a folder Which are of the below types

Ack_to_MDS_20070809141159.xml
Ack_to_MDS_20070809141157.xml
Ack_to_MDS_20070809141155.xml
Ack_to_MDS_20070809141151.xml
Ack_to_MDS_20070809141149.xml
Ack_to_MDS_20070809141148.xml
.
.
.
.

The files are of format Ack_to_MDS_YYYYMMDDHHMMSS.xml

The Content of the files are like this


<ACKNOWLEDGEMENTS>
<ACKNOWLEDGEMENT>
<ACK_TYPE>L</ACK_TYPE>
<ACK_ID>

<PROPERTY1>SPC1238</PROPERTY1>
<PROPERTY2>4</PROPERTY2>
<PROPERTY3>SHIP_TO</PROPERTY3>

</ACK_ID>
<ACK_STATUS>S</ACK_STATUS>
<ACK_COST>10.52</ACK_COST>
<ACK_ERROR_MESSAGE>NO ERROR</ACK_ERROR_MESSAGE>
</ACKNOWLEDGEMENT>
<ACKNOWLEDGEMENT>
<ACK_TYPE>L</ACK_TYPE>
<ACK_ID>

<PROPERTY1>NCB10039</PROPERTY1>
<PROPERTY2>-1</PROPERTY2>
<PROPERTY3>BILL_TO</PROPERTY3>

</ACK_ID>
<ACK_STATUS>F</ACK_STATUS>
<ACK_COST>100.55</ACK_COST>
<ACK_ERROR_MESSAGE>ERROR</ACK_ERROR_MESSAGE>
</ACKNOWLEDGEMENT>

</ACKNOWLEDGEMENTS>

Now My Requirement is In Each of the files I need to remove the
<ACKNOWLEDGEMENTS> and </ACKNOWLEDGEMENTS>
Which appear in the first and the last line of the file.

Later I need to cat the files based on the timestamps present in the file format.
The Sort order is the oldest file contents needs to be at the top and the Latest fileat the end.

The Append the Tags <ACKNOWLEDGEMENTS> and </ACKNOWLEDGEMENTS> to the first line and last line of the concatenated file.

Please Gurus
Need Help and Advice.

Thanks & Regards
Srikanth GR

Does the order in which the files are concatenated matter? If not (you don't mind plain alphabetical order), you can use this simple pseudocode:

Code:

echo "<ACKNOWLEDGEMENTS>" > new_file_name.xml
for file in *; do
   grep -v "<.*ACKNOWLEDGEMENTS>" $file
done >> new_file_name.xml
echo "</ACKNOWLEDGEMENTS>" >> new_file_name.xml

You may have to make changes in the selection of the files for the for loop (you may have other criteria, like checking only for today's files).

-EDIT
Untested

Hi BlowTorch,

Thanks for you Help.

The File order is like this .
The Oldest File content needs to be concatenated at the begining and the latest ones at end.
It will be in the descending order of the timestamp.

I found out another method like First I will remove the Tags <ACKNOWLEDGEMENTS> and </ACKNOWLEDGEMENTS> in all the files and later concatenate them.

Concatenating the files are fine but the problem is How to use the sort command


Later I need to cat the files based on the timestamps present in the file format i.e Ack_to_MDS_YYYYMMDDHHMMSS.xml.
The Sort order is the oldest file contents needs to be at the top and the Latest file at the end.

i.e. If theese are the files

Ack_to_MDS_20070809141159.xml
Ack_to_MDS_20070809141157.xml
Ack_to_MDS_20070809141155.xml
Ack_to_MDS_20070809141151.xml
Ack_to_MDS_20070809141149.xml
Ack_to_MDS_20070809141148.xml


Then The Sorted Order is
Ack_to_MDS_20070809141148
Ack_to_MDS_20070809141149
Ack_to_MDS_20070809141151
Ack_to_MDS_20070809141155
Ack_to_MDS_20070809141157
Ack_to_MDS_20070809141159

Later The concatenation order is also as the above .


Then I will Append the Tags <ACKNOWLEDGEMENTS> and </ACKNOWLEDGEMENTS> to the first line and last line of the concatenated file.

The script I've posted should cat the files in the order that you need.

to get the right order:

Code:

#!/bin/ksh
echo "<ACKNOWLEDGEMENTS>" > new_file_name.xml
for file in $(nawk -F'[_.]' '{print $(NF-1) OFS $0}' Ack_to_MDS_[0-9][0-9]*.xml | sort -n | cut -d' ' -f2-); do
   grep -v "<.*ACKNOWLEDGEMENTS>" $file
done >> new_file_name.xml
echo "</ACKNOWLEDGEMENTS>" >> new_file_name.xml

Hi srikanthgr1 ,

Let me clarify about your files content, whether all the file has same content or different?

if all the has some content of information i will suggest you one solution,

do the following steps:

1. check the line of the each and every files, if equal
2. set the line number of the every files
eg: suppose your file has 15 lines, you want remove the 14 th and 15 th lines from the file and store it in the new file.
3. sed -n '1,13p' <filename.extn> > Newfile

let me know if it's through error.

Regards,
Siva.P
Bangalore

vgersh99, can you please explain the nawk? Didn't get it...