Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Parse a string in XML file using shell script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parse a string in XML file using shell script
# 36  
Old 11-27-2007
Parse a string in XML file using shell script
Hi Matrixmadhan,

Sorry I don't know how to use the code tags. I'm just new here. I have another script tail.sh wherein I get the result of the cdr.txt. It gets the last CDR processed before CDR*.tmp which is the current CDR being processed. I don't know if I did the right thing to create a seperate script which is tail.sh to get the last complete CDR. CDR*.tmp should not be processed by the script since it's not yet completed. Hope you understand my explanation. Everytime there's a new complete CDR, the script you created should process it. Thanks again!

cat tail.sh
ls -ltr | grep CDR | tail -2 | nawk '{print $9}' > cdrfile.txt
sed -n 1p cdrfile.txt > cdr.txt

if I run tail.sh, the output of cdr.txt:
CDR3310011.7 ##### this is the CDR that the script should process, the last completed CDR. Then if there's a new CDR completed, the script should process it again and append to a file.

cdrs:
-rw-r----- 1 root transfer 5243297 Oct 27 10:39 CDR3310010.4
-rw-r----- 1 root transfer 5243090 Oct 27 10:47 CDR3310010.5
-rw-r----- 1 root transfer 5242988 Oct 27 10:54 CDR3310010.6
-rw-r----- 1 root transfer 5243269 Oct 27 11:02 CDR3310010.7
drwxrwx--- 2 root stats 24576 Oct 27 11:05 stats
-rw-r----- 1 root transfer 5243317 Oct 27 11:09 CDR3310011
-rw-r----- 1 root transfer 5242906 Oct 27 11:16 CDR3310011.1
-rw-r----- 1 root transfer 5243095 Oct 27 11:23 CDR3310011.2
-rw-r----- 1 root transfer 5243178 Oct 27 11:30 CDR3310011.3
-rw-r----- 1 root transfer 5242963 Oct 27 11:38 CDR3310011.4
-rw-r----- 1 root transfer 5243133 Oct 27 11:45 CDR3310011.5
-rw-r----- 1 root transfer 5243044 Oct 27 11:52 CDR3310011.6
-rw-r----- 1 root transfer 5243054 Oct 27 11:59 CDR3310011.7
-rw-r----- 1 root transfer 272109 Oct 27 12:00 CDR3310011.tmp
# 37  
Old 11-28-2007
Quote:
Sorry I don't know how to use the code tags. I'm just new here.
You can find the icon to enable code tags on panel above the message block

Quote:
I don't know if I did the right thing to create a seperate script which is tail.sh to get the last complete CDR.

cat tail.sh
ls -ltr | grep CDR | tail -2 | nawk '{print $9}' > cdrfile.txt
sed -n 1p cdrfile.txt > cdr.txt
From the above, you are interested in last but one file, the same could be achieved by the following

Code:
ls -lrt | awk '/CDR/ { before = curr; curr = $9 }END{ print before }'

so, with the above you could make sure, whether current file that is available needs to be processed or not.

With tail.sh or the command given above check whether the retrieved filename contains the term "tmp", if so that is not the file to be processed, if not pass that filename to the script for processing.

Hope this makes it clear ! Smilie
# 38  
Old 11-29-2007
Parse a string in XML file using shell script
Hi Matrixmadhan,

I've tried to use the command that you provided since it's better to have just on script that to call on the result of tail.sh but I get the following errors below. I tried to change some part of the syntax on the unix command that you recommended but still the same, when I'm just using cat it works. Also on how the script can determine if there's a new complete CDR? Thanks for being so patient with me. I really don't know much about creating script, sorry! Smilie Thanks a lot!

PHP Code:
root :ALCP2  # ./cdrs.pl
awk: syntax error near line 1
awk: illegal statement near line 1
Cannot open file for read :No such file or directory at ./cdrs.pl line 13. 
line 13 is:
PHP Code:
open inputfayl or die "Cannot open file for read :$!"; 
script:
PHP Code:
open(OUTPUT, ">>", "output.txt");
chomp(@inputfayl = `ls -lrt | awk '/SMSCDR/ { before = curr; curr = $9 }END{ print before }'`); 
while (<@inputfayl>) {
if (/CDR/) {
$inputfayl=$_;
}
}
open inputfayl or die "Cannot open file for read :$!";

while(<inputfayl>) {
chomp;
my @arr = split(/></);
foreach (@arr) {
if( /xml version/ ) {
$c++;
print OUTPUT "\n";
}
if( />/ && /</ ) {
if( $c == 1 ) {
s/(.*)>(.*)<.*$/\1|\2/;
my($tmp1, $tmp2) = split(/\|/);
$data .= (" " . $tmp2);
printf OUTPUT "%s ", $tmp1;
}
else {
s/(.*)>(.*)<.*$/\2/;
printf OUTPUT "%s ", $_;
}
}
}
print OUTPUT "\n";
print OUTPUT "$data\n" if( /xml version/i );
}

close inputfayl;


exit 0 
# 39  
Old 11-29-2007
Parse a string in XML file using shell script
Hi Matrixmadhan,

The command that you gave works now, I just put \ in the $9. So I will not use the tail.sh script that I created before. What I need to do now is to run the script when there's a new completed CDR which is the last CDR before the CDR*.tmp. For the example below:

the current CDR which is being dumped by the system is CDR3310011.tmp. Then after few minutes (no specific time) the CDR3310011.tmp CDR will be the completed CDR which will be CDR3310011.8 and another *.tmp will be created for the current CDR. The script that you created should process CDR3310011.8 and everytime there's a new completed CDR. Thanks a lot! Hope you can help me with it. I'm also trying to check how I can do that. Smilie

Code:
-rw-r----- 1 root transfer 5243269 Oct 27 11:02 CDR3310010.7
drwxrwx--- 2 root stats 24576 Oct 27 11:05 stats
-rw-r----- 1 root transfer 5243317 Oct 27 11:09 CDR3310011
-rw-r----- 1 root transfer 5242906 Oct 27 11:16 CDR3310011.1
-rw-r----- 1 root transfer 5243095 Oct 27 11:23 CDR3310011.2
-rw-r----- 1 root transfer 5243178 Oct 27 11:30 CDR3310011.3
-rw-r----- 1 root transfer 5242963 Oct 27 11:38 CDR3310011.4
-rw-r----- 1 root transfer 5243133 Oct 27 11:45 CDR3310011.5
-rw-r----- 1 root transfer 5243044 Oct 27 11:52 CDR3310011.6
-rw-r----- 1 root transfer 5243054 Oct 27 11:59 CDR3310011.7
-rw-r----- 1 root transfer 272109 Oct 27 12:00 CDR3310011.tmp

# 40  
Old 11-29-2007
So, the design goes like this, the script should process the last but one file which is not a tmp file ( that one that ends with tmp )

so once that is being processed, another tmp file would be created and the script should poll for it and once done it should start processing that

Please correct me if am wrong !

something like might help but it is made such a way to run ever
Code:
open(OUTPUT, ">>", "output.txt");

while (1) {

#Make it as ever running process

$file = `ls -lrt | awk '/SMSCDR/ { before = curr; curr = $9 }END{ print before }'`; 

#select the file which is last but one

next if ( $file =~ /tmp/ );

#if its the 'tmp' file dumper process is still active so don't process it else continue processing

open (FILE, "<", $file) or die  "Unable to open file $file <$!> \n";

#open the file and start processing

while(<FILE>) {
chomp;
my @arr = split(/></);
foreach (@arr) {
if( /xml version/ ) {
$c++;
print OUTPUT "\n";
}
if( />/ && /</ ) {
if( $c == 1 ) {
s/(.*)>(.*)<.*$/1|2/;
my($tmp1, $tmp2) = split(/|/);
$data .= (" " . $tmp2);
printf OUTPUT "%s ", $tmp1;
}
else {
s/(.*)>(.*)<.*$/2/;
printf OUTPUT "%s ", $_;
}
}
}
print OUTPUT "\n";
print OUTPUT "$data\n" if( /xml version/i );
}

close (FILE);

#continue with while loop, by this time dumper process might have completed and next file ready for processing
}

close(OUTPUT);

exit 0;

# 41  
Old 12-04-2007
Parse a string in XML file using shell script
Hi Matrixmadhan,

Sorry for the late reply. I tried the script that you provided. My problem is while it's waiting for a new completed CDR, it continously repeats to process the last completed file that's why on the output file the result of processing of last complete CDR repeated lots of times. What I wanted to do is to process the last completed file, then if there's a new completed CDR, it will process the new CDR and if there's no new completed CDR it will not process. I hope you can help me modify the script. Thanks a lot for being patient with me and for all the help.

Code:
#! /usr/local/bin/perl 

my $c = 0;

open(OUTPUT, ">>", "output.txt");

while (1) {

chomp(@inputfayl = `ls -lrt | awk '/SMSCDR/ { before = curr; curr = \$9 }END{ print before }'`);
   while (<@inputfayl>) {
      if (/CDR/) {
        $inputfayl=$_;
        print OUTPUT "$inputfayl ##########################################################################################\n";
      }
    }
next if ( $inputfayl =~ /tmp/ );
open inputfayl or die "Cannot open file for read :$!";

while(<inputfayl>) {
  chomp;
    my @arr = split(/></);
    foreach (@arr) {
    if( /xml version/ ) {
      $c++;
      print OUTPUT "\n";
    }
    if( />/ && /</ ) {
      if( $c == 1 ) {
        s/(.*)>(.*)<.*$/\1|\2/;
        my($tmp1, $tmp2) = split(/\|/);
        $data .= (" " . $tmp2);
        printf OUTPUT "%s ", $tmp1;
      }
      else {
        s/(.*)>(.*)<.*$/\2/;
        printf OUTPUT "%s ", $_;
      }
    }
  }
  print OUTPUT "\n";
  print OUTPUT "$data\n" if( /xml version/i );
}

close inputfayl;

}

close (OUTPUT);


exit 0;

# 42  
Old 12-04-2007
Code:
chomp(@inputfayl = `ls -lrt | awk '/SMSCDR/ { before = curr; curr = \$9 }END{ print before }'`);
   while (<@inputfayl>) {
      if (/CDR/) {
        $inputfayl=$_;
        print OUTPUT "$inputfayl ##########################################################################################\n";
      }
    }

The output of ls -lrt | ..
is going to be just one file... there is no need to process the @inputfayl,

that could be assigned to a scalar and used directly
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parse xml in shell script and extract records with specific condition

Hi I have xml file with multiple records and would like to extract records from xml with specific condition if specific tag is present extract entire row otherwise skip . <logentry revision="21510"> <author>mantest</author> <date>2015-02-27</date> <QC_ID>334566</QC_ID>... (12 Replies)
Discussion started by: madankumar.t@hp
12 Replies

2. Shell Programming and Scripting

Using shell command need to parse multiple nested tag value of a XML file

I have this XML file - <gp> <mms>1110012</mms> <tg>988</tg> <mm>LongTime</mm> <lv> <lkid>StartEle=ONE, Desti = Motion</lkid> <kk>12</kk> </lv> <lv> <lkid>StartEle=ONE, Source = Velocity</lkid> <kk>2</kk> </lv> <lv> ... (3 Replies)
Discussion started by: NeedASolution
3 Replies

3. Shell Programming and Scripting

How to Parse the XML data along with the URL in Shell Script?

Hi, Can anybody help to solve this. I want to parse some xmldata along with the URL in the Shell. I'm calling the URL via the curl command Given below is my shell script file export... (7 Replies)
Discussion started by: Megala
7 Replies

4. Shell Programming and Scripting

How to parse xml file in variable-string?

In the wake of the post: how-parse-following-xml-file Thank you for the very useful chakrapani response 302355585-post4 ! A close question. How to pass a file to xmllint in variable? For example, let it be: NEARLY_FILE='<?xml version="1.0" encoding="iso-8859-1"?><html><set label="09/07/29"... (0 Replies)
Discussion started by: OleM2k
0 Replies

5. Shell Programming and Scripting

Shell script (not Perl) to parse xml with awk

Hi, I have to make an script according to these: - I have couples of files like: xxxxxxxxxxxxx.csv xxxxxxxxxxxxx_desc.xml - every xml file has diferent fields, but keeps this format: ........ <defaultName>2011-02-25T16:43:43.582Z</defaultName> ........... (2 Replies)
Discussion started by: Pluff
2 Replies

6. Shell Programming and Scripting

Parse XML file in shell script

Hi Everybody, I have an XML file containing some data and i want to extract it, but the specific issue in my file is that the data is repeated some times like the following example : <section1> <subsection1> X=... Y=... Z=... <\subsection1> <subsection2> X=... Y=... Z=...... (2 Replies)
Discussion started by: yassine
2 Replies

7. Shell Programming and Scripting

regex/shell script to Parse through XML Records

Hi All, I have been working on something that doesn't seem to have a clear regex solution and I just wanted to run it by everyone to see if I could get some insight into the method of solving this problem. I have a flat text file that contains billing records for users, however the records... (5 Replies)
Discussion started by: Jerrad
5 Replies

8. Shell Programming and Scripting

Parse XML file into CSV with shell?

Hi, It's been a few years since college when I did stuff like this all the time. Can someone help me figure out how to best tackle this problem? I need to parse a file full of entries that look like this: <eq action="A" sectyType="0" symbol="PGR" exch="CA" curr="VEF" sess="NORM"... (7 Replies)
Discussion started by: Pcushing
7 Replies

9. Shell Programming and Scripting

Need help in creating a Unix Script to parse xml file

Hi All, My requirement is create an unix script to parse the xml file and display the values of the Elements/value between the tags on console. Like say, I would like to fetch the value of errorCode from the below xml which is 'U007' and display it. Can we use SED command for this? I have tried... (10 Replies)
Discussion started by: Anil.Wmg
10 Replies

10. Shell Programming and Scripting

Parse String in XML file

Hello All, I am new to this and I need to parse an XML file. Here's the XML Input File: <Report version="1.2"> <summary fatals="0" testcases="1" expected_fails="0" unexpected_passes="0" warnings="9" tests="21" errors="0" fails="1" passes="20" /> <testresult... (4 Replies)
Discussion started by: racbern
4 Replies
Login or Register to Ask a Question