Parsing of file for Report Generation (String parsing and splitting)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing of file for Report Generation (String parsing and splitting)
# 1  
Old 02-27-2009
Parsing of file for Report Generation (String parsing and splitting)

Hey guys,

I have this file generated by me... i want to create some HTML output from it.
The problem is that i am really confused about how do I go about reading the file.

The file is in the following format:

Code:
TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy BExpected=yyy CTime=xx CResult=yyy CExpected=yyy ....

The file can continue this way as long as it wants. The A type (for that matter any type) might be repeated again too.
But there would always be 3 fields for a given type, Time, Result and ExpectedResult.

I need to consolidate all of these values for all the different types (A, B, etc.) and create a tabular report for them.

Oh, yes, forgot to mention.... I'm using a shell based on zsh.

Thanks,
Umar
# 2  
Old 02-27-2009
Seems like a job for a higher-level code. Perl, python, awk, anything like that.
If the line can be very long you could expect unexpected problems (like storing too much data into a variable in shell or the line being truncated).

On the other side such trick could probably work as well: "for var in $(cat file); do .... done" or better - use "read".... you might need to play with the FS/IFS settings.

Anyway - I strongly suggest higher level language.
# 3  
Old 02-27-2009
Quote:
Originally Posted by adderek
If the line can be very long you could expect unexpected problems (like storing too much data into a variable in shell or the line being truncated).

There is no reason either should happen. The variable length is limited only by available memory.
Quote:

On the other side such trick could probably work as well: "for var in $(cat file); do .... done"

The is not the way to read a file. It sets var to each word in the file, not each line.
Quote:
or better - use "read".... you might need to play with the FS/IFS settings.

There is no need to play with IFS to read a file line by line.
# 4  
Old 02-27-2009
Quote:
Originally Posted by umar.shaikh
Hey guys,

I have this file generated by me... i want to create some HTML output from it.
The problem is that i am really confused about how do I go about reading the file.

The file is in the following format:

Code:
TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy BExpected=yyy CTime=xx CResult=yyy CExpected=yyy ....

The file can continue this way as long as it wants. The A type (for that matter any type) might be repeated again too.
But there would always be 3 fields for a given type, Time, Result and ExpectedResult.

I need to consolidate all of these values for all the different types (A, B, etc.) and create a tabular report for them.

What does "consolidate" mean? What do you want to do with the line?

What type of output do you want?


Is this something like it:

Code:
printf "<table>\n"
while read a b line
do
  eval "$line" ## set all the variables
  printf "<tr><td>%s<td>%s<td>%s<td>%s<td>%s<td>%s<td>%s<td>%s<td>%s<td>%s<td>%s\n" \
    "$a" "$b" "$ATime" "$AResult" "$AExpected" "$BTime" "$BResult" "$BExpected" \
    "$CTime" "$CResult" "$CExpected" 
done < "$FILE"
printf "</table>\n"

Quote:

Oh, yes, forgot to mention.... I'm using a shell based on zsh.

Use POSIX shell syntax for scripting; use extensions only when it is more efficient.
# 5  
Old 02-27-2009
Quote:
Originally Posted by cfajohnson

There is no reason either should happen. The variable length is limited only by available memory.
I have experienced variable size problems on some environments. Not sure what were the conditions though.

The is not the way to read a file. It sets var to each word in the file, not each line.
Depends on the shell/system/environment. Cannot give you details out from my head but it could be "inteligent" and read word${IFS}by${IFS}word or line by line.


There is no need to play with IFS to read a file line by line.
Might be needed if you choose the way mentioned above.
# 6  
Old 02-28-2009
Quote:
Originally Posted by adderek
Seems like a job for a higher-level code. Perl, python, awk, anything like that.
Ok. I am open to use awk or jython.

Quote:
Originally Posted by cfajohnson
What does "consolidate" mean? What do you want to do with the line?

What type of output do you want?

Is this something like it:

Code:
printf "<table>\n"
while read a b line
do
  eval "$line" ## set all the variables
  printf "<tr><td>%s<td>%s<td>%s<td>%s<td>%s<td>%s<td>%s<td>%s<td>%s<td>%s<td>%s\n" \
    "$a" "$b" "$ATime" "$AResult" "$AExpected" "$BTime" "$BResult" "$BExpected" \
    "$CTime" "$CResult" "$CExpected" 
done < "$FILE"
printf "</table>\n"

I read your code. I did think of something like that but it does not help me. Smilie
I guess I might have not been clear enough. The variables A,B,C,... could be of any number.
So i could have something ranging from A till maybe M (i.e. each A would have ATime, AResult, AExpected....... continuing like this all the way till MTime, MResult, MExpected)
Hope you follow...

Ok. I'll give you an example:
File:
Code:
TID1 Name1 ATime=2 AResult=PASS AExpected=PASS BTime=3 BResult=PASS BExpected=PASS CTime=3 CResult=PASS CExpected=FAIL
TID2 Name2 ATime=2 AResult=FAIL AExpected=PASS
TID3 Name3 ATime=2 AResult=PASS AExpected=PASS BTime=3 BResult=PASS BExpected=PASS CTime=3 CResult=FAIL CExpected=FAIL DTime=2 DResult=PASS DExpected=PASS

The output file would be more like this: (I'm thinking in these lines as of now)
HTML Code:
<html>
Below table values are in Time, Result and Expected Result format<br>
<table border=1>
<tr rowspan=2><td rowspan=2>TID1</td><td rowspan=2>Name1</td><td colspan=3>JobA</td><td colspan=3>JobB</td><td colspan=3>JobC</td></tr>
<tr><td>2</td><td>PASS</td><td>PASS</td><td>3</td><td>PASS</td><td>PASS</td><td>3</td><td>PASS</td><td>FAIL</td></tr>
<tr rowspan=2><td rowspan=2>TID2</td><td rowspan=2>Name2</td><td colspan=3>JobA</td></tr>
<tr><td>2</td><td>FAIL</td><td>PASS</td></tr>
<tr rowspan=2><td rowspan=2>TID3</td><td rowspan=2>Name3</td><td colspan=3>JobA</td><td colspan=3>JobB</td><td colspan=3>JobC</td><td colspan=3>JobD</td></tr>
<tr><td>2</td><td>PASS</td><td>PASS</td><td>3</td><td>PASS</td><td>PASS</td><td>3</td><td>FAIL</td><td>FAIL</td><td>2</td><td>PASS</td><td>PASS</td></tr>
</table>
</html>
# 7  
Old 02-28-2009
AWK seems to be a good choice (however I have not analyzed the example in details). It seems like you are going to greate a table with cells spanning across multiple rows/columns. Think twice if this is what you need. Probably it would be acceptable to have the same fields repeated in multiple rows instead of using rowspan. That would make your code much easier.
You might need to count the entries first and then make a second run to fill the output document. Easy way is to parse that file 2x with a different code - however this is not optimal (low performance, too much I/O). That might be the best option for you.
There are plenty of other ways to do this (including PERL with "split" which can be very slow).

If by any chance you experience problems with number of "columns" in AWM try changing the awk (mawk, nawk, gawk,... there is plenty of this stuff with different limitations).

Good luck
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing a log file and creating a report script

The log file is huge and lot of information, i would like to parse and make a report . below is the log file looks like: REPORT DATE: Mon Aug 10 04:16:17 CDT 2017 SYSTEN VER: v1.3.0.9 TERMINAL TYPE: prod SYSTEM: nb11cu51 UPTIME: 04:16AM up 182 days 57 mins min MODEL, TYPE, and SN:... (8 Replies)
Discussion started by: amir07
8 Replies

2. UNIX for Dummies Questions & Answers

Help on parsing Oracle RMAN output for string and print sections of a file

Hi, I need some advise on how to print 'sections' of the attached file. I am searching for some that says Marked Corrupt and print some lines after it. At the moment I am running the command below: sed -n -e '/Marked Corrupt/{N;N;p;}' rman_list_validate.txtThis gives me the following... (1 Reply)
Discussion started by: newbie_01
1 Replies

3. Shell Programming and Scripting

Parsing diff output into report

Hello all; lat week I was able to get some assistance on creating a summary report from a file generated by a "comm" comparison of twp CSV files...turn out now that I am being asked for a detail report as well...this is beyond my knowledge of perl (and yes I have to use perl)..also please keep... (5 Replies)
Discussion started by: gvolpini
5 Replies

4. Shell Programming and Scripting

Parsing a long string string problem for procmail

Hi everyone, I am working on fetchmail + procmail to filter mails and I am having problem with parsing a long line in the body of the email. Could anyone help me construct a reg exp for this string below. It needs to match exactly as this string. GetRyt... (4 Replies)
Discussion started by: cwiggler
4 Replies

5. Shell Programming and Scripting

Need help on parsing string

for i in `cat list`;do lol=`curl -m 2 -s ${i} | grep 'class=info' | cut -d '>' -f14 | cut -d '<' -f1 | sed '/^$/d'`; if ;then echo "$i,$lol" >> dirty; echo "$i,$lol";fi; done cut: you must specify a list of bytes, characters, or fields Try `cut --help' for more information. it gave me that... (0 Replies)
Discussion started by: p33plime
0 Replies

6. Shell Programming and Scripting

String parsing

Hi, name=VDSL_TTV_ HN_SUB create coid=MA5603U phone=5678 portpolicy=APortSelectionPolicy rfu10=TTV rfu3=Dot1q sz7_portmode=VDSL2 rfu5=1234 srprofile.sy_profname=$ADSL_TTV_SubProfile1 I have a line like this. Its a single line.I need the output as name=VDSL_TTV_ HN_SUB create... (1 Reply)
Discussion started by: giri_luck
1 Replies

7. Shell Programming and Scripting

String-parsing!

I need the perl solution for the following : $string="I LOVE INDIA" now, in a new string i need the first character of each word... that is string2 should be "ILN". (10 Replies)
Discussion started by: vijay_0209
10 Replies

8. Shell Programming and Scripting

Parsing out the logs and generating report

My file will contain following(log.txt): start testcase: config loading ...... error XXXX ..... end testcase: config loading, result failed start testcase: ping check ..... error ZZZZZ ..... error AAAAA end testcase: Ping check, result failed I am expecting below output. ... (4 Replies)
Discussion started by: shellscripter
4 Replies

9. Shell Programming and Scripting

Perl parsing compared to Ksh parsing

#! /usr/local/bin/perl -w $ip = "$ARGV"; $rw = "$ARGV"; $snmpg = "/usr/local/bin/snmpbulkget -v2c -Cn1 -Cn2 -Os -c $rw"; $snmpw = "/usr/local/bin/snmpwalk -Os -c $rw"; $syst=`$snmpg $ip system sysName sysObjectID`; sysDescr.0 = STRING: Cisco Internetwork Operating System Software... (1 Reply)
Discussion started by: popeye
1 Replies

10. Shell Programming and Scripting

parsing a string

hi i am new to shelll scripting I need to parse a string like "abc,def,ghie,jkl" And assign them to some variables like hs1 = abc hs2 = def hs3 = ghi hs4 = jkl Please help (4 Replies)
Discussion started by: satish@123
4 Replies
Login or Register to Ask a Question