Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

parse of lines with different delimiters

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting parse of lines with different delimiters
# 1  
Old 05-16-2008
parse of lines with different delimiters
Hi,

I am having huge file with the following lines.

2007:10:01:00:00:49:GMT: subject=BMRA.BM.T_ABTH7.FPN, message={SD=2007:10:01:00:00:00:GMT,SP=5,NP=2,TS=2007:10:01:01:00:00:GMT,VP=0.0,TS=2007:10:01:01:30: 00:GMT,VP=0.0}
2007:10:01:00:00:49:GMT: subject=BMRA.BM.T_ABTH7G.FPN, message={SD=2007:10:01:00:00:00:GMT,SP=5,NP=2,TS=2007:10:01:01:00:00:GMT,VP=0.0,TS=2007:10:01:01:30: 00:GMT,VP=0.0}


I need to parse them into the following format.

2007-10-01,T_ABTH7,2007-10-0100:00:00,5,0.0
2007-10-01,T_ABTH7G,2007-10-0100:00:00,5,0.0

Is there a way to parse the entire file without reading a single line of file and formating the output.

Thanks in advance.
# 2  
Old 05-16-2008
there are so many occurences of 2 and 0.0 in the input lines.

Highlight the portions of input line that u want to report


Thanks
Penchal
# 3  
Old 05-16-2008
Hi Penchal,

I want to parse the highlighted values.

2007:10:01:00:00:49:GMT: subject=BMRA.BM.T_ABTH7.FPN, message={SD=2007:10:01:00:00:00:GMT,SP=5,NP=2,TS=2007:10:01:01:00:00:GMT,VP=0.0,TS=2007:10:01:01:30: 00:GMT,VP=0.0}

First output column (2007-10-01): SD=2007:10:01:00:00:00

Second column (T_ABTH7): subject=BMRA.BM.T_ABTH7.FPN

Third Column (2007-10-0100:00:00): TS=2007:10:01:01:00:00

Fourth Column (5): SP=5

Fifth Column(0.0) : VP=0.0

Output for a single line :
2007-10-01,T_ABTH7,2007-10-0100:00:00,5,0.0

Please let me know if this clear.
# 4  
Old 05-16-2008
One way is to use the pattern matching and substitution capabilities of ksh93 or bash
Code:
#!/usr/bin/ksh93

IFS=','
while read c1 c2 c3 c4 c5 c6 rest
do
   tmp1=${c2##*SD=}
   tmp1=${tmp1%%:00:00:00:GMT}
   tmp2=${c1##*=BMRA.BM.}
   tmp3=${c5##TS=}
   tmp3=${tmp3%%:GMT}
   tmp3A=${tmp3:0:10}
   printf "%s,%s,%s%s,%s,%s\n" ${tmp1//:/-} ${tmp2%%.FPN} ${tmp3A//:/-} ${tmp3:11} ${c3##SP=} ${c6##VP=}
done < file

Output:
Code:
2007-10-01,T_ABTH7,2007-10-0101:00:00,5,0.0
2007-10-01,T_ABTH7G,2007-10-0101:00:00,5,0.0

# 5  
Old 05-16-2008
Hi Murphy,

Thanks for the reply.

I am getting the following error message while runnig the script.

tmp3A=${tmp3:0:10}: bad substitution
# 6  
Old 05-16-2008
Solution with gawk:

Code:
#!/usr/bin/awk -f
BEGIN {FS=","; OFS=","}
{
          print \
                  gensub(/^.+([0-9][0-9][0-9][0-9]:[0-9][0-9]:[0-9][0-9]).+$/, "\\1", 1, $2),
                  gensub(/^.+subject=BMRA.BM.(.+).FPN/, "\\1", 1, $1),
                  gensub(/^TS=(.+):GMT/, "\\1", 1, $5),
                  gensub(/^SP=(.+)/, "\\1", 1, $3),
                  gensub(/^VP=(.+)/, "\\1", 1, $6)
}

If your version of awk doesn't support gensub(), there is a solution with substr() and match(). Let me know.
# 7  
Old 05-16-2008
Hi Ripat,

Thanks for the reply. Unfortunately I am not able to run the solution. I think the gensub is not supported. Can you send me the other solution that you have with substr() and match().

Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep lines only with 3 delimiters

Hi All, my file has following Data 04:38:34 02:03 24:40 02:09:58 09:13 03:04:11 02:09:58 35:00 I want to display only lines with 3 fields. ie.. 04:38:34 02:09:58 03:04:11 (6 Replies)
Discussion started by: Arunselvan
6 Replies

2. Shell Programming and Scripting

awk: Print fields between two delimiters on separate lines and send to variables

I have email headers that look like the following. In the end I would like to accomplish sending each email address to its own variable, such as: user1@domain.com='user1@domain.com' user2@domain.com='user2@domain.com' user3@domain.com='user3@domain.com' etc... I know the sed to get rid of... (11 Replies)
Discussion started by: tay9000
11 Replies

3. Shell Programming and Scripting

Concatinating the lines based on number of delimiters

Hi, I have a problem to concatenate the lines based on number of delimiters (if the delimiter count is 9 then concatenate all the fields & remove the new line char bw delimiters and then write the following data into second line) in a file. my input file content is Title| ID| Owner|... (4 Replies)
Discussion started by: bi.infa
4 Replies

4. Shell Programming and Scripting

parse lines

I have file which is having 500 lines. I want to get the first 100 lines then sleep, then again next 100 lines sleep so now till the end of the file. Can someone tell me in perl and bash. also i want to do it in threads. Thanks.. (6 Replies)
Discussion started by: Anjan1
6 Replies

5. Shell Programming and Scripting

How to parse a numeric string without any delimiters?

Hi , I have a number say 12345001 which needs to be parsed. Its a number that has no delimiters.I have to read the last three digits and then the rest of digits irrespective of the total length of the number. The digits then have to be swapped and changed to a fixed length. The fillers to be... (10 Replies)
Discussion started by: Sheel
10 Replies

6. UNIX for Dummies Questions & Answers

How to parse 2 particular lines from Command output

Hi All, I need help on the following req. I am getting output of a command as follows: 16377612 total memory 3802460 used memory 2827076 active memory 681948 inactive memory 12575152 free memory 477452 buffer memory I want to compute used... (1 Reply)
Discussion started by: mailsara
1 Replies

7. Shell Programming and Scripting

Parse out specific lines

Hello, For the life of me, I can't figure out how to extract only certain lines of a file. For example, the file contains: project.max-sem-ids privileged 1.02K - deny - system 16.8M max deny ... (2 Replies)
Discussion started by: PointyWombat
2 Replies

8. Shell Programming and Scripting

commenting out lines between two delimiters

Hi All, I am struggling to get my head around the following issue. I am having to comment out lines between two delimiters by placing an asterix in position 7 but retain all lines in the file and in the same order. so for example a file containing: ... ... DELIM1 ... ... DELIM2... (2 Replies)
Discussion started by: Bruble
2 Replies

9. Shell Programming and Scripting

Parse and count lines

I have a data file in the following format (refer to input file) with multiple lines containing some information. I need an output file to loop thorough the input file with summarized information as seen below (refer to output file) ‘Date Time' and ‘Beta Id' input file values should be concatenated... (7 Replies)
Discussion started by: shekharaj
7 Replies

10. Shell Programming and Scripting

Insert lines between delimiters

I'm working with a file like: somestuff somemorestuff ... someadditionalstuff STARTTAG ENDTAG someotherstuff somecoolstuff ... somefinalstuffI've got some text (either in a file or piped) to put between STARTTAG and ENDTAG. I was thinking something like grepping for the line number of... (2 Replies)
Discussion started by: BMDan
2 Replies
Login or Register to Ask a Question