Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

AWK - Parse a big file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting AWK - Parse a big file
# 1  
Old 03-28-2011
AWK - Parse a big file
INPUT SAMPLE

Code:
 
Symmetrix ID          : 000192601507
Masking View Name     : TS00P22_13E_1
Last updated at       : 05:10:18 AM on Tue Mar 22,2011
Initiator Group Name  : 10000000c960b9cd
   Host Initiators
     {
       WWN  : 10000000c960b9cd
     }
Port Group Name       : 10000000c960b9cd_13E_1
   Director Identification
     {
       FA-13E:1
     }
Storage Group Name    : 10000000c960b9cd_13E_1
Sym Dev                                 Host
Name    Dir:P  Physical Device Name      Lun  Attr  Cap(MB)
------  -----  -----------------------  ----  ----  -------
0149    13E:1  Not Visible                 1              3
014A    13E:1  Not Visible                 2              3
014B    13E:1  Not Visible                 3              3
014C    13E:1  Not Visible                 4              3
                                                    -------
Total Capacity                                           12
Masking View Name     :TS00P22_4E_1
Last updated at       : 05:10:26 AM on Tue Mar 22,2011
Initiator Group Name  : 10000000c960bc81
   Host Initiators
     {
       WWN  : 10000000c960bc81
     }
Port Group Name       : 10000000c960bc81_4E_1
   Director Identification
     {
       FA-4E:1
     }
Storage Group Name    : 10000000c960bc81_4E_1
Sym Dev                                 Host
Name    Dir:P  Physical Device Name      Lun  Attr  Cap(MB)
------  -----  -----------------------  ----  ----  -------
0149    04E:1  Not Visible                 1              3
014A    04E:1  Not Visible                 2              3
014B    04E:1  Not Visible                 3              3
014C    04E:1  Not Visible                 4              3
                                                    -------
Total Capacity                                           12

OUTPUT

Code:
Symmetrix ID,DEV,Storage Group Name,Port Group Name,Initiator Group Name,Masking View Name,Port
000192601507,0149,10000000c960b9cd_13E_1,10000000c960b9cd_13E_1,10000000c960b9cd,TS00P22_13E_1,13E:1
000192601507,014A,10000000c960b9cd_13E_1,10000000c960b9cd_13E_1,10000000c960b9cd,TS00P22_13E_1,13E:1
000192601507,014B,10000000c960b9cd_13E_1,10000000c960b9cd_13E_1,10000000c960b9cd,TS00P22_13E_1,13E:1
000192601507,014C,10000000c960b9cd_13E_1,10000000c960b9cd_13E_1,10000000c960b9cd,TS00P22_13E_1,13E:1
000192601507,0149,10000000c960bc81_4E_1,10000000c960bc81_4E_1,10000000c960bc81,TS00P22_4E_1,04E:1
000192601507,014A,10000000c960bc81_4E_1,10000000c960bc81_4E_1,10000000c960bc81,TS00P22_4E_1,04E:1
000192601507,014B,10000000c960bc81_4E_1,10000000c960bc81_4E_1,10000000c960bc81,TS00P22_4E_1,04E:1
000192601507,014C,10000000c960bc81_4E_1,10000000c960bc81_4E_1,10000000c960bc81,TS00P22_4E_1,04E:1

Thanks
Last edited by greycells; 03-28-2011 at 11:34 PM..
# 2  
Old 03-29-2011
Code:
awk '
 BEGIN { OFS=",";
    print "Symmetrix ID,DEV,Storage Group Name,Port Group Name,Initiator Group Name,Masking View Name,Port"
    split("SymmetrixID:,StorageGroupName,PortGroupName,InitiatorGroupName,MaskingViewName", T, ","); 
    for(i in T) GRP[T[i]]=i; }
 ($1$2$3 in GRP) { gsub(":", "", $NF); ID[GRP[$1$2$3]]=$NF }
 /^    / {getdev=0}
 getdev { print ID[1],$1,ID[2],ID[3],ID[4],ID[5],$2 }
 /^------/ { getdev=1 }' infile

# 3  
Old 03-29-2011
Thanks for helping out again Chubler .... works fine ... just one problem , my bad ...some of the records are like this

Code:
 
Masking View Name     : BC_db00p42_43
Last updated at       : 05:09:54 AM on Tue Oct 19,2010
Initiator Group Name  : 10000000c96145d5 *
   Host Initiators
     {
       WWN  : 10000000c96145d5
     }
Port Group Name       : BL_db00p42_43
   Director Identification
     {
       FA-1G:0
       FA-16G:0
       FA-3H:0
       FA-14H:0
     }
Storage Group Name    : BL_db00p42_43
Sym Dev                                 Host
Name    Dir:P  Physical Device Name      Lun  Attr  Cap(MB)
------  -----  -----------------------  ----  ----  -------
0297    01G:0  Not Visible                 1  (M)    235521
        16G:0  Not Visible                 1  (M)   
        03H:0  Not Visible                 1  (M)   
        14H:0  Not Visible                 1  (M)   
0351    01G:0  Not Visible                 2  (M)    860160
        16G:0  Not Visible                 2  (M)   
        03H:0  Not Visible                 2  (M)   
        14H:0  Not Visible                 2  (M)   
0AD4    01G:0  Not Visible                14  (M)    102403
        16G:0  Not Visible                14  (M)   
        03H:0  Not Visible                17  (M)   
        14H:0  Not Visible                17  (M)   
0B31    01G:0  Not Visible                 3  (M)    706562
        16G:0  Not Visible                 3  (M)   
        03H:0  Not Visible                 3  (M)   
        14H:0  Not Visible                 3  (M)   
0B37    01G:0  Not Visible                 4  (M)    706562
        16G:0  Not Visible                 4  (M)   
        03H:0  Not Visible                 4  (M)   
        14H:0  Not Visible                 4  (M)   
0BB8    01G:0  Not Visible                 9  (M)    204806
        16G:0  Not Visible                 9  (M)   
        03H:0  Not Visible                 a  (M)   
        14H:0  Not Visible                 a  (M)   
0BC2    01G:0  Not Visible                 a  (M)    204806
        16G:0  Not Visible                 a  (M)   
        03H:0  Not Visible                 b  (M)   
        14H:0  Not Visible                 b  (M)   
0BCC    01G:0  Not Visible                 b  (M)    204806
        16G:0  Not Visible                 b  (M)   
        03H:0  Not Visible                 c  (M)   
        14H:0  Not Visible                 c  (M)   
0BD6    01G:0  Not Visible                 c  (M)    204806
        16G:0  Not Visible                 c  (M)   
        03H:0  Not Visible                 d  (M)   
        14H:0  Not Visible                 d  (M)   
0BE0    01G:0  Not Visible                 d  (M)    204806
        16G:0  Not Visible                 d  (M)   
        03H:0  Not Visible                 e  (M)   
        14H:0  Not Visible                 e  (M)   
0BEA    01G:0  Not Visible                 e  (M)    204806
        16G:0  Not Visible                 e  (M)   
        03H:0  Not Visible                 f  (M)   
        14H:0  Not Visible                 f  (M)   
0C32    01G:0  Not Visible                13  (M)     81923
        16G:0  Not Visible                13  (M)   
        03H:0  Not Visible                14  (M)   
        14H:0  Not Visible                14  (M)   
0D6E    01G:0  Not Visible                 f  (M)    102413
        16G:0  Not Visible                 f  (M)   
        03H:0  Not Visible                10  (M)   
        14H:0  Not Visible                10  (M)   
0D72    01G:0  Not Visible                10  (M)    102413
        16G:0  Not Visible                10  (M)   
        03H:0  Not Visible                11  (M)   
        14H:0  Not Visible                11  (M)   
1081    01G:0  Not Visible                 5  (M)    588802
        16G:0  Not Visible                 5  (M)   
        03H:0  Not Visible                 5  (M)   
        14H:0  Not Visible                 5  (M)   
1086    01G:0  Not Visible                 6  (M)    588802
        16G:0  Not Visible                 6  (M)   
        03H:0  Not Visible                 6  (M)   
        14H:0  Not Visible                 6  (M)   
14D3    01G:0  Not Visible                 7  (M)    225281
        16G:0  Not Visible                 7  (M)   
        03H:0  Not Visible                 7  (M)   
        14H:0  Not Visible                 7  (M)   
16F4    01G:0  Not Visible                 8  (M)    774145
        16G:0  Not Visible                 8  (M)   
        03H:0  Not Visible                 8  (M)   
        14H:0  Not Visible                 8  (M)   
1E89    01G:0  Not Visible                11  (M)     81923
        16G:0  Not Visible                11  (M)   
        03H:0  Not Visible                12  (M)   
        14H:0  Not Visible                12  (M)   
1E8D    01G:0  Not Visible                12  (M)     81923
        16G:0  Not Visible                12  (M)   
        03H:0  Not Visible                13  (M)   
        14H:0  Not Visible                13  (M)   
                                                    -------
Total Capacity                                      6467669

Note the "Initiator Group Name" field has the * in the end and is not the last field ( its always the 3rd field )

Also ...in the case above ,can i get the last "port" field as 01G:0_16G:0_03H:0_14H:0

Thnks again
# 4  
Old 03-29-2011
Other way:
Code:
awk 'BEGIN{print "Symmetrix ID,DEV,Storage Group Name,Port Group Name,Initiator Group Name,Masking View Name,Port"}
{sub(/:/,": ")}
/Symmetrix/{a=$4}
/Masking View Name/{b=$5}
/Initiator Group Name/{c=$5}
/Port Group Name/{d=$5}
/Storage Group Name/{e=$5}
/Not Visible/{print a","$1","e","d","c","b","$2}' input

Regards
# 5  
Old 03-29-2011
Thanks cgkamal ..... but it does not work in the 2nd scenario i mentioned ...
# 6  
Old 03-29-2011
Modified the code of cgkmal,Hope it will work for both cases. In your second inputfile, there is no Symmetrix Id.
Code:
 awk 'BEGIN{print "Symmetrix ID,DEV,Storage Group Name,Port Group Name,Initiator Group Name,Masking View Name,Port"}
{sub(/:/,": ")}
/Symmetrix/{a=$4}
/Masking View Name/{b=$5}
/Initiator Group Name/{c=$5}
/Port Group Name/{d=$5}
/Storage Group Name/{e=$5}
/Not Visible/{if(! /^ /){dev=$1;}print a","dev","e","d","c","b",01G:0_16G:0_03H:0_14H:0"}' testfilee

# 7  
Old 03-29-2011
Thanks pravin ,the second case i mentioned is an example , there are more records like that ..with diff fields , so can't hardcode the port field .There is only one Symmetrix Id field ...the very first one
thx
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split a big file into multiple files using awk

this thread is a continuation from previous thread https://www.unix.com/shell-programming-and-scripting/223901-split-big-file-into-multiple-files-based-first-four-characters.html ..I am using awk to split file and I have a syntax error while executing the below code I am using AIX 7.2... (4 Replies)
Discussion started by: etldev
4 Replies

2. Shell Programming and Scripting

Using awk to Parse File

Hi all, I have a file that contains a good hundred of these job definitions below: Job Name Last Start Last End ST Run Pri/Xit ________________________________________________________________ ____________________... (7 Replies)
Discussion started by: atticuss
7 Replies

3. Shell Programming and Scripting

Parse a file using awk

Hi Experts, I am trying to parse the following file; FILEA a|b|c|c|c|c a|b|d|d|d|d e|f|a|a|a|a e|f|b|b|b|boutput expected: a<TAB>b <TAB><TAB>c<TAB>c<TAB>c<TAB>c<TAB> <TAB><TAB>d<TAB>d<TAB>d<TAB>d<TAB> e<TAB>f <TAB><TAB>a<TAB>a<TAB>a<TAB>a<TAB> <TAB><TAB>b<TAB>b<TAB>b<TAB>b<TAB>*... (7 Replies)
Discussion started by: rajangupta2387
7 Replies

4. Shell Programming and Scripting

Parse a file with awk?

Hi guys (and gals). I need some help. I'm running an IVR purely on Asterisk where I capture the DTMFs. After pulsing each DTMF I have Asterisk write to a file with whatever was dialed (mostly used for record-keeping) and at the end of the survey I write all variables in a single line to a... (2 Replies)
Discussion started by: tulf210
2 Replies

5. Shell Programming and Scripting

awk column comparison big file

Hi all, I would like to compare a column in one file to a column in another file and when there is a match it prints the first column and the corresponding second column. Example File1 ABA ABC ABE ABF File 2 ABA 123 ABB 124 ABD 125 ABC 126 So what I would like printed to a... (6 Replies)
Discussion started by: pcg
6 Replies

6. Shell Programming and Scripting

big xml file with nested loop parse

I have an xml file with the structure: <tag1> <value1>xyx</value1> <value2>123</value2> </tag1> <tag1> <value1>568</value1> <value2>zzzzz</value2> </tag1> where I want to parse each data pair in the this single file, so something like: find first tag1 data pair... (1 Reply)
Discussion started by: unclecameron
1 Replies

7. Shell Programming and Scripting

Parse file using awk and work in awk output

hi guys, i want to parse a file using public function, the file contain raw data in the below format i want to get the output like this to load it to Oracle DB MARWA1,BSS:26,1,3,0,0,0,0,0.00,22,22,22.00 MARWA2,BSS:26,1,3,0,0,0,0,0.00,22,22,22.00 this the file raw format: Number of... (6 Replies)
Discussion started by: dagigg
6 Replies

8. Shell Programming and Scripting

Big data file - sed/grep/awk?

Morning guys. Another day another question. :rolleyes: I am knocking up a script to pull some data from a file. The problem is the file is very big (up to 1 gig in size), so this solution: for results in `grep "^\ ... works, but takes ages (we're talking minutes) to run. The data is held... (8 Replies)
Discussion started by: dlam
8 Replies

9. Shell Programming and Scripting

filter parts of a big file using awk or sed script

I need an assistance in file generation using awk, sed or anything... I have a big file that i need to filter desired parts only. The objective is to select (and print) the report # having the string "apple" on 2 consecutive lines in every report. Please note that the "apple" line has a HEX... (1 Reply)
Discussion started by: apalex
1 Replies

10. UNIX for Dummies Questions & Answers

How to view a big file(143M big)

1 . Thanks everyone who read the post first. 2 . I have a log file which size is 143M , I can not use vi open it .I can not use xedit open it too. How to view it ? If I want to view 200-300 ,how can I implement it 3 . Thanks (3 Replies)
Discussion started by: chenhao_no1
3 Replies
Login or Register to Ask a Question