Complex transpose awk script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Complex transpose awk script
# 22  
Old 01-30-2013
The comparison loop is the problem, you are right. We need to switch from testing field != fldtobe to testing !(field == fldstobe[n]) plus additional field operations. This might point you in the right direction, but it is untested, so take it as a starting point:
Code:
awk 'BEGIN {split("OCTDP TCTDP GPRSTDP",TDPAr);
            print "MSISDN,OCTDP,TCTDP,GPRSTDP,GCSO,MCSO,SSLO,GC2SO,MC2SO,TIF,GPRSSO,OSMSSO,GC3SO,MC3SO,GC4SO,MC4SO,MMSO,STAT"
     }

    NR==FNR     {fldcnt[NR] = split ($0, tmp, ","); for (i=1;i<=fldcnt[NR]; i++) cmpAr[NR,i]=tmp[i]; rowcnt=NR; next}
     
     /^MSISDN/   {getline; rec=$0}
     /^TDPTYPE/  {noflds=0
                  while (1)
                     {getline; if ($0 ~ /^$/) break   
                      OFS="|"; tmp1=$1; $1=""; sub("\|",""); tmpAr[tmp1]=$0; noflds=3 ;
                     }
                  for (i=1; i<=noflds; i++) rec=rec","tmpAr[TDPAr[i]]
                 }
     /^(GCSO|GC3SO|MMSO)/ {
                  getline; OFS=","; $1=$1; rec=rec","$0}
     /^NONE/     {getline; rec=rec ",no values"}
     /END/       {split (rec, tmpAr, ","); rec=tmpAr[1]
                  dif=0
                  for (fld=2; fld <= fldcnt[i]; fld++)                        # compare fieldwise, each field on its own against all its tobe values
                    {flddif = 1                                               # assume it's different until proven wrong
                     for (row=1; row <= rowcnt; row++)                        # here we go: check against individual tobe values
                        if (tmpAr[fld] == cmpAr[row,fld]) {flddif=0; break}   # found one! Equality proven, register Equ. in flddif and break out of loop
                     if (flddif) {tmpAr[fld]=tmpAr[fld]"_I"; dif=1}           # add "_I" to field if no match found, register in dif
                     rec=rec","tmpAr[fld]}                                    # add (modified?) field to output record
                  if (dif) rec=rec",different"
                  print rec; rec=""; delete tmpAr; dif=0}
    ' tobe file

I renamed some of the variables to somewhat more mnemonic form so as not to lose track when reading. Give it a shot and report back!
This User Gave Thanks to RudiC For This Post:
# 23  
Old 01-31-2013
Helllo RudicC,

Many thanks, I've tested and it seems it works just great.

I'll try it with a real file that have millions of lines.

Many thanks again for the patience and effort to help me, very appeciated. Smilie

Best regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Transpose table with awk

I am trying to format the table below to the output input: cand week sub1 sub2 sub3 sub4 joe 1 94.19 70.99 43.93 60.14 joe 2 94.07 51.02 41.07 38.92 joe 3 26.24 30.95 44.56 67.67 joe 4 72.36 60.92 40.78 83.25 joe 5 51 70.01 44.66 82.22... (7 Replies)
Discussion started by: aydj
7 Replies

2. Shell Programming and Scripting

Transpose rows to columns complex

Input: IN,A,1 IN,B,3 IN,B,2 IN,C,7 BR,A,1 BR,A,5 BR,C,9 AR,C,9 Output: CNTRY,A,B,C IN,1,5,7 BR,6,0,9 AR,0,0,9 (7 Replies)
Discussion started by: unme
7 Replies

3. Shell Programming and Scripting

Transpose using awk

Hi Friends, Very urgent requirement please do needful ASAP.. Input: |1||1|1||1|3||3|2||2|4||4|2||2|3||3|NA||0|5||5|NA||0|4||4|3||3 output: |1||1 |1||1 |3||3 |2||2 |4||4 |2||2 |3||3 |NA||0 |5||5 (4 Replies)
Discussion started by: bharat1211
4 Replies

4. Shell Programming and Scripting

Transpose data as rows using awk

Hi I have below requirement, need help One file contains the meta data information and other file would have the data, match the column from file1 and with file2 and extract corresponding column value and display in another file File1: CUSTTYPECD COSTCENTER FNAME LNAME SERVICELVL ... (1 Reply)
Discussion started by: ravlapo
1 Replies

5. Shell Programming and Scripting

awk to transpose every 7 rows into columns

input: a1 a2 a3 a4 a5 a6 a7 b1 b2 b3 .. b7 .. z1 .. z7 (12 Replies)
Discussion started by: ux4me
12 Replies

6. Shell Programming and Scripting

Complex match of numbers between 2 files awk script

Hello to all, I hope some awk guru could help me. I have 2 input files: File1: Is the complete database File2: Contains some numbers which I want to compare File1: "NUMBERKEY","SERVICENAME","PARAMETERNAME","PARAMETERVALUE","ALTERNATENUMBERKEY"... (9 Replies)
Discussion started by: Ophiuchus
9 Replies

7. Shell Programming and Scripting

awk script (complex)

picked this up from another thread. echo 1st_file.csv; nawk -F, 'NR==FNR{a++;next} a{b++} END{for(i in b){if(b-1&&a!=b){print i";\t\t"b}else{print "NEW:"i";\t\t"b} } }' OFS=, 1st_file.csv *.csv | sort -r i need to use the above but with a slight modification.. 1.compare against 3 month... (25 Replies)
Discussion started by: slashbash
25 Replies

8. Shell Programming and Scripting

How can i transpose this rerult by using awk?

From>>> ATOM 1 ca 2 o 3 h 4 h 5 o dE/dx 0.2057422D-01 0.2463722D-01-0.1068047D-01-0.1495280D-01-0.3725362D-02 dE/dy -0.7179106D-02-0.1554542D-01 0.1016889D-01 0.3268502D-02-0.4888578D-01 dE/dz -0.5600872D-02 0.3110649D-01-0.4088230D-02-0.2295107D-01-0.2832048D-01 ATOM 6 h 7 h 8 o 9 h 10 h... (1 Reply)
Discussion started by: wanchem
1 Replies

9. Shell Programming and Scripting

Transpose using awk

I have a requirement to transpose the below xml which is in a text file on unix: <?xml version="1.0" ?> <REQUEST> <ID>XXX</ID> <TIMESTAMP>20090720062610</TIMESTAMP> <FLAG>Y</FLAG> <TO_FLAG>Y</TO_FLAG> </REQUEST> to <?xml version="1.0"... (13 Replies)
Discussion started by: new_ds_man
13 Replies

10. Shell Programming and Scripting

How to transpose data elements in awk

Hi, I have an input data file :- Test4599,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,2,2,Rain Test90,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,1,0,Not Rain etc.... I wanted to transpose these data to:-... (2 Replies)
Discussion started by: ahjiefreak
2 Replies
Login or Register to Ask a Question