Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Parse a file using awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parse a file using awk
# 1  
Old 03-08-2013
Parse a file using awk
Hi Experts,

I am trying to parse the following file;
FILEA
Code:
a|b|c|c|c|c
a|b|d|d|d|d
e|f|a|a|a|a
e|f|b|b|b|b

output expected:

Code:
a<TAB>b
<TAB><TAB>c<TAB>c<TAB>c<TAB>c<TAB>
<TAB><TAB>d<TAB>d<TAB>d<TAB>d<TAB>
e<TAB>f
<TAB><TAB>a<TAB>a<TAB>a<TAB>a<TAB>
<TAB><TAB>b<TAB>b<TAB>b<TAB>b<TAB>

* where <TAB> is spaces.

I have tried using the this file as
Code:
 
cat FILEA |
  /usr/local/bin/gawk -F'|' '{for(i=1;i<NF;i++)
  {
   n=split($i,e,/[|]/)
   exists[e[1]]
   val[e[1],e[2],e[3],e[4],e[5]]
     }}
  END{
  no_of_names=asorti(exists)
  for(i=1;i<=no_of_names;i++)
  {
   for(j in val)
   {
    split(j,temp,SUBSEP)
    if(temp[1]==exists[i])
    print  k[1],temp[j+3],temp[4], val[j] | "sort -nr"
   }
   close("sort -nr")
   printf "\n"
  }
  }' OFS='|'

Please suggest what am i doing wrong
Last edited by vgersh99; 03-08-2013 at 10:40 AM.. Reason: fixed code tags
# 2  
Old 03-08-2013
Code:
sort file | awk -F'|' '
 A[1] == $1 && A[2] == $2 {
        n = split ($0, A, /[|]/)
        printf OFS
        for ( i = 3; i <= n; i++)
                printf OFS A[i];
        printf RS
} A[1] != $1 && A[2] != $2 {
        n = split ($0, A, /[|]/)
        print A[1], A[2]
        printf OFS
        for ( i = 3; i <= n; i++)
                printf OFS A[i];
        printf RS
} {
        n = split ($0, A, /[|]/)
} ' file

This User Gave Thanks to Yoda For This Post:
rajangupta2387
# 3  
Old 03-08-2013
Thanks afor your answer and it work for the situation i mentioned. But in actual the contents of the file are large in size and it fails to execute that.

actual file is something like:

Code:
 
Comptel_6_UAT|EventLinkUserHome|StreamParameterValue|Value|NOT EXIST|/u01/appl/u1elkpe
Comptel_6_UAT|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|Packages|EL_LIB_SFTP|1.1.0.|1.2.1.
Comptel_6_UAT|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|Packages|EL_CFG_FTP_DISTRIBUTOR|2.1.0.|2.1.0.upc2
Comptel_6_UAT|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|Packages|EL_APP_FTP_DISTRIBUTOR|2.1.4.|2.1.5.
Comptel_6_UAT|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|TargetDirectory|/u01/appl/u1elkpe/delivery/UPC_PE_Rejected_Loader_stream/INPUT/RO|${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/RO
Comptel_6_UAT|RO_OPEN_AUDIT_REJECTED_ENCODER|Packages|EL_LIB_CML|1.5.4.|1.5.5.
Comptel_6_UAT|RO_OPEN_BLN|Packages|UPC_LIB_PERL_UTILS|1.10.1.|3.2.1.
Comptel_6_UAT|RO_OPEN_BLN|Packages|UPC_CFG_RO_OPENSER_BLN|1.4.0.|1.6.2.
Comptel_6_UAT|RO_OPEN_BLN|NodeParameters|StreamName|${StreamName}|NOT EXIST
Comptel_6_UAT|RO_OPEN_BLN|NodeParameters|LsAdmin|/u01/appl/u1elkpe/EventLink/lookup_server3/bin/ls_admin|NOT EXIST
Comptel_6_UAT|RO_OPEN_BLN|NodeParameters|Lookup.ShmidDirectory|/u01/appl/u1elkpe/EventLink/lookup_server3/tmp|${EventLinkUserHome}/EventLink/lookup_server3/tmp
Comptel_6_UAT|RO_OPEN_BLN|NodeParameters|LD_PRELOAD|/usr/lib/64/libCstd.so.1|NOT EXIST
Comptel_6_UAT|RO_OPEN_BRIO_DISTRIBUTOR|NodeParameters|TargetDirectory|/u01/appl/u1elkpe/delivery/UPC_RO/OPENSER/OUTPUT/BRIO|NOT EXIST
Comptel_6_UAT|RO_OPEN_BRIO_ENCODER|Packages|UPC_CFG_RO_OPENSER_BRIO_ENCODER|1.1.1.|NOT EXIST
Comptel_6_UAT|RO_OPEN_DR_DISTRIBUTOR|NodeParameters|Username| |u1elkpe
Comptel_6_UAT|RO_OPEN_DR_DISTRIBUTOR|NodeParameters|TargetDirectory|/u01/appl/u1elkpe/delivery/UPC_RO/OPENSER/OUTPUT/DR|${EventLinkUserHome}/delivery/UPC_RO/OPENSER/OUTPUT/DR
Comptel_6_UAT|RO_OPEN_DR_DISTRIBUTOR|NodeParameters|Password| |u1elkpe
Comptel_6_UAT|RO_OPEN_DWH_DISTRIBUTOR|NodeParameters|Username|NOT EXIST|u1elkpe
Comptel_6_UAT|RO_OPEN_FILE_DUPLICATE_CHECKER|NodeParameters|SequenceMax|999999|999999999
Comptel_6_UAT|RO_OPEN_FILE_DUPLICATE_CHECKER|NodeParameters|NumberPosition|38|27
Comptel_6_UAT|RO_OPEN_FILE_DUPLICATE_CHECKER|NodeParameters|NumberLength|6|10
Comptel_6_UAT|RO_OPEN_FTP_COLLECTOR|Packages|EL_APP_FTP_COLLECTOR|2.1.4.upc1|2.1.6.upc1
Comptel_6_UAT|RO_OPEN_FTP_COLLECTOR|NodeParameters|SourceDirectory|/u01/appl/u1elkpe/delivery/UPC_RO/OPENSER/INPUT|${EventLinkUserHome}/delivery/UPCRO_OPENSER/INPUT
Comptel_6_UAT|RO_OPEN_KENAN_DISTRIBUTOR|NodeParameters|Username| |u1elkpe
Comptel_6_UAT|RO_OPEN_KENAN_DISTRIBUTOR|NodeParameters|TargetDirectory|/u01/appl/u1elkpe/delivery/UPC_RO/OPENSER/OUTPUT/KENAN|${EventLinkUserHome}/delivery/UPC_RO/OPENSER/OUTPUT/KENAN
Comptel_6_UAT|RO_OPEN_KENAN_ENCODER|Packages|UPC_CFG_RO_KENAN_ENCODER|1.2.1.|1.3.0.
Comptel_6_UAT|RO_OPEN_KENAN_RDC|Packages|UPC_CFG_RECORD_CHECKER|7.1.6.|NOT EXIST
Comptel_6_UAT|RO_OPEN_KENAN_RDC|NodeParameters|TimestampField|EL_CALL_A_TIME|NOT EXIST
Comptel_6_UAT|RO_OPEN_KENAN_RDC|NodeParameters|Keys|EL_TERMINATING_NUMBER,EL_ORIGINATING_NUMBER,I_ORIG_TRUNK_NUM,I_TERM_TRUNK_NUM,EL_SWITCH_ID|NOT EXIST
Comptel_6_UAT|RO_OPEN_KENAN_RDC|NodeParameters|CDR_DURATION|EL_DURATION|NOT EXIST
Comptel_6_UAT|RO_OPEN_KENAN_RDC|NodeParameters|B_NUMBER|EL_TERMINATING_NUMBER|NOT EXIST
Comptel_6_UAT|RO_OPEN_KENAN_RDC|NodeParameters|A_NUMBER|EL_ORIGINATING_NUMBER|NOT EXIST
Comptel_6_UAT|RO_OPEN_KENAN_RDC|DataStorages|UPC_CFG_RECORD_CHECKER|DUPLICATE_RECORDS|NOT EXIST
Comptel_6_UAT|RO_OPEN_OUTPUT_BACKUP|NodeParameters|TargetDirectory|/u01/appl/u1elkpe/delivery/UPC_RO/OPENSER/BACKUP/|NOT EXIST
Comptel_6_UAT|RO_OPEN_OUTPUT_BACKUP|NodeParameters|InputLinkDirectories|Yes|NOT EXIST
Comptel_6_UAT|RO_OPEN_OUTPUT_BACKUP|Links|ENCODED_AUDIT|1|NOT EXIST
Comptel_6_UAT|RO_OPEN_SWITCH_BACKUP|NodeParameters|TargetDirectory|/u01/appl/u1elkpe/delivery/UPC_RO/OPENSER/BACKUP/RAW|NOT EXIST
Comptel_6_UAT|RO_OPEN_TAIFUN_DISTRIBUTOR|NodeParameters|Username|NOT EXIST|u1elkpe
Comptel_6_UAT|RO_OPEN_TAIFUN_DISTRIBUTOR|NodeParameters|Password|NOT EXIST|u1elkpe
Comptel_6_DEV2|AuditOutType|StreamParameterValue|Value|OPENSER_AUDIT|RO_AUDIT
Comptel_6_DEV2|EventLinkUserHome|StreamParameterValue|Value|NOT EXIST|/u01/appl/d2elkpe
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|Packages|EL_LIB_SFTP|NOT EXIST|1.2.1.
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|Packages|EL_CFG_FTP_DISTRIBUTOR|NOT EXIST|2.1.0.upc2
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|Packages|EL_APP_FTP_DISTRIBUTOR|NOT EXIST|2.1.5.
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|Username|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|TransferType|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|TransferRetryInterval|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|TransferRetryCount|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|TransferProtocol|NOT EXIST|Local
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|TransferMode|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|TempSuffix|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|TempDirectory|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|TargetDirectory|NOT EXIST|${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/RO
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|Ruleset|NOT EXIST|default_ruleset.pm
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|RemoteSuffix|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|RemotePrefix|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|RemoteOS|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|PublicKey|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|PrivateKey|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|Port|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|Password|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|MaximumSustainedNetworkDowntime|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|MaximumReconnectionInterval|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|Kex|NOT EXIST|
Comptel_6_DEV2|RO_OPEN_AUDIT_REJECTED_DISTRIBUTOR|NodeParameters|ExternalCommand|NOT EXIST|

Last edited by vgersh99; 03-08-2013 at 10:41 AM.. Reason: fixed code tags - use [code] instead of [icode]
# 4  
Old 03-08-2013
Here is the modified code:
Code:
sort file | awk -F'|' '
  (P[1] == $1) && (P[2] == $2) {
        n = split ($0, A, /[|]/)
        printf OFS
        for ( i = 3; i <= n; i++)
                printf OFS A[i];
        printf RS
        next;
} (P[1] != $1) || (P[2] != $2) {
        n = split ($0, A, /[|]/)
        print A[1], A[2]
        printf OFS
        for ( i = 3; i <= n; i++)
                printf OFS A[i];
        printf RS
} {
        n = split ($0, P, /[|]/)
} '

This User Gave Thanks to Yoda For This Post:
rajangupta2387
# 5  
Old 03-08-2013
Hi,

Thanks it is working but still i am now facing issues w.r.t to "printf" statement.

Input:

Code:
Comptel_6_UAT|IXBS_ENCODER|Packages|UPC_CFG_NL_DMS100_IXBS_ENCODER|NOT EXIST|2.1.1.
Comptel_6_UAT|IXBS_ENCODER|Packages|EL_LIB_FNL|NOT EXIST|1.5.1.
Comptel_6_UAT|IXBS_ENCODER|Packages|EL_LIB_CML|NOT EXIST|1.5.4.
Comptel_6_UAT|IXBS_ENCODER|Packages|EL_APP_MULTI_FORMAT_ENCODER|NOT EXIST|1.5.9.
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|UniqueOutputFileNames|NOT EXIST|1
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|SeparationCondition|NOT EXIST|EL_IXBS_SEPARATION
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|SeparateSourceID|NOT EXIST|0
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|SeparateSequenceNumbersInOutput|NOT EXIST|1
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|SeparateSequenceNumbersInFileNames|NOT EXIST|0
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|ResetInterval|NOT EXIST|0
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|RemoveFileTemplates|NOT EXIST|1
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|RemoveFileTemplatesTimeout|NOT EXIST|
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|RecordWaitingTimeLimit|NOT EXIST|
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|RecordLimit|NOT EXIST|
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|RealTimeDelivery|NOT EXIST|0
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|OutputFileNameCreationMask|NOT EXIST|IXBS&CREATIONTIME(%Y%m%d%H%M%S)&FIRST(EL_OUT_FILE_SEQ)&FSIZE(9)&COUNT(*,6).CDR.&SOURCEID
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|OutputDescriptionFileName|NOT EXIST|UPC_NL_DMS100_IXBS_Description.xml
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|MinAgeOfFile|NOT EXIST|
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|MaxAgeOfFile|NOT EXIST|
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|InputFileNameExtractionMask|NOT EXIST|([a-zA-Z0-9]{7}_)(AMAUTH_0000)([0-9]{2})(_)([0-9]{14})
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|FileSizeLimit|NOT EXIST|
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|CreateEmptyOutputFiles|NOT EXIST|0
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|CreateEmptyOutputFileOnSeparation|NOT EXIST|
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|CreateEmptyOutputFileOnFlush|NOT EXIST|
Comptel_6_UAT|IXBS_ENCODER|NodeParameters|CloseFilesAtSchedule|NOT EXIST|1
Comptel_6_UAT|IXBS_ENCODER|Links|IXBS_ENCODER_OUT|NOT EXIST|1


I checked that printf is considering it to be operators. Do we have a solution for this as well?
Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!
Last edited by vgersh99; 03-08-2013 at 11:29 AM.. Reason: code tags, please
# 6  
Old 03-08-2013
Quote:
Originally Posted by rajangupta2387
Hi,
I checked that printf is considering it to be operators. Do we have a solution for this as well?
Yes, modify the printf statement, put format specifier:
Code:
printf "%s", OFS A[i];

This User Gave Thanks to Yoda For This Post:
rajangupta2387
# 7  
Old 03-08-2013
Thank you very much it worked.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to parse this file using awk and output in CSV format?

My source file looks like this: Cust-Number = "101" Cust-Name="Joe" Cust-Town="London" Cust-hobby="tennis" Cust-purchase="200" Cust-Number = "102" Cust-Name="Mary" Cust-Town="Newyork" Cust-hobby="reading" Cust-purchase="125" Now I want to parse this file (leaving out hobby) and... (10 Replies)
Discussion started by: Balav
10 Replies

2. Shell Programming and Scripting

awk parse result that match data from file

i run command that return this result,example : gigabitethernet2/2/4:NotPresent, gigabitethernet2/1/17:UP, gigabitethernet2/1/10:UP, gigabitethernet2/1/5:UP, gigabitethernet2/1/9:UP, gigabitethernet2/1/36:DOWN, gigabitethernet2/1/33:DOWN, gigabitethernet2/1/8:UP,... (19 Replies)
Discussion started by: wanttolearn1
19 Replies

3. Shell Programming and Scripting

awk or perl to parse file

I have an input file attached that I am trying to parse in tab-delimanted format: The chromosomal variant column contains all the information: parse rules: 1. 4 zeros after the NC_ and the digits before the . 2. digits after the g. repeated twice separated by a tab 3. letter before the > 4.... (10 Replies)
Discussion started by: cmccabe
10 Replies

4. Shell Programming and Scripting

awk to parse html file

Is it possible in awk to parse a webpage (EDAR Gene Sequencing - Genetic Testing Company | The DNA Diagnostic Experts | GeneDx), the source code is attached. <title> EDAR Gene Sequencing <dt>Test Code:</dt> <dd>156 </dd> <dt>Turnaround Time:</dt> <dd>6-8 weeks </dd> ... (4 Replies)
Discussion started by: cmccabe
4 Replies

5. Shell Programming and Scripting

Using awk to Parse File

Hi all, I have a file that contains a good hundred of these job definitions below: Job Name Last Start Last End ST Run Pri/Xit ________________________________________________________________ ____________________... (7 Replies)
Discussion started by: atticuss
7 Replies

6. Shell Programming and Scripting

AWK script to parse a data in a file

Hi Unix gurus.. I have a file which has below data, It has several MQ Queue statistics; QueueName= 'TEST1' CreateDate= '2009-10-30' CreateTime= '13.45.40' QueueType= Predefined QueueDefinitionType= Local QMinDepth= 0 QMaxDepth= 0 QueueName= 'TEST2' CreateDate= '2009-10-30'... (6 Replies)
Discussion started by: dd_psg
6 Replies

7. Shell Programming and Scripting

AWK Command parse a file based on string.

AWK Command parse a file based on string. I am trying to write a shell script to parse a file based on a string and move the content of the file to another file. Here is scenario. File content below Mime-Version: 1.0 Content-Type: multipart/mixed; ... (2 Replies)
Discussion started by: aakishore
2 Replies

8. Shell Programming and Scripting

Parse a file with awk?

Hi guys (and gals). I need some help. I'm running an IVR purely on Asterisk where I capture the DTMFs. After pulsing each DTMF I have Asterisk write to a file with whatever was dialed (mostly used for record-keeping) and at the end of the survey I write all variables in a single line to a... (2 Replies)
Discussion started by: tulf210
2 Replies

9. Shell Programming and Scripting

AWK - Parse a big file

INPUT SAMPLE Symmetrix ID : 000192601507 Masking View Name : TS00P22_13E_1 Last updated at : 05:10:18 AM on Tue Mar 22,2011 Initiator Group Name : 10000000c960b9cd Host Initiators { WWN : 10000000c960b9cd } Port Group Name :... (8 Replies)
Discussion started by: greycells
8 Replies

10. Shell Programming and Scripting

Parse file using awk and work in awk output

hi guys, i want to parse a file using public function, the file contain raw data in the below format i want to get the output like this to load it to Oracle DB MARWA1,BSS:26,1,3,0,0,0,0,0.00,22,22,22.00 MARWA2,BSS:26,1,3,0,0,0,0,0.00,22,22,22.00 this the file raw format: Number of... (6 Replies)
Discussion started by: dagigg
6 Replies
Login or Register to Ask a Question