Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Using sprintf and system command in awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using sprintf and system command in awk
# 1  
Old 03-29-2016
Using sprintf and system command in awk
Hello Friends,

I'm trying something hard (for me) to create a report script,normally the following script works:

Echos are just for cosmetic touch,
Code:
echo -n "\n-----\t----------\t-------------\t\t--------------\t\t--------\n
COUNT\tEVENT_TYPE\tRESPONSE_CODE\t\tINTERNAL_ERROR\t\tFLOWNAME
\n-----\t----------\t-------------\t\t--------------\t\t--------------------------------\n" 
&& gzcat *2016-03-29_*gz | gawk -F"|" '
BEGIN{format = "%-10s\t%-8s\t%-14s\t%-32s\n"}{if(( $5 == 2001 )&&( $8 != 0 )&&( $9 != 0 )&&( $12 ~ "fulfillService" )) 
printf format, "\t"$5"\t",$8"\t\t",$9"\t",$(NF-1)}' | sort | uniq -c 
&& echo -n "-----\t----------\t-------------\t\t--------------\t\t--------------------------------\n"

Output:

Code:
-----   ----------      -------------           --------------          --------
COUNT   EVENT_TYPE      RESPONSE_CODE           INTERNAL_ERROR          FLOWNAME
-----   ----------      -------------           --------------          --------------------------------
4506    2001            -3                      114                     refillVoucherless               
11472   2001            -3                      115                     refillVoucherless               
  12    2001            -4                      11111                   addAndActivateService           
   4    2001            -4                      13                      setHLRSub                       
-----   ----------      -------------           --------------          --------------------------------

I need to add up extra field to output as "OFFER NAME"
this output will come from 14th field of input files (pipe seperated) after stripping them by using SED and when I run the following command it works; it fetches what I want:

Code:
nawk -F\| '{if(( $5 == 2001 ) && ( $8 != 0 ) && ( $9 != 0 ) && ( $12 ~ "fulfillService" ) print $14}' infile 
| sed 's#.*\(externalofferid":"\)\(.*\)\(","receiversubscriber\).*#\2#'

gives something like:
Code:
ABC_OFFER

Desired Outout is :

Code:
-----   ----------	----------      -------------           --------------          --------
COUNT   OFFER_NAME 	EVENT_TYPE      RESPONSE_CODE           INTERNAL_ERROR          FLOWNAME
-----   ----------	----------      -------------           --------------          ---------------------
4506    AAA			2001            -3                      114                     refillVoucherless               
11472   BBB			2001            -3                      115                     refillVoucherless               
  12    CCC			2001            -4                      11111                   addAndActivateService           
   4    			2001            -4                      13                      setHLRSub                       
-----   ----------	----------      -------------           --------------          -----------------------

However when I want to add it to the whole command above it fails
I though I had to use a system command (but i'm not sure)
could you please suggest/correct the command?

I tried (did not show echoes part; its already confusing):
Code:
gzcat *2016-03-29_*gz | gawk -F"|" '
BEGIN{format = "%-20s\t%-10s\t%-8s\t%-14s\t%-32s\n"}{if(( $5 == 2001 )&&( $8 != 0 )&&( $9 != 0 )&&( $12 ~ "fulfillService" )) 
sprintf offer==system("sed 's@.*\(externalofferid":"\)\(.*\)\(","receiversubscriber\).*@\2@'"$14) ; 
printf format, "\t"offer"\t"$5"\t",$8"\t\t",$9"\t",$(NF-1)}' | sort | uniq -c

I believe I don't need sprintf as it prints the stripped string..

error:
Code:
gawk:      
 ^ syntax error

Thanks in advance.
Last edited by EAGL€; 03-29-2016 at 10:58 AM..
# 2  
Old 03-29-2016
Not clear. You want to print WHAT as the second output field? Where does it come from? Field 14 of the input stream? Any conditions to apply?
# 3  
Old 03-29-2016
Quote:
Originally Posted by RudiC
Not clear. You want to print WHAT as the second output field? Where does it come from? Field 14 of the input stream? Any conditions to apply?
I want to print Offernames like AAA, BOB.. Which I can strip from 14th field of gunzipped input files by using the SED command I have displayed above
14th field is actually complicated json field
I used see to strip what I want I know there could be awk scripts to strip Jason inputs..Briefly I need to add a field as an alphanumeric string which I can strip with sed
shown
# 4  
Old 03-29-2016
Why don't you post a line or two of the original (unzipped!) input file?
# 5  
Old 03-29-2016
It was long input and was on my phone not computer thats why didnt have input, thanks for asking for it Rudic..
Code:
36087|20160309115933427|20160309115933481|54|2001|||-3|114|Buiness exception|u:so1-750564413014575139599421000:ffe33-7505644130-14575139734171001-0001-01|fulfillService|ID-SSCM-ESB1-56218-1455665881255-13-319406599|{"interval":{"attributes":{"attribute":[{"name":"INITIAL_CHARGE_OPTION","value":"0"},{"name":"FULFILL_ON_RESERVE","value":"1"}]},"service":{"serviceid":800857,"externalserviceid":"EcRefillService","servicetype":"REFILL","serviceprofile":"REFILL","attributes":{"attribute":[{"name":"REFILL_AMOUNT","value":"600"},{"name":"REFILL_ID","value":"21"},{"name":"DA_UNIT_TYPE","value":"1"},{"name":"UPDATE_SERVICE","value":"0"}]},"grantedservicestartdate":"20160309115920","grantedserviceenddate":"20160309120020","requestedduration":1,"requesteddurationtype":5,"grantedduration":60000,"actualprice":"0.0","ratedprice":"0.0","prorationenabled":false},"subscriptionintervalid":61446288,"enddate":"20160408115920","startdate":"20160309115920","payersubscriber":{"msisdn":"9647505644130","paymentmethod":2,"imsi":"418400205051815","preferredlanguage":2},"campaignid":2000458,"elligibilityoption":1,"status":6,"sequence":1,"requestchannel":"USSD","transactionid":"u:so1-750564413014575139599421000","initialfeeperiodduration":0,"initialfeeperiodtype":0,"chargingperiodduration":1,"chargingperiodtype":6,"chargingperiodnumber":1},"attribute":[{"name":"QUERY_SUBSCRIPTION_HISTORY","value":"false","actiontype":"NEW"},{"name":"FRP_RETRY_REJECT","value":"false","actiontype":"NEW"},{"name":"QUERY_TARIFF_PLAN","value":"false","actiontype":"NEW"},{"name":"INITIAL_CHARGE_OPTION","value":"0","actiontype":"NEW"},{"name":"FULFILL_ON_RESERVE","value":"1","actiontype":"NEW"},{"name":"REFILL_AMOUNT","value":"600","actiontype":"NEW"},{"name":"REFILL_ID","value":"21","actiontype":"NEW"},{"name":"DA_UNIT_TYPE","value":"1","actiontype":"NEW"},{"name":"UPDATE_SERVICE","value":"0","actiontype":"NEW"},{"name":"EXT_DATA_1","value":"EC:Emergency Credit"},{"name":"EXT_DATA_2","value":"Emergency Credit:EC Plan"},{"name":"CURRENCY","value":"IQD"},{"name":"AMOUNT","value":"600000"}],"fulfillservice":{"subscription":{"attributes":{"attribute":[{"name":"QUERY_SUBSCRIPTION_HISTORY","value":"false"},{"name":"FRP_RETRY_REJECT","value":"false"},{"name":"QUERY_TARIFF_PLAN","value":"false"}]},"esdpofferid":"2000457","externalofferid":"EmergencyCreditOffer","receiversubscriber":{"msisdn":"9647505644130","paymentmethod":2,"imsi":"418400205051815","preferredlanguage":2},"startdate":"20160309115920","status":6,"externalsubscriptionid":"64750564413053192904","subscriptionid":60985319},"fulfillment":0,"fulfillmentreason":1,"requesttime":"20160309115933","transactionid":"u:so1-750564413014575139599421000:ffe33-7505644130-14575139734171001-0001-01","previousfulfillmentreason":0},"nodeidentifier":"REFILL","msisdn":"9647505644130","transactionid":"u:so1-750564413014575139599421000:ffe33-7505644130-14575139734171001-0001-01","imsi":"418400205051815","fulfill_reason":"PROV"}||{ "NodeResponse": {"status":"ERROR", "responseCode":"-3", "responseDescription":" Business exception", "internalError":"114","jObject":{} }   }|SscmAirService|refillVoucherless|

I need the red highlighted one and its in 14th field; a json input.. SED part of my command gives it.. but could not connect it to main code,
# 6  
Old 03-29-2016
OK, how about
Code:
awk -F"|" '
BEGIN           {cfmt   = "%-10s"
                 format = "\t%-10s\t%-20s\t%-14s\t%-14s\t%-32s\n"
                 UL     = sprintf (cfmt format, _, _, _, _, _, _)
                 gsub (" ", "-", UL)
                 printf "%s", UL
                 printf cfmt format, "COUNT", "EVENT_TYPE", "OFFER_NAME", "RESPONSE_CODE", "INTERNAL_ERROR", "FLOWNAME"
                 printf "%s", UL
                 cmd    =  "sort | uniq -c"
                }

END             {close (cmd)
                 printf "%s", UL
                }

$12 ~  "fulfillService" &&
$5  == 2001 &&
$8  != 0    &&
$9  != 0        {match ($14, /externalofferid[^,]*/)
                 printf format, $5, substr ($14, RSTART+18, RLENGTH-19), $8, $9, $(NF-1) | cmd
                }
' file
----------      ----------      --------------------    --------------  --------------  --------------------------------
COUNT           EVENT_TYPE      OFFER_NAME              RESPONSE_CODE   INTERNAL_ERROR  FLOWNAME
----------      ----------      --------------------    --------------  --------------  --------------------------------
      6         2001            EmergencyCreditOffer    -3              114             refillVoucherless
      2         2001            EmergencyCreditOffer    -4              114             refillVoucherless
      1         2001            EmergencyCreditOffer    -5              114             refillVoucherless
      2         2001            EmergencyCreditOffer    -6              114             refillVoucherless
----------      ----------      --------------------    --------------  --------------  --------------------------------

This User Gave Thanks to RudiC For This Post:
EAGL€
# 7  
Old 03-29-2016
Thanks a lot Rudic,

When I tested I saw that slightly the columns are shifted as the following (Event Types must be 2001; and all columns should be shifted to right side similarly):

Code:
-bash-4.1$ ./tst.sh 
----------      ----------      --------------------    --------------  --------------  --------------------------------
COUNT           EVENT_TYPE      OFFER_NAME              RESPONSE_CODE   INTERNAL_ERROR  FLOWNAME                        
----------      ----------      --------------------    --------------  --------------  --------------------------------
   4    2001            Blackberry Social       -2              320001          deactivateBBService             
8328    2001            EmergencyCreditOffer    -3              114             refillVoucherless               
21431   2001            EmergencyCreditOffer    -3              115             refillVoucherless               
   5    2001            Internet Daily          -2              320001          setHLRSub                       
  13    2001            Internet Daily          -4              11111           addAndActivateService           
   8    2001            Internet Daily          -4              13              setHLRSub                       
   5    2001            Internet Weekly 500     -2              320001          setHLRSub                       
   7    2001            Internet Weekly 500     -4              11111           addAndActivateService           
   9    2001            Internet Weekly 500     -4              13              setHLRSub                       
   2    2001            WhatAppDaily            -4              13              setHLRSub                       
----------      ----------      --------------------    --------------  --------------  --------------------------------
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Join with awk using sprintf

Hi, Trying to join 2 files with awk (file1 has variable number of fields; file 2 has constant number of fields) file1: hook1|AA|BB|CC|DD hook2|EE|FF file2: hook1|11|22 hook2|33|44 hook3|55|66 output: hook1|11|22|AA|BB|CC|DD hook2|33|44|EE|FF hook3|55|66 What I tried so far:... (3 Replies)
Discussion started by: beca123456
3 Replies

2. Shell Programming and Scripting

Awk: System command not working in awk

Hi, I have around 10 files in a folder in which I want to change the file format from tab(\t) to pipe(|) with some changes in the fields as well. Below is the code, while tmp file is getting generated but move command is not working, please help Following is the code awk -F"\t" '{print... (2 Replies)
Discussion started by: siramitsharma
2 Replies

3. Shell Programming and Scripting

How can I pass arguments to system command in a awk script?

Hi I need your help, please How can I pass arguments to system command in a awk script?... for example: byte=substr(cadena,pos,2); system("grep -n byte mapeo.txt"); Does it exist a way? Thanks for advance. (4 Replies)
Discussion started by: solaris21
4 Replies

4. Shell Programming and Scripting

Invoking system(cmd) inside awk command

Hi, I was searching for a way to grep 2 lines before and after a certain keyword, and I came across the following code.. awk "\$0 ~ /ORA-/ { cmd=\"awk 'NR>=\" NR-2 \" && NR<=\" NR+2 \"' init.ora\" system(cmd) }" input_file I could not understand how this works. What is system() ? what... (2 Replies)
Discussion started by: Kulasekar
2 Replies

5. Shell Programming and Scripting

awk - System command not working

dear All, my awk system command isn't working or rather I'm missing something in my command. Appreciated , if anyone can assist me what exactly I'm missing ?? awk ' /^/ { > c=split($3,a,"/") ;for(n=1; n<=c; ++n) > { > if (system("test -d" /home/cubedata/20120104/"$1"/"a")) { > print... (5 Replies)
Discussion started by: manas_ranjan
5 Replies

6. Shell Programming and Scripting

Problem with awk awk: program limit exceeded: sprintf buffer size=1020

Hi I have many problems with a script. I have a script that formats a text file but always prints the same error when i try to execute it The code is that: { if (NF==17){ print $0 }else{ fields=NF; all=$0; while... (2 Replies)
Discussion started by: fate
2 Replies

7. Shell Programming and Scripting

Hide status value from awk system command

Hi, When i use the system( ) function inside a awk, i am getting the ouput with a 0 appended in a new line. Can someone guide me to eliminate the extra line containing 0. Ex : awk -F"|" '{print system("convert.sh" $1}' The output is displayed with 0 in a new line. ... (8 Replies)
Discussion started by: muruganksk
8 Replies

8. Shell Programming and Scripting

system command within awk

I was trying commands within awk. i got stuck here. i take 2 files. cat input first second third fourth fifth sixth seventh eighth ninht tenthcat add *************** Nirbhay ***************i run the command awk '{ if ( NR == 5 ) { print system("cat add") "\n" $0 } else {... (4 Replies)
Discussion started by: nirbhay
4 Replies

9. Shell Programming and Scripting

awk system() command not working

I am using Sun Solaris 5.8 I am trying to run a system command such as ls and echo inside awk, but when I run the following code system echo is not displayed. bash-2.03$ ls | awk 'BEGIN { print "first print" system("echo system echo") print "second print" ... (1 Reply)
Discussion started by: rakeshou
1 Replies

10. Shell Programming and Scripting

can I pass awk variable to system command?

I wanna use a system function to deal with several data. So I use awk variable FILENAME to transfer the file directory to system command, but it does not work. I use a shell function "out_function" to deal with data and save the result in another directory with the same file name. How can I... (2 Replies)
Discussion started by: zhynxn
2 Replies
Login or Register to Ask a Question