Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help with AWK

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with AWK
# 1  
Old 10-03-2012
Help with AWK
Hello there!

I'm wondering if anyone can help me with this little (big) challenge!

I have a file, with about 30 blocks, each one with the same structure as the following:

Code:
Timestamp: 14:52:50

+++++ STCP DEVICE METERS +++++

netstat -interface #sdlmux.m5.13.1.0

MAC Type   : CSMA/CD
MAC Address: 00:00:a8:43:3f:cd
Device Name: #sdlmux.m5.13.1.0
Line Speed : 100 mb/s
Line Duplex: Full-Duplex

MAC Statistics:
  Received frames                          : 3066396
  Received multicast and broadcast frames  : 102
  Received octets                          : 813041981
  Transmitted frames                       : 3430870
  Transmitted octets                       : 1220638884
  LAN Chipset re-initialized               : 0
  SQE error                                : 0
  Transmit ring full                       : 0
  Transmit frame discarded, late collisions: 0
  Transmit frame was deferred              : 0
  Transmit frame after a single retry      : 0
  Transmit frame after multiple retry      : 0
  Transmit frame discarded, excessive retry: 0
  Receive frame discarded, lack of buffers : 0
  Receive frame discarded, improper framing: 0
  Receive frame discarded, an overflow     : 0
  Receive frame discarded, bad CRC         : 0
  Receive frame discarded, bad address     : 0
  Receive frame discarded, congestion      : 0

MAC Summary:
  Transmitted frames         : 3430870
  Transmitted octets         : 1220638884
  Retransmitted frames       : 0
  Received frames            : 3066498
  Received octets            : 813041981
  Total of lost frames       : 0
ready  18:52:49

***** END OF RECORD *****

I'm wondering if there's a way to use AWK (and SED) to get the values above in red, and put them in a line, separated with comma... like this:

Timestamp , device_name , received_frame , received_multicast , received_octets , transmitted_frames , transmitted_octets , Receive_frame_bad_crc

The timestamp does not appears every time, and when it arrives the BAD_CRC value, it should go for a new line.

This is getting me nuts. I have to get the line for every block in the text file, and I can't use the awk to get the column values, because it's not allways the same.

Anyone can help me?

Thanks!!!
Last edited by psychosb; 10-03-2012 at 05:25 PM.. Reason: Please use code tags
# 2  
Old 10-03-2012
Quote:
Originally Posted by psychosb
This is getting me nuts. I have to get the line for every block in the text file, and I can't use the awk to get the column values, because it's not allways the same.
Have you tried using ":" as the field separator? Then the number is always $2. Assign what you want into variables, and at the end of record print it all..

Code:
mute@clt:~$ ./awk input
14:52:50,sdlmux.m5.13.1.0,3066498,102,813041981,3430870,1220638884,0
mute@clt:~$ cat awk
#!/usr/bin/awk -f

BEGIN {
        FS=":[[:space:]]*"
        OFS=","
}

$1 == "Timestamp" {
  ts=$2 ":" $3 ":" $4
}
/^netstat / {
        split($0,a,/#/)
        intf=a[2]
}
$1 ~ /Received frames/ { rx_frm=$2 }
$1 ~ /Received multicast and broadcast frames/ { rx_bc=$2 }
$1 ~ /Received octets/ { rx_oct=$2 }
$1 ~ /Transmitted frames/ { tx_frm=$2 }
$1 ~ /Transmitted octets/ { tx_oct=$2 }
$1 ~ /bad CRC/ { bcrc=$2 }
/^ready / {
        print ts, intf, rx_frm, rx_bc, rx_oct, tx_frm, tx_oct, bcrc
}

---------- Post updated at 04:31 PM ---------- Previous update was at 04:25 PM ----------

If the interface doesn't always have that #, could use something like:
Code:
/^netstat -interface / {
        sub(/netstat -interface[[:space:]]*/,"")
        intf=$0
}

Last edited by neutronscott; 10-03-2012 at 05:28 PM.. Reason: fixed timestamp
# 3  
Old 10-03-2012
Hey, @neutronscott

Thanks for the reply!
I have tested the script that you sent, and it worked with the example that I've posted here in the forum.

But in the original file, it's not showing the correct info.

Code:
,,,,,,,
,,,,,,,
,,,,,,,0
,0
,0
,0
,0
,0
,0
,0
,0
,0
,0

Do you know what may be wrong?

I'm sending you the original file, if you can help me!

Moderator's Comments:
Mod Comment Link Removed
Last edited by psychosb; 10-03-2012 at 05:56 PM..
# 4  
Old 10-03-2012
pretty sure the forum allows attachments... i cannot download that. it probably has to do with either line endings and/or end of record marker (i used "ready")
# 5  
Old 10-03-2012
Sorry, I have not seen the forum attachment feature!

I'm sending the file here... Thanks!
stcp.12-09-27.zip (21.8 KB) 
# 6  
Old 10-03-2012
Try:
Code:
#!/usr/bin/awk -f
BEGIN {
    FS=":[[:space:]]*"
    OFS=","
}
$1 == "Timestamp" {
  ts=$2 ":" $3 ":" $4
}
/^[[:space:]]*netstat / {
        split($0,a,/#/)
        intf=a[2]
}
gsub(/ *$/,"",$2)&&0
$1 ~ /Received frames/ {rx_frm=$2 }
$1 ~ /Received multicast and broadcast frames/ { rx_bc=$2 }
$1 ~ /Received octets/ { rx_oct=$2 }
$1 ~ /Transmitted frames/ { tx_frm=$2 }
$1 ~ /Transmitted octets/ { tx_oct=$2 }
$1 ~ /bad CRC/ { bcrc=$2 }
/^ready / && rx_frm {
    print ts, intf, rx_frm, rx_bc, rx_oct, tx_frm, tx_oct, bcrc
    intf=rx_frm=rx_bc=rc_oct=tx_frm=tx_oct=bcrc=""
}

# 7  
Old 10-11-2012
It worked! Thanks a lot! One more question before closing the topic. For my knowledge... I've managed to solve the issue with SED + AWK. But I got the results in line, like this:

Code:
145250  #sdlmux.m5.13.1.0  102  813041981  3430870  1220638884  0  145250  #sdlmux.m5.13.1.0  222  333333  44444444  5555555555  888

Is there a way to make AWK to make a new line every 7 words?

So It would be like:

Code:
145250  #sdlmux.m5.13.1.0  102  813041981  3430870  1220638884  0 
  145250  #sdlmux.m5.13.1.0  222  333333  44444444  5555555555  888

Thanks a lot!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk output yields error: awk:can't open job_name (Autosys)

Good evening, Im newbie at unix specially with awk From an scheduler program called Autosys i want to extract some data reading an inputfile that comprises jobs names, then formating the output to columns for example 1. This is the inputfile: $ more MapaRep.txt ds_extra_nikira_usuarios... (18 Replies)
Discussion started by: alexcol
18 Replies

2. Shell Programming and Scripting

Pass awk field to a command line executed within awk

Hi, I am trying to pass awk field to a command line executed within awk (need to convert a timestamp into formatted date). All my attempts failed this far. Here's an example. It works fine with timestamp hard-codded into the command echo "1381653229 something" |awk 'BEGIN{cmd="date -d... (4 Replies)
Discussion started by: tuxer
4 Replies

3. Shell Programming and Scripting

Passing awk variable argument to a script which is being called inside awk

consider the script below sh /opt/hqe/hqapi1-client-5.0.0/bin/hqapi.sh alert list --host=localhost --port=7443 --user=hqadmin --password=hqadmin --secure=true >/tmp/alerts.xml awk -F'' '{for(i=1;i<=NF;i++){ if($i=="Alert id") { if(id!="") if(dt!=""){ cmd="sh someScript.sh... (2 Replies)
Discussion started by: vivek d r
2 Replies

4. Shell Programming and Scripting

HELP with AWK one-liner. Need to employ an If condition inside AWK to check for array variable ?

Hello experts, I'm stuck with this script for three days now. Here's what i need. I need to split a large delimited (,) file into 2 files based on the value present in the last field. Samp: Something.csv bca,adc,asdf,123,12C bca,adc,asdf,123,13C def,adc,asdf,123,12A I need this split... (6 Replies)
Discussion started by: shell_boy23
6 Replies

5. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

6. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

7. Shell Programming and Scripting

Problem with awk awk: program limit exceeded: sprintf buffer size=1020

Hi I have many problems with a script. I have a script that formats a text file but always prints the same error when i try to execute it The code is that: { if (NF==17){ print $0 }else{ fields=NF; all=$0; while... (2 Replies)
Discussion started by: fate
2 Replies

8. Shell Programming and Scripting

awk: assign variable with -v didn't work in awk filter

I want to filter 2nd column = 2 using awk $ cat t 1 2 2 4 $ VAR=2 #variable worked in print $ cat t | awk -v ID=$VAR ' { print ID}' 2 2 # but variable didn't work in awk filter $ cat t | awk -v ID=$VAR '$2~/ID/ { print $0}' (2 Replies)
Discussion started by: honglus
2 Replies

9. Shell Programming and Scripting

scripting/awk help : awk sum output is not comming in regular format. Pls advise.

Hi Experts, I am adding a column of numbers with awk , however not getting correct output: # awk '{sum+=$1} END {print sum}' datafile 2.15291e+06 How can I getthe output like : 2152910 Thank you.. # awk '{sum+=$1} END {print sum}' datafile 2.15079e+06 (3 Replies)
Discussion started by: rveri
3 Replies

10. Shell Programming and Scripting

Awk problem: How to express the single quote(') by using awk print function

Actually I got a list of file end with *.txt I want to use the same command apply to all the *.txt Thus I try to find out the fastest way to write those same command in a script and then want to let them run automatics. For example: I got the file below: file1.txt file2.txt file3.txt... (4 Replies)
Discussion started by: patrick87
4 Replies
Login or Register to Ask a Question