Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

AWK- extracting values from columns, saving them and gettins statistics

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting AWK- extracting values from columns, saving them and gettins statistics
# 1  
Old 04-12-2011
AWK- extracting values from columns, saving them and gettins statistics
Hello,

I am obviously quite new to unix and awk. I need to parse certain columns of a file (delimited by spaces), and somehow save the value of this column somewhere, together with the value of the column just after it (by pairs; so something like [$i, $i +1]).

I'm then supposed to count the times that the same pairs appear, if the second column's value increments, etc.

Problem is I don't know the syntax of awk and I'm having quite some trouble doing it.

Thanks!
# 2  
Old 04-12-2011
can you post some example data, expected output and what you have tried so far - should help us understand where you're having trouble and assist better.

:-)
# 3  
Old 04-13-2011
The data I'd be treating is the output of another program (written in C). This program essentially treats a log file and returns several fields from packet traces (specifically some of the fields from the MPEG Headers) and arranges them into columns, separated by spaces:

 Seconds Length IP@ (Blank column) MPEGField1-1 MPEGField2-1 MPEGField1-2 MPEGField2-2 ... MPEGField1-7 MPEGField2-7

The idea is to be able to treat the information contained in all the MPEG fields, by pairs (so MPEGField1-1 with MPEGField 2-1, so on and so forth until MPEGField1-7, MPEGField2-7).

As for the code I've been using, well, it just hasn't been real effective because I'm not able to extract the columns I want and even if I did, I'm not sure as to how to store the info for later treatment (statistics about the number of different channels present in the first field and their continuity).


Code:
./rt < $2 | awk ' 
            BEGIN {pidval=0; ccval=0;}
                       {
                 for (i=4;i<=16;i=i+2)
                            {
                             if(pidarray[$i]!=pidval)
                                {
                                 pidval= pidarray[$i];
                                 ccval= pidarray[$(i+1)];
                                 }
                             }
                       }
                 END {for (i in pidarray) {print pidval,ccval;}}'




Last edited by acsg; 04-13-2011 at 03:20 AM..
# 4  
Old 04-13-2011
Hi acsg,

Quote:
Originally Posted by acsg
Seconds Milliseconds Length IP@ (Blank column) MPEGField1-1 MPEGField2-1 MPEGField1-2 MPEGField2-2 ... MPEGField1-7 MPEGField2-7
Are these headers?

May you show some lines of inputfile and desired outputfile.

Regards
# 5  
Old 04-13-2011
Yes those are the headers and the ones I want to extract are all the ones that are underlined (MPEG).

Example input (first and second rows of the file):

130184 1316 efc06923 160 13 160 13 160 13 160 13 160 13 4608 13 160 13
130185 1316 efc06923 160 13 160 13 160 13 160 13 160 13 160 13 160 13

Desired output:

160 13
160 13
160 13
160 13
160 13
4608 13
160 13
160 13
160 13
160 13
160 13
160 13
160 13
160 13

Thanks for the help!
# 6  
Old 04-13-2011
acsg,

Try with:
Code:
awk '{for(i=4;i<=NF;i=i+2) print $i,$(i+1)}' inputfile


Hope this helps.

Regards
This User Gave Thanks to cgkmal For This Post:
acsg
# 7  
Old 04-13-2011
Thanks cgkmal! It works Smilie

Now, just to be sure, the outputs could be treated separately by another script, right? I printed them out to a file in a spreadsheet, just to see what they'd look like, and they're both printed in the same column (as opposed to my initial file in which all the fields were in different columns). So I'd just like to know if I could treat them both as separate fields.
Last edited by acsg; 04-13-2011 at 04:13 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Fixed length file extracting values in columns

How do I extract values in a few columns in a row of a fixed length file? If there are 8 columns and I need to extract values of 2nd,4th and 6 th columns, how do i do that? I used cut command, this I used only for one column. How do I do it more than one column? The below command will give... (1 Reply)
Discussion started by: princetd001
1 Replies

2. UNIX for Dummies Questions & Answers

Extracting rows from a text file based on the values of two columns (given ranges)

Hi, I have a tab delimited text file with multiple columns. The second and third columns include numbers that have not been sorted. I want to extract rows where the second column includes a value between -0.01 and 0.01 (including both numbers) and the first third column includes a value between... (1 Reply)
Discussion started by: evelibertine
1 Replies

3. Shell Programming and Scripting

Extracting multiple columns with awk

Hi everyone!! I need to apply a simple command to extract columns from a matrix, but I need to extract contemporary from the first to the tenth columns, than from the eleventh to the twentyth and so on... how can i do that? (1 Reply)
Discussion started by: gabrysfe
1 Replies

4. Shell Programming and Scripting

Extracting columns from multiple files with awk

hi everyone! I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next} {print a, $2}' file1 file2 I added the file3, file4 and... (10 Replies)
Discussion started by: orcaja
10 Replies

5. UNIX for Dummies Questions & Answers

Extracting columns from multiple files with awk

hi everyone! I already posted it in scripts, I'm sorry, it's doubled I'd like to extract a single column from 5 different files and put them together in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is: awk 'NR==FNR{a=$2; next}... (1 Reply)
Discussion started by: orcaja
1 Replies

6. Shell Programming and Scripting

extracting columns with awk

Friends, I have a file with fileds in the following order sda 4.80 114.12 128.69 978424 1103384 sdb 0.03 0.40 0.00 3431 0 sda 1.00 0.00 88.00 0 176 sdb ... (14 Replies)
Discussion started by: achak01
14 Replies

7. Shell Programming and Scripting

saving values in file in an array in awk

hi i am trying to save values in a file in an array in awk..the file is as follows: 0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0, so far i have this: awk 'BEGIN {RS="\n";FS=","} { for(i=1;i<=NR;i++) { for(j=1;j<=NF;j++) { a=$j; } } (4 Replies)
Discussion started by: npatwardhan
4 Replies

8. Shell Programming and Scripting

extracting multiple consecutive columns using awk

Hello, I have a matrix 200*10,000 and I need to extract the columns between 40 and 77. I dont want to write in awk all the columns. eg: awk '{print $40, $41, $42,$43 ... $77}'. I think should exist a better way to do this. (10 Replies)
Discussion started by: auratus42
10 Replies

9. Shell Programming and Scripting

saving values from awk expression into shell array

hi i am trying to save the values i extract from a file with the help of awk in a bash shell array. i have: exec 10<file2 while read LINE <&10; do ARRAY1=$(awk '{print $1}' file2) ((count++)) done echo ${ARRAY1} it prints just blank lines. file1 has two columns and i... (4 Replies)
Discussion started by: npatwardhan
4 Replies

10. UNIX for Dummies Questions & Answers

Extracting lines and saving - awk

Hi All, I am trying to extract lines bsed on pattern matching../mp straight-flow/ Extracted output should be saved in meta_string , but the code is not working in that manner,saving repeated lines. can anyone please suggest where am i going wrong. /mp straight-flow/ {... (6 Replies)
Discussion started by: madhaviece
6 Replies
Login or Register to Ask a Question