Sponsored Content
Top Forums Shell Programming and Scripting AWK - printing certain fields when field order changes in data file Post 302185695 by eric4 on Tuesday 15th of April 2008 02:32:11 PM
Old 04-15-2008
Question AWK - printing certain fields when field order changes in data file

I'm hoping someone can help me on this. I have a data file that greatly simplified might look like this:

Code:
sec;src;dst;proto
421;10.10.10.1;10.10.10.2;tcp
426;10.10.10.3;10.10.10.4;udp
442;10.10.10.5;10.10.10.6;tcp
sec;src;fac;dst;proto
521;10.10.10.1;ab;10.10.10.2;tcp
525;10.10.10.5;ac;10.10.10.6;tcp
522;10.10.10.3;ab;10.10.10.4;udp
535;10.10.10.5;ac;10.10.10.6;tcp
...

Periodically throughout the file is a header line and the lines underneath that are the actual data where the fields correspond to the header line. However, sometimes the order of the fields change, and different fields are used, but that change is always marked by a new header line. The new header lines can be anywhere in the data file. Fields might be next to each other at one point in the file, then separated in a later part of the file.

I'd like to be able to produce a simplified output of just a few of the fields in the data file. For instance, I'd like to extract the src, dst, and proto from the example above. src is normally field 2, but dst is field 3 and then changes to field 4. My desired output would look something like this:

Code:
src;dst;proto
10.10.10.1;10.10.10.2;tcp
10.10.10.3;10.10.10.4;udp
10.10.10.5;10.10.10.6;tcp
10.10.10.1;10.10.10.2;tcp
10.10.10.5;10.10.10.6;tcp
10.10.10.3;10.10.10.4;udp
10.10.10.5;10.10.10.6;tcp

I've worked with AWK quite a bit and know how to work with field numbers, if/then, etc, but I can't figure out how to change a field number to a new value as directed by the header line.

Can anyone help me? I'd sure appreciate any advice. Is AWK the right tool to do this with?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

printing select fields in awk

Hi, I want to print certain fields from my data file depending on certain conditions. Somebody pls let me know how to send it to awk. The command below is the one which I want to use in a shell script and this prints fine cat ./datafile.dat | grep -i $SEARCH_STR | awk -F: '{ print $1 $2 $3... (5 Replies)
Discussion started by: maverix
5 Replies

2. Shell Programming and Scripting

need help with awk in printing the fields in a file

hi all i get a file from a server and i do not know how many fields that file will contain. i need to cut from the second column of the file to the last, irrespective of how many fields are there. is there any way to make the awk command dynamic for fetching from the second to the last... (4 Replies)
Discussion started by: sais
4 Replies

3. UNIX for Dummies Questions & Answers

printing fields in reverse order

command/script(apart from awk) to print the fields in reverse order that is last field has to come first and so on and first field has to go last Input store-id date sale ............. ............. ... (3 Replies)
Discussion started by: tsurendra
3 Replies

4. Shell Programming and Scripting

Putting values into order in a field using awk

Hi, I am using UBUNTU 12.04. I have a dataset as follows: Column#1 Column#2 Column#3 .... Column#50 1 154878 1 145145 2 189565 2 454121 ... (5 Replies)
Discussion started by: Homa
5 Replies

5. Shell Programming and Scripting

awk help: Match data fields from 2 files & output results from both into 1 file

I need to take 2 input files and create 1 output based on matches from each file. I am looking to match field #1 in both files (Userid) and create an output file that will be a combination of fields from both file1 and file2 if there are any differences in the fields 2,3,4,5,or 6. Below is an... (5 Replies)
Discussion started by: ambroze
5 Replies

6. Shell Programming and Scripting

How to print 1st field and last 2 fields together and the rest of the fields after it using awk?

Hi experts, I need to print the first field first then last two fields should come next and then i need to print rest of the fields. Input : a1,abc,jsd,fhf,fkk,b1,b2 a2,acb,dfg,ghj,b3,c4 a3,djf,wdjg,fkg,dff,ggk,d4,d5 Expected output: a1,b1,b2,abc,jsd,fhf,fkk... (6 Replies)
Discussion started by: 100bees
6 Replies

7. Shell Programming and Scripting

awk processing of variable number of fields data file

Hy! I need to post-process some data files which have variable (and periodic) number of fields. For example, I need to square (data -> data*data) the folowing data file: -5.34281E-28 -3.69822E-29 8.19128E-29 9.55444E-29 8.16494E-29 6.23125E-29 4.42106E-29 2.94592E-29 1.84841E-29 ... (5 Replies)
Discussion started by: radudownload
5 Replies

8. Shell Programming and Scripting

awk Selective printing of fields

Hi Gurus, I have following input file. I tried multiple awk combinations to print selected columns without success. HEX ID Name ver FLRGT Start Time Total Shared End Date ----- -------- --- ------ ------------------------ -------------- -------... (4 Replies)
Discussion started by: shunya
4 Replies

9. UNIX for Beginners Questions & Answers

Continued trouble matching fields in different files and selective field printing ([g]awk)

I apologize in advance, but I continue to have trouble searching for matches between two files and then printing portions of each to output in awk and would very much appreciate some help. I have data as follows: File1 PS012,002 PRQ 0 1 1 17 1 0 -1 3 2 1 2 -1 ... (7 Replies)
Discussion started by: jvoot
7 Replies

10. UNIX for Beginners Questions & Answers

awk to clean up input file, printing both fields

In the f1 file below I am trying to clean it up removing lines the have _tn_ in them. Next, removing the characters in $2 before the ninth /. Then I remove the ID_(digit- always 4). Finally, the charcters after and including the first _. It is curently doing most of it but the cut is removing $1... (5 Replies)
Discussion started by: cmccabe
5 Replies
IGAWK(1)							 Utility Commands							  IGAWK(1)

NAME
igawk - gawk with include files SYNOPSIS
igawk [ all gawk options ] -f program-file [ -- ] file ... igawk [ all gawk options ] [ -- ] program-text file ... DESCRIPTION
Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1). AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like @include getopt.awk in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path. OPTIONS
See gawk(1) for a full description of the AWK language and the options that gawk supports. EXAMPLES
cat << EOF > test.awk @include getopt.awk BEGIN { while (getopt(ARGC, ARGV, "am:q") != -1) ... } EOF igawk -f test.awk SEE ALSO
gawk(1) Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995. AUTHOR
Arnold Robbins (arnold@skeeve.com). Free Software Foundation Nov 3 1999 IGAWK(1)
All times are GMT -4. The time now is 11:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy