Merging Adjacent Lines Using Gawk Post: 302479005

Sponsored Content

Top Forums Shell Programming and Scripting Merging Adjacent Lines Using Gawk Post 302479005 by Jahn on Thursday 9th of December 2010 11:18:55 AM

12-09-2010

Registered User

Merging Adjacent Lines Using Gawk

Hi all,

I have a text file consisting of 4 columns. What I am trying to do is see whether column 2 repeats multiple times, and collapse those repeats into one row. For example, here is a snippet of the file I am trying to analyze:

1 Gamble_Win 14.282 0.502
1 Sure_Thing 14.858 0.174
1 Sure_Thing 15.043 0.39
1 Gamble_Loss 15.496 1.236
1 Gamble_Loss 16.982 0.402
1 Gamble_Loss 17.647 0.19
1 Gamble_Win 17.914 0.236
1 Arrow 18.203 0.371

Here is what I am trying to do: For the conditions where "Sure_Thing" and "Gamble_Loss" repeat, I want to collapse it into a single line, adding up all of column 4 over the repeats. So after I gawk it, I want it to look something like this:

1 Gamble_Win 14.282 0.502
1 Sure_Thing 14.858 0.564
1 Gamble_Loss 15.496 1.828
1 Gamble_Win 17.914 0.236
1 Arrow 18.203 0.371

Here is the code I have used to analyze it so far, but it only works for 2 adjacent repeats; I want to generalize it for multiple repeats:

Code:

igawk '

BEGIN{

OFS=" "

prevTrial = "-";
prevTime = "0";
prevDur = "0";
}

{

if ($2 == prevTrial)print $1, prevTrial, prevTime, prevDur+$4;
else if ($2 != prevTrial)print $0;


prevTrial = $2;
prevTime = $3;
prevDur = $4;
}

' $*

I appreciate any input!

Jahn

View Public Profile for Jahn

Find all posts by Jahn

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

print adjacent lines

how do you print the lines before and after the line you are interested in? Example: Line to be printed: line 344 Output: line 343 line 344 line 345 Thanks

2. Shell Programming and Scripting

gawk print some special lines

Hi every body, i have this file example : TD1 TD2 TD3 . . .TDn <DIE_1> xxxxxx <\DIE_1> <TD1> information 1 inormation n <\TD1> <TDq> information

3. Shell Programming and Scripting

Gawk combining lines unexpectedly

I am attempting to combine sections of log that should be one line but are spaced out over 10-30 lines due to how the software is outputting the info. (If I am making a newbie mistake I apologize) Example of log I am working with: 2009-04-14 14:51:22 access data here info. Info. Info. …….....

4. Shell Programming and Scripting

How to subtract the adjacent lines from a single column?

Hi All, I have a file with only one column and i need to subtract the adjacent lines of the same column and print it in the same column. For Example: (Input) Col1 5 10 12 6 9 12 5 . . . .output should be like this:

5. UNIX for Dummies Questions & Answers

Joining lines of a text file using GAWK

sir... am having a data file of customer master., containing some important fields as a set one line after another., what i want is to have one set of these fields(rows) one after another in line.........then the second set... and so on... till the last set completed. ...

6. Shell Programming and Scripting

Gawk / Awk Merge Lines based on Key

Hi Guys, After windows died on my netbook I installed Lubuntu and discovered Gawk about a month ago. After using Excel for 10+ years I'm amazed how quick and easily Gawk can process data but I'm stuck with a little problem merging data from multiple lines. I'm an SEO Consultant and provide...

7. Shell Programming and Scripting

merging two .txt files by alternating x lines from file 1 and y lines from file2

Hi everyone, I have two files (A and B) and want to combine them to one by always taking 10 rows from file A and subsequently 6 lines from file B. This process shall be repeated 40 times (file A = 400 lines; file B = 240 lines). Does anybody have an idea how to do that using perl, awk or sed?...

8. Emergency UNIX and Linux Support

[Solved] AWK to parse adjacent matching lines

Hi, I have an input file like F : 0.1 : 0.002 P : 0.3 : 0.004 P : 0.5 : 0.008 P : 0.1 : 0.005 L : 0.05 : 0.02 P: 0.1 : 0.006 P : 0.01 : 0.08 F : 0.02 : 0.08 Expected output:

9. Shell Programming and Scripting

Gawk Find Pattern Print Lines Before and After

Using grep I can easily use: cvs log |grep -iB 10 -A 10 'date: 2013-10-30' to display search results and 10 lines before and after. How can this be accompished using gawk?

10. Shell Programming and Scripting

Merging multiple lines to columns with awk, while inserting commas for missing lines

Hello all, I have a large csv file where there are four types of rows I need to merge into one row per person, where there is a column for each possible code / type of row, even if that code/row isn't there for that person. In the csv, a person may be listed from one to four times...

LEARN ABOUT FREEBSD

colrm

COLRM(1)						    BSD General Commands Manual 						  COLRM(1)

NAME

     colrm -- remove columns from a file

SYNOPSIS

     colrm [start [stop]]

DESCRIPTION

     The colrm utility removes selected columns from the lines of a file.  A column is defined as a single character in a line.  Input is read
     from the standard input.  Output is written to the standard output.

     If only the start column is specified, columns numbered less than the start column will be written.  If both start and stop columns are spec-
     ified, columns numbered less than the start column or greater than the stop column will be written.  Column numbering starts with one, not
     zero.

     Tab characters increment the column count to the next multiple of eight.  Backspace characters decrement the column count by one.

ENVIRONMENT

     The LANG, LC_ALL and LC_CTYPE environment variables affect the execution of colrm as described in environ(7).

EXIT STATUS

     The colrm utility exits 0 on success, and >0 if an error occurs.

SEE ALSO

     awk(1), column(1), cut(1), paste(1)

HISTORY

     The colrm command appeared in 3.0BSD.

BSD
								  August 4, 2004							       BSD

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

print adjacent lines

Discussion started by: apalex

2. Shell Programming and Scripting

gawk print some special lines

Discussion started by: kamel.kimo

3. Shell Programming and Scripting

Gawk combining lines unexpectedly

Discussion started by: demanche

4. Shell Programming and Scripting

How to subtract the adjacent lines from a single column?

Discussion started by: Fredrick