Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Merge two non-consecutive lines based on line number or string

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merge two non-consecutive lines based on line number or string
# 1  
Old 05-17-2011
Merge two non-consecutive lines based on line number or string
This is a variation of an earlier post found here:

unix[dot]com/shell-programming-scripting/159821-merge-two-non-consecutive-lines.html

User Bartus11 was kind enough to solve that example.

Previously, I needed help combining two lines that are non-consecutive in a file. Now I need to do the same thing, but the operation needs to be done on the first 6 lines and then start again on line 7 through line 12. I am having a difficult time explaining, so please see the pattern below.

Original File Here:

Please note, the "Line#:" is there only for reference. The lines can only be distinguished by whether field3 is either "Foo" or "Bar" and if field4 is either START or DONE.

Code:
Line01:  field0,field1,field2,Foo,START 
Line02:  field0,field1,field2,Foo,START 
Line03:  field0,field1,field2,Foo,START 
Line04:  field0,field1,field2,Foo,DONE 
Line05:  field0,field1,field2,Foo,DONE 
Line06:  field0,field1,field2,Foo,DONE
Line07:  field0,field1,field2,Bar,START
Line08:  field0,field1,field2,Bar,START
Line09:  field0,field1,field2,Bar,START
Line10:  field0,field1,field2,Bar,DONE
Line11:  field0,field1,field2,Bar,DONE
Line12:  field0,field1,field2,Bar,DONE

The "Foo, Start" line needs to be combined with the "Foo, Done" line to form "Foo, Start, Foo, Done" and the "Bar, Start" line needs to be combined with the "Bar Done" line to form "Bar, Start, Bar, Done". After combining the lines the file looks as follows:

Code:
field0,field1,field2,Foo,START,field0,field1,field2,Foo,DONE
field0,field1,field2,Foo,START,field0,field1,field2,Foo,DONE
field0,field1,field2,Foo,START,field0,field1,field2,Foo,DONE
field0,field1,field2,Bar,START,field0,field1,field2,Bar,DONE
field0,field1,field2,Bar,START,field0,field1,field2,Bar,DONE
field0,field1,field2,Bar,START,field0,field1,field2,Bar,DONE

So the pattern of the lines is as follows:

Code:
Line01,Line04
Line02,Line05
Line03,Line06
Line07,Line10
Line08,Line11
Line09,Line12

Thanks for any help.
# 2  
Old 05-17-2011
Sort the data so done is just before start, and then combine adjacent lines:
Code:
sort | sed '
  N
  s/\(.*\)\n\(.*\)/\2,\1/
 '

# 3  
Old 05-17-2011
Almost
Quote:
Originally Posted by DGPickett
Sort the data so done is just before start, and then combine adjacent lines:
Code:
sort | sed '
  N
  s/\(.*\)\n\(.*\)/\2,\1/
 '

Thanks for the reply! That almost works, but it's producing output as follows:
Code:
Line04,Line01
Line10,Line07
Line05,Line02
Line11,Line08
Line06,Line03
Line12,Line09

So the "DONE" line is merging before the "START" line and "Foo" and "Bar" lines are alternating, producing the following pattern:

Code:
field0,field1,field2,Foo,DONE,field0,field1,field2,Foo,START
field0,field1,field2,Bar,DONE,field0,field1,field2,Bar,START
field0,field1,field2,Foo,DONE,field0,field1,field2,Foo,START
field0,field1,field2,Bar,DONE,field0,field1,field2,Bar,START
field0,field1,field2,Foo,DONE,field0,field1,field2,Foo,START
field0,field1,field2,Bar,DONE,field0,field1,field2,Bar,START

---------- Post updated at 02:53 PM ---------- Previous update was at 02:39 PM ----------

Ok, so by simply switching the numbers the order is how I need it.

Code:
sort | sed '
  N
  s/\(.*\)\n\(.*\)/\1,\2/
 '

instead of

Code:
sort | sed '
  N
  s/\(.*\)\n\(.*\)/\2,\1/
 '

With that change the "START" line merges before the "DONE" line. Now how can I solve the alternating problem "Foo" line, "Bar" line issue?

The new pattern is as follows:

Code:
Line01,Line04
Line07,Line10
Line02,Line05
Line08,Line11
Line03,Line06
Line09,Line12

Thanks for any help.
Last edited by munkee; 05-17-2011 at 03:45 PM..
# 4  
Old 05-17-2011
Code:
# ./justdoit.sh
field0,field1,field2,Foo,START,field0,field1,field2,Foo,DONE
field0,field1,field2,Foo,START,field0,field1,field2,Foo,DONE
field0,field1,field2,Foo,START,field0,field1,field2,Foo,DONE
field0,field1,field2,Bar,START,field0,field1,field2,Bar,DONE
field0,field1,field2,Bar,START,field0,field1,field2,Bar,DONE
field0,field1,field2,Bar,START,field0,field1,field2,Bar,DONE

Code:
#!/bin/bash
inc=3;mul=$(sed 'N;s/\n//' file1|sed -n $=);x=$((1+$inc ))
f=1;ff=$f;x=$((f+inc)) ;c=$(($mul/2-1))
for((i=0;i<$mul;i++)) ; do
sed -n "$f p;$x p" file1|sed 'N;s/\n/,/'
if [ $i -eq $c ] ; then f=$(($ff+$mul-1)) ; x=$(($f+$inc));fi
((f++)) ; ((x++))
done

regards
ygemici
This User Gave Thanks to ygemici For This Post:
munkee
# 5  
Old 05-17-2011
Try:
Code:
awk '{a[NR]=$0}END{for (i=1;i<=NR/2;i++){print a[i+3*(int((i-1)/3))]","a[i+3*(int((i-1)/3))+3]}}' file

This User Gave Thanks to bartus11 For This Post:
munkee
# 6  
Old 05-17-2011
Code:
# awk -F'[: ,]' '{
        if(/START/){
                        a[$(NF-1)]=((a[$(NF-1)])?a[$(NF-1)] OFS:x)$1
                        }
        if(/DONE/){
                        b[$(NF-1)]=((b[$(NF-1)])?b[$(NF-1)] OFS:x)$1
                        }
}
END{
        for (i in a){
                        split(b[i],d,OFS)
                        split(a[i],c,OFS)
                        for(j=1;j<=length(c);j++){
                                        print c[j],d[j]
                                        }
                        }
}'  file
Line01 Line04
Line02 Line05
Line03 Line06
Line07 Line10
Line08 Line11
Line09 Line12

This User Gave Thanks to danmero For This Post:
munkee
# 7  
Old 05-17-2011
Normally, S sorts after D, but there must be other details.

For foo and bar, use multiple sort keys.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep three consecutive lines if each lines contains certain string

say we have : 2914 | REQUEST | whatever 2914 | RESPONSE | whatever 2914 | SUCCESS | whatever 2985 | RESPONSE | whatever 2986 | REQUEST | whatever 2990 | REQUEST | whatever 2985 | RESPONSE | whatever 2996 | REQUEST | whatever 2010 | SUCCESS | whatever 2013 | REQUEST | whatever 2013 |... (7 Replies)
Discussion started by: Saumitra Pandey
7 Replies

2. Shell Programming and Scripting

Print lines based on line number and specified condition

Hi, I have a file like below. 1,2,3,4,5,6,7,8,9I would like to print or copied to a file based of line count in perl If I gave a condition 1 to 3 then it should iterate over above file and print 1 to 3 and then again 1 to 3 etc. output should be 1,2,3 4,5,6 7,8,9 (10 Replies)
Discussion started by: Anjan1
10 Replies

3. Shell Programming and Scripting

Grep couple of consecutive lines if each lines contains certain string

Hello, I want to extract from a file like : 20120530025502914 | REQUEST | whatever 20120530025502968 | RESPONSE | whatever 20120530025502985 | RESPONSE | whatever 20120530025502996 | REQUEST | whatever 20120530025503013 | REQUEST | whatever 20120530025503045 | RESPONSE | whatever I want... (14 Replies)
Discussion started by: black_fender
14 Replies

4. Shell Programming and Scripting

join based on line number when one file is missing lines

I have a file that contains 87 lines, each with a set of coordinates (x & y). This file looks like: 1 200.3 -0.3 2 201.7 -0.32 ... 87 200.2 -0.314 I have another file which contains data that was taken at certain of these 87 positions. i.e.: 37 125 42 175 86 142 where the first... (1 Reply)
Discussion started by: jackiev
1 Replies

5. Shell Programming and Scripting

Merge two non-consecutive lines

Hello - First post here. I need help combining two lines that are non-consecutive in a file. Using sed, awk or perl preferably. So the file looks as follows. Please note, the "Line#:" is there only for reference. The lines can only be distinguished by whether they have "start" or "done" in... (2 Replies)
Discussion started by: munkee
2 Replies

6. Shell Programming and Scripting

Extract string from multiple file based on line count number

Hi, I search all forum, but I can not find solutions of my problem :( I have multiple files (5000 files), inside there is this data : FILE 1: 1195.921 -898.995 0.750312E-02-0.497526E-02 0.195382E-05 0.609417E-05 -2021.287 1305.479-0.819754E-02 0.107572E-01 0.313018E-05 0.885066E-05 ... (15 Replies)
Discussion started by: guns
15 Replies

7. Shell Programming and Scripting

Delete lines based on line number

I have a file with ~200K lines, I need to delete 4K lines in it. There is no range. I do have the line numbers of the lines which I want to be deleted. I did tried using > cat del.lines sed '510d;12d;219d;......;3999d' file > source del.lines Word too long. I even tried... (2 Replies)
Discussion started by: novice_man
2 Replies

8. Shell Programming and Scripting

merge lines into single line based on symbol \t

The symbols are \t and \t\t (note: not tab) If the line starts with \t merge them into a single line upto symbol \t\t \t\t to end and start new line I able to join in a single line but not ending at \t\t and I completely confused help would be appreciated:b::D Input \ta tab XXXXXXXXXX \te... (5 Replies)
Discussion started by: repinementer
5 Replies

9. UNIX for Dummies Questions & Answers

want to merge two consecutive lines.

Hi All, I want to merge two consecutive lines. Currently the output is :--> crmplp1 cmis461 No Online cmis462 No Offline crmplp2 cmis462 No Online cmis463 No ... (6 Replies)
Discussion started by: pank29
6 Replies

10. Shell Programming and Scripting

searching and storing unknown number of lines based on the string with a condition

Dear friends, Please help me to resolve the problem below, I have a file with following content: date of file creation : 12 feb 2007 ==================== = name : suresh = city :mumbai #this is a blank line = date : 1st Nov 2005 ==================== few lines of some text this... (7 Replies)
Discussion started by: swamymns
7 Replies
Login or Register to Ask a Question