Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

bash - joining lines in a file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting bash - joining lines in a file
# 8  
Old 03-02-2011
Quote:
Originally Posted by Cultcha
Sorry to come back to you on this again.

It seems that not all rows have joined appropriately. Here's the input:

Code:
 
1234567DR$MYSELF              $ANDI                $9876543H $          12346.87$     497.52$     123.56$XY$ $12
1234567DR$MYSELF              $ANDI                $9876543H $              0.00$       0.00$       0.00$TR$ $10

Did you check the line count after running the sed command..? I guess since the posted above 2 lines are lengthy, it appears as if they are not joined. Pls check again.
# 9  
Old 03-02-2011
From my test file, I should have 19 lines after the merge. I have 23. Maybe I would be best to use AWK to substitute the new lines based on the two conditions?
# 10  
Old 03-02-2011
If the conditions are complex then awk would do the job better. Post your sample real data with your requirements correctly.
# 11  
Old 03-02-2011
Hi again, thanks for your help with this!

This file contains records matched on column 1 and column 4.

There can be up to four rows for each record, these rows appear sequentially on the file. However, duplicate rows can appear in the file also, so I would like to only join up to four rows and then delete duplicates after the join (as occasionally 2/3 of the four rows are duplicates of each other validly).

Where column 1 and column 4 are the same, I want to join the lines together (really I just want to add the different data (ie. columns 8 and 10) to the end of the first row.

Code:
 
1234567D $J                  $P                  $1234567N $          26575.00$   1670.89$   4527.81$XY$ $ 1$
1234567D $J                  $P                  $1234567N $              0.00$      0.00$      0.00$SY$ $51$
2456789B $P                  $T                  $8888888U $           5577.00$    157.00$    756.00$XY$ $13$
2456789B $P                  $T                  $9999999B $          30938.00$   1916.36$   5223.85$SY$ $25$
3333333I $L                  $G                  $1111111H $          39068.00$   2609.11$   6808.98$SY$ $52$
4444444F $GE                 $WI                 $6656656G $          21850.00$    683.13$   3032.06$XY$ $50$
4444444F $GE                 $WI                 $6656656G $              0.00$      0.00$      0.00$SY$ $ 2$
5555555H $J                  $BU                 $4545454D $          46698.00$   3159.91$   8179.99$SY$ $52$
6666666J $FR                 $CU                 $6372232D $          13448.00$     25.44$   1180.20$GP$ $51$
6666666J $FR                 $CU                 $6372232D $              0.00$      0.00$      0.00$SY$ $ 1$
7777777U $CH                 $TO                 $4444444P $           8667.00$    624.42$   1556.19$SY$ $ 9$
7777777U $CH                 $TO                 $4444444P $              0.00$      0.00$      0.00$XY$ $ 1$
1234112P $JI                 $TI                 $4582809N $          23117.00$      0.00$      0.00$R $ $52$
1234112P $JI                 $TI                 $9508160H $          14243.00$    768.00$   2299.00$SY$ $17$
4445566P $RI                 $B                  $4521440T $           7788.00$      0.00$      0.00$X1$ $52$
4445566P $RI                 $B                  $4567892P $          17234.00$    497.00$   2279.00$XY$ $26$
4445566P $RI                 $B                  $4567892P $              0.00$      0.00$      0.00$GP$ $17$
4445566P $RI                 $B                  $4567892P $              0.00$      0.00$      0.00$HU$ $ 5$
4445566P $RI                 $B                  $4567892P $              0.00$      0.00$      0.00$SY$ $ 4$
3334455E $G                  $GR                 $1239875F $          22149.00$    680.78$   3061.65$SY$ $ 2$
3334455E $G                  $GR                 $1239875F $              0.00$      0.00$      0.00$XY$ $50$
0000001V $RI                 $RA                 $1212121R $           1400.00$      0.00$      0.00$R $ $52$
0000001V $RI                 $RA                 $4556455F $          15384.00$    734.61$   2257.14$SY$ $16$
0000001V $RI                 $RA                 $4556455F $              0.00$      0.00$      0.00$R $ $32$
0000001V $RI                 $RA                 $4556455F $              0.00$      0.00$      0.00$XY$ $ 4$
0000001V $RI                 $RA                 $4556455F $              0.00$      0.00$      0.00$X1$ $ 2$
2222222N $K                  $G                  $7878787R $          35340.00$   2327.12$   6126.20$SY$ $52$
2222222N $K                  $G                  $6205125L $          37110.00$   1237.00$   1237.00$X1$ $52$
6565656S $PR                 $GU                 $5645564D $          60000.00$   4135.32$  10585.32$SY$ $52$
6565656S $PR                 $GU                 $9595959F $           2848.00$      0.00$      0.00$R $ $45$

This is a test file that I'm using. Let me know if you need any other info!
# 12  
Old 03-03-2011
This is what are you looking ?
Code:
awk -F"\$" '{if(!a[$1$4]){a[$1$4]=$0;printf NR==1?$0:"\n"$0}else{printf "%s%s",$8FS,$10}}' infile

This User Gave Thanks to pravin27 For This Post:
Cultcha
# 13  
Old 03-03-2011
Thanks so much! This was exactly what I was looking for.

Just in case this is useful to anyone else at any stage I then replaced the ^M characters created with $s using:

sed 's/^M/$/g' file1 > file2

^M is typed as ctrl + v + enter

thanks again!!
Last edited by Cultcha; 03-03-2011 at 07:45 AM.. Reason: stupidity
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Joining especific lines in "2n" lines file

Hi to everybody. I have a "2n" lines file. I would like to create a new file with only "n" lines, each line in the new file formed by the proper odd line of the old file joined with the following even line (separated by a space) of the old file. I'd prefer using sed or bash. -example-... (5 Replies)
Discussion started by: felino
5 Replies

2. Shell Programming and Scripting

Issue in Concatenation/Joining of lines in a dynamically generated file

Hi, I have a file containing many records delimited by pipe (|). Each record should contain 17 columnns/fields. there are some fields having fields less than 17.So i am extracting those records to a file using the below command awk 'BEGIN {FS="|"} NF !=17 {print}' feedfile.txt... (8 Replies)
Discussion started by: TomG
8 Replies

3. Shell Programming and Scripting

Joining lines in TXT file based on first character

Hi, I have a pipe delimeted text file where lines have been split over 2 lines and I need to join them back together. For example the file I have is similar to the following: aaa|bbb |ccc ddd|eee fff|ggg |hhh I ideally need to have it looking like the following aaa|bbb|ccc ddd|eee... (5 Replies)
Discussion started by: fuji_s
5 Replies

4. Shell Programming and Scripting

joining multi-line file into single lines

Hi, I have a file like mentioned below..For each specific id starting with > I want to join the sequence in multiple lines to a single line..Is there a simple way in awk or sed to do this >ENST00000558922 cdna:KNOWN TCCAGGATCCAGCCTCCCGATCACCGCGCTAGTCCTCGCCCTGCCTGGGCTTCCCCAGAG... (2 Replies)
Discussion started by: Diya123
2 Replies

5. Shell Programming and Scripting

Joining lines in a file - help!

I'm looking for a way to join lines in a file; e.,g consider the following R|This is line 1 R|This is line 2 R|This is line 3 R|This is line 4 R|This is line 5 what i want to end up with is R|This is line 1 R|This is line 2 R|This is line 3 R|This is line 4 R|This is line 5 so... (15 Replies)
Discussion started by: Storms
15 Replies

6. Shell Programming and Scripting

Joining lines in a text file using AWK or SED

Hi All I'm struggling a bit here :( I need a way of joining lines contained in a text file. I've seen numerous SED and AWK examples and none of them seem to be working for me. The text file has 4 lines: DELL1427 DOC 30189342 79 Now bear with me on this one as I'm actually... (4 Replies)
Discussion started by: huskie69
4 Replies

7. UNIX for Dummies Questions & Answers

JOINING MULTIPLE LINES IN A TEXT FILE USING GAWK

sir... am having a data file of customer master., containing some important fields as a set one line after another., what i want is to have one set of these fields(rows) one after another in line.........then the second set... and so on... till the last set completed. I WANT THE DATA... (0 Replies)
Discussion started by: KANNI786
0 Replies

8. UNIX for Dummies Questions & Answers

Joining lines of a text file using GAWK

sir... am having a data file of customer master., containing some important fields as a set one line after another., what i want is to have one set of these fields(rows) one after another in line.........then the second set... and so on... till the last set completed. ... (0 Replies)
Discussion started by: KANNI786
0 Replies

9. Shell Programming and Scripting

Joining 2 lines in a file together

Hi guys, I've got a log file which has entries that look like this: ------------------------------------------------------------------------------- 06/08/04 07:57:57 AMQ9002: Channel program started. EXPLANATION: Channel program 'INSCCPQ1.HSMTSPQ1' started. ACTION: None. ... (3 Replies)
Discussion started by: m223464
3 Replies

10. Shell Programming and Scripting

Joining lines in log file

Hi, I need to develop a script to join multiple three lines in a log file into one line for processing with awk and grep. I looked at tr with no success. The first line contains the date time information. The second line contains the error line. The third line is a blank line. Thanks, Mike (3 Replies)
Discussion started by: bubba112557
3 Replies
Login or Register to Ask a Question