Sponsored Content
Top Forums Shell Programming and Scripting Join columns across multiple lines in a Text based on common column using BASH Post 303014159 by nv186000 on Tuesday 6th of March 2018 07:06:50 AM
Old 03-06-2018
Attempts so far

In another thread some one suggested below AWK code , unfortunately our awk version did not match and i was not able to use this .

Code:
{ a[$2][$1] };
END {
    for (col in a) {
        printf "%s", col;
        for (tab in a[col])
            printf "|%s", tab;
        print ""
    }
}

Another method i am trying now is using a self join

Code:
join -t "|" -1 2 -2 2  -o '1.1,2.1,1.2'  file  file

and i am getting below output

Code:
Table1|Table1|Column1
Table1|Table2|Column1
Table1|Table5|Column1
Table2|Table1|Column1
Table2|Table2|Column1
Table2|Table5|Column1
Table5|Table1|Column1
Table5|Table2|Column1
Table5|Table5|Column1
Table3|Table3|Column2
Table3|Table2|Column2
Table2|Table3|Column2
Table2|Table2|Column2
Table4|Table4|Column3
Table4|Table2|Column3
Table2|Table4|Column3
Table2|Table2|Column3
Table2|Table2|Column4
Table2|Table5|Column4
Table5|Table2|Column4
Table5|Table5|Column4
Table2|Table2|Column5

Now at least 2 problem with this approach

1) At max I can get 2 tables with common join columns . If there are 3 tables with same columns i will have to split it into 2 separate SQL statement with one join each.

2) I need to remove functional duplicates . that is joins between same tables , joins where just table names are reversed table1 join table2 == table2 join table1

Note:
1) I am open to using awk , or sed ..or anything that works.
2) outside Bash i meant , i can use hard coded concatenation to generate most part of the SQL statement apart from the join part.
Moderator's Comments:
Mod Comment Please use CODE tags for sample input and output as well as for code segments.

Last edited by Don Cragun; 03-06-2018 at 08:26 AM.. Reason: Add CODE and ICODE tags.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

find common lines using just one column to compare and result with all columns

Hi. If we have this file A B C 7 8 9 1 2 10 and this other file A C D F 7 9 2 3 9 2 3 4 The result i´m looking for is intersection with A B C D F so the answer here will be (10 Replies)
Discussion started by: alcalina
10 Replies

2. Shell Programming and Scripting

sum multiple columns based on column value

i have a file - it will be in sorted order on column 1 abc 0 1 abc 2 3 abc 3 5 def 1 7 def 0 1 -------- i'd like (awk maybe?) to get the results (any ideas)??? abc 5 9 def 1 8 (2 Replies)
Discussion started by: jjoe
2 Replies

3. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
Discussion started by: Katabatic
1 Replies

4. Shell Programming and Scripting

Join multiple files based on 1 common column

I have n files (for ex:64 files) with one similar column. Is it possible to combine them all based on that column ? file1 ax100 20 30 40 ax200 22 33 44 file2 ax100 10 20 40 ax200 12 13 44 file2 ax100 0 0 4 ax200 2 3 4 (9 Replies)
Discussion started by: quincyjones
9 Replies

5. Shell Programming and Scripting

join files based on a common field

Hi experts, Would you please help me with this? I have several files and I need to join the forth field of them based on the common first field. here's an example... first file: 280346 39.88 -75.08 547.8 280690 39.23 -74.83 538.7 280729 40.83 -75.08 499.2 280907 40.9 -74.4 507.8... (5 Replies)
Discussion started by: GoldenFire
5 Replies

6. UNIX for Dummies Questions & Answers

How to use the the join command to join multiple files by a common column

Hi, I have 20 tab delimited text files that have a common column (column 1). The files are named GSM1.txt through GSM20.txt. Each file has 3 columns (2 other columns in addition to the first common column). I want to write a script to join the files by the first common column so that in the... (5 Replies)
Discussion started by: evelibertine
5 Replies

7. UNIX for Dummies Questions & Answers

How to join 2 .txt files based on a common column?

Hi all, I'm trying to join two .txt file tab delimitated based on a common column. File 1 transcript_id gene_id length effective_length expected_count TPM FPKM IsoPct comp1000201_c0_seq1 comp1000201_c0 337 183.51 0.00 0.00 0.00 0.00 comp1000297_c0_seq1 ... (1 Reply)
Discussion started by: alisrpp
1 Replies

8. Shell Programming and Scripting

Join common patterns in multiple lines into one line

Hi I have a file like 1 2 1 2 3 1 5 6 11 12 10 2 7 5 17 12 I would like to have an output as 1 2 3 5 6 10 7 11 12 17 any help would be highly appreciated Thanks (4 Replies)
Discussion started by: Harrisham
4 Replies

9. Shell Programming and Scripting

Join multiple lines from text file

Hi Guys, Could you please advise how to join multiple details lines into single row, with HEADER 1 as the record separator and comma(,) as the field separator. Input: HEADER 1, HEADER 2, HEADER 3, 11,22,33, COLUMN1,COLUMN2,COLUMN3, AA1, BB1, CC1, END: ABC HEADER 1, HEADER 2,... (3 Replies)
Discussion started by: budz26
3 Replies

10. Shell Programming and Scripting

Paste columns based on common column: multiple files

Hi all, I've multiple files. In this case 5. Space separated columns. Each file has 12 columns. Each file has 300-400K lines. I want to get the output such that if a value in column 2 is present in all the files then get all the columns of that value and print it side by side. Desired output... (15 Replies)
Discussion started by: genome
15 Replies
All times are GMT -4. The time now is 01:16 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy