Join 3 or more files using matching column Post: 302517571

Sponsored Content

Top Forums Shell Programming and Scripting Join 3 or more files using matching column Post 302517571 by sk1418 on Wednesday 27th of April 2011 07:17:39 AM

04-27-2011

Registered User

three files version:

Code:

awk 'ARGIND==1{a[$1]=$0 " null null";next;} 
    ARGIND==2{ if($1 in a)sub(/ null /, " "$1" ",a[$1]);else a[$1]="null "$1" null";next; }
    ARGIND==3{if($1 in a)sub(/ null$/," "$1" ",a[$1]); else a[$1]="null null "$1;}
END{for(i in a)print a[i]}' file1 file2 file3

---------- Post updated at 13:17 ---------- Previous update was at 13:06 ----------

multi-files version:

just fill the files in "files" list

PHP Code:



#!/usr/bin/python
files=['file1','file2','file3']
dict={}
len = files.__len__()
for s in files:
    idx = files.index(s)
    f = open(s)
    line = [x.replace("\n","") for x in f.readlines()]
    for l in line:
        if(not dict.has_key(l)):
            dict[l] = ["null"]*len
        dict[l][idx] = l
    f.close()
keys = dict.keys()
for k in keys:
    print  reduce(lambda x,y: x + " " + y,dict[k])

sk1418

View Public Profile for sk1418

Find all posts by sk1418

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Join 3 files using key column in a mapping file

I'm new of UNIX shell scripting. I'm recently generating a excel report in UNIX(file with delimiter is fine). How should I make a script to do it? 1 file to join comes from output of one UNIX command, the second from another UNIX command, and third from a database query. The key columes of all...

2. UNIX for Dummies Questions & Answers

Join 2 files using first column

Hi, I'm trying to compare the first column of two files (tab or whitespace delimited, either way's fine, I`ve got both) and print the lines that are identical for the first column of both files. Something like this: File1 AAA 26 49 7 27 36 33 46 75 73 69 AAAAA 4 10 4 7 10 18 21...

3. Shell Programming and Scripting

Join multiple files by column with awk

Hi all, I searched through the forum but i can't manage to find a solution. I need to join a set of files placed in a directory (~1600) by column, and obtain an output with first and second column common to each file, but following columns are taken from the file in the list (precisely the fourth...

4. Shell Programming and Scripting

Matching the substring and join two files

Hi I had two files like below. file-1 101001234567890 101001234567891 101001234567892 101001234567893 101001234567894 101001234567895 101001234567896 101001234567897 101001234567898 101001234567899 file-2

5. Shell Programming and Scripting

join two files based on one column

Hi All, I am trying to join to files based on one common column. Cat File1 ID HID Ab_1 23 Cd 45 df 22 Vv 33 Cat File2 ID pval Ab_1 0.3 Cd 10 Vv 0.0444

6. UNIX for Dummies Questions & Answers

How to use the the join command to join multiple files by a common column

Hi, I have 20 tab delimited text files that have a common column (column 1). The files are named GSM1.txt through GSM20.txt. Each file has 3 columns (2 other columns in addition to the first common column). I want to write a script to join the files by the first common column so that in the...

7. Shell Programming and Scripting

Join two files with matching columns

Hi, I need to join two files together with one common value in a column. I think I can use awk or join or a combination but I can't quite get it. Basically my data looks like this, with the TICKER columns matching up in each file File1 TICKER,column 1, column, 2, column, 3, column 4 ...

8. UNIX for Dummies Questions & Answers

Join files by second column

I have file input file1 1/1/2013 A 553.0763397 96 16582 1/1/2013 B 459.8333588 195 11992 1/2/2013 A 844.2973022 306 19555 1/2/2013 B 833.9300537 457 20165 1/3/2013 A 563.6917419 396 13879 1/3/2013 B 632.0749969 169 ...

9. UNIX for Dummies Questions & Answers

Join 2 files based on certain column

I have file input1.txt 11103|11|OTTAWA|City|AA|CAR|0|0|1|-1|0|8526|2014-09-07 23:00:14 11103|11|OTTAWA|City|BB|TRAIN|0|0|2|-2|6|6359|2014-09-07 23:00:14 11104|11|CANADA|City|CC|CAR|0|0|2|-2|0|5947|2014-09-07 23:00:14 11104|11|CANADA|City|DD|TRAIN|0|0|2|-2|1|4523|2014-09-07 23:00:14...

10. Shell Programming and Scripting

Join 2nd column of multiple files

Dear All, I have many files formatted like this: file1.txt: 1/2-SBSRNA4 18 A1BG 3 A1BG-AS1 6 A1CF 0 A2LD1 1 A2M 1160 file2.txt 1/2-SBSRNA4 53 A1BG 1 A1BG-AS1 7 A1CF 0 A2LD1 3 A2M 2780