How to join one file with multiple files in a directory in UNIX?

Login or Register to Ask a Question and Join Our Community

How to join one file with multiple files in a directory in UNIX?

Tags

Login to Discuss or Reply to this Discussion in Our Community

Top Forums Shell Programming and Scripting How to join one file with multiple files in a directory in UNIX?

04-23-2015

Registered User

44, 0

Join Date: Sep 2013

Last Activity: 16 November 2016, 12:10 PM EST

Posts: 44

Thanks Given: 12

Thanked 0 Times in 0 Posts

How to join one file with multiple files in a directory in UNIX?

Dear folks
Hello

I have a one file called (file1) which the structure looks like this

Code:

1       gi|358484521|ref|NW_003764373.1|
1       gi|358484520|ref|NW_003764374.1|
1       gi|358484519|ref|NW_003764375.1|
.
.
.
30   gi|368484519|ref|NW_00449375.1|

In addition, I have around 300 files which can called (file2) in a directory with this structure:

Code:

gi|358484513|ref|NW_003764381.1| 1751 T C 0 1 1
gi|358484498|ref|NW_003764396.1| 481 A C 1 1 2
gi|358484498|ref|NW_003764396.1| 2603 T G 0 1 1

Now my desire output is join the Second column of file1 with first column of file2.

Code:

1  gi|358484513|ref|NW_003764381.1| gi|358484513|ref|NW_003764381.1| 1751 T C 0 1 1
1  gi|358484498|ref|NW_003764396.1| gi|358484498|ref|NW_003764396.1| 481 A G 1 1 2
1  gi|358484498|ref|NW_003764396.1|  gi|358484498|ref|NW_003764396.1| 2603 A G 0 1 1

I was trying a for loop to get the join files for all of the 300 files in the directory but it does not happened.

I will appreciate if any one give me a suggestion.

Last edited by sajmar; 04-23-2015 at 02:34 PM..

sajmar

View Public Profile for sajmar

Find all posts by sajmar

04-23-2015

Moderator

12,296, 3,792

Join Date: Nov 2008

Last Activity: 1 January 2021, 1:47 AM EST

Location: Amsterdam

Posts: 12,296

Thanks Given: 679

Thanked 3,792 Times in 3,282 Posts

You could try:

Code:

awk 'NR==FNR{A[FNR]=$0; next} {print A[FNR], $0}' file1 otherfile*

If that does not exceed the maximimum command line argument length
or

Code:

echo otherfile* | xargs awk 'NR==FNR{A[NR]=$0; next} {print A[FNR], $0}' file1

Last edited by Scrutinizer; 04-23-2015 at 01:29 PM..

Scrutinizer

View Public Profile for Scrutinizer

Find all posts by Scrutinizer

04-23-2015

Registered User

44, 0

Join Date: Sep 2013

Last Activity: 16 November 2016, 12:10 PM EST

Posts: 44

Thanks Given: 12

Thanked 0 Times in 0 Posts

Thank for your suggestion. I tried those codes But these command will not meet my need.

sajmar

View Public Profile for sajmar

Find all posts by sajmar

04-23-2015

Moderator

12,296, 3,792

Join Date: Nov 2008

Last Activity: 1 January 2021, 1:47 AM EST

Location: Amsterdam

Posts: 12,296

Thanks Given: 679

Thanked 3,792 Times in 3,282 Posts

I made a correction to the second suggestion.

What does not work exactly? Can you elaborate?

Scrutinizer

View Public Profile for Scrutinizer

Find all posts by Scrutinizer

04-23-2015

Registered User

44, 0

Join Date: Sep 2013

Last Activity: 16 November 2016, 12:10 PM EST

Posts: 44

Thanks Given: 12

Thanked 0 Times in 0 Posts

As I mentioned, I have 300 files in the folders which are like file2 and they all have the same ending file name which is *.geno . now each of these file have different rows. I want to know how many of the id fields like (gi|358484521|ref|NW_003764373.1|) are present in each of these 300 files.

sajmar

View Public Profile for sajmar

Find all posts by sajmar

04-23-2015

Registered User

15,129, 5,008

Join Date: Jul 2012

Last Activity: 4 May 2020, 4:31 PM EDT

Location: Aachen, Germany

Posts: 15,129

Thanks Given: 735

Thanked 5,008 Times in 4,483 Posts

... which doesn't work with the samples you have given. With a slightly modified file2, try

Code:

awk 'FNR==NR {T[$2]=$0;next} $1 in T {print T[$1], $0}' file1 file2
1       gi|358484521|ref|NW_003764373.1| gi|358484521|ref|NW_003764373.1| 3303 T G 0 1 1

Is that what you're after?

RudiC

View Public Profile for RudiC

Find all posts by RudiC

04-23-2015

Registered User

44, 0

Join Date: Sep 2013

Last Activity: 16 November 2016, 12:10 PM EST

Posts: 44

Thanks Given: 12

Thanked 0 Times in 0 Posts

Dear Rudic

Thanks for your command. your command did not give me anything in the output. By the way, if when I have 300 files, I think with this command, I should to run 300 times.

sajmar

View Public Profile for sajmar

Find all posts by sajmar

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Join files on multiple fields

Hello all, I want to join 2 tabbed files on the first 2 fields, and filling the missing values with 0. The 3rd column in each file is constant for the entire file. file1 12658699 ST5 XX2720 0 1 0 1 53039541 ST5 XX2720 1 0 1.5 1 file2 ...

2. Shell Programming and Scripting

Comparing two files in UNIX and create a new file similar to equi join

I have 2 files namely branch.txt file & RXD.txt file as below Ex:Branch.txt ========================= B1,Branchname1,city,country B2,Branchname2,city,country B3,Branchname3,city,country B4,Branchname4,city,country B5,Branchname5,city,country RXD file : will...

3. Shell Programming and Scripting

Join multiple files with filename

Please help, I want to join multiple files based on column 1, and put the missing values as 0. Also the colname in the output should say which file the values came from. FILE1 1 11 2 12 3 13 FILE2 2 22 3 23 4 24 FILE3 1 31 3 33 4 34 FILE1 FILE2 FILE3 1 11 0 31

4. Shell Programming and Scripting

Join multiple files

Hi there, I am trying to join 24 files (i showed example of 3 files below). They all have 2 columns. The first columns is common to all. The files are tab delimited eg file 1 rs0001 100e-34 rs0003 2.8e-01 rs008 1.9e-90 file 2 rs0001 1.98e-22 rs0004 3.77e-10...

5. UNIX for Dummies Questions & Answers

How to use the the join command to join multiple files by a common column

Hi, I have 20 tab delimited text files that have a common column (column 1). The files are named GSM1.txt through GSM20.txt. Each file has 3 columns (2 other columns in addition to the first common column). I want to write a script to join the files by the first common column so that in the...

6. Shell Programming and Scripting

Awk - join multiple files

Is it possible to join all the files with input1 based on 1st column? input1 a b c d e f input2 a b input3 a e input4 c

7. Shell Programming and Scripting

Join multiple files by column with awk

Hi all, I searched through the forum but i can't manage to find a solution. I need to join a set of files placed in a directory (~1600) by column, and obtain an output with first and second column common to each file, but following columns are taken from the file in the list (precisely the fourth...

8. Shell Programming and Scripting

How to join multiple files?

I am trying to join a few hundred files using join. Is there a way to use while read or something else to automate this. My problem is the following. Day 1 City Temp ABC 20 DEF 30 HIJ 15 Day 2 City Temp ABC 22 DEF 29 KLM 5 Day 3

9. UNIX for Dummies Questions & Answers

Join 2 files with multiple columns: awk/grep/join?

Hello, My apologies if this has been posted elsewhere, I have had a look at several threads but I am still confused how to use these functions. I have two files, each with 5 columns: File A: (tab-delimited) PDB CHAIN Start End Fragment 1avq A 171 176 awyfan 1avq A 172 177 wyfany 1c7k A 2 7...

10. UNIX for Dummies Questions & Answers

Join multiple Split files in Unix

Hi, I have a big file of 50GB size. I need copy it to a second ftp from a ftp. I am not able to do the full 50GB transfer as it timesout after some time. SO i am trying to split the file into 5gb each 10 files with the below command. split -b 5368709120 pack.tar.gz backup.gz After I...

Login or Register to Ask a Question