Join Files

choppas · #1 (**permalink**) 10-17-2006

Hi Gurus,
I have to join two flat files based on two key field columns. I concatenated two key fields and i tried the join command. It is working
fine. But, without using temporary files can't i use like this:

join -t ':' `awk -F ":" '{ printf("%s%s:%s\n", $1,$2, $0) }' file1` `awk -F ":" '{ printf("%s%s:%s\n", $1,$2, $0) }' file2`

My each file is having the size nearly 2 GB. Similary i need to run 27 jobs parellely (scripts which are using join).
So i need to join 54 files approximately. If i use temporary files 54 * 2 = 108 GB space needs to be utilized more.

Without using temporary files is there any other approach to proceed?

Thanks in advance
Srinivas Choppa

jim mcnamara · #2 (**permalink**) 10-17-2006

try:

Code:

awk '
    FILENAME=="file1" {
        Keys[$1 $2]=$0
    }
    FILENAME=="file2" {
    	if (Keys[$1 $2]>" "){
         printf("%s ", Keys[$1 $2]);
         for(i=3;i<=NF;i++) printf("%s ",$i);
         printf("\n");
        }
    }
' file1 file2

This produces $0 (whole record) from file1 joined with alll of the record after they key from file2.

choppas · #3 (**permalink**) 10-18-2006

Hi Jim,

Thank you for the reply.

But, i am unable to use the array contents (Key[$1 $2]) created in file1 block in file2 block. Means everytime it is returning NULL value in File2 block.

I am very much new to AWK programming. Can you please explain?

Thanks in advance,
Srinivas Choppa

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
join two files	koti_rama	Shell Programming and Scripting	5	08-05-2008 04:20 AM
Join two files	koti_rama	Shell Programming and Scripting	4	06-10-2008 06:15 AM
Join columns from 2 files	osramos	Shell Programming and Scripting	2	11-14-2007 05:25 AM
how to join files	jxh461	UNIX for Dummies Questions & Answers	5	08-23-2007 07:11 AM
join files	mohan705	Shell Programming and Scripting	3	03-15-2007 06:51 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by