compare the column from 3 files and merge that line


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting compare the column from 3 files and merge that line
# 1  
Old 04-10-2008
CPU & Memory compare the column from 3 files and merge that line

I have 3 file, each of has got 80000 records.

file1.txt
-----------------------
ABC001;active;modify;accept;
ABC002;notactive;modify;accept;
ABC003;notactive;no-modify;accept;
ABC004;active;modify;accept;
ABC005;active;no-modify;accept;

file2.txt
---------------------------
ABC001;change;modify;numbers;
ABC002;no-change;print;fractions;
ABC003;different;color;accept;
ABC005;done;modify;accept;

file3.txt
---------------------------
ABC001;got it;modify;numbers;
ABC002;happening;print;fractions;
ABC003;different;color;accept;
ABC004;classify;modify;accept;


Now i have to compare the first field in the file1 (ABX001/002/003...)
with the file2 , whether file2 has got the first field of file1 , same comparison with file3.
if exists then merge all three to get output like below.

ABC001;active;modify;accept;change;modify;numbers;got it;modify;numbers;

Merge such that

Line1 from file1 and filed2.. to field_n of the matched line from file2 and file3.


thanks
Ganesh
# 2  
Old 04-10-2008
you can combine them into one file first.
Code:
cat file file1 file2 > newfile
awk 'BEGIN{OFS=FS=";"}
{
  org=$1
  $1=""
  a[org]=a[org]";"$0  
} 
END {
 for (i in a) print i,a[i]
}' newfile

# 3  
Old 04-10-2008
Try join command. It would be simple. But the limitation is only two files can be joined at a time.

This works like join in SQL.

Code:
man join

# 4  
Old 04-11-2008
CPU & Memory

Quote:
Originally Posted by ghostdog74
you can combine them into one file first.
Code:
cat file file1 file2 > newfile
awk 'BEGIN{OFS=FS=";"}
{
  org=$1
  $1=""
  a[org]=a[org]";"$0  
} 
END {
 for (i in a) print i,a[i]
}' newfile


thanks for the reply

with the above code i am getting output like this

ABC001;;;active;modify;accept;;;change;modify;numbers;;;got t;modify;numbers;
ABC002;;;notactive;modify;accept;;;no-change;print;fractions;;;happening;print;fractions;
ABC003;;;notactive;no-modify;accept;;;different;color;accept;;;different;color;accept;
ABC004;;;active;modify;accept;;;classify;modify;accept;
ABC005;;;active;no-modify;accept;;;done;modify;accept;


but i need like this

ABC001;active;modify;accept;change;modify;numbers;got it;modify;numbers;
ABC002;notactive;modify;accept;no-change;print;fractions;happening;print;fractions;
ABC003;notactive;no-modify;accept;different;color;accept;different;color;accept;
ABC004;active;modify;accept;;;;classify;modify;accept;
ABC005;active;no-modify;accept;done;modify;accept;;;;


No two semicolons after ABCXXX (;Smilie .

ABC004 is there in file1 and file3 but not in file2. so output should be 3 blank semicolons .ABC004;active;modify;accept;;;;classify;modify;accept;

same for ABC005 blank in place of output of file3.

if ABCxxx is there in file2 and file3 and not in file1 then output should be
ABCxxx;;;out21;out22;out23;out31;out32;out33;


thanks
Ganesh
# 5  
Old 04-11-2008
Code:
awk '{ s = $1; sub(/^[^;]*;/,"")
  if (FILENAME == "file1") 
    _[s] = $0
  if (FILENAME == "file2" && (s in _)) 
    f[s] = _[s] $0
  if (FILENAME == "file3" && (s in f)) 
    f_[s] = f[s] $0
} END {
for (k in f_)
  print k FS f_[k]
}' FS=\; file1 file2 file3

Use nawk or /usr/xpg4/bin/awk on Solaris.
# 6  
Old 04-11-2008
its left as exercise for you, on purpose
# 7  
Old 04-14-2008
thanks for the reply its working.
i need to add few more conditions to this . i ll try these.

but i am not able to append numbers to the string
Code:
for(i=0;i<max;i++) {
_[s_i] = i;
print _[s_i]
}

like s_1 = 1;s_2=2; ........


it does not work ?
how do i make it work
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare and merge two big CSV files

Hi all, i need help. I have two csv files with a huge amount of data. I need the first column of the first file, to be compared with the data of the second, to have at the end a file with the data not present in the second file. Example File1: (only one column) profile_id 57036226... (11 Replies)
Discussion started by: SirMannu
11 Replies

2. Shell Programming and Scripting

Compare two files and merge into third

Hello: Newbie with Awk. Trying to compare two files and merge data based on CID. Please see the input file format and desired output. Any help is appreciated. TIA Input File1 CID1 --- TYP1 --- DCN1 --- INDATE1 --- IN-DATA1 CID2 --- TYP2 --- DCN2 --- INDATE2 --- IN-DATA2 CID3 ---... (6 Replies)
Discussion started by: wincrazy
6 Replies

3. Shell Programming and Scripting

Compare and Merge files

Hi All, I have two different files as shown below separated by a "|". I need to compare the first column from both the files and if they match merge both the columns. File 1 "S00172012"|"CHRONIC RENAL FAILURE"|""|"I" "S00159962"|"SUBENDO INFRC-INIT EPISD"|""|"I" "S00255303"|"BENIGN... (6 Replies)
Discussion started by: nua7
6 Replies

4. Shell Programming and Scripting

Merge two files line by line and column by column

Hi All, I have two files having oracle query result. I want to merge to files line by line and also with column File1 23577|SYNC TYPE 23578|Order Number|ConnectionState 23585|Service State|Service NameFile2 23577|AR Alarm Sync 23578|A5499|9 23585|7|test_nov7Result... (18 Replies)
Discussion started by: Harshal22
18 Replies

5. Shell Programming and Scripting

merge same pattern of same column in one line

Hello, I have some problem in the modified shell script. I would like to merge the same word in column 1 in one line. Example : A1 B1 2 A2 B1 4 A3 B1 7 A1 B2 1 A2 B2 10 A3 B2 8 Expected output : A1 B1 B2 2 1 A2 B1 B2 4 10 A3 ... (6 Replies)
Discussion started by: awil
6 Replies

6. Shell Programming and Scripting

compare files and remove a line from a file if first column is greater than 25

my files are as follows fileA sepearated by tab /t 00 lieferungen 00 attractiop 01 done 02 forness 03 rasp 04 alwaysisng 04 funny 05 done1 fileB funnymou120112 funnymou234470 mou3raspnhdhv rddfgmoudone1438748 so all those record which are greater than 3 and which are not... (4 Replies)
Discussion started by: rajniman
4 Replies

7. UNIX for Dummies Questions & Answers

loop? print max column in each line for 800 files and merge

Hello, I have 800 or so files with 3 columns each and >10000 lines each. For each file and each line I would like to print the maximum column number for each line. Then I would like to 'paste' each of these files together (column-wise) so that the file with expression in label '_1' is the... (6 Replies)
Discussion started by: peanuts48
6 Replies

8. Shell Programming and Scripting

Shell Scripting: Compare pattern in two files and merge the o/p in one.

one.txt ONS.1287677000.820.log 20Oct2010 ONS.1287677000.123.log 21Oct2010 ONS.1287677000.456.log 22Oct2010 two.txt ONS.1287677000.820.log:V AC CC EN ONS.1287677000.123.log:V AC CC EN ONS.1287677000.820.log:V AC CC EN In file two.txt i have to look for pattern which column one... (17 Replies)
Discussion started by: saluja.deepak
17 Replies

9. UNIX for Dummies Questions & Answers

compare columns from 2 files and merge

Dear all, Being new to Unix i have a problem. I have 2 files: File 1: 118,1,0,2,3,0,5,0.3,0,0.3,0.6,1 118,2,1,2,2,0,5,0.4,0,0.4,0.4,1 118,4,2,0,3,0,5,0.7,0,0.3,0.6,1 118,6,4,1,0,0,5,0.8,0,0.2,0,1 File 2: 118,1,BFGL-NGS-109695,3610326,0,18,1,0.556,0.389,0.056,0.25,0.8183... (2 Replies)
Discussion started by: samwilkinson
2 Replies

10. Shell Programming and Scripting

Compare two files and merge columns in a third

Hi, I'm working with snmp, with a little script I'm able to obtain from a switch a list with a couple of values with this format Port Mac 1 00:0A:0B:0C:0D:0E .... (hundred of entries) Now with a simple arp on a router I am able to obtain another list 00:0A:0B:0C:0D:0E... (20 Replies)
Discussion started by: CM64
20 Replies
Login or Register to Ask a Question