Compare two file & output


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare two file & output
# 1  
Old 03-10-2009
Compare two file & output

Hi every body

i have a problem need help urgently

file 1 (approx 200K entries)

aaaaa
bbbb
cccccc
dddd
ffff

file 2 (approx 2 million entries)

aaaaa,1,ee,44,5t,6y,
bbbb,3,ff,66,5u,8r,
cccccc, .....
dddd, .....
eeeeee, .....
ffff, ......
ggg, .......

i wanna compare two file
if entry in file 1 exist in file 2 , output show full row content from file 2.

result =

aaaaa,1,ee,44,5t,6y,
bbbb,3,ff,66,5u,8r,
cccccc, .....
dddd, .....
ffff, ......

++++++++++++++++++++++++++++++++++++++++++++

Thanks in advance

The_Archer
# 2  
Old 03-10-2009
Assuming an entry in file1 is to be matched against the first field in file2:
Code:
nawk -F, -v OFS=',' 'FNR==NR {f1[$1]; next} $1 in f1' file1 file2

# 3  
Old 03-10-2009
Code:
awk -F, ' { if (FILENAME=="file1") {arr[$0]=1}
          if(FILENAME=="file2" )
           {
                for(i=1; i<=NF; i++) { if ($i in arr) {print $0; next}}
           }
          ' file1 file2


Last edited by vgersh99; 03-10-2009 at 02:23 PM.. Reason: code tages
# 4  
Old 03-10-2009
Grep example

#!/bin/sh
for name in `cat file1`
do
grep $name file2 >> file_results.txt
done
# 5  
Old 03-10-2009
Code:
fgrep -f file1 file2

# 6  
Old 03-10-2009
THANKS A LOT
You Guys Made my day

one thing to ask

which process is fast
as my file 2 has 2 million rows
& in each row approx 200 comma seperated field

i will run it on V890 with 16 G.

just for information
i have tried grep option before but it was tooooooo slow even on V890

What you Guys suggest
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script (sh file) logic to compare contents of one file with another file and output to file

Shell script logic Hi I have 2 input files like with file 1 content as (file1) "BRGTEST-242" a.txt "BRGTEST-240" a.txt "BRGTEST-219" e.txt File 2 contents as fle(2) "BRGTEST-244" a.txt "BRGTEST-244" b.txt "BRGTEST-231" c.txt "BRGTEST-231" d.txt "BRGTEST-221" e.txt I want to get... (22 Replies)
Discussion started by: pottic
22 Replies

2. Shell Programming and Scripting

Compare & print content of file which is not matching

Hi All I want to compare 2 files using awk and get output of content which is not matching I have 2 files a.txt 123 456 780 143 b.txt A|B|C|167|D|E C|K|D|123|D|E A|B|D|789|G|F C|D|G|143|A|B Not matching line from b.txt O/P A|B|C|167|D|E A|B|D|789|G|F (3 Replies)
Discussion started by: aaysa123
3 Replies

3. UNIX for Dummies Questions & Answers

To compare two files,Output into a new file

Hi Please help me to compare two files and output into a new file file1.txt 15114933 |4001 15291649 |933502 15764675 |4316 15764678 |4316 15761974 |282501 15673104 |933505 15673577 |933505 15673098 |933505 15673096 |933505 15673092 |933505 15760705 ... (13 Replies)
Discussion started by: Ankita Talukdar
13 Replies

4. Shell Programming and Scripting

Compare File & Copy Replace if Successful

Hi All, I have written a shell script that creates a backup of my MySQL database. The script performs the following functions: Creates a Backup of the MySQL database Compresses the Backup Copies the Backup to a Remote Server Send an E-Mail displaying the size of the Backup Removes any... (6 Replies)
Discussion started by: SalientAnimal
6 Replies

5. Shell Programming and Scripting

Match list of strings in File A and compare with File B, C and write to a output file in CSV format

Hi Friends, I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving... File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Discussion started by: asnandhakumar
6 Replies

6. Shell Programming and Scripting

Compare two files A & B and accordingly modify file A

Friends, i have two huge complex files (for eg :A & B)as output , the sample contents of the files are as follows : A == ID,DATE,SUM1,SUM2,TOTAL(SUM1+2) A5066,20/04/2010,25000,50000,75000 A5049,20/04/2010,25000,60000,85000 B == ID,DATE,SUM1,SUM2,TOTAL(SUM1+2)... (2 Replies)
Discussion started by: appu2176
2 Replies

7. UNIX for Dummies Questions & Answers

How to compare 2 files & get specific value & replace it in other file.

Hiiii Friends I have 2 files with huge data. I want to compare this 2 files & if they hav same set of vales in specific rows & columns i need to get that value from one file & replace it in other. For example: I have few set data of both files here: a.dat: PDE-W 2009 12 16 5 29 11.11 ... (10 Replies)
Discussion started by: reva
10 Replies

8. Shell Programming and Scripting

Compare & replace contents within a file

I have 2 files file1 1 TMQUEUE QUE1 STMW633A 100 DMADM DOMGRPSTMW633A STMW633A 100 GWADM GWTGRPSTMW633A STMW633A 100 GWADM GWTGRPSTMW633AA STMW633A 100 GWADM GWTGRPSTMW638A STMW638A 100 TMSYSEVT EVTGRPSTMW633A STMW633A 100 TMSYSEVT ... (2 Replies)
Discussion started by: kaustubh137
2 Replies

9. UNIX for Dummies Questions & Answers

Aggregate values in a file & compare with sql output

Hi, I have a file containing the following data: junk123junk723itemcode001qty01price10total10junkjunk junk123junk723itemcode002qty02price10total20junkjunk .. .. .. could be 5000+ lines I have an algo and need a code to implement this: 1. Linecount = wc -l (should give 5000) 2. For i... (1 Reply)
Discussion started by: shiroh_1982
1 Replies

10. Shell Programming and Scripting

File Compare & Move between 2 servers

Greetings - I am a newbie in shell scripts. I have been thru the whole forum but there has been no similar query posed. The objective of my system is to have a unified filebase system. I am using RSync to synchronise files between the location & central server with both of them having the... (4 Replies)
Discussion started by: evolve
4 Replies
Login or Register to Ask a Question