Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

help with comparing fields

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting help with comparing fields
# 1  
Old 10-20-2011
Network help with comparing fields
I have a file having records like below
Code:
Field1;Filed2;Filed3
< '393200103052';'H3G';'20081204'
< '393200103059';'TIM';'20110111'
< '393200103061';'TIM';'20060206'
< '393200103064';'OPI';'20110623'
> '393200103052';'HKG';'20081204'
> '393200103056';'TIM';'20110111'
> '393200103088';'TIM';'20060206'

What i need is take
Code:
if('>firstfiled==''<firstfiled'){
     if(check other fileds){
          print what differece in fileds
     }
}
else{
     print no match found >filed1 with all other fields
}

Advance thanks

Moderator's Comments:
Mod Comment
Please use code tags. Can't be that hard. You got 3 PMs already and this here is a link to a video tutorial. If you collect more infraction points by constantly ignoring that, you will be automatically banned soon. Maybe think about that if it is worth it.
You also had an awk line of code as subject which is not helping that informative as a subject. Yet again, you just ask for help in the subject, but not at least a descriptive word in it with what problem.
Last edited by zaxxon; 10-20-2011 at 05:06 AM.. Reason: code tags, non-descriptive subject again, see PM.
# 2  
Old 10-20-2011
Requirement not clear!

--ahamed
# 3  
Old 10-20-2011
Network
HI ahamed101,
I had two files say file1.txt file2.txt

file1.txt
Code:
'393200103001';'TIM';'20080205'
'393200103025';'OPI';'20041025'
'393200103032';'OPI';'20080218'

file2.txt
Code:
'393200103001';'TIM';'20080205'
'393200103017';'TIM';'20040521'
'393200103032';'ABC';'20080218'

Now i fired the diff command for these two files and stored the difference in file3.txt,the following was the generated file3.txt
Code:
< '393200103025';'OPI';'20041025'
< '393200103032';'OPI';'20080218'
> '393200103017';'TIM';'20040521'
> '393200103032';'ABC';'20080218'

If you look at the file3 it has the records missing from file1 and records missing from file2 and records having common first filed but other fileds are different.
Using this file3.txt i need to generate three files like this
1.missed_file1.txt-should have records missing from file1.
2.missed_file2.txt -should have records missing from file2.
3.common.txt-should have records having common first filed.
sample out of these files should like this

missed_file1.txt
Code:
'393200103017';'TIM';'20040521'

missed_file2.txt
Code:
'393200103025';'OPI';'20041025'

common.txt
For '393200103032 file1:'OPI' file2;'ABC'
i hope now its more clear.

---------- Post updated at 04:20 AM ---------- Previous update was at 04:13 AM ----------

I dont want to use file1.txt and file2.txt to generate other three files, i want to use file3.txt only to generate other three files.
Becuase my file1.txt and file2.txt are very big in size having millions of records.
so its taking much time to genreate files.
I had tried like this functionality is working fine but taking much time with large files because for every record in file1 it has to compare with all the records of file2.
so comparison is n*n times.
If i use file3.txt which was generated using diff command,i think i can optimize time.please help me if possible.
Code:
awk -F \; 'NR==FNR{a[$1];next} !($1 in a)' file1.txt file2.txt > missed_file1.txt

'393200103056';'TIM';'20110111'
'393200103088';'TIM';'20060206'

awk -F \; 'NR==FNR{a[$1];next} !($1 in a)' file2.txt file1.txt > missed_file2.txt

'393200103059';'TIM';'20110111'
'393200103061';'TIM';'20060206'
'393200103064';'OPI';'20110623'

awk -F \; 'NR==FNR{a[$1]=$2;next} $1 in a && a[$1]!=$2{print $1, $2,a[$1]} '  file1.txt file2.txt  > common.txt

'393200103052' 'H3G' 'HKG'

Last edited by zaxxon; 10-20-2011 at 06:24 AM.. Reason: code tags
# 4  
Old 10-20-2011
Check the comm command (yes, you'll need to sort the files before using that command):

Code:
bash-4.1$ head file[12]
==> file1 <==
'393200103001';'TIM';'20080205'
'393200103025';'OPI';'20041025'
'393200103032';'OPI';'20080218'

==> file2 <==
'393200103001';'TIM';'20080205'
'393200103017';'TIM';'20040521'
'393200103032';'ABC';'20080218'

Code:
bash-4.1$ comm -13 <(sort file1) <(sort file2)
'393200103017';'TIM';'20040521'
'393200103032';'ABC';'20080218'
bash-4.1$ comm -23 <(sort file1) <(sort file2)
'393200103025';'OPI';'20041025'
'393200103032';'OPI';'20080218'
bash-4.1$ comm -12 <(sort file1) <(sort file2)
'393200103001';'TIM';'20080205'

# 5  
Old 10-20-2011
Bug i think you were not clear with requirement
i think you were not clear with requirement
# 6  
Old 10-21-2011
Shruthi,

You got the diff command working wrong.
It will not display the common lines in file1 and file2
It does a line by line comparison

This is the diff output
Code:
 2,3c2,3  #this means in file1 lines 2-3 have changed compared to file2 in the same line range
#Following lines are present in file1
< '393200103025';'OPI';'20041025'
< '393200103032';'OPI';'20080218'
---
#compared to the lines in file2
> '393200103017';'TIM';'20040521'
> '393200103032';'ABC';'20080218'

You can try radoulov suggestion itself...
Something like...Instead of running the diff command you run the comm command once
Code:
comm <(sort file1) <(sort file2) > output
 
 
awk '/^\t\t/{print $1> "comm.txt";next} 
/^\t/{print $1> "missing_file1.txt";next}
{print $1> "missing_fil2.txt"} ' output

You will have the following files
Code:
comm.txt #Common in both the files
missing_file1.txt #Contents missing in file1
missing_file2.txt #Contents missing in file2

--ahamed
Last edited by ahamed101; 10-22-2011 at 07:34 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Need urgent help in comparing two fields in two files

Hi all, I have two files as below. I need to compare field 2 of file 1 against field 1 of file 2 and field 5 of file 1 against filed 2 of file 2. If both matches , then create a result file 1 with first file data and if not matches , then create file with first fie data. Please help me in... (1 Reply)
Discussion started by: sivarajb
1 Replies

2. Shell Programming and Scripting

Comparing two files using four fields

Dear All, I want to compare File1 and File2 (Separated by spaces) using four fields (Column 1,2,4,5). Logic: If column 1 and 2 of File1 and File2 match exactly and if the File2 has the same characters as any of the characters present in column 4 and 5 of file1 then those lines of file1 and file2... (6 Replies)
Discussion started by: NamS
6 Replies

3. Shell Programming and Scripting

Comparing two files using four fields

I want to compare File1 and File2 (Separated by spaces) using four fields (Column 1,2,4,5). Logic: If column 1 and 2 of File1 and File2 match exactly and if the File2 has the same characters as any of the characters present in column 4 and 5 of file1 then those lines of file1 and file2 are... (1 Reply)
Discussion started by: NamS
1 Replies

4. Shell Programming and Scripting

Join fields comparing 4 fields using awk

Hi All, I am looking for an awk script to do the following Join the fields together only if the first 4 fields are same. Can it be done with join function in awk?? a,b,c,d,8,,, a,b,c,d,,7,, a,b,c,d,,,9, a,b,p,e,8,,, a.b,p,e,,9,, a,b,p,z,,,,9 a,b,p,z,,8,, desired output: ... (1 Reply)
Discussion started by: aksijain
1 Replies

5. Shell Programming and Scripting

comparing two files for matching fields

I am newbie to unix and would please like some help to solve the task below I have two files, file_a.text and file_b.text that I want to evaluate. file_a.text 1698.74 1711.88 6576.25 899.41 3205.63 4187.98 697.35 1551.83 ... (3 Replies)
Discussion started by: gameli
3 Replies

6. Programming

comparing two fields from two different files in AWK

Hi, I have two files formatted as following: File 1: (user_num_ID , realID) (the NR here is 41671) 1 cust_034_60 2 cust_80_91 3 cust_406_4 .. .. File 2: (realID , clusterNumber) (total NR here is 1000) cust_034_60 2 cust_406_4 3 .. .. (11 Replies)
Discussion started by: amarn
11 Replies

7. Shell Programming and Scripting

Comparing two files and inserting new fields

Hi all, I searched the forum and tried to learn from the similar posts. However, I am new and I need to get help on this. I hope an expert kindly help me to sort this out. I need to compare field 1 and 2 of the first file with the same fields of the second file and if both fields matches... (9 Replies)
Discussion started by: GoldenFire
9 Replies

8. Shell Programming and Scripting

Comparing fields in two files

Hi, i want to compare two files by one field say $3 in file1 needs to compare with $2 in file2. sample file1 - reqd_charge_code 2263881188,24570896,439 2263881964,24339077,439 2263883220,22619162,228 2263884224,24631840,442 2263884246,22612161,442 sample file2 - rg_j ... (2 Replies)
Discussion started by: raghavendra.cse
2 Replies

9. Shell Programming and Scripting

Comparing two files and replacing fields

I have two files with ids and email addresses. File 2 cotains a subset of the records in file 1. The key field is the first field containing the id. file 1: 123|myadr@abc.com 456|myadr2@abc.com 789|myadr3@abc.com file 2: 456|adr456@xyz.com Where the record appears in the second... (3 Replies)
Discussion started by: tltroy
3 Replies

10. Shell Programming and Scripting

Merging two files by comparing three fields

Hi Experts, I need your timely help. I have a problem with merging two files. Here my situation : Here I have to compare first three fields from FILE1 with FILE2. If they are equal, I have to append the remaining values from FILE2 with FILE1 to create the output. FILE1: Class ... (3 Replies)
Discussion started by: Hunter85
3 Replies
Login or Register to Ask a Question