Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

script to compare two columns in a file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting script to compare two columns in a file
# 1  
Old 08-25-2011
script to compare two columns in a file
Dear everyone,
I need any sort of shell script or perl script would do the following. I have a txt file as follows:
;Stretnumber Resident Resdient (not in file)
Code:
   16  John        Mary       
   16  Mary        Parker
   16  Nancy       Smith
   16  Mary        John
   18  Trey        Steve
   20  Mandy       Greg
   22  Mary        John

I want to remove duplicate entries i.e., line 1 (John Mary) and line 4 (Mary John). Line 7 also has Mary John, but a different street. So I want the file to be printed as:
Code:
   16  John        Mary       
   16  Mary        Parker
   16  Nancy       Smith
   18  Trey        Steve
   20  Mandy       Greg
   22  Mary        John

Thanks for help!
Last edited by Franklin52; 08-25-2011 at 05:55 AM.. Reason: Please use code tags for code and data samples, thank you
# 2  
Old 08-25-2011
Try this:
Code:
awk 'a[$0] || a[$1$3]==$2{next} {a[$0]=$0; a[$1$2]=$3}1' file

This User Gave Thanks to Franklin52 For This Post:
sasharma
# 3  
Old 08-26-2011
Quote:
Originally Posted by Franklin52
Try this:
Code:
awk 'a[$0] || a[$1$3]==$2{next} {a[$0]=$0; a[$1$2]=$3}1' file

Thanks for the above post but sorry, it fails to work if I have following file data:
Code:
   16  John        Mary       
   16  Mary        Parker
   16  Nancy       Smith
   16  John        Macey      
   16  Mary        John
   18  Trey        Steve
   20  Mandy       Greg
   22  Mary        John

Here the input thas two John but with different partners. Only 5th line should be deleted. Instead the output from above code still is in output.
Code:
  16  John        Mary      
   16  Mary        Parker
   16  Nancy       Smith
   16  John        Macey     
   16  Mary        John
   18  Trey        Steve
   20  Mandy       Greg
   22  Mary        John

Any help. Thanks.
# 4  
Old 08-26-2011
Code:
awk 'a[$0] || a[$1$3]==$2{next} {a[$0]=$0; a[$1$2]=$3}1' file

# 5  
Old 08-26-2011
Hi Franklin thanks a lot, but it does not work for me.
regards,
# 6  
Old 08-26-2011
Code:
$ perl -lane '$k1="$F[0] $F[1] $F[2]";$k2="$F[0] $F[2] $F[1]";$h{$k1}+=0 if !(exists $h{$k1} || exists $h{$k2}) }{print foreach keys %h' inputfile

This User Gave Thanks to getmmg For This Post:
sasharma
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Awk: compare values in two columns of the same file

I'm trying to learn awk, but I've hit a roadblock with this problem. I have a hierarchy stored in a file with 3 columns: id name parentID 4 D 2 2 B 1 3 C 1 1 A 5 I need to check if there are any values in column 3 that are not represented anywhere in column 1. I've tried this: awk '{arr;}... (7 Replies)
Discussion started by: kaktus
7 Replies

2. Shell Programming and Scripting

Compare 2 columns from the same file and print a value depending on the result

Hello Unix gurus, I have a file with this format (example values): label1 1 0 label2 1 0 label3 0.4 0.6 label4 0.5 0.5 label5 0.1 0.9 label6 0.9 0.1 in which: column 1 is a row label column 2 and 3 are values I would like to do a simple operation on this table and get the... (8 Replies)
Discussion started by: ksennin
8 Replies

3. Shell Programming and Scripting

Compare columns in a single file

i have the following files (all separated by tabs): file 1.txt 1 yes 2 no 3 yes 4 yes file 2.txt a no b no c yes d no i combine the above files in file 3 which looks like file 3.txt 1 yes a no 2 no b no 3 yes c yes 4 yes d no now, i need to compare the values between column 2... (3 Replies)
Discussion started by: msonoth
3 Replies

4. Shell Programming and Scripting

Compare two date columns in same file

Hi All, Need to compare two date columns from the filname FinalDate.txt. My data's are like below D_OT_START D_EXP_STR Amount 1/3/2012 1/3/2012 5000 6/21/2011 6/25/2011 6000 2/28/2011 2/28/2011 7000 7/16/2010 8/16/2010 8000 7/14/2010 10/26/2010 9000 ... (3 Replies)
Discussion started by: suresh_target
3 Replies

5. Shell Programming and Scripting

Compare Multiple Columns in one file

Hello guys, I am quite new to Shell Scripting and I need help for this I have a CSV file like this: Requisition,Order,RequisitionLineNumber,OrderLineNumber REQ1,Order1,1,1 REQ1,Order1,1,3 REQ2,Order2,1,5 Basically what I want to do is compare the first 3 fields If all 3 fields are the same... (5 Replies)
Discussion started by: jeffreybsu
5 Replies

6. UNIX for Dummies Questions & Answers

To compare first two columns in an excel file

Hi All, i have a excel sheet with two columns as below. column1 column2 100 100 200 300 300 400 400 400 500 600 i need to compare the values these two columns and the output should be printed in the third column...if these values are equal the output should be green and if these... (2 Replies)
Discussion started by: arunmanas
2 Replies

7. Shell Programming and Scripting

Compare data and columns script

Hello all, I have below SQLs to compare data between 2 identical tables.in my database. Can any body help me to convert this into a script db2 "select *from TAB1 where PK IN (select PK from TAB2) order by BEDG_NR". Note: These SQL should run for any given 2 tables as input TAB1... (4 Replies)
Discussion started by: kanakaraju
4 Replies

8. Shell Programming and Scripting

compare two columns of different files and print the matching second file..

Hi, I have two tab separated files; file1: S.No ddi fi cu o/l t+ t- 1 0.5 0.6 o 0.1 0.2 2 0.2 0.3 l 0.3 0.4 3 0.5 0.8 l 0.1 0.6 ... (5 Replies)
Discussion started by: vasanth.vadalur
5 Replies

9. Shell Programming and Scripting

Compare selected columns from a file and print difference

I have learned file comparison from my previous post here. Then, it is comparing the whole line. Now, i have a new problem. I have two files with 3 columns separated with a "|". What i want to do is to compare the second and third column of file 1, and the second and third column of file 2. And... (4 Replies)
Discussion started by: kingpeejay
4 Replies

10. Shell Programming and Scripting

compare file columns

I need help in file comparision. I have two files in below format: FILE_A: ------- COL1 COL2 COL3 COL4 COL5 FILE_B: ------- COL1A COL1B COL1C COL1D COL1E i want to compare for a for each row in FILE_A and FILE_B COL1 of FILE_A with COL1B of FILE_B COL3 of FILE_A with COL1E of... (1 Reply)
Discussion started by: learnoutmore99
1 Replies
Login or Register to Ask a Question