Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Check for data between two files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Check for data between two files
# 1  
Old 06-13-2014
Check for data between two files
I have two files

Code:
 
File1.txt
 
000199458
000199463
000200442
000200831
000200866
000201009
000201050
000201405
000201666
000201682

Code:
 
File2.txt
 

DHP5104   1BWEL9                             000199458
DHP5104   1BWELD                             000199463
DHP5104   1BWEK3                             000200442
DHP5104   1BWEK3                             000200831
DHP5104   1C1FZD                             000200866
DHP5104   1BWEK3                             000202405
DHP5104   1C1FXX                             000202707
DHP5104   1C1FZD                             000203202
DHP5104   1BWEL9                             000204450
DHP5104   1BWEL9                             000204453
XVP5104   1BWEL9                             000199458

I want to check each row in File1.txt with file2.txt. If match is found then the record needs to be deleted in file2.txt

Code:
 
Output.txt
DHP5104   1BWEK3                             000202405
DHP5104   1C1FXX                             000202707
DHP5104   1C1FZD                             000203202
DHP5104   1BWEL9                             000204450
DHP5104   1BWEL9                             000204453

If we had a particular string that we need to find then this can be achieved using below command

Code:
grep -v 000199458 File2.txt > Output.txt

But in this case I want to check each row from File1.txt with File2.txt and want to remove all those that do not exist.

Any help would be appreciated.

Thanks!!
# 2  
Old 06-13-2014
Code:
awk 'FNR==NR {f1[$1];next} !($3 in f1)' file1.txt file2.txt

This User Gave Thanks to vgersh99 For This Post:
halfafringe
# 3  
Old 06-13-2014
Quote:
Originally Posted by vgersh99
Code:
awk 'FNR==NR {f1[$1];next} !($3 in f1)' file1.txt file2.txt


Thanks for the reply. Can you tell me what $3 is ? Is it the 3rd column ?
# 4  
Old 06-13-2014
Quote:
Originally Posted by halfafringe
Thanks for the reply. Can you tell me what $3 is ? Is it the 3rd column ?
yes
This User Gave Thanks to vgersh99 For This Post:
halfafringe
# 5  
Old 06-13-2014
If the values of file1 can only appear in the third column, you could try with:-
Code:
grep -vf file1  file2 > file2.tmp
mv file2.tmp  file2

This will get all records from file2 that do not match an entry in file1. Be careful because a null line in file1 will match every line in file2, so you will get no output.


Does that help?


Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

In PErl script: need to read the data one file and generate multiple files based on the data

We have the data looks like below in a log file. I want to generat files based on the string between two hash(#) symbol like below Source: #ext1#test1.tale2 drop #ext1#test11.tale21 drop #ext1#test123.tale21 drop #ext2#test1.tale21 drop #ext2#test12.tale21 drop #ext3#test11.tale21 drop... (5 Replies)
Discussion started by: Sanjeev G
5 Replies

2. Shell Programming and Scripting

Combine data from two files base on uniq data

File 1 ID Name Po1 Po2 DD134 DD134_4A_1 NN-1 L_0_1 DD134 DD134_4B_1 NN-2 L_1_1 DD134 DD134_4C_1 NN-3 L_2_1 DD142 DD142_4A_1 NN-1 L_0_1 DD142 DD142_4B_1 NN-2 L_1_1 DD142 DD142_4C_1 NN-3 L_2_1 DD142 DD142_3A_1 NN-41 L_3_1 DD142 DD142_3A_1 NN-42 L_3_2 File 2 ( Combination of... (1 Reply)
Discussion started by: pareshkp
1 Replies

3. Shell Programming and Scripting

Check data in mysql

Hi, I just would like to write a perl script , this perl script do the following . Check the parameter $data in a mysql database ( with user ID and password ) , if the $data is exist then return true , if not then return false , would advise how to write this script ? thanks ---------- Post... (1 Reply)
Discussion started by: ust3
1 Replies

4. Shell Programming and Scripting

data integrity check needed

Hi friends I need copied 100gd of data to other Solaris server. Could anyone help me guiding appropriate way of checking data integrity at source and destination so can I delete the data at source location . How can print/check cksum of individual file in each folder and match it with... (7 Replies)
Discussion started by: hk_kamozalwar
7 Replies

5. Shell Programming and Scripting

Request to check: compare two files , match same entries, write data before it

Hi all, I have 2 files:Column1 of first file has to be matched with column 3 of second file first file contain DATA like this in 2 columns one with gene name second with whether CAD,HT,RA T2Dor any one column 1 column2 ARFGEF2 CAD DDEF2 CAD PSCD3 CAD PSCD4 CAD CAMK1... (5 Replies)
Discussion started by: manigrover
5 Replies

6. Shell Programming and Scripting

Perl code to check date and check files in particular dir

Hi Experts, I am checking how to get day in Perl. If it is “Monday” I need to process…below is the pseudo code. Can you please prove the code for below condition. if (today=="Monday" ) { while (current_time LESS THAN 9:01 AM) ... (1 Reply)
Discussion started by: ajaypatil_am
1 Replies

7. UNIX for Dummies Questions & Answers

How to check hardware data compression

Hi, I'm looking to check the compression ratio applied by hardware to work out how much free space we have on tapes. mt -f /dev/rmt0 status shows: If I add up the output of the du command and what we backup from the script, it gives me the total space we tape up. So the plan was to find the... (4 Replies)
Discussion started by: rab
4 Replies

8. Shell Programming and Scripting

how to check availability of data of one file in other

hi all i have two files in unix 1.table.name 2.all.tables i need to check the availability of all data of table.name in all.tables.both the table contains n number of data.i need a script which will check one by one whether the data exist in table.name or not.I want this in SH. (1 Reply)
Discussion started by: alisha
1 Replies

9. Shell Programming and Scripting

Check data and time

I am attempting to figure out how to do a time check within my script. For some reason I can not seem to get this to work correctly. I want the script to first see if it is Saturday. If it is Saturday then check to see if it's between the time 5:30am and 6:30am. If it is between 5:30am and... (4 Replies)
Discussion started by: LRoberts
4 Replies

10. Linux

check written data

I'm using growisofs to write DVD, e.g. $ growisofs -Z /dev/dvd -V "Personal Data, `date +"%b, %d %Y"`" -R -J /mnt/d/* How can I test whether data was written correctly? md5sum or so? (0 Replies)
Discussion started by: Hitori
0 Replies
Login or Register to Ask a Question