Search and filter between two files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Search and filter between two files
# 1  
Old 01-03-2011
Search and filter between two files

Hello,

I have two files in this form that consist of three columns, a name (L*contig*), the length (length=**) and the sequence

LT_file.txt
HTML Code:
LTcontig1   length=13 acccatgctttta
LTcontig5   length=8 ggattacc
LTcontig8   length=20 ccattgaccgtacctgatcg
LTcontig23   length=5 accta
and LU_file.txt

HTML Code:
LUcontig1   length=15 acccatgcttttacc
LUcontig18   length=13 acccatgctttta
LUcontig16   length=25 ccattgaccgtacctgatcgaaaat
I want to compare every sequence in the first file with every sequence in the second file in order to find matches and I want to write both sequences in another file.

For example, in this example, LTcontig8 is included in LUcontig16 since

LTcontig8 length=20 ccattgaccgtacctgatcg
LUcontig16 length=25 ccattgaccgtacctgatcgaaaat

so I want both of these lines in a separate file.

Thank you in advance,
Felipe
# 2  
Old 01-03-2011
Code:
awk '{print $NF}' LT_file.txt | grep -f - LU_file.txt

Code:
awk '{print $NF}' LT_file.txt | while read a
do
grep $a LU_file.txt && grep $a LT_file.txt
done

This User Gave Thanks to ctsgnb For This Post:
# 3  
Old 01-03-2011
Thank you very much, it works fine!
So you used "a" as a temporary name for the last column of the file you read with awk?
# 4  
Old 01-03-2011
Quote:
Originally Posted by FelipeAd
Thank you very much, it works fine!
So you used "a" as a temporary name for the last column of the file you read with awk?
Yep,
For every line awk takes the last field and give it to the pipe.
for every entry that is passed to the pipe i read it and store it in the "a" variable (of course you could have chosen another name for it).
# 5  
Old 01-03-2011
Quote:
Originally Posted by ctsgnb
Code:
awk '{print $NF}' LT_file.txt | grep -f - LU_file.txt

Code:
awk '{print $NF}' LT_file.txt | while read a
do
grep $a LU_file.txt && grep $a LT_file.txt
done

Also, I have a huge question about the while-loops.
In the code you gave me, if I want to fine how many matches it finds, I tried having an increasing variable inside my loop, but this does not work. I wrote this:


Code:
let h=0;awk '{print $NF}' LT_file.txt | while read a
do
grep $a LU_file.txt && grep $a LT_file.txt
h=$h+1
done

but the variable (h) does not grow inside the loop...
# 6  
Old 01-04-2011
maybe you can try
Code:
let h=0;awk '{print $NF}' LT_file.txt | while read a
do
grep $a LU_file.txt && grep $a LT_file.txt && h=$(($h+1))
done


Last edited by ctsgnb; 01-04-2011 at 04:25 AM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Filter using awk in CSV files

Hello Gentlemen, Finding difficulties to play with my Input files:confused: . Your guidance will certainly help as always. After converting to csv file from XLSM file, I am getting some extra ""(double quote) characters which I want to terminate inside shell script and process it further. ... (6 Replies)
Discussion started by: pradyumnajpn10
6 Replies

2. UNIX for Dummies Questions & Answers

Filter and merge 2 files problem

Hi, I'm trying to combine two files which have 1 column in common and filter out rows I don't need. File 1: ID Start End Matched Coverage 1 1 254 1515 5.96 2 1 135 402 2.98 File 2 (has 2 rows per entry): >1... (4 Replies)
Discussion started by: Yarinka
4 Replies

3. UNIX for Dummies Questions & Answers

Filter lines common in two files

Thanks everyone. I got that problem solved. I require one more help here. (Yes, UNIX definitely seems to be fun and useful, and I WILL eventually learn it for myself. But I am now on a different project and don't really have time to go through all the basics. So, I will really appreciate some... (6 Replies)
Discussion started by: latsyrc
6 Replies

4. Shell Programming and Scripting

Copying the files after filter

Hi Guys, i want copy the all files another direcotry after filtering the command. and tried as like below...it's not working. ls -ltr|awk '{print $9}'|grep "images\|\.htm"|cp *.* /home/oracle Thanks (13 Replies)
Discussion started by: bmk
13 Replies

5. Shell Programming and Scripting

how to filter files with given format

Hi, all, I have files like: nameserver 216.66.22.2 ; tserv1.ash1.ipv6.he.net. tserv13.ash1.ipv6.he.net. nameserver 216.66.38.58 ; tserv1.tor1.ipv6.he.net. tserv21.tor1.ipv6.he.net. nameserver 216.218.221.6 ;... (3 Replies)
Discussion started by: esolvepolito
3 Replies

6. SuSE

Search all files based on first and in all listed files search the second patterns

Hello Linux Masters, I am not a linux expert therefore i need help from linux gurus. Well i have a requirement where i need to search all files based on first patterns and after seraching all files then serach second pattern in all files which i have extracted based on first pattern.... (1 Reply)
Discussion started by: Black-Linux
1 Replies

7. Shell Programming and Scripting

Search and filter by TAG

Hello all, searching on a text file (log file) is quite simple: grep -i texttosearch filename | grep somethingWhat I'm trying to do is filter the result by TAG and remove the double entries. Like if the log file contains the following text (fields are separated by commas): ... (18 Replies)
Discussion started by: Lord Spectre
18 Replies

8. Shell Programming and Scripting

Filter files and print

Hi, I need to filter and store the files ends with log extension in the array and need to write the file names in the array to a file. I need to use array to derive this solution. Please help me out. Thanks (2 Replies)
Discussion started by: Sekar1
2 Replies

9. Shell Programming and Scripting

Filter only gz files from list of subdirectories

Hi, I have a very big directory structure that consists of many sub-directories inside.There are around 50 ".gz" files under this dir structure. I want to copy all the gz files alone to a seperate location. Plz help me. (2 Replies)
Discussion started by: villain41
2 Replies

10. UNIX for Dummies Questions & Answers

ldap search filter

Hi, I am trying to do an ldapsearch with a filter that checks the uid and the userpassword: $filter= "(&(uid=$user) (userpassword=$password)"; $objs = $ldap->search( base => $basedn, filter => "($filter)"); i based it on this example i found on CPAN: $mesg = $ldap->search( ... (2 Replies)
Discussion started by: tine
2 Replies
Login or Register to Ask a Question