Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Count and search by sequence in multiple fasta file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Count and search by sequence in multiple fasta file
# 1  
Old 03-10-2014
[Solved] Count and search by sequence in multiple fasta file
Hello,

I have 10 fasta files with sequenced reads information with read sizes from 15 - 35 . I have combined the reads and collapsed in to unique reads and filtered for sizes 18 - 26 bp long unique reads. Now i wanted to count each unique read appearance in all the fasta files and make a table with sample names as columns and reads as rows. I tried to use "grep -w "sequence name" file name " to count the tags but this seems to take long time. does anyone know how to do this faster?
# 2  
Old 03-10-2014
Post example input and example output of what you want.
# 3  
Old 03-10-2014
Sorry for the confusion. Here are the input files and required output

This is the input file where it contains unique sequences. i have more than million such unique sequences.
Query:
Code:
>tag1
TCGGA
>tag2
TCTCA
>tag3
TCTCGC

These are multiple files. for example i am showing with 3 files. i have more than 20 such files. each file contains more than 10 million sequences each
File1:
Code:
>file1_id1
TCGGA
>file1_id1
TCGGAT
>file1_id2
TCTCA
>file1_id3
TCTCA

File2:
Code:
>file2_id1
TCTCA
>file2_id2
TCTCA
>file2_id3
TCTCACTA
>file2_id4
TCTCGC
>file2_id5
TCTCGCCTAT
>file2_id6
TCTCGC

File3:
Code:
>file1_id1
TCGGA
>file1_id1
TCGGAT
>file2_id4
TCTCGC
>file2_id5
TCTCGCCTAT
>file2_id6
TCTCGC

I need the following output. Search has to be exact for the count.
output:
Code:
		sequence	file1	file2	file3
tag1	TCGGA		1		0		1
tag2	TCTCA		2		2		0		
tag3	TCTCGC		0		2		2

Moderator's Comments:
Mod Comment Please use CODE tags to mark sample code, sample input, and sample output; not your entire message.
Last edited by Don Cragun; 03-11-2014 at 03:17 AM.. Reason: Fix CODE tags.
# 4  
Old 03-11-2014
Without seeing your code, it is impossible to guess at whether or not something else is faster. You could try something like:
Code:
awk '
FNR == NR {
	if($1 ~ /^>/)
		t[++tn] = substr($1, 2)
	else {	s[$1] = tn
		seq[tn] = $1
	}
	next
}
FNR == 1 {
	fn++
}
$1 in s {
	tc[s[$1], fn]++
}
END {	# Print header:
	printf("\t\tsequence")
	for(i = 1; i <= fn; i++)
		printf("\tfile%d", i)
	printf("\n")
	# Print tag data:
	for(i = 1; i <= tn; i++) {
		printf("%s\t%s", t[i], seq[i])
		for(j = 1; j <= fn; j++)
			printf("\t\t%d", tc[i, j])
		printf("\n")
	}
}' Query File1 File2 File3

which with your sample input files produces:
Code:
		sequence	file1	file2	file3
tag1	TCGGA		1		0		1
tag2	TCTCA		2		2		0
tag3	TCTCGC		0		2		2

# 5  
Old 03-11-2014
a bash approach
Code:
#!/bin/bash

query=query.txt
file_list=(*.txt)
declare -A pattern

printf "%-10s\t%-10s\t" "" "Sequence"
for file in ${file_list[@]}
do
  printf "%-10s\t" $file
  while read line
  do
    [[ "$line" =~ ">" ]] && continue
    ((pattern[$file,$line]+=1))
  done < $file
done
echo

while read line
do
  [[ "$line" =~ ">" ]] && printf "%-10s\t" ${line/>/} && continue
  printf "%-10s\t" $line
  for file in ${file_list[@]}
  do
    printf "%-10s\t" ${pattern[$file,$line]:-0}
  done
  echo
done < $query

# 6  
Old 03-13-2014
Thank you , Don. This works really well. easy to run and easy to understand as well.

---------- Post updated at 04:17 PM ---------- Previous update was at 04:16 PM ----------

ahamed, thank you for your response but don's version was faster and finished my query in less time. Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to find a specific sequence pattern in a fasta file?

I have to mine the following sequence pattern from a large fasta file namely gene.fasta (contains multiple fasta sequences) along with the flanking sequences of 5 bases at starting position and ending position, AAGCZ-N16-AAGCZ Z represents A, C or G (Except T) N16 represents any of the four... (3 Replies)
Discussion started by: dineshkumarsrk
3 Replies

2. UNIX for Beginners Questions & Answers

How to count the length of fasta sequences?

I could calculate the length of entire fasta sequences by following command, awk '/^>/{if (l!="") print l; print; l=0; next}{l+=length($0)}END{print l}' unique.fasta But, I need to calculate the length of a particular fasta sequence specified/listed in another txt file. The results to to be... (14 Replies)
Discussion started by: dineshkumarsrk
14 Replies

3. Shell Programming and Scripting

Getting unique sequences from multiple fasta file

Hi, I have a fasta file with multiple sequences. How can i get only unique sequences from the file. For example my_file.fasta >seq1 TCTCAAAGAAAGCTGTGCTGCATACTGTACAAAACTTTGTCTGGAGAGATGGAGAATCTCATTGACTTTACAGGTGTGGACGGTCTTCAGAGATGGCTCAAGCTAACATTCCCTGACACACCTATAGGGAAAGAGCTAAC >seq2... (3 Replies)
Discussion started by: Ibk
3 Replies

4. Shell Programming and Scripting

To search duplicate sequence in file

Hi, I want to search only duplicate sequence number in file e.g 4757610 4757610 should display only duplicate sequence number in file. file contain is: 4757610 6zE:EXPNL ORDER_PRIORITY='30600022004757610' ORDER_IDENTIFIER='4257771056' MM_ASK_VOLUME='273' MM_ASK_PRICE='1033.0000' m='GBX'... (5 Replies)
Discussion started by: ashfaque
5 Replies

5. Shell Programming and Scripting

Extract sequence from fasta file

Hi, I want to match the sequence id (sub-string of line starting with '>' and extract the information upto next '>' line ). Please help . input > fefrwefrwef X900 AGAGGGAATTGG AGGGGCCTGGAG GGTTCTCTTC > fefrwefrwef X932 AGAGGGAATTGG AGGAGGTGGAG GGTTCTCTTC > fefrwefrwef X937... (2 Replies)
Discussion started by: ritakadm
2 Replies

6. UNIX for Dummies Questions & Answers

Change sequence names in fasta file

I have fasta files with multiple sequences in each. I need to change the sequence name headers from: >accD:_59176-60699 ATGGAAAAGTGGAGGATTTATTCGTTTCAGAAGGAGTTCGAACGCA >atpA_(reverse_strand):_showing_revcomp_of_10525-12048 ATGGTAACCATTCAAGCCGACGAAATTAGTAATCTTATCCGGGAAC... (2 Replies)
Discussion started by: tyrianthinae
2 Replies

7. UNIX for Dummies Questions & Answers

How to change sequence name in along fasta file?

Hi I have an alignment file (.fasta) with ~80 sequences. They look like this- >JV101.contig00066(+):25302-42404|sequence_index=0|block_index=4|species=JV101|JV101_4_0 GAGGTTAATTATCGATAACGTTTAATTAAAGTGTTTAGGTGTCATAATTT TAAATGACGATTTCTCATTACCATACACCTAAATTATCATCAATCTGAAT... (2 Replies)
Discussion started by: baika
2 Replies

8. UNIX for Dummies Questions & Answers

Breaking a fasta formatted file into multiple files containing each gene separately

Hey, I've been trying to break a massive fasta formatted file into files containing each gene separately. Could anyone help me? I've tried to use the following code but i've recieved errors every time: for i in *.rtf.out do awk '/^>/{f=++d".fasta"} {print > $i.out}' $i done (1 Reply)
Discussion started by: Ann Mc Cartney
1 Replies

9. Shell Programming and Scripting

Parsing a fasta sequence with start and end coordinates

Hi.. I have a seperate chromosome sequences and i wanted to parse some regions of chromosome based on start site and end site.. how can i achieve this? For Example Chr 1 is in following format I need regions from 2 - 10 should give me AATTCCAAA and in a similar way 15- 25 should give... (8 Replies)
Discussion started by: empyrean
8 Replies

10. Shell Programming and Scripting

Search and find total count from multiple files

Please advice how can we search for a string say (abc) in multiple files and to get total occurrence of that searched string. (Need number of records that exits in period of time). File look like this (read as filename.yyyymmdd) a.20100101 b.20100108 c.20100115 d.20100122 e.20100129... (2 Replies)
Discussion started by: zooby
2 Replies
Login or Register to Ask a Question