06-17-2010
Deleting sequences based on character frequency
This is what I would like to accomplish, I have an input file (file A) that consist of thousands of sequence elements with the same number of characters (length), each headed by a free text header starting with the chevron ‘>' character followed by the ID (all different IDs with different lenghts) and a number (prevalence). Something like this:
Quote:
> ID 1 Prev 1
C-TGCTAGCTACGTCGTACGT
> ID 2 Prev 31
A-TGCTAGCTACGTCGTACGT
> ID 3 Prev 30
A-TGCTAGCTACGTCGTTCGT
> ID 4 Prev 30
A-TGCTAGCTACGTCGTACGA
> ID 5 Prev 2
A-TGCTAGCTACGTCG-----
> ID 6 Prev 2
A-TGCTAGCTACNTCGTACGT
> ID 7 Prev 2
A-CGCTAGCTACGTCGTACGT
> ID 8 Prev 2
A-TGCTAGCTA-GTCGTACGT
> ID 9 Prev 1
AGTGCTAGCTACGTCGTACGT
I need to calculate the frequency of A,G,C,T,- and N in each position. Then, I need to evaluate the first position in the first sequence and if the frequency of the character in that position does not reach 5% the entire sequence along with the ID should be removed. If the frequency is higher then I need to determine the frequency of the character in the second position so on and so forth till the entire sequence has been scanned. Then, I should do the same thing for each and every sequence in the file.
Thus, in my example above Seq ID 1 should be removed since "C" accounts for only 1% (Prev=1) in that column (the ID is not considered in the analysis). As result of this process, the output file (File B) should only contained Sequences 2, 3 and 4 since the frequency of each character in every position along the entire sequence in the three entries is higher than 5%. The output file should have the same format (FASTA) as the input file:
Quote:
> ID 2 Prev 31
A-TGCTAGCTACGTCGTACGT
> ID 3 Prev 30
A-TGCTAGCTACGTCGTTCGT
> ID 4 Prev 30
A-TGCTAGCTACGTCGTACGA
Any help will be greatly appreciated.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All,
I have a big log file i want to delete all characters (between 350th to 450th characters) starting at 350th character position to 450th character position.
please advice or sample code. (6 Replies)
Discussion started by: rajeshorpu
6 Replies
2. Shell Programming and Scripting
My files look like this
And I need to cut the sequences at the last "A" found in the following 'pattern' -highlighted for easier identification, the pattern is the actual file is not highlighted.
The expected result should look like this
Thus, all the sequences would end with AGCCCTA... (2 Replies)
Discussion started by: Xterra
2 Replies
3. Shell Programming and Scripting
If I have a file with the following information
And I would like to remove all the sequences with Freq less than 3, so I end up having the following file:
I am currently using awk to accomplish this task but I am not getting the results I actually want.
Any help will be greatly appreciated. (3 Replies)
Discussion started by: Xterra
3 Replies
4. Shell Programming and Scripting
My file looks something like this
Wnat I need is to look for the Reference sequence (">Reference1") and based on the length of that sequence trim all the entries in that file. So, the rersulting file will contain all sequences with the same length, like this
Thus, all sequences will keep... (5 Replies)
Discussion started by: Xterra
5 Replies
5. Shell Programming and Scripting
Hi,
I have a file with more than 28000 records and it looks like below..
>mm10_refflat_ABCD range=chr1:1234567-2345678
tgtgcacactacacatgactagtacatgactagac....so on
>mm10_refflat_BCD range=chr1:3234567-4545678...
tgtgcacactacacatgactagtatgtgcacactacacatgactagta
.
.
.
.
.
so on
... (2 Replies)
Discussion started by: Diya123
2 Replies
6. Shell Programming and Scripting
I have two files with thousands of sequences of different lengths. infile1 contains the actual sequences and infile2 the scores for each A, T, G and C in infile1. Something like this:
infile1:
>HZVJKYI01ECH5R
TTGATGTGCCAGCTGCCGTTGGTGTGCCAA
>HZVJKYI01AQWJ8
GGATATGATGATGAACTGGTTTGGCACACC... (4 Replies)
Discussion started by: Xterra
4 Replies
7. Shell Programming and Scripting
I have to remove sequences from a file based on the distance value. I am attaching the file containing the distances (Distance.xls)
The second file looks something like this:
Sequences.txt
>Sample1 Freq 59
ggatatgatgatgaactggt
>Sample1 Freq 54
ggatatgatgttgaactggt
>Sample1 Freq 44... (2 Replies)
Discussion started by: Xterra
2 Replies
8. Shell Programming and Scripting
I have a list of IDs in file1 and a list of sequences in file2. I can print sequences from file2, but I'm asking for help in printing the sequences in the same order as the IDs appear in file1.
file1:
EN_comp12952_c0_seq3:367-1668
ES_comp17168_c1_seq6:1-864
EN_comp13395_c3_seq14:231-1088... (5 Replies)
Discussion started by: pathunkathunk
5 Replies
9. UNIX for Dummies Questions & Answers
I have files with hundreds of sequences with frequency values reported as "Freq X" and missing characters represented by a dash ("-"), something like this
>39sample Freq 4
TAGATGTGCCCGTGGGTTTCCCGTCAACACCGGATAGTAGCAGCACTA
>22sample Freq 15
T-GATGTCGTGGGTTTCCCGTCAACACCGGCAAATAGTAGCAGCACTA... (12 Replies)
Discussion started by: Xterra
12 Replies
10. Shell Programming and Scripting
I have this file:
>ID1
AA
>ID2
TTTTTT
>ID-3
AAAAAAAAA
>ID4
TTTTTTGGAGATCAGTAGCAGATGACAG-GGGGG-TGCACCCC
Add I am trying to use this script to output sequences longer than 15 characters:
sed -r '/^>/N;{/^.{,15}$/d}'
The desire output would be this:
>ID4... (8 Replies)
Discussion started by: Xterra
8 Replies
LEARN ABOUT DEBIAN
uuencode
UUENCODE(5) File Formats Manual UUENCODE(5)
NAME
uuencode - format of an encoded uuencode file
DESCRIPTION
Files output by uuencode(1) consist of a header line, followed by a number of body lines, and a trailer line. The uudecode(1) command will
ignore any lines preceding the header or following the trailer. Lines preceding a header must not, of course, look like a header.
The header line is distinguished by having the first 6 characters begin The word begin is followed by a mode (in octal), and a string
which names the remote file. A space separates the three items in the header line.
The body consists of a number of lines, each at most 62 characters long (including the trailing newline). These consist of a character
count, followed by encoded characters, followed by a newline. The character count is a single printing character, and represents an inte-
ger, the number of bytes the rest of the line represents. Such integers are always in the range from 0 to 63 and can be determined by sub-
tracting the character space (octal 40) from the character.
Groups of 3 bytes are stored in 4 characters, 6 bits per character. All are offset by a space to make the characters printing. The last
line may be shorter than the normal 45 bytes. If the size is not a multiple of 3, this fact can be determined by the value of the count on
the last line. Extra garbage will be included to make the character count a multiple of 4. The body is terminated by a line with a count
of zero. This line consists of one ASCII space.
The trailer line consists of end on a line by itself.
SEE ALSO
uuencode(1), uudecode(1), uusend(1), uucp(1), mail(1)
HISTORY
The uuencode file format appeared in BSD 4.0 .
UUENCODE(5)